Computers in Industry 61 (2010) 318–328
Contents lists available at ScienceDirect
Computers in Industry journal homepage: www.elsevier.com/locate/compind
A design-and-play approach to accessible user interface development in Ambient Intelligence environments Sokratis Kartakis a, Constantine Stephanidis a,b,* a b
Foundation for Research and Technology – Hellas (FORTH), Institute of Computer Science, N. Plastira 100, Vassilika Vouton, GR-70013 Heraklion, Crete, Greece Department of Computer Science, University of Crete, Greece
A R T I C L E I N F O
A B S T R A C T
Article history: Available online 13 January 2010
User interface development in Ambient Intelligence (AmI) environments is anticipated to be a particularly complex and programming intensive endeavor. Additionally, AmI environments should ensure accessibility and usability of interactive technologies by users with different characteristics and requirements in a mainstream fashion. Therefore, appropriate user interface development methods and tools are required, capable of both reducing development efforts and ‘injecting’ accessibility issues into AmI applications from the early design stages. This paper introduces two tools, named AmIDesigner and AmIPlayer, which have been specifically developed to address the above challenges through automatic generation of accessible Graphical User Interfaces in AmI environments. The combination of these two tools offers a simple and rapid design-and-play approach, and the running user interfaces produced integrate non-visual feedback and a scanning mechanism to support accessibility. AmIDesigner and AmIPlayer have been evaluated to assess their usability by designers, and have been put to practice in the redevelopment of a light control application in a smart environment as a case study demonstrating the viability of the design-and-play approach. The results confirm the usefulness and usability of the tools themselves. Overall, the proposed approach has the potential to contribute significantly to the development, up-take and user acceptance of AmI technologies in the home environment. ß 2009 Elsevier B.V. All rights reserved.
Keywords: Ambient Intelligence (AmI) Smart home User interface development User interface generation Accessibility
1. Introduction The increased importance of user interface development in the context of the evolution of the Information Society has been widely recognized in recent years in the light of the profound impact that interactive technologies are progressively acquiring on everybody’s life and activities, and of the difficulty in developing usable and attractive interactive products and services. As the Information Society further develops, the issue of efficiently developing high quality user interfaces becomes even more prominent in the context of the next anticipated technological generation, that of Ambient Intelligence (AmI) environments. The concept of AmI provides a vision of the Information Society where the emphasis is on greater user-friendliness, more efficient services support, user-empowerment, and support for human interactions. Humans are surrounded by intelligent intuitive interfaces that are embedded in all kinds of objects and an
* Corresponding author at: Foundation for Research and Technology – Hellas (FORTH), Institute of Computer Science, N. Plastira 100, Vassilika Vouton, GR-70013 Heraklion, Crete, Greece. E-mail address:
[email protected] (C. Stephanidis). 0166-3615/$ – see front matter ß 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.compind.2009.12.002
environment that is capable of recognizing and responding to the presence of different individuals in a seamless, unobtrusive and often invisible way [9]. AmI as a research and development field is rapidly gaining wide attention worldwide, and the notion of AmI is becoming a de facto key dimension of the evolving Information Society, since many of the new generation industrial digital products and services are shifted towards an overall intelligent computing environment. AmI will have profound consequences on the type, content and functionality of the emerging products and services, as well as on the way people will interact with them, bringing about multiple new requirements for the development of the Information Society (e.g., [3,2]). One of the main intrinsic benefits of AmI is identified in the emergence of new forms of interactive applications supporting every day activities in a variety of environments and application domains. A key application of AmI is smart home environments supporting home automation [5]. Although the cost and availability of AmI technologies will be important factors for their wider spread and market success, the ultimate criterion for their public acceptance and use will be the extent to which the complex environment will be able to communicate with its users, and developers will be facilitated
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
in providing appropriate interaction means in an effective and efficient way. Therefore, the issue of supporting user interface development becomes critical. In this respect, the following main requirements can be identified: User interface development in AmI will be a particularly complex endeavor, due to the complexity of the environment itself. Therefore, tool support becomes crucial. A graphical design environment combined with automatic generation of AmI applications can provide an appropriate solution for facilitating rapid prototyping and development of Graphical User Interfaces (GUIs). AmI is perceived as a technological development that should ensure accessibility and usability of interactive technologies by users with different profile characteristics and requirements in a mainstream fashion [3,10]. This is due to the fact that, in such an in fieri technological environment, it will not be possible to introduce ad hoc solutions that are implemented once the main building components of the new environment are in place. Therefore, appropriate user interface development methods and tools will be required, capable of ‘injecting’ accessibility into AmI applications from the early design stages and throughout the entire user experience lifecycle. This paper introduces two tools, named AmIDesigner and AmIPlayer, which have been specifically developed to address the above challenges through automatic generation of accessible GUIs in AmI environments, and have been applied and tested in the context of smart home applications. AmIDesigner is as a graphical design environment, whereas AmIPlayer is a support tool for GUI generation. The combination of these two tools is intended to significantly reduce the complexity of developing GUIs in AmI environments through a design-and-play approach, while at the same time offering built-in accessibility of the generated user interfaces, by integrating non-visual feedback and a scanning mechanism. The tools can be easily extended with additional interaction styles and services, as well as interaction platforms. AmIDesigner and AmIPlayer have been evaluated to assess their usability by designers, and have been put to practice in the redevelopment of a light control application in a smart environment as a case study demonstrating the viability of the design-and-play approach. The results of the conducted evaluation and of the case study confirm the usefulness and usability of the tools themselves. An additional benefit is that the developed tools, besides from the fact that they provide built-in accessibility, support very rapid prototyping-assessment-redesign cycles to efficiently address usability problems. This paper is structured as follows. Section 2 discusses smart home environments and their requirements regarding user interaction and user interface development. Section 3 overviews current approaches to automated user interface generation. Section 4 provides a rationale for the design-and-play approach, outlines the underlying process and its implication in a smart home environment. Section 5 presents the main characteristics of the user interfaces produced by the developed tools, focusing on built-in accessibility. Section 6 presents the design of AmIDesigner, and Section 7 outlines the main features of AmIPlayer. Section 8 reports on the usability assessment of AmIDesigner, as well as the conducted case study.
anticipation of market expansion. For example, Philips has a facility in The Netherlands called HomeLab,1 which is specifically designed as a development and test site for in-home AmI technology, and where technologies like Easy Access2 are emerging. Easy Access recognizes a hook line from somebody humming, automatically compares it with a song database and plays the song on the room’s stereo equipment. The underlying goal is to create a ‘‘perceptive home environment’’, which is able to identify and locate humans in it, as well as to determine their intentions, with the ultimate aim to provide them with the best service possible. Similar developments are taking place in the academic and research community. The Georgia Institute of Technology has developed a 5040-square-foot ‘‘Aware home’’3 used as a laboratory for AmI development. Various products have been tested, providing valuable insight into the scope of applications that are available. Such an initiative is the Gesture Pendant,4 which exploits simple hand gestures to control multiple devices, thus helping reduce the number of remote controls necessary in most homes today and minimizing technological clutter. Other related initiatives are the PlaceLab of MIT5 and the inHaus Innovation Center of Fraunhofer-Gesellschaft.6 A common objective of many current projects is to enable, in the near future, the control of a large number of standard electrical and electronic devices residing in the home environment through computing technologies. In [8], an experiment is described with the main target to analyze different kinds of ambient assistance in living room environments. The participants had to interact with a networked and partly autonomously acting home entertaining system, which presented integrated functions like TV, radio, audio and video playing, telephone, and light controls. Based on the obtained results, seven kinds of assistance were identified, ranging from situations where the user is fully informed about all details of the environment changes to situations where the environment changes without any direct access of the user. The vision of Ambient Intelligence promises to deliver more natural ways for controlling devices (e.g., by using hand gestures and speech) [1] or even offer proactive, non-user-initiated, controls (e.g., a ‘‘smart’’ room knows when and how to change the state of each device in it) [4]. However, due to the intrinsic characteristics of the new technological environment, it is likely that interaction will pose different perceptual and cognitive demands on humans compared to currently available technology [6], and the challenge emerges to identify and avoid forms of interaction, which may lead to negative consequences such as confusion, cognitive overload, frustration, etc. This is particularly important given the pervasive impact of the new environment on all types of everyday activities and on the way of living. In [19], an experiment is described, that targeted the matching of user interface intelligence in smart home environments with the type of task to be carried out and the age of the user. Following [17], tasks are classified as skill-based tasks, rule-based tasks and knowledge-based tasks. In skill-based tasks, performance is governed by patterns of preprogrammed behaviors in human memory, without conscious analysis and deliberation. In rulebased tasks, performance is governed by conditional rules. Finally, at the highest level of knowledge-based tasks, performance is governed by a thorough analysis of the situation and a systematic comparison of alternative means for action. Interfaces are also 1
http://www.research.philips.com/technologies/misc/homelab/index.html. http://www.research.philips.com/technologies/syst_softw/easyaccess/ index.html. 3 http://awarehome.imtc.gatech.edu/. 4 http://www.imtc.gatech.edu/projects/archives/gesture_p.html. 5 http://architecture.mit.edu/house_n/placelab.html. 6 http://www.inhaus-zentrum.de/site_en/. 2
2. Smart home AmI applications Currently, large companies worldwide are investing on research and development of smart home environments in
319
320
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
Fig. 1. Tools for Graphical User Interface prototyping and generation.
distinguished according to three levels of intelligence (low, medium and high). The results of the study, involving two user groups of young and older people respectively, show that the level of intelligence of the user interface significantly affects performance for different types of tasks. In particular, the study arrives at the conclusion that, when completing skill-based tasks, users have better performance (less time and fewer errors) on low-intelligence-level interface, whereas when completing rule-based tasks, users perform better on high intelligence user interfaces. However, due to the decline of the cognitive abilities, the performance of older users does not differentiate as clearly as that of young people while using different user interfaces of smart home, especially for high cognitive demanded tasks. Therefore, the need arises for highly usable GUIs allowing direct and accessible user control, avoiding the need to use a variety of different applications to control different devices, and providing an intuitive overview of the overall current state of the smart environment. The development of appropriate user interfaces also needs to be facilitated through the availability of suitable tools to reduce development efforts and guarantee built-in accessibility. 3. User interface development tools User interface development is a complex task that is widely recognized as requiring appropriate tool support. In current development practice for graphical environments, tool support is mainly based on: the provision of graphical environments for user interface prototyping supporting a WYSIWYG (‘‘What You See Is What You Get’’) design paradigm; the provision of markup languages for the description of GUIs; and the provision of mechanisms for the automatic generation of user interfaces, intended to fill the so called design-implementation gap, thus considerably reducing development time.
A schematic classification of existing tools is provided in Fig. 1. Available GUI WYSIWYG editors offer graphical editing facilities that allow designers to perform rapid prototyping visually. Such editors may be standalone or embedded in integrated environments, and may differ with respect to: (i) the output of the GUI design process: this may be a GUI description in some markup language or GUI code that needs to be integrated in the overall application by programming; and (ii) the facilities made available to designers. Various user interface markup languages are available, whose main role is to help keeping the GUI separated from the logic of an application, but also provide a common language between GUI design and automatic GUI generation. Based on the markup description produced by GUI editors, GUI generators can produce a running user interface automatically, thus saving much programming efforts. Alternative more research-oriented efforts towards support for user interface development focus on model-based tools. A variety of user interface design tools and environments are based on modeling relevant user interface related knowledge (e.g., [18]). Model-based design environments vary according to many factors, including overall objectives, number and types of models, available design support and design outcomes [15]. Almost all design environments include graphical editors for task hierarchies and other models. Design environments also differ with respect to the number and type of design activities that are automated. In some tools, the entire user interface is automatically produced [16]. The emergence of a variety of interaction platforms has led to various model-based approaches for supporting the design of multiplatform user interfaces (e.g., [14]). These approaches usually address issues related to handling the constraints posed by different platforms and screen sizes, selecting and generating appropriate presentation structures, and managing contexts of use. Some model-based tools for user interface development in AmI have also been proposed (e.g., [12]). Overall, however, design automation from models appears to be a difficult task, due to the large dimensions of the design space, and the difficulty in automatically
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
321
Fig. 2. AmI environment architecture.
selecting appropriate designs. The impact of many of the above tools on design practices has been limited so far [18,15]. Many tools have been criticized for lack of usability due to their complexity. Additionally, automatic generators are usually bound to generate specific or limited types of interfaces, and do not take into account accessibility requirements for the generated user interfaces. Therefore, although automatic user interface generators are often criticized for the quality of the code they produce, the combination of a GUI WYSIWYG Editor with a markup description and an automatic GUI generator currently appears to be a more promising approach towards rapid and efficient development of user interfaces, more familiar to designers and in-line with current practice. However, no tool currently available offers the possibility of creating GUIs for AmI services provided through various devices in a smart home environment, as no tool is appropriately equipped for communicating with such an environment. Additionally, another very important limitation of available tools is that they do not produce accessible user interfaces, and, therefore, they are not suitable for addressing the mainstream accessibility requirement posed by AmI environments, and smart home applications in particular. Finally, many of the described tools are bound to specific development platforms or interaction object toolkits. In AmI, however, it is particularly important to ensure that new interaction metaphors, appropriate to the new types of applications that the AmI environment integrates, can be elaborated, evaluated and used. Therefore, the extensibility of the basic toolkit is particularly important when designing for AmI. AmIDesigner and AmIPlayer aim to address the above gaps through a design-and-play approach inspired by current tools for prototyping and automatic generation of GUIs.
The AmI environment architecture, in which AmIDesigner and AmIPlayer operate, is depicted in Fig. 2. Such an environment includes many devices, such as PDAs, projectors, monitors, computers, cameras and sensors that need to communicate with each other. The Middleware is the AmI environment’s layer that is responsible for communication. The devices are activated depending on external measurements indicated by the sensors. These measurements are processed by the Decision Engine, which is considered as the ‘‘brain’’ of the AmI Environment and defines the state of the environment. AmIDesigner is a graphical WYSIWYG editor, which creates descriptions of a designed application in XML and offers a set of widgets available to design a user interface. The widgets are detected dynamically by the editor, and developers can program their own widgets. This possibility is provided by the dynamic architecture of the system. AmIDesigner communicates with the Middleware of an AmI environment, and automatically detects the available services in a room, as well as the Middleware signals. Therefore, it supports defining how the application manipulates the devices in the room and how the external environment interacts with the GUI.
4. Design-and-play user interface development in smart home environments AmIDesigner and AmIPlayer have been specifically developed to achieve the following development support objectives: easy and quick development of user interfaces, along the lines of the GUI generators discussed in the previous section; automatic integration of the developed interfaces in smart home environments without the need of further programming; and extensibility of the tools themselves with additional interaction styles and additional services, as well as, potentially, additional interaction platforms.
Fig. 3. Design-and-play development process.
322
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
Fig. 4. AmIDesigner Graphical User Interface.
AmIPlayer uses the export XML description produced by AmIDesigner to generate GUIs ready for run time. These GUIs can send and receive signals from the middleware and call services, thus allowing control of the room devices. In the current implementation, AmIDesigner is limited to the MS Windows platform due to the usage of the .NET platform. Nevertheless, AmIPlayer can potentially run on various operating systems with minor changes. Future extensions of the tools will address multiplatform development. In summary, a very simple design-and-play process is carried out for building applications with AmIDesigner and AmIPlayer, which is illustrated in Fig. 3. The design of the GUI is performed using AmIDesigner, which exports the GUI description to an XML file. Then, AmIPlayer uses the XML file in order to generate an accessible GUI that controls the devices of the AmI environment, and the user can directly interact with it. 5. Generated user interfaces The user interfaces generated by AmIDesigner and run by AmIPlayer are rich Flash GUIs that can include animations. Since interface widgets can be freely created by designers, alternative interaction metaphors beyond the traditional desktop and more appropriate for the home environment can easily be designed and implemented, and attractive graphical effects can be included. However, the GUIs generated by the proposed tools also integrate accessibility features, which make them usable for a large population of target end users, including hand-motor and visual impaired users, children, elders, and inexperienced users. Accessi-
bility is integrated in a multimodal fashion, i.e., alternative rendering of widgets and alternative interaction techniques are supported. In particular, such interfaces embed directly from design: speech and audio feedback associated to interface widgets; hierarchical scanning of interface elements; and compatibility with various input/output devices (e.g., touch screen, remote control, binary switches, etc.). Speech and audio feedback, in combination with keyboard input or input through other appropriate devices, addresses the needs of blind users in the context of smart room devices control. Additionally, non-visual feedback (in association with visual feedback) can also be useful for sighted users, especially when they are busy with other tasks. In conventional GUIs, non-visual feedback is usually limited to few sounds and does not include speech. Non-visual rendering of GUIs is usually achieved through screen readers, i.e., assistive technologies for blind users that ‘read aloud’ user interface items to the user. However, besides necessitating extra software that the user needs to possess and install, the use of screen readers is problematic with Flash GUIs. These problems can be completely overcome in the proposed approach through built-in non-visual feedback. Scanning is an interaction technique that enables motorimpaired users to have access to interactive applications and services [13]. The basic idea is that a special ‘‘marker’’ (e.g., a colored frame) indicates the interaction item that has the input focus. The user can shift the focus marker to the next/previous interaction object using any kind of switches (e.g., keyboard keys, special switch hardware, remote control). When the focus is over
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
323
Fig. 5. AmIDesigner Widget toolbox.
an object the user wants to interact with, another switch is used to perform ‘‘selection’’. Additionally, in cases where the user can use just a single switch, focus movement can be automatically generated by the system at constant time intervals. This variation of the technique is referred to as ‘‘automatic scanning’’, while any other case is generically called ‘‘manual scanning’’ (even if hands are not actually used at all). Scanning is difficult to add to a user interface after development, as the scanning mechanism needs information on the user interface structure implementation. AmIDesigner embeds a scanning mechanism directly in the produced user interfaces, and the designer only has to set the order in which interface widgets need to be scanned. As a particularly innovative feature, scanning can also be associated with non-visual feedback in a multimodal fashion, thus making switch-based interaction accessible to blind users. 6. AmIDesigner AmIDesigner is targeted not only to experienced, but also to beginner designers and developers of smart home applications. The user interface of the tool has been designed according to the following requirements: Organization of GUI design based on the home environment rooms, so that the designed application is able to control specific rooms. Minimum possible number of steps to complete each design subtask (usually two steps) through edit-in-place facilities. Main editing area inspired by Microsoft Visual Studio .Net,7 as the latter is the design environment currently most familiar to many designers. Fig. 4 depicts the main areas of the GUI of AmIDesigner. These are: the main menu and toolbar, the widgets toolbox, the stage, the widget properties editor, the signals editor and the scanning order editor. Accordingly, the steps that need to be carried out, so that a user interface is produced, are: (1) Creating a project, selecting the smart home room which will be controlled by the application, and setting the dimensions of the GUI. 7
http://msdn.microsoft.com/en-us/vstudio/default.aspx.
(2) Selecting the widgets to be included in the interface. (3) Setting the widget properties, including layout issues, dialogue flow and calls to services. (4) Associating user interface elements and Middleware signals. (5) Establishing a scanning order for the widgets in the user interface. (6) Generating an XML description of the designed GUI, viewing it through AmIPlayer and testing it. This process is described in more details in the following subsections. 6.1. Widgets The widget toolbox displays the widgets that can be used to design a GUI. Three different presentations of the widgets are offered within the toolbox: large icons, list and small icons (see Fig. 5). To insert a widget in the GUI being created, the corresponding item is dragged from the toolbox and dropped on the stage. The only actions that can be performed within the stage are the creation of new items and moving and resizing existing items. 6.2. Widget properties The widget properties editor on the right side of the AmIDesigner interface displays the following information: (a) the properties of the currently selected item on the stage; (b) a list of all the items within the stage with the currently selected item marked; and (c) a list of all available services and functions, as well as the arguments of the currently selected function. Using this editor, the designer can set the properties of the currently selected item. Properties vary according to each item category, are detected dynamically, and include name, size, dimension, etc. An important property is the voice text associated to an item, which is used at run time to ‘‘speak aloud’’ the interface, so as to provide non-visual interaction (for example, for blind users, or for users busy with other tasks in the environment). As previously mentioned, action widgets can be used to define the dialogue flow in the designed GUI. More specifically, action widgets can be used to alter the visibility state of container widgets. This is achieved by binding an action widget to a container widget, implying that the action widget will change the visibility status of the widget it binds.
324
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
Fig. 6. AmIDesigner widget properties editor.
Additionally, widgets can be grouped to produce complex artifacts. This is achieved by attaching widgets to containers. When an item is attached to a container, by moving the container on the stage, the attached item is also moved. An action widget can also be bound to a function of a service. On the bottom-right part of the interface, a list containing the services and functions of the selected room is displayed. A function can be bound to an action widget in a way similar to binding interface items (see Fig. 6). 6.3. Signals The smart environment should be able to interact with the generated interface to inform it about changes occurring in the environment, thus affecting the status of the GUI. To this purpose, two types of signals are implemented: Middleware Signals and GUI Signals. Middleware signals are transmitted from the middleware with specific arguments to produce GUI Signals. Middleware signals are automatically detected by AmIDesigner. When GUI signals are produced, they modify properties of items within the current stage. The items that are affected by each signal, as well as
the new properties resulting from changes in the environment, are presented at the bottom of the AmIDesigner interface. In the signals editor, the designer can select Middleware signals and associate them to GUI signals, so that that, when a middleware signal is received with specific arguments, the respective GUI signals are produced in the interface (see Fig. 7). 6.4. Scanning A scanning mechanism is built-in in the user interfaces produced by AmIDesigner. The designer has no other task than setting the order according to which elements in the interface will be scanned at run time. This is achieved by defining (and modifying) a scanning hierarchy in the scanning order editor in the left bottom part of the interface (see Fig. 8). Only container widgets are allowed to have children in the scanning hierarchy. 7. AmIPlayer AmIPlayer has no specific interface. It is an application that parses the XML file generated by AmIDesigner and communicates
Fig. 7. AmIDesigner signals editor.
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
Fig. 8. AmIDesigner scanning order editor.
Fig. 9. AmIPlayer GUI generation process.
with the Middleware so as to activate devices and receive signals from them. AmIPlayer consists of two parts: a C# application, which communicates directly to the Middleware, and a Flash stage, which is responsible for the GUI generation. Robust communication between the C# application and the flash stage is essential, and is achieved through calls of Middleware functions with specific arguments. The steps that are carried out, so that a GUI is generated and the user is able to use the devices, are the following (see Fig. 9). Firstly, the configuration file of AmIPlayer is modified to define the XML file that the AmIPlayer will use. The next step is to start the AmIPlayer application. This is a simple flash object that is embedded in the C# application. Finally, the GUI is generated and the user can interact with it and control the corresponding smart room.
325
the perspective of the designer. Additionally, AmIDesigner is a unique tool, i.e., there are no known existing tools providing equivalent functionality that can be used for comparative assessment. These two facts have determined the selection of empirical evaluation methods. On the other hand, the existence of a running prototype made the use of a user-based method to assess the users’ subjective opinion on the perceived usability of the system feasible. The IBM Usability Satisfaction Questionnaires [11] were adopted for subjective usability measurement in a scenario setting. These include an After-Scenario Questionnaire (ASQ), which is filled in by each participant at the end of each scenario, and a Computer System Usability Questionnaire (CSUQ), which is filled in at the end of the evaluation. The two questionnaires adopt a scale from 1 (strongly agree) to 7 (strongly disagree). Additionally, given the target user group of the tool, i.e., designers, who are by definition expert users, it was decided to combine user satisfaction measurement with expert user interface evaluation in order to obtain detailed designers’ comments and suggestions on the interface design of AmIDesigner. The evaluation was conducted in an in-house AmI laboratory space where, among other technologies, several computercontrolled lights have been installed. Seven expert users were involved, five males and two females. All users had at least a University degree in Computer Science or a related subject. All of them had at least a few years experience in the field of Human– Computer Interaction and in the design of accessible user interfaces, but no previous experience in the design of AmI applications. The following evaluation procedure was adopted. The group of users was briefly introduced to the functions and goals of the room, as well as to the objectives of the evaluation experiment, i.e., the evaluation of user satisfaction and the collection of experts’ comments on the system, and was provided with the related material. After that, AmIDesigner was demonstrated to each user. All participants were then asked to perform the following series of design tasks for a light control application: Task Task Task Task Task
1: 2: 3: 4: 5:
Design a Simple GUI. Change Widgets’ Properties. Bind Items and Services and attach Widgets. Signals’ Manipulation. Create a Scanning Order.
8. Evaluation and validation
Upon completion of the design tasks, the participants exported the XML description and ‘‘played around’’ with their own user interface (changing the lighting in the room). During the experiment, the participants were requested to freely express their opinion about the system and mention any problems they faced. The designers were then requested to fill-in the IBM user satisfaction questionnaire, as well as a brief evaluation report including their expert evaluation comments on AmIDesigner and their suggestions for further improvement.
8.1. Evaluation of AmIDesigner
8.2. Evaluation results
AmIDesigner targets a particularly expert user population, namely user interface designers. The evaluation process of the tool has, therefore, followed a hybrid approach involving a mixture of user satisfaction and expert evaluation methods. The evaluation method was selected based on the consideration that, besides general heuristics, there are no available heuristics or guidelines for evaluating a design support tool from
The results of the user satisfaction questionnaire are reported in Table 1 (ASQ), Table 2 (CSUQ) and Fig. 10. The mean user satisfaction, presented in the diagram of Fig. 10, was 87.61, that is considered very high. In general, the participants were satisfied with AmIDesigner. In particular, they were impressed by the possibility of using the application immediately after designing it, and very much enjoyed
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
326 Table 1 AmIDesigner usability evaluation results (ASQ). ASQ results Task 1
User User User User User User User
1 2 3 4 5 6 7
Total
Task 2
Task 3
Task 4
Task 5
(a)
(b)
(c)
(a)
(b)
(c)
(a)
(b)
(c)
(a)
(b)
(c)
(a)
(b)
(c)
2 1 1 1 1 1 2
1 1 1 1 2 2 2
1 1 1 2 1 2 2
2 2 2 1 1 1 4
2 2 1 1 1 1 4
1 1 1 1 1 2 4
3 2 3 1 3 3 4
3 2 1 1 3 3 4
3 1 2 1 1 2 5
1 2 1 1 3 2 2
1 2 1 1 2 2 2
1 3 1 1 1 2 2
1 1 1 1 1 1 4
1 1 1 1 1 1 4
1 1 1 1 1 2 4
29 (21–147)
36
51
34
31
Table 2 AmIDesigner usability evaluation results (CSUQ). CSUQ results General questions 1 User User User User User User User Total
1 2 3 4 5 6 7
10
11
12
13
14
15
16
17
18
19
1 3 1 1 2 2 3
2 2 2 2 1 2 2 3
3 2 1 1 1 1 2 3
4 1 1 1 1 2 2 3
5 1 2 1 3 2 2 3
6 1 1 1 2 1 2 3
7 1 1 1 2 2 1 4
8 1 2 1 2 1 2 3
9 1 1 1 1 3 3 4
1 1 1 2 3 3 5
1 1 2 2 1 2 4
1 1 1 1 2 2 4
1 1 1 1 1 2 5
1 1 1 2 1 2 4
4 l l l 3 2 3
1 1 1 1 1 1 2
1 1 1 1 1 2 2
2 2 2 2 1 2 3
1 1 1 1 2 2 3
13
14
11
11
14
11
12
12
14
16
13
12
12
12
15
8
9
14
11
the design-and-play approach. Most of them also provided useful comments towards improving the user interface of the tool. Those comments are briefly summarized below:
developed tools, and most importantly, of the potential role of the design-and-play approach in AmI environments. 8.3. A case study
The visibility of the binding process for services and items can be improved. The same holds for the process of attaching widgets to each other. The design stage can be enriched with additional functionality. Apart from drag and drop, widgets could be inserted in the stage by simply clicking on their image in the widget toolbox. The property grids could be improved by offering tools for easier selection of graphical images, colors and other features. Overall, this evaluation experiment, although small-scale, provided a clear indication of the usefulness and usability of the
In order to test them in practice, AmIDesigner and AmIPlayer have been employed in the redevelopment of an accessible light control application for smart home environments, named CAMILE. The original development and evaluation of CAMILE is reported in [7]. Target users are home inhabitants of any age, including people with visual or motor disabilities. In order to accommodate such a wide range of users, CAMILE provides four different modes of interaction: (a) touch screen-based for sighted users with no motor impairments; (b) remote controlled operation in combination with speech feedback for visually impaired users or tele-operation by sighted users; (c) scanning (both manual and automatic) for
Fig. 10. Evaluation results of AmIPlayer (user satisfaction).
Fig. 11. CAMILE user interface.
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
motor-impaired users using binary switches; and (d) speech enabled scanning for visually impaired users. The GUI of CAMILE is illustrated in Fig. 11. For the redevelopment of CAMILE, the following steps were followed. AmIDesigner was used to recreate the user interface by means of simple widgets such as labels, panels and image buttons. The light services were then bound to specific action widgets, followed by the definition of the widgets’ scanning order. Finally, GUI signals were created to influence the GUI depending on the incoming Middleware Signals. After completing the redesign and producing the XML description, AmIPlayer was ran on the touch screen PC, and the application that controls the lights in the AmI laboratory was ready for use. The original development of CAMILE required three weeks of extensive development, as well as specialized programming knowledge and experience. On the other hand, the redevelopment through AmIDesigner by an expert user interface designer took approximately two working days, without any programming, while obtaining the same functionality, user interface and communication with the AmI environment. Thus, the use of AmIDesigner can be claimed to greatly simply development and reduce the required time and experience resources. Additionally, AmIDesigner made the development easy because the produced results are visible at design time, and changes in the user interface need only a few ‘‘clicks’’. This offers the opportunity of very fast prototyping cycles for effective usability evaluation, where adjustments to the user interface can be made directly during experiments when needed. The above provide strong evidence that the developed tools, AmIDesigner and AmIPlayer, have significant potential to bring positive impact on the development, adoption and acceptance of AmI technologies. 9. Conclusions This paper has introduced a design-and-play approach to the development of accessible GUIs in AmI environments, based on automatic GUI generation. Such development process is supported through two tools, AmIDesigner and AmIPlayer. AmIDesigner is a graphical editor that creates descriptions of a designed application in XML. It includes a variety of widgets which are detected dynamically. Developers can program their own widgets. AmIDesigner communicates with the Middleware of the AmI environment, is continuously informed about the available services of a room and detects Middleware signals. Therefore, it supports the definition of how the application manipulates the devices in the room and how the external environment interacts with the GUI. AmIPlayer, on the other hand, uses the export XML description of AmIDesigner to generate running GUIs. The produced rich GUIs can be manipulated through several input devices, embed built-in non-visual feedback and scanning, and allow end users to directly control smart room environment services. The tools have been undergone evaluation and obtained very positive usability scores. Additionally, they have been employed in a case study concerning the development of a light control application in a laboratory environment. This effort has demonstrated in practice the value of the tools towards simplifying and considerably shortening the user interface development process. Overall, it is claimed that the proposed approach and its supporting tools achieve the following objectives: easy and quick development of user interfaces without the need of programming; automatic integration of the developed interfaces in smart home environments without the need of further programming;
327
accessibility of the produced user interfaces through a multimodal combination of interaction devices and techniques without reducing the use of rich graphics; support for efficient usability assessment lifecycle through the design-and-play approach; and potential for further development through extensibility. The above can provide a significant contribution to the development and up-take of AmI technologies in the home environment. In particular, offering the possibility to decouple user interface provision from extensive programming, the reported work can facilitate the emergence of new professional roles targeted to effectively and efficiently set-up smart environments for their diverse inhabitants, including people of all ages, abilities and backgrounds, and test them in place. Foreseen extensions of AmIPlayer include enhancement of the editing facilities, multilingual support, selection of input devices and association of devices to specific task, and integration of recognition techniques (e.g., speech and gesture). Foreseen extensions of AmIPlayer mainly concern multiplatform support, thus making the generated interfaces available through PDAs, mobile phones and other interactive devices in the AmI environments. References [1] N. Carbonell, Ambient multimodality: towards advancing computer accessibility and assisted living, Universal Access in the Information Society 5 (1) (2006) 96–104. [2] V. Coroama, J. Bohn, F. Mattern, Living in a smart environment—implications for the coming ubiquitous information society, in: Proceedings of the International Conference on Systems, Man and Cybernetics 2004 (IEEE SMC 2004), vol. 6, The Hague, 2004, pp. 5633–5638, The Netherlands, October 10-13. [3] P.L. Emiliani, C. Stephanidis, Universal access to ambient intelligence environments: opportunities and challenges for people with disabilities, IBM Systems Journal. Special Issue on Accessibility 44 (3) (2005) 605–619. [4] A. Ferscha, S. Resmerita, C. Holzmann, Human computer confluence, in: C. Stephanidis, M. Pieper (Eds.), Proceedings of the 9th ERCIM Workshop on User Interfaces for All, Ko¨nigswinter, Germany, September 27–28, 2006, Universal Access in Ambient Intelligence Environments, 2007. [5] M. Friedewald, O. Da Costab, Y. Punieb, P. Alahuhtac, S. Heinonen, Perspectives of ambient intelligence in the home environment, Telematics and Informatics 22 (3) (2005) 221–238. [6] A. Gaggioli, Optimal experience in ambient intelligence, in: G. Riva, F. Vatalaro, F. Davide, M. Alcan˜iz (Eds.), Ambient Intelligence, IOS Press, Amsterdam, The Netherlands, 2005, pp. 35–43. [7] D. Grammenos, S Kartakis, I Adami, C. Stephanidis, CAMILE: controlling AmI lights easily, in: Proceedings of the 1st ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2008), 15–19 July 2008, Athens, Greece, [CD-ROM], New York, ACM Press, 2008. [8] M. Hellenschmidt, R. Wichert, Goal-oriented assistance in ambient intelligence. Fraunhofer-IGD Technical Report, Darmstadt, 2005, http://www.igd.fhg.de/ igd-a1/publications/publ/ERAmI_2005.pdf. [9] IST Advisory Group, Ambient intelligence: from vision to reality (2003) [On-line]. Available at: ftp://ftp.cordis.lu/pub/ist/docs/istag-ist2003_consolidated_report.pdf. [10] E. Kemppainen, J. Abascal, B. Allen, S. Delaitre, C. Giovannini, M. Soede, Ethical and legislative issues with regard to ambient intelligence, in: P.R.W. Roe (Ed.), Impact and Wider Potential of Information and Communication Technologies, Published by COST, Brussels, 2007, pp. 188–205. [11] R.J. Lewis, IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use, International Journal of Human–Computer Interaction 7 (1) (1995) 57–78. [12] K. Luyten, C. Vandervelpen, J. Van Den Bergh, K. Coninx, Context-sensitive user interfaces for ambient environments: design, development and deployment, http://hdl.handle.net/1942/7935, in: Dagstuhl Seminar 05181: Mobile Computing and Ambient Intelligence: The Challenge of Multimedia (Dagstuhl Seminar 05181), 2005. [13] S. Ntoa, A. Savidis, C. Stephanidis, FastScanner: an accessibility tool for motor impaired users, in: Proceedings of the 9th International Conference on Computers Helping People with Special Needs (ICCHP 2004), LNCS 3118, Springer, Berlin Heidelberg, 2004, pp. 796–804. [14] F. Paterno`, C. Santoro, One model, many interfaces, in: Proceedings of the 4th International Conference on Computer-aided Design of User Interfaces, Kluwer Academics Publishers, 2002, pp. 143–154. [15] P. Pinheiro da Silva, User interface declarative models and development environments: a survey, in: Ph. Palanque, F. Paterno` (Eds.), Interactive Systems: Design, Specification, and Verification (7th International Workshop DSV-IS, Limerick, Ireland, June, 2000), LNCS vol. 1946, Springer, Berlin Heidelberg, 2000 , pp. 207–226.
328
S. Kartakis, C. Stephanidis / Computers in Industry 61 (2010) 318–328
[16] A. Puerta, J. Eisenstein, Towards a general computational framework for modelbased interface development systems, in: Proceedings of the ACM Conference on Intelligent User Interfaces (IUI’99), ACM Press, New York, 1999, pp. 171–178. [17] J. Rasmussen, A.M. Pejtersen, L.P. Goodstein, Cognitive Systems Engineering, John Wiley & Sons, Inc., New York, 1994. [18] P. Szekely, Retrospective and challenges for model based interface development, in: F. Bodert, J. Vanderdonckt (Eds.), Proceedings of the 3rd Int. Eurographics Workshop on Design, Specification and Verification of Interactive Systems (DSVIS’96), Eurographic Series, Springer, Berlin Heidelberg, 1996, pp. 1–27. [19] B. Zhang, P.L.P. Rau, G. Salvendy, Design and evaluation of smart home user interface: effects of age, tasks and intelligence level, Behaviour and Information Technology 28 (3) (2009) 239–249. Sokratis Kartakis holds a B.Sc. and an M.Sc. in Computer Science from the Department of Computer Science of the University of Crete, Greece. From July 2003 to September 2006 Mr. Kartakis worked at the Human–Computer Interaction Laboratory of the Institute of Computer Science of the Foundation for Research and Technology – Hellas (ICS-FORTH) as an assistant with an undergraduate scholarship. From October 2006 to September 2008, Mr. Kartakis worked at the Human–Computer Interaction Laboratory as an assistant with a postgraduate scholarship.
Constantine Stephanidis, Professor at the Department of Computer Science of the University of Crete, is the Director of the Institute of Computer Science, and member of the Board of Directors of the Foundation for Research and Technology – Hellas. He is the Head of the Human–Computer Interaction Laboratory, and of the Centre for Universal Access and Assistive Technologies, and Head of the Ambient Intelligence Programme of ICS-FORTH. He is also the President of the Board of Directors of the Science and Technology Park of Crete. He is member of the National Council for Research and Technology (Hellenic Ministry of Development), National representative (General Secretariat for Research and Technology, Hellenic Ministry of Development) in the Specific Configuration Programme Committee of FP7, and member of the Management Board of ENISA (European Network and Information Security Agency). He has been engaged as the Scientific Responsible on behalf of ICS-FORTH in more than 50 National and European Commission funded projects, and he introduced the concept and principles of Design for All in Human–Machine Interaction and for Universal Access in the evolving Information Society. He has published two Handbooks and more than 300 technical papers, and he is Editor-in-Chief of the Springer international journal ‘‘Universal Access in the Information Society’’. URL: http://www.ics.forth.gr/stephanidis/index.shtml.
