Local Support to Plastic User Interfaces: an ...

Local Support to Plastic User Interfaces: an Orthogonal Approach Sendín, M. and Lorés, J. University of Lleida (Spain). GRIHO: HCI research group.

1 Introduction Nowadays technology allows users to keep on moving with computing power and network resources at hand. Computing devices are shrinking while wireless communications bandwidth keeps increasing. These changes have increasingly enabled access to information; “anytime and anywhere”. This makes computing possible in multiple and varied contexts of use1 in which a great amount of variable parameters come together. This provides us tremendous versatility. Consequently, current interactive systems should be prepared to face up this continuous and diverse variability. They should be able to offer the capacity of systematically producing as many User Interfaces (henceforth UIs) as contextual circumstances have been identified. This fact raises a first challenge. Moreover, the adaptive capacities should evolve in runtime, as the real time constraints vary. This would offer, in addition, continuous adaptation to each concrete UI in use, with the aim of solving the contextual changes in a dynamic and automatic way. We identify this problem as a second challenge different from the previous one. Until now these two issues have been designated a single term: plasticity of UIs2, introduced by Thevenin and Coutaz in [19] in 1999 along a framework and research agenda development work. This work was put in practice in these two works: ARTStudio [18] and Probe [4]. It was also revised in 2002 in [5], within the Cameleon project [6]. Nevertheless, according to the distinction between the two challenges mentioned above, we understand plasticity as a dichotomy, that is to say, a separation into two parts. We see these two parts closely associated with the diverse stages of reconfiguration and readjustment the UI comes cross during the whole system lifetime, respectively. They can be considered as the design operation level and the runtime operation level. Both of them were distinguished and defined in [13] as two sub-concepts of plasticity (explicit plasticity and implicit plasticity, respectively) with two different goals clearly identified. They have also been presented as an extension to the Thevenin and Coutaz work in [16]. As it appears in [16], we define the explicit plasticity as the capacity of automatically generating a specific UI valid to a concrete context of use, starting from an abstract specification -generic UI3. In a parallel way, we define implicit plasticity as the capacity of incremental adaptation that the specific UI obtained in the production stage should show in real time as the user goes across new contexts of use. The explicit plasticity tackles important changes in the UI that are caused either by unforeseen situations –a change in the user or the device- or by the request of new contents –changes in the user location or in the task to perform. Both cases 1

2

3

Set of environmental parameters that describe a particular context in which a set of conditions of real time come together, related not only to the restrictions in hardware resources, but also related to the user, and even to the environment. Capacity of a same generic UI to support variations in multiple contextual circumstances preserving usability and at the same time minimizing the development and maintenance costs [6]. Unique and flexible abstract specification of the layout and composition of the resulting interface, as well as a description of how the UI evolves in time. It is related to the concept of canonical expression of the UI.

Sendín, M. and Lorés, J.

involve a reconfiguration of the UI to be solved in the server side upon explicit request. This is the reason why we use the term “explicit”. Comparably, the implicit plasticity tackles specific modifications in the UI, originated by predictable contextual changes -changes in the brightness level, server connectivity level, bandwidth, small advances made by the user, etc. We state that this kind of changes should be solved with an automatic local readjustment on the client side, without any express request or action. This is why we chose the term “implicit”. In the works mentioned previously, we also propose a client/server-based architectural framework to solve both challenges on the opposite sides of the architecture. The framework proposed to solve the explicit plasticity is located on the server side and is called explicit plasticity engine. The architecture proposed to solve the implicit plasticity is located on the client side (on his/her own device) and is called implicit plasticity engine. The former consists of a systematic development support capable of generating or redesigning (reconfiguring) remotely a suitable UI for each case presented. See [14] for more detail. The later consists of an architecture with evolving capacity capable of detecting the environment and of reacting appropriately. It is in charge of applying locally the necessary adjustments to the UI and thereby adapting to the contextual changes on the fly. This architectural framework tries to reach a certain balance between both sides. Certainly, it contributes to some benefits derived from certain autonomy that allows adaptivity also to be performed on the client side. As a result, the client only contacts with the server in the necessary cases. On the one hand, as the contextual representation is stored in the client side, the resultant system becomes more robust to possible fails in the server or communication system. This makes possible to keep on running till new reconfiguration needs appear. On the other hand, this autonomy contributes to an automatic and transparent reaction in real time. Moreover, the timely communication between client and server provides a mechanism for propagating contextual changes in both ways. This is one of the most important shortcomings detected in the literature. The client notifies to the server the changes produced in the current UI by means of a trustworthy request (i.e. that reliably reflects the real time constraints). On receiving the request, the server has the chance of updating its contextual map and, as a consequence, of generating a UI as adjusted as possible to the current situation. This mechanism allows keeping both sides in continuous feedback and updating. This paper deals with the implicit plasticity problem associated with a spontaneous adaptation to contextual changes. We aim to offer a local solution capable of accommodating a continuous variability. This one is to be implanted in the client device. The major difficulty for offering a local support to plastic UIs is dealing with the multiplicity of contextual factors that a multiappliance, multi-user and multi-contextual interactive system is supposed to control. Moreover, the needs for dynamic and real time adaptation become more and more demanding. Nevertheless, runtime constraints not only related to resources constraints -bandwidth, server availability, physical resource restrictions, etc.-, but also others related to the user, and even related to the environment, are volatile and require sophisticated adaptive capabilities that today are still challenging. In order to apply the necessary adjustments to the UI in a transparent and dynamic way, the system has to be able to self-recognize and self-modify in runtime. So we are thinking of a technology that provides properties such as retrospection, reification and reflection [11]. Apart from that, we need a solution to the separation of concerns as orthogonal as possible, in order to offer major clarity and minor coupling in the treatment of a specific property. We have adopted the Aspect Oriented Programming (AOP) paradigm [10], in combination with reflection techniques [11], offering an application of such a relevant technology in the Human Computer Interaction field. This proposal implies an improvement regard a previous architecture based only on reflection techniques [14]. The present contribution proposes to modularise the set of contextual factors that affect the UI using aspects. The aim is to solve the tangling code and scattering code problems that the previous solution had. This will contribute to reusability and flexibility, allowing each contextual factor to evolve independently. This paper is composed of three sections more. In section two the implicit plasticity concept is featured giving an overview. In section three the AOP paradigm is presented. Then, in section

Local Support to Plastic User Interfaces: an Orthogonal Approach four we present the adaptive architecture we propose. This section is complemented presenting the implementation tools chosen. Finally, we expose the conclusions.

2 Systems with Implicit Plasticity

2.1 Featuring the Domain We define a system with implicit plasticity as a system capable of detecting contextual information –where it is also considered the user and the hardware resources as part of the context-, taking advantage of it to adapt its own appearance to different contexts of use. In this sense this concept comprises both the adaptivity purpose as well as the context-awareness one. In accordance with Dix in [9], adaptivity is “the modifiability of the UI by the system”, referring to an automatic modifiability and taking into account exclusively the user profile. In our opinion, the concept of implicit plasticity can be considered equal to the context awareness –which is a decade old-, since the moment in which the properties related to the user as well as the restrictions in hardware resources are incorporated as properties that characterize the context. The goal is that all of them are taken into consideration in the adaptation process. In this way, our conception about context corresponds to the Schilit one in [17], who divides it in three categories: computing context, user context and physical context. In [1, 7] a detailed survey of all of these terms is offered. The combination of varied contextual properties lets us attain a more precise understanding of the current situation. However, apart from the treatment of the location and the user modeling, any other context parameter has been so widely studied [7]. Moreover, it has not been employed too much effort in exploring techniques that combine the user modeling with the context modeling, as is exposed in the next section. To develop an implicit plasticity engine and being able to adapt the system behavior, it is necessary to know the environment that surrounds it, in order to characterize the context. Next, it must be established the method to model and to integrate such context in the system performance. The aspect of how to effectively use that information is still a challenging problem for developers of this type of systems. Dix in [9] presents the Context-Awareness computing as one of the applications that Ubiquitous Computing makes possible. He defines the Ubiquitous Computing as “any computing technology that permits human interaction away from a single workstation. This include pen-based technology, handheld or portable devices, wireless networking infrastructure,...”. Hence, according to this consideration, Mobile Computing constitutes one of the fields embraced by Ubiquitous Computing. He also points that Context-Awareness Computing is especially restricted to Location-Awareness Computing for mobile applications. Anyway, Context-Awareness Computing tackles multiple variable parameters, between them considerations about the hardware resources that acquire today essential importance, either for its wide range of possibilities of selection (Mobile Computing) or for the implicit necessity of interconnection among devices (Ubiquitous Computing). 2.2 Overview It is worthy to remark that, despite the apparent correlation and proximity between the adaptation provided by the user modeling and the notion of context-awareness, the explicit combination of both techniques is a relatively unexploited research area, as it is analyzed in [3]. In fact, to date there are few clear examples of developed applications that explicitly combine sensor-based context-aware techniques in conjunction with traditional user modeling techniques. As user modeling-based applications that implicitly use context data as a part of their user models we can mention, for example, user’s interests in [2], and user’s tasks in [20]. Similarly,


there are few examples of context-aware applications that incorporate the notion of user model to support their adaptive behavior. As a significant example we can mention the GUIDE system [8], which uses a user model in order to represent visitor’s interests, preferences and the attractions that they have already visited. Nevertheless, there exists a great amount of context-aware applications that work without the intervention of a user model. We can mention the applications within the field called memory prosthesis, which in the early nineties became an important application domain in the research field of context-aware computing, introducing the notion of augmenting human memory. Examples in this field can be consulted in [3]. To finish this analysis, we want to comment that some of the challenges raised by the incipient everyday computing [1] are to consider the techniques that have arisen from the approaches developed through user-modeling and context-aware computing. This term describes a paradigm of use in which constant interactions may occur between users and ubiquitous computing resources, implicitly or explicitly, based on continuously changing user contexts.

3 Aspect Oriented Programming The increasing complexity of software products as well as the need for finishing developments in shorter and shorter periods of time is promoting the Software Engineering community to design and adopt new technologies that provide a more orthogonal separation of concerns. Among other the Aspect Oriented Programming (AOP) is gaining relevance, in comparison with other separation of concerns techniques, among them reflection, program transformation, or subjective programming. It is a paradigm extensively developed and used offering a new and promising evolution in the line of separation of concerns technologies. To strengthen the separation of concerns, the AOP focuses on those interest points or concerns of the application characterized by two factors: • On the one hand its influence and treatment crosscuts the system basic functionality, scattering along diverse modules that not necessarily are related each other (code scattering). We are referring to the well-known crosscutting concerns. Typical crosscutting concerns are the persistence, synchronization, exception handling, the logging operations, authentication, etc. The code scattering problem generates code redundancy problems. • On the other hand, these parameters are not orthogonal, but they appear interweaved, getting the main purpose of that code fragment tangled, and causing so a decrease in the code’s clarity (code tangling). The multiplicity of parameters to be taken into account in such a complex kind of applications, which inherently are related each other makes this problem unavoidable. The solution to this double issue will consist of extracting and encapsulating this type of problematic concepts, providing them modularization. Hence, we obtain not only separation, but also the possibility of condensing in a single program unit the whole treatment of the concept. Now then, this modularization requires a composition process (weaving) subsequent to compose the final operational system. It is the weaving phase, to be carried out by an especial compiler called weaver. The modularization unit is a program entity called generally aspect. It consists of an analogous structure to a class, which encapsulates all of its mechanisms. The program whose behavior is affected by aspects is called base program. The AOP [10] is the technology resulting from the effort in modularizing and de-coupling crosscutting concerns, avoiding at the same time redundancy and code tangling. It was proposed by Kiczales et al., (Xerox PARC) in 1990, in an attempt to surpass the increasing complexity of software systems, although it was designated with the term of AOP in 1996. This novel discipline causes more and more interest and relevance in the research community dedicated to the Software Engineering. Proof of that is the fact that today there exist international conferences dedicated to it.

Local Support to Plastic User Interfaces: an Orthogonal Approach Ubiquitous Computing, Mobile Computing and Context-Awareness are fields where meet certain circumstances such as treatment of crosscutting concerns that entangle between them and with the application code, presenting diverse decomposition dimensions. This is due to the multiplicity of factors that should be handled, with the aim of offering the maximum adaptation, and which rebound in a considerable increase of the complexity. The AOP is especially applicable to this type of systems to obtain the desired orthogonality in the separation of concerns that we do not achieve working only with reflection. AOP offers numerous advantages: major productivity; major ability to extension without having to re-implement, facilitating its evolution; smartness, as well as improvement in the validation phase, constituting all of them the key to a major code reusability. All that causes that AOP is a powerful programming paradigm.

4 Our Proposal

4.1 Previous Approach and Improvements On the client side our aim consists of developing an engine capable of adapting a specific UI in real time according to a set of factors related to the context of use. This kind of factors (henceforth real time constraints) should be represented (reification process4, depicted as the arrow 3 in figure 1) and treated separately from the core functionality of the system, as nonfunctional properties that affect the UI. The aim is to maintain a self-representation (explicit modeling) from the part of the system that we are interested in accessing, controlling and manipulating (the UI, together with the contextual factors that characterize it). The aim is to apply the necessary adjustments to the system’s behavior, in a transparent and dynamic way. This is precisely what makes up the essence of a reflective system or architecture [11]. Therefore we use a reflective architecture5 to implement the implicit plasticity engine we propose, due to its capacity of self-recognition and self-modification in runtime. This makes possible to extend and adapt the system behavior. Following this approach, the components related to the functionality of the application are represented in the base level of the reflective architecture (logic layer in the figure 1), and they are manipulated by the metalevel (adaptive layer in figure 1). Furthermore, the selfrepresentation of the system is located in the metalevel, making directly explicit and accessible certain properties that usually are not so – those related to the UI and to contextual factors. As these properties are normally common to diverse systems it is interesting to build and consider them as reusable and extensible components. Lastly, the metalevel is also responsible for controlling changes in the different contextual factors. A third level (context-aware layer in figure 1) will be in charge of detecting the environment and notifying the changes to the metaobjects. Hence, following the principle of abstraction, in the base level the system works without any conception of interface. This one is relegated to the metalevel where is being continually remodeled during runtime, putting introspection6 and reflection7 capacities to work. These

4

Process consisting of making concrete and materializing by means of objects the set of non-functional properties. This means making hidden aspects explicit and accessible, so amenable to inspection and adaptation. 5 Architecture that provides a system capacity to reason about and act upon itself, adjusting so its behavior according to certain conditions. Under an object-oriented reflective architecture view a system is considered to be made up of two parts: the application core and the reflective part to be separated in thus called base level and metalevel, respectively, making possible the system to self-recognize itself and to self-updating at run-time. 6 Capacity of a system to examine its own state in a fixed moment, making possible to control key events.


properties, which are inherent to a reflective architecture, are depicted as arrows 2 and 1 respectively in figure 1). As a result, the changes will be reflected in the system (base level) at an opportune time. Undoubtedly, the separation between the functional (base level) and the non-functional parts (metalevel) proposed by means of reflection prevents the non-functional properties from interfering with the basic functionality of the application, bringing flexibility in the development of the application core. However, the treatment of the non-functional properties is still being carried out as a whole, causing the same problems of code scattering and code tangling, although now located in the metalevel. To solve that and contribute to a major flexibility and reusability we propose to model the real time constraints, which constitute inherently crosscutting concepts, as aspects8 to influence in the metalevel, using the Aspect Oriented Programming (AOP). This idea, resumed in Figure 1, expresses the improvement proposed compared to the previous architecture.

Fig. 1. Architecture for the implicit plasticity engine.

The notation used in Figure 1, which is based in UML, has been taken from [12]. An example that illustrates an application of the implicit plasticity engine is showed in [15]. In particular, this example corresponds to a tele-aid system for high-mountain rescue. It is showed a possible aspectual decomposition that could be applied in this concrete case, according to the considerations exposed there. 4.2 Implementation Tools To carry out the adaptive layer structure presented before, in which reflection and AOP techniques intervene, and after having analyzed diverse platform options, we have decided to 7

Susceptibility to changes in the system, caused by a change in his self-representation. This property guarantees that each change made to the self-representation is immediately mirrored in the underlying system’s actual state and behavior. 8 Program unit to encapsulate crosscutting concerns.

Local Support to Plastic User Interfaces: an Orthogonal Approach adopt the J2ME (Java 2 Micro Edition) language, in combination with the AspectJ as an aspect oriented extension for Java. This platform guarantees the integration between both engines presented, as well as the dissemination of the implicit plasticity engine to the most wide range of mobile devices, including cellular phones due to the multi-platform character of Java. In this sense we are thinking in terms of a generic implicit plasticity engine. In relation to that, one of the most important reasons why we have chosen Java is that the solution offered by AspectJ is the most extended and consolidated solution to AOP, as is proven in [21]. It constitutes the most simple and practical extension of AOP for Java and it has a lot of strong points. Moreover, the weak point of Java, which is reflection capacity, can be compensated by means of the AspectJ weaving possibilities. To be exact, it uses the term pointcut (appeared in figure 1) to refer to those points in the code from which the code corresponding to the crosscutting concerns has been extracted in order to be injected afterwards. The entity aspect is in charge of intercepting them making possible reification and reflection, enhancing this way the Java reflection power. Finally, we would like to remark that as AspectJ generates pure Java byte code, applications using it will be able to be deployed in any mobile device that supports J2ME, which definitively are composed of the Java Virtual Machine.

5 Conclusions Though the benefits of incorporating a context-sensitive and anticipative component into a mobile and multi-appliance interactive system may be immense, the actual practicalities of doing so present some difficult design and implementation problems. A successful solution to the problem will mark a milestone in mobile computing. This paper tackles this notable problem and presents an architecture to support the implicit plasticity engine, in which the reflection and the AOP techniques are combined. On the one hand, the reflection arises as a mechanism to solve the context-awareness, allowing designing for change. On the other hand, the AOP helps to model and integrate the contextual factors in the system performance. Furthermore, AOP offers a mechanism to modularise and de-couple the crosscutting concerns optimally, thus obtaining the desired orthogonality in its treatment. Definitely, we can assert that the structure we propose here for the adaptive layer satisfies a double purpose. Firstly, it fulfils with the initial expectances offering an adaptive behavior and UI. Secondly, it is adjusted to the canons of flexibility, reusability and smartness that today should be demanded to any software product. Therefore it constitutes a valid option to model and manage the intrinsic concepts to the systems with implicit plasticity.

Referentes [1]

[2] [3]

[4] [5] [6] [7]

Abowd, G.D.; Mynatt, E. (2000). Charting Past, Present and Future Research in Ubiquitous Computing. ACM Transactions on Computer-Human Interaction, Special issue on HCI in the new Millenium, Vol 7(1), 29-58. Ardissono, L.; Goy, A. (2000). Dynamic Generation of Adaptive Web Catalogs. Proceedings of Conference on Adaptive Hypermedia and Adaptive Web-base Systems, Trento. Byun, H.E. ; Cheverst, K. (2001). Exploiting User Models and Context-Awareness to Suport Personal Daily Activities. Proceedings of Workshop on User Modelling for Context-Aware Applications, User Modelling Conference. Calvary, G.; Coutaz, J.; Thevenin, D. (2001). Supporting Context Changes for Plastic User Interfaces: a Process and a Mechanism. Proceedings of IHM-HCI’2001, 349-363. Calvary, G.; et al. (2002). Plasticity of User Interfaces: A Revised Reference Framework. Proceedings of TAMODIA 2002, 127-134. CAMELEON – Context Aware Modelling for Enabling and Leveraging Effective interactiON. (2003) http://giove.cnuce.cnr.it/cameleon.html Chen, G.; Kotz, D. (2000). A survey of context-aware mobile computing research. Technical Report TR2000-381. Computer Science Department, Dartmouth College, Hanover, New Hampshire.


[8]

[9] [10]

[11] [12]

[13]

[14]

[15]

[16]

[17]

[18] [19] [20]

[21]

Cheverst, K.; Davies, N.; Mitchell, K.; Smith, P. (2000). Providing Tailored (Context-Aware) Information to City Visitors. Proceedings of Conference on Adaptive Hypermedia and Adaptive Web-based Systems, Trento. Dix, A.; Finlay, J.; Abowd, G.; Beale, R. (1998). Human-Computer Interaction (2nd ed.). Prentice Hall Europe. Kiczales, G.; Lamping, J.; Mendhekar, A.; Maeda, C.; Lopes, C.; Loingtier, J.-M.; Irwin, J. (1997). Aspect-oriented Programming. M. Aksit and S. Matsuoka, ed., 11th ECOOP’97, Vol. 1241 of LNCS 220-242. Maes, P. (1987). Concepts and Experiments in Computional Reflection. Proceedings of the 2nd OOPSLA’87 147-156. Pawlak, R.; Duchien, L.; Florin, Gl.; Legond-Aubry, F.; Seinturier, L.; Martelli, L. (2002). A UML Notation for Aspect-Oriented Software Design. Proceedings of AO Modelling with UML Workshop at the AOSD 2002 Conference. Sendín, M. (2004). Dichotomy in the Plasticity Process: Architectural Framework Proposal. Advances in Pervasive Computing. Austrian Computer Society. Doctoral Colloquium in Pervasive Computing. 141-147. Sendín, M.; Lorés, J. (2004). Plastic User Interfaces: Designing for Change. Proceedings of the Making model-based UI design practical: usable and open methods and tools Workshop (IUICADUI'04). Sendín, M.; Lorés, J. (2004). Plasticidad Implícita en dispositivos móviles: hacia la Ortogonalidad deseada en la Separación de Conceptos. Proceedings of the V Congreso en Interacción PersonaOrdenador. Sendin, M.; Lores, J. (2004). Plasticity in Mobile Devices: a Dichotomic and Semantic View. Proceedings of Workshop on Engineering Adaptive Web, supported by AH 2004, Eindhoven, 58-67, ISSN: 0926-4515. Schilit, B.; Adams, N.; Want, R. (1994). Context-aware Computing Applications. Proceedings of IEEE Workshop on Mobile Computing Systems and Applications, Santa Cruz, California, 85-90. IEEE Computer Society Press. Thevenin, D. (2001). Adaptation en Interaction Homme-Machine: Le cas de la Plasticité. PHD Thesis, Joseph Fourier University, Grenoble. Thevenin, D.; Coutaz, J. (1999). Plasticity of User Interfaces: Framework and Research Agenda. Proceedings of Interact’99, Edinburgh, 110-117. Vassileva, J. (1997). Ensuring a Task-based Individualized Interface for Hypermedia Information Retrieval through Use Modeling. Maybury M.T. (ed.) Intelligent Multimedia Information Retrieval, MIT Press, London. Xerox PARC. (2002). AspectJ home page. Web.