A Conceptual Framework for Adaptation and Personalization in Virtual ...

A Conceptual Framework for Adaptation and Personalization in Virtual Environments Johanna Renny Octavia Chris Raymaekers Karin Coninx Hasselt University - tUL - IBBT Expertise Centre for Digital Media Wetenschapspark 2, 3590 Diepenbeek, Belgium Email: {johanna.octavia,chris.raymaekers,karin.coninx}@uhasselt.be

Abstract—When interacting in a virtual environment, users are confronted with a number of interaction techniques. These interaction techniques may complement each other, but in some circumstances can be used interchangeably. Because of this, it is difficult for the user to determine which interaction technique to use. Furthermore, the use of multimodal feedback, such as haptics and sound, has proven beneficial for some, but not all, users. This complicates the development of such a virtual environment, as the designers are not sure of the implications of the addition of interaction techniques and multimodal feedback. A promising approach for solving these problems lies in the use of adaptation and personalization. By incorporating knowledge of a user’s preferences and habits, the user interface should adapt to the current context of use. This could mean that only a subset of all possible interaction techniques is presented to the user. Alternatively, the interaction techniques themselves could be adapted, e.g. by changing the sensitivity or the nature of the feedback. In this paper, we propose a conceptual framework for realizing adaptive personalized interaction in virtual environments. Furthermore, we discuss how a user model can be built, as this model forms the important first step in implementing the conceptual framework. Keywords-virtual environments; adaptation; framework;

I. I NTRODUCTION Performing complex tasks in virtual environments (VE) requires to employ highly interactive three-dimensional (3D) user interfaces, while keeping the user interaction to be natural and intuitive [1]. In medical care for example, an interactive VE can facilitate surgeons to perform complex operations such as practicing new surgical procedures on simulated individuals, but still maintaining their interaction as natural as possible. Therefore, employing multimodal interaction becomes essential in the establishment of the VE applications since it makes optimal use of human senses. The nature of multimodal virtual environments utilizes a wide range of 3D interaction techniques from simple to very complex. When users need to carry out a particular task in such environments, they are confronted with a high degree of freedom in deciding which to choose from the various interaction techniques. This situation may possibly introduce

additional complexity and cognitive load for users, which in the end might hinder their interaction. To enhance user interaction in virtual environments, we envisage providing adaptive personalized 3D user interfaces to be a potential solution. Adaptive and personalized interaction in virtual environments, which has the ability to adapt to users intelligently, may increase user satisfaction and performance, and eventually enhance user interaction. This vision is disclosed in a conceptual framework for adaptive personalized 3D user interfaces, with the user model as one main building block. This paper describes the proposed conceptual framework and the results of ongoing work on constructing the user model. II. R ELATED W ORK In the context of this paper, we define adaptive and personalized interfaces as interfaces that reflect user characteristics, preferences, needs and abilities to automatically adapt the behavior accordingly using intelligent mechanisms. Over the past two decades, there has been a quite amount of work in the area of adaptive and personalized user interfaces. Early work included an adaptive personalized interface for a menu-driven application of a telephone-directory system [2], which was shown to be viable and superior to a non-adaptive version. This finding raised the belief that designing systems or interfaces that adapt to users is possible and beneficial. Several studies have focused on interfaces that adapt to users interacting in the window, icon, menu and pointing (WIMP) applications. Gajos [3] proposed the idea of automatically personalized user interfaces, mainly in Microsoft Office applications, with the design of SUPPLE and ARNAULD. Rocchi et al. [4] developed an adaptive multimedia mobile guide, which operates on a PDA and presents adaptively selected information to museum visitors based on their expressed interests. As interaction in 3D interfaces is more complex than in WIMP, this could be a reason why adaptation and personalization of 3D interfaces has been explored less often. Nevertheless, there has been a number of attempts towards it. Wingrave et al. [5] investigated an approach to VE interface design that adapts to users’ preferred method

of interaction, that later on was introduced as personalized nuance-oriented interaction. Celentano et al. [6] discussed adaptivity of interaction in 3D environments by identifying user recurring interaction patterns and using the learned user behavior to shorten user interaction in a virtual world. Much research into three-dimensional (3D) interaction techniques has been conducted over the past years. However, the usability of many interactive VE applications has not yet achieved a desirable level. Bowman et al. [1] proposed considering user-specificity in the design of 3D user interfaces to improve the usability of 3D interaction techniques. This is justifiable since we realize that individual differences have influences on user performance. To accommodate user diversity, building user models becomes crucial to provide adaptation and personalization for users. User models contain information and assumptions about users that play an important role in the adaptation process of user interfaces to the needs of different users. Jameson and Wittig [7] proposed a user modeling approach that learns two types of user models, namely general user models and individual user models. A general user model is learned from all observations of a sample of users and can be applied to all users in general. An individual user model is learned completely from observing one particular user and later can be adapted during the interaction. Another user modeling method is the stereotype approach discussed by Kobsa [8], which identifies user groups and generalizes their interaction behaviors into patterns to construct group user models. Not much attention has been spent on the effort of establishing user models for the purpose of adaptation and personalization in virtual environments. III. C ONCEPTUAL F RAMEWORK We have been aware of the importance of enhancing user interaction in virtual environments through the establishment of adaptive personalized 3D user interfaces. Our goal is to support adaptation and personalization in order to assist users when interacting in virtual environments. This includes determining which interaction technique is suited for a user in a certain situation (e.g. by only providing a limited number of interaction techniques) and adapting the interaction techniques themselves (e.g. by adjusting the sensitivity of the interaction). The adaptation should occur not only according to the users’ characteristics, such as their preferences and abilities, but also to the actions they performed in the past and the task they are currently executing. To achieve this, we propose our conceptual framework as depicted in Figure 1. The main goal of this framework is to come up with most suited 3D interaction techniques for users through adaptation and personalization. With the framework, we intend to gather information and build knowledge about users on their working methods, performances, preferences and abilities when interacting in virtual environments. This knowledge will later on be used to assess the adaptation

Figure 1. interfaces

Conceptual framework for adaptive personalized 3D user

and personalization of interaction techniques, either deciding which particular interaction technique should be offered or regulating how an interaction technique should behave, depending on the performance and preference of users. When interacting in virtual environments, users carry out certain actions. The recurring interaction patterns are identified and delivered into the user model. User actions are also recognized and monitored as triggers for adaptation in the monitoring module. When it identifies new patterns, the user model will be updated. When assessing the trigger, the user model closely collaborates with the knowledge base, which contains factual information about the past interaction and adaptation. This collaboration provides information to the adaptation engine, where the adaptation and personalization take place with the help of intelligent algorithms. As a result, adaptive personalized 3D user interfaces can be realized. We realize that the conceptual framework consists of several building blocks that need to be investigated further. In this work, we focus on the establishment of one component proposed in the framework, namely the user model. A. User Model The user model is acquired through user modeling activity, which attempts to gather users’ interaction patterns, preferences, interests and characteristics. In the framework, we employ a user model template combined with individual user models. The user model template is developed by establishing general and group user models. This template is used as a starting point in adaptation process for users that do not have any prior interaction history.

General user models are advantageous to people using a virtual environment for the first time to benefit instantly from adaptation and personalization. Group user models are employed because we believe users interacting in virtual environments can be classified into different user groups based on different user characteristics such as experience level, gender or age. However, basing the adaptation only on the user model template is insufficient because users remain individuals that vary so much from each other. Therefore, individual user models are employed in parallel with the user model template. There are various levels of information provided in the user model. The general user model provides the most basic information that can be used for adaptation to all users. The group user model gives more specialized information to be applied to a group of users, while the individual user model delivers the most detailed information about one particular user. When the information is conflicted between the levels, the information from the higher level takes priority over the information from the lower level. B. Adaptation Engine The adaptation engine determines the kind of adaptation and personalization with the help of three intelligent algorithms, namely the mapping algorithm, the predicting algorithm and the adapting algorithm. The mapping algorithm can help to selectively offer interaction techniques or metaphors to users depending on the actual context of use. Users may have different ways of interacting in 3D environments, depending on the devices used during the interaction (e.g. large display, desktop screen, or interactive table). The predicting algorithm can help to enable or disable interaction techniques based on the prediction made according to the semantic of objects and the history of prior used interaction techniques. Users may have built a preference of interaction technique when selecting objects in dense environments, which might differ when in sparse environments. The adapting algorithm can help to determine whether it is desirable to adapt the interaction technique or enhance it with other modalities such as audio or force feedback. Users may have a certain characteristic that influences their performance in 3D environments, such as the level of experience might hinder or encourage the use of a particular interaction technique. IV. E STABLISHMENT OF U SER M ODELS We initiate the implementation of the conceptual framework by constructing the user models. As proposed, we will build a user model template (consisting of general and group user models) and individual user models. To date, we have made an effort to construct the user model template and succeeded in constructing the general user model.

(a) Figure 2. depth ray

(b)

The selection techniques used: (a) The bubble cursor (b) The

A. Context In this work, we focused on investigating user interaction when performing selection tasks in a virtual environment with controlled variables, such as high density of objects, target distance and occlusion. Object selection is a fundamental task in virtual environments and any interactive 3D user interfaces must support performing this task. Two selection techniques as shown in Figure 2, the bubble cursor and the depth ray, were considered because these techniques were found to perform better than the others when selecting objects in dense and occluded virtual environments [9]. The bubble cursor is based on the hand extension metaphor, which resembles the action of reaching out one’s hand to the target object to be selected. The depth ray is based on the ray casting metaphor, which resembles the action of pointing one’s hand at the target object to be selected. B. Group User Model In order to build the user model template, we conducted an experiment on 3D target acquisition task in virtual environments with four user groups differing in experience level and gender [10]. In the experiment, participants were asked to select a target object randomly positioned among distractor targets, as fast as possible while minimizing errors. As shown in Figure 3, a Polhemus Fastrak 6 DOF magnetic tracker was used as input device, together with a polarization projection screen (2.4 m x 1.8 m ) with passive stereo using two DLP projectors. Sixteen participants between 18 and 31 years old were recruited for this experiment, which lasted for about 45 minutes. In the experiment, we found that experience level has effects on users’ performance for 3D target acquisition task in a virtual environment, while gender not. Experienced users perform significantly better than novices, while males and females do not perform significantly different. Even so, it was also found that there is no difference between the user groups with regard to the selection technique itself. Prior to the experiment, we expected to be able to construct group user models based on the differences between those user groups with respect to their performance and preference

(a)

(a)

(b) Figure 3. The experiment apparatus: (a) The magnetic tracker (b) The projection screen (b)

for interaction technique. Due to the fact that no trends were found, group user models were unsuccessful to be constructed. Nevertheless, we are aware that besides experience level and gender, there are other user characteristics that can be taken into account when constructing group user models, such as age, physical characteristics, physical and cognitive abilities. For a more detailed description of the findings and the experiment design and procedure, please refer to [10]. C. General User Model Another element of the user model template is the general user model. As shown in Figure 4, we constructed the general user model based on the performance and preference of participants as a whole. Table I shows the general user model in summary. This provides information on the appropriate selection technique for executing 3D target acquisition task in a virtual environment, for all users in general. This general user model can be beneficial for first-time users interacting in a virtual environment who do not have any interaction history at all. For instance, users can be offered to use the depth ray technique when they start interacting in a virtual environment with dense objects,

Figure 4. General user model: (a) Performance-based (b) Preference-based

Table I G ENERAL USER MODEL Environment condition Interaction technique Occluded-Dense-Near (ODN) Bubble Cursor Occluded-Dense-Distant (ODD) Bubble Cursor Occluded-Sparse-Near (OSN) Bubble Cursor Occluded-Sparse-Distant (OSD) Bubble Cursor Visible-Dense-Near (VDN) Bubble Cursor Visible-Dense-Distant (VDD) Depth Ray Visible-Sparse-Near (VSN) Bubble Cursor Visible-Sparse-Distant (VSD) Depth Ray

visible and distant target. When the environment switches to the one with sparse objects, occluded and distant target, they can be offered the bubble cursor technique. By applying the general user model, we expect that the users’ interaction may be optimal, even though they are still novice to virtual environments.

D. Individual User Model Aligned with the user model template, we will also employ individual user models. The individual user model will serve an important role in the adaptation process of user interfaces for every particular user. Especially since we observed in the experiment that every user has his/her own interaction pattern that differs from person to person. To construct an individual user model, we need to build knowledge on a user’s performance and preference by analyzing each user individually. This user model requires prior traces of interaction of one user, and will be continuously adjusted during the interaction itself. Compared to a general user model that is static, an individual user model is dynamic. The establishment of the individual user models for this study will be carried out in a next experiment. Furthermore, another goal of the experiment is to verify the general user model illustrated in Section IV-C. Through this experiment, we would also like to investigate how users react to adaptation of interaction technique based on the result of combining the general and individual user models. The same experimental set-up will be applied, with some alteration in the experimental design and procedure. The experiment will consist of two sessions. The first session will be conducted to build the individual user model, while the second session will be for testing the general user model and inquiring insights into the user’s reaction to adaptation. The individual user model resulted from the first session will be combined with the general user model in runtime, then this action will inform the system to decide further whether the adaptation of interaction technique should occur or not. V. C ONCLUSIONS AND F UTURE W ORK We presented a conceptual framework for enhancing user interaction in virtual environments by means of adaptation and personalization of interaction techniques. We also have described our effort to build the user model, both the user model template and individual user models, which is an important building block of the proposed framework. The general user model has been presented although it still needs to be verified. The construction of the user models will be carried on in the upcoming experiment. As a priority in the next phase of the research, we would like to construct individual user models, bring the general and individual user models in practice, and examine users’ response to adaptation and personalization (both subjectively and objectively) based on the user models’ application. We also realize that further studies of the various building blocks

of the framework still need to be conducted. Performing similar studies that consider other 3D interaction techniques such as navigation or manipulation may be compelling to explore more possibilities for adaptation and personalization in virtual environments. ACKNOWLEDGMENT Part of the research at EDM is funded by the ERDF (European Regional Development Fund) and the Flemish government. The work described in this paper is funded by the transnational University Limburg (tUL). R EFERENCES [1] D. A. Bowman, J. Chen, C. A. Wingrave, J. Lucas, A. Ray, N. F. Polys, Q. Li, Y. Haciahmetoglu, J.-S. Kim, S. Kim, R. Boehringer, and T. Ni, “New directions in 3d user interfaces,” Int. J. Virtual Reality, vol. 5, no. 2, pp. 3–14, 2006. [2] S. Greenberg and I. Witten, “Adaptive personalized interfaces: A question of viability,” Behaviour and Information Technology, vol. 4, no. 1, pp. 31–45, 1985. [3] K. Z. Gajos, “Automatically generating personalized user interfaces,” Ph.D. dissertation, University of Washington, 2008. [4] C. Rocchi, I. Graziola, D. Goren-Bar, O. Stock, and M. Zancanaro, “Adaptive multimedia guide,” in PEACH: Intelligent Interfaces for Museum Visits, 2007, pp. 3–22. [5] C. A. Wingrave, D. A. Bowman, and N. Ramakrishnan, “Towards preferences in virtual environment interfaces,” in Proceedings of the 8th EGVE, 2002, pp. 63–72. [6] A. Celentano, M. Nodari, and F. Pittarello, “Adaptive interaction in web3d virtual worlds,” in Proceedings of the 9th 3D Web, 2004, pp. 41–50. [7] A. Jameson and F. Wittig, “Leveraging data about users in general in the learning of individual user models,” in Proceedings of the 17th IJCAI, 2001. [8] A. Kobsa, “User modeling: Recent work, prospects and hazards,” in Adaptive User Interfaces: Principles and Practice, M. Schneider-Hufschmidt, U. Malinowski, and T. Kuhme, Eds. Elsevier Science Inc., 1993, pp. 111–128. [9] L. Vanacken, T. Grossman, and K. Coninx, “Multimodal selection techniques for dense and occluded 3d virtual environments,” Int. J. Hum.-Comput. Stud., vol. 67, no. 3, pp. 237–255, 2009. [10] J. R. Octavia, C. Raymaekers, and K. Coninx, “Investigating the possibility of adaptation and personalization in virtual environments,” in UMAP 2009, ser. LNCS, G. Houben, G. McCalla, F. Pianesi, and M. Zancanaro, Eds., vol. 5535. Springer, 2009, pp. 361–366.