Learning About the User – User Modeling and Machine Learning Wolfgang Pohl Universit¨at GH Essen Department of Mathematics and Computer Science c/o GMD, FIT.MMK
[email protected]
Abstract User modeling is employed by applications that need to maintain explicit models of their users in order to exhibit individualized behaviour. The user modeling task involves representation and acquisition of assumptions about the user. Particularly user model acquisition is closely related to the machine learning task of automatically acquiring new information as well as new representations of existing information. This paper shows how and for which purposes machine learning techniques have been and could be employed in user modeling. Also usage modeling, a more action-centered approach to user modeling, is considered. Finally, the LaboUr approach to user modeling is sketched, which regards user modeling as learning problem.
1 Introduction User modeling is aimed at providing an explicit information basis (a socalled user model), about a user’s knowledge, goals, preferences etc. to application systems that try to adapt their behaviour to their users’ individual characteristics. The main tasks of user modeling systems are the proper representation of the user model and the acquisition of assumptions about the user. In current user modeling systems, among them a number of general user modeling tools (so-called user modeling shell systems; e.g., BGP-MS [Kobsa
and Pohl, 1995] or TAGUS [Paiva and Self, 1995]), the facilities for user model representation are often quite sophisticated. On the one hand, there are classical, symbolic approaches the representational power of which ranges from propositional facts to full-fledged modal logic. On the other hand, the importance of probabilistic techniques has lately been growing (for an overview, see [Jameson, 1995]), since they seem to be more appropriate for dealing with the uncertainty that is inherent to user models. While representation of assumptions about user beliefs, goals, and plans is a well-researched problem, the acquisition of user models is still very difficult. In applications that are in control of the user’s behaviour (like tutoring systems), the mapping of user actions to user model contents is quite straightforward. But when the interaction between user and application is user-controlled, it is hard to identify relevant information that may later on serve to justify an adaptation step. Some standard user model acquisition techniques in such situations are:
user interviews (should occur only occasionally because the system takes control of the interaction) rules that describe – often in a simple and straightforward way – which entries into the user model should be made as the consequence of some observed user action stereotypes, i.e. models of user groups that serve as source for default inferences about the user in case of his being a group member inference procedures of the representation system that acquire implicit or secondary assumptions A common problem of these techniques is that they are quite inflexible and that their validity crucially depends on pre-development investigations. In the case of numeric representation, the flexibility problem seems to be less hard, but here it is not obvious which user actions should influence which available numeric entitities. Furthermore, the problem of defining the correct inference rules often is only replaced by the problem of choosing the right conditional probabilities (e.g., in Bayesian networks). So, in general, a lot of a priori work and knowledge is required for reliable user model acquisition. In the following, I will try to show how machine learning techniques could be used to improve current user modeling techniques. First, possible benefits for user model representation will be described. The subsequent section then
focusses on user model acquisition and mainly discusses a first approach of employing learning methods for this purpose. It turns out that a shift to a more action-oriented view of user modelling is required. Consequently, in the fourth section, a fairly new issue in user modeling shall be discussed, namely the problem of “usage modeling”, that may also benefit from the employment of machine learning methods. Finally, an approach to learning about the user is suggested, which integrates the before-mentioned techniques and assigns machine learning methods a central role in the user modeling game.
2 Machine Learning for User Model Representation Methods and formalisms for representing user models must satisfy several demands, like the abilities to distinguish several aspects of user models (e.g., beliefs, goals, and preferences) and to control the consistency of user models. One of the main tasks in user model representation is to cope with the dynamic nature of users and – consequentially – user models. New assumptions about the user must be added to the user model, and existing assumptions must be changed, revised, or completely deleted. Dependencies between user model contents, whether resulting from domain constraints (you cannot have goal X if you believe in proposition Y) or inference processes (belief A has been inferred from beliefs B and C), must be taken into account. In order to deal with these requirements, reason maintenance techniques have been developed for several user modeling to handle such dependencies. Dynamically changing knowledge bases and domain models are typical for machine learning systems. Therefore, problems like consistency detection and knowledge revision are often tackled by machine learning tools. For example, MOBAL [Morik et al., 1993] offers solutions to consistency detection, knowledge revision [Wrobel, 1994], and knowledge base restructuring [Sommer, 1995]. Tools like MOBAL must be examined, in how far they can deal with specific user modeling problems like default reasoning based on stereotypes. Moreover, it is not obvious how systems like MOBAL that are aimed at the interactive development of knowledge bases can be utilized for the mainly non-interactive process of user modeling. A large number of machine learning methods realize inductive reasoning. Inductive inference techniques like rule discovery and concept formation are probably most welcome add-ons to the inference capabilities of current user
model representation methods. For example, a standard approach to individualized information presentation is to tailor system output to the user’s assumed knowledge about the concepts of an underlying domain. Concept formation techniques [Wrobel, 1994] might be used to trace the user’s conceptualizations and to compare them to the concepts in the domain knowledge base in order to decide if a user knows a concept (to some degree) or not.
3 Machine Learning for User Model Acquisition Current user modeling systems very often rely on their input being wellprepared and corresponding tightly to the available representation entities. That is, applications have to explicitly tell the user modeling system about what the user is supposed to believe or what his goals might be, or which probabilities in the user model should increase or decrease. This requires that applications themselves transform observed user behaviour into assumptions about the user. An example is the adaptive hypertext system KN-AHS [Kobsa et al., 1994] that observes the user ask for an explanation about a hotword in a hypertext document and, as a consequence, tells the user modeling system that the user is not familiar with the concept that corresponds to the hotword. In this system, like in many others, the burden of user model acquisition is mainly laid upon the application. This situation can be improved only if the transformation of observed user behaviour into assumptions about the user is done by the user modeling system. A first approach would be to shift the mostly heuristic acquisition rules of current adaptive applications into user modeling systems. Application developers, who employ the user modeling shell system BGP-MS, can define simple acquisition heuristics in a declarative manner. These heuristics will later be processed by the resulting user modeling system to generate user model entries from single user actions that have been observed by an application and reported to the user modeling system [Pohl et al., 1995]. However, if more satisfactory solutions shall be found particularly for user modeling shell systems, methods are required that process observed user behaviour incrementally and take former behaviour into account. Considering these criteria, machine learning methods seem to be promising candidates. Such a learning approach to user model acquisition has been pursued by Jon Orwant. In [Orwant, 1995] he describes the general user modeling system
¨ DOPPELGANGER , which employs several statistical and (machine) learning techniques in order to process different streams of observations about users into user and so-called community models:
Beta distribution is used for computing the strengths of simple assumptions about user preferences from one-bit observations (e.g., the user likes an article of an electronic newspaper or not). These preference assumptions help to compose personalized electronic news. Linear prediction is employed to predict the times at which users will be logging in or reading their electronic newspaper, based on previously observed series of login and reading times. Reading time prediction helps to create personalized news in time. Markov models assist in the prediction of transitions in locality and working state of the user. An unsupervised clustering algorithm (ISODATA) partitions the user model space into community models. Like stereotypes, they are a source for default assumptions about users; missing information in a user model is substituted by data from the most similar community model. Orwant switches from a paradigm of user model acquisition, which presupposes application-controlled communication between application and user modeling system about normally application-specific user model contents, to another that automatically transforms streams of lower-level data from normally application-independent “sensors” into general assumptions about and predictions of user behaviour. There are several basic problems with this approach, which are also admitted by Orwant: First, observation data often has to be pre-processed in order to become appropriate input for a specific learning technique. Second, it is not trivial to choose suitable learning algorithms for each data stream. In addition, more general problems arise: On the one hand, a great part of the user modeling data that is useful for an application remains applicationspecific. Also, a lot will be lost if application-specific acquisition is given up. A mix of techniques must be sought that, e.g., can combine pre-defined application stereotypes with community clustering. Decisions about the transformation of sensor data and about the selection of learning techniques can probably be made better and more precisely, if they are based on the
background knowledge that exists for an application or at least a domain. On the other hand, Orwant deals with completely pragmatic assumptions about the user. It is not obvious, how machine learning techniques can help with more mentalistic notions like beliefs and goals of the user. Assumptions of these kinds are used by a considerable range of user modeling applications. Here is perhaps a potential for learning tools that go beyond the mainly statistical methods employed by Orwant in their ability to produce own hypotheses about users’ mental states.
4 Machine Learning and Usage Modeling Currently, individualizing an application mostly relies on user models that are concerned about what the user is. In the early times of user modeling in (natural-language) dialog systems [Kobsa and Wahlster, 1989], user models were supposed to be mainly inferred from user utterances. However, utterances can be regarded as a special kind of actions, and since user modeling has shiftedits focus from dialog systems to all kinds of interactive applications, it can be more generally said that assumptions about what the user is have to be acquired from what the user does. But recently, there have also been systems that treat user actions not only as source of information for modeling mental attitudes like beliefs and goals of users, but focus on modeling the user’s behaviour.
4.1 Interface Agents In the last years, an important stream of research has been concerned with interface agents [Maes, 1994] or personal assistants [Mitchell et al., 1994]. Interface agents are software systems that assist the user in a personalized way. A characteristic of such a system is that it “becomes gradually more effective as it learns the user’s interests, habits and preferences” [Maes, 1994]. Currently, the predominant capability of interface agents is to learn correlations between situations the user may encounter and the corresponding actions he performs. These data are used to predict user behaviour in future situations, to suggest appropriate actions to the user, and perhaps automatically perform actions on the user’s behalf. Examples are an agent for scheduling meetings, which suggests reactions or itself reacts to meeting invitations [Kozierok and Maes, 1993], and the assistant system ‘WebWatch-
er’ [Armstrong et al., 1995], which advises the user concerning hyperlink selection on a WWW page, given an information seeking goal of the user. While the learning methods used by interface assistants seem to provide a promising solution to the acquisition problem (Maes calls it the “competence” problem), their possibilities are restricted. [Maes, 1994] presents several applications that follow the same principle: learning the importance of situation features for each user and their correlations to user actions. Such interface agents can be employed for assistive tasks that mainly consist of predicting actions in given situations. However, there are a lot of applications that do not fit into this assistant scheme. E.g., in information presentation (technical manuals, expert system explanations) the user’s knowledge about the information domain must be considered. Also behaviour-oriented tech¨ niques like time and location prediction in DOPPELGANGER cannot solely rely on situation-action correlations. Common to these applications is that they do not relieve the user from actions that he could have performed himself (perhaps worse), but try to personalize system actions that would be performed anyway, or give additional, personalized support to improve interaction. In spite of its lack of generality, the interface agents approach shows that it can be very beneficial to watch user actions and utilize learning techniques for acquiring personalization knowledge based on these observations. Orwant has already shown first steps in generalizing this approach to other kinds of assumptions about the user than only situation-action correlations. Still, a more general notion of how adaptivity and individualization can be based on learning from user actions must be found.
4.2 Usage modeling Recently, there have been other approaches to adaptivity that are based on observing and recording user behaviour. In these works, the term “usage profile” [Krogsæter et al., 1994] has been coined for collections of usage data, and “usage modeling” [Grunst et al., 1993] has been used to name this action-centered approach to application individualization. Examples for systems working with usage profiles are Flexcel [Krogsæter et al., 1994] and Basar [Thomas and Fischer, 1996]. Flexcel makes use of statistics about how frequently different function parameterizations were selected by the user in order to suggest new key shortcuts, to change parameter defaults, or to eliminate the dialogue step of functions. Basar stores calls and results of World-Wide Web search processes (performed by search agents)
in the usage profile, together with the user’s assessment of the search results. This assessment stems from an interpretation of the actions the user performed on the search results and is the basis for an individualized behaviour of the search agents in further search processes. Usage modeling, as presented in these systems, is similar to the idea of interface agents in that assumptions about the user are not only acquired from user actions but also deal with what the user does and how he does it. Particularly the functionality of Basar might be implemented as a learning assistant. However, both Basar and Flexcel demonstrate that usage modeling can be more than learning agents. They build explicit usage models that contain more information than situation-action correlations, and particularly Flexcel goes beyond pure assistivity by trying to suggest interaction improvements. A disadvantage of these systems is that their usage models are still quite simple and therefore of only limited and straightforward applicability.
4.3 Intelligent Usage Modeling For more difficult tasks, more powerful techniques are needed. Machine learning methods like those employed by learning interface agents will certainly be crucial for intelligent usage modeling. However, concerning personalized support that goes beyond situation-action prediction, several questions will have to be answered: How can learning examples be identified? How can learning data be coded? Which learning method shall be chosen for ¨ given inputs? It can be foreseen that, like in DOPPELGANGER , a variety of techniques must be available, and that a usage modeling system will have to decide which method to employ under which circumstances. A good example for intelligent usage modeling is programming by demonstration (PBD, cf. [Cypher, 1993]). A PBD system generates generalized programs from recorded actions. For the generalization process, machine learning techniques in form of induction algorithms are used. However, in standard PBD systems, the user still has to explicitly instruct the system to “watch”. A less obtrusive approach is to permanently watch the user and automatically identify action sequences that could be generalized to macro operations. Such an approach has already been suggested for the system X-AiD, a prototype for an adaptive and knowledge-based user interface [Thomas et al., 1987], and heuristics for finding macro examples were introduced in [Pohl, 1992]. Macro learning at the user interface shows what intelligent usage modeling can be. Besides, it is an example for a possible
application of machine learning techniques where the learning does not only consist of processing a stream of fairly well-formed example data (this is the case in Orwant’s work), but also of detecting relevant examples in a more noisy data stream.
5 The LaboUr approach In the preceding sections, several aspects of an employment of machine learning techniques for acquiring and maintaining assumptions about the users of interactive computer systems were discussed. The following theses result from this discussion:
Machine learning methods can help to cope with the problem of getting user models dynamically updated. The acquisition task should be assigned to the user modeling system. It can be regarded as modular processing of several, more or less noisy data streams with suitable learning and prediction techniques. Not only shall assumptions about the mental attitudes of users be considered. Assumptions about the user must also be concerned with user behaviour and activities. This leads to usage models. If one adopts these ideas, user modeling (in its above-mentioned general sense of acquiring and maintaining all kinds of assumptions about users) evolves into a learning problem. In future work, we will try to develop an approach to user modeling pursuing this notion, which will be referred to as LaboUr (Learning about the User) approach. The following problems will have to be tackled:
Develop a general interface between interactive applications and user modeling systems that allows the transmission of data about observed user behaviour along different data streams. Identify classes of observation data and appropriate learning methods for processing these data. This should result in rules (of probably heuristic nature) for selecting from a pool of learning methods. This leads to the notion of intelligent learning: methods must be selected and combined, and learning examples must perhaps be discovered in noisy data streams.
Utilize clustering and other aggregation methods to find commonalities in user models, to build or add to group models, and to make applications share assumptions about users. Integrate the classical way of modeling mental attitudes (e.g. beliefs and goals) of users with modeling habits or patterns of behaviour. Identify relationships between the corresponding kinds of assumptions about the user. Apply knowledge revision techniques for dynamically updating particularly knowledge-based user models. In addition, LaboUr systems will face a general problem: The circumstances of learning for user-adapted interaction are different from classical machine learning situations. In order to really improve the interaction between human and computer, user modeling activities should disturb the user as little as possible. For the development and application of learning algorithms, this means that training sessions should be avoided, that resourceintensive processing must be delayable to off-line and idle times of the user, and that the user should not be unnecessarily forced to interact with the learning process (i.e., learning should be unsupervised and thus transparent to the user).
6 Conclusion User models are a prerequisite for personalized interaction between computer systems and their users. Current approaches to user modeling often do not adequately cope with the problems of acquiring assumptions about users (the user model contents) and dynamically handling changes in user models. This paper has tried to show that machine learning techniques may be benificially applied to solve these problems. Existing applications of learning methods in user modeling and personalized interaction, their potentials and problems have been described. It has turned out that modeling user behaviour, also called usage modeling, is crucial for providing intelligent, personalized user support. Finally, the LaboUr approach to user modeling has been sketched, which is based on the idea of user modeling as a learning process.
References [Armstrong et al., 1995] R. Armstrong, D. Freitag, Th. Joachims, and T. Mitchell. WebWatcher: A Learning Apprentice for the World Wide Web. Proc. of the 1995 AAAI Spring Symposium on Information Gathering from Heterogeneous, Distributed Environments, March 1995. [Cypher, 1993] A. Cypher, editor. Watch What I Do: Programming by Demonstration. The MIT Press, Cambridge MA, 1993. [Grunst et al., 1993] G. Grunst, R. Oppermann, and C. G. Thomas. Benutzungsmodellierung bei kontext-sensitiver Hilfe und adaptiver Systemgestaltung. In A. Kobsa and W. Pohl, editors, Arbeitspapiere ABIS-93. WIS Memo 7, pages 69–77. University of Konstanz, 1993. [Jameson, 1995] A. Jameson. Numerical Uncertainty Management in User and Student Modeling: An Overview of Systems and Issues. User Modeling and User-Adapted Interaction, 5(3-4):193–251, 1995. [Kobsa and Pohl, 1995] A. Kobsa and W. Pohl. The User Modeling Shell System BGP-MS. User Modeling and User-Adapted Interaction, 4(2):59– 106, 1995. [Kobsa and Wahlster, 1989] A. Kobsa and W. Wahlster, editors. User Models in Dialog Systems. Springer, Berlin, Heidelberg, 1989. [Kobsa et al., 1994] A. Kobsa, D. M¨uller, and A. Nill. KN-AHS: An Adaptive Hypertext Client of the User Modeling System BGP-MS. In Proc. of the Fourth International Conference on User Modeling, pages 99–105, Hyannis, MA, 1994. [Kozierok and Maes, 1993] R. Kozierok and P. Maes. A Learning Interface Agent for Scheduling Meetings. In W. D. Gray, W. E. Hefley, and D. Murray, editors, Proc. of the International Workshop on Intelligent User Interfaces, Orlando FL, pages 81–88, New York, 1993. ACM Press. [Krogsæter et al., 1994] M. Krogsæter, R. Oppermann, and C. G. Thomas. A User Interface Integrating Adaptability and Adaptivity. In R. Oppermann, editor, Adaptive User Support. Lawrence Erlbaum Associates, 1994. [Maes, 1994] P. Maes. Agents that Reduce Work and Information Overload. Communications of the ACM, 37(7):31–40, July 1994.
[Mitchell et al., 1994] T. Mitchell, R. Caruana, D. Freitag, J. McDermott, and D. Zabowski. Experience with a Learning Personal Assistant. Communications of the ACM, 37(7):81–91, July 1994. [Morik et al., 1993] K. Morik, S. Wrobel, J.-U. Kietz, and W. Emde. Knowledge Acquisition and Machine Learning – Theory, Methods, and Applications. Academic Press, London, 1993. [Orwant, 1995] J. Orwant. Heterogeneous Learning in the Doppelg¨anger User Modeling System. User Modeling and User-Adapted Interaction, 4(2):107–130, 1995. [Paiva and Self, 1995] A. Paiva and J. Self. TAGUS – A User and Learner Modeling Workbench. User Modeling and User-Adapted Interaction, 4(3):197–226, 1995. [Pohl et al., 1995] W. Pohl, A. Kobsa, and O. Kutter. User Model Acquisition Heuristics Based on Dialogue Acts. In International Workshop on the Design of Cooperative Systems, pages 471–486, Antibes-Juan-les-Pins, France, 1995. [Pohl, 1992] W. Pohl. Beispielerkennung f¨ur die induktive Generierung von Benutzermakros. GMD-Studien 205, GMD, St. Augustin, 1992. [Sommer, 1995] E. Sommer. Induction, Evaluation, Restructuring: Data Analysis as a Machine Learning Loop. In G. Lasker, editor, Proc. of the Conference on Intelligent Data Analysis (IDA-95), 1995. [Thomas and Fischer, 1996] C. G. Thomas and G. Fischer. Using Agents to Improve the Usability and the Usefulness of the World-Wide Web. In Fifth International Conference on User Modeling, pages 5–12, 1996. [Thomas et al., 1987] C. G. Thomas, G. M. Kellermann, and H.-W. Hein. X-AiD: An adaptive and knowledge-based human-computer interface. In H.-J. Bullinger and B. Shackel, editors, Proc. of Human-Computer Interaction INTERACT’87, pages 1075–1080, Amsterdam, 1987. Elsevier Science Publishers. [Wrobel, 1994] S. Wrobel. Concept Formation and Knowledge Revision. Kluwer Academic Publishers, 1994.
