User Modeling in Dialog Systems Pontus Johansson Department of Computer and Information Science, Link¨ oping university
[email protected]
Abstract. In this paper the definitions of and approaches to user modeling in natural language dialog systems have been reviewed. The contents of user models are discussed; how user modeling in different research areas relates to user modeling are reviewed; examples of some techniques for building user models through natural language interaction and observed behavior in other media channels are given. Furthermore, the uses of user models in natural language dialog systems are discussed, as well as the implications of employing user models. It is concluded that natural language aspects of user modeling is needed to complement traditional user modeling in high-functionality applications.
1
Introduction
One of the areas in the field of natural language processing is to design dialog systems, where users interact with computers using spoken or written natural language. The traditional goal of this research community is to create systems that allow users to interact with them in similar ways as they interact with other humans. Accordingly, early research was inspired by the Gricean maxims [1]. There was also a heavy reliance on plan recognition, and complex inference mechanisms for modeling user knowledge and beliefs in a particular domain (e.g. [2, 3]). This has lead to the emergence of the research field of user modeling. The earliest work is regarded to be that of Allen, Cohen and Perrault (in [4]), and Elaine Rich [5]. In this early work, the user modeling was handcrafted and often little distinction was made between user modeling components and other system components. Since then, work has been carried out to design generic and reusable user modeling components or shells (e.g. [6, 7], but substantial work is still needed before robust, reusable, and generic user modeling is a reality. User modeling in the context of natural language dialog systems has slowly progressed. It has generally been claimed that the NLP techniques is something of a progress bottleneck, and even today - almost 30 years after the first thriving attempts - user modeling success stories in the context of natural language dialog systems are still rare [8, 9]. User modeling has made more progress in non-natural language systems and interfaces. It has, for example, been successful in the context of user profiling and various recommending systems. Other fields of interest include adaptive interfaces, information retrieval, intelligent tutoring, help and guidance systems, and expert systems [8].
Recently, however, natural language interaction with computers has become more feasible both in scientific and commercial terms. This is due to scientific research in speech technology, dialog modeling, and language processing, as well as the arrival of faster computers [10]. This recent development provides opportunities to incorporate user modeling in natural language dialog systems. The purpose of this paper is to present an overview of current research in the field of user modeling, with focus on natural language dialog systems. Since there are several definitions of the concept of user models and user modeling, I will not give a clear-cut definition already in this section. Instead, I will first sketch some of the proposed benefits of having a user model (section 1), where for now, a user model simply is a ”model of the user”. Then, the various definitions and approaches of user modeling will be reviewed in more detail in section 2. This section also includes some examples of applications. Section 3 then deals with how user models are acquired and built up. The overview is then completed in section 4, which deals with how a user model is put to use. This section is more oriented towards where user modeling components reside within a system, and their relationship to other modules. Section 4 is concluded with some important implications of the use of user modeling, from social, ethical, and usability perspectives. Finally, section 5 discusses the overview and provides some pointers for future work.
2
Why Model Users?
In the beginning of the eighties, user models were expected to improve several abilities of dialog systems: To ”understand” a user, help achieve adaptivity, and increase robustness, to mention a few. Looking from a wider perspective, one of the central problems for system developers - and as a result: users - can be traced to the field of Human-Computer Interaction (HCI). It is pretty hard to write software for potentially millions of users that will fit the needs for each individual (who are only known at use-time). To customize software to fit an individual user so that s/he can perform his or her tasks as efficiently and enjoyably as possible is one issue user modeling is trying to address. In short, Fischer [11] (p. 1) informally captures the benefits of a user model: The challenge in an information-rich world is not only to make information available to people at any time, at any place, and in any form, but specifically to say the right thing at the right time in the right way. In the context of general user modeling, this problem has recently grown; mainly due to the increase of complexity in today’s so-called High-Functional Applications (HFA), and the increase of information and information channels available to users. A HFA is a complex software system (such as Unix, MS-Office, etc.) that serves the needs of large and diverse user populations. Since software applications have become essential and everyday tools for professionals and students they have grown increasingly powerful, but also more complex. These applications are so feature-rich that they represent a significant design effort for their
creators as well as a significant learning effort for their users. According to Fischer [11] (p. 7) the design of HFAs must address three problems: 1. The unused functionality must not get in the way of used functionality. 2. Unknown existing functionality must be accessible or delivered at times when it is needed. 3. Commonly used functionality should not be difficult to be learned, used, and remembered. It is also noted by Fischer that the old notion of an ”expert” of a software system is obsolete in this context. Depending on the task, different users use the same application very differently. An example of a HFA is the MS-Word software, which can be used for a wide range of tasks, several of them more related to for example graphical layout as opposed to more traditional word processing. HFAs naturally require adaptivity in order to suit the needs of any particular user. User modeling is therefore a very important issue as HFAs emerge and increase in complexity and functionality. For natural language dialog systems, the potential benefits of user models seem to be many. Kass and Finin [2] list the following: 1. Supporting the task of recognizing and interpreting the information seeking behavior of a user (i.e. plans and goals). 2. Providing the user with tailored help and advice, such as handling misconceptions, and knowing when to volunteer information. 3. Eliciting information, getting input, and resolving ambiguity. 4. Providing output (i.e. deciding what to say and deciding how to say it). Morik [12] states that a cooperative natural language system requires a user model. Schuster [13] agrees, and adds that a user model that can be modified during the interaction is a mechanism that can also contribute to a robust, and coherent dialog. Wahlster and Kobsa [14] point out that user modeling can also be required in non-cooperative dialog for identifying and analyzing objects, indirect speech acts and meanings. The user model is thus not merely a method for ”smoothing” the dialog. The following capabilities of a user model are also listed by Kobsa [15]: – Take into account the goals and plans for which the user needs some requested information, and supply additional relevant information, if necessary. – Take into account what the user probably already knows or does not know about a situation, and thus avoid redundancy and incomprehensibility in its answers and explanations. – Detect wrong beliefs and misconceptions of the user and inform the user about them. User model contribution to dialog systems can be divided into natural language generation and natural language understanding, where generation improves content planning, and understanding improves plan recognition. The use of these aspects is discussed in further detail in sections 4.1 and 4.2.
When considering spoken dialog systems, new types of user models and user model acquisition techniques have emerged, and thus new benefits; such as mixed-initiative issues in spoken dialog, modeling of verification of user input, and assessing cognitive load of a user based on his/her speech [9]. Billsus and Pazzani [16], add that the infamous issue of information overload could be helped by user modeling in the context of intelligent information agents (p. 148): [Such] agents locate and retrieve information with respect to users’ individual preferences. As intelligent information agents aim to automatically adapt to individual users, the development of appropriate user modeling techniques is of central importance. This particular issue is admittedly not specific to natural language applications, but when considering the application of a dialog system in a particular domain, this issue becomes relevant just as well. Sparck Jones [17] lists the following benefits of employing a user model: – Effectiveness. The prime object of the user model is that the system reaches the correct decision. A correct user model is thought to help the system achieve this. – Efficiency. A user model can also serve to reach the correct decision in an economical way. – Acceptability. The system may use a user model to support its decisionmaking in a comprehensible and agreeable way. Even though formal utility evaluations of user modeling in natural language systems are rare thus far, one can conclude that there seems to exist several potential benefits of applying user models.
3
What is a User Model?
Having decided that a user model might be beneficial we already have a commonsense picture of what it is. But further clarification is needed. I will try to achieve this by reviewing some definitions given in the literature; point out some different approaches and views of user models; and give a brief overview of user modeling research in other disciplines. Finally, I will give some examples of applications of user modeling, which also shed some light of some of the techniques that are used. 3.1
Definitions
This section reviews various definitions and approaches to user models1 . To lay a basic foundation for this discussion we start off with a very intuitive ”definition” 1
Since this paper deals with user models within dialog systems, the definitions might appear too narrow for user modeling in non-natural language systems.
of a user model, i.e. that a user model is knowledge about the user of a system, encoded for the purpose to improve the interaction. The following definition for user models in natural-language dialog systems is taken from [14] (p. 3): A user model is a knowledge source in a natural-language dialog system which contains explicit assumptions on all aspects of the user that may be relevant for the dialog behavior of the system. Kass and Finin [2] argue that this definition is both too strong, and too weak at the same time. They view user models as a subclass of agent models. An agent model is a model of any entity, regardless of its relation to the system doing the modeling. A user model is the model of the agent currently interacting with the system. Furthermore, Kass and Finin note that implicit user models are often not interesting, since they merely represent assumptions about the agent made by designers of the system (at design-time). Their discussion is thus focused on explicit agent models, which often are related to knowledge base design (and utilized at use-time). There are four features that characterize agent models [2] (p. 6): 1. Separate Knowledge Base (i.e. information about an agent is not distributed throughout the system). 2. Explicit Representation (i.e. the knowledge about the agent is encoded in an expressive language, with support for inferential services). 3. Support for Abstraction (i.e. the modeling system can distinguish between abstract and concrete entities, such as classes of users and individual users). 4. Multiple Uses (i.e. the agent model can be used for various purposes such as support dialog, or to classify a new user etc.). Taking the above discussion into account, Kass and Finin claim that agent models (and hence user models) is a part of the field of knowledge representation. Since the user model concept is approached from different directions, it is multi-facetted and can be categorized along the lines of several dimensions. One of the earliest works in the field (in [5]) identifies three dimensions, and other authors add to this list. Kass and Finin [2] summarize these contributions as: – Specialization: The user model may be generic or individual. Typically, the stereotype model (Rich, 1989) can act as a ”bridge” between a generic and an individual model. – Modifiability: If the user model is changed during the course of an interaction, it is dynamic. Otherwise, it is static. User models that tracks goals and plans of the users are dynamic. – Temporal extent: The dimension of temporal extent is defined on a shortterm - long-term scale. At the extreme of short-term models, the user model is discarded as soon as the interaction ends. On the other hand, static models (as well as individual models) need to be long-term. – Method of use: User models may be descriptive (i.e. described in a simple data base which can be queried), or prescriptive (where the system simulates the user to check the user’s interpretation of the response). – Number of agents: Some systems are not limited to a one-to-one relationship between user and system. There might be several agents involved in the
interaction, such as in a medical diagnosis system where there is one doctor interacting with the system, and one patient. Both the doctor and the patient can be modeled in separate agent models. The system could also have a model of itself. – Number of models: For each given agent, it is possible to have several models. Separate models for an individual agent corresponds to real-life situations where humans can ”wear different hats” depending on if they act as a private person, or represent a company etc. Kass and Finin claim that there has to be a central model responsible for deciding which sub-model to employ in any given situation. It is possible to imagine more dimensions of a user model. Zukerman and Litman [9] for example, present the concept of multi-dimensional user models. While their use of ”dimension” in this case is not directly comparable to that of Kass and Finin2 , it gives rise to the concept of modality, which can be viewed as an addition to the list above. By including modalities other than language, such as mouse pointing and clicking etc., in the user model the system’s ”understanding” of what the user is trying to accomplish can be enhanced. The multi-modal approach is discussed in section 3.2. In the literature the terms student modeling, and student model often occurs. According to [18], user models and student models are both sub-classes of agent models. A student model is more specialized, since it is a model of an individual that an intelligent tutoring system (ITS) views as a student. Since student and user models are used in different ways, their respective modeling approach is different. Kass concludes that user models can benefit greatly by adopting the capabilities of student models (e.g. relationship between student and expert knowledge models, generative models, and deep modeling). Hence, ITSs and student modeling remains central to user modeling research. (See section 2.4.1 for more information about ITSs). Karen Sparck Jones [17] introduces the notion of a patient model, and compares that to an agent model3 . The patient is defined as (p. 343): [...] the subject of the system’s decision-making processes, confining the system’s decision-making processes here to those the system was primarily constructed to carry out. An agent in Sparck Jones’s sense is the person who conducts the interaction with the system, and is thus not the subject of the system’s decision-making process. This distinction is admittedly very fine-grained since the patient and agent often refer to the same entity, but serves a purpose in for example medical diagnosis systems where a nurse using the system is the agent, and the sick person that the nurse treats is the patient. 2
3
Rather, Zukerman and Litman’s use of ”dimension” seems to be more related to Kass and Finin’s ”number of models”. Sparck Jones’ view of an agent model differs from Kass and Finin’s [2] view. Since both patients and agents can have beliefs and goals they can both be viewed as agents in the general sense of Kass and Finin.
Alfred Kobsa [19] brings the notion of adaptivity into his definition of user models (p. 1): User models are collections of information and assumptions about individual users (as well as user groups), which are needed in the adaptation process. Identifying the relationship between user models and adaptivity is not trivial. This is mostly due to the fact that it is hard to find an exhaustive definition of what adaptation or adaptivity really is. As we have seen, a clear definition of what a user model is, is also lacking. Implicit user modeling is too general4 to be useful in this discussion [2], and explicit user models thus far seem to be rare in software systems. Adaptivity however, is more common. This suggests that adaptivity may exist in systems without an explicit user model with support for inference and abstraction. Since there (according to some of the definitions above) are other uses for user models than adaptivity the two concepts seem to be disjoint. Adaptivity may not require an individual, explicit user model, but can be helped by one. Since there is no great distinction between agent models and user models in most of the literature, and some of the definitions of user models really apply to agent models, we will have to accept the fuzzy boundaries of the concept as such. 3.2
Approaches
The earliest work on user modeling in dialog systems approached the issue from an emulating point of view (e.g. [20, 2]). That is, to improve the interaction by having the computer trying to mimic a human conversational participant, or at least endow the system with human-like attributes. This approach has been criticized from both the language technology field [21], and the more general HCI field [22]. The main argument is that design should instead exploit the dissimilarities between humans and computers. According to [11], this recent complementing approach is more efficient and feasible for interaction design in general, and user modeling in particular. Another shift of approach to user modeling in dialog systems concerns uncertainty. The following sub-sections show how methods for handling uncertainty can be applied to the task of recognizing the user’s plans, and the task of categorizing user groups. Lastly, the two areas of Psychology and Human-Computer Interaction and their relationship with user modeling will be discussed. Plan Recognition and Uncertainty Early work was focused on text-based cooperative systems, where the user models contained goals, plans, beliefs, and attitudes regarding a domain. Plan recognition was a cornerstone in all user modeling work. According to [23], these systems often relied either on poorly understood ad hoc techniques or on general techniques, neither of which were 4
In fact, all man-made artifacts can be said to possess an implicit user model.
suitable for handling the relevant problems. Furthermore, the utility of these prototype systems were never evaluated, and has now largely disappeared [9]. As [24] points out, empirical evaluation techniques of user models and adaptive systems have only recently started to mature. It is clear that user modeling in the NLP community has benefited from artificial intelligence (AI) research on plan recognition, and vice versa. Current work is more focused on shallow and robust systems, often tolerant to uncertainty. Plan recognition remains as an important part of user modeling, even though there is a shift from handcrafted plan libraries towards predictive statistical methods from the field of Machine Learning. The early user modeling systems’ cumbersome plan libraries were used to make inferences from observations about users and their intentions and preferences. These knowledge bases were usually built by carefully analyzing several instances of the problem at hand, which were deemed to be representative of this problem. Traditional plan recognition starts with a set of goals that an agent might be expected to pursue in the domain and an observed action by the agent. The plan inference system then infers the agent’s goal and determines how an observed action contributes to that goal. The system has a set of allowed actions that the agent typically executes. These are compared to a plan library, which is a set of recipes that defines how an agent might perform the actions. The plan library also contains information about preconditions, sub-goals, and effects [25]. While this approach is still used, it suffers from two shortcomings: construction is a resource-intensive process (the so-called knowledge bottleneck problem), and usually they are not adaptable or extendable. The early systems also ignored noise, such as interruptions and false starts. To come to terms with this, predictive statistical methods have been proposed since they have the potential of handling uncertainty and noise [26]. Machine Learning research provides stochastic methods for dealing with uncertainty, open worlds, and noise. The trend of using stochastic frameworks, derived from the AI disciplines, is evident also in other areas of user modeling. One example is - as we will see in the next section - the task of categorizing the users in stereotypes or clusters. Stereotypes and Clustering The stereotype approach is one of the first contributions to the field of user modeling, and was originally proposed 1979 by Elaine Rich (in [5]). The concept of stereotyping is simple, yet powerful. It is something people rely heavily on in everyday life, and it is used in several areas (e.g. advertisement). In user modeling, a stereotype captures default information about groups of people. (Rich’s original system used people’s descriptions of themselves to recommend books). Rich’s ideas have influenced the field greatly. Kobsa [8] summarizes stereotypes as a means of providing a quick - but not necessarily correct - assessment of the user. Kobsa identifies three tasks that the developer has to carry out when developing stereotypes: 1. User subgroup identification 2. Identification of key characteristics
3. Representation of stereotypes (i.e. defining a hierarchy) Kay [27] concludes that stereotypes is a special form of knowledge based reasoning, but should be used only initially, while the system waits to collect something better. Schwab and Kobsa [28] distinguish between clusters and stereotypes, where clusters are the outcome of the collaborative filtering approach. The collaborative filtering approach - also known as the clique-based approach [28] - is based on the assumption that people belonging to a particular group tend to behave similarly, given a set of circumstances. This approach is thus used when a user’s behavior is predicted from the behavior of like-minded people [26]. The clustering approach is sometimes referred to as the interaction history approach [19]. User interactions are clustered, and the system tries to identify a user group or task from these clusters. Then interaction patterns are extracted from the data in the clusters. New users are classified as belonging to a specific cluster, and the associated navigation pattern resulting from previous users of the same cluster is used as a guideline. The collaborative approach is derived from the area of Machine Learning, where predictive statistical methods are frequently used. There is another type of approach from this field that can be used in user modeling applications as well: the content-based approach. The content-based approach assumes that a user behaves the same way, given a set of circumstances. This is used when behavior is to be predicted from past behavior. One example that encapsulates both collaborative and content-based approaches is the Microsoft Office Assistant (see section 3.2), which utilizes a Bayesian network (BN) for dealing with user’s intentions, goals, beliefs, and needs [29]. BNs have gained in popularity within the user modeling community. This is probably due to the fact that they support the combination of the collaborative and content-based approach. The collaborative approach may be used to obtain the conditional probability tables and the initial beliefs of a BN. These beliefs can then be updated in a content-based manner when the network is accessed by a user [26]. In summary: Stereotypes are typically handcrafted, whereas clusters are dynamically built using a stochastic framework. The obvious advantage of the clustering approach is that it does not require the pre-definition of stereotypes/user populations. Psychology and User Modeling Some work in the field of user modeling touches on areas of Psychology. One example is Personality (or Trait) Psychology. The application of for example the ”Big Five” structures has previously been used in giving computer systems personality (e.g. [30]5 ). The Big Five characteristics is simply a summary description of attributes, which can be utilized in theories such as The Five Factor Model [32]. It is rarely discussed whether such theories could be of use in natural language user modeling (i.e. letting the 5
Reeves and Nass [31] go even further when they claim that people treat computers as social actors.
system ”give users personality” in terms of the Big Five characteristics6 in its user model). Intuitively it would seem very plausible, since the initial support for the Big Five in fact came from factor analysis of natural language [33]. There are other areas of Psychology that also could prove beneficial for user modeling. Mostly this involves Cognitive Psychology, which in turn is related to Human-Computer Interaction (HCI) issues. Traditionally, HCI has been known to focus on (graphical) user interfaces, in order to make systems more usable. More recent HCI research goes deeper than the interface, and is concerned with tasks, and goal-directed and contextual design (cf. [34, 35]). The current research in HCI seems to rest on the complementing approach to user modeling [11]. With the emergence of high-functionality applications (HFA), interaction design is faced with new challenges. As computer systems increase in complexity and the same software application is used for a multitude of tasks, it is reasonable to think that an individual user will never need - or want - to use all the functions available. S/he just wants to get his or her specific tasks done. To ensure this a user model might prove beneficial [11]. This leads to the relationship between usability and adaptive user interfaces, which is discussed by (among others) [36, 37].
3.3
What Does a User Model Contain?
Since user models are so heterogeneous, there are a multitude of things that can be included in a user model. Due to the often dynamic content of a user model, Kobsa (1993) notes that entries in the model are mainly assumptions and must be possible to revise. This requires a complex inference system along with the model data themselves. Kass and Finin [2] considers four major types of content: user goals, user capabilities, user attitudes, and user knowledge and beliefs. From the system’s point of view it might be hard to distinguish between what a user knows and what s/he believes, and most systems seem to treat this concept as beliefs. Exactly what goals, capabilities and knowledge/beliefs (and other intentional states) really are is seldom discussed in the user modeling literature. In fact, Kobsa [3] states (p. 53): ...the ascription of these and similar ’intentional’ states to people, animals, computers etc. is abundant in our commonsense explanation of behavior. Further dwelling in these matters is left to philosophers of mind. Horvitz et al. [29] also include needs, as well as goals, in their user model. They define goals as ”target tasks or sub-tasks at the focus of a user’s attention” (p. 257), and needs as ”information or automated actions that will reduce the time or effort required to achieve goals” (p. 257). 6
The Big Five factors include neuroticism, extraversion, openness, agreeableness, and conscientiousness [32].
Sparck Jones [17] proposes the following taxonomy for user model properties7 : – Objective/Subjective – Static/Dynamic – Decision/Non-Decision The first distinction is between objective facts and subjective (from the possessor’s point of view) facts. An objective fact could be readings from a metrical instrument such as a thermometer, whereas subjective facts include an agent’s (or patient’s) sense of anxiousness. The static/dynamic distinction is intuitive. Gender, for example, is a static property, while the user’s belief about what is currently on TV is a dynamic property. The property refers to changes in the user over time - not the effect of time on the system’s knowledge of a user property. As Sparck Jones points out, having the system realize the distinction between whether its knowledge or the user’s state has changed, can be tricky. (And sometimes the consequences might be the same thing). The most important distinction, however, is the decision and non-decision property distinction. This property cuts across the other two, so that a decision property can be static as well as dynamic, objective as well as subjective. What distinguishes a decision property from a non-decision one is applicationdependent. A movie recommender system might consider ”action comedy preference” as a decision property, whereas a medical diagnosis system most likely considers the same property a non-decision one. In most systems decision properties would be the most significant ones. It is possible to use both types of properties, but the decision properties typically require most attention, both in use and verification. According to Sparck Jones, a user model should only contain properties that set the user apart from other users. This note gives a hint as to the wide span of user model contents. 3.4
Example Applications of User Modeling
In this section, two types of systems utilizing user modeling will be briefly overviewed. Hopefully the following examples will help clarify and sum up what a user model really is, and give a sketch of some commonly used techniques. Intelligent Tutoring Systems Intelligent tutoring systems (ITS) have a rich history of helping students in certain scientific domains, like geometry, chemistry, and programming. These sorts of domains are ideal for ITSs, because they can be easily represented. However, most implemented ITSs to date rely on traditional interaction techniques, and not so much on natural language processing. Again, the bottleneck has been the lack of NLP techniques. As [38] puts it (p.127): 7
Note the difference in semantics compared to Kass and Finins’s [2] taxonomy of user models themselves; e.g. static/dynamic properties have nothing to do with static/dynamic user models (see section 3.1).
Without significant advances in the application of natural language understanding techniques, the use of an ITS in a domain like history, for example, would be very difficult, and for philosophy would be out of the question. Despite this, models for ITS applications have been proposed. The crucial problem for ITSs is to model what a user believes to be true about the application domain. Kass and Finin [2] separate between the overlay model and the perturbation model. The simpler overlay model relies on the assumption that the user’s knowledge is a subset of some ”total” domain knowledge. Domain concepts can thus be marked ”known” or ”not known”. Although this is easy to implement (once the domain knowledge representation is fixed), it is fundamentally wrong. The user’s conceptual model of the domain knowledge might not fit into the system’s (i.e. system developer’s) conceptual model of the domain. A variant of the overlay model - differential modeling - is presented by [18]. In strict overlaying, the system assumes that knowledge in the expert model but not in the student model is not known by the student. In the differential model, this knowledge is subdivided into what the student should know if s/he is to behave like an expert, and what s/he could not be expected to know. Kass notes that the differential modeling overcomes the system being (p. 392): overly presumptuous about the knowledge of the student, but still suffers from most of the difficulties of the overlay model. The perturbation model is used to model student knowledge beyond the range of the expert model, while still having a close link between the student and expert knowledge models. Even though the perturbation approach still assumes that the user and the expert are similar enough to be directly comparable. Kass [18] claims that this is basically the case when humans interact with each other: ”people expect other individuals to be basically like themselves.” (p. 393). As pointed out in section 2.2.3, this is consistent with the findings of [31]. While there certainly is strong evidence for this, this is related to how people interact with other entities: Not on how knowledge should be modeled in a system. As such, the argument can to some extent be rejected by the recent shift towards the complementing approach, and the fact that we are discussing how computers model their human users, and not vice versa. It is possible that the distinction between the interaction and the knowledge modeling could in fact be modeled by two separate user models. The interaction model would be responsible for modeling how a user is handling the interaction with the system (e.g. anthropomorphism), and thus tailor the interaction model. The model of the user’s knowledge about the domain could be viewed as separate from his/her interaction model. The interaction between these two separate models in an ITS could yield overall better performance. Recommender Systems Recommender systems are independent of the mode of interaction and hence not natural language-specific. Simply put, recommender
systems help people make decisions in complex information spaces. Recommenders suggest items to the user that she may value based on knowledge about her and the space of possible items. A movie recommending service, for example, remembers the movies a user has seen. The next time the user accesses the service; the system can recommend new movies to her based on the ones she has seen before. There are several techniques in use, but collaborative filtering is one of the most popular (see section 2.2.2): Given a domain of choices (items), users can express their opinions (ratings) of items they have tried before. The recommender then compares the ratings of other users, finds the ”most similar” users based on some criterion of similarity, and recommends items that similar users have liked in the past. User modeling is a demanding area for knowledge representation. Kay [39] notes that AI research is often based upon assumptions of closed worlds, and the user having correct, and well-formed plans and goals without critical misconceptions. These and other simplifying assumptions are not appropriate for user knowledge modeling. There are two general ”cold-start” problems with user model acquisition for recommender systems [40]: 1. The new-system cold-start problem. There are no initial ratings by any users, and hence no user profiles. In this situation most recommender systems have no basis on which to recommend, and hence perform very poorly. 2. The new-user cold-start problem. The system has been running for a while and a set of user profiles and ratings exist, but no information is available about a new user. Most recommender systems perform poorly in this situation too. The use of ontologies has been proposed to solve these cold-start problems. Ontologies and knowledge representation is a sub-field to the AI discipline. Middleton et al. report a successful integration of a recommender system (Quickstep) and an ontology to come to terms with both the new-system and the new-user ”cold-start” problems. One (non-natural language) example of a recommender system utilizing the collaborative approach is the movielens recommendation system [41], where predictions are made based on explicit ratings provided by users; implicit ratings derived from navigational data (e.g. viewed products); and implicit ratings derived from transactional data (e.g. purchased products). alfresco [42] is an interactive, natural language centered system for users interested in frescoes and paintings. The system aims not only at providing information, but also of promoting other masterpieces that may attract the user. The domain is Fourteenth Century Italian frescoes and monuments. The system, besides understanding and using (written) language, uses hypermedia both as input and output. The user can interact with the system by typing a sentence, navigating in the underlying hypertext, and using the touch screen. alfresco uses a dynamic user model that is incrementally built as the dialog proceeds. It consists of two separate modules: one representing what the user has been exposed to through images or language, and one representing what the user seems to be interested in. The former could be viewed as a model of the user’s knowledge, and is based on a user profile. The latter model, (the ”interest model”),
models the potential interest of the user and consists of an activation/inhibition network whose nodes are associated with ordered sets of individual concepts. The implementation is built on an activation/inhibition network, where each node represents an area of interest such as a specific painting style or period of time.
4
How is a User Model Acquired?
The issue of this section - how a user model is acquired - is of central importance. The ability of a user model to provide an application with user modeling benefits is directly dependent on the relevance, accuracy, and amount of knowledge the user model has, which in turn depend on the acquisition of the data that the user model is built from. Apart from plan recognition, [39] concludes that user model acquisition needs to rely on other areas of AI as well, such as Machine Learning, automated reasoning (see section 2.2.1 and 2.2.2), and Knowledge Representation/ontologies (see section 2.4.2). There is a close relationship between knowledge acquisition and the form of representation of the knowledge. The representation must be able to accommodate the dynamic nature of the user model. According to [2], the representation system needs some form of truth maintenance system of utilize evidential reasoning of some sort. The first sub-section deals with how natural language input can be used to build user models. The following sub-section then discusses how other modalities can be used. 4.1
Natural Language Input
Kass and Finin [2] discuss two types of acquisition methods in natural language systems: implicit and explicit acquisition. Explicit knowledge is acquired when the user directly states a fact that can be used in the user model. This could for example be to state an explicit goal, such as ”I want to watch a movie on TV tonight”. This is to contrast with the implicit version of this statement: ”Is there a movie on TV tonight?”. Here the system may well assume that the second speaker has the same goal as the first - i.e. the wish to watch a movie tonight. In other cases, it might not be trivial to assign goals to implicit utterances. Explicit acquisition can also occur when a system starts asking direct questions to first-time users. Usually, implicit acquisition is more difficult than explicit. Implicit acquisition can often benefit from inferring facts about the user from his or her observed behavior. This means that the user modeling component must be able to ”eavesdrop” on the system-user interaction [2]. As the name implies, dynamic user models need to be modified. This is typically done in an on-going dialog. Instead of relying on handcrafted models provided in advance, a system may incrementally use the content of user input (explicitly or by inference) to update the user model. Should the system require
more information, a sub-dialog can be initiated. Designers of such ”active” systems justify this action by the intuition that the extra knowledge acquisition sub-dialog in the long run reduces the total dialog length, thus rendering a more efficient system (in [9]). User preferences extracted from dialog can be used both for evaluating and improve user plans, and for generate replies. User preferences can be extracted either through explicit utterances (e.g. ”I like action movies” in a TV/movie domain), or implicit as a result of rejected suggestions in the past [9]. Pohl et al. [7] propose that dialog acts with schematic presuppositions can be used as a means for formulating and generalizing user model acquisition heuristics. Their generic user modeling shell system (BGP-MS) analyses dialog acts, and allows a developer of an adaptive application to define relevant dialogue act types and their associated meaning (”presupposition patterns”). During runtime, the application informs the shell about observed dialogue acts. BGP-MS then updates the user model accordingly. Pohl et al’s system can thus be able to handle user plan extraction based on both domain and dialog act type. An (implicit) utterance may correspond to different user goals depending on the domain, as the following example utterance (from Pohl et al.) shows: When is the next train to Montreal? The user goal that can be extracted here is that the user wants to travel to Montreal by train. But this is only true in a travel domain; in a transport domain the user goal is probably to send cargo to Montreal. For a simple informationproviding system, the result might be the same (i.e. a departure time), but for a more advanced cooperative system the result could include domain-specific material in addition to the departure time (i.e. by also suggesting bus departure times in the travel domain, or asking for amount and type of cargo in the transport domain). The generality in this approach relies on the fact that there exist general dialog act types. For example, any request for explanation means that a user is not familiar with some concept, whereas a request for detail information implies that the user is familiar with the basics and wants to know more. Pohl et al. mention several other examples of such dialog act heuristics. 4.2
Multimodal Input
Allowing a user model to observe what a user is actually doing with the system (clicking, using menu commands, etc.), and combining this knowledge with explicit (natural language) queries requires a multi-modal (or multi-dimensional) user model. Successful applications of this acquisition technique are the Lumi`ere and MS-Office Assistant projects, which use probabilistic reasoning when inferring the users goals from observations of the user’s behavior in combination with queries [29]. Users of the HFA MS-Office application8 have the option to 8
The facts mentioned here refer to the MS-Office97 suite, and may not apply to more recent versions, such as MS-Office2000.
use a virtual ”assistant”, which can take several graphical forms, where the most known one is the paper clip. The Office Assistant was created from the more advanced Lumi`ere project [29], where the system attempts to recognize an agent’s goal from observations of behavior. In short, the Lumi`ere system tries to infer both the likelihood that a user needs assistance, and the type of help the user needs in that case. In order to identify the need for assistance, a few heuristics (derived from Wizard-of-Oz studies) have emerged. For example, when a user attempts to return to a prior state after an action (using the undo command, or closing a dialog box without invocate an operation by it, etc.) we have an instance of the ”undesired effect” heuristic. Other heuristics include inefficient command sequences, mapping the focus of attention, search patterns, and introspection (e.g. a sudden pause after a period of activity). A set of probabilistic links to user goals is set up from these heuristics when they are observed. By using an ”event synthesis” derived from both observed actions, and queries, the Bayesian inference network along with a control system (responsible for time-stamping the events, and analyzing the event queue) determine if the system should provide assistance. Furthermore, the Lumi`ere system maintains a ”competency profile” in the operating system’s registry. To give a hint of the complexity, the MS-Office Assistant employs a broader but shallower model than the Lumi`ere system, and comprises several thousands of user goals for each MS-Office application. The assistant does not reason about competency and does not employ a persistent user profile (Horvitz et al., 1998). While the Office Assistant is especially noted for its social character - and sometimes for its bad performance in identifying what the user wants to do - the Lumi`ere project remains as a rather successful application of Bayesian networks and multi-modal user modeling.
5
How is a User Model Used?
This section deals with the employment of user modeling, and how it affects the natural language part of a system. First, the natural language understanding part (such as plan recognition) will be discussed, followed by the natural language generation part. Finally, I will point out other areas that will be affected by user modeling (such as social, ethical, and usability issues). 5.1
Natural Language Understanding
Carberry [25] concludes that plan recognition can greatly enhance the effectiveness of a system. It therefore remains a central research area in the field of user modeling in general, and user modeling in natural language systems in particular. There are however, several serious problems that need to be addressed. In other areas of natural language processing, plan recognition has been used in for example story understanding [9]. In the context of dialog systems Zukerman and Litman list the following benefits of plan recognition: – Understanding indirect speech acts
– – – – –
Responding to ill-formed queries Detecting and correcting misconceptions Handling queries based on invalid plans Recognizing discourse acts such as expressions of doubt Resolving referring expressions
Systems that consult user models in terms of surface features at utterance level of the user input can tailor the language understanding to suit the user’s deviation from the system’s language. An example of this is a system that dynamically adapts a parser’s grammar to facilitate a user’s interaction, thus learning the ”ungrammatical” style of the user. Interesting work in natural language understanding and user modeling happens at discourse level. Issues involved here include adaptation of dialog strategy on the basis of the progress of the dialog that has been conducted so far, allowing for flexible control of mixed-initiative interaction, and using dialog as a tool for incrementally acknowledge user preferences (see section 3.1). In spoken dialog systems, part of the user model can be used to cater for speech recognition failures (due to, for example, foreign or domestic accents). Several speech recognizers rely on this sort of ”training” of a user model by having a user reading pre-defined texts. 5.2
Natural Language Generation
If understanding the user’s intentions and plans is one side of the coin, the other is to generate coherent and relevant discourse. As [43] notes, the amount of detail in an explanation as well as the kind of information given depends on the user’s expertise in a domain. The user’s personal preference in style, and knowledge about terminology affects the actual words used at surface generation. Finally, making decisions about if text, graphics or sound etc. is more appropriate depending on the user and context is a task for user modeling as well. User modeling in generation can be thus be summarized as: – Content planning (i.e. deciding what to say) – Surface generation (i.e. deciding how to express it in natural language) – Modality (i.e. deciding if and how to include other channels than natural language) 5.3
The User Model vs. the Discourse Model
Both natural language understanding and generation in dialog systems refer to some sort of discourse model. There has been a heated debate concerning the relationship between user models and discourse models (and implied components, such as dialog memory). In the September 1988 issue of Computational Linguistics, a number of short papers discuss this relationship. In summary, the authors view the user model as being either disjoint [44, 45], part-of [13], or intersecting with [46, 12, 47] the discourse model. To some extent, it seems like much of the
discussion is really about definitions. For example, it seems like Schuster’s use of the term ”discourse model” is in fact the same thing as Morik’s ”dialog memory”, and Morik’s discourse model is directly derived from Grosz and Sidner’s use [48] (i.e. ”the syntactic structure of a dialog”). With a confusing terminology it is hard to maintain the discussion. But the debate is obviously not entirely attributed to terminology: these issues are complex individually, and the same goes for the relationships between them. Chin [46] for example, tries to establish the relationships between linguistic structure, attentional state, intentional structure, user goals, beliefs, knowledge, and preferences/capabilities. Since the former three are traditionally encapsulated in the discourse model, and the latter five intuitively are contents of a user model (e.g. [2, 20]) Chin claims that the two models intersect (e.g. the attentional part of the discourse model is in fact included in the user model, and the intentional structure is intimately related to user goals, etc.).
5.4
Other Implications of Using User Models
Except for the implications (or benefits) listed in the preceding sub-sections, there are other implications that a user modeler needs to take into account. These are more on the ”negative” side and include social, ethical, and usability issues.
Social and Ethical Issues All new technologies run the risk of being subject to misuse. Kobsa [8] draws attention to the fact that even if user modeling aims at making computer systems more cooperative, there is always a potential risk of misuse and abuse. Other researchers have also voiced some warnings concerning user modeling. While this is a healthy sign, the most dangerous problems are probably the ones not identified yet. Kay [27] warns about over-using stereotypes. There could be negative effects from bad stereotyping, probably resulting in worse systems. Two obvious examples would be negative stereotyping based on gender and race. Some research has implicitly pointed out potential ethical problems before. For example: Kass and Finin’s [2] notion of biased consultation, where the system is pretending objectivity. The underlying objective for pretending objectivity certainly lies at the system designer’s, and we now touch on the same ethical issues that advertisement and commercials battle with (i.e. how deceptive can we allow commercial user modeling to be?). Fischer [11] claims that (p. 14) ”it will be a major challenge to find ways to avoid misuses, either by not allowing companies to collect this information9 at all or by finding ways that the individual users have control over these user models”. In order to avoid some of these ethical and social pitfalls, [8] provides the following guidelines: 9
I.e. ”user models”. (Author’s note).
– Users should be aware of the fact that the system contains a user modeling component. (There seems to be a general agreement in the research community about this). – Users should be instructed that computer systems make errors, and merely rely on assumptions. – Users should be instructed that a system might pursue non-cooperative interests. – Users should have the possibility to inspect and modify their user model. – If technically possible, users should be able to ”switch off” the user modeling component. – Long-term characteristics should be modeled with caution, since misuse is more likely to have a larger effect, and because they are often more intimate/personal than short-term characteristics. – Results in user modeling research should be made accessible to the general public, since they will eventually be affected by it. Systems following these guidelines would tend to have an ”interactive user model”. As an example of a system that abide to (at least some of) these guidelines, the owl system [49] could be noted, since it adds a toolbar to a popular word processing application (making the user aware of the user model). Furthermore, the user can switch it off anytime by simply clicking a button in the toolbar. Another interesting problem, which could yield negative social implications, is the issue of incorrect models. A recommending system, such as a TV guide, with an incorrect user profile could happily start recommending TV shows that a user is not interested in - or still worse: would not want to be affiliated with. In some contexts (e.g. watching TV with friends), such recommendations could result in a social faux pas. Usability and Non-Deterministic Systems Another implication of adaptive systems is that they tend to become non-deterministic. That is to say, the interface and the available commands differ depending not only on who the user is; but could also differ for the same user depending on the task s/he is currently attending. This is really walking on the edge in terms of usability, since it is very close to violate established usability principles, such as recognition rather than recall, the principle of making things visible [50], etc. H¨ o¨ ok et al. [37] point out that adaptive systems run the risk leaving the user without a sense of control. It is necessary for intelligent systems that they are inspectible, controllable and predictable. This is addressed by transparent systems. Transparency occurs when the system is built as a ”glass box” (i.e. the user can see the internal workings of the system). This approach seems to fit nicely to the recommendation of letting the user be able to view and control the user model (see section 4.4.1). The obvious problem is of course that revealing the inner machinery of complex applications greatly raises the information load on users. The viewing of the system thus has to be at a suitable level of abstraction. This leads to another problem: What is a ”suitable” level? If this level could vary between users, there seems to be need for yet another user model!
As applications become more complex, and more ”intelligent”, part of the solution could lie in systems with adjustable autonomy [51]. Adjustable autonomy means to dynamically adjust the level of autonomy depending on the current situation. Such a system realizes when a decision is too important for the system to make, and could allow another entity (such as a human user) to override if they can perform a part of a task better. Of course, having a user override the system too often defies the goal of having autonomy and intelligent systems in the first place. A system employing adjustable autonomy distributes the system’s autonomy over all the system’s entities (e.g. users and software agents) to optimize the system’s overall performance. The connection between user models and adjustable autonomy remains to be defined.
6
Discussion
The field of user modeling has grown huge since it was first introduced in the late 1970’s. It has an interdisciplinary character, making it hard to focus only on natural language applications. In the context of multi-modal applications and HFAs, it may not even be desirable to limit user modeling to natural language dialog systems. Additionally, since natural language is starting to establish itself as an interaction technique for making systems easier to use, recent HFAs (e.g. Microsoft Windows XP) offer speech recognition as a standard part of the system. One implication of this is that user modeling will need to incorporate both the more traditional user modeling components in HFAs, as well as the more natural language-oriented user models from traditional dialog systems. As the complexity and uses increase, designers employ natural language interaction for usability. This requires advanced natural language user modeling techniques in combination with existing user modeling. Projects (such as Lumi`ere) have shown that performance increases when using several media channels such as observed behavior in combination with natural language queries. When relating this to the discussion of the relationship between the user model and discourse model (see section 4.3), I believe it might be fruitful to divide the concept of a user model into several user models. It is, in my view, likely that we need a finer distinction. The concept of a single user model seems a little clumsy in the light of the recent achievements. As we aim to build advanced, adaptive natural language dialog systems, and possibly connect them to HFAs, it seems fruitful to divide the notion of user model into several models. This distinction is not equal to Kass and Finin’s [2] dimension of ”number of models”. As mentioned in section 2.1, they propose the use of several models for one agent depending on which ”hat” the user is currently wearing. This is distinct from a division based on different responsibilities of the models (and related to where various user models should reside within a dialog system architecture). It is possible that user models should be distinguished on for example the level of situation and context, discourse, behavioral/interaction, domain preferences, and domain knowledge. The outline, support and details of this claim are beyond this review paper however. One important point that
needs immediate research is to refine the terminology. For example, the notion of adaptivity and user modeling and their relationship, the distinction between discourse modeling and user/agent modeling, and the relationship between user models and systems with adjustable autonomy. Interesting work is also to create a general dialog system architecture that includes user modeling at all the levels noted above. A dialog system architecture that includes a user modeling component needs to take several issues discussed in this paper into account. Addressing these issues would be interesting from a software-engineering point of view, such as finding and applying design patterns. For example, due to the need of separating the actual data from the inference or controlling system, and taking the ethical implication of letting the user view his/her user model into account, the model-view-control design pattern could be applied. Accordingly, there is a need for designing new methods (or modify established ones) for customizing such systems. A final pointer to future research is to find and exploit research from other fields that can be of use. As we have seen; AI, Psychology, Linguistics, and HCI all have contributed (and vice versa) to the field of user modeling. Further exploration within these fields could be fruitful, and it is likely that other research areas could be of use for the user modeling community as well.
7
Acknowledgements
This paper was written as a part of the Dialogue Systems course at the Graduate School of Language Technology (GSLT), G¨oteborg, Sweden. The work was financed by Santa Anna IT Research Institute. I would like to thank my supervisor Arne J¨onsson, and my reviewer Loredana Cerrato for their support and help.
References 1. Grice, H.P.: Logic and conversation. In Cole, P., Morgan, J.L., eds.: Syntax and Semantics: Vol. 3: Speech Acts. Academic Press, San Diego, CA (1975) 41–58 2. Kass, R., Finin, T.: Modeling the user in natural language systems. Computational Linguistics 14 (1988) 5–22 3. Kobsa, A.: A taxonomy of beliefs and goals for user models in dialog systems. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer-Verlag, Berlin (1989) 52–73 4. Kobsa, A.: Generic user modeling systems. User Modeling and User-Adapted Interaction 11 (2001) 49–63 5. Rich, E.: Stereotypes and user modeling. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer-Verlag, Berlin (1989) 35–51 6. Finin, T.W.: GUMS—A general user modeling shell. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer-Verlag, Berlin (1989) 411–430 7. Pohl, W., Kobsa, A., Kutter, O.: User model acquisition heuristics based on dialogue acts (1995)
8. Kobsa, A.: User modeling: Recent work, prospects, and hazards. In SchneiderHufschmidt, M., Kuhme, T., Malinowski, U., eds.: Adaptive User Interfaces: Principles and Practice. Springer Verlag, Berlin, Amsterdam, North-Holland (1993) 9. Zukerman, I., Litman, D.: Natural language processing and user modeling: Synergies and limitations. User Modeling and User-Adapted Interaction 11 (2001) 129–158 10. McTear, M.F.: Spoken dialogue technology: enabling the conversational user interface. ACM Computing Surveys 34 (2002) 90–169 11. Fischer, G.: User modeling in human-computer interaction (2000) 12. Morik, K.: Discourse models, dialog memories, and user models. Computational Linguistics 14 (1988) 95–97 13. Schuster, E.: Establishing the relationship between discourse models and user models. CL 14 (1988) 82–85 14. Wahlster, W., Kobsa, A.: Dialogue-based user models. In: Proceedings of the IEEE. (1986) 948–960 Published as Proceedings of the IEEE, volume 74, number 7. 15. Kobsa, A.: User modeling in dialog systems: Potentials and hazards. Technical Report Bericht Nr. 64, Universit¨ at des Saarlandes, Informatik, Saarbr¨ ucken (1990) Sonderforschungsbereich 314 — K¨ unstliche Intelligenz — Wissensbasierte Systeme. 16. Billsus, D., Pazzani, M.J.: User modeling for adaptive news access. User Modeling and User-Adapted Interaction 10 (2000) 147–180 17. Sparck Jones, K.: Realism about user modeling. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer-Verlag, Symbolic Computaion Series (1989) 341–363 Also available as Technical Report 111, University of Cambridge Computer Laboratory. 18. Kass, R.: Student modeling in intelligent tutoring systems – implications for user modeling. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer Verlag, Symbolic Computation Series, Berlin Heidelberg New York Tokyo (1989) 386–410 19. Kobsa, A.: Supporting user interfaces for all through user modeling. In: Proceedings of the Sixth International Conference on Human-Computer Interaction. Volume I. Human and Future Computing of I.4 User Interface for All – Everybody, Everywhere, and Anytime. (1995) 155–157 20. Wahlster, W., Kobsa, A.: User models in dialog systems. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer-Verlag, Berlin (1989) 4–34 21. J¨ onsson, A.: A model for habitable and efficient dialogue management for natural language interaction. Natural Language Engineering 3 (1997) 103–122 22. Suchman, L.A.: Plans and situated actions: the problem of human-machine communication. Cambridge University Press, Cambridge, England (1987) 23. Jameson, A.: Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction 5 (1996) 193–251 24. Chin, D.: Empirical evaluation of user models. User Modeling and User-Adapted Interaction 11 (2000) 25. Carberry, S.: Techniques for plan recognition. User Modeling and User-Adapted Interaction 11 (2001) 31–48 26. Zukerman, I., Albrecht, D.W.: Predictive statistical methods for user modeling. User Modeling and User-Adapted Interaction 11 (2001) 5–18 27. Kay, J.: Lies, damned lies and stereotypes: Pragmatic approximations of users. In: Proceedings of the 4th International Conference on User Modeling, Bedford, MA, USA, MITRE Corporation (1994) 175–184
28. Schwab, I., Kobsa, A.: Adaptivity through unobstrusive learning (2002) 29. Horvitz, E., Breese, J., Heckerman, D., Hovel, D., Rommelse, K.: The Lumi`ere project: Bayesian user modeling for inferring the goals and needs of software users. In Cooper, G.F., Moral, S., eds.: Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI-98), San Francisco, Morgan Kaufmann (1998) 256– 265 30. Nass, C., Moon, Y., Fogg, B., Reeves, B., Dryer, C.: Can computer personalities be human personalities? In: Proceedings of ACM CHI’95 Conference on Human Factors in Computing Systems. Volume 2 of Short Papers: Agents and Anthropomorphism. (1995) 228–229 31. Reeves, B., Nass, C.: The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press, Cambridge, England (1996) 32. Engler, B.: Personality Theories: An Introduction. Houghton Mifflin Company, Boston, New York (1999) 33. Catell, R.B.: Handbook of Multivariate Experimental Psychology. Rand McNally, Chicago (1966) 34. Cooper, A.: The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity. Sams (1999) 35. Beyer, H., Holtzblatt, K.: Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann Publishers, San Francisco (1998) QA76.9.S88B493. 36. Benyon, D.: Adaptive systems: A solution to usability problems. User Modeling and User-Adapted Interaction 3 (1993) 65–87 37. H¨ o¨ ok, K., Karlgren, J., Waern, A., Dahlback, N., Jansson, C., Karlgren, K., Lemaire, B.: A glass box approach to adaptive hypermedia (1996) 38. Wiemer-Hastings, P., Wiemer-Hastings, K., Graesser, A.: Approximate natural language understanding for an intelligent tutor. In: Proceedings of the 12th Interanational Florida Artificial Intelligence Research Society Conference, Menlo Park, CA, AAAI Press (1999) 192–196 39. Kay, J.: Vive la difference! individualised interaction with users. In Mellish, C.S., ed.: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, San Mateo, Morgan Kaufmann (1995) 978–984 40. Middleton, S.E., Alani, H., Shadbolt, N.R., Roure, D.C.D.: Exploiting synergy between ontologies and recommender systems. In: The Eleventh International World Wide Web Conference (WWW2002), Hawaii, USA, ACM, Sementic Web Workshop 2002, WWW2002 (2002) Semantic Web Workshop 2002 At the Eleventh International World Wide Web Conference Hawaii, USA. 41. Rashid, A.M., Albert, I., Cosley, D., Lam, S.K., McNee, S.M., Konstan, J.A., Riedl, J.: Getting to know you: learning new user preferences in recommender systems. In Gil, Y., Leake, D.B., eds.: Proceedings of the 2002 International Conference on Intelligent User Interfaces (IUI-02), New York, ACM Press (2002) 127–134 42. Stock, O., Strapparava, C., Zancanaro, M.: Human-computer interaction through natural language and hypermedia in alfresco. ACM SIGCHI Bulletin 28 (1996) 102–107 43. Paris, C.L.: The use of explicit user models in a generation system for tailoring answers to the users level of expertise. In Kobsa, A., Wahlster, W., eds.: User Models in Dialog Systems. Springer Verlag, Symbolic Computation Series, Berlin/Heidelberg/New York/Tokyo (1989) 200–232 44. Cohen, R.: On the relationship between user models and discourse models. CL 14 (1988) 88–90
45. Wahlster, W.: Distinguishing user models from discourse models. CL 14 (1988) 101–103 46. Chin, D.N.: User models and discourse models. CL 14 (1988) 86–87 47. Kobsa, A.: User models and discourse models: United they stand... CL 14 (1988) 91–94 48. Grosz, B.J., Sidner, C.L.: Attention, intentions, and the structure of discourse. Computational Linguistics 12 (1986) 175–204 49. Linton, F., Schaefer, H.P.: Recommender systems for learning: Building user and expert models through long-term observation of application-use. User Modeling and User-Adapted Interaction 10 (2000) 181–208 50. Norman, D.A.: The Design of Everyday Things. The MIT Press (1988) 51. Scerri, P.: Designing Agents for Systems with Adjustable Autonomy. PhD thesis, Link¨ oping University (2001)
