MODIFIER : Improving an End-User Modifiable System Through User Studies Andreas Girgensohn NYNEX Science and Technology 500 Westchester Avenue White Plains, NY 10604, USA
[email protected] Abstract. End-user modifiability allows different users to tailor a system to pursue different tasks, to have different preferences, and to adapt it to changing needs over time. User studies were performed with a system called MODIFIER, an end-user modification component for design environments. These studies helped refine previously hypothesized principles of end-user modifiability and identify new principles. The studies provided new insights about the requirements for system explanations. It also became apparent that system-maintained task agendas and critics can guide users through modification tasks. Unsolved problems were discovered and will be used as a guideline for future research.
1
Introduction
End-user modifiable systems support user modifications of systems according to the users’ needs. End-user modifiability is necessary because different users pursue different tasks, have different preferences, and have evolving needs or requirements due to changes in the world. The intended users of end-user modifiable systems are knowledgeable in the application domain but unwilling or unable to modify a system at the programming language level. MODIFIER is the end-user modification component of JANUS, a knowledge-based design environment for kitchen design [4]. Knowledge-based design environments consist of a construction kit that provides a direct-manipulation interface and a domain-oriented palette of design units and critics that critique suboptimal construction situations. JANUS supports the construction of kitchens from domainoriented building blocks such as appliances. Critics [3] in JANUS apply their design knowledge to critique the designer’s partial solutions. User studies were helpful in improving several prototypes of MODIFIER. General principles for achieving end-user modifiability have also been refined and extended in these studies [2, 6, 7]. The studies provided new insights about the requirements for explanations that are given during modification tasks. It is also apparent that systemmaintained task agendas and critics can guide users through modification tasks. Unsolved problems such as the need for a clear indication for the supported scope of modifications have been found. The principles and methods developed during the research on MODIFIER have been used in other application domains, most recently in an application for Business Marketing Operations at New York Telephone.
2
End-User Modifiability
Users must be in control of the interaction designed to achieve their goals. Otherwise, they have to put up with an unsatisfactory state of affairs, or they may not use a system at all if it does not fit their needs. Users of computer systems want to be independent of computer specialists. Therefore, tools should be usable directly by those who need them rather than indirectly by intermediaries. But more control is not always better. Users should be able to delegate their tasks, but it must be possible for them to intervene if such delegation does not work. End-user modifiable environments will free designers of tools from the impossible task of anticipating all possible uses of a tool and all people's needs. However, making systems modifiable by their users does not mean that the responsibility for good design of the system is transferred to the users. In fact, normal users may not be capable of building tools of the quality a professional designer would. Grudin and Barnard [8] report that command names designed by naive text-editing users were much less effective than those supplied by skilled designers. The goal of end-user modifiable systems is that only if the system does not satisfy the needs and preferences of the users, the users should be able to adapt it. Enabling the user to customize menus, screen layout, or similar properties is important but does not go far enough. The user should be able to change the behavior of the program and to extend the model of the problem domain with which the program operates. Users should be supported more than simply in the mechanics of the changes. For example, users might not have a good understanding of the consequences of the changes or whether the changes are the best solution for their problems. One might think that automatic programming will make end-user modifiable systems unnecessary. Rich and Waters [12] give a “cocktail party” description of automatic programming: There will be no more programming. The end user, who only needs to know about the application domain, will write a brief requirement for what is wanted. The automatic programming system, which only needs to know about programming, will produce an efficient program satisfying the requirement. Automatic programming systems will have three key features: They will be end-user oriented, communicating directly with the end user; they will be general purpose, working as well in one domain as in another; and they will be fully automatic, requiring no human assistance. They point out that this description is based on a number of myths. It is not possible for end users to communicate with an automatic programming system that knows nothing about the application domain, a point that is discussed in detail by Barstow [1]. Requirements cannot be complete because there is no practical limit for details in a specification. Creating a program is not a two-step process in which first an end user creates a requirement and than the automatic programming system makes a program. Since requirements are not complete, a dialog must be established for clarifying them, and in addition, the requirements of users change with the feedback of the growing program. End-user modifiability is an answer to some of the problems with automatic programming systems. End-user modifiable systems have
Figure 1: JANUS domain knowledge, and instead of requiring detailed and complete specifications they provide immediate feedback and support users in finding a solution. Knowledge-based systems, such as expert systems, are becoming widely used for both large and small tasks. The difficulty of the knowledge acquisition process for such systems has been documented by McDermott [10] and by Mittal and Dym [11]. This process is costly in both the time and money required to develop the knowledge base. This work focuses on minimizing the need for knowledge engineers in the knowledge acquisition process by providing tools and representations that allow end users to make modifications to the knowledge base of the system they are using. Besides making existing knowledge-based applications more affordable to create and maintain, we believe these techniques allow for a new range of applications of knowledge-based systems, which were previously insupportable due to limitations in existing knowledge acquisition techniques.
3
An Example of End-User Modifiability: JANUS and MODIFIER
JANUS is a design environment that allows designers to construct residential kitchens. The system enriches traditional design practice by augmenting the designer’s creative and analytical skills. JA N U S is a knowledge-based system supporting the construction of kitchens from domain-oriented building blocks called design units. JANUS contains knowledge about how to distinguish “good” designs from “bad” designs and can explain that knowledge. The system knows how to combine building blocks into functional kitchens. Its knowledge includes design principles [9] based on three areas: building codes, such as “the window area shall be at least 10% of the
Figure 2: Help for the “Descriptions” Field of a New Class floor area” (Such a critic could not be formulated in JANUS because only relationships between individual design units can be critiqued.); safety standards, such as “the stove should not be installed under a window or within 12 inches of a window”; and functional preferences, such as “the work triangle should be less than 23 feet.” Critics in JANUS apply their design knowledge to critique the designer’s partial solutions. The critics display messages, such as “the refrigerator is not near a stove” in a critic window (see Figure 1). JANUS employs several knowledge representation strategies; the goal is to support end users in modifying all these representations. The system is used as a test case for the end-user modification component MODIFIER . MODIFIER implements methods for achieving end-user modifiability in knowledge-based, domain-oriented design environments. These methods are not restricted to the domain of kitchen design; they can be and have been transferred to other domains. 3.1
A Scenario
This scenario gives an overview of the features of MODIFIER. It shows how a user would add a new appliance to the palette. A designer wants to design a kitchen with a freezer in it. She discovers that the palette does not include a freezer. She decides to add one to the palette. A window with several subwindows represents a form for describing the new class (see Figure 2). MODIFIER suggests entering a name for the new class and explains that the name is used for identifying classes. The corresponding field in the “Class Attributes” window is highlighted. After the user enters the name “Freezer.” MODIFIER suggests entering descriptions for the new class. The user clicks on the “Descriptions” field and presses the HELP key. The help window explains how descriptions are used and provides a list of descriptions already used in MODIFIER . The user picks the description “(Stores Self Food)” from the list. Selecting a description from the list of used ones requires only that the user can decide whether a description seems fit to the new class. If the new class is described with existing descriptions instead of new ones, it is possible for MODIFIER to list classes in the “Matching Classes” window that might have related useful descriptions and later to classify the new class in relation to existing ones. MODIFIER
Figure 3: Suggestions for Embedding a New Class into the Inheritance Hierarchy suggests copying the remaining descriptions of the matching classes because they might be useful for describing the new class as well. The user follows the suggestion by clicking on the “Do It” button and gets all local and inherited descriptions of the classes “Refrigerator” and “Cabinet,” which both have the description “(Stores Self Food).” Now the user removes descriptions that are inappropriate for a freezer such as “(Stores Self Dishes)” and adds the new description “(Freezes Self Food).” The next step is to decide which critic rules should apply to the new class. MODIFIER supports that decision by listing rules whose requirements are fulfilled by the description of the new class in the “Matching Rules” window as likely candidates for rules for the new class. Instead of following the suggestion to copy some of the matching rules, the user executes the command “Copy All Rules” from the menu of “Refrigerator” in the “Matching Classes” window that copies all rules applicable for refrigerators. The rationale for copying all these rules is that refrigerators are similar to freezers so that most of the refrigerator rules should be applicable for freezers as well. MODIFIER suggests three different alternatives for embedding the new class in the hierarchy of existing classes (see Figure 3). For each alternative a window shows in a graph how the class hierarchy would look if that alternative were chosen. These graphs use a fisheye view in which classes are printed smaller if they are farther away from new classes and classes that are too far away are not shown at all. By suggesting how the new class could be embedded into the class hierarchy, MODIFIER frees the user from the burden of locating related classes, changing super class fields appropriately, and moving descriptions and rules around. In order to understand these alternatives better, the user gives the generated classes meaningful names. The user selects the second alternative and the class “Freezer” and the two generated classes “Food-Storage” and “Cooling-Device” are installed.
4
User Studies
Two user studies were performed. The goal of these studies was to evaluate MODIFIER and to gain new insights on how people perform modifications. In the first study, three subjects (designated S1 - S3) with computer science backgrounds were observed. Each subject was asked to complete different tasks in MODIFIER such as “define a critic rule that critiques design units that are positioned away from walls,” or “add a microwave oven to the palette.” The second study was done eight months after the first. Nine subjects (S4 - S12) were tested. Seven of these subjects had computer science backgrounds and one had also been in the first study. In addition to the given tasks, they were asked to identify a part of JANUS that should be modified and to do the necessary modifications. The free tasks were supposed to indicate how well the system could support unforeseen modification tasks in contrast to the other tasks for which the intended outcome was completely known. The first study had shown that the subjects were not familiar enough with MODIFIER in order to do the tasks without any training. To overcome that obstacle, the subjects in the second study had either a 15-minutes period to read the manual and to experiment with the system or they were given a 15-minute demonstration of the system. The intended users of MODIFIER are regular users of JANUS . Because JANUS has not been used by anybody on a regular basis, it was not possible to find subjects who were familiar with JANUS . Many of the problems were related to the interaction style of a SYMBOLICS LISP machine. Many of the subjects had not used a SYMBOLICS LISP machine at all or at least not on a regular basis. Problems caused by the lack of familiarity with either JANUS or a SYMBOLICS L ISP machine are not discussed below. The focus is on MODIFIER. Many insights for the directions of future research were gained from the free tasks in the second user study. The impossible tasks made the limits of the current implementation of MODIFIER apparent. The remaining free tasks showed how users would approach modification tasks that had not been foreseen in the design of the system and what kind of support tools would be necessary to support these tasks. 4.1
Impossible Tasks
Three subjects tried to do modification tasks that were impossible to perform in the implementation of MODIFIER . That shows that MODIFIER does not give its users a sufficient model of its scope and limitations. This is an important issue for end-user modifiable systems that needs to be looked into in the future. S4 wanted to define tall freezers that could not provide set-off space for sinks. Set-off space is used for putting down items next to the sink. He proposed to add a height attribute to the class “Freezer” that could be checked in the requirement of the critic rule that would watch for set-off space around sinks. This modification was impossible because attributes could not be checked in the requirements of critic rules. A plausible solution would have been a LISP expression such as the following: (> (height freezer) (inches 30)) That solution or a similar one could not be used because at the time of the study the requirements could contain only references to fact descriptions and no L I S P
expressions. After being told that the proposed solution was not possible, the subject used a description that distinguished tall freezers from others. S8 wanted to add a grid or a ruler to the work area. Extensions of the system in that direction are not supported at all so that such a task had to be done completely in LISP. S10 wanted to define a critic that checked whether the kitchen has a door. That would have required changing the representation of critic rules and the mechanism that applied rules because all critic rules check the relationships between design units in the work area in the current implementation. Such a mechanism cannot check for the existence of a door because no critic rule for a door would fire if no door existed in the work area. The main problem with these three failures was not that the subjects could not do the intended tasks but that they could not tell that the tasks would not be possible. In general, it was almost impossible for the users to judge the limits of the end-user modification component. Future research will have to find methods to represent these limits and to communicate them to the users. This problem is discussed in more detail in Section 6.2. 4.2
Free Tasks
The subjects were asked to define their own modification tasks for parts of JANUS that needed improvements. Some of these tasks were impossible to do as already discussed above. The remaining tasks uncovered problems that did not appear in the given tasks. Free tasks are the real test for an end-user modification component because the purpose of such an component is to allow users to make additions to a system that are not part of the original system. S4 wanted to define tall freezers that could not provide set-off space for sinks because they are too tall to have a counter top. He realized that he should start with modifying the class “Freezer,” but he did not know whether he should modify the attributes or the descriptions of that class. The conceptually better solution would have been to add an attribute “Height” to the class “Freezer” but it was impossible to access attributes in the requirement of a critic rule. After learning that, the subject decided to use a description that would represent whether a freezer could provide set-off space. First he looked through the existing descriptions and decided to use “(Affords Self Set-Off-Space)” because that was similar to the existing descriptions “(Needs Self Set-Off-Space).” He then proceeded with defining a new critic rule “Not-NextTo(Freezer,Sink)” that had “(Not (Affords Freezer Set-Off-Space))” as its requirement. S6 wanted to add a microwave oven to the appliance palette. He only had a few problems during the definition of the class but did not recall immediately all the steps he had done in task 1. This might indicate that the system should provide more guidance for the modification task. The subject defined a critic rule for microwave ovens. Like a few of the other subjects, he tried to start the definition of the critic rule by manipulating a design unit in the work area. After getting past that mistake, he used an example during the definition of the critic rule.
S10 wanted to add to the appliance palette a detached cabinet that must not be surrounded by appliances or other cabinets. He wanted to copy and edit an existing cabinet and was told to look in the menu of a palette item after he asked the study supervisor about it. He discovered the “Copy & Edit” command and started to define the new class as a copy of the class “Base-Cabinet” without problems. He had a few difficulties changing the appearance of the new class in the “Graphics Objects” field but succeeded with it. After completing the definition of the new class, the subject started to define a critic rule that would check the constraints for detached cabinets. He knew that he wanted a relation such as “Not-Next-To” as the condition of the critic rule but overlooked it at first in the help window for the rule condition. He put too many classes into the parameter field because he wanted both cabinets and appliances to be checked with respect to detached cabinets. A critic told him that the rule had too many parameters and he realized that he had to define two rules in order to accomplish his goal. After filling the parameters field, the subject wondered whether he needed a requirement and decided against it. He used the “Example” button to decide which value to choose for the “Apply To” field and would have liked to see an example of a rule that had “Not-Next-To” as its condition. The algorithm for finding an example did not consider the condition field the most important and showed a different example. The subject decided to leave the value at “One” and to test the rule. After some testing the subject realized that the value for “Apply To” should be “All” and changed it accordingly but did not remember at first how to locate the new rule. When the subject defined the second critic rule that checked appliances, he did not use “Copy & Edit” but defined the rule from scratch. After the subject filled the fields of the second rule, the system suggested merging the two rules and replacing the two classes “Cabinet” and “Appliance” by “Design-Unit.” The subject did not like the suggestion because it was too general and would have included walls and windows as well. 4.3
Discussion
These studies point out some problems with the MODIFIER approach. First, it is necessary to provide rationale for the existing critic rules in order to aid the users in deciding which rules to copy. The requirements of critic rules were another source of confusion. A few subjects did not understand the concept of an applicability condition and many subjects had some problems with the LISP-like format of the requirement field and its relationship to the descriptions of objects. The improved help windows used in the second study and the use of modification critics reduced these problems but could not eliminate them. In addition to pointing out how the system had to be improved, the studies also helped to refine and extend the principles for making systems end-user modifiable. The first study already showed that providing a list of possible values is not enough of an explanation for a field in a property sheet. The second study confirmed that explanations for the purpose of the object and field being modified increased the understanding of the subjects. But the second study also indicated that verbose explanations are as bad as no explanations at all because most subjects did not read them. Examples are often the preferred form of explanations. System-maintained task agendas seemed to be very helpful for guiding users through modification tasks. However, system developers have to be very careful in designing the actions suggested by the system because some of the subjects followed them almost blindly.
Modification critics proved to be very valuable in immediately pointing out misconceptions to the subjects. The combination of system-maintained task agendas and critics seems to be well suited for guiding users through a variety of modification tasks. The need for domain orientation became apparent during the studies. S12, who had an architectural design background, especially had problems with concepts used in the system such as classes, rules, and relations. Representations have to be found, at least for the top layer of a layered architecture, that hide the system concepts better. Visualization seems to be a good way for dealing with inheritance hierarchies. None of the subjects in the second study had problems in understanding the suggestions of the system for embedding the new class into the inheritance hierarchy. The “fisheye” view [5] used for reducing the size of the shown hierarchies did not seem to trouble the subjects at all, so that this method should scale up nicely to larger hierarchies. In some cases the subjects did not understand why the system had come up with a certain set of suggestions. That indicates the need for an explanation for the suggestions of the system.
5
Future Research
Although the current version of MODIFIER was successful in enabling the subjects to complete their tasks during the user studies, there are areas for future research that promise improvements. In order to assess the potential of new and existing principles for achieving end-user modifiability, more formal user studies have to be performed that can tell whether the improvements are statistically significant. 5.1.
Domain-Oriented Representations
The user studies showed that although subjects without a computer science background were able to complete the tasks, they felt uncomfortable with representations that were too far from the problem domain. These subjects commented that concepts such as classes, rules, and relations were too far from the kitchen domain. In order to overcome these problems, representations have to be found that are more familiar to the users. The inheritance hierarchy of classes could be simplified by using the results of Rosch’s [13] research on categories. She observed that there are basic level categories which are the least-specific categories for which prototypes can be envisioned. An example of such a category is a chair. Super ordinate categories such as furniture do not have prototypes and subordinate categories such as a recliner do not add much knowledge compared to the basic level categories. A system that emphasizes basic level categories should make it more natural for users to deal with an inheritance hierarchy. In the kitchen domain, examples for basic level categories are stoves, sinks, and refrigerators. Subordinate categories can be represented by a catalog of concrete appliances from different manufacturers and descriptions such as “three heating elements” or “two doors.” Super ordinate categories such as appliances can be represented by grouping basic level categories. The representation of critic rules in JANUS proved to be difficult to understand for many of the subjects in the user studies. The applicability condition that relates a critic rule to descriptions of design units such as “(Cooks Self Food)” was a major
source of problems. The L ISP-like format of descriptions and the use of keywords such as “Self” was a part of these problems. A representation has to be found that is more familiar to users and, at the same time, constrained enough for the system to reason with it. Another source of confusion was the mechanism for specifying how many combinations of design units had to be tested. For example, a stove has to be near to only one refrigerator but it should be away from all windows. In many cases this distinction is of no importance because most kitchens contain only one stove and one refrigerator. A related problem is that critic rules that check for the absence of design units cannot be formulated in the current representation. For example, it is impossible to check whether a kitchen has a door or whether the window area is at least 10 percent of the floor space. Spatial relations in JANUS use LISP predicates as their conditions. Users need to use only a very small subset of LISP consisting of arithmetic and comparison functions in addition to the provided library of spatial functions. But the use of LISP is unfamiliar to most domain experts. An infix notation might be a slight improvement but the problem remains that many end users might not be able to formulate appropriate predicates. Fortunately, most users will not have to define new spatial relations because most new critic rules can be defined just by using existing relations. Minor adaptations of relations can be done by copying existing ones and by changing conditions by textually replacing parts of it, such as a number that represents a distance. However, this is not satisfying in a spatial domain. Most spatial relations could be specified with a direct-manipulation interface. 5.2
Indicating Limits
As part of the user studies, subjects were asked to choose a modification task of their own. This part of the study showed that it was difficult for the subjects to assess the limits of the end-user modification component. Some of the subjects chose tasks such as adding a grid or a ruler to the work area that were far outside of the scope of the system. If users do not have a good understanding of the limits of the system, they might waste much time in looking for a solution that does not exist or they might give up prematurely although a solution could have been found. In order to be able to notify a user if an intended task is outside the scope of the system, the system needs to have a representation of the task the user wants to perform. The user could specify such a task or the system could try to infer it and ask the user for confirmation. If the scope of the task representation component is significantly broader than the scope of the modification component, the system will be able to tell the user which tasks are impossible. Users can be prevented from giving up prematurely if the system guides them through a task with a task agenda and critics. The current implementation of these support tools proved to be very helpful in the user studies but they need to be extended so that they cover the full range of possible modification tasks. 5.3
Adaptive Component
End-user modifiable systems belong to the category of adaptable systems, but they can benefit from adaptive mechanisms [14]. In order to achieve adaptivity, a user model has to be maintained. Such a model collects information about a user such as knowledge about certain topics, command usage, and preferences. A user model can be used for providing minimalist explanations; that is, the system adapts the
information provided in an explanation to the user. In order to adapt the interface of a system to a user two models are needed in addition to a user model. An interaction model represents the available interaction objects such as windows and menus and a task model represents the tasks a system can perform and on a lower level commands and their results. These models are already needed for a demonstration component and a tool for indicating the limits of the system discussed in the previous sections. If all these models are being used by the system, it can adapt the interface of the system to different users by hiding commands and providing different views of the system. Different views can support the transition from the system model to different situation models of different users.
6
Conclusions
Two user studies were performed with twelve subjects to identify problems with MODIFIER and to learn more about the conceptual framework of end-user modifiability. Several conceptual problems were observed that led to new principles for the design of end-user modifiable systems. Explanations were discovered to be insufficient and the importance of concise explanations that describe the purpose and use of a parameter was recognized. Examples were discovered to be a preferred form of explanation. Modification critics proved to be effective in pointing out misconceptions of the subjects immediately. A system-maintained task agenda was successful in guiding the subjects through modification tasks. Several problems still existed by the time of the second study. Most of these problems were related to system concepts such as classes or rules that were confusing to some of the subjects. This shows the importance of domain orientation. The representations used in MODIFIER were not close enough to the application domain. Several promising approaches for simplifying the inheritance hierarchy, the critic rule mechanism, and spatial relations need to be tested and refined. In the second study, the subjects were asked to perform one task of their own choice. Several of the subjects chose tasks that were impossible to perform with M ODIFIER. The subjects had no indication that these tasks were outside the scope of the system, a problem that needs to be addressed in the future. An explicit representation of the kinds of tasks an end-user modification component can perform can be used for indicating the limits of the system to the users. An adaptive component of an enduser modifiable system can adapt explanations to different users, provide different views, and hide certain commands. A serious shortcoming of MODIFIER has been that it was tested only in laboratory settings. That will be overcome by integrating MODIFIER in an application for Business Marketing Operations at New York Telephone and deploying the system in a real-use situation.
Acknowledgments I would like to acknowledge the support of Gerhard Fischer. As advisor and dissertation committee chairman he has been a source of advice and constructive criticism that has served to guide my research. I thank Mike Atwood for his comments that improved this paper greatly and for giving me the time to work on the paper. The research was conducted at the University of Colorado and supported by grants No. IRI-8722792 and No. IRI-9015441 from the National Science Foundation.
References 1.
Barstow, D.R., A Perspective on Automatic Programming. AI Magazine, 1984. 5(1): p. 5-27.
2.
Fischer, G. and A. Girgensohn. End-User Modifiability in Design Environments. in Human Factors in Computing Systems, CHI’90 Conference Proceedings (Seattle, WA). 1990. New York: ACM.
3.
Fischer, G., et al., Critics: An Emerging Approach to Knowledge-Based Human Computer Interaction. International Journal of Man-Machine Studies, 1991. 35(5): p. 695-721.
4.
Fischer, G., R. McCall, and A. Morch. JANUS: Integrating Hypertext with a Knowledge-Based Design Environment. in Proceedings of Hypertext’89 (Pittsburgh, PA). 1989. New York: ACM.
5.
Furnas, G.W. Generalized Fisheye Views. in Human Factors in Computing Systems, CHI’86 Conference Proceedings (Boston, MA). 1986. New York: ACM.
6.
Girgensohn, A., End-User Modifiability in Knowledge-Based Design Environments. 1992, Department of Computer Science, University of Colorado:
7.
Girgensohn, A. and F. Shipman. Supporting Knowledge Acquisition by End Users: Tools and Representations. in Proceedings of the Symposium on Applied Computing (SAC’92). 1992. New York: ACM.
8.
Grudin, J. and P. Barnard. When Does an Abbreviation Become a Word? And Related Questions. in Human Factors in Computing Systems, CHI’85 Conference Proceedings. 1985. New York: ACM.
9.
Jones, R.J. and W.H. Kapple, Kitchen Planning Principles - Equipment Appliances. 1984, Urbana-Champaign, IL: Small Homes Council - Building Research Council, University of Illinois.
10. McDermott, J., R1: The Formative Years. AI Magazine, 1981. 2(2): p. 21-29. 11. Mittal, S. and C.L. Dym, Knowledge Acquisition from Multiple Experts. AI Magazine, 1985. 6(2): p. 32-36. 12. Rich, C.H. and R.C. Waters, Automatic Programming: Myths and Prospects. Computer, 1988. 21(8): p. 40-51. 13. Rosch, E., et al., Basic Objects in Natural Categories. Natural Categories, 1976. 8: p. 382-439. 14. Totterdell, P.A., M.A. Norman, and D.P. Browne. Levels of Adaptivity in Interface Design. in Proceedings of INTERACT’87, 2nd IFIP Conference on Human-Computer Interaction (Stuttgart, FRG). 1987. Amsterdam: NorthHolland.
