Minimizing the Learning Curve: Towards a Holistic

Minimizing the Learning Curve: Towards a Holistic Design for Context Adaptive Systems Edward R. Sykes1

Abstract. This paper presents an outline for a context adaptive system with a particular emphasis on minimizing the training time required to make effective use of complex software. The purpose of this type of adaptive system is to provide an environment in which users are highly productive even on their first encounter with the software. While a generalized adaptation engine for training is overly ambitious, several components of the proposed architecture are generalizable and can be reused across adaptive applications. This paper discusses the rationale for this type of application and discusses specific reusable aspects of the architecture. Keywords: Context Adaptive Systems, User Modeling, Context Analysis

1 Introduction The need for user adaptive systems is rapidly growing and is outstripping the ability of researchers to provide robust, usable systems. Three key factors in this growing need are the rapid increase in number of complex, interactive software systems available to users, the increasing use of mobile computing devices, and the increased responsibility that is delegated to software systems by users. • Interactive System Growth. The number of systems that people deal with continues to rise (Jameson, 2001, 2006; Kobsa, 2001; Kobsa & Fink, 2006). Generally, users do not have the time or desire to gain complete understanding of any particular software package. Instead users simply want to complete their tasks as efficiently as possible. As software becomes more powerful, multipurpose, and complex, the identification of software features useful for specific tasks requires significant knowledge about the software system; knowledge that users frequently don’t have or want. Adaptive software that supports users to achieve their goals without significant time invested in learning the software package will offer an advantage to users, and should enjoy increased popularity as a result. Unlike software packages that are designed to teach a specific skill (tutoring systems), or software that supports a small set of jobspecific tasks this research is concerned with creating an adaptive architecture that allows adaptation of complex, multipurpose, software to the tasks and requirements of the immediate interaction. 1

Dr. Judi McCuaig assisted in elements of this work. Some of the sentences, paragraphs are hers. I submitted a compressed version of this paper to the UM Conference 2007: http://www.iit.demokritos.gr/um2007/index.php We find out if it is accepted in February.

•

Mobile Computing The popularity of computers and miniature computing devices is on an exponential growth path (Nair, Chintagunta, & Dubé, 2004; Smith, 2002). The devices are used by an increasingly diverse population in equally diverse ways (Brusilovsky, 2001; Jameson, 2006). As a result, it is becoming incredibly difficult to design and construct software that both meets the requirements the potential clientele and runs within the restrictions of a mobile computing environment. Software designed with a context adaptive component can aid in the solution to this problem since the adaptation can restrict the number of software components loaded onto the mobile environment to be the minimum set required to meet the individual user’s requirements, rather than loading a bloated software application designed to meet the needs of all potential users. The inclusive notion of context proposed by this research allows the adaptation to be made with respect to hardware as easily as it is made with respect to user characteristics. • Delegation of Responsibilities. There is a tremendous amount of information available to users primarily due to the Internet and resources openly available on it. The growth experienced to date is expected to be exponential for some time to come (Nair et al., 2004; Smith, 2002). The chore of searching for appropriate information for a user can become overwhelming and is non-optimal in terms of use of resources. For example, people are already delegating responsibilities to applications by setting “watches” on news sources, RSS feeds, eBay, travel sites, etc. In these situations, the application sends a customized report to the user, however, for the most part, it is generally a one-size-fits-all model. Thus, delegation of responsibilities is a fitting direction to pursue for the design of context adaptive systems. In other words, a context adaptive system should have the capability to perform background work (such as searching for documents on a specific topic in a set of libraries on the Internet). The system needs to be able to do these tasks as precisely as possible even if the system does not have a perfect model of the user’s needs or of the task being performed. This paper presents an architecture for a context adaptive system with a particular emphasis on minimizing the training time required to use the software effectively. The effect of this adaptive system will be that users are as productive as possible (from the user’s perspective) even on their first encounter with the software. While a generalized adaptation engine for training is overly ambitious, several aspects of the architecture are generalizable and promote a reusable component model for other adaptive systems. Users of such a system will be engaged in productive, work related tasks immediately and will not be hampered by restrictive training-modes of the software as they gain expertise. To facilitate this kind of adaptive system, both the interface and the underlying processes must be adaptive and adaptable and the system must be designed with a clear understanding of the effect of adaptation on user productivity to balance the risk of interrupting the user at a task-critical moment against the risk of not providing assistance when it is needed. Studies have shown that user adaptive systems hold the key to creating software that truly assists the user rather than software that simply does what is asked of it (Cheverst, Davies, Mitchell, & Efstratiou, 2001; Strang & Claudia, 2004; Webb, Pazzani, & Billsus, 2001). However, it is also known that adaptations can reduce the usability of software if they are intrusive and impact on the attention that the user has

for the task at hand (Horvitz, 1999). This research is focused on identifying mechanisms and techniques to give users the maximum benefit of an adaptive system while carefully managing the cost of making adaptations so that usability is not impacted negatively and so that the user is not unnecessarily interrupted or redirected by the system. This paper is structured as follows: section two presents a general architecture for a context-adaptive system that minimizes the software learning curve; section three discusses the three specific aspects of the architecture: (a) reducing the user’s cognitive load during learning; (b) minimizing interruptions while scaffolding the learner (c) decision-theoretic mechanisms for making choices about learning. The final section discusses the evaluation of such systems and proposes possible future directions. 1.1 Terms and Definitions The following terms and definitions are provided for additional clarity in this paper. These definitions are based on a survey of numerous manuscripts in the fields of Adaptive Systems, User Interfaces, Context-aware systems, and in general, Human Computer Interaction (Brusilovsky, 2001; Jameson, 2006; Krzysztof & Daniel, 2004; Schmidt, 1999; Weld et al., 2003). S: an interactive computing system or device U: user of the specified system S Adaptive: the quality of a system that is able to adapt its behavior according to changes in its environment, the context of operation, or in parts of the system itself Adaptable: the quality of a system that may be altered in behavior and/or appearance by an individual for personalization purposes with the goal of increased usability for that individual. Context: The combined information that S can collect about U’s characteristics, characteristics of the domain, characteristics of the task, the physical environment, and characteristics of the computing environment. This is a more inclusive definition of context than is commonly given. Interaction: a specific type of action that occurs between U and S (or any subcomponent of S) such that there is an effect upon one another. The concept that this type of action is a two-way effect is essential. (For example, performing a query on a database is not interaction since there is not a two-way effect). Context Adaptive System (CAS): an interactive system that adapts its behaviour, the content or the presentation to more than one dimension of the surrounding context. 1.2 Reducing the Learning Curve The broad goal for this research is to create software (S) that assists U in U’s efforts to learn to use S. The goal is reachable if S can be fitted with the ability to understand the goals of U with respect to S, and to understand the knowledge and skills that U brings to the interaction. This paper is concerned with giving S the

ability to recognize and react appropriately to the goal of U. With only that ability, S can provide support to reduce the cognitive load of U and help U be immediately effective with the software. The research goal will be realized through success in (a) identifying suitable adaptation strategies; (b) determining which of several possible context models permit the most accurate characterization of goals; and (c) supporting different learning styles with appropriate adaptations. Each of these subgoals for the research is the basis for a key component in the proposed architecture. 1.3 Representing Goals and Tasks with Contextual Models Adaptive systems rely on contextual knowledge to determine what should be presented to the user and how this presentation should take form. Adaptive systems face challenges of assessment of state, in the selection of adaptation strategies to employ, and in doing more than simple adaptation such as adjusting size and message placement without disorienting or confusing users. The representation of the context of goals and tasks is especially challenging because users frequently have more than one goal when they are interacting with software systems and because humans are adept at task switching with little or no warning. A hierarchical representation of goals or tasks is practically a necessity but it is not sufficient for identifying adaptive mechanisms to support the accomplishment of a specific subgoal or task. Different users of computing software commonly develop vastly different strategies for accomplishing the same task. The strategy selected depends on several dynamics including the previous experience of the user, the preferred learning style of the user, the type of task and the domain or data involved. This research will show that an adaptive system can accommodate different approaches or strategies for tasks using models of tasks, models of completion strategies and models of learning style. The proposed architecture must enable adaptive systems capable of more than content and presentation adaptation. In order to minimize U’s learning curve the software system must be able to adapt the process as well. The challenge is to make adaptations of process without interfering with the user’s activities and independence. 1.4 Dynamic Adaptation to Different Cognitive Styles Different styles of learning are appropriate for different tasks, for different content domains, and for different types of people. Individuals usually have a preferred style of learning. Learning styles also vary with the occupation, educational choices and job role of the individual (Boyatzis & Kolb, 1995; Wolfe, Bates, Manikowske, & Amundsen, 2005). The most appropriate techniques for a particular interaction is some combination of the preferred techniques for each of the associated learning styles. An adaptive system to reduce U’s learning curve must have the ability to adapt to U’s actions and make adaptations that are appropriate for U’s preferred learning style

for the situation. A major contribution of this research is the development of guidelines and heuristics for incorporating measures and descriptions of cognitive style into the adaptation process without restricting the user to a single sequence of activities. 1.5 Identification of Suitable Adaptation Strategies The usability and effectiveness of adaptive and context aware systems is difficult to determine for several reasons. The field is new and a consensus about what makes a system usable and useful has not been reached (Jameson, 2006). More traditional Human Computer Interaction research is applicable to the domain of adaptive system usability, but there are enough differences between an adaptive system and a static one to make direct application of HCI principles to adaptive systems an impossibility. For the purposes of this research, a suitable adaptation strategy is one that helps the user learn aspects of the software without causing the user to lose task focus due to interruptions by the adaptive system.

2 Research in Context: Document Authoring Adaptive systems have been used in many applications with varied degrees of success. The primary use to date has been to customize information delivery, for applications such as tutoring systems and advertising (Van Lehn, 2006). Other context aware systems will customize the display for different hardware devices (Jameson, 2006). The thesis of this research is that context-adaptive systems are ideally suited to assist users of complex software to learn to use the software without direct training. The concept of reducing the learning curve for complex software systems becomes clearer with an example. Consider a software application that supports the traditional writing process. This application provides support for research, note-taking, reference management, and draft management. It has templates for a wide variety of written work including technical writing, textbooks, fiction, poetry, etc. It includes mechanisms for importing and converting text and images to and from a variety of formats. It includes facilities for creating artwork and diagrams for inclusion in the authored materials. It even includes some sort of assistance for publication such as catalogues of appropriate venues and assistance with advertising and presentation. In short, this hypothetical application is a large, complex piece of software that facilitates every step of the authoring process. Now, further suppose that an individual must use this specific piece of software— perhaps because its use is mandated by the individual’s employer. It is easy to imagine difficulties in learning to use such a software system. Because the software is complex and multipurpose, there are several paths to accomplishing even the simplest task. Some of those paths are efficient and effective; some will work, but will be fraught with frustration and workarounds. (e.g., many people format text into tables using the tab key in their word processor even though it makes editing tables and adding rows or columns extremely difficult). A user whose work productivity depends on their ability to learn to use a complex software system can easily become

disillusioned and frustrated when the time to learn to use the software exceeds the time to perform a task manually. Users are often unwilling to make the investment of time to learn software because they cannot see how the software will improve their productivity or ability to perform their duties, especially when their initial encounters with the software are laborious and unproductive. This research proposes mechanisms that will allow complex software systems to help users discover the features most likely to assist with immediate tasks. The software system adapts to U’s learning style and U’s task using the principles of andragogy rather than pedagogy (Knowles, 1980).

3 Approach Software that adapts to contextual inputs can easily become disorienting or confusing if the adaptations are slow or result in an less responsive system, or if they are disruptive to the user’s workflow. The approach taken in this research is to minimize user disruption and adaptation complexity by focusing on only a small part of the contextual information that could be made available to such a system, and by ensuring that adaptations fit with U’s learning style.

3.1

Contextual information

Context-awareness once was the characteristic of systems that monitored the physical environment, such as location and temperature. Increasingly however, the definition of context awareness is expanding to include a holistic sense of context that includes the purpose of the activity, the domain of discourse, the user’s characteristics, characteristics of the computing equipment and sometimes even temporal information (Krzysztof & Daniel, 2004; Schmidt, 1999). A system S depends on a distinct user and context related items of information to effectively adapt its behavior for U. The items include: (a) environment and domain; (b) The current state of U and the task; (c) U’s behavior; and (d) U’s longer-term properties (Jameson, 2006; Krzysztof & Daniel, 2004; Tsandilas & Schraefel, 2004). Few adaptive systems factor in all of the different types of contextual information when making decisions; but often several different types are taken into account (Cheverst et al., 2001; Jameson, 2006; Krzysztof & Daniel, 2004; Strang & Claudia, 2004). Additionally, each type of contextual information has one of more subtypes, and individual subtypes may or may not be considered within any one adaptive system. For example, information about U’s behaviour might include a log of keystrokes, information from eye-tracking software, a history of applications used previously, a log of computer access times for some period of time in the past and a record of phone conversations. A specific adaptive system may require only the eyetracking data to make decisions and would not use any of the other behaviour related context.

This research builds upon Schmidt’s, and more recently, Cheverst’s work, who identified that there are four main ways in which context may be used to simplify the user’s interaction with an interactive system (Cheverst et al., 2001; Schmidt, 1999): 1. Simplifying the task specification. Simplifying the task specification requirement from the user in order to achieve their desired goals is a necessary component for a UAS (Cawsey, Jones, & Pearson, 2000; Cheverst et al., 2005; Goodman et al., 2006). Therefore, one goal of the system is to reduce the need for input and/or action by U. The research is this project focuses on reducing the complexity of rules constituting the user’s model of the system, thereby enabling the user to be productive with the system. 2. Changing the output produced by the system. This can be accomplished by two distinct ways: (a) reducing the quantity of information that has to be processed by the user; and/or (b) increasing the quality of information presented (Jameson & Krüger, 2005; Kobsa, 2001). Reduction in output may be achieved by attempting to ascertain the value of the output that is likely to be required by the user. 3. Reducing and/or normalizing the Model. In the field of HCI a number of researchers have determined that the user model does not need to capture everything about the user (Aleven, McLaren, & Koedinger, 2006; Koedinger, 2001; Van Lehn, 2006). This research proposes that effective adaptations can be made with relatively limited modeling and use of stereotypes based on learning styles research. 4. Reducing the User’s Cognitive Load. In a number of situations the cognitive demands on the user can be lessened. Research literature in this area has shown significant benefits from the user’s perspective by minimizing the user’s mental processing during task activities (Jameson & Krüger, 2005; Kobsa & Pohl, 1994; Sykes, 2007a, 2007b, 2007c; Sykes & Franek, 2004; Van Lehn, 2006). If it is possible to reduce the cognitive load for U, either by performing a task on U’s behalf or by minimizing the distraction caused by adaptations, and there is a sufficiently high degree of confidence that the computation can safely be done accurately, then it should be performed on U’s behalf. For this research, significant information required for adaptation decisions includes a knowledge of the task, knowledge of the current state of U and knowledge of the learning styles expected to be effective given knowledge about U and knowledge about the task. The different learning styles associated with tasks and users form the basis of the stereotyped models that bootstrap the system.

3.2

Stereotyped Models using Learning Styles

This section explores learning styles with the goal to design context adaptive systems in such a way to minimize U’s cognitive demands by “understanding” and supporting U’s unique form of learning. Studies have shown that when users were in a learning environment that matched their preferred learning style, they experienced higher achievement (Dunn, Beaudry, & Klavas, 1989; Kolb, 1984). We use Kolb’s

Experiential Learning Theory (ELT) as the framework to guide our research since it is a well-established and a recognized model that describes how learners see and interpret information (Kolb, 1984). Kolb’s theory is especially well suited to context adaptive systems because of the task-oriented activities users are involved with and the high degree of information (in various modalities) required to deal with when working with these systems. Kolb’s ELT is shown in Figure 1.

do

feel

perception continuum

Accommodators (feel and do) processing continuum Convergers (think and do)

Divergers (feel and watch) watch Assimilators (think and watch)

think Fig. 1. Kolb’s Experiential Learning Theory.

Kolb’s ELT consists of a two dimensional framework involving two axes (a) perception continuum (vertical axis; U’s emotional response to a task); and (b) processing continuum (horizontal axis; how U approaches a task). The four quadrants show the four types of learning styles: Accommodators; Divergers; Convergers; and Assimilators. Table 1 shows the characteristics of each type of learning style. Table 1. Learning Style and Characteristics. Learning Style Accommodators (A)

Divergers (D)

Convergers (C)

Assimilators (AS)

Characteristics 1. prefer “hands-on” experience over logical analysis 2. seek new experiences, take risks 3. good leaders 1. view concrete situations from different points of view 2. good at understanding 3. creative 1. combine active experimentation with abstract conceptualization 2. pragmatic and logical 3. good at solving problems and making decisions 1. prefer abstract conceptualization with reflexive observation 2. good at developing models and theories 3. good at planning

Research has shown that learning styles vary not only with individuals, but with occupation, job role, task type and educational focus (Boyatzis & Kolb, 1995; Strang & Claudia, 2004; Taylor & Tisdell, 2000). The characteristics of learning styles, combined with a knowledge of U’s background, role and current task, can be used to create effective but simple models of interaction that can then be used to select the adaptations that will facilitate U’s productivity and learning.

3.3

Modeling Goals and Tasks

The goal of U can be modeled as three individual parts. The first part of the model is an element selected from an ontology of tasks that describes typical user tasks performed with the adaptive system. The second part of the model is a description of the desired end result, or the product that U will create while reaching the goal being modeled. The third part of the model is a description of task completion strategies, or steps for accomplishing the goal and creating the product. These parts contain, respectively, the task decomposition, the temporal and sequential constraints on goal completion, and the goal hierarchy. A portion of a prototypical task model is shown in Figure 2. The task is to create a research paper, one of the types of tasks that the hypothetical system described in section 2 would commonly be used for. The ovals in the figure represent products, the rectangles represent steps or completion strategies. Products can be subclasses or composites of one another. Completion strategies are ordered (temporally or sequentially) and have a product as a result. Completion strategies can have associated learning styles that represent the stereotype strategy most often used to achieve a particular step. When a discrepancy between U’s preferred style of learning and the style associated with a task completion strategy is detected, the adaptive system can more closely guide U through the task. Conversely, when U’s preferred style matches the style associated with the task, the adaptive system can stay in the background. New tasks can easily be modeled or novel approaches to completing the tasks (task completion strategies) can be exploited.

Written Product

subclass

Survey

Presentation Research Paper

pa rtO

partOf

f

Background Section completionStrategies (1..N)

completionStrategies (1..M) literature collection

identify topic s de ce pre

Thesis pro du ce d

Arguments

produced

background research pr e

e pr

s de ce

cursory read

ce de s

pre

ce d

es

pr od e uc

identify thesis

d

s de ce pre

Conclusion

Abstract

pr od u

pr od

build case ce d

uc e

d

pr ec ed e

detailed read s de ce pre

analysis pre

s

create paper

ce de s

reflection

Fig. 2. An illustration of the components for a written product.

Figure 3 shows a possible set of reasoning steps that the adaptive system could make in its efforts to accommodate U’s efforts to use the software system. In response to U’s activities (or lack of activities) the system can initiate a chain of reasoning to determine if U needs assistance to complete the task-step under consideration.

Model of U

U’s learning style

Does U need assistance?

Recent Internet/ Library searches?

Assessment of current task

How focused is U on the task?

Pause assessment: too long?, too short? etc.

Is U performing an action related or unrelated to the task?

Fig. 3. Partial Bayesian Reasoning Network.

Each task can have several associated completion strategies, each associated with different styles of learning. The adaptive system can select the strategy most likely to accommodate the needs of the user and demands of the task and adapt itself accordingly. The algorithm for the selection of appropriate completion strategies such that following factors are richly considered: (a) environment (context); (b) current state; (c) U’s behavior; and (d) U’s longer-term properties based on the following desired aspects: (a) U’s cognitive load is minimized; (b) supporting U by generating a personalized user interfaces automatically, and (c) using computational model (e.g., decision-theoretics and machine learning techniques) to extract salient information about U and U’s goals.

3.4

Making Adaptations

Four ways to use context to simplify interactions were discussed in an earlier section. This research focuses only on reducing the model used by the system and on reducing the cognitive load faced by the user as they attempt to understand the system. Reducing the modeling overhead is accomplished by creating stereotyped models of user and task based in learning theory and then refining those stereotypes with information gleaned from the user’s activities. Information about user ability or background knowledge is not required nor is information about the domain of discourse. The cognitive overhead of the user is reduced by adapting the software system so that the steps required to accomplish the task at hand match, as closely as possible, to the preferred learning style of the user. Many adaptations are possible using learning styles as a basis for the models. The strategy is illustrated through the use of examples. Consider a scenario in which U is working in a spreadsheet and wishes to write a function to compute the average on a set of values (e.g., grades for a course). Let T be this task and U explicitly asks the system for assistance with T. Depending

on U’s learning style, the context adaptive system would respond appropriately as described below. Convergers: These types of learners like the “hands-on” approach, however, they are content to think at an abstract level. In this situation, CAS would accentuate the “hands-on” aspect yet support the activity by providing high-level description of how to accomplish T. The CAS would first provide a brief explanation of the process, followed by the exact steps needed to accomplish T. Since these types of learners are good at abstraction, the actual dataset used may be different than U’s current dataset. In other words, the presentation of analogies (and generalizations) would be something U would appreciate. Accommodators: These types of learners are like Convergers in that they like “hands-on” activities, however, they have a strong “feeling” characteristic which influences the CAS to emphasis on the practical and the personal. In relation to the scenario, the CAS would first provide a brief overview and the steps to follow to accomplish T using U’s dataset to ensure personal relevance to the activity. Divergers: These types of learners “feel and watch.” As a result, the CAS would focus on the personal aspects of T that relate to U by providing a brief overview of solution, and then demonstrate the solution to U using U’s dataset. Assimilators: These are the most abstract type of learners—they think and they watch. If U were of this type, then the CAS would explain the high level details to solve T, display the steps, and then perform some (or all) of the steps on U’s behalf. The execution of steps would be presented in such a way that through watching, U would be able to effectively learn. Obviously, there are many other variations and adaptations possible given the different learning strategies. The adaptation possibilities presented above can be dynamically altered depending on the specific characteristics of U and of the task. For example, suppose U were an Assimilator but the task at hand required Converger characteristics; the CAS would respond by using analogies and offering high-level explanation and descriptions of solutions. Secondly, the CAS would track U’s profile so that half of the time, the CAS would perform the operation (U watches), and the other half of the time, U would do the steps outlined by CAS. Other such adaptive techniques could be implemented to more closely match context with preferred learning styles. Within the adaptive system, possible adaptations are annotated with information about which style of learning is supported by the adaptation. Decisions about adaptations are made by comparing the various styles of learning associated with the user, the task, the user’s job and the current interaction and making adaptations to reduce the differences as much as possible.

3 Concluding Remarks This paper represents a partial snapshot of an active, in-progress research program. At the time of writing, the development of a working proof-of-concept prototype is underway and experiments to demonstrate the effectiveness of the approach are being designed.

Most experimental studies in the field of HCI involve two or more variants of a given system are compared within a controlled environment in which users perform predefined tasks. When the system is a UAS, generally adaptive and non-adaptive variants of the system are compared. Jameson, 2006, points out “in a few years, we may see more studies in which the main focus is on the comparative evaluation of different variants.” (Jameson, 2006) This research will employ comparative evaluation to assess the usability and effectiveness of the adaptations.

Acknowledgements I wish to acknowledge Dr. Nonnecke and the students of HCI class of 2006. I was inspired by the great discussions and presentations.

References Aleven, V., McLaren, B., & Koedinger, K. R. (2006). Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor. Paper presented at the International Journal of Artificial Intelligence in Education. Boyatzis, R. E., & Kolb, D. (1995). From learning styles to learning skills: the executive skills profile. Journal of Managerial Psychology, 10(5), 3-17. Brusilovsky, P. (2001). Adaptive Hypermedia. User Modeling and User-Adapted Interaction, 11, 87-110. Cawsey, A. J., Jones, R. B., & Pearson, J. (2000). The Evaluation of a Personalised Health Information System for Patients with Cancer. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 10(1). Cheverst, K., Byun, H. E., Fitton, D., Sas, C., Kray, C., & Villar, N. (2005). Exploring Issues of User Model Transparency and Proactive Behaviour in an Office Environment Control System. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 15(3-4), 235-273. Cheverst, K., Davies, N., Mitchell, K., & Efstratiou, C. (2001). Using Context as a Crystal Ball: Rewards and Pitfalls. Personal and Ubiquitous Computing, 5, 8-11. Dunn, R., Beaudry, J., & Klavas, A. (1989). Survey of research on learning styles. Educational Leadership, 46(6), 50-58. Goodman, B. A., Linton, F. N., Gaimari, R. D., Hitzeman, J. M., Ross, H. J., & Zarrella, G. (2006). Erratum: Using Dialogue Features to Predict Trouble During Collaborative Learning. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 16(1), 83-84. Horvitz, E. (1999). Principles of Mixed-Initiative User Interfaces. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit Pittsburgh, Pennsylvania, United States Jameson, A. (2001). Modelling both the Context and the User. Personal and Ubiquitous Computing, 5, 29-33.

Jameson, A. (2006). Adaptive Interfaces and Agents. In J. A. Jacko & A. Sears (Eds.), Human-computer interaction handbook (pp. 305-331). Mahwah, NJ: Erlbaum. Jameson, A., & Krüger, A. (2005). Preface to the Special Issue on User Modeling in Ubiquitous Computing. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 15(3-4), 193-195. Knowles, M. S. (1980). The modern practice of adult education: from pedagogy to andragogy. Chicago, IL: Association Press Kobsa, A. (2001). Generic User Modeling Systems. User Modeling and UserAdapted Interaction: The Journal of Personalization Research, 11(1-2), 4963. Kobsa, A., & Fink, J. (2006). An LDAP-Based User Modeling Server and its Evaluation. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 16(2), 129-169. Kobsa, A., & Pohl, W. (1994). Workshop on Adaptivity and User Modeling in Interactive Software Systems. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 3(4), 359-367. Koedinger, K. R. (2001). Cognitive tutors. In K. D. Forbus & P. J. Feltovich (Eds.), Smart machines in education (pp. 145-167). Cambridge, MA: MIT Press. Kolb, D. (1984). Experiential learning: experience as the source of learning and development. Englewood Cliffs, N.J.: Prentice-Hall. Krzysztof, G., & Daniel, S. W. (2004). SUPPLE: automatically generating user interfaces. Funchal, Madeira, Portugal: ACM Press. Nair, H., Chintagunta, P., & Dubé, J.-P. (2004). Empirical Analysis of Indirect Network Effects in the Market for Personal Digital Assistants. Quantitative Marketing and Economics, 2(1), 23-58. Schmidt, A. (1999). Implicit human-computer interaction through context. Paper presented at the Proceedings of the 2nd Workshop on Human Computer Interaction with Mobile Devices., Edinburgh, Scotland. Smith, R. (2002). Adapting a new technology to the academic medical library: personal digital assistants. Journal of the Medical Library Association, 90(1), 93-94. Strang, T., & Claudia, L.-P. (2004). A Context Modeling Survey. Paper presented at the Workshop on Advanced Context Modelling, Reasoning and Management - The Sixth International Conference on Ubiquitous Computing,, Nottingham, England. Sykes, E. R. (2007a). Determining the Effectiveness of the 3D Alice Programming Environment at the Computer Science I Level. Journal of Educational Computing Research. Sykes, E. R. (2007b). Developmental Process Model for the Java Intelligent Tutoring System. Journal of Interactive Learning Research, 18(3). Sykes, E. R. (2007c). Qualitative Evaluation of the Java Intelligent Tutoring System. Journal of Systemics, Cybernetics, and Informatics, 3(5). Sykes, E. R., & Franek, F. (2004). A Prototype for an Intelligent Tutoring System for Students Learning to Program in Java. International Journal of Computers and Applications, 1, 35-44.

Taylor, E. W., & Tisdell, E. J. (2000). Adult Education Philosophy Informs Practice. Adult Learning, 11, 6-10. Tsandilas, T., & Schraefel, M., C. (2004). Usable adaptive hypermedia systems. New Review of Hypermedia and Multimedia, 10(1), 5-29. Van Lehn, K. (2006). The Behavior of Tutoring Systems. International Journal of Artificial Intelligence in Education, 16(1), 195-223. Webb, G. I., Pazzani, M. J., & Billsus, D. (2001). Machine Learning for User Modeling. User Modeling and User-Adapted Interaction: The Journal of Personalization Research, 11(1-2), 11-21. Weld, D., S. , Anderson, C., Domingos, P., Etzioni, O., Gajos, K., Lau, T., et al. (2003). Automatically Personalizing User Interfaces. Funchal, Madeira, Portugal: ACM Press. Wolfe, K., Bates, D., Manikowske, L., & Amundsen, R. (2005). Learning Styles: Do They Differ by Discipline? Journal of Family and Consumer Sciences, 97(4), 18-22.