Learning technology and usability: a framework for ... - CiteSeerX

MAYES, J.T. & FOWLER, C.J.H. Learning technology and usability: a framework for understanding courseware. Interacting with Computers 11, 485-497, 1999

Learning technology and usability: a framework for understanding courseware

J. Terry Mayes1 and Chris J. Fowler2 1 Centre for Learning and Teaching Innovation Glasgow Caledonian University St Andrew House 141 West Nile Street Glasgow G1 2RD tel: 0141 331 1271 email: [email protected] 2 BT Labs Martlesham Heath Ipswich IP5 7RE Tel:01473 644318 Email: [email protected]

1

Abstract This paper argues that the usability of educational software cannot be measured in the same terms as other work contexts. This is because learning is a byproduct of understanding rather than an activity which can be supported directly. Although it is best achieved through the performance of meaningful tasks, these tasks need to be designed to support different kinds of learning. We approach the problem through an attempt to derive a framework for understanding courseware. Conceptual learning is characterised as a cycle, involving the three stages which we term conceptualisation, construction and dialogue. These are mapped onto primary, secondary and tertiary courseware. Each kind of courseware is discussed in terms of efficiency, effectiveness and usability.

Introduction Usability, in the straightforward sense, is a self-evident requirement for all software. Yet there is an interesting paradox in the case of some educational applications, where a seamless fluency of use is not necessarily conducive to deep learning. The learner needs to move effortlessly to the conceptual level, but then must engage with the underlying meaning. To put it simply, the software must make the learner think. Learning cannot be approached as a conventional task, as though it were just another kind of work, with a number of problems to be solved, and various outputs to be produced. This is because learning is a byproduct of doing something else. It is the 'something-else' that needs support. This can be achieved in many ways, but none of them have much to do with the conventional multimedia properties of the content. Today, the constructivist approach to learning is widely supported (Duffy et al, 1992), although the epistemological foundations of this movement are under attack (Phillips, 1995). Constructivist approaches emphasise the active building of understanding through the performance of learning tasks in which the learner decides how to proceed, based on his or her current understanding of the task and of the domain of knowledge in question. Often the task will involve some kind of problem-solving, although this can take many different forms. The goal is for learners to build their own knowledge. This contrasts with an instructivist view emphasising the delivery of explanation. A crude version of this is the conventional approach to training in computer skill, characterised by Carroll (1990) as the Nurnberg Funnel in which knowledge is poured into the head of a pupil. In its more sophisticated and enticing form, represented by the intelligent tutoring approach (eg Ohlsson, 1993), learning is seen as a process of knowledge acquisition rather than knowledge construction, where the learner’s

2

misconceptions are identified and corrected through individualised instructional explanation. These contrasting pedagogical assumptions imply different approaches to software usability. Both assume a straightforward ‘operational’ usability, allowing the functionality of the software to be discovered and used, but then they diverge on the question of learning. Instructivist approaches involve factors which will emphasise the impact of content presentation on the learner - its accessibility, its vividness, the power of its explanation, the appropriateness of its representation. Constructivist approaches, on the other hand, focus on supporting the learner in the performance of tasks which have been designed to engage the learner in active problem solving, questioning and conceptual manipulation. Even more distant from the conventional usability of the software itself are factors which partly determine motivation. These will include the way in which the technology has been introduced to the wider learning environment, the impact on time spent in face-to-face teaching, the resources which have been relinquished in order to purchase the hardware, gender attitudes, and a host of other variables which constitute a social and organisational dimension to usability. In general we will argue for a view of usability which emphasises learning outcomes. We will attempt to justify this argument by: • briefly characterising conceptual learning as an iterative conceptualisationconstruction-dialogue cycle, where the construction of meaning, and the testing of this against other judgements, are the crucial stages for educational software to support. • giving some examples of classes of software which support learning at each stage of this cycle. • examining the relationship between usability and educational effectiveness for each stage. Usable systems involve capitalising on the user's pre-existing knowledge. Similarly, effective educational material involves information for which a framework for extracting meaning is already in place. So both concepts (usability and learning effectiveness) involve the role of prior learning. Attempting to learn material for which no previously established conceptual framework exists is, literally, a meaningless task. Equally, trying to use technology without the assumed level of prerequisite understanding leads to usability failure. The main challenge in the design of educational software is to support the user in the creation of such a framework for extracting meaning: this requires the design of effective tasks, rather than interfaces. For example, while an important usability metric for all other kinds of work is to avoid allowing the user to get

3

lost in information space, in some learning situations it would be beneficial for deep learning for the user to do just that. The learning which would often be associated with navigating out of such situations might not occur with an interface which did too much of the work. Effective design of educational software also requires being clear about the prior knowledge of the users, and the educational setting in which the software will be used.

Assumptions about learning The enthusiasm with which multimedia technology has been embraced in education, not only by those with a vested interest in its uptake, but also by policy makers who see learning technology raising the efficiency of the entire educational process, should be treated with caution. Accompanying this enthusiasm has been an insidious shift in the language used to describe the learning process. Increasingly, the power of access and display for multimedia content, coupled with the dramatic growth of the Web, has encouraged a crude ‘delivery’ model of education. To counter this, we re-emphasise four fundamental points about learning. Each of these is well understood, but their importance may sometimes have become obscured by this shift of emphasis. The first point is that learning is best viewed as a by-product of understanding, and that understanding must be built by the individual learner in the performance of tasks. Second, that the growth of understanding depends on frequent feedback, particularly from teachers and peers. Third, that it is helpful to model learning as a progression through stages, and finally that all learning is situated in a personal, social and organisational context. A fundamental description of learning was provided by William James (1890): "the one who thinks over his experiences most, and weaves them into systematic relations with each other will be the one with the best memory". The simple point is that conceptual learning depends on thinking, and the more one thinks about something the better it is remembered. During the 1970's and 80's this point was elaborated into a theory of memory called "levels of processing" (Craik & Lockhart, 1975). How well something was learned could be described as being directly related to the depth to which it had been analysed. This theory assumes that information which seems immediately meaningful is easily learned because of "compatibility with previously existing cognitive structures". Craik and Tulving (1975) obtained the empirical evidence necessary to build the levels-ofprocessing approach into a full-scale theory. They found that what is most critical is not simply that information is made meaningful, but the richness or degree of elaboration of that meaning. In addition, superior learning was found to result from material that has been acted upon,and so long as deep processing occured it did not seem to matter whether there was actually an intention to

4

learn. The crucial point is the modelling of learning as a by-product of comprehension. Although this point was buried in the literature of the cognitive psychology of memory, it underpins the constructivist approach to education. Comprehension is rarely an absolute matter, except in fairly closed domains. In general comprehension will grow as current understanding is applied in different contexts. Thus the adequacy of the understanding needs to be tested, and the results of the test fed back into further conceptual development. The concept of feedback has always played a central role in learning theory, and was described in a way that bridged between behaviourist, cybernetic and cognitive approaches in the TOTE (Test-Operate-Test-Exit) formulation of Miller, Galanter & Pribram (1960). This can be thought of as a building block for more complex descriptions. It corresponds to the goal-action-feedback cycle and it captures the dynamic and iterative nature of learning: as understanding grows so knowledge is rebuilt through results. A constructivist approach to learning must provide not only the environment and the tools for the active construction of knowledge, but also the availability of appropriate feedback on the learner’s progress. Another central theme uniting approaches to learning has been the idea that the essential characteristics of the learning process change as learning progresses. It is important to match resources and support to the appropriate stage. Most commonly, three stages have been identified (Fitts & Posner,1967; Anderson,1982) describing the transition from novice to expert performance. Rumelhart & Norman’s (1978) account uses the language of schema theory to describe the three stages. In this account structuring is the formation of new schemata. A new schema is formed when existing schemata will no longer be sufficient for full understanding. Structuring occurs relatively rarely and often involves considerable cognitive effort. It involves questioning, hypothesis testing and problem formulation and is usually observed in the context of active work of struggling to understand a problem, of presenting one’s own summary of a complex area, or of trying to teach something to someone else. Once a structure is in place then incoming information can be “chunked” into meaningful patterns. Accretion is the adding of new knowledge to existing schemata. Once the higher-level framework for understanding has been successfully structured, accretion is the mode by which new data are entered. This is by far the most common form of learning. However, a third stage of learning involves the fine adjustment of knowledge to the demands which are made of it. The schema exists, with appropriate knowledge in all the slots. Yet the application of the knowledge in the performance of real tasks is not yet optimal. So with practice a process of tuning ensures that the schema becomes developed. This classification helps to focus our attention on the learning process. Learning is always situated in a context which will shape the approach adopted by the individual. Not all learners will seek understanding, and some learning 5

about the use of technology will be deliberately short-term, discarded when a goal is reached. Also, the acquisition of procedural skill will often not involve comprehension in the usual sense. Clearly, the constructivist approach must take account of the fact that not all learning needs to be underpinned by the understanding of underlying principles and concepts. Nevertheless, in educational settings it is hard to think of situations where learning through comprehension would be inappropriate. The most important situational variables in education will usually be the nature of the assessment, and the attitudes towards learning of peers. The organisational setting will set the parameters of the learning task, and the most direct expression of this will be the 'contract' between students and teachers which will set the expectations and norms. Much of what is referred to as student motivation is determined by these factors. The effectiveness of learning technology will, like all other variables in the educational setting, be determined largely by these wider factors, rather than by the intrinsic features of the technology or the content.

The Learning Cycle It is helpful to describe the basic unit of conceptual learning as a cycle (Kolb, 1984; Mayes, 1995). Learning is often thought of as a discrete activity, yet this is misleading since it involves a process of continuing growth, in any domain (Dewey, 1938). The notion of continuous development involves the repeated testing of a current conceptualisation for adequacy. Adequacy of understanding requires testing against criteria, whether these are defined internally or externally, the latter through assessment. The notion of a conceptualisation cycle integrates the three kinds of learning described by Rumelhart & Norman (1978) into one continuous process of the refinement of understanding. The starting point is illustrated in Figure 1.

Conceptualisation

Test

Reconceptualisation

6

Exit

Figure 1: First order description of conceptualisation This merely describes learning as a cyclical dynamic feedback process. To build from this, we need to elaborate the component of reconceptualisation. Two further processes can be seen to be fundamental to the developmental aspect of conceptualisation. These are the performance of learning tasks resulting in some output from which learning occurs as a by-product, and secondly, applying the developing conceptualisation in new contexts. A second-order description can therefore be based on the following: • Conceptualisation - refers to the users’ initial contact with other peoples’ concepts. This involves an interaction between the learner’s pre-existing framework of understanding and a new exposition. • Construction - refers to the process of building and combining concepts through their use in the performance of meaningful tasks. Traditionally these have been tasks like laboratory work, writing, preparing presentations etc. The results of such a process are products like essays, notes, handouts, laboratory reports and so on. • Application - the testing and tuning of conceptualisations through use in applied contexts. In education, however, as Laurillard (1993) has pointed out, the goal is testing of understanding, often of abstract concepts. This stage is best characterised in education, then, as dialogue. The conceptualisations are tested and further developed during conversation with both tutors and fellow learners, and in the reflection on these.

Conceptualisation

Primary courseware

Construction

Secondary courseware

Dialogue

Tertiary courseware

Figure 2: The (re)conceptualisation or learning cycle

7

Figure 2 describes a framework which we can use to distinguish different kinds of courseware. This will enable us to approach the notion of usability with a clearer understanding of the functional requirements of the technology.

The nature of courseware Most learning technology provides access to courseware, normally interpreted to mean the computer based delivery of learning materials. The simple term is too general since it obscures differences both in the pedagogy and in the associated technology. A new classification is offered here, based both on the way the courseware is originated, and on its mapping to types of learning. Primary Courseware is courseware intended mainly to present subject matter. It would typically be authored by subject matter experts but is usually designed and programmed by courseware specialists. Increasingly, primary courseware will be seen as a publishing product, for wide distribution. Secondary Courseware describes the environment and set of tools by which the learner performs learning tasks, and the tasks (and task materials) themselves. Here, the products are volatile and of varied quality. Tertiary Courseware is material which has been produced by previous learners, in the course of discussing or assessing their learning tasks. It may consist of dialogues between learners and tutors, or peer discussions, or outputs from assessment. One kind of tertiary material will be compiled from the questions, answers and discussion that will typically be generated by a computer conference. Most examples of current courseware are primary. These come in many forms, some of which will involve impressive interactivity in simulated environments. The learner may explore the exposition, and may even be able to pose ‘what if’ questions of the software. Nevertheless, the purpose of the courseware is to provide an exposition of subject matter. Secondary courseware, on the other hand. directly supports the learner's task-based learning activity. One form of this will comprise descriptions, instructions and materials for the learning tasks themselves. The term also refers to the task-support environment. One range of tools, mindtools or cognitive tools (Kommers et al, 1992), have been designed specifically to encourage users to have to think conceptually about the subject matter being manipulated. Other examples would include languages like LOGO, and authoring software such as Authorware or Director. Also included would be HTML (the hypertext markup language) or Microcosm. However, our concept of secondary courseware is wider than specialist software designed specifically to support the structuring of material. It would involve any use of the computer to produce output when the task is primarily for learning. By this definition a word processor, or any other productivity software, could be defined

8

as secondary courseware since the defining characteristic is the nature of the task, rather than the attributes of the software. Learning tasks will also require information to be sought, identified and retrieved, and this will mean that information search and retrieval tools will also support the task. The difficulty in classifying educational software by its overt functionality forces us to focus on its use in real learning contexts. Some ten years ago (Mayes et al, 1988) one of the current authors was involved in the development of a computerbased learning environment called StrathTutor. It is interesting to trace how our thinking about how to use StrathTutor developed with our experience of trying to use it effectively. At first we saw it as a learning-by-browsing system, where primary courseware could be explored using a number of search and navigation techniques. Then we came to realise that its most effective features lay in problem solving, with the learner competing against the system (i.e. the author) in the playing of a kind of subject-matter game. Finally we came to realise that the main learning gains were achieved by the authors of StrathTutor materials, rather than by the recipients of that material, and that its most effective use was to put its authoring tools into the hands of learners directly. We would now see StrathTutor as offering both primary and secondary courseware, the main classification depending on the particular context of use. Tertiary courseware is a new conception of courseware, and there are few current examples. The defining characteristic is the 're-use' of the learning experiences of other students. There are many possible approaches, one of which would be to provide a distributed database of answers to "frequently asked questions", where the questions have been collected from real learning episodes. A somewhat different version of the idea has been described as "reification” (Boder, 1992). This emphasises the value of recording discussions between peers, and structuring those into an evolving database. The inspiration for our own thinking about this idea was the "Answer Garden" described by Ackerman & Malone (1990), and by Ackerman & McDonald (1996) . The Answer Garden allows the development of databases of commonly asked questions that grow "organically" as new questions arise. It is designed to be a resource for dealing with situations where there is a continuing stream of questions, many of which occur over and over again, but some of which are novel. Questions for which there are not yet satisfactory answers in the database are automatically routed to appropriate experts (tutors), and then inserted, together with their answers, into the network. So the database grows with real use. It is easy to imagine how this idea could be used by tutors, or teaching assistants, dealing with large classes. It might also provide a courseware layer in an integrated system, where primary courseware would represent another, linked, layer (Cumming, 1993). This would allow the primary teaching

9

material to be continually improved. Many variations of the basic idea can be considered. Its potential in education and training has been recognised in the "The Answer Web" (Smeaton & Neilson, 1995) which seeks to contextualise the dialogue by attaching states of simulation software that ground the questions and answers in the courseware giving rise to the learning issue under discussion. Another example derived from the Answer Garden, also developed in the TLTP Programme, is the 'Knowledge Garden' (Brailsford et al; 1996). In our own work on deriving tertiary coursework we have used the HyperNews conferencing tool, integrated with Netscape, to structure peer discussion. We have also experimented with giving current students access to the courseware annotations of previous students, to their coursework, and even to previous attempts at assessment tasks. In general we have found that only a minority of the spontaneous HyperNews discussions are suitable for re-use, and we have turned to the design of special tasks which make learner perspectives more visible than they might otherwise be. The idea of tertiary courseware seems important because it offers a way in which computers might be able to provide a partial experience of dialogue in educational situations where it is simply not possible for teachers to engage in one-to-one conversations with students (McKendree & Mayes, 1997). In that sense it offers a real alternative to intelligent tutoring. The effectiveness of the idea depends on the validity of the underlying assumptions about learning: these assumptions are conveyed by the term vicarious learning.

Vicarious Learning To what extent can learners benefit from the dialogues produced by other learners? We assume that some real learning occurs through observation of other learners engaged in real learning episodes. In real classroom experience the questions asked by other learners, and the resulting discussion, can often articulate and expose shared aspects of conceptual difficulty. Such dialogues can involve discourse between learner and teacher, discussion between peers, or even direct interaction with courseware. Such dialogues may constitute a new kind of ‘reusable’ learning resource, tertiary courseware. This idea contrasts with the traditional approach to instructional dialogue by assuming that there is considerable scope for learning without being a direct participant. Although over a quarter of a century has passed since SCHOLAR (Carbonell, 1970) demonstrated that it was possible to simulate a tutorial dialogue through machine representation of domain knowledge, it is still not possible to construct dialogue-providing tutors for subject-matter where discussion and reflection are important components. This is because we cannot build programs which can sufficiently-well understand the conceptual difficulties a learner is currently

10

experiencing and which can offer direct help. The vicarious learning approach to is to take real discussion between learners and tutors as the raw material from which a new kind of courseware can be compiled for new learners. This involves recording real dialogues of previous learners in equivalent situations and making these accessible at just the right moment in the conceptualisation cycle for new learners. The challenge, of course, is to design a way in which appropriate dialogues can be matched to the learner's immediate learning need. The idea is attractive because it partially addresses the fundamental problem of educational resourcing, the fact there are always too many learners and too few teachers. It also partly avoids the need for conventional courseware development, which is notoriously costly. The approach of evolving courseware out of real teaching and learning experiences also conveys a satisfying usercentred approach. Essentially, the learner is told: "here are some problems previously experienced by other learners, see if you can find one that is similar to your own difficulty". In recent experiments (McKendree & Mayes, 1997) the idea of tertiary courseware has been extended to include the outputs of coursework from previous learners, including the feedback from tutors which also serves to provide a kind of vicarious resource. Obviously, the idea of tertiary courseware raises some fundamental design problems. It is clearly valuable for a tutor directly to answer questions while sensitive to the context in which it has been asked. It is less clear what the value of the same answer will be to a future questioner whose learning context will be different. Contextualising the dialogues so that an optimum match can be found on the underlying conceptual difficulty shared by a previous and a new learner represents one kind of design difficulty. Identifying suitable dialogue fragments is another.

Courseware usability To gain insight into usability requirements, at least beyond the level where the user has become fluent in gaining access to the learning support system, we need to understand where the value of the technology mainly lies. We have distinguished three kinds of courseware by arguing almost entirely from a learner-centred perspective. These three kinds of educational software attempt to support quite different kinds of activity. The usability of each will entail quite different kinds of design criteria. Primary courseware In the conceptualisation stage, the state of development of the learner’s schema will determine the amount of meaning extracted from the information presented. The match between pre-existing knowledge, and the available learning materials, is all-important. Designing the usability of this material is the

11

familiar problem of matching the characteristics of a system to the expectations and knowledge of the user. However, the medium of expression of that information is likely to be of only marginal importance for learning (Mayes, 1995). This means that a primary exposition - an effective lecture, a powerful notation, an animated illustration, will serve to provide an initial contact with the conceptual knowledge that constitutes the subject matter. For the learning cycle to operate, however, the stages of construction and/or dialogue must come into play. The interaction between the learner’s prior understanding and the primary exposition produces only an initial interpretation. However, the most important single function of the primary exposition is to orient the learner towards the subject matter. It gives the learner a map of what is to be learned and understood through subsequent learning activity. A live lecture is highly effective for orientation. Similarly, the traditional media of print, film and video achieve the goal of providing an engaging and compelling primary exposition of the subject matter. At this level there is no real case to be made for learning technology in terms of effectiveness, ie the gains are not directly reflected in the quality of the learning outcomes. However, networked delivery can provide the learner with access to impressive quantities of primary courseware, and thus will rate highly for efficiency. Adjustment of the level of description of learning material to match the momentto-moment needs of the individual learner is one of the primary goals of tutoring. This can be achieved through offering the learner a range of descriptions and allowing choice, as in hypertext/hypermedia, or through discussion, where tutors can adjust their descriptions according to the conceptualisations revealed by the learner. In the strongest, but most important sense of the term, the usability of primary courseware will be mainly determined by the match between material and learner on the dimensions of conceptual demands and requirements for prior understanding. Compared with these the design of the media characteristics of the presentation itself are of secondary importance. Secondary courseware The construction stage emphasises the need for a task-based approach to learning. Learning tasks will vary on many dimensions, but the essential requirement is that the learner should engage at a conceptual level. The case for effectiveness is that some structuring software - mindtools - have been explicitly designed to force users to think about the subject matter at a deeper conceptual level than would otherwise have been the case, even when the learner is writing a conventional essay or lab report. Certainly, a great deal of current software is designed to improve the efficiency of construction tasks. Learners have immediate access to large sources of information, and a whole range of

12

functionality aimed at the creation of new structures. Most of the conventional approaches to usability will apply here - the need for consistency, memorability, learnability and satisfaction in using the tools (Nielsen, 1993). Underpinning this, however, is the basic suitability of the task for its educational purpose. To assess the wider usability of this kind of software will involve devising cognitive measures of conceptual engagement (Newman et al, 1996). Tertiary courseware Dialogue is fundamental to education (Laurillard, 1993). Of course, it is perfectly possible to learn without dialogue, but the need to test one’s developing understanding through asking questions, offering opinions, or challenging other positions is central to the educational process and is absolutely dependent on receiving a response. Reflective thinking, as a kind of dialogue with oneself, has also been regarded as a vital component of conceptual learning. John Dewey, for example, discussed the need to support reflection in the very first issue of the Science Education Journal in 1916. Learning dialogues can be supported, or made possible, at a distance by the new technology of synchronous communication. Thus, videoconferencing will allow tutors and learners to participate in real-time dialogues of various kinds, where the achievement of 'telepresence' becomes important. Audiographics and shared workspaces also providing opportunities for real-time discussion across a distance, as well as for sharing applications in co-operative tasks. Asynchronous communication, on the other hand, is particularly effective in fostering reflective learning. There will be effectiveness gains where dialogue can be offered to distance learners. In general, though, communications technology will not directly address the problem that there are too few teachers to provide individual dialogues to all learners. Tertiary courseware is aimed at exactly this problem. The efficiency gain here will be to retain some of the features of tutorial dialogues which can no longer be sustained within the current system. The larger the scale on which the tertiary courseware idea is implemented, the more attractive it becomes. It is possible to see how very large efficiency gains might emerge if an Answer Garden were to be compiled for an entire discipline. In general, the added value of technology for learning becomes greater as we move from primary exposition to construction to dialogue. Traditional media support exposition well. Communications technology becomes important in construction, particularly in group learning tasks, and especially strong in supporting dialogue. The design of tertiary courseware presents some formidable challenges, as well as opportunities. Ultimately the test of its effectiveness is to detect some of the same kind of benefits for the learner that will result from real dialogue with tutors or peers. At present we know too little about what contributes to an

13

effective learning dialogue in conventional teaching situations, and an iterative design process for reusable dialogue needs to be informed by a long-term research programme on the nature of learning dialogues in general. The issue of the usability of such courseware should probably be approached first through straightforward measures of its frequency of use. If learners find the dialogues of previous learners of genuine benefit in learning then they will seek to use tertiary courseware as a valued resource.

Conclusions Our framework maps three fundamental stages of learning onto three kinds of courseware. The main thrust of the argument is that primary courseware has been over-emphasised at the expense of supporting task-based learning and, especially, dialogue. As the technology of learning becomes ubiquitous at all levels of education, so it will be seen to be a tool for supporting the kind of tasks that teachers have always encouraged learners to undertake. We can expect a more focused effort to go into the design of task support environments for particular disciplines. The idea of tertiary courseware, however, opens up a new kind of thinking about learning technology, and we expect to see many versions of this idea emerge in education. Will the learner-initiated dialogue become the cornerstone of learning support? The main message here is that nothing can be as effective as the dialogue between a teacher and a learner. It remains to be seen whether technology can provide a partial substitute for this, in situations where the real thing is not readily available. It is worth noting that the approaches to courseware outlined in this paper might also be directed towards the design of help systems, and online tutorials in the use of software. The idea of vicarious usability, where a user observes the attempts of previous users to learn the system, is interesting but beyond the scope of our present discussion.

Acknowledgements Some of the arguments of this paper were first presented in a report for BT Labs, "Learning through Telematics", available on request from the authors. Support for the research on vicarious learning is being provided by the ESRC Cognitive Engineering programme, and the tertiary courseware idea is being explored in a project funded by the EPSRC Multimedia Networking Applications programme. We are pleased to acknowledge the contributions of our colleagues on these projects: John Lee, Jean McKendree, Patrick McAndrew, Keith Stenning, Jonathan Kilgour, Finbar Dineen, Richard Cox, Neil Finlayson and Richard Tobin.

14

References Ackerman, M., & Malone, T. (1990). Answer Garden: a tool for growing organisational memory. Paper presented at the Proceedings of the Conference on Office Information Systems, New York. Ackerman, M. S., & McDonald, D. W. (1996). Answer Garden 2: Merging Organisational Memory with Collaborative Help. Paper presented at the Proceedings of the ACM Conference on Computer-Supported Collaborative Work (CSCW'96). Anderson, J.R. (1982) Acquisition of cognitive skill. Psychological Review, 89, 369406. Boder, A. (1992). The process of knowledge reification in human-human interaction. Journal of Computer Assisted Learning, 8, pp 177-185. Brailsford, T. J., Davies, P. M. C., Scarborough, S.C., & Trewhella, W.J. (1996) Knowledge Tree: Putting discourse into computer-based learning. ALT-J, 5(1), 19-26 Carbonnel, J.R. (1970) AI in CAI: an artificial intelligence approach to computerassisted instruction. IEEE Transactions on Man-Machine Systems, 11, 190-202 Carroll, J. M. (1990). The Nurnberg Funnel: Designing minimalist instruction for practical computer skill. Cambridge, MA.: MIT Press. Craik, F. I. M., & Tulving, E. (1975a). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104, 268-294. Craik, F. I. M., & Lockhart, R.S. (1975b). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behaviour., 11, 671-684. Cumming, G. (1993). A perspective on learning for intelligent educational systems. Journal of Computer Assisted Learning, 9, pp 229-238. Dewey, J. (1938). Logic, the theory of inquiry. New York: Holt, Rinehart & Winston. Duffy, T.M., Lowyck, J., & Jonassen, D.H. (1992) (Eds.) Designing Environments for Constructive Learning. Heidelberg: Springer-Verlag. Fitts, P. M., & Posner, M. I. (1967). Human Performance. Belmont California: Brooks/ Cole. James, W. (1890). The Principles of Psychology (Vol. 1 and 2). New York: Holt. Kolb, D. A. (1984). Experiential Learning: Experience as a source of learning and development. Englewood Cliffs, New Jersey: Prentice Hall. Kommers, P., Jonassen, D., & Mayes, J.T. (Ed.). (1992). Cognitive Tools for Learning. Heidelberg, FRG: Springer-Verlag. Laurillard, D. (1993). Rethinking University Teaching. A framework for the effective use of educational technology. London: Routledge.

15

Mayes, J. T., Kibby, M.R. and Watson, H. (1988). StrathTutor: the development and evaluation of a learning-by-browsing system on the Macintosh. Computers and Education, 12, 221-229. Mayes, J. T. (1995). Learning Technology and Groundhog Day. In V. B. S. W. Strang, & D. Slater (Ed.), Hypermedia at Work: Practice and Theory in Higher Education. . Canterbury: University of Kent Press. McKendree, J., Reader, W., & Hammond, N. H. (1995). The "homeopathic" fallacy in learning from hypertext. Interactions, 2(3), 74-82. McKendree, J and Mayes, J.T. (1997) The Vicarious Learner: investigating the benefits of observing peer dialogues. Paper presented to CAL '97 Conference, Exeter, UK, March. Miller, G. A., Galanter, E., & Pribram, K. (1960). Plans and the structure of behaviour. New York: Holt. Nielsen, J. (1993) Usability Engineering. London: Academic Press Ohlsson, S. (1993) Impact of cognitive theory on the practice of courseware authoring. Journal of Computer Assisted Learning, 9, 4, 194-221 Phillips, D.C. (1995) The Good, the bad, and the Ugly: The Many Faces of Constructivism. Educational Researcher, 24, 7, 5-12 Rumelhart, D. E., & Norman, D. A. (1978). Accretion, tuning and restructuring: three modes of learning. In J. W. Cotton & R. L. Klatzky (Eds.), Semantic Factors in Cognition . Hillsdale NJ: Erlbaum. Smeaton, C., & Neilson, I. (1995). The Answer Web . Report: Institute for Computer-Based Learning, Heriot-Watt University, Edinburgh http://www.orbital.co.uk/AW/paper.html

16