case-based reasoning in virtual reality: a framework ...

CASE-BASED REASONING IN VIRTUAL REALITY: A FRAMEWORK FOR COMPUTER-BASED TRAINING

Leonardo Rocha de Oliveira B.Eng. (Civil Engineering), M.Sc. (Construction Management)

T.I.M.E Research Institute Department of Surveying University of Salford Salford, UK

Submitted in Partial Fulfilment for the Degree of Doctor of Philosophy

JULY, 1998.

Declaration

This is to certify that this thesis: 1. embodies the results on my own course of study and research; 2. has been composed by myself; 3. has not been submitted as an exercise for a degree at any other university; and 4. has been seen by my supervisor before presentation.

Signature of the Candidate: ……………..…………………

Date: 07/ July/ 1998.

Page ii

Acknowledgements First I would like to thank the Brazilian Government that provided the funding to develop this thesis at the University of Salford through CNPq, one of the National Research Councils. I would also like to express my gratitude to my supervisor, Dr. Ian Watson, for his expert guidance, friendship and for always making me believe that it would be possible to conclude this thesis. His expertise in case-based reasoning and artificial intelligence played a key role in this thesis. “The simpler the better” is something that I now believe and I will certainly take to my future career. My gratitude is extended to my co-supervisor Dr. Arkadi Retik for his expert guidance, friendship and for always making himself available despite the distance between Salford and Glasgow. His expertise in virtual reality played a key role in this thesis. “Not only what you see but what you can get from it” are words that I shall not forget. I owe special thanks to Martin Holden (Manchester City Council), Peter Gordon (ACE Scaffolding) and Mark Pearce (Manchester Scaffolding) for participating in the prototyping process. My appreciation goes also to people such as Carlos Formoso, Luis Fernando Heineck, Carin Schmidt, and Carlos Bonin, from the University Federal of Rio Grande do Sul (UFRGS – BRAZIL) for supporting me to develop this work abroad. The development of this thesis has also involved my personal life and the people from the Surveying Department have provided not only the technical support but also friendship. Ghassan Aouad, Miguel Mateus, Sheila Walker, Claudia de Cesare, Simon Osbaldiston, Antonio Grilo, Jason Underwood, Sandra Heyworth, Ian Hanbridge, Peter Unsworth, Lynn Williamson, Martin Betts, and Vanda Tomlinson are also people whom I will never forget. My special thanks to my family in Brazil that has always been supportive and made me believe that whatever happens in my future, they will always be there to help. My special gratitude goes to my mother, for all the encouragement and moral support. Finally, my special and most sincere thanks to my wife Sylvie, for her love, encouragement, and support reading and correcting the English language.

Page iii

In a vocational evaluation I did prior to joining the University, I went accompanied by a same-age cousin, close friend, and we were asked questions such as: Would you prefer to know a)

how the engine of an aeroplane work to make it fly; or

b)

why some people fear air flights?

This question, for some reason, was the starting point of our further conversation, perhaps because we both had a straight answer: •

I said - why would someone be interested in knowing why people fear air flights when there is so much technological challenge in an aeroplane?

•

She said - why would someone be interested in knowing how those noisy big things work when you live surrounded by people?

Surprisingly, after all those years we are still very good friends!

Tolerance allows people to see the evergreen not only in black and white.

Page iv

Table of Contents CHAPTER 1 – INTRODUCTION___________________________________________________ 1 1.1 – Overview _________________________________________________________________ 1.2 – Research background _____________________________________________________ 1.4 – Hypothesis, aims and objectives ___________________________________________ 1.4.1 – VR as an interface for case representation ______________________________ 1.4.2 – VR and the CBR model of cognition ____________________________________ 1.4.3 – Common objectives____________________________________________________ 1.5 – Research methodology ____________________________________________________ 1.6 – Outline of this dissertation ________________________________________________

1 2 4 4 5 5 6 7

CHAPTER 2 - COMPUTER-BASED TRAINING ____________________________________ 9 2.1 – Overview _________________________________________________________________ 9 2.2 – Computer-based training _________________________________________________ 10 2.3 – The role of training in today’s society______________________________________ 11 2.4 – Training in the construction Industry _____________________________________ 12 2.5 – Training alternatives _____________________________________________________ 13 2.6 – CBT as an alternative solution ____________________________________________ 14 2.6.1 – The cost advantage of CBT____________________________________________ 15 2.6.2 – General advantages of CBT ___________________________________________ 17 2.7 – CBT and human learning_________________________________________________ 18 2.8 – CBT and the dynamic memory theory _____________________________________ 22 2.9 – The dynamic memory theory______________________________________________ 23 2.10 – Learning from cases ____________________________________________________ 27 2.11 – Classroom and case-based instruction___________________________________ 29 2.12 – Synthesis of the chapter ________________________________________________ 30 CHAPTER 3 - ARTIFICIAL INTELLIGENCE AND TRAINING _____________________ 32 3.1 – Overview ________________________________________________________________ 3.2 – The origins of AI _________________________________________________________ 3.3 – AI in education __________________________________________________________ 3.4 – Intelligent tutoring systems_______________________________________________ 3.4.1 – The Knowledge module _______________________________________________ 3.4.2 – The Student module__________________________________________________ 3.4.3 – The Pedagogical module ______________________________________________ 3.5 – A review of ITS applications ______________________________________________ 3.5.1 – SCHOLAR ___________________________________________________________ 3.5.2 – SOPHIE______________________________________________________________ 3.5.3 – WEST________________________________________________________________ 3.5.4 – WHY_________________________________________________________________ 3.5.5 – BUGGY ______________________________________________________________ 3.5.6 – GUIDON _____________________________________________________________ 3.5.7 – CALAT _______________________________________________________________ 3.5.8 – EPITOME ____________________________________________________________ 3.5.9 – A brief recap on ITS __________________________________________________

32 33 35 37 38 38 39 41 41 42 42 43 43 44 44 45 45

Page v

3.6 – Intelligent computer aided training _______________________________________ 3.6.1 – AI training applications_______________________________________________ 3.6.2 – A brief recap on the ICAT applications_________________________________ 3.7 – The limitations and the future in ITS and ICAT ____________________________ 3.7.1 – The limitations _______________________________________________________ 3.7.2 – The future in ITS and ICAT ___________________________________________ 3.7.3 – Instruction and the World-Wide-Web (WWW) __________________________ 3.8 – Synthesis of the chapter__________________________________________________

46 48 51 52 53 54 55 55

CHAPTER 4 - ARTIFICIAL INTELLIGENCE AND CASE-BASED REASONING _____ 58 4.1 – Overview ________________________________________________________________ 4.2 – The origins of CBR _______________________________________________________ 4.3 – An overview of CBR ______________________________________________________ 4.4 – Case representation ______________________________________________________ 4.4.1 – Case acquisition _____________________________________________________ 4.4.2 – Case indexing ________________________________________________________ 4.4.3 – Case Retrieval________________________________________________________ 4.4.4 – Case utilisation ______________________________________________________ 4.4.5 – Case-base maintenance ______________________________________________ 4.5 – The CBR interface________________________________________________________ 4.6 – Review of CBR applications _______________________________________________ 4.6.1 – CLAVIER_____________________________________________________________ 4.6.2 – ARCHIE______________________________________________________________ 4.6.3 – CASEline ____________________________________________________________ 4.6.4 – SMART ______________________________________________________________ 4.6.5 – GIZMO TAPPER ______________________________________________________ 4.6.6 – Brief recap on the applications reviewed_______________________________ 4.7 – Synthesis of the chapter__________________________________________________

58 59 61 63 65 66 69 72 73 75 76 76 77 77 78 79 79 80

CHAPTER 5 – TRAINING WITH CASE-BASED REASONING ______________________ 83 5.1 – Overview ________________________________________________________________ 83 5.2 – Learning from past memories_____________________________________________ 84 5.3 – Case-based instruction___________________________________________________ 85 5.4 – Case-based instructional activities ________________________________________ 87 5.3.1 – ID and ID2___________________________________________________________ 88 5.3.2 – ECAL ________________________________________________________________ 89 5.3.3 – Recap of the applications _____________________________________________ 90 5.5 – Learning from CBR_______________________________________________________ 90 5.5.1 – Discovery learning ___________________________________________________ 91 5.5.2 – Situated learning_____________________________________________________ 92 5.5.3 – Task centred learning ________________________________________________ 93 5.5.4 – Goal driven learning__________________________________________________ 94 5.5.5 – Common characteristics ______________________________________________ 95 5.6 – Review of CBR instructional applications__________________________________ 96 5.6.1 – ASK-TOM ____________________________________________________________ 97 5.6.2 – DUSTIN______________________________________________________________ 98 5.6.3 – CREANIMATE ________________________________________________________ 99 5.6.4 – SPIEL (YELLO) ______________________________________________________ 100 5.6.5 – SCI-ED _____________________________________________________________ 101 5.6.6 – CADI _______________________________________________________________ 101 5.6.7 – A brief recap on the applications reviewed ____________________________ 102 5.7 – Case-based training _____________________________________________________ 103 5.8 – Synthesis of the chapter_________________________________________________ 105

Page vi

CHAPTER 6 – VIRTUAL REALITY CASE REPRESENTATION ___________________108 6.1 – Overview _______________________________________________________________ 108 6.2 – VR: from the labs to the industry ________________________________________ 109 6.3 – VR interface capabilities_________________________________________________ 111 6.3.1 – Interactive VR_______________________________________________________ 112 6.3.2 – Immersive VR _______________________________________________________ 113 6.3.3 – Augmented VR ______________________________________________________ 113 6.3.4 – Networked VR_______________________________________________________ 114 6.3.5 – A recap on the interaction modes.____________________________________ 115 6.4 – VR instruction __________________________________________________________ 118 6.5 – Visualisation and memory recall _________________________________________ 120 6.6 – VR interface for CBR ____________________________________________________ 122 6.6.1 – VR case contents____________________________________________________ 123 6.6.2 – Modelling VR cases__________________________________________________ 125 6.6.3 – Featuring case contents _____________________________________________ 126 6.6.4 – Retrieving VR cases _________________________________________________ 129 6.6.5 – Adapting the VR cases_______________________________________________ 131 6.7 – Synthesis of the chapter_________________________________________________ 133

CHAPTER 7 – CONCEPTUAL VECTRA DESIGN_________________________________136 7.1 – Overview _______________________________________________________________ 136 7.2 – The choice for a methodology ____________________________________________ 137 7.3 – Developing a CBR _______________________________________________________ 138 7.4 – VECTRA design requirements ___________________________________________ 140 7.4.1 – Training requirements_______________________________________________ 141 7.4.2 – CBR instructional capabilities _______________________________________ 143 7.4.3 – VR capabilities ______________________________________________________ 145 7.5 – Instructional activities design____________________________________________ 146 7.5.1 – Accessing learning capabilities_______________________________________ 149 7.5.2 – Media for instructional delivery ______________________________________ 152 7.6 – The VECTRA development methodology __________________________________ 154 7.7 – Synthesis of the chapter_________________________________________________ 157

CHAPTER 8 – THE VECTRA FRAMEWORK ____________________________________159 8.1 – Overview _______________________________________________________________ 159 8.2 – Building the VECTRA framework ________________________________________ 160 8.3 – The CBR–VR integration_________________________________________________ 161 8.3.1 – The choice for the software tools _____________________________________ 163 8.3.2 – VR development tools _______________________________________________ 164 8.3.3 – The choice for Superscape VRT 4 ____________________________________ 166 8.4 – The VECTRA framework _________________________________________________ 168 8.5 – Case memory structure _________________________________________________ 171 8.5.1 – Case featuring ______________________________________________________ 172 8.5.2 – The retrieval mechanism ____________________________________________ 173 8.5.3 – Case adaptation_____________________________________________________ 175 8.6 – Instructional facilities ___________________________________________________ 177 8.7 – Synthesis of the chapter_________________________________________________ 179

Page vii

CHAPTER 9 - THE VECTRA-SI PROTOTYPE ___________________________________181 9.1 – Overview _______________________________________________________________ 181 9.2 – The VECTRA-SI prototype _______________________________________________ 182 9.3 – The task of scaffold inspection ___________________________________________ 184 9.3.1 – Experts approach to scaffold inspection ______________________________ 185 9.3.2 – Experts approach to training ________________________________________ 186 9.3.3 – VECTRA-SI training approach _______________________________________ 187 9.4 – VECTRA-SI case-based instruction ______________________________________ 190 9.4.1 – Case gathering ______________________________________________________ 191 9.4.2 – Case-based instructional strategy ____________________________________ 192 9.4.3 – Case implementation ________________________________________________ 194 9.4.4 – Featuring cases and Scripts _________________________________________ 197 9.4.5 – Retrieval of cases and Scripts ________________________________________ 198 9.4.6 – Case adaptation_____________________________________________________ 199 9.5 – User interface of the VECTRA-SI prototype _______________________________ 201 9.5.1 – The novice interface _________________________________________________ 202 9.5.2 – The intermediate interface ___________________________________________ 203 9.5.3 – The expert interface _________________________________________________ 203 9.6 – Feedback of the experts on the VECTRA-SI_______________________________ 205 9.6.1 – The experts and the VECTRA-SI _____________________________________ 206 9.6.2 – VECTRA-SI and classroom training __________________________________ 207 9.6.3 – VECTRA-SI and descriptions of past experiences _____________________ 209 9.7 – Synthesis of the chapter_________________________________________________ 209 CHAPTER 10 – CONCLUSIONS_________________________________________________212 10.1 – Overview ______________________________________________________________ 212 10.2 – Review ________________________________________________________________ 213 10.3 – Conclusions ___________________________________________________________ 214 10.3.1 – The CBR – VR integration __________________________________________ 214 10.3.2 – VR case representation _____________________________________________ 214 10.3.3 – VECTRA instruction________________________________________________ 215 10.3.4 – The VECTRA prototype _____________________________________________ 216 10.3.5 – The VECTRA framework____________________________________________ 217 10.4 – Recommendations _____________________________________________________ 218 10.4.1 – The CBR model of cognition ________________________________________ 218 10.4.2 – Design of VR cases _________________________________________________ 219 10.4.3 – VECTRA instructional capabilities __________________________________ 219 10.5 – Future research _______________________________________________________ 219 APPENDIX 1 – THE VECTRA-SI INTERFACE ___________________________________221

APPENDIX 2 – CAPABILITIES OF VR WORLD BUILDERS ______________________228

REFERENCES _________________________________________________________________232

Page viii

List of Figures Fig. 2.7a - Accessing long-term memory: adapted from Gagne (1985) and Wingfield (1979). .................. 20 Fig. 2.7b - The stages of learning (adapted from Gagne 1985). ................................................................ 20 Fig. 4.1 - CBR and its main components ................................................................................................. 61 Fig. 4.2 - The CBR process (adapted from Watson, I.D. (1997)). ............................................................. 62 Fig. 5.1 - Achieving an instructional goal (adapted from Gagne (1992)) .................................................. 88 Fig. 6.6.3.a - Contents of a digitised image file.......................................................................................127 Fig. 6.6.3b - Contents of a Superscape VR file....................................................................................127 Fig. 6.6.5 - Object-oriented hierarchical architecture. .............................................................................132 Fig. 7.3 - Development stages of CBR applications. ...............................................................................140 Fig. 7.4 - Decision factors for the appropriateness of CBT......................................................................141 Fig. 7.4.1 - Main elements of decision for the appropriateness of CBT....................................................142 Fig. 7.5 - Designing instructional activities.............................................................................................148 Fig. 7.6 - Development tasks of VECTRA applications. .........................................................................156 Fig. 8.5 - Object-oriented hierarchies in the VECTRA framework. .........................................................171 Fig. 8.5.1 - Structure for featuring MOP and Scripts...............................................................................173 Fig. 8.5.2 - The Retrieval algorithm .......................................................................................................175 Fig. 8.5.3 - Structure for case adaptation. ...............................................................................................176 Fig. 8.6a - Evaluation test for instructional activity.................................................................................178 Fig. 8.6b - Passing parameters for an evaluation object...........................................................................178 Fig. 9.2 - Overview of the VECTRA-SI prototype..................................................................................183 Fig. 9.4.2 - Synchronising sounds and viewpoint movements .................................................................193 Figs. 9.4.3a – 9.4.3d - Pictures of an on-site scaffold structure................................................................195 Fig. 9.4.3e - Case model for further implementation in VR.....................................................................196 Fig. 9.4.4 - Accessing case and Script features. ......................................................................................197 Fig. 9.4.5 - Dataflow diagram of the for case/Script retrieval process......................................................199 Fig. A1.1 - First screen of the VECTRA-SI prototype.............................................................................222 Fig. A1.2 - Options for case/Script retrieval ...........................................................................................223 Fig. A1.3 - Choosing case features.........................................................................................................223 Fig. A1.4 - Retrieval for the case that best match the inputted features....................................................224 Fig. A1.5 - VR case showing a scaffold structure ...................................................................................224 Fig. A1.6 - Overhand of scaffolding boards............................................................................................225 Fig. A1.7 - VR case of a scaffold structure .............................................................................................226 Fig. A1.8 - VR case of a scaffold structure .............................................................................................226 Fig. A1.9 - View from the roof top of a building ....................................................................................227 Fig. A1.10 - VR case of a scaffold structure ...........................................................................................227

Page ix

List of Tables Table 4.4.2 – Features of printers as CBR indexes................................................................................... 67 Table 6.6.4 – Retrieval on visualisation by digitised images and VR.......................................................130 Table 7.5.1 – Influencing learning outcomes. .........................................................................................151 Table 7.5.2 – The design of instructional events.....................................................................................154 Table 8.3.3 – The capabilities of VRT and WTK for building the VR cases. ...........................................167 Table 9.3.3 – Checklist of activities inspecting scaffold components. .....................................................189 Table 9.6.2 – Comparing instructional activities for classroom and VECTRA-SI training. ......................208 Tab. A2.1 – Aspects of reality supported by VR tools ............................................................................230

Page x

List

of Abbreviations

3D ______ Three Dimension AI ______ Artificial Intelligence ASCII ___ American Standard Code for International Interchange CBR ____ Case-Based Reasoning CBT ____ Computer-Based Training DDE ____ Dynamic Data Exchange DLL ____ Dynamic Link Library ICAT ____ Intelligent Computer Aided Training ITS _____ Intelligent Tutoring System KADS ___ Knowledge Acquisition and Design System SCL _____ Superscape Control Language VR ______ Virtual Reality VRML ___ Virtual Reality Modelling Language VRT ____ Superscape Virtual Reality Toolkit WTK ____ Sense8 World Tool Kit WWW ___ World Wide Web

Page xi

Abstract This thesis involves the development of a case-based training framework that holds a repository of past experiences (cases) of domain experts. The cases are represented in Virtual Reality (VR) and contain a real-time 3D simulation of experts performing their job. The VR case representation also includes the guidance these experts would provide when training novices. Users can thus retrieve the VR cases and learn by re-experiencing 3D simulations of on-job activities with expert guidance. This framework involves research in domains such as Case-Based Reasoning (CBR), Computer-Based Training (CBT), and VR. CBR plays its role by providing the foundations for the development of a computer tool that handles a repository of past experiences. CBT contributes with the requirements for instructional strategies in training tools. VR addresses the 3D representation of human memories and the user interface with the instructional activities. The hypothesis behind this work is that this approach can prove useful for training for reasons such as: (i) it uses past experiences to support training that is a natural process of human cognition; (ii) it allows users to learn-by-doing and interacting with the VR interface; and (ii) it provides the advantages of CBT where users can access the training course at the time and pace they wish. The acronym VECTRA stands for Virtual Environment for Case-based TRAining and it is a framework to ease the development of CBR instructional applications. This framework has provided the development of the VECTRA-SI application where on-job experiences of experts in Scaffold Inspection are implemented. This thesis shows that the VECTRA framework provides a tool that can be used for the development of intelligent instructional applications for a range of domains.

Page xii

Chapter 1 – Introduction

1.1 - Overview

This chapter provides an overview of this thesis describing the research background and the reasons that motivated its development. The hypothesis and objectives of this work are also discussed in this chapter and are followed by a description of the research activities involved in the development of this work. The final section provides an outline of this thesis that briefly describes the contents of each chapter.

Page 1


1.2 - Research background Intelligence is the ability to respond successfully to new situations and the capacity to learn from one's past experiences. Gardner, H. (1992)

Artificial Intelligence (AI) is a research field that attempts to create computer systems that emulate human intelligent behaviour (Minsky 1968; Barr 1981; Feigenbaum 1995). This attempt can either regard (i) the development of computer systems that use knowledge models to solve or provide advice for problems that otherwise would require human expertise, or (ii) the study of models of human cognition that allow the representation of knowledge. The former deals with the development of computer systems that emulate human intelligent behaviour when performing a task. These systems hold a body of knowledge of the application domain that a human expert would need to perform the same task (McCordick 1979; Boden 1987). Results of this AI research area cover capabilities related to human intelligent behaviour such as natural language processing, automatic programming, planning, image analysis, decision making and problem solving. The latter research area of AI regards the study of structures and paradigms that emulate the processes associated with human intelligence such as thinking and learning. Research in this area involves the study of human cognition, intelligent behaviour and their representation in computer machines (Schank 1973; Winston 1975; Norman 1975). Results of research in this area are paradigms for knowledge representation such as semantic networks, predicate logic, frames, object-oriented languages and case-based reasoning (CBR). CBR is the AI paradigm focused on in this thesis and it emerged from research in cognitive science where the act of recalling a previous experience is emulated. This act is a common practice in intelligent human behaviour where the remembrance of past experiences supports human reasoning to perform tasks such as problem-solving, learning and decision making (Schank 1982; Riesbeck 1989, Kolodner 1993, Leake 1996). Another aspect of CBR is that even applications that are not originally designed with instructional purposes can provide learning as a ‘side effect’ of using CBR tools (Anderson 1985; Veloso 1992; Kolodner 1993). In CBR, this ‘side effect’ learning is Page 2


achieved by comparing the situation users are facing with similar past cases in the computer, in a process named analogical reasoning (Schank 1988; Burstein 1989; Veloso 1989). Regardless of this ‘side effect’ learning, there are CBR applications specially designed to provide instruction. Their instructional strategy usually relies on discovery learning where users dig into the systems searching for a case that contains the knowledge they want to acquire. More refined instructional strategies involve CBR applications challenging the users’ knowledge by asking them questions before presenting a case with the correct answer. The advantage of this latter instructional strategy stems from its more active instructional approach in comparison to the passive discovery learning. This thesis focuses on the representation of cases and introduces Virtual Reality (VR) both as technique for the visualisation of past experiences and as a technique involved in the whole working cycle of CBR. The VR technology taken in this work uses an object-oriented language to build the VR cases that allows access to the properties of the VR objects. Every object in a VR world has its own attributes that can be accessed and dynamically modified by the developers of the VR cases. This brings new possibilities for the design of CBR applications, combining the dynamic memory theory with an object-oriented programming language that also originates from a model of human cognition (King 1988). The implications that VR technology could have over the whole working cycle of CBR as a paradigm of human cognition motivated this work. VR is also a powerful interface to provide instruction due to its capabilities to simulate reality. Thus, the VR interface’s capabilities to simulate on-job situations and stimulate the use of CBR instructional applications were also key motivations for this work. The VR cases in this thesis are built using an object-oriented language that incorporates the processes of case featuring, retrieval and adaptation of past memories. Differently from digitised multimedia files that require an external description of their contents (see Section 6.6.3), VR cases allow to access the contents of their files. This access gives a new perspective to the CBR model of human cognition that is investigated in this thesis.

Page 3


This work also involves the development of a framework to build instructional applications where VR simulations of on-job past experiences are held in a case repository so that users can retrieve and learn from them. This framework has been used to build a prototype and the application domain regards training in the inspection of health and safety regulations on scaffold structures. Past experiences of experts in this task are modelled in VR and users can retrieve and take their learning from them.

1.4 - Hypothesis, aims and objectives CBR can be seen both as a methodology to build AI systems and as a model of human cognition. The hypothesis behind this work is that VR can play an important role in these two aspects of CBR. VR as a system’s interface allows users to have access to past experiences in a simulation environment as close to reality as computers can currently provide. For the model of cognition, the object-oriented language of the VR tool allows direct access to the contents of the files and the retrieval of individual objects. This language also makes possible the access to distinct pieces of the past experiences held in each case. This thesis has therefore two aims, exploiting issues regarding VR (i) as an interface for case representation and (ii) as a framework capable of holding the CBR model of cognition. The objectives of this thesis regarding each of these two aspects of CBR and those which are common to both of them are described below.

1.4.1 – VR as an interface for case representation Dearden (1995) stated that “the success of any interactive intelligent system is dependent not only on the quality or on the appropriateness of the knowledge encapsulated within the system but also on the quality of the interaction that the system supports”. From this statement, it can be inferred that the interface in CBR plays an important role in the quality of the support provided to users. The user interface is also a major concern for computer-based training applications for reasons such as stimulating users to take the tool, accessing different learning preferences and providing an instructional methodology that complies with the domain considered. An objective of this work is to investigate VR’s capabilities to represent past experiences. More specifically, the objectives regarding the VR interface for CBR are: Page 4


•

to analyse the VR requirements to perform the task of case acquisition for the creation of the VR cases;

•

to investigate the VR capabilities to provide the instructional events and contribute to accessing different learning styles and preferences;

•

to identify the role that the VR cases can play in issues related to computer-based instruction such as the stimulation of users to take the tool, learning effectiveness and simulation of on-job situations

•

to exploit the VR-CBR instructional approach for the specific domain of inspection of scaffold structures.

1.4.2 - VR and the CBR model of cognition The second aim of this thesis is to exploit the capabilities of VR technology to emulate the human process of reasoning that is at the foundations of CBR. The specific objectives regarding this issue are: •

to investigate the role that the access to the contents of the VR files can play over the CBR working cycle and its capabilities for retrieving, featuring, indexing, and adapting cases in the repository;

•

to develop a framework that makes the broad set of the ideas of the dynamic memory theory operational, such as the breaking down of past experiences into small pieces and allowing the featuring and retrieval of each independent piece of memory;

•

to identify the feasibility of creating a framework that allows the development of a shell aiming at speeding up the process of building applications;

1.4.3 - Common objectives This thesis involves objectives that are common to the CBR-VR model of cognition and interface simulating past on-job experiences. These objectives are related to the instructional capabilities of this integration between CBR and VR and are: •

to investigate the amount of work involved in acquiring and representing the VR cases to build CBR instructional applications;

Page 5


•

to evaluate the VR cases in terms of computer hardware requirements such as storage space, processing speed, graphic cards and interaction devices;

•

to determine the programming requirements of a framework allowing to work over the VR cases and coping with the demands of the instructional strategies;

•

to identify the requirements for case featuring and retrieval that best fit the CBR as a model of cognition and the domain of instruction;

1.5 – Research methodology A prototype has been developed as part of this research to further explore the objectives described in the previous section. The VECTRA acronym stands for Virtual Environment for Case-based TRAining. In this thesis the VECTRA acronym will be used to refer to the framework for the development of case-based instructional applications. As a prototype, VECTRA-SI refers to the application domain of Scaffold Inspection that is used to evaluate the hypothesis, aims and objectives of this thesis. The development of the VECTRA prototype is part of the research methodology adopted in the development of this thesis. This methodology follows the development life cycle of information systems and includes a literature review on the main issues involved in this research work. The combination of the methodological requirements from research and information systems development guided the work in this thesis and involved the following stages: •

Literature review – it is the first step and involves a review of issues such as CBT, AI instruction, CBR and VR education;

•

Prototype conceptualisation – involves the decision for the application domain and the identification of domain experts willing to support this research;

•

Choice for computer tools – involves the choice for the hardware and software to develop the VECTRA framework and VECTRA-SI prototype so as to fit the domain aspects;

•

Knowledge acquisition – involves interviews with the experts to gather their onjob past experiences and index the case repository;

Page 6


•

Case analysis – involves the analysis of case contents and the requirements prior to their implementation in the computer tool;

•

Prototype implementation – deals with the implementation of the prototype in the computer and involves tasks such as the design of the cases in the VR world builder in accordance with the domain’s instructional requirements and the CBR working cycle;

•

Prototype verification – takes the feedback of the experts and other people involved with IT and education on the qualitative aspects of the application. The original proposal involved one last stage of prototype validation where the

instructional capabilities of the prototype were to be tested with students and trainees. However, learning evaluation is a complex and time-consuming task that was left for future developments. Further details about the work carried out in each stage of the adopted methodology are discussed further in Chapter 7 and the following section briefly describes the contents of each chapter of this thesis.

1.6 - Outline of this dissertation This dissertation contains two central themes regarding the use of VR in the CBR paradigm that are: the VR capabilities to represent the ideas that conceived the CBR as a model of human cognition and the use of VR as an interface for the representation of past experiences. These two issues have been evaluated in an application prototype that aims at providing training for the inspection of scaffold structures. In order to provide a sequence to explore each of these issues, this dissertation has been organised as follows: Chapter 1 gives an overview of this dissertation and briefly describes the main topics involved. This chapter also sets the issues regarding the research work such as its motivation, hypothesis, objectives and the domain of its application. Chapter 2 discusses training and focuses on the role that CBT can play for organisations and their employees. The case-based instructional approach is then introduced along with a discussion about the human process of learning.

Page 7


Chapter 3 discusses the origins of AI, focusing on instructional applications and reviewing their origins, the development tasks and instructional strategies involved as well as state-of-the-art architectures and applications. Chapter 4 introduces CBR as a technique for the development of AI systems, discusses its working cycle and reviews some state-of-the-art CBR applications. Chapter 5 provides an overview of CBR instruction focusing on the learning strategies it can support and reviews some applications. Towards the end of this chapter, VR is introduced as an alternative interface for instructional applications. Chapter 6 presents an overview of the VR technology and its capabilities to interface with the users and simulate on-job situations. VR is analysed according to its potential for the representation of CBR cases focusing on the design, the internal architecture and the role that the VR cases can play for CBR. Chapter 7 presents the conceptual stage of development of the VECTRA-SI prototype, describing the choice for a methodology and the work carried out prior to its implementation in the computer. Chapter 8 describes the development of the VECTRA framework and its capability to hold interdisciplinary VR case-based instructional applications. Chapter 9 describes the whole development process of a prototype for training in scaffold inspection built in the VECTRA framework. Chapter 10 presents the conclusions drawn from the development of this thesis, provides recommendations for further developments in the VECTRA framework and directions for further research. Appendix 1 presents a training session with the VECTRA prototype. Appendix 2 describes the capabilites of some VR modellers for IBM-PC machines.

Page 8

Chapter 2 - Computer-based training

2.1 - Overview

Computer-based training (CBT) has lately been perceived as an attractive technology for those who sponsor training as much as for those who receive it (Shlechter 1991; Dean 1992; Ravet 1997). However, to take the most from CBT, this chapter will show that a joint effort between employers and the designers of the instructional course is required. This joint effort includes issues such as motivating trainees to take the training, addressing individual learning preferences, providing a instructional methodology and evaluating trainees’ performance at work. The design of the CBT is the focus of the discussion in this chapter that starts by reviewing the state-of-the-art in CBT and its role in professional training. Then, human learning preferences and cognitive aspects of training are reviewed. Next, an alternative methodology for the development of instructional computer tools is discussed. It relies on the dynamic memory theory (Schank 1982): a theory that matches the training requirements of the industry with the human learning process. Finally, the implication of this instructional approach over the development of CBT is reviewed and the synthesis of this chapter is drawn.

Page 9


2.2 – Computer-based training CBT can be defined as an instructional experience between the computer and the learner (Harrison 1990; Shlechter 1991; Dean 1992). The computer provides the stimulus and the learner responds, in an interaction resulting in progress towards increased skills or knowledge. For instance, to enable someone to acquire the knowledge and skills that comprise competence to become an expert in a certain task, such as safety regulations for scaffolding, CBT can be used as an alternative media to provide the instruction. Computer science has been providing software for education since the early 60s, though it is over the last few years that these applications have been receiving greater attention (Gery 1995, Brooks, D.W. 1997; Schank 1997). Changes on the business side of training, where companies require a quickly adaptable and skilled work force (Senge 1994; Schank 1997) are reasons behind the interest for new training alternatives. The interest in CBT can be justified for reasons such as the power of current hardware and software to handle training applications, the developments of hardware devices (joysticks, steering wheels, gloves and head-sets) providing new interface capabilities, multimedia facilities making applications more attractive and useful to users and the increasing availability of computers at home and at the office (Dean 1992; Boschmann 1995; Heinich 1996; Schank 1997; Brooks, D.W. 1997). The availability of hardware at affordable prices and software tools that do not demand highly skilled programmers to develop applications have also contributed to this interest in CBT tools (Cardinale 1994; Reynolds 1996; Tucker 1997). Commercial software for the development of CBT tools (also called as CBT Shells or authoring training tools) reduce the cost and minimise design obstacles for the development of applications (see Section 3.7.1). These Shells also provide facilities allowing checks of how much time has the learner spent on the instructional program and print out reports of a user’s performance for each instructional session, thus helping to determine the effectiveness of the training. CBT Shells usually include a programming language where special routines can be developed to integrate applications to software packages such as databases and spreadsheets.

Page 10


In spite of the growing popularity of CBT as an alternative instructional media, different domains, instructional requirements, and learning preferences, still constitute barriers to the use of this instructional media. Therefore, even if the tool contains a robust body of domain knowledge, it can fail to motivate the learners to take the tool (Schank 1997; Ravet 1997). Reasons behind the possible failure of CBT are varied and include the lack of users’ motivation to improve their skills, the instructional methodology adopted for the domain, the system’s interface that is difficult to work with, the lack of users’ enthusiasm in taking the instruction, and the difficulties in accessing users’ learning preferences. Further details related to the effective implementation of CBT are given later in this chapter and the following section discusses the role that training can play for organisations.

2.3 - The role of training in today’s society The industry depends on the skills of thinking, collaboration, creativity, inquiring, innovation, and endless learning. Every company is a product of its employees' abilities. (Senge, 1990).

As a response to an international dimension of competition in the marketplace, companies of all sectors have come under pressure to offer higher quality and more competitively priced products (Senge 1994, Reed 1994). Enterprises that want to succeed have to keep their work force properly trained and up-to-date with the technological advances and the changes they bring to the skills required to perform the work (Senge 1994; Schank 1997). Today’s professional expertise soon becomes outdated and companies are required to invest in lifelong training programs to keep a skilled work force (Senge 1994; Schank 1997). A survey revealed that in 1990 in the USA already 44% of corporations were willing to spend about one thousand American dollars a year per employee for training (Senge 1990). Thus, companies are aware of this need for training and of the improvements it can bring to their work. Schank (1997) observed that people do not like to spend time learning new skills. In fact, learning new skills are “viewed as necessary evil by management and with disdain by employees” (Schank 1997). Other reasons that have been discouraging training are the cost and time involved with the training courses and the poor Page 11


instructional approaches that often fail to motivate trainees to go through the training courses (Pea 1989). Authors such as Senge (1990), Dean (1992) Lee (1995) and Ravet (1997) show that motivating employees to undertake training is the role of the leaders of a company. The management staff is also responsible for defining the training goals, allocating the resources and deciding the instructional approach that best fits the company’s work force. These professionals have to assure that (i) the instructional approach is consistent with the company’s goals and (ii) these goals are being met by regularly evaluating the work force (Senge 1990; Ishikawa 1991; Jenkins 1996). Further details on the role played by employers, course designers and employees towards training are discussed in this chapter. The following section focuses on training in the construction industry that is the application domain of this work.

2.4 – Training in the construction Industry It is difficult to imagine an industrial sector that could not benefit from training and the construction industry is no exception. Difficulties that are inherent to the domain such as the need to move the work force from site to site, the different local conditions faced at each work place, the differences in each construction project and the ever changing weather conditions make it difficult to establish comparisons of work efficiency and the need for training (Tatum 1988; Strassman 1988; Latham 1994; Prais 1995). In manufacturing industries, a large number of unskilled workers can efficiently operate the machinery when set up under the supervision of a single skilled person (Strassman 1988). This does not apply to the construction industry where the tasks and work are far less sequential and uniform than in the manufacturing industry. Moreover, most of the work in construction is performed individually and under poor supervision (Latham 1994). Due to these characteristics and to difficulties in assessing productivity, the construction industry continues to use an inadequate proportion of skilled labour (Prais 1995). The result is that the low level of efficiency of the work in construction occurs both at a company’s internal level and at the general industry level (Tatum 1988).

Page 12


Another difference between the manufacturing and construction industries is the scale of the competition that they face. In the manufacturing industry the competition occurs on an international scale where the price and quality of the final products quickly addresses inefficiencies (Strassman 1988; Tatum 1988). On the other hand, difficulties in evaluating working inefficiencies and the local scale of competition that the construction industry is exposed to (Lathan 1994; Prais 1995) have been allowing the presence of low productivity, low innovation rate, high price of the final products and high accident rates. Prais (1995) presented a survey where standard examination tests were performed with British and German building craftsmen, allowing for a comparison between the two countries. The evaluation included such tasks as brickwork, carpentry, plastering, roadwork, and painting. Results of this survey showed that the British labour was well behind the Germans. Although questions were raised as to whether influences emerging from the whole educational process in these countries had played a role in the results, the author made clear that the training provided by governmental institutions and construction companies were the main reasons behind the higher German score. Studies like Prais (1995) and Lathan (1994) provide an indication that training is a key factor to compensate for the construction industry’s current inefficiencies.

2.5 - Training alternatives In order to supply training, companies can choose from a range of methods, though the most popular form of training has been instructor-led, with face-to-face contact in the classroom (Milheim 1994; Dean 1992; Tucker1997). Nonetheless, Harasin (1995) shows that there is no evidence to support that this is the best form of training. In fact, authors such as Wells (1990) and Hiltz (1994) show the opposite, where CBT applications have achieved superior training outcomes than classroom training. Under increased pressures from global competition, the effectiveness of hiring professional trainers to improve working skills has been questioned by authors such as Shlechter (1991) and Dean (1992). Instructor-led training as any other training method also presents difficulties and limitations (Gagne 1992). For instance, it is likely that a

Page 13


group of classroom trainees will include individuals with different learning preferences and knowledge background. It may slow down the group’s instruction at various stages of the training sessions as the trainer tries to address these differences individually. The training sessions can thus easily fail to provide employees with equal time privileges or to cater for their own personal learning preferences. The hiring of professional trainers and educational facilities can also prove difficult and entail costly investments for a company without guarantee of a proper return (Lee 1995; Gery 1995; Masie 1995). The effectiveness of other training methods has also come under judgement. Onthe-job guided training, videotapes and reading materials have also become controversial. For instance, videotapes are expensive to develop and edit, and once outdated, a whole new process of filming becomes necessary to update the training. Therefore, there are pros and cons for each of the training media mentioned, as it is further reviewed in Section 9.6.2 where a comparison between these training media is presented. The following section introduces CBT as an alternative training media and discusses aspects such as its cost, its development time and the role that CBT can play as a corporate knowledge for companies.

2.6 - CBT as an alternative solution Training in the industry is labour-intensive, costly and highly dependent on the availability of skilful experts. Although these professionals may have an extensive knowledge of their domains, they may fail to be skilful trainers or may simply not have the time to spend training novices. As a result, private organisations have become open to new training alternatives (Shlechter 1991; Dean 1992; Brooks, D.W. 1997). While CBT was not intended to replace live instructors or teachers, many businesses realised that computers could handle certain training tasks. CBT has come in as an interesting training alternative (Schank 1997). In fact, advances in computer technologies, such as computer networks, databases, interactive multimedia, friendly interfaces, hardware devices, and now VR running on personal computers, have enabled companies to get quality training opportunities with limited budgets.

Page 14


Not much longer than a decade ago, CBT started to gain popularity in the form of tutorials for secretaries and writers on how to use word processing programs (Brooks, D.W. 1997). This kind of applications became popular because they were easier than reading through pages of unfriendly user’s manuals and also presented the advantage of being on-line with the word processor package. These early CBT programs soon branched out into other computer-related training functions including tutorials on database programs and spreadsheets. Nowadays, software applications often do not even provide a user’s manual on paper, as it can be quite tedious to read, expensive to print and edit, and even lacking effectiveness in several domains (Reynolds 1996). On-line help for office tasks such as changing the toner of a printer where the multimedia CBT tool includes animated illustrations associated to text is now standard. An advantage is the real-time on-job training that avoids the hassle of keeping training manuals available to everyone. Advances in computer technology and the use of computers at the office support the growing popularity of CBT. Faster CPUs with powerful graphic interfaces have allowed CBT programs to become highly sophisticated, effective and interesting to use. Interactive interfaces, colourful illustrations, and other graphics interface capabilities have pushed the growth of CBT. As a result, CBT applications are no longer restricted to computer related topics. Today’s CBT market offers a range of training applications and several domains are getting benefits from CBT. CBT is currently available in a number of applications that range from relatively simple topics (such as providing typing skills) to topics as complex as the training of astronauts to perform their job in space (see Section 3.3). The advantages that CBT can offer justify its popularity and the ever-growing number of applications available in the market. Further details on the cost advantage of CBT are discussed in the next section.

2.6.1 - The cost advantage of CBT For a company, the cost factor plays an important role in choosing the type of training to apply (Schank 1997). The cost advantage CBT can provide seems to have played an important part in its growing popularity. However, evaluating the cost of implementing CBT properly is not an easy task (Dean 1992; Ravet 1997). If CBT can

Page 15


represent an expensive initial investment, this investment could soon be “reduced” as the company spreads it over a number of trained employees. Another advantage CBT can offer in terms of cost is that future updates in the training course may be performed at a very low expense, a fact that contrasts with other training methods such as classroom training where the update cost is usually equivalent to the initial investment. An example of the cost-benefit of CBT was discussed in a meeting of the VRT1 users’ group. A company needs to train its 100 employees who are scattered throughout Europe (or even the world) to sell a new product over the next couple of weeks. Different training options were studied and their cost compared. First, hiring a trainer to fly from location to location and present this course material was considered, but the cost entailed by such an option seemed rather high. Another alternative discussed was to bring all the employees together for a seminar. This implied taking into consideration not only the daily expenses of the trainer, but also those of the 100 trainees, not to mention the difficulty of co-ordinating the schedules of all the participants. Therefore, this second option seemed to entail a rather overwhelming task and expensive cost. A third alternative proposed was to take a couple of weeks to develop a CBT course. Once finished and tested, copies (floppy-disk or CD-ROM) could be made and sent to all of the 100 employees with a message indicating that the course must be completed within a week. One point to consider is the initial cost of the development of the CBT course. Nonetheless, this cost also exists for the "traditional" training programs and when this initial cost is spread over 100 distribution copies, this project may turn out to be a relatively easier and less expensive solution as scheduling conflicts and travel costs are avoided. Still considering the CBT alternative, future updated releases of the training package can significantly reduce its cost. Obviously, there are situations when the capabilities of CBT may not fit well with the training task. For instance, the example given above fits well with the update of products such as mobile phones or new releases of products that the salespeople already have the skills to sell. However, a novice that does not yet have the skills to sell any product has different needs that CBT may not so 1

Users’ group of the Superscape virtual reality toolkit.

Page 16


easily be able to address. Therefore, the application domain also plays an important role in deciding whether the CBT alternative will be an appropriate solution.

2.6.2 - General advantages of CBT Several reasons support the use of CBT. Perhaps the most popular is that learners can take the instructions at their own pace, moving onto new stages only when they have mastered the current, and free from any pressure from other learners. With the current multimedia interfaces, CBT can provide an instructional environment that is attractive to users and makes learning fun (Schank 1997), thus reducing the potential for distraction or disruptive classroom behaviour. In spite of the reasons provided above and the power of current software tools facilitating the development of applications, there are other advantages that CBT can provide when compared to classroom training. The following list presents advantages of CBT that have been compiled from a review of the work of authors such as Dean (1992); Cardinale (1994); Gery (1995); Reynolds (1996); Brooks, D.W. (1997); Ravet (1997); and Schank (1997): • CBT can provide instructional events matching individual learning preferences by covering a variety of multimedia instructional deliveries; • CBT can help overcome potential barriers to training such as the instructional level of the training activities that can be too high for some and too low for other students; • learners can start, stop, restart and repeat the training session as they wish, independently from the availability of a tutor, and allowing training for people that have time limitations for traditional courses due to childcare, transportation problems, or scheduling conflicts; • CBT can count on the help of Internet delivery and portable computers, thus reaching people with disabilities or living in remote areas more easily; • CBT constitutes a kind of simulation that learners can use until they feel confident enough to face the real situations; • CBT can simulate on-job activities that are of rare occurrence, too expensive to create real simulations for, potentially dangerous to the learners’ health, or that learners may interfere with and cause damage to; Page 17


• learners do not lose part of the training due to temporary distraction, tiredness or difficulties related to the oral comprehension of the trainer as they can replay and re-read the material until fully comprehended; • CBT programs can include the expertise and teaching experiences of various professionals, thus reducing biases on apprenticeship; • CBT can avoid or diminish the need of often rare and usually busy experts to perform the instructional task; • the contents of the CBT instruction can be updated and include the feedback given by the users; • CBT programs can be installed on private networks, allowing online availability for employees, security for copyrights and other advantages that Intranet/Internet support can provide to companies; • CBT programs can be linked with other training techniques or be part of training courses involving other activities such as classroom and on-job work. Gagne (1992) cited that no computer can offer the same level of personal contact that face-to-face trainer/trainee interaction can provide. Nonetheless, even for domains where the presence of trainers is required, these professionals can use CBT as part of their training courses for such tasks as helping to illustrate their point, providing homework adapted to the students’ background and evaluating apprenticeship. The way different individuals learn and react to the idea of learning from a computer tools is a key factor to the acceptance of CBT. It is essential to have a clear idea of the users’ background and learning preferences prior to choosing CBT as a training alternative. Further details on this issue are discussed in the following section.

2.7 - CBT and human learning Educational methods based on research in cognitive science are the educational equivalents of the polio vaccine and penicillin. Yet, few outside the educational research community are aware of these breakthroughs or understand the research that makes them possible. John T. Bruer, 1994.

Instruction from a book, from a teacher, or from a computer can provide learning that is related to the instructional strategy of these different media and not to the learning source. For instance, learning from past experiences can be provided by Page 18


reading about them from a book, by watching their filmed illustrations or either by reading about them and watching them on a computer screen. Section 2.5 shows that different instructional media can be more adequate to deliver certain types of instructional strategies. In this section, the human learning process is seen as independent from the learning source. Another possible source of misunderstanding is the language used in references on pure cognitive psychology and computer related learning such as CBT and CBR. For instance, when learning occurs by associations between things or experiences, CBR references use the term “learning by analogy”. On the other hand, references in cognitive psychology use instead the term “intellectual skills” for this type of learning (Gagne 1975; 1992). To avoid possible misunderstanding with the jargon used in the dissertation, a glossary is provided at the end of this work. Preference is given, though, to the jargon used in references related to the model of cognition behind CBR. Even the words training and learning are sometimes misused in references. Learning and training are two distinct activities and must be addressed differently. The Concise Oxford Dictionary presents training “as the act or process of teaching or learning a skill” and learning as “the act, process or experience of acquiring knowledge by study” (reading books, observing someone developing a task, attending a training course or simply reasoning based on one’s own mental process). Training is an instructional process that aims at acquiring skills to carry out a specific task. In terms of computer applications, Dean (1992) have defined CBT “as a tool to help people learn to do something previously beyond their capabilities”. On the other hand, learning is an individual process that differs from one individual to another. Thus, individuals attending the same training course may achieve different levels of learning. Klatzky (1980), Anderson (1985), and Gagne (1985) cited that learning occurs by transferring information from the short-term to the long-term memory (Figure 2.7a), taking place as an individual response to a stimulus from the external environment. On its way from the short to the long term memory, learning requires people to think about it, relate it to other things they know, question it, and transform it into their own words. The authors also cited that the result of learning is a permanent change in the learner’s mind. Page 19


Instructional methodology

Human memory Training Course

Instructional events

Short-term memory

Instructional media

Long-term memory

Feedback

Fig. 2.7a - Accessing long-term memory: adapted from Gagne (1985) and Wingfield (1979).

Authors such as Kolb (1984), Wingfield (1979) and Gagne (1975) have decomposed learning in sub-processes that occur in the human mind. For instance, Wingfield (1979) cited that the three major stages of learning are (i) input, (ii) storage and (iii) retrieval. Kolb (1984) describes learning as a four-step process that is (i) perceiving information, (ii) reflecting on how it will impact an aspect of our life, (iii) comparing how it fits into our own experiences, and (iv) thinking about how this information offers new ways for us to act. Gagne (1975) has gone further in decomposing the learning process into nine stages that were reduced to six events in later publications (Gagne 1992). Figure 2.7b shows the reviewed version of the author and indicates the sequence of the occurrence of these events and the processes associated with them. Further details on the work of Gagne (1992) are given in Section 5.4 that describes two CBT applications relying on his theories of learning capabilities.

- response - retrieval

Stages

- storage - understanding - coding - attention - perception - expectancy - stimulus

Motivation

Performance

Recall

Retention

Acquisition

Apprehension

Time

Fig. 2.7b - The stages of learning (adapted from Gagne 1985).

Page 20


Gagne (1985) explains the role played by each stage in human learning and how to properly achieve them. A brief description of each learning stage, as given by the author, is shown below. •

The Motivation stage - it establishes expectancy in the learners. They anticipate the reward that they will obtain when the learning goal is achieved.

•

The Apprehension stage - it concerns catching the learners’ attention, a process initiated with the stimulus caused by the first stage but a process that may not last long if the training does not appeal to the learners. It is also important to note that different individuals have a different perception of the stimulus and therefore a different response to it because of their different learning preferences.

•

The Acquisition Stage - it refers to the stage where the learners transform the original information into neural information. This information is first stored into the short-term memory and whether it gets through to the long-term memory will depend on the effectiveness of the learning.

•

The Retention stage - it deals with the storage of the information acquired into the learners’ long-term memory. This is the stage of learning the scientific community has the least knowledge of. Some aspects of this stage are known, such as the fading with the passage of time, the possibility of new memories replacing older ones and the gathering of different aspects of the same experience by different individuals. Others aspects, such as the limit of the capacity of the long-term memory, how to access it effectively and how long it will keep an experience for, are still a mystery.

•

The Recall stage - it is the stage that allows the learners to apply the knowledge they have gained in one context to other situations, by retrieving the memories they have stored. Although the retrieval appears to be most effective when close to the time of the learning, cues can also help the retrieval process (i.e. recall can be induced by different means than the one used to transmit the information). Thus, effective instruction needs to provide the learner with the means to trigger and make resurface the relevant information that has been stored.

•

The Performance stage - it refers to the stage where the learners respond to a situation using the knowledge they have stored, thus proving that learning has Page 21


actually occurred. This is the stage when learners perceive they have achieved the goal set in the first stage and hence close the “learning loop”. Questions related to the number of times the learners should be tested to prove that learning has actually occurred and the length of time this new learning will stay in the learners’ mind remain unanswered. The breaking down of the human learning process helps to identify the role that employers, employees and designers play throughout the implementation of CBT. Another important factor is to recognise that the means to reach each stage of learning, like motivation for instance, vary from individual to individual. Learning preferences are also influenced by age, background and other personal characteristics that are proper to each individual. Although it seems difficult for CBT to cope with all these individual differences, there are instructional approaches that have been producing satisfactory results. One of these instructional approaches is the case-based instruction that was introduced by Schank (1982, 1995; 1997) and is further discussed in the next section.

2.8 – CBT and the dynamic memory theory Improvement in memory rests almost entirely on improvement in techniques of learning. First, it is important to attend to the material; second, to give it organisation; and third, to rehearse the material as much as possible. Wingfield (1979)

Previous sections of this chapter have shown that CBT is an alternative method of instruction and like any other instructional alternative, its effectiveness faces barriers imposed by individual learning preferences. Studies in cognitive psychology, as shown in the preceding section where the learning process is broken down into instructional stages, can also be seen as efforts towards providing instruction that access each stage properly. Another area of cognitive psychology that has been contributing to the design of instruction deals with studies of human memory and cognition. One of the results of the studies conducted in this area is the dynamic memory theory (Schank 1982; 1995; 1996) that proposes a model where human memory is seen as a repository of past experiences. Intelligence that helps learning is related to the act of remembering past experiences and the knowledge stored in memory helps processing new situations.

Page 22


Learning is thus seen as a dynamic process where new experiences re-align with the pasts, modifying the original memory structure. Theories of human cognition linked to the sequence of learning show that the effectiveness of CBT requires a joint effort between the companies where the trainees work and the designers of the training tool. The companies play a role at the initial and final stages of the learning process. At the initial stage, by motivating the trainees and showing that the learning will be rewarded. At the final stages, by giving trainees the opportunity to use their new skills, evaluating their performances and showing the results of the improved skills. The design of the instructional tools plays its role at the intermediate stages of the learning process. This role concerns the providing of applications containing a sound body of domain knowledge, enabling users to understand the instructional events, catching trainees’ attention and interest, and accessing learners’ long-term memory. The dynamic memory theory provides an approach to cope with these instructional difficulties. The three major reasons behind the use of this theory in this work are: 1. it represents a theory of learning for both computers and people and the CBT tool that relies on this theory can improve itself as much as the users; 2. it provides an instructional methodology that relies on the gathering and storage of real on-job experiences that users can retrieve and take their learning from; and 3. it supports instruction by accessing memories of real on-job past experiences that is a natural form of human learning on the journey from novice to expert. This work proposes an alternative instructional approach that the designers of training tools can follow. This approach is founded on the dynamic memory theory (Schank 1982) that is further described in the following section.

2.9 - The dynamic memory theory Learning means the dynamic modification of memory. Schank (1995)

The dynamic memory theory (Schank 1982) is a model of human cognition. This model represents intelligent behaviour that involves the collection, use and Page 23


modification of past experiences in human memory. CBT applications relying on this model thus access a natural form of human learning where past experiences are modelled and stored in a case repository. Users can retrieve those past experiences and take their learning from them. (Schank 1996). The origins of the dynamic memory theory could go back to the work of Bartlett (1932). The author worked on the conditions of human learning and on the nature of the errors that seemed common in memory recall, based on the meaning and understanding of learned materials. Bartlett (1932) cited that memory could be seen as “nothing more than a collection of anecdotes whose true accuracy and validity could be as dubious as that of our wayward witness”. In his experiments, Bartlett (1932) had people reading short stories and then tested their recall by asking them to re-tell these stories at various time intervals. These experiments led the author to believe that memory had a dynamic reconstructive aspect, influenced by the individuals’ understanding of what they had learned. From this study, the author elaborated a theory whose central theme was the reconstructive nature of memory where newly acquired information is mapped onto a pre-existing memory structure that the author called schemata. Schemata were thus described as a dynamic structure of concepts as they change and are made more complete by the acquisition of new information (Bartlett 1932). Another aspect of the schemata was that they were individually unique as two persons experiencing the same event will later reproduce similar recall “only to the extent that their schemata are similar or at least allow for equivalent mapping” (Bartlett 1932). The author went further by citing that “when a person reproduces meaningful material exactly as it was first experienced, this is more a happy coincidence of a valid transformation than evidence that no transformation of the input has occurred”. Another effort of relevance for both studies of human cognition and this thesis was the experiment of Bower (1969). This author gave two groups of volunteers, with a similar knowledge background, twelve lists of ten words to memorise. One group studied the lists as an exercise of memorisation of the words. The second group was instructed to invent stories including all the words on each list. Both groups were given the same amount of time to perform the learning of each ten-word list. The difference

Page 24


was dramatic, since each member of the first group only recalled an average of seventeen words while the members of the second group recalled about hundred out of the original one hundred twenty words. The author concluded that schemata were fragmentary and that a process of reconstruction could help recall. Tulving (1972) also presents a theory of cognition where memory is classified as episodic and semantic. The episodic memory relates to concrete experiences that have a sequence in time or space. The recall of these memories can be triggered by a similar sequence of events that composed these previous experiences. Examples of this type of memory are found in tasks where someone has a sequence of tasks to perform until certain equipment becomes operational. Another classic example is when going to a restaurant, the customer expects a sequence such as finding a table, choosing from the menu, ordering the meal, eating, paying and leaving. Semantic memories are abstract, individual, do not follow a timed sequence and usually involve a conceptual representation of the world. An example of semantic memory is the use of words and language where there is no unique sequence in choosing the words that will communicate well. Another aspect of semantic memory is the influence of someone’s culture when receiving the words. For instance, for some people the word snow may invoke the image of a beautiful place covered in white where one can ski and have fun. For others snow may be associated to cold weather and to the difficulties it imposes to people who cannot travel freely. Perhaps the first author to provide a structure for the human cognition was Norman (1975) who cited that knowledge is structured in the form of an interconnected semantic net containing ideas and concepts. Learning occurs when people acquire new information and integrate it into their existing knowledge structure. Learning thus means an alteration in the knowledge network where either a new structure is added or the actual structure is modified to cope with the new knowledge (Norman 1975). Schank’s (1982) dynamic memory theory is an approach that brings together the work of all these authors. The premise of his theory is that “remembering, understanding, experiencing and learning cannot be separated from each other” (Schank 1982). The dynamism of memory comes from the changes it goes through as a result of new learning or new experiences. For instance, when learning a new method of performing an task such as inserting paper in a new printer model, old experiences in Page 25


performing this task can be recalled and provide expectations that will drive the learning. The dynamic memory theory was first described by Schank (1982). Since then, this theory has been evolving and the parts that compose its structure have also been changing. For the sake of brevity, this work will not discuss the evolution of this theory along with the work of Schank (1982; 1996). It will only present the core structure of the theory that was presented in Schank (1996). This structure is described below. •

Memories Organisation Packets (MOP) – it holds both a general description of a past experience and its organisation i.e. it gives the sequence of the events that constitute this experience. An example of MOP given by Schank (1995) is the experience of visiting a doctor that includes the whole sequence of events such as booking an appointment, checking in with the nurse, reading a magazine in the waiting room while waiting to be called, and finally seeing the doctor. A MOP is thus a representation of a series of events that lead to the achievement of a goal (in this case the goal is consulting a doctor).

•

Scripts – are the specific situations contained in a MOP and can also be seen as “a set of expectations about what will happen in a given situation” (Schank 1996). Each individual event of the “visiting a doctor MOP” is considered as a Script in the dynamic memory theory. Sometimes an MOP such as visiting a doctor and visiting a dentist can have a series of Scripts that are common to both MOPs. Thus, recalling one of these MOPs and its sequence of scripts can help define the expectations for the sequence of scripts of the other.

•

Meta-MOP – are structures that work as a template organising the MOP. MetaMOP are at the top of the hierarchy in the dynamic memory theory and deal with general goals such as learning a skill where a series of MOPs are involved. The value of the dynamic memory theory for CBT lies both in its being an

instructional activity that accesses a model of cognition and in its providing an architecture that allows the development of computer applications. The dynamic memory theory is at the origins of the CBR methodology for the development of applications (see Chapter 4) and is also at the foundations of the case-based instructional approach proposed in this thesis.

Page 26


The case-based instructional approach emulates a situation where learners with a problem describe their situation to the system that retrieves a similar past experience and presents it to the learner. Providing the descriptions that properly address the contents of each case in the repository is thus an important requirement. However, this requirement is not exclusive to the case-based instructional approach but is common to any application relying on CBR. These issues that concern the requirements of general CBR applications are discussed in Section 4.4. Further details on the learning that can be accessed from the case-based instructional methodology are given in the next section.

2.10 - Learning from cases The way memory is organised has great importance for theories of learning. Schank (1995).

Case-based instruction is a form of education where tutors, lead by the questions of students, tell a relevant story (or a past experience) and allow students to figure out the answer based on the example given. This educational approach is as old as human education (Schank 1995) and was used by teachers such as Salomon, Jesus Christ, Buddha and Plato. Case-based instruction is a natural human form of education that is also part of common dialogue where people tell past experiences that are relevant to the point they are trying to make in their conversation. Teaching from cases is at the core of the work of Schank (1995; 1996; 1997) and his research group at the Institute for the Learning Sciences (Northwestern University). They explore a case-based teaching architecture where the cases are represented as digitised films of experts telling stories about their experiences. Schank (1990) cited that this architecture “exploits the basic capacity of students to learn from stories and the basic capacity of teachers to tell stories that are indicative of their experiences”. The case-based teaching architecture used at the Institute for the Learning Sciences is similar to the instructional approach in this thesis. Cases are represented containing simulations of experts’ past experiences. Rather than the recording of an expert telling a story, this work represents the cases in a virtual environment simulating the location where the experience took place and the on-job actions of the

Page 27


experts. The instructional approach of this work aims at training and will be referred to as case-based instruction. Shank’s (1997) case-based teaching and the case-based instruction adopted in this thesis share common aspects related to their effectiveness for instruction. For instance, both approaches are based on the premise that learning is best taken in functional contexts and with similarities to real situations (either described or simulated) rather than from bare facts (see Section 5.2). Moreover, students can acquire knowledge in real time with the problem-situation that they encounter. When the need for learning comes, the tools are available to present a similar experience that can provide the learning. Other instructional aspects that are common to these two works are: •

the cases must provide instructions that are relevant to understand the domain and the instructional goal of the application;

•

the cases must be designed to avoid users’ misinterpretation of their contents;

•

the cases must be attractive in order to capture and maintain users’ interest;

•

the application requires a comprehensive set of cases covering the domain to be instructed;

•

the application must retrieve cases that are relevant and applicable to the learners’ request; and

•

the cases must be presented in such a form that helps learners draw out useful generalisations. Another aspect discussed in Section 5.3 is that expertise is built upon a rich set

of experiences and learning is acquired in relation to previous knowledge. Therefore, even if the case retrieved is not relevant to the situation users are facing, it could bring knowledge background to users, thus speeding up future learning. The following section discusses other advantages that come from the case-based instructional approach, focusing on its differences to classroom training.

Page 28


2.11 - Classroom and case-based instruction A review of the literature related to instructional approaches such as computerbased training, classroom training and on-job supervised training leads to the conclusion that all forms of instruction have their strengths and weaknesses. Section 2.6 shows a list of advantages of CBT as an instructional alternative. This section discusses classroom and case-based instruction from the cognitive point of view that each takes on providing learning. An issue raised by Merril (1993) and Schank (1995) is that instructor led training courses are tutor-oriented rather than trainee-oriented. Tutors try to provide students with as much information as they can during the time available for the course. Trainees are faced with an amount of information that, though previously organised by the tutor, follows an instructional approach that fits the tutor’s viewpoint in instructing the subject rather than the trainees’ experiences and difficulties in performing their jobs. Instructors are thus presenting information that, though relevant to the domain, answers to questions that the trainees have not asked and perhaps providing solutions to problems that the trainees have not had. Therefore, instead of having their own experiences to recall, alter and learn from, students are left with the memories of the instructors providing information on how to proceed. Authors such as Twining (1991), Dean (1992), Weller (1994); Schank (1995) and Ravet (1997) recognise that learning would be more effective if the trainees could make their own mistakes, acknowledge their own failures and learn from them. Those authors also suggest that the ideal situation is to have an instructor looking over the trainees’ shoulder when doubts emerge. However, companies cannot afford having neither employees making mistakes nor instructors permanently watching over each employee. CBT tools relying on the case-based instruction can provide help at the time the trainees are facing a situation that requires skills they do not have. For instance, in the domain of scaffold inspection, the case-based instruction will be most useful when users have to inspect a component that they have never inspected before. They will then be able to use the case-based instructional tool and retrieve a case where the component is inspected. Page 29


There is no evidence that the instructor led training is less appropriate or effective than case-based instruction. It is evident, though, that trainees relying on the former instructional approach are dealing with an instruction where someone tells them what to do whereas the latter makes them find out what has previously been done, reason and compare it to the situation they are facing. From the cognitive psychology viewpoint, case-based instruction is therefore more in accordance with the theories of cognition of Bartlett (1932), Bower (1969), Norman (1975) and Shank (1982).

2.12 - Synthesis of the chapter CBT provides an instructional experience that aims at enhancing levels of performance. Reasons behind the growing interest for CBT are its competitive cost in comparison to other training alternatives, the potential of current hardware and software to provide training applications, and the increasing use of computers for professional and home tasks. However, to take the most from CBT, a joint effort between the employers and the designers of the CBT applications is required. The design of CBT is the focus of this chapter that introduces case-based instruction. Cases are past experiences of experts performing their job and are modelled and kept in a repository. This approach has its foundations in studies of human cognition, where ”much of human reasoning is case-based and people constantly experience such reminding, comparing one experience to another so as to learn from both” (Schank 1982). This model of human cognition also asserts that past experiences in memory are broken down into pieces that can be recalled and altered independently. For instance, in the domain of scaffold structures, experts usually have their own way of inspecting health and safety regulations. Each time they are faced with a new type of scaffold, they bring past inspections from their memory and check the components that are relevant. If there is a new component or a different structure configuration, this new piece of inspection can be added into the memory repository of the experts, thus improving their experience and skills. The memory storage of this new case can even change in the future if another expert gives instructions regarding the inspection of new components and how the inspections should be performed. Once the implications that the new component brings

Page 30


to the scaffold inspection are memorised, a new experience is added to memory. For future similar inspections, the expert will be able to retrieve this experience from memory and rely on its contents to perform new inspections. This work models the cases in VR, simulating the physical space where the experience and the actions of the experts took place. Users can thus retrieve the cases and take their learning from them. Further details on the working cycle of this instructional approach are shown in Chapter 4 that describes CBR as a technique for the development of AI applications.

Page 31

Chapter 3 - Artificial intelligence and training

3.1 - Overview

The previous chapter discusses CBT as an alternative form of training and introduces a theory of human cognition as both an instructional approach and as a methodology for the development of CBT applications. This model of human cognition corresponds to an effort in artificial intelligence (AI) where past experiences of experts performing their jobs are represented in the computer. Users can retrieve these past experiences and learn from them. This chapter provides an overview of the concepts of AI, describing its current state-of-the-art in instructional applications. Previous research work in AI covering both paradigms for knowledge representation and computer systems that embody human instructional performances is reviewed. This chapter finishes describing applications of AI instruction, highlighting their strengths and weaknesses.

Page 32


3.2 - The origins of AI AI is not the study of computers, but of intelligence in thought and action. Computers are its tools, because its theories are expressed as computer programs that enable machines to do things that would require intelligence if done by people. Boden (1987) - preface to first edition

AI is a research field that associates computer science and intelligent behaviour, involving interdisciplinary areas such as cognitive psychology, paradigms for computer knowledge representation and the design of systems that attempt to emulate aspects of human intelligent behaviour (Barr 1981, Boden 1987; Schank 1990). Research in AI has led to the development of computer systems using knowledge models to solve (or provide support for) problems that otherwise would require human expertise (Rich 1983; Partridge 1990). The origins of AI are associated with a combination of intellectual efforts in research areas such as the evolutionary behaviour of living organisms, theories of language, mathematical logic and studies of cognitive psychology modelling aspects of human memory and reasoning (Barr 1981; Schank 1990; Partridge 1990). Studies in human cognition that are at the foundations of AI are described in Chapter 2. Studies in mathematical logic and symbolic deduction carried out by authors such as Whitehead (1925), Church∗ (1996), Tarski∗ (1995), Turing∗ (1992) and Kleene∗ (1971) helped the formalisation of logical reasoning and intelligence that led to the birth of AI programming (Barr 1981). Barr (1981) cited that Turing∗ (1992) could be considered the father of AI. His work on mathematical theories applied to both modelling of patterns in living organisms and non-numerical computation behaving as models of intelligence, are the foundations of AI. The work of Turing∗ (1992) in symbolic processing and his “universal machine” capable of executing describable algorithms has contributed to the creation of computer machines accepting programming languages. Right after the creation of programmable computers appeared software packages dealing with tasks associated with human intelligence such as solving puzzles, playing chess and translating texts from one language to another. The work of

∗ The original work of these authors could not be found and instead compilations of their work are presented in the references of this dissertation.

Page 33


researchers such as Feigenbaum (1995), Minsky (1968), Simon (1969) and Newell (1963) on the semantics of information processing and the programming techniques supporting aspects of human intelligence also helped establishing the foundations of AI. McCarthy (1969) first introduced the term AI at a conference at Dartmouth College in 1956 (Partridge 1990). The participants of this conference, such as Minsky (1968) who founded the first AI lab at MIT, Shannon (1956) from Bell Labs, Newell (1957) who became the first president of the American Association of AI, and Simon (1969) who won a Nobel prize working at the Carnegie Mellon University, can be considered the AI pioneers (Partridge 1990). Forsyth (1990) cited that the 1980s were the golden age of AI, when academic and commercial institutions from all over the world became involved with AI research and developments. For instance, in 1981 the Japanese Ministry of Trade and Industry announced its interest in projects involving machines capable of learning and communicating in natural language. In 1983 the UK launched the ALVEY programme of advanced information technology with a budget of £ 350 million, stimulating research and development in AI in the country. It was also in the 1980s that the Europeans invested 1600 million ECUs over a period of five years in the ESPRIT program, promoting European co-operation and the establishment of standards for AI developments. Results from research in the 1980s include software tools (Shells) for the development of intelligent systems, standard methodologies for AI developments and the emergence (and re-emergence) of AI techniques such as neural networks, fuzzy systems and genetic algorithms. Architectures and Shells for the development of AI instructional applications and the CBR paradigm also emerged from the 1980s research. Despite the achievements of AI in the 1980s, fundamental issues related to the complex nature of AI developments are still to be cleared (Hickman 1992; Russel, S.J. 1995; Bailey 1997). For instance, Partridge (1990) cited that the major discovery of AI research was that the “phenomenon of intelligence is astonishing complicated to be represented in computer machines”. Moreover, authors such as Rich (1991), Kolodner (1993), Schank (1994), Feigenbaum (1995) and Russel, S.J. (1995) cited that results of Page 34


AI research in the 1980’s lead to the conclusion that AI applications were too expensive for the benefits they provided. Current AI research, rather than only studying human intelligence applied to computer sciences, also involves studies of computation to understand human intelligence (Barr 1981). Computer hardware evolution has also contributed to the level of performance of current intelligent systems (McCordick 1979). Work in AI currently covers a range of domains and applications and thus cannot be defined in terms of its results but as a research area that links computers and human intelligent behaviour (Rich 1991; Schank 1994, Russel, S.J. 1995). AI involves hardware, software and programming techniques emulating human mental and physical processes, such as thinking, reasoning, learning, vision, and motor skills (Boden 1989; Tuthill 1990). Applications of AI cover interdisciplinary research in areas such as knowledge modelling, natural language processing, computer vision and intelligent computer-aided instruction. Further details on the current accomplishments of AI presenting a special interest for intelligent computer-based instruction are discussed next.

3.3 - AI in education The previous section shows that AI is closely related to cognitive psychology and human intelligent behaviour. For instance, Schank (1990) cited that AI systems contain characteristics associated to human intelligence such as a body of knowledge of the application domain, inference mechanisms to interpret the significance of the knowledge stored, the capacity to draw conclusions and to improve its capabilities by learning from new inputs of information. In spite of the limitations of AI applications comparing to the human process of cognition, learning for both the user and the system has always been an issue in intelligent systems. AI has much to do with knowledge and learning is a “side effect” of AI applications designed to provide problem-solving strategies (Anderson 1985; Johnson, W.L. 1985). AI involves both machine learning and user learning. This work will only focus on the latter and thus deal with applications of AI techniques providing instructional activities.

Page 35


Early applications of AI instruction focused on creating a domain body of knowledge and on carrying out dialogues capable of providing instruction and diagnosing user’s mistakes (Carbonnel 1970; Wexler 1970). Efforts in this area included the geography tutor SCHOLAR (Carbonnel 1970a) and the electronic troubleshooting tutor SOPHIE (Brown 1974). These applications hold a repository of domain knowledge allowing these systems to be responsive in a range of problem-solving interactions. Further details on these applications are discussed in Sections 3.5.1 and 3.5.2. Researchers such as Laubsch (1975) and Brown (1977) claimed that AI systems, even when containing comprehensive domain knowledge, were holding neither expert skills to teach the application domain nor a model of the students’ learning capabilities. AI applications designed to replicate the reasoning of experts could thus be inadequate to emulate the actions of an expert tutor addressing individual students’ requirements (Brown 1977; Clansey 1990). Two main research topics related to AI research in education thus emerged. The first was the teaching methods, i.e. the procedures, principles and techniques that the systems should incorporate to encourage and ease learning. The second was the evaluation of individuals’ learning preferences, capabilities and knowledge background thus allowing the development of applications to help learners acquire knowledge from the instructions. Current research in AI instruction take into consideration the elements of design of applications that are sensitive to the user’s strengths, weaknesses and preferential learning styles so as to provide instructional strategies capable of coping with these individuals differences (Barr 1981; Wenger 1987; Sleeman 1982). As a new step in computer education, AI instruction has also brought new perspectives for CBT, both for the design of applications encapsulating the experts’ knowledge and for the techniques used to teach learners. Almost three decades have passed since the first publication of an application of AI concepts to education (Carbonell 1970) was found and an agreement on the vocabulary of AI education has yet to be reached. Instructional computer systems involving AI have been conducted under the acronyms of Intelligent Computer Aided Instruction (ICAT), Intelligent Tutoring Systems (ITS) and Intelligent Computer Aided Training (ICAT). This work uses ITS for general instructional applications and ICAT Page 36


for applications that specifically address the task of training. Further details of the vocabulary used in this thesis are available in the Glossary and the following section describes the state-of-the-art on ITS.

3.4 - Intelligent tutoring systems The main differences between ITS and earlier CAI systems do not reflect teaching methods and underpinning philosophies of learning. Rather, they reflect engineering and psychological enhancements that permit ITS to tutor in a knowledge-based fashion. Burns. (1991)

Early applications of AI in education employed a procedural representation of domain knowledge and users relied on the systems to take the information they wanted. The idea was to provide a comprehensive body of domain knowledge that could be used to answer questions involving causal or relational reasoning (Carbonell 1973; Laubsch 1975). These systems thus provided instructions in a form of discovery learning where the apprenticeship did not include any form of teaching expertise (Brown 1977; Sleeman 1982). ITS was first introduced in the mid 1970s from the work of Brown (1974) and Burton (1982) at Bolton Beraneck and Newman Inc. and focused on identifying tutoring strategies and diagnosing students’ behaviour. The first application involving these concepts was WEST (see Section 3.5.3), a children’s board game that exercised a guided discovery learning, identifying gaps in user’s prior domain knowledge. The work at Bolton Beraneck and Newman Inc. introduced the current modular architecture of ITS. This architecture encapsulates the knowledge related to the tutoring of a specific task or domain (Knowledge Module), addresses pedagogical questions related to the teaching faculty (Pedagogic Module) and tracks information about the learner (Student Module). Researchers such as Wenger (1987), Polson (1988), Larkin (1992) and Merri (1992) claim that this architecture provides a versatile tool for instructional activities since developers can work independently in each module. The state-of-the-art in ITS applications indicates that, due to the complexity involved in instructions, none of the three component modules is fully implemented yet in current ITS. None of these independent modules has reached a stage that does not require further research (Major 1992; Murray 1996; Stern 1996). Moreover, studies from Schank (1995) and Piaget (1982; 1985) have questioned the role of the Pedagogic

Page 37


module in ITS, citing that instructional applications should focus on challenging user knowledge and providing learning-by-doing activities (see Section 5.5.5). Further details on the anatomy and objectives of the ITS architecture are discussed next.

3.4.1 - The Knowledge module This component works as a knowledge repository of the domain containing the relevant information to be taught. The development of this module involves the modelling of the domain knowledge and its implementation into the application. The domain knowledge includes facts, concepts, procedures and the heuristics that experts use when performing their work (Wenger 1987; Larkin 1992). The design of this module usually requires a knowledge engineer to implement the domain knowledge into the application, thus making the module’s contents accessible to the other modules of the system. (Wenger 1987; Psotka 1988; Stern 1996). The Knowledge module contains a repository of the domain knowledge that is to be communicated to the student. It contains descriptions of the concepts and skills of an expert in a particular domain and a model that can perform the domain tasks, hence holding a corporate source of expertise. The Knowledge module is thus not only a repository of information, but also a model of how experts skilled in a particular domain perform their tasks (Carbonell 1970). Other functions attributed to this module are: • providing a transparent interface, allowing learners to access the procedural representation of domain knowledge; • dealing with the enquiries as similarly as possible to the relational reasoning process used by human experts; • containing knowledge regarding both correct and incorrect learners’ responses, thus providing a picture of the consequences of inadequate decisions.

3.4.2 - The Student module The Student module deals with information regarding the students’ understanding of the domain knowledge and their performances over the apprenticeship. The simplest learner module tracks an individual student’s performance over the instruction, avoiding repetitions of the lessons where the student

Page 38


performed well. More sophisticated modules include a record of the learner's performance, the identification of individual learning preferences and knowledge background. The purpose of the Student module is to provide information for the Pedagogical module of the system, allowing it to decide which instructional strategy needs to be adopted in order to cope with the students’ learning capabilities and preferences. The Student module works as a dynamic evaluator of the student’s performance over the domain, thus allowing the system to make pedagogical decisions. Other tasks that this module is expected to perform are: • tracking the state of learners’ knowledge in comparison with the contents of the Knowledge module, thus allowing evaluation of system’s usefulness for each user; • providing learners with the reasons behind their failures; • diagnosing learning efforts, identifying areas where the user repeatedly fails and why, and if the system proves to be incapable of providing the support required, suggesting the users to first overcome their lack of background knowledge; • inferring predictions about a particular learner’s performance in accordance to his/ her preferential learning styles and capabilities; • ability to deal with the presence of noise in the user’s records and its effects on the accuracy of the evaluation of the user’s performance; • capability to diagnose the learner’s performance even during the learning session, providing active and passive observations.

3.4.3 - The Pedagogical module Teaching is a skill that requires knowledge additional to the knowledge comprising mastery of the subject domain. Brown (1977)

This module has also been called Tutoring or Instructional module and can be seen as an independent AI system. It contains its own knowledge-base and inference mechanisms handling expertise related to cognitive preferences and teaching skills. This module thus plays the role of an expert tutor of the application domain teaching individual students. Page 39


The Pedagogical module controls the interaction with the learner using its own knowledge of teaching to perform the instructional process. Its reasoning process begins with receiving information from the Student module regarding the needs of individual learners. Based on this information, this module decides for the best approach to provide learners with the domain knowledge contained into the Knowledge module. The Pedagogical module is hence responsible for tasks such as determining when and how to present a new topic, and revising previous instructional events. In spite of the importance of this module, authors such as Wenger (1987), Clancey (1990) and Winkels (1990) acknowledged that it has not been researched as much as the others. Perhaps one of the major reasons behind the lack of research in this field is the fact that instructional methodologies rely heavily on the skills of experienced teachers and that it is difficult to model this knowledge (Gagne 1985; Piaget 1985; Dean 1992; Gardner 1993). Early developments in the Pedagogical module explored instructional methodologies diagnosing students’ understanding by giving them tasks to perform and then evaluating their responses (Brown 1974; Koffman 1975). The module was capable of determining, from the students’ feedback, the skills or areas that needed improvement and thus reinforced the corresponding subject. Current modules are capable of monitoring students’ performance, acting at “the right time with the right knowledge” (Burton 1982; Wenger 1887; Schank 1995) and encouraging students to reason by identifying gaps in their own knowledge (Brown 1989; Stern 1996; Suthers 1992; Schank 1997). Although there has been promising work in this area, Suthers (1992) shows that developments in the Pedagogical module still have a long way to go. For instance, its potential to adapt to individual learners and to improve the pedagogical strategy over time has yet to be fulfilled and this represents an important subject for further research. Other tasks attributed to the Pedagogical module are: • identifying user errors and guiding the path to a correct solution, presenting explanations in a way that is understood by the learner; • adapting to user needs, providing instruction and tests at the correct level of difficulty considering the learner’s prior performances;

Page 40


• when unable to answer learner’s enquiries, providing the reasons for this inability; • adjusting the instructional performance based upon the user’s response to the domain knowledge; • taking control of the learning activity, establishing the sequence of instructional events and following a didactic methodology; • allow learners to have the initiative in asking questions, without letting them take full control of the instructional activity.

3.5 - A review of ITS applications This section presents a chronological review of eight ITS applications and aims at allowing readers to follow the evaluation of the ITS architecture and capabilities. Readers are also expected to recognise that technological advances in computer-based instruction lead to a direction that has made the instructional approach adopted in this work (see Section 7.5) a natural evolution from the concepts of AI education.

3.5.1 - SCHOLAR This application was developed at Bolt Beranek and Newman Inc. and was a pioneer application of AI developed exclusively for instructional purposes (Carbonell 1970). The knowledge-base contains information about South American geography represented in a semantic net of concepts and properties programmed in LISP. Special emphasis in this system is given to the pedagogical approach and the nature of the instructional dialogue. The system provides instruction through question/answer sessions that are picked at random, in a format of a Socratic dialogue, where the system is constantly challenging students’ skills. The pedagogical strategy relies on finding an interesting way of making students memorise the instructions. The system is capable of keeping track of the topics discussed and allows students to change the direction of the discussion. This pedagogical approach was later improved becoming capable of making inferences in the knowledge-base to answer students’ questions (Collins, 1978). These inferences rely on keyword recognition from the dialogues and on emulating a conversation in natural language. This application has been tested with children in classroom and the authors cited that the children were quite enthusiastic with using the system but no evaluation of the system’s instructional capabilities in comparison with the traditional classroom tutoring was performed. Page 41


3.5.2 - SOPHIE This acronym stands for SOPHisticated Instructional Environment and its application domain was the debugging of electronic circuits (Brown 1974). The system was developed at Bolt Beranek and Newman Inc. and was the first ITS to provide a simulation environment where the students had to examine a malfunctioning piece of equipment taking measurements such as voltage, current and resistance. The instructions were provided following an approach of discovery where, based on the results of the measurements, the students could make observations about possible reasons for the faults and come up with an hypothesis. SOPHIE would then use its domain knowledge to decide if the hypothesis was correct, if the inferences the student made were reasonable and what the possible diagnoses were, thus emulating the presence of an expert in the lab. The system’s instructional capabilities rely almost exclusively on the knowledge-base since the pedagogical approach was only providing a faulty circuit to start with and keeping track of the simulations already displayed. The emphasis of this system was on exploring students’ initiative in problem-solving skills rather then providing general domain knowledge. This learning-by-doing approach was considered an effective instructional strategy and the systems was updated a few times and used for a number of years (Brown 1977; Burton 1982).

3.5.3 - WEST This application was the first ITS to be used as a game and was developed at Bolt Beranek and Newman Inc. (Burton 1982. The application domain was arithmetic, and the knowledge-base was holding information about constructing an arithmetic expression using the operations of addition, subtraction, multiplication and division. The pedagogical approach in this application worked as “looking over the students’ shoulder” during the game and deciding when pedagogical interruption would be profitable. The application involved diagnosing the students’ misunderstanding based on their actions during the game and providing the instruction at the right time without interrupting the students too often and thus without ruining the fun of the game. The system was also monitoring the players' moves, their scores and listing the possible moves that they could take next. The system was thus capable of determining if the students' moves were optimal, identifying the factors that influenced the players’ selection of moves and presenting the moves that players missed according to the opportunity used. The system was put to use in elementary-school classrooms and Page 42


compared to a non-coached version of the system. The evaluation showed that the coached version resulted in a greater learning and fun for the users than the noncoached version (Burton 1982).

3.5.4 - WHY WHY followed the SCHOLAR and was also developed at Bolt Beranek and Newman Inc. providing instruction regarding meteorology forecast (Collins 1978). The knowledge-base contained information regarding the causes for rainfall, such as heuristics rules related to common-knowledge in rainfall forecast and the geophysical processes that influence the weather. The application had a special emphasis on its pedagogical activity, providing an interaction based on Socratic dialogue of questionand-answer sessions. The system included the instructional strategies of domain tutors, offering guidance for the dialogues and diagnosing the students' misconceptions based on their incorrect inferences and statements. Students were thus not expected to learn facts about the domain but to understand a model of the domain and the relationship between the factors capable of influencing rainfall.

3.5.5 - BUGGY This system was designed as a complement to WEST and focused on diagnosing the mistakes that mathematics students may make while trying to solve arithmetic problems and explaining the reasons behind these mistakes (Brown 1977). Instead of the traditional focus on the knowledge-base, BUGGY worked on the pedagogical skill needed to expose and remedy student mistakes. This application concentrated on developing a diagnostic model that simulated a student with erroneous thinking. The system was built by modelling the expertise of experienced math tutors on the most common mistakes students’ make. The author cited that students of basic arithmetic exhibit consistent buggy behaviour and the application relied on creating a repository of possible mistakes. The knowledge-base of the application also held general diagnostic tests to identify students' misconceptions and pedagogical approaches to fix the problems. Based on the diagnosis of the students’ common mistakes, the system would provide an instructional approach capable of correcting specific errors. Therefore, rather than modelling the domain knowledge, the system approached instruction based on expected failures.

Page 43


3.5.6 - GUIDON This application was the first ITS to be built on top of an existing AI system (Clancey 1982). GUIDON used the MYCIN knowledge-base that was one of the pioneer AI applications and provided medical diagnosis on infectious diseases. GUIDON was designed to choose a diagnosis from MYCIN and guide the student through the rules that the system used to make its decision. The guidance would assist and critique the students according to their performances in comparison with the system’s diagnosis. The Pedagogical module was held separately from the knowledge-base and followed an instructional approach that guided the dialogue, summarised evidence, and tracked student’s understanding. GUIDON was the first application that proposed the current architecture of ITS as a combination of three independent modules with different purposes. The instructional premise was that the rules in MYCIN could be taken to provide instruction. The authors later concluded that students found rules difficult to understand, remember, incorporate in a global problem-solving approach and did not easily provide users with an organised hierarchy of hypotheses (Clancey 1982).

3.5.7 - CALAT CALAT has been reported as under development at the NTT Information and Communication Laboratories and learners can access the system via the Internet (Cherhal 1994). This tool keeps track of the students' needs and presents the appropriate instruction, consisting of text, images, animation and audio. The capabilities of this tool include access to the domain knowledge and information such as on-line libraries, museums and heuristics kept at the NTT headquarters’ server. Currently, the project is dealing with the obstacles posed by using the Internet, such as enabling users to personalise the interaction, controlling what information users can access and allowing the use of any Internet browser. This system relies on the ITS architecture of three independent modules and the Student module keeps track of users achievements. The Pedagogical module provides users with assistance and guidance to navigate the sites that will match their interests, educational goals, and competence in the material. Educational issues regarding the instructional capabilities of the Internet are being raised with the development of this project, one of them being that the freedom to access the instructions may lead students to check the answer page while still on the question page. The CALAT is still under development at the NTT

Page 44


Information and Communication Laboratories and no evaluation of user learning performances or of the usefulness of the system has been provided.

3.5.8 - EPITOME This acronym stands for Engineering Platform for Intelligent TutOrs and Multimedia Experiences. EPITOME is still under development at the School of Civil and Environmental Engineering at the Georgia Institute of Technology (Baker 1996). This project proposes the creation of an ITS Shell that requires the input of the domain knowledge model incorporating multimedia, since the Pedagogical and Student model are independent and available from the ITS Shell. This ITS Shell is to be used for the creation of instructional applications in different engineering domains relying on the standard modularity of ITS. The authors claim that engineering education requires instructional methods that are standard for experienced tutors. These instructional methods, incorporated in the Pedagogical module will allow the applications to go beyond the standard of current systems, allowing learners the choice of different instructional styles. The developers claim that the instructional techniques used in engineering classrooms by experienced tutors are a key factor to provide models for the instructional strategies in the Pedagogical module.

3.5.9 - A brief recap on ITS There is evidence that ITS can be effective and increase learner performance and motivation. Work carried out by Shute (1989) shows, for instance, that students using an ITS for Economics performed equally well to students taking a traditional Economics course, but requiring half as much time to cover the material. ITS research also highlights that if computers can somehow be endowed with human instructional behaviour, emulating tasks such as understanding and reasoning from learners’ attitudes, the result would closely resemble the interaction between an expert instructor and a student. However, the field of artificial intelligence-based pedagogy has still a long way to go in emulating the skills of experienced tutors. The architecture of ITS includes the modelling and implementation of domain knowledge model and this is a task that usually requires knowledge engineers to code domain information into computer paradigms for knowledge representation. Moreover, ITS include a model of expertise in instructional delivery strategies that not only

Page 45


presents its own difficulties related to the unstructured nature of pedagogical expertise, but also adds the difficulties related to learners’ individual preferences, motivation and capabilities in following the instruction. Although the ITS architecture may seem a comprehensive and powerful approach to provide education, it represents a complex task for ITS developers. Clancey (1982) cited that classroom and computer-based instruction are different processes that have the goal of striving to improve student learning in common. The ITS tools reviewed above are more focused on the search for didactic strategies for the Pedagogical module rather than on evaluating student performance strengths and weaknesses in the Student module. However, authors such as Schank (1997) and Piaget (1985) cited that the student is the key for effective instruction. Examples of this interest for instructional strategies that focus on the student’s stimulus to learn are the Guided Discovery Tutoring approach (Elsom-Cook 1990), the Case-Based Teaching approach (Shank 1990), the Hypermedia Browsing systems (Weller 1994) and the Knowledge Negotiation approach (Moyse 1992a). These instructional methodologies do not include tutors’ expertise in instructional strategies but work by challenging students to search for the information they need. With these methodologies, learning takes place as a “side effect” of performing a task (learning-bydoing) or answering to a system’s challenge. This focus on the learner prevails in training applications where the emphasis lies on simulations of on-job activities and evaluation of user performance. Further details on ICAT and a description of applications are provided in the following section.

3.6 - Intelligent computer aided training Intelligent Computer-Aided Training (ICAT) applications, similarly to general AI applications and ITS, emulate the knowledge and performance of an expert trainer in a well defined and narrow domain. The key issue behind the ICAT technology is to provide a skill beyond the trainees’ current capabilities in a more advantageous manner than other training alternatives (see Section 2.6). To achieve this objective, ICAT systems rely on the domain knowledge model, the ability to provide an individual training performance and emulating the capability of an expert trainer delivering instruction.

Page 46


ICAT systems provide much of the same experience than ITS, only the emphasis lies on the improvement that could be achieved over an on-job performance. The tasks performed by ICAT include providing training for domain skills, monitoring and evaluating the actions of a trainee, providing meaningful comments in response to the trainee’s mistakes, answering requests for information and storing the strengths and weaknesses displayed by the trainee so that appropriate future exercises can be designed. Researchers such as Loftin (1989), Burns (1991), and Chappell (1995) recognise that, in spite of the similarities and results that can be taken from ITS research, training differs from teaching or tutoring. Training involves skills over on-job performances, thus implying differences between ICAL and ITS, i.e. ICAT applications involve a simulation model of the job activities. The differences between ICAT and ITS are reflected over the development of ICAT applications, as these systems have to cope with aspects such as: • motivation: trainees are aware of their responsibilities and the consequences of their failures, sometimes even involving the safety of human lives; • background: trainees may already have an academic background and practical experience that will bear implications on their training performances; • basis for comparison: trainees often have experiences with the hassle of training from manuals and are aware of the cost involved with on-job training under the supervision of more experienced personnel; • flexibility: professional tasks usually offer considerable freedom in the way that they may be accomplished; • timing: the best time to have the support from a training tool is in real time on-job situations when the new skills are required; • pedagogy: more important than memorising a sequence of activities to perform the work, trainees need to understand their working process and the consequences of alternative ways of doing it. Unfortunately, this ultimate training tool is still far away from the ICAT applications available today (Boman 1992; Cardinale 1994; Eliot 1995; Shank 1997). Page 47


However, new technologies are bringing computers closer to work environments each day. For instance, VR technology has brought the capability to create simulated models to represent the work environment. The following section describes some ICAT applications that present special interest for this thesis.

3.6.1 - AI training applications Two groups are developing similar applications that are relevant to this research project. The first group is at the Institute for the Learning Sciences at Northwestern University (Schank 1991; 1995; 1997) and created a number of academic and commercial instructional applications. These tools rely on a case-based teaching architecture, where on-job past experiences of professionals are modelled in digitised video clips that can be retrieved by trainees. The cases are displayed in a story telling format and contain business practices and actual past examples of professional stories. The second group is located at NASA’s Johnson Space Center (Loftin 1987; 1988; 1989) and involves a project called Intelligent Computer-Aided Training and Virtual Reality (ICAT-VR) technology. This project uses an integration of ICAT with VR for a number of applications, mostly related to the training of astronauts, flight controllers and in space simulations. The ICAT-VR integration from NASA’s Johnson Space Center was introduced in the late 1980’s when VR technology was still expensive (see Section 6.2). Nonetheless, the budgets allocated to such space projects probably justified the costs of using VR. The emphasis of the project was placed over the simulation of on-job experiences. Another project has recently begun at the National Science Foundation and aims at the development of a Shell capable of using the ICAT-VR integration framework and then transferring this technology to the government, the industry and education. The case-based teaching architecture from the Institute of the Learning Sciences lays the emphasis on the cognitive aspects of instruction. The applications aim at stimulating trainees to use the tool rather than performing on-job simulations. Further details on the case-based teaching architecture and a description of applications are discussed in the next chapter. The following sections describe four tools developed at the NASA/Johnson and two other ICAT applications relying on the ITS architecture.

Page 48


The description of these applications provides an idea of the current state-of-the-art in ICAT.

3.6.1.1 - PD - Payload-assist Deploys PD - ICAT is an application to train NASA flight controllers in performing the computations and other operations necessary for satellite deployment from the space shuttle (Loftin 1987). This system was the first to involve the ICAT-VR technology and has served as a test-bed for the development of a general framework for the ICAT-VR framework used for further applications. Although the cost of VR representation of the actions required on job was high, especially at the time the system was developed, it was minimal considering the price of the equipment to be taken to space. This application was also positive due to the personnel turnover and frequent introduction of new technologies and systems. This application does not follow the ITS architecture and the pedagogical approach is provided in the form of a tutor guiding the user’s interaction with the system. Users either succeed or fail the simulation and in the latter case, the system provides the reasons for the failure. The simulations create a range of situations that trainees could face in real life and keeps track of the strengths and weaknesses of each trainee so that future appropriate exercises can be resimulated and special exercises for further instruction designed.

3.6.1.2 - MPP - Main Propulsion Pneumatics MPP - ICAT is an application to help engineers achieve the skills required to perform testing of the space shuttle’s main propulsion pneumatics system at NASAKennedy Space Centre (Loftin 1988; 1989; 1989a). This system utilises the ICAT-VR framework proposed in PD/ICAT and the VR environment simulates the propulsion system and the equipment used to test it. Trainees follow the operations and maintenance instructions pertinent to the pneumatics system that controls the space shuttle propulsion system. In addition to training engineers in nominal test procedures, the application intends to address the development and implementation of test procedures employed when faults are detected. Similarly to the previous application, this system does not contain a Pedagogical or Student module. The emphasis of this tool lies on the virtualisation of the situations that professionals may face in on-job situations. The cost of VR in this application is proportionately minimal when considering the price and danger involved in failures of the real propulsion system. Page 49


3.6.1.3 - IPS - Instrument Pointing System IPS - ICAT is an application for the training of astronauts in the operation of the IPS in space (Loftin 1988a). The IPS device is a platform used for mounting and pointing astronomical telescopes during the Space-lab missions. The function of the IPS is to provide precision pointing and tracking capabilities for telescopes in space, establishing an inertial stable base from which observations can be made. The training tool provides a simulation of the operation of the pointing equipment, displaying a VR representation of the vision from the space shuttle to the equipment. Trainees can thus access interactive VR simulation of the control panels as well as the displays used for the operation of the IPS in space. This ICAT is designed to train astronauts in the activation, deactivation, and initial pointing of the IPS as well as in the final pointing of the telescope in space. The system uses the ICAT-VR framework and focuses on the simulation of on-job situations that professionals will have to deal with. The pedagogical approach followed by the Pedagogical module emulates the presence of a tutor guiding the work of the trainees. The application does not have a Student module.

3.6.1.4 - CISCO - Centre Information System Computer Operations CISCO - ICAT addresses the training of mainframe computer operators of the Johnson Space Centre Information System (Loftin 1989b). Operators are provided, in a PC and Windows environment, with a console operator display as well as a "map" of the hardware locations. From the VR simulated displays of the computer network and hardware equipment, trainees are instructed in standard operations of computer maintenance, including power-up, power-down, control of user accesses, replacing hardware devices and fixing network problems. CISCO also uses the ICAT-VR framework that has been developed by NASA’s Johnson Space Center but running for the first time on the local computer network and under the hardware and software limitations of personal computers. AI is used to model the guidance of experts operators and their knowledge of fixing network problems. The system does not provide any evaluation of trainees’ learning but simply a simulation of on-job activities.

3.6.1.5 - AEGIS CIC ITS AEGIS CIC ITS was designed by the Stottler Henke Associates Inc. (Richard 1993) to be used by the US Navy. The AEGIS ITS includes a series of applications Page 50


designed to train crew members in a military Combat Information Center (CIC) for positions such as tactical action officer (who is the person that controls the CIC) and weapons director, helping them understand the tactical situations they are likely to face. The applications include the operation of equipment for the long-range detection of submarines and the targeting weapon systems. These systems rely on the notion that the best way to learn is by multimedia animations of tactical simulations. Adhering to the cognitive aspect of learning from experiences, each simulation describes a problem, the strategies and steps necessary to find a solution, and the consequences of the actions taken. Other capabilities of this system includes testing the students' learning from the simulations. The focus of this application is the evaluation of the user’s capabilities to learn from simulations. The authors have neither provided an evaluation of user’s learning nor details on the usefulness of the tools compared to other alternative forms of instruction.

3.6.1.6 – SHERLOCK This application was developed by XYZ Corporation and used by the U.S. Air Force for the training of technicians diagnosing electronic faults on the F-15 avionics test station (Lesgold 1992). The diagnoses of electronic faults consist of taking electrical measurements and figuring out possible faults. The tool provides a simulation of the equipment that the technicians work with using a combination of digital video-clips and computer graphic displays. The simulation involves the equipment controls that can be operated with the mouse and the display changes reflecting the response of the meters attached to the device test points. The instructions rely on a learning-by doing approach where the system challenges the user’s skills to solve a simulated fault and provides advice when the trainees reach an impasse. The system also keeps track of the trainees’ actions and if they fail on the diagnosis, an expert solution is available to coach trainees through each step of the process. There is also an option for a check-list guidance of an expert solution reviewing the trainees' recent efforts with the diagnosis. The authors claim that the system has worked “remarkably well in terms of fostering high levels of job expertise and promoting transfer to new electronics troubleshooting tasks on novel equipment” (Lesgold 1992).

3.6.2 - A brief recap on the ICAT applications Schank (1995) cited that the most effective way to achieve a skill is by allowing trainees to carry out an assigned task hence performing a learning-by-doing activity

Page 51


(Shank 1995). Loftin (1989) goes further by citing that training could be even more effective when learners are provided with an instructional environment where they can perform the training by any valid means, learning also from their mistakes. Providing simulations of the work activity that users can experience with thus seems to be the major area of concern for ICAT designers. As mentioned in Section 2.5, training and teaching (or tutoring) present differences. Training is related to working performances and tutoring to academic and educational environments. This difference reflects in the design of ICAT applications that emphasise the instruction of the skills to perform a job. To satisfy this working related approach, ICAT designers seem to prefer to develop applications focusing on the simulations of on-job activities rather than on pedagogical strategies or the modelling of a domain body of knowledge. The applications discussed above and a literature review on ICAT reveal that ICAT designers strive for the simulation of the work environment with the training tools. VR and its capabilities to simulate work situations can play an important role for training applications. Nonetheless, the Pedagogical module is usually limited to the guidance of an expert instructing the best working practices possible and the Student module to evaluating trainees’ performances.

3.7 - The limitations and the future in ITS and ICAT Wenger (1987) and Russel, D. (1988) cited that CBT is a maturing technology that “offers an unprecedented avenue for the delivery of experiential education to students of all age levels and for all disciplines” (Shute 1995). Objectives of instructional applications include reducing the cost, increasing the quality and the flexibility of the courses, allowing users to take them whenever they want or need. Research and development in VR technology has been stimulated by its usefulness in domains where 3D real time simulation is required, such as industrial design (Orr 1989), architecture (Brooks, F.P. 1988), medicine (Fuchs 1989), telematics and teleoperation (Loftin 1989; McKendree 1990). Instructional applications involving VR offer the benefits that this technology can bring to instruction, but the cost-barrier has slowed its use in commercial applications (see Chapter 6). Nonetheless, the cost of VR having recently declined, commercial VR modelling products running on personal

Page 52


computers are now affordable by small and medium sized companies. The following sub-sections present more details on the current limitations and future in computerbased instruction.

3.7.1 - The limitations The development of ITS and ICAT systems is a time consuming and costly task that usually involves a team including computer programmers, domain experts, and educational theorists (Dean 1992; Davies 1996, Brooks, F.P. 1997). For instance, Murray (1996) estimates that one hundred hours of development time translates into one hour of instruction. ITS-ICAT Shells or authoring training tools can play an important role by reducing the time and cost of developing applications. These tools: (i) provide a friendly package where educators can build their own courseware; (ii) supply facilities to represent the domain knowledge; (iii) enhance teaching strategies to cope with individual learning differences; and (iv) provide facilities to keep track of user’s activities and learning performance. ICAT Shells such as Challenger (Desktop Training Systems Inc.), IDE (Russell, D. 1988) and EON (Murray 1996) provide facilities to cope with multimedia representations of the domain knowledge. Instructional activities in these Shells are limited to the modelling of the guidance of an expert tutor included as a part of the instructional events. Facilities for the Student module include passwords for registering trainee records, on demand progress reports, test analysis to evaluate trainees’ performances and bookmarks to restart the training at the last exit point. Wenger (1987) expressed that ITS Shells have proven capable to speed up the development process of ICAT applications. However, the word “intelligent” in ITS and ICAT is limited to the capturing of the domain knowledge and the instructional sequence from experienced instructors. The instructional capabilities related to the personal experience of a human instructor in these Shells “are still far behind those of experienced tutors” (Wenger 1987). A common instructional approach adopted by training tools has been identified by Schank (1990) as “the page-turning architecture” that works by displaying a screen of information and waiting for the users to move onto the next page or to answer a Page 53


question. Based on the user’s answer, the system decides whether to go further with a new page of instructions or to give additional instruction regarding the failed topic. Instruction from this page-turning approach, as cited by Schank (1990) provides learning out of memorisation. Current ITS and ICAT tools hardly provide a stimulus for users to use the tool to perform the apprenticeship. Authors such as Weller (1994), Wessel (1996) and Brooks, F.P. (1997) lead readers to believe that the novelty of multimedia features can stimulate users to use the tool. However, learning and having fun while taking an instruction are two different things. Activities such as playing a game can be fun, yet instructionally poor (Towne 1992; Schnotz 1993).

3.7.2 - The future in ITS and ICAT Current research involving instructional applications has focused on improving the modules of the ITS architecture described in Section 3.4. On the other hand, ICAT applications are more focused on the simulation of the working environment and on the modelling of the guidance of experts performing the job. The interface between the instructional tool and the student has been an area drawing particular attention. Friendly interfaces imply lowering the level of user’s computer expertise to use the training tool. Rich interfaces also mean that the training can be delivered over various multimedia and access a wider range of learning preferences. The interface is the component part of ITS and ICAT where VR technology has the potential to play a major role. VR interactions with the simulated environment could bring users to become active participants in instructional courses (see Section 6.4). Other AI techniques, including neural networks, genetic algorithms and fuzzy systems, where the domain knowledge and inferences are not easily accessed, may offer benefits to model trainee’s behaviour (Klir 1988; Lea 1990). This claim is based on the assumption that human behaviour and knowledge levels can seldom be represented as a set of fully understood practices. Therefore, these AI techniques, where the knowledge encapsulated is not transparent for the users, may help both the Pedagogical and Student module, encapsulating learner behaviour and enhancing a realistic model of what an individual knows, has learnt or has done (Klir 1988; Lea 1990).

Page 54


3.7.3 – Instruction and the World-Wide-Web (WWW) WWW instruction seems to be less expensive for institutions than conventional instruction in many areas. Wow! That is a very scary thought. Preface of Brooks, F.P. (1997)

Over the last five years, the WWW interface has changed from a styled text display to tools capable of presenting animated graphical images, sounds, and digitised films. The Virtual Reality Modelling Language (VRML) has also been allowing the creation of interactive VR sites. These technological advances mean for ITS and ICAT an improvement in the capabilities related to the delivery of applications. The WWW is already the number one media for data transfer (Brooks, F.P. 1997) and it holds an incredible potential to reach people, even at remote areas. Furthermore, it is difficult to imagine a domain where teachers, trainers and students could not benefit from its use. There are still many difficulties to overcome prior to reaching the conclusion that WWW instruction will end up substituting traditional classroom education. However, new technologies such as video-conferencing and fast ISDN transmissions are bringing every day more applications, greater reliability and new users to the WWW information media.

3.8 - Synthesis of the chapter For almost as long as computers have existed, work has been conducted in AI and its benefits for education. The objectives of the work in AI instruction include the development of computer systems capable of holding domain knowledge, emulating pedagogical behaviour of expert tutors and providing instruction that satisfies individual learning preferences (Bork 1986). After more than four decades of AI research, the current state-of-the-art in ICAT and ITS is still modest compared to the capabilities of a professional tutor. Wenger (1987) cited that the main issue related to the role of the word intelligence in instructional systems is still the creation of the knowledge-base, so as to compose the instructional interaction. Instead of decisions and inferences in the knowledge encapsulated in applications, it is the domain knowledge itself that is represented so that it can be used in computer systems.

Page 55


In addition to the continuing work on these aspects of intelligent behaviour, one important research area is reducing the time and cost to develop such systems. ICAT and ITS Shells, where instructors with modest computer background can develop their own applications, have been seen as offering potential to reduce time and cost of building instructional tools (Milheim 1994). Despite the similarities between instruction and training, there are differences that reflect over the development of ICAT and ITS applications. For instance, the ITS applications reviewed in this chapter focused on modelling human educators and their ability to use a variety of instructional strategies performing their work. On the other hand, the ICAT applications reviewed in this chapter focused on the simulations of onjob situations where trainees can learn-by-doing. VR simulations form part of the ICAT reviewed and in spite of its cost, VR has been useful to represent the level of performance expected in the real workplace. The cost of VR is now decreasing and other computer technological advances such as multimedia, ICAT Shells and the WWW are also helping to increase the capabilities of ICAT applications and are reducing the time and cost necessary to their development.

Page 56

Chapter 4 - Artificial intelligence and case-based reasoning

4.1 - Overview

Chapter 3 reviews AI instruction describing its origins, the tasks involved with the development of applications and the state-of-the-art in current technologies. This review shows that AI instruction deals with three main issues that are (i) the principles of instructional design, (ii) the knowledge models of the skill to be acquired and (iii) the access to student’s performances and learning preferences. This chapter discusses Case-Based Reasoning (CBR) from its origins as a model of human cognition to its evolution as a methodology for the development of AI systems. The dynamic memory theory discussed in Chapter 2 is reviewed focusing on how this model of human cognition has been adapted to allow the development of CBR applications. Previous CBR instructional applications are also reviewed, highlighting the internal architecture and techniques for case representation, indexing, retrieval and adaptation.

Page 58


4.2 – The origins of CBR Intelligence is the ability to respond successfully to new situations and the capacity to learn from one's past experiences. Gardner, H. (1992)

CBR as an AI paradigm for knowledge representation is based on studies of human cognition (Riesbeck 1989; Slade 1991; Barletta 1991). Unfortunately, literature on the origins of CBR prior to the work of Schank (1977; 1979; 1982; 1999) and his research group at the University of Yale is scarce and fragmented (Gutierrez 1997). The studies of cognitive psychology and human learning carried out by Norman (1975; 1983; 1988; 1990) and Rumelhart (1977) and their research group at the University of California – San Diego seem to have played a major influence on the work of Schank (1982). This research group proposed a model for human learning dividing it into a three-stage process that consists of (i) accretion, (ii) restructuring, and (iii) tuning. The premise to this learning theory is that knowledge is structured in the learner's mind in the form of an interconnected web of ideas and concepts. Learning occurs when people acquire new information (accretion) and integrate that information into their existing knowledge structure (restructuring). Any increase or change in their knowledge causes effects over the structure and the process of tuning is activated to ensure that this operation is properly performed (Norman 1975). The work of Schank and his research group at the University of Yale was inspired by Norman’s theory of human cognition. Their dynamic memory theory (Schank 1982) provided the theoretical background to CBR. The central issue they raised was that the act of remembering a previous situation (seen as an episodic memory or case) is a common practice in intelligent human behaviour for situations such as problem-solving, decision-making and learning. The first CBR application reported was CYRUS (Schank 1979; Kolodner 1980), which contained a repository of the travels and meetings of the ex US Secretary of State Cyrus Vance. CYRUS was followed by applications such as MEDIATOR (Simpson, R.L. 1985), CHEF (Hammond 1986), PERSUADER (Sycara, 1987), and JULIA (Hinrichs, 1992), which were developed as part of PhD theses at the University of Yale (Kolodner 1993). These applications soon inspired developments outside Yale

Page 59


such as PROTOS (Porter, B.W. 1986; Bareiss 1988), HYPO (Ashley, K.D. 1987) and CABARET (Rissland 1989)∗. In the mid 1980s, CBR research expanded throughout the world. In Europe, for instance, CBR projects took place at Trinity College in Dublin (Keane 1988), the University of Aberdeen in Scotland (Sharma 1988), the University of Kauserslautern in Germany (Althoff 1989) and at the University of Trondheim in Norway (Aamodt 1994). In the late 1980s and early 1990s, a series of conferences and workshops in the USA funded by the DARPA program helped bringing CBR from academic research in cognitive science to the commercial arena (Watson, I.D. 1997). Watson, I.D. (1997) cited that this transition was marked by the launch of the CBR shell called ReMind (Cognitive Systems Inc.). This Shell was soon followed by others such as CBR3 (Inference Corp.), ESTEEM (Esteem Software Inc.) and ReCall (ISoft) that were also marketed and disseminated CBR world-wide. One of the reasons behind the acceptance of CBR relies on its capability to help in domains where the knowledge is unstructured and ill-defined (Riesbeck 1989, Kolodner 1993). CBR applications hold a repository of past experiences upon which users can rely and base their reasoning (Kolodner 1993; Leake 1996). Authors such as Kolodner (1993), Riesbeck (1989) and Schank (1996) cited that the task of developing CBR that relies on the gathering and organisation of past experiences, played an important role in bringing CBR into the commercial arena. The current state-of-the-art in CBR applications involves systems covering domains such as planning, architectural design, legal assistance, the diagnosis of diseases and instructional activities (Kolodner 1993). In spite of the success of these CBR applications, there is still work left on the theoretical foundations of CBR. For instance, authors such Aamodt (1994), Plaza (1993), Jurisica (1994), Leake (1996) and Schank (1996) claim that CBR as a process of human cognition has still much to contribute to when a broader view of human-learning, machine-learning and reasoning support is required.

∗

Kolodner (1993) provides a chronological description of the applications cited in this work and many others CBR applications worldwide .

Page 60


4.3 - An overview of CBR You can close your eyes to reality but not to memories. Stanislau Lec (Quoted by McCordick 1979)

CBR provides a methodology for the development of AI systems that handles case representations and emulates the way people employ deductive reasoning to relate past experiences to the help of new situations. The terms methodology and deductive reasoning highlight the two most relevant aspects of CBR for this thesis that are (i) the process of building the applications and (ii) the cognitive model of human reasoning. CBR is an AI technique that models domain knowledge as cases, i.e., representations of past experiences. These cases are kept in a repository (or case base) and can be retrieved to support user’s cognition for situations that are similar to those represented in the system (Kolodner 1993, Leake 1996). The mechanisms used to identify and retrieve the right case for the right situation (Schank 1988; Riesbeck 1989; Kolodner 1993) are at the core of CBR as a methodology for building AI systems. CBR architecture is composed of three main components: (i) the case repository of domain experiences, (ii) the retrieval mechanism that performs the search and retrieval in the case repository, and (iii) the indexes describing case contents and allowing differentiation between cases in the repository. Figure 4.3a shows how this architecture resulting from the dynamic memory theory becomes functional in CBR.

Fig. 4.3a – CBR and its main components

The CBR interface relies on users inputting the descriptions of the past experience they want to retrieve and accessing case contents. The working process of a CBR application is described in Figure 4.3b and begins with users describing the situation they are facing in order to check whether the system contains a similar case. Page 61


Schank (1990; 1996) cited that the problem description emulates a conversation with an expert, such as a medical doctor, where the patient gives leads and asks whether the professional has faced a similar problem before.

U

Input problem description

REtrieve

REuse

Review case contents

U

S

S E

Case Repository

E R

R Assign new indexes

REtain

REvise

Review case proposals

Fig. 4.3b - The CBR process (adapted from Watson, I.D. (1997)).

The next step is the “retrieval” of a similar case, where a retrieval mechanism is activated to narrow down the matching to a few possible cases. This process emulates the human process of recalling past experiences where, based on the leads given, the brain uses inferences in memories to search for a similar experience. Using Schank’s (1996) example of the interview with the medical doctor, based on the symptoms given by the patient, the doctor narrows the diagnosis to a few possible causes matching the patient’s situation. CBR users can then revise the case retrieved and either accept the recommendations made and apply them to the situation they are facing, or provide further descriptions to the system, if the case retrieved does not match their needs. Following the doctor-patient conversation, the doctor may say that the patient’s headache is being caused by excessive readings from computer screens. The patient may add that this may not be the cause since he/she suffers from headaches even when he/she does not work with the computer. The CBR working cycle is successfully completed when the user finds a perfectly matching situation in the case repository. The CBR may also make other suggestions, such as cases not fitting 100%, but similar enough that they can be adapted to the new situation. For instance, the patient’s headache may be diagnosed as being caused by too much exposure to the TV screen, rather than to the computer screen.

Page 62


There are situations where CBR may not help at all, but this is also common in real life. Using the interview with the doctor once more, this is the situation where the professional may fail to provide a diagnosis and recommend the patient to see another specialist. With the insight of the new doctor, the patient may find that the real cause of the headache was his/her sleeping position. Reporting this back to the first doctor means that he/she learns a new diagnose for the patient’s symptoms. It is equivalent to inserting a new case in the CBR repository, thus expanding the systems’ capabilities. This new case can either be inserted as a brand new case or as an adaptation from the cases already in the repository. The description of the CBR working cycle highlights some of the major issues in CBR research, i.e. case representation, indexing, retrieval, and system maintenance. All these issues are related to the use of cases for knowledge representation and are discussed further in the following sections.

4.4 - Case representation In a CBR application, cases need to be represented in a form that makes them useful to the user. Depending either on the domain or on the application, the contents of a case might be best presented through different forms such as drawings, photographs, graphs, charts, videos (Kolodner 1993) or VR, as proposed in this thesis (see Section 7.2) . Schank’s (1982) dynamic memory theory models each past experience as a group of pieces that together form the representation of a memory (see Section 2.9). In CBR, these pieces can be as simple as the description and the solution of a past experience. Cases can also be represented as a group of several pieces describing case contents and allowing retrieval. Kolodner (1993) describes cases as containing three major parts: 1. the description of the case, allowing its identification and retrieval; 2. the case itself that contains information relevant to the domain of its application; and 3. the resulting state of the domain when the solution was carried out. For instance, one of the cases in the VECTRA prototype deals with a scaffold structure for the construction of a residential house. The description of the case refers Page 63


to the information that differentiates it from the others held in the repository, such as the type of building, type of scaffold, and type of work being performed as well as the structural components of the scaffold. The information relevant to the domain describes how the scaffold inspection was performed and gives the sequence of items checked. The resulting state of the domain describes the situation found on site and the recommendations for similar inspections. Although Kolodner (1993) gives the description of a general case structure, this configuration can change depending on the application. For instance, parts that are used to describe a case in some applications can be used in others to provide information about the domain. Taking once more the VECTRA prototype as an example, if the domain is to learn about how to build scaffold structures, case attributes would have to be relevant to issues related to the construction of scaffold structures. If the domain was to provide training for the inspection of safety regulations, the attributes that are relevant to the domain will relate to approaches for inspection of safety regulations on scaffold structures. Case descriptions are usually referenced as case properties, case attributes or case features. In the light of this work, these three words have different meanings and applications, as described below. •

Properties - encompass all the possible pieces that might help to describe, identify or retrieve a case and are identified prior to the implementation of CBR.

•

Attributes – are the properties that are represented within the computer model, thus limiting attributes to the properties relevant to the application domain.

•

Features – are the attributes that differentiate the cases in the repository and are used as indexes, thus allowing the retrieval mechanisms to perform their search over them. Taking the VECTRA prototype as an example, one of the cases deals with the

inspection of a scaffold structure used to repair the roof-top of a three storey building. The information that users input into the system when describing the case they want to retrieve are the features. Attributes are those properties relevant to the training of scaffold inspection, such as how to inspect the soil plates, boards, and ties of the scaffold. Properties involve all the possible aspects of the inspector’s memory, such as Page 64


the weather conditions and the colour of the building. Properties can thus turn out to be irrelevant to the goals of the application. To find the attributes and features that properly represent the domain and allow case retrieval is a task that is usually performed with the support of domain experts. The experiences of these professionals are one of the major sources of knowledge for the development of CBR applications. Further details on the acquisition of cases are reviewed in the following section.

4.4.1 - Case acquisition Riesbeck (1989), Kolodner (1993) and Watson, I.D. (1997) cited that the task of case acquisition is one of the advantages that CBR offers when compared to the development task of AI applications such as knowledge-based systems and neural networks. The domain knowledge modelling, a critical task usually considered as the bottleneck in the development of AI applications (Murray 1992; Luger 1993; Greer 1995), is limited in CBR to the simple gathering of past cases. Schank (1997) cited that the acquisition of cases in CBR is facilitated for two main reasons. The first is because people can easily understand what a case is and how useful it can be for the domain. The second is because people frequently use descriptions of their past experiences to make a point relevant to a conversation. Other reasons making case acquisition in CBR simple are: •

there is no need to follow any particular methodology to structure the process of interviewing the experts, who can simply be asked to tell their past experiences in the domain;

•

there are situations where experts do not even need to know much about the process that makes an input become an output and simply need to know about the resulting state of the world when a solution was applied;

•

the knowledge of experts is made from both successful and unsuccessful experiences that can be gathered to become part of a case repository. There are domains where cases may be unavailable or difficult to acquire and

model. Leake (1996) called these domains as unnatural for the application of CBR. The same author cited that these domains may “depend on a significant case engineering

Page 65


effort to delimit the information that cases must contain”. This is perhaps the type of domain where CBR should give room to other computer techniques (Sycara 1992; Watson, I.D. 1997). In these type of domains, difficulties for indexing cases are expected as well (Voss 1994; 1996; Domeshek 1993). Case indexing is further discussed in the following section.

4.4.2 - Case indexing From the users’ point of view, the most important part of a CBR tool is the proper access to the cases stored into the system’s repository. The retrieval of the right case for the right situation is a key factor for the credibility of a CBR application (Barletta 1988; Domeshek 1993, Fox 1995). The task of providing the right descriptions for the cases is known as the indexing problem and has a major role not only in CBR but at any computer system involving information retrieval. Early references in CBR describe indexes as pointers to cases (Schank 1979; 1988; Hammond 1988; Kolodner 1989), a useful concept to help understand how indexes work. According to this viewpoint, an index works as a pointer to a case as it can work as a pointer to a book in a library or as a pointer to a record in a database. More recent references have been using the word label or case labelling instead of indexes (Kolodner 1993; Burke 1996). However, in the light of this work, the word label or case labelling refers to one of the techniques for indexing cases. Indexing a case in the repository requires the same competency as the indexing of a record in a database or a book in a library. The indexes must allow an easy, spontaneous and instinctive retrieval of the right case at the right time. It is essential for the indexing to: provide good descriptions of the cases; allow the distinction between the cases in the repository, be significant to the application domain that the cases refer to, be predictive for the users, and making cases easy to find and retrieve. A common approach for case indexing deals with using descriptions that are capable of differentiating the cases into the repository. This approach is used by CBR applications that represent cases in a database format and developers are just required to choose amongst the case attributes for those that are the most convenient to work as indexes. An example of this approach is presented in Table 4.4.2, where the features of hypothetical printers (cases) are used as CBR indexes.

Page 66


Printer A

Printer B

Printer C

Printer D

Speed (ppm):

4

4

6

10

Dimension (cm):

35x30x20

35x30x20

32x40x25

34x47x30

Memory (Mb):

1

2

2

4

Resolution (dpi):

300

600

600

720

Price (UK £):

180

260

284

586

Table 4.4.2 - Features of printers as CBR indexes.

The questions users are asked to answer (such as the price they want to pay, the printing speed they require, and the resolution they desire) and the value (or weight) of each index is a task forCBR developers. For instance, if the first question regards the price of the laser printer and the user says that he/she will not pay over £ 300, Printer D is automatically excluded from further search. However, if the user answers that he/she would like to pay around £ 200 and requires at least 600 dpi, Printers B and C match the user’s request and, although the price of Printer A matches the user’s requirement, its 300 dpi resolution excludes it from further search. Another usual approach to perform case indexing deals with the use of labels. Labels work as keywords for the information contained in the cases that is relevant to the application domain. Labels are especially useful for cases using multimedia to help the representation of past experiences. These digitised multimedia files are generated by software programs that convert the files into something the users can see and hear but that do not allow programmers to access the contents of these coded files (see Section 6.6 for further details on digitised images). Labels (or keywords) have been largely used by CBR developers and a vast literature covers the subject (Barletta 1988; Schank 1990; Katz 1990; Sycara 1991; Domeshek 1993; Kolodner 1993; Burke 1996). However, the task of generating labels for case indexing can be both time consuming and difficult (Leake 1991; Domeshek 1993), as it requires looking through case contents to identify labels relevant for all the possible situations a case can be useful in.

Page 67


In certain domains, the past experiences may deal with a great diversity of issues. This, coupled with the user's lack of familiarity with the description process of past experience, can create problems. It can lead to difficulties in evaluating labels that properly describe the past experiences. Nonetheless, this problem is not unique to CBR systems as it applies to any system involving information retrieval (Riesbeck 1989; Ferguson 1992; Domeshek 1993). The range of problems in conjunction with the variance of users’ needs may lead to a great number and diversity of indexes. Users may not be fully familiar with case descriptions, resulting in the systems’ failure in finding a situation matching users’ queries. Thus, designers must be aware that it may not be simple to find indexes unifying all the information contained into cases. Kolodner (1993) cited that the three main factors CBR developers must consider are: (i) the application domain of the CBR application; (ii) the range of cases available to support the application domain; and (iii) the future improvements and expansions in the case repository and system’s capabilities. Kolodner (1993) also describes the two main approaches for index selection: •

the functional approach - designers should look for the cases’ contents that properly describe the tasks they support;

•

the reminding approach - designers should look for the mental leads that actually remind the experts of the past experiences related to their job performance. Although these two approaches for case indexing fit most CBR applications,

difficulties can emerge when the case repository reaches great dimensions (Riesbeck 1989). Kolodner (1996) cited that, since most CBR applications do not contain more than a few hundred cases, a simple indexing approach could work “just fine”. However, difficulties can emerge when the case repository reaches great dimensions (Riesbeck 1989) and more elaborated indexing approaches might then be required. More sophisticated case indexing approaches are found in CBR and information retrieval systems (Spark 1981, Ase 1990, Katz 1990). Nonetheless, even for applications like SQUAD (Kitano 1993; 1996), where the repository comprises more than 20.000 cases, a simple approach can work. SQUAD has

Page 68


its cases represented in a database format where records contain information about software bugs and the fields used for retrieval contain descriptions of the bugs. The search is simply performed over indexed case features and the authors found that, in spite of the dimension of the case repository, the retrieval time has not affected system’s usefulness. Kolodner (1996) and Watson, I.D. (1997) cited that most CBR applications use simple indexing procedures. Kolodner (1996) also mentioned that the indexes are usually taken naturally from the experts’ description of the situations when the cases occurred, or the resulting state of the world after the occurrence of the case. The author concludes that the main reason supporting this simplicity is because “what is important in a case does not change much from situation to situation”. The same simplicity is expected for the algorithms performing the retrieval that is the issue discussed in the following section.

4.4.3 - Case Retrieval Case retrieval is the searching process for the cases that achieve the highest overall match given the user’s description of their needs. The matching process is performed by algorithms called retrieval engines or searching mechanisms. These algorithms are derived from multidisciplinary research in computer science, involving areas such as the retrieval of computer files, library documents, image retrieval, and more recently, from algorithms performing Internet search. CBR retrieval engines work by requesting users for information and searching for cases that match user requirements. Case search is performed over a set of similarities between the cases in the CBR repository (Domescheck 1993). Since it is usual to deal with situations where the cases do not perfectly match user’s description, retrieval mechanisms are expected to cope with partial matching. Facilities for searching and ranking the cases that are the most similar to the user-inputted description are also found in retrieval mechanisms. Case retrieval plays a major role in CBR and algorithms can range from fairly passive approaches that only respond to well conceived requests to an active retrieval capable of interacting with users and helping them by correcting inputs and even

Page 69


misspellings. Three common approaches for case retrieval are described in the following sub-sections.

4.4.3.1 – Nearest Neighbour Nearest-neighbour is the simplest approach for case retrieval and searches the case repository for the closest matches by determining how similar each case is to the situation described by the user. The higher the similarity score is, the more similar is the case in the repository. The simplest approach to nearest-neighbour matching considers all case features with the same value. Therefore, a case that matched on ten features is considered more similar than a case that matched on six. Another approach uses weighted-features to differentiate their importance over the matching process. The scores in weighted-nearest-neighbour are computed by using the formula (Barletta 1991):

∑ Similarity( In

[i ]

, Re t[i ] ) xWeight[i ] ]

Total _ Weight Where: i is the feature used for matching; In[i} is the input-case-value; and Ret [i] is the retrieved case-value As nearest neighbour searches all features and cases in the repository, this approach can be slow especially for case-bases containing thousands of cases and/or hundreds of features. In addition, the similarity between two cases must be correctly defined for the retrieval mechanism to work properly. This is due by the fact this retrieval mechanism is class blind. The similarity between to different cases such as my printer keeps the character too close and my monitor keeps the characters too close can be high, merely because the words used for the description are similar (Kriegsman 1993).

4.4.3.2 – Inductive retrieval The inductive retrieval mechanism compares case features using a decision tree that is defined to classify the cases in the repository. The user descriptions are propagated down the tree path until a leaf node is found. The search in the repository is faster than the nearest neighbour approach since the tree structure could exclude many cases from the searching process (Barletta 1991).

Page 70


The drawbacks of this retrieval approach are the need to classify the case features in the tree structure. Moreover, the creation of a degree of distinction to establish a similarity score between cases in different branches is difficult and may reject cases for one single feature that does not match Watson, I.D. 1997).

4.4.3.3 – Knowledge-guided retrieval The knowledge-guided retrieval mechanism relies on domain knowledge to perform the searching process. Domain knowledge can be used to help the retrieval process and may also help to define the case features that match user description. For instance, rules may be established to specify those previous experiences that could match some common mistakes in the user description (Smyth 1994a). Watson (1994) stated that a difficulty in this retrieval approach is due to the acquisition of the knowledge models guiding the retrieval. Another difficulty is the task of partitioning the search space so the partial matching could properly be done on potentially relevant cases.

4.4.3.4 –A brief recap on retrieval mechanisms The simplest retrieval mechanism is the nearest neighbour and Watson, I.D. (1997) suggests that this approach should be used whenever possible, “until retrieval time becomes an important issue”. For CBR applications containing more than a thousand cases, this method could prove too slow to perform the retrieval. The algorithms performing the retrieval depend on the approach adopted in the CBR for case indexing (Leake 1991) and play a key role in defining the retrieval strategy (Stotler 1989; Edelson 1993; Zacherl 1993). The ideal situation is to direct the retrieval in some way so that matching is only attempted on those cases with potential relevance to the user description. There are thus situations where more than a single retrieval approach can be used. For instance, Smyth (1994) provides an example where a two-stage retrieval approach is used: the first stage relies on the inductive retrieval filtering out irrelevant cases; and the second stage performs a detailed similarity analysis using the nearest-neighbour approach. Schank (1994), Kolodner (1996) and Leake (1996) cited that the fast growing success that CBR has been achieving is due to its simplicity. This simplicity also includes the searching mechanisms CBR usually requires. Schank (1994) quotes that Page 71


the underlying well structured nature of CBR allows the use of simple retrieval mechanisms. Schank’s (1994) viewpoint seems validated by a number of applications described by authors such as Kolodner (1993), Leake (1996) and Watson, I.D. (1997) who reveal that most CBR applications involve simple retrieval algorithms.

4.4.4 - Case utilisation Once a case is retrieved, users are required to judge whether it is relevant to the situation they are facing. This task can be performed either exclusively by users reasoning over the case contents and relying on their background of the domain knowledge, or the system can provide a simulation environment where users can test their assumptions. The reasoning process over the case contents requires the users’ ability to understand the case retrieved and its relevance to the problem. The theoretical concept of CBR is that a case must play the same role as a previous experience that is part of human expertise. The richer the case representation, the easier future access is and the better the feedback provided for similar situations. Therefore, a main issue of CBR is case (knowledge) representation in the computer. Section 2.9 shows that Schank’s (1982) dynamic memory theory has the representation of past memories broken down into pieces that are used to represent cases in the computer. When implemented in the CBR system, these pieces become the attributes of a case. Some of these attributes can work as indexes for case retrieval, emulating the leads that allow people to retrieve the right memories for the right situation. As a basic principle, a case can be broken down into as many parts as the system developers want. From a point of view of efficacy in case representation, the system should represent cases containing all those details that are pertinent to the past experience. Efficient case representation would get just those pieces that are significant to the systems’ application domain. For instance, in applications aiming at training, designers must include representations of past experiences that convey the skill to be achieved. According to Schank’s (1982) dynamic memory theory, a case is a contextual piece of memory and it is common to recall only parts of a past experience (see Section Page 72


2.9 for details on MOP and Scripts). When it comes to CBR, cases should also be represented in a manner that allows indexing, retrieval and provides easy understanding of its contents so as to support users’ reasoning. Authors such as Riesbeck (1989), Kolodner (1993) and Schank (1994) cited that cases must hold the information needed by users in such a way that it should: •

preserve the semantic of the domain;

•

represent a contextual piece of memory, i.e., associate the outcomes with the descriptions of the situation when the case occurred;

•

ensure that the applications’ goals are reached;

•

reflect a sharp conceptual view of the domain and the context it represents; In order to ease understanding of case contents, several media have been used to

improve case representation. For instance, CBR applications can use databases, worded descriptions, pictures, sounds, image animation and video clips. Altogether, these multimedia aim at improving the representation of cases and the usefulness of CBR tools. For instance, the CBR community∗ (Kolodner 1993; Burke 1996; Schank 1997) has been using cases represented as digitised films containing experts telling stories. The reasoning support process provided by the cases represented is the main goal of CBR applications. Rather than providing a solution, CBR aims at providing a similar situation to support users’ reasoning. However, there are situations when the case repository does not contain cases capable of providing any helpful feedback. When this situation occurs (or becomes quite frequent), it is time to update the case repository with new cases. This process is called case-base maintenance and is discussed in the following section.

4.4.5 - Case-base maintenance Although most CBR systems do not contain any kind of facility allowing users to update the systems (Kolodner 1993; Wilke 1996; Watson, I.D. 1997), CBR maintenance is an important research issue. CBR maintenance deals with keeping the application up-to-date with the domain knowledge. As the domain knowledge is represented by ∗

Examples involving story-telling and multimedia case representation, such as the ASK systems, can be found at The Institute for the Learning Sciences at Northwestern University http://www.ils.nwu.edu/~e_for_e/ (last visited 12/11/97).

Page 73


cases, CBR maintenance involves the addition of new cases to the CBR repository and the increase of the system’s capabilities in providing support for reasoning. In accordance with the dynamic model of human memory (Schank 1982), CBR maintenance is seen as the system’s learning by experiencing new situations. Theoretically, the maintenance in CBR is potentially simpler, only requiring the updating of the case repository by (i) adding new cases to the repository, or (ii) adapting from the cases already there. The first requires to go through the stages of breaking down the new case into pieces, representing the case into the computer, and indexing it for future retrieval. Adaptation involves changes in the cases present in the repository, such as fitting in a new solution to an old problem, fitting a new problem to an old solution or repairing solutions that are no longer found optimum. Adaptation seems simpler since it relies on the case(s) already in the repository and attributes can be changed to fit with the new situation. It entails just part of the work required at each stage of inputting a new case as part of the case is already represented and indexed. Depending on the situation, adaptation can be “as simple as substituting one component of a solution for another or as complex as modifying the overall structure of a solution” (Kolodner 1993). The adaptation emulates the aspects of human learning, where new experiences re-align with the pasts (Schank 1982). This process requires the understanding of case contents and a certain degree of domain knowledge to project the results of a new solution over the attributes of the case (or cases) used to support the adaptation (Piaget 1982; Branting 1994). Case adaptation can be performed manually, automatically or mixing both approaches. Manual adaptation requires user’s feedback to modify the original case. Automatic adaptation relies on special computer algorithms that are the focus of machine learning research. The user (or developer) interacting with the system and altering case contents is the intermediate form of case adaptation. Automatic case adaptation is performed by the CBR itself by taking a problem description and a solution that does not fully match users’ need and creates a new case that fits the users’ feedback on the situation they are facing (Kolodner 1993; Leake 1996). Automatic adaptation makes use of different approaches and algorithms that are

Page 74


mostly derived from an active research area, namely Machine Learning that aims at providing systems capable of improving themselves through experience (Mitchell 1997). Machine learning algorithms have been classified by Aha2 as lazy and eager learners. Lazy learning algorithms only act under the user’s consent. They seem to fit CBR better since they are usually simpler than eager algorithms and take user’s feedback into consideration prior to creating new cases. Further details on automatic case adaptation related to the current state-of-the art in CBR applications are discussed in Section 4.4.5. A common practice in commercial environments is to leave the adaptation to be performed by the team who developed the system (Kolodner 1993; Aamodt 1994, Watson, I.D. 1997). Commercial applications involving either manual or automatic facilities for system’s maintenance are still quite rare, and rely on the user interface, which is the topic discussed next.

4.5 - The CBR interface Dearden (1995) stated that the success of any interactive intelligent system, whether it is rule-based or case-based, “is dependent not only on the quality or on the appropriateness of the knowledge encapsulated within the system but also on the quality of the interaction that the system supports”. From the user’s point of view, the interface is the component that addresses and supports the tasks and functional requirements of the CBR application. The interface in CBR basically involves the input of the problem description and the receiving of the information contained in the case retrieved. This interface can also display the results of the search, include an interactive mechanism to narrow down the search as well as multimedia features to present case contents. Facilities for CBR maintenance, guiding case adaptation and the input of new cases can also be part of CBR interfaces. The state-of-the-art in interfaces for inputting the description of the case that users want to review still relies on the traditional line-oriented interfaces that include question-answer and command line dialogues. Current CBR Shells such as CBR3 (Inference Corp.), ESTEEM (Esteem Software Inc.) and Art* Enterprise 2

David Aha maintains a web site containing key information about machine learning and CBR.

Page 75


(Brightware Inc.) even provide facilities such as form filling and menu systems where developers can quickly build such interfaces. Another important aspect of the CBR interface deals with the representation of the case contents into the computer and how these contents are delivered to the users. Cases represented as digitised multimedia features, such as pictures, sounds and films have been largely used by CBR developers. The main issue regarding these media is always the same: to improve the representation of previous experiences and the usefulness of cases. The following section describes five CBR applications and provides a picture of the current state-of-the-art in CBR interfaces.

4.6 - Review of CBR applications The previous sections of this chapter discus the tasks involved with the development of CBR applications and give a comparison with the human model of cognition that was at the origin of this AI technique. This section provides a review of five CBR applications focusing on their application domain, techniques for case representation, users’ interface, indexing and systems’ maintenance.

4.6.1 - CLAVIER This system is perhaps the first commercial application of CBR and it has been used since the autumn of 1990 at Lockheed Missiles and Space Company (Hennessy 1991; 1992; Hinkle 1994). CLAVIER was developed in Common LISP and its application domain deals with the curing of carbon fibre products in an autoclave. The cases contain information regarding the layout configuration of successful cure of products within the autoclave. Case contents involve the graphical representation of previous autoclave layouts, the names of the products, the position where they were placed in the autoclave, their relative position to the other products, curing time, internal pressure and temperature. The system’s interface allows users to provide a list containing the name of the products for curing. Case retrieval is performed over the name of the products in previous successful cures in the autoclave. For instance, for a given list of parts waiting to be cured, the system searches for previous layouts that successfully cured the greatest number of parts on the waiting list. This retrieval approach was further modified to help finding the smallest number of autoclave layouts needed to cure all the parts in the waiting list. The system then shows the layout and the settings for each new curing process. Case adaptation is performed if the products Page 76


in the waiting list do not match any previous layout. The system then suggests the placing of a similar product on the place of the missing product. Users can thus decide to accept the suggestion, but must first check out whether there was no previous failed layout with such configuration. If users decide to accept the system’s suggestion, users can, after curing this new layout, include the new case either as a successful or failed layout.

4.6.2 – ARCHIE This system was developed at the Georgia Institute of Technology to help architects with a case-library of office building designs (Goel 1991; Pearce 1992). The system was developed using the CBR Shell Remind (Cognitive Systems Inc.) and its cases contain information such as design goals, constraints, construction plans and how well the design satisfies the given goals. Each case contains more than one hundred and fifty attributes and involves textual descriptions, dimensions, annotated plans, photographs, drawings, and animations. The design description reports the viewpoint of architects, engineers, include post-occupancy evaluations and try to cover the building life cycle. The system’s interface lets users describe the type of design they want to build or the problem they are facing in an aspect of the building design. The retrieval mechanism gets this textual input from the user and searches for cases that match most of the words. The system thus allows users to access case contents through hypertext links and move between case attributes. The authors cited that the complexity of representing building design cases is the major reason why the system does not allow for any case adaptation. The authors also affirm that building design knowledge covers a wide range of domains and CBR developers should narrow the application domain to get better results from CBR applications.

4.6.3 – CASEline This system was developed at British Airways to assist technical support engineers with aircraft fault diagnosis and repairs on the Boeing 747-400 (Magaldi 1994; Dattani 1996). The system was developed in the CBR Shell REMIND (Cognitive Systems Inc.) and its cases contain information involving worded descriptions of past defects and the associated repair procedures. The system’s interface prompts a command line where users can input the description of a fault given by the aircraft pilot. Case retrieval is performed either over the fault number shown on the Page 77


plane’s panel and maintenance manual or over a textual description of the fault. The system does not contain facilities for case adaptation and only expert engineers can input new cases and update CASEline. This application does not involve any special difficulty to develop and its richness relies on the modelling of domain knowledge performed by the gathering of past cases. The authors of CASEline report that a major reason supporting the system is the contextual information contained in each case: should an accident occur, the expert developers could demonstrate that they have conducted procedures that proven successful in the past. The usefulness of this system can also be proven by the fact that British Airways is developing another similar application to deal with the maintenance of the Concorde.

4.6.4 - SMART This system was developed to help customer support personnel working at the telephone service centre at Compaq Computers and was the first commercial help-desk application using CBR (Acorn 1992; Nguyen 1993). The system involves an integration of case repositories of problems from different Compaq products, and involves hardware running under UNIX, DOS, Windows, Novell, OS2 and LAN Manager. The case representation contain the symptoms of a problem and the description of a solution. The system’s interface provides the customer support personnel with a series of questions to be asked to the customer over the phone. Based on customer’s answers, the system narrows down the fault diagnosis to find the cause and provide a solution. Only if the system is incapable of providing a solution the call is passed on to the Compaq engineers who can then include this new case into the system. The user interface starts by asking questions regarding the type of product the customer is using and then allows an input of the problem description. Case retrieval is performed over the descriptions of the problem. Based on the customer’s answers, the system keeps bringing questions about the highest ranked problem to identify whether the customer’s description fully matches the symptoms of a specific problem. The authors cited that the difficulty in building the systems was not the CBR methodology, but the integration of the different case-bases into the local network. Several help desk personnel had to be able to work on-line and to solve problems on different equipment at the same moment in time. This system had to allow for the input of new cases with the system remaining online.

Page 78


4.6.5 – GIZMO TAPPER This system was developed by Broderbund Software Inc.∗ to provide customer technical support and was the first system of this type working on the Internet. The system came on-line near Christmas 1995, with the launch of the Broderbund’s game, MYST, to avoid the need for extra staff over that holiday period when the company expected a great number of sales. The system was developed using an adaptation of the Inference Corp. CBR3 Shell to support Internet applications. Case contents contain descriptions of technical problems with the MYST game and the symptoms of the problem. The system’s interface begins by asking general questions regarding the computer platform and hardware configuration. The system can either keep asking questions until a solution is found or users can input a description of their problem. The problem description involves facilities to correct misspellings and search for synonyms. The system searches for cases containing descriptions that match users’ input and retrieves a ranked list of possible solutions. Users can thus click the mouse over the solutions on the list and obtain more information about each listed case. Another relevant factor for Broderbund was that the support personnel working on the phone could use the same system that provides support via Internet.

4.6.6 – Brief recap on the applications reviewed The descriptions of the CBR applications above cover domains such as machine set up, building design, mechanical diagnosis and help desk technical support. Authors such as Allen (1994), McCarthy (1996) and Watson, I.D. (1997) cited that customer services applications is a domain that still has much to gain from CBR. Reasons why CBR fits the customer service domain are the ease to classify problem situations and the suitability of CBR to diagnose problem-solving domains. Other conclusions taken from the systems reviewed that are relevant to this thesis are: •

a comprehensive set of cases is not required to start using the CBR and can be built as the system is put to use;

•

there are no special requirements for programming languages for CBR development although Shells can provide a structure for case representation, retrieval mechanisms and speed up the development process;

∗

Further details on this application can be found in Borron (1996), Watson, I.D. (1997) and at the Inference Corp. web-site (http:\\www.inference.com).

Page 79


•

case representation can make use of multimedia such as pictures, animated images, and video clips telling stories but the retrieval mechanisms act only over the alphanumeric case attributes;

•

CBR interfaces for inputting case descriptions rely mostly on users’ typing a description of the situation they face or answering yes/no to the system’s questions;

•

even if the past experience was a failure, it can provide feedback and help avoid repeating an unsuccessful action;

•

case adaptation requires a well-defined case structure and a reliable knowledge source;

•

there does not seem to be any limits for the domains where CBR can help, but CBR is more likely to succeed in narrow and well-defined domains. The applications reviewed show that CBR upholds an emulation of the human

process of reasoning from past experiences rather than being a specific technique for the development of AI applications. Cases can be represented, indexed and retrieved by several means. The aim is to provide a clear description of a past experience and allow its retrieval “at the right time for the right situation” (Schank 1996). Further details on the CBR methodology applied to instructional applications are given in the next chapter.

4.7 - Synthesis of the chapter This chapter reviewed CBR as a technique for the development of AI systems. The review covered CBR from its origins as a model of human cognition to the state-ofthe-art technologies and applications. The CBR architecture, working cycle, case representation, case acquisition, case indexing, case retrieval, and development process of applications were also considered in this chapter. CBR applications rely on the representation of past experiences into the computer. However, developing applications requires more than simply collecting a representative number of domain cases. CBR development involves tasks such as structuring the case repository, defining criteria for case indexing, assessing similarities between cases and developing retrieval algorithms and interfaces for the display of case contents. Page 80


CBR also includes techniques for the systems’ maintenance that rely either on the input of new cases or adapting from the cases already in the repository. Both the input of new cases and adaptation require a reliable source of cases that neither the user nor the system itself can provide. Moreover, indexing new cases in accordance with the requirements discussed in Section 4.4.2, usually demands a joint effort between domain experts and the system’s developers. For instructional CBR applications, where the users are learning about the domain, case adaptation is thus not expected. Currently, academic and commercial applications dealing with a wide range of domains and applications can be found. For instance, this chapter reviewed CBR applications dealing with domains such as machine set up, mechanical diagnosis, building design, and customer technical support. However, applications in a welldefined domain that can be classified after asking a few questions have greater chance to succeed commercially.

Page 81

Chapter 5 - Training with case-based reasoning

5.1 - Overview

The previous chapter described CBR focusing on its origins, its working cycle and the tasks involved in the development of applications. This chapter describes CBR from the viewpoint of instruction, addressing issues related to CBT and ICAT such as domain modelling and instructional methodologies. The state-of-the-art in CBR training is reviewed and applications are described. The review of these applications provides feedback for the instructional approach adopted in the VECTRA prototype. Most of this chapter is based on the work of Schank (1982; 1995, 1997) and the Institute for the Learning Sciences (ILS) indicating that the CBR involves a model of cognition

integrating

problem-solving,

understanding,

learning

and

memory

organisation in a way that can be useful to instruction. CBR holds a repository of past experiences that can also work as a repository and source of knowledge for training. CBR and its relation to the instructional activities it can provide is the focus of the discussion in this chapter.

Page 83


5.2 - Learning from past memories We tend to assume that learning is bound up in the ability to articulate the theory behind some phenomena, but an important aspect of learning is simply being able to predict what will happen next, on the basis of experience. Roger Schank(1995a)

Chapter 4 shows that CBR originated from a model of human cognition and provides a technnique for the development of AI applications. This model of cognition simplifies memory as a warehouse of past experiences where knowledge is kept. This notion of a warehouse, although only assumed as a basis for comparison, implies that learning can be seen as the storing of new memories. Schank (1995) gives an example explaining the role past memories can play over human learning and how the process works inside the human mind. For instance, considering the question “how old is the ex Prime Minister John Major?” People can use different approaches to answer this question. The three most usual approaches are: (i) knowing the answer immediately as a fact kept in memory; (ii) knowing how old John Major was at a certain point in time and make calculations to determine his current age; or (iii) using another person whose age is known and bears a resemblance in age to John Major. Even if this person looks slightly older or younger, the comparison can still be made. Each of these strategies to answer the question relies on a different type of memory recall. The last strategy is the most complex as it requires the person questioned to know more than a single fact or to apply a rule. This strategy requires searching for a memory of an individual whose age is known and appearance is similar enough to allow an inference on Major’s age. The memory retrieved can be seen as a case in CBR and the comparison of similarities as the user’s reasoning process. Learning occurs by keeping this new information in memory and if the same question is asked again, the person will be able to answer it promptly. This human capability to learn by associations between similarities has been referenced by Gagne (1985; 1992) as learning by applying intellectual skills. This cognitive capability is also denominated as learning by analogy in CBR references such as Holyoak (1984), Ashley, K.D. (1987), Burstein (1989), Veloso (1989; 1992) and Schank (1996). Learning by analogy is of major importance for any instructional application that involves CBR and is further discussed in the following section.

Page 84


The CBR notion of remembering is the search and retrieval of a stored piece of knowledge that can provide a basis for comparison with the new problem situation. Schank (1982) cited that human learning depends upon the inputs received through life. “Each word read and each sight seen changes human memory in some way”. The human reasoning process plays its role by interpreting and deciding what is worth keeping in memory and how it fits in relation to previous knowledge (Wingfield 1979). References to the human learning process and learning capabilities provided by CBR are made throughout this chapter. There might be situations where the jargon from pure cognitive psychology conflicts with the jargon of the CBR research and publications. In this chapter, preference is given to CBR references. In order to avoid any possible misunderstanding, a glossary of the terms and their meaning is provided at the end of this thesis.

5.3 – Case-based instruction Our prior experience, knowledge, and expectations are key to learning. Wingfield (1979)

The implications of the CBR model of cognition that provides instructional applications have been the focus of studies at the Institute for the Learning Sciences (ILS). These ILS studies have been carried out by authors such as Schank (1991; 1995; 1997), Burke (1996), Ferguson (1992), Chandler (1993). Other studies have been conducted by authors such as Bareiss (1988; 1989), Redmond (1989; 1992), Veloso (1989; 1992), Goel (1991) and Bhata (1997). The general consensus between these authors is that previous experiences play an important role for learning: past experiences provide a source for analogical reasoning that users can rely on. Wingfield (1979) has divided this process of applying analogies to learn from past experiences into four smaller processes that are: (i) recognising the source for the analogy; (ii) transferring the analogy, or part of it, onto the target; (iii) evaluating those parts transferred; and (iv) consolidating the reasoning process. From a pragmatic point of view, this whole process is similar to the CBR working cycle discussed in Section 4.3 because of aspects such as: recognising the source for analogy – cases, both in the human memory and in CBR, contain a previous experience that was found somehow of relevance and thus

Page 85


stored, nevertheless, these cases would be meaningless if they were not recalled at the right time and did not provide the right source for analogy; transferring the analogy: the repository provides a case, but it is the responsibility of the user to identify the contribution that can be taken from the case and this may entail a creative task, such as applying cognitive strategies to derive new solutions from those similar kept in the computer; evaluation of the parts transferred: the evaluation of past memories is a human task that can only be helped by a system holding case contents and structures that facilitate human understanding; consolidating the reasoning process: the reasoning can lead not only to a solution but to learning by focusing on the details that differentiate the cases, thus creating a model of the actions and results that can be used in the future. This CBR working cycle highlights the areas where CBR instructional applications should focus. Another aspect of learning by analogy relevant to CBR instruction is that learning does not happen by recalling a past experience that perfectly matches the problem situation. As described in Section 2.7, learning takes place when the new information goes from the short-term memory through to the longterm memory. Thus, learning implies a permanent behavioural change that requires the understanding of the case contents and leads to a generalisation from the problem situation to its underlying concepts. Another CBR learning issue that has been raised in the previous chapter is the importance of making the representation of cases become properly memorable to the users. Kolodner (1993) cited that people “tend to remember things that are presented in compelling ways”. In instructional applications where the learner is free to take the tool whenever it is convenient, case presentation and guidelines to provide instruction are key issues to consider. Authors such as Schank (1996), Leake (1996) and Kolodner (1996) have pointed that the instructional potential of the CBR paradigm has barely been tapped. Case libraries containing instructional activities can serve as corporate memories that employees can rely on and as teaching archives that make expertise available on an asneeded basis. Page 86


In order to build such instructional CBR applications, designers have to deal with instructional issues in the CBR. Instructional methodologies are an issue of concern for all types of computer and non-computer based instruction. The following section discusses further the instructional capabilities of CBR.

5.4 – Case-based instructional activities Chapter 2 reviewed Schank’s (1982) dynamic memory as a theory of cognition and memory organisation inside the human mind. The previous section shows how this theory fits with the CBR working cycle. This section discusses a theory of instruction that was introduced by Gagne (1985; 1992) and deals with the achievement of an instructional goal by breaking down the process into instructional events. This instructional approach and how it fits CBR instruction is the focus of this section. Gagne (1985) sees cognition as a general term that describes the mental processes that transform the various forms of sensory input by coding the information, storing it in memory and retrieving it for later use. Learning is thus seen as a cognitive activity that involves, organising and reorganising information. However, for learning to take place and lead to a permanent change in the memory structure by reaching the long-term memory (see Section 2.7), instructional strategies capable of transforming the sensory experience into organised concepts are required. Gagne (1992) also cites that different learning outcomes require special conditions to be organised in the memory. Learning conditions are thus meaningful instructional modes that help learning to reach a permanent state and ease later recall. With his conditions of learning, Gagne (1985; 1992) postulates that there are five different learning categories: verbal information, intellectual skills, cognitive strategies, motor skills, and attitudes. Each category requires different types of instruction that are supported by different external and internal conditions. This theory of Gagne (1985) fits with the computer-aided instruction premise that

different

learning

outcomes

require

different

instructional

approaches.

Applications should thus be individualised to suit the learner, the domain, and employ different multimedia providing information to aid domain learning. Each instructional activity should thus be part of a structured instructional strategy.

Page 87

Chapter 5 - Training with case-based reasoning - response - retrieval - storage

Learning stages

- understanding - coding - expectancy - stimulus

- attention - perception

Performance

Recall

Retention

Acquisition

Apprehension

En ret han en cin tio g n

pe Acc rfo es rm sin an g ce

P fee rovi db din ac g k

pe Eli rfo cit rm ing an ce

L gu earn ida in nc g e

pe Se rce lec pti t on

Re lea call rni prio ng r

I ob nfor jec min tiv g es

G att ain en ing tio n

Motivation

Instructional steps

Fig. 5.1 – Achieving an instructional goal (adapted from Gagne (1992))

Gagne (1985; 1992) arranges these learning conditions into a hierarchy in the sense that basic types of instruction are necessary before an individual can advance to more complex forms of learning. The author identifies nine learning conditions to be considered for each instructional goal. The order that eases learning, with each prior level becoming a desired prerequisite for the next higher condition, is presented in Figure 5.1. This Figure fits these learning conditions in with the learning stages discussed in Section 2.7. CBR, like any computer-based instructional application, provides learning as a sequence of instructional activities. In CBR, this sequence of activities must fit with the CBR working cycle. The user’s learning process of input, storage and retrieval should hence be provided in an instructional cycle that gives meaning to the information to be assimilated into the more permanent store of knowledge and past experiences. Including instructional strategies into the CBR working cycle is a challenging task for the developers of applications. Section 7.5 provides further details on how this theory of learning conditions is mixed with the instructional activities of the CBR working cycle in the VECTRA framework. The following sub-sections describe two ICAT applications using Gagne’s (1992) conditions of learning.

5.3.1 - ID and ID2 An example of a ICAT application using Gagne’s (1985; 1992) conditions of learning is the ID Expert (Merrill 1989) and its successor, ID2 (Merrill 1990). These

Page 88


systems work as a consultation tool that provides instruction emulating a dialogue between the learner and the course designer. Both ID and ID2 follow an instructional strategy that takes into consideration different learning goals requiring special learning conditions. The domain knowledge follows the instructional hierarchy shown in Figure 5.1 and was called Knowledge Analysis and Acquisition (KAAS). It employs a network of related frames, each representing an instructional goal. Each frame provides all the instructional activities needed to achieve an instructional goal. The frames also contain links to other frames, thus composing a frame-network of domain knowledge. This frame network contains the domain knowledge to be taught and emulates a mental model of the domain knowledge. The authors of the ID and ID2 suggested a general framework for instructional purposes based on their approach for knowledge representation and tutoring. Limitations were nonetheless raised and are similar to those attributed to ITS (see Section 3.7), such as the difficulty in coping with different instructional strategies and in keeping track of the learners’ performance. On the other hand, these applications present positive aspects, such as their flexibility to incorporate new frames into the system, thus improving domain coverage and the instructional approach that was designed to work as a free consultation system holding an instructional strategy.

5.3.2 - ECAL Another application that uses Gagne’s (1985) instructional approach is the Extended Computer-Assisted Learning (ECAL) system (Elsom-Cook 1990; Posner 1994). The basic unit in this system is an instructional goal in the domain of producing a curriculum for planning the lecturing units. The courseware provides a sequence of instructional activities for each instructional goal in accordance with Gagne’s (1985) theory of learning and instructional stages. ECAL also uses frames for knowledge representation and each frame contains an intended learning outcome. This tool does not provide any guidance for the instructional events. Instead, each frame is indexed by list of keywords (or labels) allowing users to search for the instructional goals.

Page 89


This application keeps track of user’s performance, relying on mathematical algorithms that assign scores from zero to ten to the instructional goals. The scores indicate the system’s evaluation of a user’s performance over the instructional goal contained in each frame. When the contents of a frame are retrieved, users are also presented with a “diagnostic” frame. User’s performance over each frame and overall performance are constantly evaluated since average results over the system are also available.

5.3.3 - Recap of the applications The two applications reviewed rely on Gagne’s (1992) principles of instruction theory. These applications are relevant to this thesis as they are providing a structured hierarchy for the instructional activities, guiding users through a sequence of instructional goals (only ID2), and providing an approach to evaluate learning. The approach used to evaluate learning also plays the role of stimulating users to keep exploring the system’s instructional capabilities (only ECAL). Differently from the applications above, CBR represents knowledge as past experiences. The traditional CBR working cycle does not include guidance or instructional methodologies other than the users’ search for the information they require. This fact is confirmed by the CBR commercial Shells that have been reviewed by authors such as Althoff (1995), Watson, I.D. (1997) and those found at the AI-CBR web-site∗. These Shells do not provide any special feature regarding the development of instructional application. Rather, this is a task that requires developers to include special programming so that instructional strategies can be suggested and the users’ learning performances can be evaluated.

5.5 – Learning from CBR Knowledge is of two kinds: we know a subject or we know where we can find information upon it. Johnson, W.L. (1984)

CBR related learning involves two major issues, sometimes confusing the readers, that are: (i) the human learning related to the use of CBR applications and (ii) machine learning. The former is related to the human cognition process that can be activated by the CBR tool. The latter deals with computer algorithms that allow the ∗

The AI-CBR web site contains a continually updated review of CBR tools along with information and links to vendors (last visited 15/01/98).

Page 90


CBR tool to learn by increasing the number of cases in its repository. Machine learning and human learning are issues with several common interests (Bradshaw 1986; Aha 1997; Mitchell 1997). However, this thesis focuses only on the human learning activities related to the use of CBR. Researchers such as Kolodner (1993), Schank (1994) and Dubitzky (1996) have attempted to categorise the forms of learning that CBR can provide. Some of these learning possibilities are likely to happen as a by-product of the interaction with the CBR. As an active instructional tool, the four main instructional strategies that can be taken from CBR are described in the following sub-sections.

5.5.1 - Discovery learning Discovery learning involves letting users explore the case repository freely and discover the knowledge contained in the past experiences. CBR systems providing this instructional strategy can either passively allow free case search and retrieval or provide guidance for the exploration. One of the earlier systems based on the discovery learning approach has been presented by White (1987). The system lets users experiment dynamic simulations of electrical circuits’ behaviour. It allows users to build simulated electric circuits and explore the behaviour associated with the resulting configuration. Users can then attempt to solve problems or observe how the examples work out. As the system does not provide any guidance for the exploration, users can spend their time experimenting with new configurations and evaluating the results obtained. Cognitive psychologists, such as Kolb (1984) consider that certain types of learners prefer discovery learning to the traditional tutor led instruction. Schank (1991) cited that discovery learning depends on the users’ will to take the tools and their knowledge background influences what they retrieve and learn. Hence, there might be situations when learners can make assumptions that are not intended by the instructional designers. These are the situations when learning evaluation must be present to ensure that the learning has been taken properly. Differently from decision support or problem-solving CBR applications, instructional systems require more than letting users free to discover the instructional

Page 91


contents (Schank 1995). Instructional applications require methodologies to guide the instructional activities (Briggs 1981; Harrison 1990; Dean 1992; Cardinale 1994).

5.5.2 - Situated learning Situated learning is a theory of knowledge acquisition also accomplished by transferring skills from learning activities to new and real situations (Lave 1990; McLellan 1995). Learning is performed by enabling people to acquire, develop and use their cognitive skills by dealing with authentic or simulated domain activities. The learning occurs with knowledge transfer between tasks and thus depends on the amount of practice and the degree of shared concepts (Brown 1989). The situated learning theory is at the foundation of Schank’s (1995) proposals for a learning-by-doing education. This instructional approach claims that domain knowledge is not a self-contained entity but depends on the context and applications that highlight their meaning. Learning is thus acquired through practice and is linked to the real activities. A general prescription is that learners should be given simulation environments to practise and apply the concepts that are relevant to the apprenticeship. Applications addressing situated learning usually allow learners to appreciate expert behaviour when performing their work. Learners then acquire a similar “teleological” approach for the understanding of a problem situation from real activities. This “teleological” view from the learner allows the creation of references or leads to the relevant aspects of the domain. It is thus important for the instructional applications to provide clues connecting the real situation to the instructional activities. McLellan (1995) cited that this connection may require learners to enter a new culture, distinct from classroom instruction, that entails new ways of reasoning about the domain situations. Examples of this cognitive process can be found in Lave (1991), who provides an analysis of situated learning in five different settings: Yucatec midwives, native tailors, navy quartermasters, meat cutters and alcoholics. In these examples, a gradual acquisition of knowledge and skills takes place as novices learn from experts in the context of everyday activities. These examples emphasise two issues related to the use of supporting cues that are (i) helping replicating a practitioners’ problem solving

Page 92


expertise by indexing features of the environments, and (ii) supporting knowledge transfer between the instruction and the real situation. Situated learning requires authentic activities capable of reflecting the actual thinking of experts when tackling real problems (McLellan 1995). An authentic task in the context of situated learning is a real task. Representations of these tasks, whether using CBR or not, must not only hold the attributes that allow the understanding of the tasks, but also the conception of cues capable of transferring knowledge between similar activities. Another major area of concern is that novices can create these cues supported only by their own personal knowledge background and this could lead to an undesired instruction. In CBR, situated learning requires structuring the instructional activities and selecting special case attributes to support user’s reasoning. It also involves the representation of conditions embodying actual problem solving situations that are to be encountered in the real domain tasks. This learning approach can be useful for training, and “often more effective when nearly independent parts are practised first, before combining them” (McLellan 1996).

5.5.3 - Task centred learning Similarly to the concept of learning by doing, task centred learning focuses on performing a task and acquiring the knowledge involved in its performance. The instructional activity relies on a task to be performed and the learning occurs from the activities surrounding the accomplishment of this task. These activities can be the identification of the skills to be performed, the sequence of the actions to accomplish, the consequence of the decisions made, and the strategies involved in the different ways of performing the task. This learning approach makes learners face a challenging situation as its instructional strategy relies on learners recognising by themselves if there are gaps in their knowledge and skills. When they do, learners should be able to ask the system for advice. CBR applications involving task centred learning should thus allow users to search for similar cases and then apply their skills to the original new task. The number of adventure games available on the market illustrates how interesting applications involving task centred learning can be. These systems provide Page 93


leads and measure achievements as an incentive for users to find more information and reach the end of the game. Some games also encourage users to try different ways of solving problems. Users can also stop the game, save it, and return to it later on, when convenient. Applications involving multimedia tools are also called ‘edutainment’ as they involve a blend of education and entertainment (Agnew 1996; Boyle 1996). Task centred learning involves challenging learners and expecting failures so that they can get further interest in developing the skills to perform the task properly. Difficulties associated to this learning approach regard making the tasks interesting enough to get learners’ attention. Another common concern is to make sure that only the designed instructional goal is achieved. Educators such as Falk (1995), Beattie (1995), Agnew (1996) and Ivers (1997) are still investigating the pedagogic benefits of using this instructional approach in education. A general conclusion is that youngsters in primary schools seem to enjoy “edutainment” task centred applications.

5.5.4 - Goal driven learning Differently from task centred learning, goal driven learning focuses on the setting a goal rather than a challenging task. In AI, psychology and education, a growing body of research supports the view that human learning is naturally a goal driven process (Schank 1995; Boyle 1996). Learning goals specify a state of need or desire to learn that is highly dependent on the user’s motivation. This learning desire is also subject to a timing factor and it is stronger at the moment the users are facing a situation they are not capable to deal with. It means that the best time for learning is at the very moment when the goal emerges (Schank 1995). Research in goal-driven learning involves a range of issues, such as how to make goals arise and the ways that goals influence the broad range of the human learning process. There are also questions about goal driven learning in computer applications. They are for instance related to the types of goals that can influence learning, the domains that could benefit from adopting goal-driven applications, a design of instructional activities that can access goal-driven learning, the technical issues that goal-driven learning applications must address, (Ferguson 1992; Falk 1995; Boyle 1996).

Page 94


In spite of all these possible difficulties, goal driven learning is undoubtedly the learning approach that the majority of instructional computer systems, whether CBR or not, are involved with. CBR instructional applications hold a case repository of past experiences of the domain that can be accessed at any time to satisfy the user’s need for learning. Users thus can interact with the system, searching for a past experience that allows them to acquire the knowledge related to the goal they are trying to achieve. The richness of the learning depends on several factors involved in the CBR design, such as case contents, indexing and domain coverage. The central principle of goal driven learning is that users are involved in an active process, attempting to identify and satisfy their need for information. Experiments show that goals have effects not only over the degree of learning, but also over what people actually learn (Lave 1990; Schank 1995, Brooks, F.P. 1997). The users’ prior knowledge, skills, and capabilities are all involved in exercising their learning capabilities and thus achieve their goal(s). However, the level of usefulness of this exercise will depend on the computer systems’ capabilities to conduct the pursuit of the learning goals.

5.5.5 - Common characteristics A number of learning theories viewing the learner as the centre of the instructional universe have come to the fore in the last few years (Collins 1994). These theories challenge instructional approaches that have long governed educational practice and held the tutor at the core of instruction (Collins 1994, Schank 1995). These ideas are theoretically grounded in the concepts of discovery learning, situated learning, task centred learning and goal driven learning. The instructional strategies reviewed in the previous section share common qualities and attributes that have been highlighted when related to CBR. For instance, discovery learning, situated learning, task centred learning and goal driven learning could all engage users in learning from past experiences. Additionally, all four approaches of instruction have an affective dimension as the learning often takes place from real past experiences that incorporate characteristics of the workplace. Chapter 2 has shown that the best time to learn is when people need this learning for specific work situations. CBR training applications available in the Page 95


workplace that access motivated learners in an as-needed basis have higher chances of succeeding as instructional tools. However, there are no guarantees as to whether the users will actually learn or just rely on the system’s help. An evaluation of the learning can help make users pay more attention to the instructional activities. Although the main propeller in these four instructional approaches is the learners’ will to acquire the instruction, the amount of learning they can get from the computer tool depends on several factors related to the CBR’s broadness and interface (Dearden 1995). Conditions that enhance learning and are common to the four approaches are as follows: The users’ comparisons between the case and the real situation (analogical reasoning) are part of CBR, independently of the instructional strategy adopted by the applications; the learners’ reflection to identify the problem situations and make assumptions on possible solutions is an individual process, even though it strongly relies on the systems’ capabilities; cases that are similar but not equal can make learners use their creativity and make assumptions that can lead to challenging the way things were done before; learners are provided with a variety of different internal processes to base their reasoning on and therefore, they may not be able to face the real situation. Elsom-Cook (1990) cited that systems should vary the amount of guidance given to support learning and gradually decrease the level of support as the user’s need for guidance decreases. This approach is taken into the VECTRA project, as described in Section 7.4. The following section describes six CBR instructional applications and special attention is given to the instructional strategies adopted.

5.6 - Review of CBR instructional applications This section reviews six CBR instructional applications and focuses on their instructional requirements and capabilities. This review provides a source representative of the instructional strategies that were adopted by these applications and shows the state-of-the-art in CBR instruction.

Page 96


5.6.1 - ASK-TOM ASK-TOM (Schank 1991; Ferguson 1992) is part of the ASK systems developed at the Institute for the Learning Sciences (ILS) and was designed as an advice-giving platform that emulates a conversation with an expert. The knowledge domain of ASKTOM is trust bank consulting. Cases are represented as digitised video clips of experts telling stories they faced in real on-job situations. This multimedia case presentation aims at providing an easily understandable and attractive past experience that carries the instructional activity (Ferguson 1992). The system’s interface allows users to describe a problem situation they are facing or want to learn more about. The system can also ask for more specific clues and suggesting the kind of interactions users must have with the system, thus emulating a conversation with an expert. Each video clip contains a textual description of its contents that works as features and allows the retrieval by matching keywords from the descriptions users have inputted. Case descriptions provide a browsing interface where users can find information or facts that conduct them to a particular case. When the system identifies an area of interest to users, by matching their inputted descriptions with the textual case description of the cases, the retrieval is activated and the video clip of the best match will be played. The interface can also challenge the user’s knowledge by asking questions and applying an opportunistic approach for the retrieval of past experiences according to the user’s answers. Past experiences are represented as stories and thus bring up cognitive issues related to story telling, such as: they approach a reminder that story telling is a common instructional activity, they bring users close to a real expert and thus deal with the affective nature of learning, and they bring real expertise to users as they took place at real situations. None of the ASK systems allows case adaptation. The major reason for the lack of adaptation was pointed out by authors such as Burke (1996) and Ferguson (1992) and is the fact that the contents of video clips cannot be changed. Modifying the contents of video clips not only requires the recording of new videos to be added to the system, but it also means that the contents of the new cases have to go through the steps of being described and being featured in the context of the application. Page 97


From a pedagogical viewpoint, users take the instructions following a selfdiscovery approach mixed with situated learning. The coverage of the case repository plays an important role over the instruction. On the other hand, a great number of cases could bring difficulties related to (i) the provision of textual descriptions capable of differentiating the cases within the repository and (ii) the large amount of disk space necessary for the storage of digitised video clips. The pedagogical approach only relies on the users taking the system when they are most receptive to information. This application also investigates the usefulness of video as an instructional media. Disappointingly, no literature was found regarding the evaluation of the learning achievements provided by the applications, specially in comparison to the traditional turning-page instruction provided by ICAL tools or to onjob instructor-led training courses.

5.6.2 - DUSTIN DUSTIN was designed as a language teaching system for non-English speaking employees of a private consulting company who were about to come to the USA for training (Schank 1990). A later release was developed for the teaching of the English language to foreign students that would be joining an academic institution in the USA (Schank 1991). This system performs a simulation of real situations that the students are likely to face when entering the USA to attend their courses. The cases are represented by digitised video clips and, for instance, given a task of checking in at the airport customs, the system retrieves a video clip presenting a receptionist speaking English who greets the user. The user has to type the adequate responses to the receptionist’s questions and If he/she passes this airport customs simulation test, the system then moves on to another task, such as meeting a secretary at the school. The system’s instructional strategy complies with the task centred and goal driven learning approaches and is very similar to that of adventure games, where the users have to properly perform a series of tasks to achieve an important goal. The instructional activities try and catch the user’s attention by using multimedia simulations of real situations. However, there are no guarantees that learners will be able to structure their knowledge about the language. Moreover, language in the

Page 98


system is seen as a tool for achieving goals in specific situations and not as a creative process of expressing personal feelings and requirements. Another concern is that the spoken and written languages have their own aspects and structures that are not addressed by the system’s instructional activities. The system thus works more as a guide to users in identifying how capable they are to deal with entering the USA.

5.6.3 - CREANIMATE This system was developed to provide instruction for children about animal behaviours associated to their anatomies and skills (Edelson 1991; 1993). This application was developed at the ILS (1990) and presents cases in digitised video clips. The cases contain situations displaying an animal behaviour, such as flying for an escape (or for hunting) in its natural habitat and explaining how such an animal uses this capability for survival. The instructional activity relies on the system giving students an opportunity to use their imagination and design a new animal, such as a flying cow. The system initiates a discussion about the reasons why animals fly using illustrations of animals flying in their own environment. The students are then asked to provide a reason for their chosen animal, i.e. a cow to fly, and the system shows how flying would change the things that cows actually do. The retrieval of the video clips is performed over a textual description of the animal’s behaviour and the principle that is illustrated. The system tries to match the user’s input and the textual description of the video clips containing the animal and the associated behaviour. Video clips containing the instructions are thus retrieved to show the associated animal behaviour. The instructional strategy is mainly discovery learning, but the system is also capable of challenging users to achieve goals by asking questions, such as “how do giraffes use their long necks to help their survival?” Answers to this question are also provided by videos displaying the animal using such behaviour in their natural habitat, i.e. a video clip of a giraffe using its long neck to eat on top of small trees and bushes. The system also lets learners specify instructional goals by themselves, such as creating a flying giraffe and how this animal could use this new feature to improve its survival.

Page 99


This application emphasises case featuring, indexing and retrieval strategies for the achievement of instructional goals. The digitised video clips integrated with the task environment for learning guidance and indexing make this application cover a wide range of media for the delivery of the instruction. The instructional events enhance retention, promote knowledge transfer, and illustrate general principles that can make the system a useful complement to instruction in the classroom (Edelson 1993).

5.6.4 - SPIEL (YELLO) SPIEL stands for Story Producer for IntEractive Learning and is a system that investigates strategies for case retrieval for instructional purposes (Burke 1996). The system was designed for users who are learning the social skills required by diplomatic or business related jobs. The SPIEL architecture was later used to develop the YELLO system that provides instruction for employees selling yellow pages advertising. Cases in the SPIEL architecture are digitised video clips of practitioners telling stories about their on-job experiences. The system uses an architecture that was called by the authors Guided Social Simulation (GuSS). The guidance works like an experienced tutor watching over the users’ interactions and retrieving stories that are relevant to them. This relevance is established by monitoring the stream of actions performed by the users over the interaction. The SPIEL architecture allows the creation of applications where the developers’ task is reduced to the gathering and video-recording of the experts telling relevant onjob stories. The stories also require a textual description of their contents and must contain keywords for further retrieval. This case description also provides a previous quick read of the video contents prior to retrieval. This helps to avoid unnecessary viewing of the whole video clip retrieved, as the systematic viewing of the whole video, so as to get its instructional meaning, can sometimes be annoying. The SPIEL architecture allows neither case-adaptation nor the input of new cases changing the course of the story retrieval. Similarly to the story-telling systems previously described, the SPIEL provides instruction following the discovery and task centred learning approaches. The authors cite that users find video clips quite useful

Page 100


and stimulating to take the tool. However, no learning evaluation or comparison with other instructional alternatives has been reported.

5.6.5 - SCI-ED The SCIence EDucation advisor (Kolodner 1991, 1993; Chandler 1993) is a CBR application that assists teachers with the creation, critiquing and evaluation of lecture notes for elementary science education. The system provides novice teachers with past experiences of classroom experts giving advice to novice teachers on how to plan their lessons. These past experiences are presented in video clips and use the story telling approach as a decision aid. The video clips contain stories of difficult situations that are likely to happen when a particular classroom teaching approach is taken. The instructional strategy involves discovery learning and the system’s interface allows users to describe a problem regarding the lecture planning. The system searches for descriptions of video clips addressing user input and shows users the case that best matches their input. Similarly to other story-telling systems developed at the ILS, SCI-ED cases contain experts telling stories of real on-job situations. Although the system does not contain any special instructional strategy besides discovery learning, it provides help on an as-needed basis for users having a lesson to prepare. The authors cited that although stories can play an important role for instruction, the gathering and recording of past stories can be quite difficult. The gathering of past stories involved the traditional CBR approach of inviting experts to tell stories relevant to the applications domain. However, issues have been raised regarding difficulties that can emerge during this process, such as the stories were not complete or detailed enough to provide the required instruction. Moreover, stories can be told using domain jargon without the person in charge of gathering them realising it as he/ she can also be a domain expert. This may lead to the necessity for further revision.

5.6.6 - CADI CADI stands for Cardiac Auscultation Diagnosis Instructor and is a CBR teaching cardiac auscultation that plays digitised sounds emulating heart beatings

Page 101


(Fenstermacher∗). Users are instructed by figuring out the underlying physiology that produces certain sounds. The author claims that a student who is having trouble with a patient could also use the system and access similar heart beatings. In this situation, the user would have to describe the case since the system is not capable of performing retrieval based on heartbeat sounds. The system’s interface can either guide users through the instructional activities or letting them use the tool in a discovery learning approach. The former simulates an expert who introduces patients and challenges students to diagnose the problem the patient might have. CADI uses multimedia not only to play the sounds but also to present graphs such as electrocardiogram and echocardiogram (a device that uses ultrasound and displays the structures within the heart) as additional help to diagnosing heart diseases. The author cited that the multimedia simulate the exams and equipment available for diagnosing heart diseases. These facilities are also motivating students to take the tool. The instructional strategy addresses discovery, situated, task centred and goal driven learning. The system provides instructional activities either by following the instructions of an expert or allowing users to search for a case containing an aspect that they want to reinforce learning in. The system does not hold capabilities to evaluate learner performance.

5.6.7 - A brief recap on the applications reviewed The systems previously described as well as most instructional CBR applications found in the literature could be included into a category classified by Dean (1992) as "learning by exploring software". Interacting with these systems, users are able to search for answers to their question. Even when the case repository does not contain the right answer, users can learn what the answer would be if the question was different. The number of publications reporting CBR applications that address instructional issues is growing. However, applications addressing training are still not easily found. From the review of CBR related instruction above, only the CADI makes ∗

Dr. Kurt D. Fenstermacher keeps a web site at the Artificial Intelligence Laboratory at the University of Chicago describing the CADI system (last visited 12/01/98).

Page 102


reference to guidance through the instruction. The ASK systems and the SPIEL architecture use a metaphor where users are having an advisory conversation with an expert who opportunistically provides past experiences. This approach does not provide any tutoring strategy, but guidelines for case retrieval. Storytelling is undoubtedly a natural instructional and communication medium that has been used by man for centuries. Stories can give meaning to several types of learning that “could otherwise be perceived as dry and uninteresting facts” (Schank 1995). However, the instructional capabilities of story telling are limited. The course of action adopted by the CADY system contains past cases as past examples that cover a wide range of instructional strategies. Cases modelled as simulations of on-job situations rather than stories being told can, even though this may imply a more elaborated modelling process, supply instructional strategies addressing a wider range of learning requirements. Another issue is the use of digitised multimedia files enhancing the learning by providing attractive learning environments capable of getting the user’s attention. However, file contents cannot be accessed and the systems thus lack capabilities for case adaptation. Even when multimedia cases are modelled as small pieces, as in the CADI system, each piece requires special tools for editing its contents and, in the end, these digitised files can only be featured for retrieval. The applications reviewed seem more involved with the instructional capabilities that CBR can hold than to test whether CBR can be more effective than traditional CBT or ICAT (see Section 3.7). Moreover, none of the applications reviewed has performed any evaluation of instructional capabilities. In any case, all these tools can somehow help alleviate the demand for experts’ time.

5.7 - Case-based training Having a broad, well-indexed set of cases is what differentiates the expert from the textbook-trained novice. Roger Schank (1995a)

The previous chapter shows that past experiences can be useful for a variety of instructional tasks. For instance, applications of CBR can be found in such domains as planning, design, diagnoses, decision-support and problem solving. Individuals often use past experiences prior to making decisions and the amount of experience related to Page 103


their job often makes the difference between experts and novices. However, as shown in Section 2.5, general instruction and training have differences that reflect on the development of computer applications. As an example of real training, novices on scaffolding inspection are usually taken to real construction sites. As part of the course, different scaffold structures and sites are visited. Pictures and films are also part of the training course. The instructor guides the apprenticeship and novices learn by watching the trainer in action, listening to the instructions, and participating by asking questions. Based on the students’ behaviour and questions, the trainer can either proceed with the planned instructional activities or pause to give remedial instruction. After the training course, throughout the journey from novice to expert, the trainees will see several scaffolds in different sites and build their own apprenticeship from these new experiences. In work situations, it is important to know cases and skills. Thus, theories from books or even stories that have not been personally experienced can contribute to building a body of knowledge in the professionals’ mind. This issue is emphasised by Schank’s (1995), who cited that the kind of education provided at schools is difficult to access at work. The author cited that today's educational system “overemphasises facts, under emphasises skills, and grossly neglects such processes as communication and human relations”. Traditional CBR applications rely on the ability to properly retrieve cases containing the information users need. Although this ability can make users learn by having the opportunity to describe, retrieve, understand, and readapt cases, CBR applications rarely cover all of the instructional steps presented in Figure 5.1 or provide a pedagogical methodology structuring the instructional activities. Although they are capable of providing real time instruction, the CBR applications discussed in the previous section contain instructional activities and guidelines for retrieval but do not fully satisfy training requirements. CBT systems involve a context capable of guiding the users’ apprenticeship and provide on-job situations that trainees can exercise with. Training systems applying the CBR working cycle have to cope with this situation, i.e., they have to supply guidelines to assist trainees through the skills to be achieved and help them retrieve simulations of real onjob past experiences. Page 104


Training applications must enable learners to access real work situations, performing learn-by-doing activities within simulated environments that replicate real working situations. Building such simulations is still not an easy task. To provide realistic simulations of social and physical environments, it is necessary to have tools that facilitate the creation of such environments. This project uses VR to provide the simulation environment and the following chapter discusses the role that this technology can play for training applications and special attention is given to CBR applications holding a repository of VR cases.

5.8 - Synthesis of the chapter The CBR working cycle holds a natural way of learning. An example of how natural the CBR working cycle is in human reasoning is given by Schank (1995) when the author cites that novices usually describe a problem to an expert who can be reminded of a past experience. Another aspect of CBR learning is lazy learning∗ , i.e., even when applications do not aim to provide instruction, users can learn about the domain, by the simple fact of reviewing past experiences. There are CBR applications where the provision of instruction is the main objective. The two main instructional approaches supported by CBR instruction are (i) discovery learning and (ii) task centred learning. The former can be achieved by the CBR cycle passively allowing learners to retrieve the cases and build their own conclusions. The latter requires users to perform a task, benefiting from a methodology guiding the instructional activities. A relevant instructional CBR application reviewed in this chapter is the CADY system. The system deals with training for cardiac auscultation and offers learners the opportunity to practise from simulations of on-job situations under the guidance of experts. The system uses multimedia for case representation, i.e. past cases of cardiac auscultation and covers most of the steps in Gagne’s (1992) instructional hierarchy. Should an evaluation of users’ learning and feedback on their performances be included, the application would cover all the aspects of Gagne’s (1992) instructional theory.

∗

Lazy learning is also a concept of machine learning research and deals with algorithms that rely on users to improve computer systems’ capabilities.

Page 105


Unfortunately, the CBR applications reviewed in this chapter have not conducted an evaluation of user’s learning in comparison to other forms of instruction. Rather, CBR developers seem more interested in uncovering the instructional traits that can be achieved from the design of case-based instruction. CBR limits the instructional activities to the retrieval of cases and to the way cases are presented to the users. Although it represents a limitation, it seems easier to develop applications where the knowledge modelling is limited to the gathering of domain past experiences. Moreover, CBR involves an investigation of the cognitive process that includes not only the instructional capabilities of past experiences but also the clues that allow retrieval in human memory. From an instructional viewpoint, three major principles can be derived from the reasoning from past experiences and CBR instruction. The first is that representing expertise as past experiences and using them to facilitate the learning of new information is a natural process of human cognition. The second is that the usefulness of the past experiences depends on the ability to keep them and locate them in memory, and computers are very powerful when it comes to keeping records of great amounts of information. Finally, in instructional activities, the mechanisms triggering the retrieval of experts’ past memories are not very different from the case descriptions that allow case retrieval from the CBR repository. Thus, failures caused by recalling the wrong memory or by facing a situation never encountered before are likely to happen to both humans and machines (Gentner 1989; Brooks, L.R. 1989, Kolodner 1993).

Page 106

Chapter 6 – Virtual reality case representation

6.1 - Overview

The previous chapter shows that case representation plays a major role in CBR especially for instructional applications, where user learning depends on their understanding of the case contents. In order to enrich case representation, CBR developers employ multimedia that are not only improving the display of case contents, but are also capable of increasing the user’s interest in the instructional tool. Apart from the instructional strategies in CBT, users are offered with an entertaining instructional environment. This chapter introduces Virtual Reality (VR) as an interface for case representation. VR provides an interactive interface for the information contained in the cases and plays an instructional role by allowing users to experience the 3D simulations of past experiences. VR provides the feeling of ‘being there’ in the simulations with a degree of freedom of movement and a safety that is rarely possible in the real equivalent training experience.

Page 108


6.2 - VR: from the labs to the industry VR technology refers to the “use of 3-D displays and interaction devices to explore real-time computer-generated environments” (Brooks, F.P. 1993). VR was a dream long before the creation of computers that allowed it to become a reality (Rheingold 1992; Earnshaw 1993). The first practical results of VR technology were reported in the early 1960’s and dealt with experiments on 3D visualisation through head mounted displays (Sutherland 1965). Until the late 1980’s, military and secret government agencies (Larijani 1994) did not divulge VR achievements to the public. The first VR project made public was developed by NASA (Larijani 1994) in 1985 and by the end of the 1980’s, a number of applications were in use for domains such as training, strategic military defence, and aircraft design (Loftin 1987, Chorafas 1995). Rheingold (1992) cited that in the late 1980s VR technologies including devices such as data-gloves, body-data-suit, and Headsets were already being used in the NASA labs. Some of these projects were even employing head-mounted displays capable of stereo imaging. The same author also cited that these NASA applications had one point in common, i.e. the need for powerful and expensive computers to handle the virtual worlds and the devices for interaction. By the early 1990s, VPL Research Inc. launched SWIVEL, the first desktop commercial tool for the development of VR worlds. This tool provided facilities for 3D navigation, support for data-gloves and head mounted display that were running on expensive workstations. SWIVEL was soon followed by commercial tools such as Superscape (Superscape VR Plc.), WorldToolKit (Sense8 Corp.) and VREAM (Virtual Reality drEAM by Platinun Technology Inc.). Despite the launch of these tools, only a few applications left the research laboratories (Brooks, F.P., 1993). The difficulty in building the VR worlds requiring interdisciplinary computer professionals and the limited capabilities of desktop computers in the early 1990s are reasons behind the slow entrance of VR in the commercial arena (Bryson 1993; Wexelblat 1993). Nonetheless, applications in flight simulations, special effects in the film industry, aeroplane design and medical simulations did reach the operational and commercial stages (Brooks, F.P. 1993). Once again, the cost of VR limited its use for

Page 109


tasks where the overall budget was large enough to accommodate the expenses generated by the use VR. Surveys of VR activities in the UK conducted by Howard (1996) and Stapleton (1997) show that, during the 1990’s, University based-research was quite successful in attracting governmental funds for VR research. Nowadays, many Universities in the UK and USA have a VR research centre. Most of these academic research centres benefit from industrial links and funding. Despite the valuable VR research and development accomplished by academia, most technological advances are actually coming from private and governmental labs (Larijani 1994). Larijani (1994) stated that results coming from Universities usually involve extensive feasibility studies, exhaustive reporting and bureaucratic paper work that detracts from the time researchers can actually dedicate to development. As results from private research institutions are not easily shared, it is difficult to identify the capabilities of the latest VR applications. It is believed that current VR developments in private research labs and military institutions go far beyond the state-of-the-art found in research publications. Judging from the latest references, VR is now at a stage where it is finding acceptance in industrial areas such as building design, car manufacturing and the maintenance of manufacturing equipment (Earnshaw 1993; Harrison 1996; Dai 1997). For instance, companies such as General Motors, Ford, Mercedes-Benz, and Rolls Royce, are running their own VR labs (Drews 1997). Nonetheless, few other manufacturing sectors apply VR operationally as it is still considered to be a costly and complex undertaking and thus remains limited to applications where the budget allocated is large enough to cover the costs of the VR technology (Harrison 1996; Seidel 1997; Dai 1997). Authors such as Wexelblat (1993), Brooks, F.P. (1993), and Vacca (1996) cite that VR has an important role to play in improving communications between the groups involved in the initial stages of conception and design of new products. VR could provide perspective views for sketches, allowing an interactive visualisation of the various parts of a new product being considered. More recent references such as Harrison (1996), Watson, M. (1996) and Seidel (1997) cited that VR is a technology that

Page 110


can be used for the whole process of building new products. Providing training for the work-force, reducing prototyping costs and performing simulations that could be used for future improvements are also tasks that have much to gain from VR (Stuart 1996; Harrison 1996; Drews 1997). VR is still at an embryonic stage in the industry (Harrison 1996; Dai 1997; Seidel 1997). Nonetheless, authors such as Chorafas (1995), Watson, M. (1996) and Drews (1997) predict that in the near future VR will be used by the industry wherever it is proven to be helpful. VR, integrated with other computer technologies such as databases, decision-support systems, CAD, and AI, may thus soon become an industrial reality. Computer technologies handling 3D images and animations have already reached a stage where non-specialists can rely on development tools to build applications on reasonably priced personal computers. Advances in VR software and the increasing power of standard personal computers shall shortly allow VR to be available on a broader basis, in both industrial and home-based applications (Harrison 1996; Porter, L.R. 1997).

6.3 - VR interface capabilities The interface is the application. Pimentel, K. and Teixeira, K. 1994

VR is a 3-D computer interface that allows users to interact with the simulated environments (Rheingold 1992; Pimentel 1994). This interaction can take several forms and use several hardware devices, such as head-mounted displays (HMD), data-gloves, wands, force-feedback joysticks, space-balls, and body-suits. Depending on these devices, the degree of interaction and embodiment in the VR world differs. The interaction with the simulated VR environments ranges from a simple walk-through the simulation that is only displayed on a computer screen, to a fully immersive interaction where user movements are fully tracked inside the VR environment. The simpler the interaction with the VR worl, the less resources are required for the development of applications (Rheingold 1992). Note that the resources necessary for the development of VR systems range from hardware to software to programmers and

Page 111


designers. From a cost perspective, a stereo head-mounted display alone can, for instance, cost more than the whole equipment required to run a simple VR world on a computer screen∗. The following sub-sections describe four approaches for user interaction with the VR worlds.

6.3.1 - Interactive VR Interactive VR is also called “desktop VR” or “fish-tank VR” and deals with applications where the virtual world is displayed over a flat computer screen or as an image projected over a larger flat screen. Authors such as Arthur (1993) and Hendrix (1995) mention that interactive VR works like watching a water tank with fish inside through a glass window, very much like an aquarium. Users can view the virtual world displayed on a computer monitor and interact with it through conventional devices such as the keyboard and mouse. Special devices such as the track-ball or space-mouse may also be used to provide 6D movements. The 3-D visual effects are achieved thanks to the capability of the VR package to map the surface of the virtual objects and apply directional lightening over them (Hollands 1996). This lightening can either be automatically controlled following the user’s viewpoint or set by developers in different locations of the virtual world. Although this lightening control is the most common form of interactive VR, some systems allow stereo vision, relying on special software and displays. Stereo displays present the images slightly offset to each eye and require simple stereo glasses fusing the display into a single image with depth in the human brain. Interactive VR is the most popular choice for commercial, industrial and academic applications (Pimentel 1994). The functionality of VR is available at the user’s fingertips, without having to resort to cumbersome and expensive head mounted displays or powerful and costly computers. Moreover, the creation of the virtual worlds is also facilitated since designers do not have to worry about tracking the user’s view within the virtual worlds, but only about controlling the virtual world display on the flat screen.

∗

A useful list of VR products, manufacturers and prices has been maintained by Ian Feldberg at the following URL: (last visited 10/11/97).

Page 112


6.3.2 - Immersive VR The Immersive VR interface allows interaction with the simulated environment rather than only standing aside and observing it. Immersive VR presents a 3-D interaction where the virtual world surrounds the users. Following the fish-tank example given in the previous section, in immersive VR, the user is scuba diving inside the aquarium rather than only watching it through the glass window (Rheingold 1992; Arthur 1993; Hendrix 1995). This interaction uses more of the sensory organs of the users as it involves their visual, auditory, tactile and kinaesthetic senses. Movements of the user operator are tracked, relayed to the computer system, and generated by the computer displays. The user's body itself may become part of the virtual world and the participant operates in an extended virtual space created by the interaction between his/ her human perception and the computer generated displays. The most popular form of Immersive VR allows users to view the virtual worlds through a head-mounted display. Sensors are used to track the user’s head movements, allowing the computer to generate the appropriate image. Users are then surrounded by or “immersed” in the virtual world. Special devices, designed specifically for this kind of VR interaction, such as the data glove and body-suits, can also be used for the interaction with the user. However, these devices are still expensive and therefore, most commercial and most academic applications do not use them (Howard 1996; Stapleton 1997).

6.3.3 - Augmented VR Augmented reality refers to a state where the users are in the real world and are simultaneously able to interact with the virtual world (Bajura 1995; Azuma 1997). Unlike the immersive interface, where the virtual worlds replace the real world, AR systems enhance the real world by superposing information onto it. In fact, the users can see the real world around them, but with the virtual world superimposed onto it. AR thus works as a supplement to the real world and allows users to view both the real and virtual worlds at the same time. The most usual way to achieve AR is with special see-through HMDs or headworn displays. These devices display the virtual graphic images in front of the user’s

Page 113


eyes while allowing the light from the real world to come through (Webster 1996). A combination of the real and virtual worlds is thus drawn in front of the user’s eyes. These HMD can also be designed to overlay the naturally occurring sounds of the background or conversation. AR is seen by authors such as Bajura (1995), Webster (1996) and Azuma (1997) as the next generation of human-computer interface technology. Some applications are already using this technology that seems ideal for design, repair and construction tasks. For instance, to build or repair complex equipment, users could see the real objects and a real-time simulation of the construction or repair process superimposed onto them. Oral explanations of what needs to be done and where are also available to enhance the interaction (Caudell 1992; Hriber 1993, Webster 1996; Azuma 1997).

6.3.4 - Networked VR Networked VR enables the experience of a multi-user interactive environment where users can be distributed over different places and connected via computer networks. Rather than experiencing a one-to-one user-VR interaction, as described in the previous sections, two or more users can observe and interact with the same VR world. The VR world where the interaction takes place is also called “cyberspace” and allows users to join the interaction simultaneously from different computers. There are two main ways of achieving networked VR: (i) the users interact with the VR world from different computers, connected by a network, or (ii) the virtual world is represented by only one computer and is shared by all the users. When comparing networked VR with the other forms of interaction previously mentioned, the task of designing the virtual worlds remains the same. Nonetheless, in networked VR, the computer has to deal with tracking each user within the VR worlds and this is a task that can be quite demanding in terms of computational processing. References such as Salomon (1991), Alluisi (1990), Psotka, (1993) and Moshell (1993) highlight the potential of networked VR for instructional applications. For instance, instructors or experts can interact with novices located in remote sites and share the same experience. This technology also allows learners to work together over the same virtual task, with the supervision of the (visible or invisible) instructor.

Page 114


Networked VR can allow the three types of interactions presented in the previous sections. However, the greater the immersion level, the higher the demand for computational processing. Obviously, there are application domains that are more suitable for a particular form of VR interaction and each approach presents its own pros and cons. A brief review of the advantages and difficulties of each form of interaction is presented in the next section

6.3.5 - A recap on the interaction modes. The previous section presented four different types of interaction with virtual worlds: interactive, immersive, augmented and networked VR. Networked VR, differently from the other three interaction modes does not refer to the type of visual interaction, but to the fact that different users share the same virtual world. Networked VR can thus use any of the three other forms of interaction. The degree of complexity of the design of the virtual worlds and the requirements for software, hardware and developers’ expertise increase in parallel to the degree of VR interaction, i.e. they increase from the simplest interactive to the most immersive VR. The degree of complexity for Networked VR also increases the complexity for development of applications. Apart from the level of complexity, the three modes of interaction share common factors involved in the building tasks of their virtual worlds. A list of these common aspects is presented below: •

objects’ dimensions, locations and other general and domain specific attributes are needed prior to the implementation of the VR worlds and there is a lack of techniques allowing modelling of these domain requirements;

•

different computer monitors, graphics cards and other hardware devices contain particular settings that can cause unexpected differences when interacting with the virtual worlds;

•

low graphics and processing performance can make the VR interaction uncomfortable for the users and this fact can be crucial for applications using headmounted displays or running on the Internet;

•

the sensitivity calibration of the controllers over the interaction involves tasks such as setting the maximum frame rate, speed of movement and angular step and these Page 115


tasks must be performed at various stages before, during and after the development of the virtual worlds; •

the interface should be conceived so as to take users to well known places since it is easy to get lost in the VR worlds by lack of control over the navigation;

•

the VR interface has to be appealing and comfortable for the eyes and other senses especially for applications where users have to spend several hours using the systems. Immersive VR can rely on a range of hardware devices providing the immersion

effect. The quality and the degree of immersion into the VR worlds vary as a result of the hardware chosen for the interaction. Immersive interaction adds other factors that the designers of VR worlds must take into account, such as: •

devices tracking the user’s viewpoint and positioning in the virtual worlds are still expensive, not very accurate and involve difficulties for calibration;

•

software and hardware requirements for developing immersive VR and navigating through it bring difficulties for the set up that comprise the whole life-cycle of the VR applications;

•

the work necessary to obtain quality images with the current head-mounted display technologies is still quite cumbersome, leading to a lower quality of the visual interaction as designers avoid complex image definition and textures;

•

researchers such as Chung (1990), Rushton (1993) and Slater (1993) cited that people often feel uncomfortable, sick or lost when their senses are immersed into a virtual environment and there are also gender and age∗related influences. One of the arguments justifying augmented reality is its capability to provide an

interaction with the VR worlds without losing references in the real world. However, augmented reality also plays its own influence over the VR experience, requiring designers to deal with issues such as:

∗

VR psychology is a field that covers issues from 3D design to the effects of VR over the life of users. A recommended reference covering age and gender issues is The Journal of Artificial Societies and Social Simulation (Last visited 12/01/98).

Page 116


•

the hardware and software set up for controlling the navigation and the tracking of user’s location taking consideration of the position and the scale of the augmented images in relation with the real world is a time consuming task;

•

the high demand for the tracking of user movements leads to the need for devices that are accurate, expensive and require powerful machines to handle;

•

the difficulty in scaling the user’s viewpoint in relation with the VR objects from different distances and in correctly shaping them so as to provide the right sensory feeling of space is still a challenge to VR developers (Rushton 1993);

•

the quality and resolution of the images projected in front of the user’s eyes are still not very accurate and are even worse than those provided by HMD in immersive interaction; Networked VR also brings its own aspects to the design of virtual worlds and an

area of major concern is the fact that each moving object, animation, walk-through, collision as well as other aspects related to the VR go through the network, thus overloading the communication lines. Other aspects involving the design of networked VR are: •

technical problems related to the network communication cover a range of aspects such as the achievement of synchronisation between sites, differences between the hardware used to run the VR worlds in each location and inconsistencies caused by communication delays;

•

a common technique of networked VR relies on each independent computer maintaining local copies of the world objects and this is an operation that brings difficulties related to the integration of the real time objects, i.e. problems may emerge in the indexing of objects and their hierarchies for each location;

•

time synchronisation is usually achieved by using a global clock and the setting of the different hardware devices adds difficulties to set up the navigation; The factors listed in this section represent a threat to the quality of the

simulation that VR technology can achieve. In spite of these difficulties, VR research and applications are growing all over the world and covering a wide range of application domains. Hardware devices increasing the quality of the VR simulations are challenges for VR research. For instance, devices providing tactile sensations such Page 117


as pressurised gloves, force-feedback joysticks, gloves producing sounds when objects are touched and preventing motion through the objects in the visual range are all possible subjects for VR research. There are interaction modes relying on navigation devices capable of providing quality interaction with the VR worlds. However, their effect over learning and training is still largely unexplored and unknown (Psotka 19933). Further details on this issue are provided in the following section.

6.4 - VR instruction “It may be that all human beings have the same perception of space at the biological level of perception. But certainly every society uses its space differently, both technologically and artistically” (Bolter 1986)

The potential of VR for instruction is limitless and it is difficult to encounter domains that could not benefit from VR instruction (Rheingold 1992; Psotka 1993; Bricken 1993). Learning topics such as languages, car driving, and the use of complex equipment can all enjoy the benefits of VR (Pantelidis 1993; Moshell 1994; Seidel 1997). For instance, who would not like to have a virtual instructor sitting next to the novice and capable of explaining how to proceed with things? Very few people would though prefer to read instruction manuals. VR is a technology capable of enhancing the pursuit of knowledge and creating enthusiasm with the way instruction is provided (Larijani 1994; Moshell 1994; Earnshaw 1993). Actually, most of the applications involving VR are somehow related to either providing instruction or simulations that help reasoning over a decision to be taken. Since VR was first referenced as a technology, its use has been mainly related to the field of instructional simulations (Rheingold 1992; Larijani 1994; Seidel 1997). VR instructional capabilities cover aspects ranging from providing visually simulated pedagogical methodologies to working as a virtual laboratory that allows learners to try various courses of actions and thus stimulate creative thinking. Issues related to the potential of VR instruction are listed below: •

virtual “laboratories” can be cost-effective and constructed in an as-needed basis alternative to instruction;

3

Paper available from

Page 118


•

virtual environments allow developers to apply different instructional strategies, ranging from letting users perform discovery learning to performing a guided simulation of learning-by-doing activities;

•

VR technology can bring a degree of realism to the simulation and thus to the instructional actions that fit domain requirements;

•

VR provides environments that are not simply simulating a course of action, but that also allow users to interact with it and intervene with the results.

•

VR instructional activities can include written, spoken, visual, and motor components that can all be presented simultaneously;

•

instruction has much to do with learners enjoying their courses and VR instruction is capable of combining entertainment with instruction; In spite of the advantages that VR can provide for instruction, there are limits

that one must be aware of. For instance, experiencing the physical world simulation is at the core of the VR technology. This combination of simulation with learning by-doing activities is one of the most recommended ways of learning (Schank 1995). However, developers must not assume that the instruction will be successfully encoded into the users’ long term memory and therefore properly learned simply because a virtual environment is present. Another easy assumption to make regards the degree of VR interaction and the quality of the instructional activities provided. There is evidence that immersive VR, as opposed to simply viewing the virtual world from outside and from different viewpoints, does not necessarily improve the understanding of the domain modelled (Slater 1993; Davis 1996). Slater (1993) presented an evaluation of learning chemical combinations where students performed better using interactive VR rather than immersive. Therefore, it should not be assumed that the use of more resourceful forms of VR such as immersive, augmented, and networked ensure the provision of a better understanding of the instructional activities than simply viewing them on the computer screen. Unfortunately, the literature review on VR instruction has not provided many evaluations of the effectiveness of the different interaction technologies over the

Page 119


learning process. It is expected that users of different age, gender and learning preferences will achieve different results over VR instructional packages. There is, nonetheless, a general consensus indicating that instructional activities involving VR technology can be entertaining. The future role of VR education is uncertain and a question has been raised about the influence that general computer-based instruction can play over the educational system as a whole (Bricken 1993; Harrison 1996; Seidel 1997). With the increasing capabilities of hardware and software providing education and the growing number of people using computers for work, computer-based education is representing a threat to the current status quo of education. For centuries, education has been culturally associated with going to school and joining classmates The whole educational system has been built around the institution of the school and of teachers providing instruction. Nowadays, professionals involved in higher education in Universities already perform more and more of their work with the help of computers. It is expected that, soon, students at a high level education, mainly involved in individual research such as doctoral and masters, will no longer need to attend classes. Obviously, VR is not the only computer technology threatening traditional education. However, the 3D simulation and the learning-by-doing activities allowed by VR represent a push for general computer-based instruction. Further details on the role that VR can play over the human cognitive process are discussed in the following section.

6.5 - Visualisation and memory recall “Images must be lively, active, striking, charged with emotional affects so that they may pass through the door of the storehouse of memory… however, we need to ask ourselves what would constitute the lively, active, striking and emotionally charged equivalents for our own time”. (Yates, 1966)

In his book “Art of Memory”, Yates (1966) describes the role of images and spatial descriptions over human memories and learning. Although the author does not refer to the VR technology, his work offers support for the use of visual simulations of past experiences in instructional applications. Yates (1966) cited that people often remember things in the context of place, even when there is no relevant connection between the thing remembered and the place where it happened.

Page 120


An example provided by Yates (1966) involves a vehicle accident where a survivor was able to recognise the victims only by recalling the places where they had been seated. Spatial environments and visual memories play an important role in triggering memory recall and in human learning. This potential has been recognised by educators and computer-based instructional references mention that a picture can be worth a thousand words. The human 3D faculty bares implications for the instructional activities as pointed by Yates (1966) and summarised below. •

Imaginary or real structures – instruction in domains, such as inspection of scaffolds, a textual description of the structure would never provide the same level of remembrance than its visual representation. A textual description of a scaffold would force learners to create a mental picture of the structure. VR would immediately provide the picture of the structure described. VR would also highlight and simulate instructional issues, allowing users to create their own visual cues to help them retain and recall the instruction.

•

Concreteness and memorability - for the majority of learners, recalling a concrete view of a space is easier than recalling abstract symbols (such as pieces of language). This human preference implies that memory storage is a key factor for learning and that it is performed more easily with the spatial support of the virtual worlds than from abstract descriptions.

•

Taxonomies for thought - spaces have coherence and logic that can be used to connect one idea to another and VR spatial simulations can provide models helping users’ mnemonic thoughts. These visual mental models also cover issues associated to user’s movements, physical constraints and sequences of events that are part of the standard VR instructional applications.

•

Representing realities - although some aspects of realities only exist inside the mind of an individual, VR representation based on a single observer’s view can be used as a place of mediation for various individuals. In this sense, VR can go further than the simple description of real models as it provides places where whatever is there can be seen as real.

•

Detection of motion - movement is a strong element in visual perception and users can gain much information from it (Kolb 1984; Gagne 1992). The current VR

Page 121


technology supports effects related to movement, such as clash detection, associated noises and gravity that could play an important role for instructional activities. •

Visual-spatial - individuals naturally think in terms of the physical space, as professionals such as architects, civil engineers, decorators and drivers do. Individuals thus become aware of their environment by making use of drawings, maps, and mental representations. VR can be used to represent these spatial models and even help novices “see” through the eyes of experienced professionals. Current VR technologies can support various aspects of memory recall that are

associated to spatial simulations. However, there are still aspects that VR cannot yet help with, such as temperature, aroma, or tactile textures. Another aspect of VR is that it can work as a filter for space representation and display only the information that is relevant to the domain. Other aspects of reality that can be simulated by VR applications supported by VR development tools are presented in Appendix 2. Further details on the role of the VR interface for case representation are discussed in the following section.

6.6 - VR interface for CBR Human-computer interaction and interfaces have been a research area since the creation of computers. This research area involves issues that range from the capabilities of hardware devices, such as high fidelity images on head-mounted displays, to the mysterious human-software psychology. This section focuses on the VR interface capabilities and the benefits they can bring to the CBR working cycle (see Section 4.3). A major aspect supporting the use of a VR interface for CBR is its use to simulate past experiences of experts allowing users to enjoy a 3D interaction with the VR cases. VR can transform a worded description of a past experience into a space where users are guided through an interactive adventure story. VR instructional applications allow an interaction that not only simulates the instructional activities but also holds the level of difficulty that the real training situations demand. VR is not simply an interface, but a whole computer environment that contains a programming language for the development and handling of the VR environments. VR facilities allowing interaction, human perception, sound effects, and 3D navigation, Page 122


have a software architecture supporting the development of applications. For instance, VR development packages currently available on the market rely on the object-oriented programming architecture. This object-oriented architecture is considered an evolution that has made the programming task easier (King 1988; Watson, M. 1996). The also common Virtual Reality Modelling Language (VRML) can be seen as an evolution of object-oriented architecture focused on the specific needs of VR. VR development tools such as VR Creator (from Platinum Technology), V*Realm Builder (from Integrated Data Systems) and Pioneer Pro (from Caligari Pioneer) already provide WYSIWYG (What You See Is What You Get) tools allowing an easy development of VR worlds and libraries of objects developed in VRML. These VR programming languages and architectures can play a role over the whole CBR working process and should not be limited to the representation of cases. In this thesis, VR is applied over the whole working cycle of CBR and provides the interface for the user’s input of case descriptions and for the mechanism for case matching and retrieval. These interfaces rely on algorithms written with the same programming language that was used to build the VR worlds. Chapter 7 describes the role of VR regarding the application domain and the development of the VECTRA prototype. The following sub-sections describe the general role of VR for tasks related to CBR such as case representation, featuring, retrieval and adaptation.

6.6.1 - VR case contents VR capabilities to represent past cases are only limited by the current computer technologies, the dedication and skills of professionals and developers. Every day, hardware and software developments are closing the gap between the professional effort and the complexity in building the virtual worlds. It is expected that for the representation of past experiences, the greater the contents and the knowledge embedded in a case, the greater its applicability and usefulness for users. On the other hand, rich case representation increases the demand for hardware as well as the time and skills from VR designers. In a VR instructional application, the focus is on the instructional capabilities, independently of how immersive and entertaining the application could be for the final

Page 123


users. The instructional effectiveness is thus part of the development process of applications and developers must focus their attention on the instructional activities that comprise domain knowledge. For instance, a simple character input such as the name “Santa Claus” can trigger memory recall for a user that could help save thousands of data inputs forming the picture of the good old man who wears red clothes and brings presents at Christmas. Modelling techniques can use pages of information to create a model of Santa Claus that is already present in the mind of nearly every adult. In an instructional application for children, a picture showing what Santa Claus looks like could be necessary, but for adult education words alone can bring that image into the user’s mind. From the perspective of usefulness of the information to be modelled and encompassed by the integration of VR and CBR, three main issues have been identified and are listed below. •

The volume of information - deals with the knowledge of possible interest and its on-demand presentation, in a way that its forms and sources are coherent, useful, optimise storage space, and development time for applications.

•

Type and adequacy of information - refers to the physical universe of the constructed reality that is relevant to represent. The information must be organised, culled, highlighted and presented to the user in order to provide the best achievement of the instructional goals of the application.

•

Relationships between the information - refers to the connection of the knowledge present within each instructional goal and amongst the cases in the repository, allowing access to the attributes that make the presentation of a case as well as its indexing and retrieval possible. Designers should thus prevent the repository from holding ubiquitous knowledge. Other issues regarding the influence of VR over the process of modelling and

representing past experiences that have been particularly relevant to this work are: (i) presenting the cases in the way they have been visualised through the eyes of the experts; (ii) making cases useful in transmitting the experts’ experiences; (iii) presenting cases in the adequate level of detail that is required to display the Page 124


knowledge present in the VR case; (iv) ordering and interconnecting the information to be displayed; and (v) adapting cases from the repository for a new situation, thus creating a new case that will be available for further retrieval.

6.6.2 – Modelling VR cases In CBR applications, building the VR cases requires a process of design and the more information is available describing the cases the better the support for modelling the VR cases is. The experience acquired with this work suggests that those interested in the design of VR cases should model the information contained in each case prior to their implementation in the computer. The idea is to have a modelling technique that designers and experts can interact with allowing them to verify whether the VR case represents the past experience well prior to its implementation into the computer. The ideal modelling technique should be simple enough for a non-VR expert to understand its contents and also flexible enough so that the designers can represent all the necessary information. Section 7.6 describes a modelling approach for development of training applications using the VECTRA framework. For the sake of brevity, this section just highlights the three main general modelling approaches that developers should consider prior to choosing the modelling technique that best fits the application domain. •

Free modelling: developers can use their own preferences and previous experiences in choosing the technique they feel most comfortable with. The final product is the responsibility of the designers as they are the ones who will build the VR cases.

•

Domain oriented modelling: developers can use modelling techniques that are common to the application domain and well known by the experts. The modelling task can be made easier if the experts can understand the models used for their past experiences as they will be able to provide feedback to the designers of these models.

•

Standard modelling: developers can choose modelling techniques that are part of an international standard of communication. Modelling standards such as those provided by STEP and Object Oriented modelling languages are available to provide this international standard. Currently, VRML is only being used as a Page 125


programming language. It is nonetheless believed that it will soon be able to work as an intermediate representation prior to the computer implementation of the VR worlds. Unfortunately, current efforts have not yet provided any technique especially designed to cope with the requirements of building VR environments. Section 9.4.3 provides further details on the choice made in this work and the following section goes onto the stage of the CBR development that deals with the featuring of the VR cases.

6.6.3 - Featuring case contents Featuring case contents in CBR involves two main issues that are (i) the descriptions of the cases that will allow their differentiation and retrieval; and (ii) the technical computational approach to attach those features to the case in the CBR application. The former is related to the domain of application and conceptual details are discussed in Section 9.4.4. The latter depends on the software architecture chosen to build the CBR application and is the focus of this section. The role of visualisation for instructional issues and for the representation of case contents has been discussed earlier in this chapter. These visualisation techniques have relied on digitised files containing pictures, animated images and video clips. Research in digitised multimedia files brings two issues that are of major importance for CBR: (i) image and video compression techniques; and (ii) image and video indexing and retrieval techniques. The first has resulted in the creation of standards for digital images, and focuses on producing compressed image files containing the highest possible quality and resolution of image. Although image-compressing techniques facilitate the development of applications and the distribution of image files, they do not allow developers to access the visual contents of these images from the contents of the digitised files. For instance, Figure 6.6.3a shows an example of the contents of a digitised image file. Although the contents of these digitised files mean something for the software that will be used to convert the file, it means little or nothing for the programmers trying to access any visual attribute from this image without seeing it on display. The current approach for image retrieval thus requires the creation of some sort of database Page 126


containing descriptions of the images or keywords related to the visual contents of the image files.

Fig. 6.6.3.a - Contents of a digitised image file.

Apart from VR where the simulation is represented in an object-oriented architecture, current visualisation techniques present “static” information. Note that the term “static” refers to the information only, and has no relation with the way information is displayed for users. This definition also comprises animated information presented in multimedia where the animated images are binary files.

Fig. 6.6.3b - Contents of a Superscape VR file. Page 127


Differently from digitised image files, VR technology holds a software architecture that allows access to the contents of the visualisation. VR used to represent past cases thus gives a more active interface in the sense that developers can access the attributes of the objects contained into the virtual worlds. Figure 6.6.3.b shows part of the contents of a Superscape VR file. It shows that developers can add and change properties of the objects either in the world builder in a WYSIWYG format or directly over the programming code. Advantages that can be gained with the access to the contents of the files are: •

the possibility to create libraries containing VR objects and hierarchies that can be shared between users and developers, thus speeding up the process of case design and implementation into the computer environment;

•

the prospect to create new cases, either automatically or manually, by extracting and combining the contents of various VR cases; and

•

the possibility to develop computer algorithms to perform an automatic creation of new cases by adapting from the objects and object-hierarchies, thus reducing the time-consuming task of VR case-design. VR does not amplify the range of attributes and features that can be given to a

case. However, it opens new possibilities for the development of algorithms that can handle case attributes and features. VR is particularly useful to handle attributes of objects in the virtual worlds such as rotations, velocities, positions within the space, and contact with other objects. VR does not just improve the presentation of a case, but also provides an architecture for case featuring that can be used either to represent the case in the CBR or to allow its indexing and retrieval. VR gives a dynamic representation of a case, where attributes such as movement, colour and geometry can be attached as slots of an object. The access to the case attributes within the VR files allows working directly over the contents of the VR cases, avoiding the need for the extra database holding image descriptions. The implications that this architecture has over the retrieval of the VR cases are discussed in the following section.

Page 128


6.6.4 - Retrieving VR cases Case retrieval in the CBR paradigm is performed by special algorithms that search the case repository for information that matches the users’ queries. Section 4.4.3 describes how these algorithms perform the search over the case repository and Section 8.5.2 describes the algorithm developed for the VECTRA prototype. This section discusses the role that the VR architecture plays over the retrieval process of CBR cases. The previous section shows that the VR programming architecture allows objects to have their properties attached to them. This differentiates VR files from other digitised image files where properties are kept in external databases. The VR architecture changes the focus of current searching algorithms used for the retrieval mechanisms of CBR. Instead of searching for external records in databases, the algorithm searches for attributes in the VR object-oriented hierarchies. The search for objects’ attributes rather than records in databases, does not bring any extra difficulty for developing the search mechanisms as there are already powerful algorithms performing the search both in object-oriented hierarchies and databases. However, changing the attributes of an image is simpler in VR than in other digitised visual files. For instance, changing the colour of a grey house to a blue house in a digitised picture requires either a new picture or editing work to change its colour to blue. Once the house colour is changed, the description of its colour in the attached database file will have to be changed as well. In VR, simply changing this attribute in the file automatically modifies the house colour and the new attribute is automatically available for the searching mechanisms as well. Table 6.6.4 presents examples of queries, comparing how digitised image files and VR would perform the retrieval. The example uses an image presenting different species of birds in a cage, and the same representation in a VR world. The example limits the search to the contents of one image and the contents of one virtual world. The way this search is performed by the two visualisation techniques is explained so that readers can establish the differences between retrieving from VR and digitised image files.

Page 129


Retrieval with Digitised Images (DI) and VR • Retrieval by colour - it enables retrieval for objects with specified colour. • Query - I would like to see an image of a yellow bird. • DI - the retrieval mechanism would perform a mapping for an RGB within the range of yellow available from a colour histogram and afterwards a search for a shape that matches the format of a bird. • VR - the system would limit the search for objects called birds and search for either an RGB within the range of the yellow or for an attribute in natural language identifying the birds’ colour. • Retrieval by shape - it allows retrieval based on the shape and format of objects. • Query - I would like to know which is the smallest bird in the cage: • DI - several algorithms have been proposed to characterise an object’s shape and the similarity between shapes (Gudivada 1995). One of the most applied and accurate technique is called sweep line representation (Gudivada 1996) and creates a polygonal approximation of the objects’ shape. The searching mechanism then performs a search for objects that match the shape of a bird in a image and calculate each area to identify the smallest. • VR - the retrieval mechanism calculates either the area or volume of each bird based on the objects’ spatial co-ordinates. • Retrieval by topological constraints - it allows retrieval based on the spatial relationships among the objects, such as “underneath the bed”, “on top of the television set”. • Query - I would like to know which bird is on the top left side of the cage. • DI - the retrieval mechanism performs a search for shapes that match the configuration of birds, cage and for the birds inside the cage. It then searches for the positioning based on the centroid of each shape. • VR - the cage objects can be hierarchically organised as the parent object for the birds objects inside it and the retrieval mechanism checks the spatial co-ordinates of each bird object searching for the one located at the position given. • Retrieval by text - it is the same retrieval technique that is used for databases. There are powerful methods and algorithms allowing language-based searching. The most traditional approaches for retrieval use indexed keywords representing the images’ contents. Retrieval by text is still the most used technique to handle image retrieval. Due to the weaknesses of the previous retrieval techniques for DI, this method is preferred when the image collection is large and an interactive query processing is needed (Gudivada 1996). • Query - I would like to see a picture of an adult male Robin. • DI - retrieval by text would require an external database and each record would require fields containing information such as a worded description of the images, the name of each image file, and the address where each image is kept. • VR – it does not require an external database file since the VR files and objects can hold attributes and worded descriptions of their contents. The files can be searched by natural language processing techniques. Table 6.6.4 - Retrieval on visualisation by digitised images and VR

Page 130


Although Table 6.6.4 presents a few possible features for image retrieval, as many features as the CBR designer can provide can be used to perform the search and retrieval. For instance, image retrieval can be based on the spatial constraints of objects, domain concepts, or the sequence of events in animation. These forms of matching cases can make a great difference for the capabilities of the CBR to create new cases by adapting from the contents of various visual files. Further details on the role that VR files can play over the adaptation and creation of new cases are discussed in the following section.

6.6.5 - Adapting the VR cases Case adaptation takes place at the last stage of the CBR working cycle and regards the use of parts of cases already present in the repository that can be modified to form a new case. The current state-of the-art in case adaptation and the difficulties behind its performance are described in Section 4.4.5. This section discusses the role that VR case representation can play over the adaptation task. Case adaptation is perhaps the task that can benefit the most from the VR case representation. VR allows access to the contents of the case files that can be used to perform the case adaptation. VR packages provide different facilities to access the contents and attributes of the objects in the virtual environments. This thesis relies on the object-oriented architecture that has been used in commercial VR development tools such as the Superscape and Sense8 that were evaluated as the most suitable tools to fit the VECTRA project (see Section 8.3.2). Objects in a virtual world are usually seen as buildings, walls, doors or pieces of furniture. These objects can also be part of a hierarchy, holding attributes in the same way as in object-oriented programming languages. The hierarchical structure in VR serves to organise the creation of VR worlds upon it and does not necessarily require physical connections between the objects contained in the virtual worlds. For instance, although a building is expected to be the parent object for the several walls it contains, and the pieces of furniture in a room to be the children of the room object, this is not an essential requirement. The object hierarchies of the VR files may also contain objects that are invisible for the environment and hold attributes related to the interaction, such as starting

Page 131


animations, narratives and changing another object’s shape or colour. Objects can also hold programming algorithms such as functions, rules and procedures, thus enriching the VR simulation. Case representation by VR becomes a task that does not differ from any object-oriented language and it corresponds to choosing an object framework that meets domain requirements. The object-oriented hierarchy allows inheritance of properties through the objects. The inheritance works from the Parent object to its Children and a child can be a Parent for other objects further down in the hierarchy, as shown in Figure 6.6.5. The hierarchical structure and the inheritance of properties have proven to be convenient in this work. For instance, when an object is the Child of another, its position within the VR world is defined in terms of its relative position to the Parent rather than in terms of its world co-ordinates. This means that a complex hierarchical structure, such as a house containing objects like walls, roof, windows and furniture can be used in another VR world and the objects inside the house will keep their original position inside the Parent-house object.

Parent file

Parent to Child

Sibling files

Case Files' hierarchy

Case 1 Retrieval Interface Menus Index

Second Level

Case 2 Building Site

First Level Front Ground Level

Scaffold Case 3 L-SIde Scaffold

Fig. 6.6.5 - Object-oriented hierarchical architecture.

This parenthood between objects allows Children objects to inherit attributes from their Parents. If the Parent object in given an attribute such as spinning clockwise and shrinking, its Children objects can follow automatically. This inheritance of attributes allows a Parent object, for instance representing a car that is composed of several parts, to receive one single attribute such as move forward and all its Children objects will follow.

Page 132


The advantage of the adaptation of VR cases is obviously the time saved over the design process of new VR cases. Computer algorithms can also be developed to access the attributes of the objects’ hierarchy and help building new cases by automatic adaptation (see Section 4.4.5 for further details on automatic adaptation). Therefore, the adaptation of VR case files becomes a task that does not differ from case adaptation performed over CBR applications relying on the object-oriented architecture for case representation.

6.7 - Synthesis of the chapter VR is a computer technology that provides 3D environments and real-time navigation. VR capabilities have been used for instructional and training applications since it first emerged. At its beginning, VR was a technology demanding many resources such as expensive hardware, software and professionals specialised in the development of applications. The continuous work in the field of VR led to a decreasing need of resources and thus to a price decrease of its applications. The amount of resources involved in VR developments is closely related to the type of interaction supported. For instance, when simply watching 3D images on flat surfaces such as computer screens or flat white boards, the viewer is subject to a primary level of VR interaction. The cost involved in the development of applications with this type of interaction is relatively low. When images are projected in various directions around the viewers as augmented realities, the degree of interaction is increased and so are the resources required for the development of applications. If inside this augmented reality the user can wear a glove and manipulate the virtual objects, the degree of interaction and the costs of the application are once again increased. There is evidence that VR is suitable for instructional applications. It allows users to manipulate objects’ properties, navigate through the VR world, experience views from different spatial references, and enables users to alter these aspects in a real time basis. Users are thus given the freedom to move through the VR worlds and experience a similar level of complexity than that of the real situations. Past experiences represented as computer simulations that incorporate tutorial guidance provide an environment that is particularly suited to integrate the instruction

Page 133


and practice that is often required for training. The VECTRA project provides such integration by allowing access to VR cases that simulate the place where the real experience took place and the actions of the experts. This environment uses the latest computer technologies and provides instructional activities that access learning-bydoing with the guidance of a virtual instructor. VR case representation in CBR instructional applications represents an improvement from the digitised video cases that are so often used in CBR. Differently from VR cases, programming languages cannot access the instructional contents of digitised video files. Moreover, case adaptation from digitised videos requires the recording of a new picture or film, as it cannot be performed from the contents of the existing video cases. In spite of the advantages that CBR-VR applications can bring, there is a danger that applications may become more entertaining than addressing educational issues. The trainees may enjoy the 3D real-time simulation and struggle with the navigational devices and fail to adopt the goals and the lines of reasoning that are promoted by the system. Users may instead end up using the system with an anecdotal approach and just enjoy the “fun” of the VR interfaces.

Page 134

Chapter 7 – Conceptual VECTRA design

7.1 – Overview The previous chapters describe the state-of-the-art on the issues behind the development of this thesis. A literature review in topics such as computer-based training (CBT), intelligent tutoring systems (ITS), intelligent computer-aided training (ICAT), case-based reasoning (CBR) and virtual reality (VR) was conducted focusing on the influence of these topics on the hypothesis and objectives of this thesis. This chapter describes the design of a conceptual methodology to determine how to construct VR case-based instructional applications. This methodology involves a combination of the capabilities of CBR instruction, the VR interface and the requirements for instructional strategies in CBT tools. The proposed methodology helps guiding the tasks involved in the development of VECTRA framework and does not include particular domain aspects. Rather, it gives an approach for the development of interdisciplinary VECTRA applications.

Page 136


7.2 – The choice for a methodology The definition of a methodology prior to the development of the VECTRA applications has its use in organising ideas and suggesting a course of action to be followed. Designing a CBR instructional application with VR cases involves a number of alternative options that include the choice for the VR tool for case representation, the representation of domain cases and the instructional strategy in the CBR. The definition of a suitable course of action can lead to a more effective design of the system (Crinnion 1992; Connell 1994). It can also help to provide a model and/or document for the exchange of ideas and viewpoints between the author, the domain experts and the other people involved in the development of the system. In order to choose a methodology for the development of VR case-based instructional applications a literature review on methodologies for development of computer systems, AI applications, CBR and CBT was carried out. This review raised the possibility to use established methodologies such as the Client-Centred approach (Watson, I.D. 1992), Knowledge Acquisition and Design Structuring (KADS) (Schreiber 1993), the Prototyping Methodology (Crinnion 1992) and methodologies for development of CBT (Dean 1992; Seidel 1995) and CBR (Althoff 1992; Kolodner 1993). The Client-Centred approach involves the whole life cycle of knowledge-based systems and guides the work of the knowledge engineers focusing on “what the client can see rather than being technology-centred” (Watson, I.D. 1992). The inadequacy of this methodology for VECTRA applications stemmed from the fact that the student should be the one to play the role of the client. However, the domain experts’ are actually the ones with the past experiences to fill in the case repository and to supply the instructional activities of VECTRA applications. The KADS methodology was also a possibility considered, as it has become a standard structured methodology for building knowledge-based systems and models of domain knowledge. The major contribution of this methodology comes from its process of domain knowledge modelling (Hickman 1992), the area that usually is the bottleneck in the development process of knowledge-based systems (see Section 4.4.1). However, the knowledge modelling process in CBR does not require such an approach as this task is performed by simply gathering domain cases.

Page 137


The Prototyping Methodology provides guidelines for the development of general information systems rather than limiting them to AI applications and focuses on the establishment of a step-by-step plan to be followed prior to the development of information systems. This plan includes elements such as the domain requirements, the system’s functionality, the expected outcome and a set of procedures structuring and sequencing activities, thus guiding the developers throughout the development processes of the information system (Crinnion 1992). This methodology does not indicate the use of any specific modelling technique, which can be as simple as a series of screen displays holding information such as their purpose and relationship to other screens or data-flow diagrams and/or object-oriented models (Connell 1994). The prototyping methodology is thus about creating a model of the systems that is interactively improved through the development stages of the system. As the prototype eventually becomes the installed system, its model becomes the document of the system’s development process. The reason for choosing the Prototyping Methodology in this thesis stems from the fact that it could be combined with methodologies for development of CBR and CBT applications. For the sake of brevity, this thesis does not provide further details on the Prototyping Methodology nor on the other methodologies considered and declined for this work. The following sections provide details on the combination of the characteristics and domain requirements that resulted in the conceptual model for the development of VECTRA applications.

7.3 – Developing a CBR The Prototyping Methodology reviewed in the previous section provides an approach for the anticipation and evaluation of the tasks involved in the development of information systems. Obviously, certain types of systems, such as CBR, involve tasks that would not be found in other systems. For instance, the combination of case gathering, case indexing and case representation has so far been exclusively found in CBR applications. To model the tasks involved in these activities requires a review of specific references. VECTRA applications, independently from its instructional role and the use of VR for case representation, is a CBR and should thus be developed as such. The CBR in

Page 138


VECTRA applications is at the core of the tasks to be carried out prior to the development of the prototype. Authors such as Barletta (1991), Henessy (1992) and Kolodner (1993) suggest approaches for the development of CBR systems that have been combined in 4 stages as shown in Figure 7.3 and described below. •

First stage – involves the identification of the domain cases and the definition of the way they will be represented before the whole case-base is stored. This stage involves domain experts and to make sure the features and attributes of the cases are properly covered and represented.

•

Second stage – involves the implementation of the whole set of cases and evaluates the consistency of the case repository. The system should then be tested by domain experts and to face real domain problems. This stage thus helps building confidence about the system’s coverage.

•

Third stage – identifies the requirements for the addition of new cases and helps to choose and debug the adaptation technique to be implemented. This stage also involves testing the indexing mechanism of the system.

•

Fourth stage – involves the evaluation of issues such as the accuracy of the system’s solutions, the speed of the system, the friendliness of the user interface, the accuracy of adapted solutions, and the overall usefulness of the system. As reviewed in Section 5.6.7, CBR instructional applications are not likely to

include case adaptation mechanisms. In this thesis, this applies also for VECTRA applications where case adaptation is not part of the original proposal. Therefore, as Figure 7.3 provides a general approach for the development of CBR applications, the third stage can be skipped for VECTRA applications where case adaptation is not present.

Page 139

Acquire cases of the domain

Start

Implement part of the case base

Test case representation

No

Redefine features and/or attributes

Stage 1


Yes

Apply real domain problems

Test case-base consistency

Irrelevant cases

Complement the case-base

Stage 2

Remove irrelevant cases

Insufficient number

Yes

Review adaptation mechanism

No

Test adaptation approach

Yes

Apply new problems

Test indexing approach

No

Stage 3

This stage only applies when the system allows case adaptation

Add adaptation mechanisms

Yes

No

Yes

No

Test user friendliness

Test overal usefulness

Apply real domain use

No

Review indexing approach

No

Stage 4

Test running time

Yes

Yes

End

Modify user interface

Fig. 7.3 – Development stages of CBR applications.

The following sections describe aspects such as the requirements of CBT applications and the instructional capabilities of the VR-CBR integration proposed in this thesis.

7.4 - VECTRA design requirements The design of VECTRA applications combines computer-based instructional requirements, the CBR working cycle and the capabilities of the VR interface. VR provides the spatial simulation of the workplace where the past experiences occurred. CBR involves the representation of past on-job experiences and the ability to match and retrieve them appropriately. Behind these computer techniques are the CBT Page 140


requirements and instructional strategies guiding user learning. Therefore, developing an instructional CBR that presents the cases as VR simulations of experts’ past experiences and provides guidelines to assist user learning are also issues on the choice for a methodology to design the prototype.

Training requirements Topic synopsis

Learners' profile

Hardware requirements

CBR instruction

VR interface

Suitability of past experiences

Software requirements

Requirements for case representation

Instructional guidelines

Instruction from past experiences

Interface design

Fig. 7.4 - Decision factors for the appropriateness of CBT.

The Figure 7.4 describes the three main issues involved in the development of VECTRA applications and suggests a sequence for the aspects to be considered that are further reviewed in the following sub-sections.

7.4.1 - Training requirements The first task for an individual or a company willing to develop a training application is to identify the areas where improved skills would bring benefits to the work performed. Although it seems an easy task, authors such as Briggs (1981), Dean (1992), Lee (1995) and Tucker (1997) show that it can be quite challenging, specially for companies with hundreds of employees. The next task is to identify whether the CBT is the right medium to provide the training course. This task requires investigating whether the instructional activities could be provided by computer applications. The requirements for hardware devices to provide the intended instructional outcome, the availability of hardware for users to run the application and the amount of resources necessary to develop the CBT are all aspects that have to be taken into consideration.

Page 141


These tasks are of strategic importance and an analysis of today’s market for CBT shows that the majority of applications are related to the improvement of skills related to computer applications. This should come as no surprise as these CBT applications are used by professionals who are familiar with computers and their interfaces, are used to read and follow instructions from the computer screen, are aware of the benefits computers can bring to their daily life and probably have their own computers at home. This association between the CBT market and computer professionals does not mean that CBT applications cannot apply to other professional sectors. For instance, flight simulators deal with learners who do not often use computers in their jobs and may even be totally unfamiliar with them. However, the cost and risk of training in real aeroplanes justify the investment in CBT tools. Unfortunately, there is no methodology to identify when CBT is the most appropriate form of training, matching the application domain and learner’s profile (Harrison 1990; Dean 1992; Gery 1995; Tucker 1997). The most common decision factor has been relying on the sole comparison between the cost incurred in the development of the CBT tool and the cost of running other alternative forms of training (Lee 1995; Tucker 1997; Brooks 1997; Ravet 1997).

PROBLEM INVESTIGATION

Domain synopsis

Equipment requirements

Learners' profile

Describe the skill(s) that learners aim to acquire

Identify hardware devices for the training course

Evaluate users' background and learning capabilities

Cost evaluation Compare cost between the VECTRA and other training alternatives

Fig. 7.4.1 - Main elements of decision for the appropriateness of CBT.

In the absence of such a methodology, Figure 7.4.1 provides some help in deciding whether CBT is an appropriate media for the delivery of the instructional course. The main issues presented in this picture and described below: Page 142


•

Domain synopsis – instructional activities have their own characteristics and this task deals with the understanding and documenting of the skill to be acquired and its implications for the way the work is performed;

•

Equipment requirements - a wide range of hardware devices such as mouse, joystick, steering wheel, sound card, space-mouse, 3D monitor, and Head-Mounted Display are currently available and can be involved in the CBT course;

•

Learners’ profile – this task provides an indication on how the employees would react to the CBT course and help to identify the type of interface and learning strategy that would best fit the trainees;

•

Cost evaluation- this task establishes comparisons between the instructional tool and other training methods so that both developers and clients can have an idea of the total costs involved in the development of the CBT tool. If a CBT tool is judged appropriate for the training course, the next task is to

decide for the appropriate computer technology that best suit the application domain. This thesis proposes CBR and the following section helps to identify domains that are best suited by this AI technique.

7.4.2 – CBR instructional capabilities VECTRA applications rely on the working cycle of CBR presented in Section 4.3. Chapter 5 presents four instructional strategies that can be taken from CBR and Section 7.3 provides a methodology for the development of applications. This section highlights the VECTRA instructional capabilities so as to help defining the domains that can get the most from this instructional approach. The four main tasks involved in deciding whether VECTRA applications fit an application domain are: 1. Availability of cases – identifies whether domain cases are available and the work involved in the case gathering process; 2. Suitability of cases – identifies whether the training sessions based on VR representations of past experiences are able to provide the skill required;

Page 143


3. Instructional strategy – helps deciding for the instructional strategy (i.e. deciding to let users freely retrieve the cases or providing approaches such as those discussed in Section 5.5); and 4. Case representation – identifies the structure and the requirements for case representation that best suits the instructional domain. Authors such as Kolodner (1993), Leake (1996) and Watson, I.D. (1997) provide insight on the appropriateness of CBR for general application domains. The recommendations of these authors rely on the advantages of CBR in comparison to other computer technologies and are specific to the application domain considered. The appropriateness of VECTRA applications is also related to the skill to be achieved and aspects of domains that could take advantage of the VR case-based instruction are listed below: •

when a case library already exists or the experts’ knowledge is mostly learned from on-job experiences;

•

for domains where cases are easy to acquire or where it is difficult to create structured models of the domain knowledge;

•

when VECTRA cases are capable of simulating on-job experiences;

•

when VECTRA simulations provide a creative reasoning process that allows users to propose new solutions from different inputs;

•

when VECTRA cases can provide a shortcut for a solution, avoiding time consuming descriptions of inputs and long inferences in rule chains;

•

when unsuccessful experiences can be part of the VECTRA case repository and could help to anticipate problems that might result from the actions of unskilled trainees;

•

where new cases can be added to the repository, thus improving the VECTRA instructional capabilities; Much of the effort in building CBR instructional applications goes into collecting

and representing cases for the repository. Nonetheless, there are domains where the VR cases proposed in this thesis are either unsuitable to represent the domain or where

Page 144


the benefits the VR technology can bring do not compensate for the effort and cost involved in developing the VR cases. These are situations when the VECTRA is not suitable and CBT designers should look for either commercial CBR tools or other CBT techniques. Further details about the suitability of VR for case representation are discussed in the following section.

7.4.3 - VR capabilities General benefits of VR technology for training and its influence over the CBR working cycle are discussed in Chapter 6. This section discusses the VR instructional capabilities in relation with what has been learned from the VR cases and helps to decide for domains that can get the most from VECTRA applications. The main issue involved in deciding whether the VR cases are appropriate for the application domain regards the possibility to build VR simulations of on-job activities. The creation of these VR cases involves tasks such as: •

Identifying domain aspects – deals with the appraisal of the domain aspects required for the instruction and thus need to be simulated in the VR cases;

•

Identifying software requirements – identifies the capabilities of commercial VR packages to represent the domain aspects;

•

Interface design – evaluates the amount of work and skills required for the implementation of the VR cases. The VR cases are represented as simulations of past on-job experiences.

Unfortunately, the task of building the VR cases is usually more complex than representing cases via databases or multimedia video-clip files. This difficulty is mostly due to the task of designing and positioning the VR objects in the VR environment. Details on the Superscape capabilities to build the VR cases are presented in Section 8.3.2. The types of domains that can benefit from the use of VR cases are listed below: •

domains where the 3D interaction is important to aid understanding ;

•

domains involving representation of physical objects rather than abstract concepts;

•

domains where the simulation of past experiences plays an important role over the apprenticeship of experts;

Page 145


•

domains where simulations are important for the training course and VR is an alternative to diminish their cost and improve their safety;

•

domains where libraries of VR objects can be built to speed up development of VR cases;

•

domains that do not involve too many different complex objects and time-consuming efforts for building the shapes;

•

domains where the users can take advantage of the interactive work with the VR cases supporting a real-time simulation of a problem-solving situation;

•

domains where the interaction with the VR cases do not require a high degree of immersion and expensive hardware devices for the interaction; Most of the difficulties listed above are due to limitations of the current tools

available to build the VR cases and may no longer exist in the near future with the improvements in these tools. This factor is also an advantage of VR cases in comparison to digitised multimedia video clips since the former can be upgraded from the previous representation and video clips would require the filming of new domain activities. Another issue related to the use of VR technology that was identified is the entertainment that users can take from the 3D interaction with the instructional activities. With users for whom catching attention is a problem, VR cases can be an alternative solution.

7.5 - Instructional activities design Authors such as Harrison (1990), Shlechter (1991), Dean (1992) and Gery (1995) mention that the design of instructional activities is one of the most problematic aspects of the development of instructional courses. There is no unique way to conduct the instructions and learning is influenced by individual learning capabilities and motivational aspects that make it difficult to provide “the right instruction at the right time and with the right format” (Dean 1992). The process of designing training courses using a repository of VR cases involves modelling and sequencing the instructional activities and choosing the media for the delivery that is part of an instructional strategy. The planning of each instructional activity in VECTRA applications relies mostly on the work of Gagne (1992) and Schank Page 146


(1995) and on the instructional capabilities from the CBR model of cognition (see Section 5.3). The work of authors such as Gagne (1985, 1992), Gardner (1993) and Schank (1982; 1995) shows that learning a skill requires prior learning of simpler parts of that skill. Moreover, instructional applications should include instructional strategies, and authors such as Wingfield (1979) and Bednar (1992) defend that the effective design of instructional systems requires the application of a particular theory of learning. Developers of VECTRA applications must be aware of the theoretical basis underlying the system design. Each VR case in the VECTRA is hinted at embodying the simulation of past experiences and the actions taken by the professional who had them. The domain knowledge is held in the case repository and can be seen as a break down of the domain into representations of past experiences containing instructional activities. Each case is designed to embody one or more instructional activities. For instance, training on the inspection of health and safety regulations on scaffold structures may contain such topics as structural safety, safety for the people in the vicinity and safety for the workers on top of the structure. An instruction regarding these issues can be given from one single case of scaffold inspection thus containing several instructional activities (see Section 9.4.2). The

instructional

strategy

in

VECTRA

applications

follows

the

recommendations of these authors and provides past experiences accompanied by theoretical information about their outcomes. A common instructional strategy used for educational and training applications goes from presenting general domain knowledge (e.g. general implications of health and safety on scaffolding) to describing more specific tasks (e.g. how to properly inspect scaffold foundations). Apart from this general guideline for sequencing the instructional activities, the VECTRA framework comprises with the instructional actions and the learning stages presented by Gagne (1992). Figure 7.5 has been adapted from the work of Gagne (1992) and suggests an approach for designing each independent instructional activity contained in a case. The figure shows a sequence of instructional actions fulfilling the learning stages and further details on each instructional action are discussed below.

Page 147


Learning stages

Apprehension

Informing objectives

Recalling prior learning

S pe ele rc cti ep ng tio n

INSTRUCTIONAL ACTIVITIES DESIGN

Learning guidance

Recall

Ac qu isit ion

Pe rfo rm an ce

g in nc n ha ntio En ete r

n tio tiva Mo

g inin Ga ntion atte

Instructional actions

Accessing performance

n tio ten Re

E perf liciting orm anc e

Providing feedback

Fig. 7.5 – Designing instructional activities

•

Gaining attention - is the first step in the design of instructional activities and involves a motivation to make users engage in the learning process. (Further details on the importance of this stage are discussed in Section 2.7 and should take into consideration the VR representation of on-job experiences.)

•

Informing objectives - provides users with the contents of each instructional activity and helps them to identify the instruction they are looking for.

•

Recalling prior learning - identifies the knowledge required to properly learn the contents of the instructional activity and links to other domain concepts.

•

Selecting perception - is the task of deciding on the media to deliver the instructional activity that best matches the users’ learning capabilities and these media include sounds, written information and moving the viewpoint.

•

Learning guidance - involves deciding for the strategy guiding the users’ through the instructional activities and relies on the instructional strategies that can be taken from CBR (see Section 5.5).

•

Enhancing retention - highlights the main issues in the instructional activity and this task should also provide leads for further recall of the learning taken.

•

Accessing performance - involves tests to evaluate the learning taken from the instructional activity and can be performed by approaches that range from questions about the main issues in the instructional activity to making users resimulate the instructional performances.

Page 148


•

Providing feedback - involves an evaluation of the tests results and gives users feedback on their weaknesses or their capability to move onto another instructional activity.

•

Eliciting performance - is the last step in the design of each instructional activity and involves the motivational aspects that make users relate the learning taken to the domain concepts and make them keep using the tool. Many steps in the design of instructional activities, such as defining objectives

for each instructional event, recalling prior learning and eliciting feedback require an understanding of the domain that can be obtained from the contact with experts. Modelling techniques can play an important role in helping putting down on paper (or electronically) the information taken from the experts and thus facilitating the discussions about the aspects of the domain. Other aspects that can help the design of instructional activities are: •

experts with teaching experience to accompany the design process;

•

a literature review of the domain and an analysis of the instructional sequence adopted by the authors;

•

organise the issues from fairly general (e.g. the importance of safety on scaffold structures) to very specific (e.g. checking the position for the sole plates);

•

interviews with the targeted learners in order to analyse their knowledge background and preferential learning styles; and

•

include facilities to allow users switch between the instructional activities or even to perform tests checking the possibility to skip some of the instructions. Another issue related to the organisational structure of the instructional

activities regards choosing a media for the delivery of the instructional goal that catches the learners’ attention and accesses their learning capabilities. Further details on these issues are discussed in the following sub-sections.

7.5.1 – Accessing learning capabilities Section 3.7 shows that when designing instructional tools, accessing individual differences is a major weakness of ITS and ICAT tools. The characteristics of the intended users play an important role over the design of instructional applications and

Page 149


include aspects such as the users’ goal in using the system, their motivation for learning and pre-existing knowledge of the application domain. Apart from general aspects of motivation, the task of providing instructional activities that properly access individual differences is the responsibility of the CBT designers. Gagne (1985) stated that learners are usually not aware of their own learning processes when taking instructions. They are engaged in learning from the material provided in the training course and do not think of what could be the best way for them to learn (Gagne 1995). VECTRA applications and the VR case repository conform to Schank’s recommendation for educational practices of “learning by doing” (Schank 1995; 1996; 1997). The author claims that “the best way to learn how to do a job is to simply try doing the job…but with an expert available for help as needed”. Applying this to computer systems, the recommendation would be to use simulations of on-job activities with the guidance of an expert instructor. The designer’s task is to choose approaches capable of attracting users’ attention regarding the skill to be improved. In order to carry out this task, Gagne (1985; 1992) suggests an approach based on conducting the instructional activities accessing the five learning capabilities that are related to the types of skill to be acquired. Gagne’s (1985; 1992) learning capabilities are (i) verbal information, (ii) intellectual skills, (ii) cognitive strategies, (iv) attitudes and (v) motor skills (see Section 5.4). Guidance on conducting instructional activities dealing with these learning capabilities is presented in Table 7.5.1 that has also taken the viewpoints of authors such as Harasin et al. (1995), Lee (1995), Sims (1995) and Fairhurst (1995). The suggestions to access Gagne’s (1985; 1992) learning capabilities are independent from the cognition theory the authors have followed and take into account the type of instruction the VECTRA application aims to provide.

Page 150


Type of instruction

Verbal Information

Intellectual Skills

Cognitive Strategy

Attitude

Motor Skill

Accessing learning capabilities - Stating the objectives of each instructional activity and their role in the training course. - Providing concepts and meanings of words. - Activating attention by varying the rhythm of the verbal instructions. - Giving a meaningful context (including images) representing the domain or skill to be learned. - Calling participation by challenging learners’ attention or questioning the issues in discussion. - Providing examples that are relevant to the instructional goal. - Stimulating retrieval of previous knowledge. - Giving leads that allow the combination of previous knowledge. - Using different media and contexts to accomplish the same learning. - Providing rules to structure the domain concepts. - Presenting and describing the learning plan. - Discussing the pace and the sequence of the training session. - Providing leads to help future recall of the issues in discussion. - Providing guidelines to organise the knowledge involved. - Encouraging learners to provide alternative solutions. - Providing learners with past experiences, either successful or not, and the course of action that was selected. - Letting learners perform the actions. - Providing information about the personal skills involved in past actions. - Providing the possible benefits for the learners in taking the instructions. - Stimulating participation by giving feedback on actions taken. - Providing timed guidance for the expected actions. - Providing learning-by-doing activities. - Arranging facilities for repeated practice; - Anticipating the expected results of the actions taken; - Providing links for the actions involved in the training course. Table 7.5.1 - Influencing learning outcomes.

Independently of the type of instruction, preferential learning capabilities and the ensuing ways of presenting the instructional activities, VECTRA applications have to be delivered by computational media such as sound, images and written information. The task of choosing the media to deliver the instructional activities so as to match individual preferences is the issue discussed in the following section.

Page 151


7.5.2 – Media for instructional delivery Individual preferences in taking instruction from particular instructional strategies and media such as sounds, written language or visual information are also a problem to be considered in designing the instructional activities. For instance, leads for knowledge retention, tables, diagrams, pictures and verbal clues that are effective to illustrate some domain characteristics may fail to provide an effective instruction to some individuals. Choosing the media for the delivery of the instructional activities is a task that requires attention from designers. For instance, instructional activities that are wellsequenced and provide instructional actions covering the whole learning process could be considered as an example of a well-organised presentation. However, the designer should also choose the media to deliver the instruction that fits the domain and the users’ learning preferences. Current commercial VR tools contain capabilities to emulate several aspects of reality (see Appendix 2) including sounds, animation, written information and pictures attached as textures enhancing the visual details of objects. However, the literature review shows that there is no methodology indicating the type of media that best matches certain domains or individual learning preferences (Dean 1992; Gagne 1992). Gagne (1992) cited that designers should try reaching a balance between the requirements of the instructional goal, the learner’s background and individual preferences. The choice of computer media should also take into consideration the contributions the characteristics of each media can make to the application domain. Some characteristics of the possible media for the instructional delivery in VECTRA applications are presented below. •

Written language - it is based on the formulation of meaningful sentences providing the instruction of domain concepts and activating learners’ cognition. Authors such as Briggs, (1981) Gagne (1992) and Brooks (1997) cited that this is perhaps one of the least effective techniques to deliver training but the most used and every instructional application contains this media.

•

Sounds - this media is naturally important when the skills to be learned require this mode of communication, as for instance instruction in music or foreign languages. It can also be useful for learners who do not know how to read and for

Page 152


learners who are slow readers or visually impaired. Although it is still not clear that auditory messages play a valuable function in controlling attention (Gardner, 1993), they have been frequently used to deliver instruction. •

Static images - they play a major role in learning and can be used for reasons such as to direct attention to portions of a text, to illustrate objects and the spaces where the past experiences occurred. Images can also help to instruct abstract concepts such as colour and geometry by providing illustrations facilitating their understanding or concepts such as near/distant, closed/open by giving examples of their influence on the domain.

•

Animated images - they can play an important a role, like the media above, but adding a number of potential advantages such as providing instruction by conveying movements and including sequences of actions and objects’ behaviour. Moreover, watching moving objects has been pointed out as capable of motivating the use of the instructional tool (Wingfield 1979; Gagne, 1992, Schank 1995). However, the main function of animation is to provide a meaningful context to which learners can relate new information and associate their learning to the corresponding real on-job activities. Authors such as Harasin (1995), Reynolds (1996) and Brooks (1997) cited that a

way around the influence of multiple learning differences is to opt for more than one media to provide the instructional activities and to let users get the information as they prefer. This approach though useful, could extend the time spent on the development of applications and increase the hardware requirements to run the VR cases. Table 7.5.2 provides a tool to help designers track the richness of each instructional activity for VECTRA applications. Section 2.7 shows that instructional activities should go through the whole learning stages and can be broken down into instructional actions. Designers can thus tick in the boxes over the numbers 1 to 4 and specify the media used to deliver each instructional action corresponding to the learning capability that is accessed.

Page 153


Table 7.5.2 - The design of instructional events

Once designers have decided for the elements to take into account and identified the tasks involved in the design process, they should then start thinking about performing tests to check and evaluate user learning. Further details on the importance and techniques for learning evaluation are discussed next.

7.6 - The VECTRA development methodology This section discusses the elaboration of a model to anticipate and to guide the tasks

in

the

development

of

VECTRA

applications.

This

model

involves

recommendations taken from the literature review and rather than providing a rigid sequence of tasks to follow, it proposes general guidelines for the development of VECTRA applications. The three main issues that have been combined into the elaboration of the VECTRA development methodology are: 1. the recommendations of authors such as Althoff (1992), and Kolodner (1993) on the development of the CBR systems,

Page 154


2. an instructional strategy suggested by Gagne (1992), which has been applied for the design of the VECTRA instructional activities; and 3. the Prototyping Methodology (Crinnion 1992); Section 7.3 shows that in considering the requirements of instructional applications, designers should focus on investigating the application domain and job profile. The next stage involves an evaluation of the capabilities of computer applications, and more specifically of CBR, to satisfy the training requirements. The last stage involves the modelling of the training course, the instructional activities and the implementation of the system. Obviously, every domain has its particular characteristics and it is difficult to provide a general methodology for the development of VECTRA applications. Chapter 9 describes the particular aspects of an application for inspection of scaffold structures and Figure 7.6 shows the compilation of the tasks involved for the development of VECTRA applications independently of the domain. Further details on these stages are described below. •

Domain characteristics - It is a conceptual stage where the interests turn to identify the skills to be acquired, the requirements for hardware devices and whether the case-based instructional approach is capable of matching the domain requirements. This stage also involves an evaluation of the trainees’ profiles, including their educational levels, existing computer skills and whether they would be willing to take the training from a computer tool.

•

Defining design standards – This stage starts when the decision is made to use the VECTRA training approach and the first task is to identify the source of domain cases and acquire them. For applications involving a team of developers, it is recommended to set standards for fonts, colours, software settings, file naming conventions, type of menus, interaction controls to forward and repeat instructional activities and their general positioning on the computer screen. This stage also involves the identification and design of the VR objects that are part of the simulations and the implementation of examples of VR cases into the VECTRA repository.

Page 155


Determine training needs

Start

Evaluate learners' profile

Evaluate VR suitability

No

Choose another training alternative

Evaluate total cost

No

Define software requirements

Stage 1

Yes

Define hardware devices

Yes

Evaluate case availability

Choose another CBT alternative

No

Yes

Acquire cases of the domain

Define instructional strategy

Implement part of the case base

Stage 2

Identify case source

Model domain cases Redefine features and/or attributes

No

Test case representation Yes

Irrelevant cases

Complement the case-base

Test case-base consistency

Yes

Test instructional capabilities

No

Stage 3

Apply real domain problems

Remove irrelevant cases

Insufficient number

Yes

Review indexing approach

No

Test running time Yes

No

Test overal usefulness

Yes

Test user friendliness

No

Modify user interface

Stage 4

Apply real domain use

Review retrieval mechanism

Yes

End

Fig. 7.6 – Development tasks of VECTRA applications.

Page 156


•

Assembling the system - This stage involves the implementation and evaluation of the usefulness and correctness of the application regarding its instructional goal(s). The evaluation should be carried out via the analysis of the qualitative and quantitative aspects involved in the application and this is an iterative process of testing and refining that requires the presence of domain experts.

•

Evaluating performance – This stage involves monitoring and evaluating the training achievements from a sample of the users’ responses to the tests and could also involve an evaluation of the behavioural change(s) over the performance of their on-job activities. This evaluation will reveal the usefulness of the training application and may lead to improvements in the case repository and user interface. The experience gained from this thesis shows that all the documents, procedures

and supporting files related to the design of the VECTRA application can easily be dispersed and become unmanageable. For applications involving a team of designers, the presence of a project manager is thus required. This professional’s job is to establish the procedures for everyone to follow in supplying and maintaining the project files and to put the work together in a full instructional package. This task will be performed more easily if the prior stages have been well documented. Moreover, new projects can also benefit from the experience gained from (well documented) past ones.

7.7 - Synthesis of the chapter A conceptual model for the development of VR case-based instructional applications has been presented in this chapter. The model considers the CBR development methodologies, the instructional requirements of CBT, and the capabilities of VR technology for the instruction and simulation of on-job activities. Most of the decisions to be made throughout the development of VECTRA applications are of qualitative nature. This chapter does not provide criteria to help these decisions but simply a series of hints that can be taken into consideration for decisions on the suitability of case-based instruction, the VR capabilities for case representation and the instructional strategy to be adopted. This conceptual model does not include an evaluation of aspects of particular instructional domain. Rather, the model provides an approach for the development of interdisciplinary instructional applications. Further details on the structure of the Page 157


VECTRA framework and its capabilities to provide interdisciplinary training applications are described in the next chapter.

Page 158

Chapter 8 – The VECTRA framework

8.1

Overview

The previous chapter describes the conceptual stage of the development of VR case-based instructional applications and presents a guideline covering the process of building VECTRA applications. The development methodology proposed identified the necessary components to be part of the VECTRA framework. This chapter describes the development of the VECTRA framework and each of its components regarding the role they play in providing VR case-based instructional applications. The VECTRA framework is a combination of CBR and VR that provides an open architecture for the development of intelligent instructional applications. The word open means that the VECTRA framework could be applied for the development of interdisciplinary applications and this chapter thus describes the VECTRA framework independently from any application domain.

Page 159


8.2 – Building the VECTRA framework The VECTRA acronym stands for Virtual Environment for Case-based TRAining and a major concern in its development was providing an open framework that could hold a training methodology and shorten the time to build applications. It was also intended that developers with modest technical computer expertise could use this framework to build instructional applications. The VECTRA training approach addresses instructions relying on the model of cognition and working cycle that are at the origins and heart of CBR (see Sections 4.2 and 4.3). The representations of past on-job experiences provide a domain body-ofknowledge that can be retrieved as an instructional element. Instruction from this framework addresses two main topics: i - the aspects of the CBR working cycle such as case representation, case indexing, retrieval mechanism, instructional strategy and case adaptation; and ii - the learning that can be taken from past cases by comparing the case retrieved and the intended learning goal and involves processes such as recognising the source for analogy, transferring the analogy, evaluating the parts transferred, and consolidating the reasoning process. The previous chapter shows that the author considered the Prototyping Methodology the most suitable to start developing the VECTRA framework. Another aspect supporting the use of the Prototyping Methodology was the fact that the literature review did not reveal previous attempts to integrate CBR and VR. The innovative aspect of this thesis relies on the creation of a framework to hold VR case based instructional applications where the feedback gained throughout the prototyping task was the only support for its development. The Prototyping Methodology refers to building a simple computerised solution containing some (but not all) of the essential elements of the final system (Berry 1990). The prototype thus allows to obtain feedback about the system’s good and bad parts, thus guiding the next version of the prototype development. Mayhew (1987) proposes three approaches for prototyping that could be used in the VECTRA framework:

Page 160


1. rapid prototyping – also described as a throw-it-away approach and involves the creation of fake data-bases or knowledge-bases where the inferences can be tested and the domain problems understood; 2. incremental and evolutionary prototyping – proposes the break down of the system into individual components that can each be tested independently and then added to the whole; 3. evolutionary prototyping – suggests a process where the system is successively refined until it is adequate for use. Following the model identified in section 7.6 for the development of VECTRA applications, the first stage should be an evaluation of the application domain. However, at this stage there is a need to evaluate the means to integrate VR and CBR and how it would work in terms of holding the CBR working cycle. The initial idea was to take these three approaches from the Prototyping Methodology and apply them at different stages of the prototyping task. For instance, a few fake VR cases were to be built and the independent components of the CBR working cycle, such as the retrieval mechanism, the attributes and indexes could then be tested independently. Once the way these components worked in the CBR-VR integration was defined, the VECTRA framework would thus be refined until adequate for use. Further details on the development of the CBR-VR integration that originated the VECTRA framework are discussed in the next section.

8.3 - The CBR–VR integration CBR applications can be developed using either programming languages or tools specially designed. The latter are usually called CBR Shells and they provide facilities comprising with the CBR working cycle where developers are only required to input and index the cases in accordance with the Shell’s capabilities for case representation. The advantage in using these tools is to speed up the development of applications. The weakness is the lack of flexibility in representing cases as developers are restricted to the Shell’s capabilities. There are also commercial VR tools (or VR world builders) that can help the design of the VR cases. VR world builders such as Superscape, WorldToolKit, IDS

Page 161


and VREAM allow designers to create the VR cases relying exclusively on the facilities provided by these tools. Like CBR Shells, programming skills are not required to develop VR cases using VR world builders. However, the task of choosing the tool that best suits domain requirements involves knowledge of their capabilities and evaluation criteria. Due to the capabilities of these CBR Shells and VR world builders, the initial approach for integrating CBR and VR was by choosing two appropriate tools and establishing communication between them. The first attempt to integrate CBR and VR was designed as event-based links between the CBR tool and the VR environment. Every action in CBR that had influence in the VR case, and vice-versa, would be communicated to each other via Dynamic Data Exchange (DDE) links. At this stage some objects such as a cube, a sphere, a pyramid and a cylinder were created in AUTOCAD Release 11 and imported into a VR tool called World Tool Kit that was being used in another research project in the Department of Surveying. These objects were thus converted in to an object-oriented language were worded attributes such as colour, object’s name, object’s behaviour, and shape were added emulating features of VR cases. Most of the potential CBR Shells to be used in the CBR-VR integration were able to handle DDE links and keep track over the visualisation process. However, these links would start and finish invisibly to the users who lacked control over the system’s actions. Another weakness of this approach was the need for the representation of each VR case and its possible dynamic changes such as object’s position, colour and features in both the CBR Shell and the VR world builder. Other factors discarding this integration were the unreliability of the DDE links, the dependency on the CBR Shell’s capabilities to handle dynamic changes on case attributes and the great amount of DDE links to keep track over the interaction with the VR cases. The second approach for the CBR-VR integration was based on an event-based blackboard implemented using a third part software. The blackboard was responsible for reactive and strategic activities, keeping track of the state of the domain and coordinating the set of events to be performed in real time. The blackboard eased the task of controlling the system’s actions as well as making the integration less dependent on the CBR Shell capabilities for case representation. However, this approach helped Page 162


neither the unreliability of the DDE links nor the inclusion of facilities to keep track of the interactive VR interface. The final decision was to use only the VR environment supporting the entire application. There were some known inconveniences in this approach, such as programming the retrieval mechanism, user interface to input case descriptions, and the user interaction with the instructional strategy in the training sessions. However, several other advantages could be acquired, such as no more machine crashes caused by the DDE links, straightforward case modelling in VR, and full control over the system’s actions. The VR package used for the development of this application was Superscape VRT version 4.0. This tool incorporates an environment for building VR worlds and a programming language that allowed the development of the CBR mechanism and the structure of guidelines for the training sessions. Further details on the evaluation of the software tools for building the VECTRA framework are discussed in the following subsections.

8.3.1 – The choice for the software tools Commercial CBR Shells come in a wide range of price and allowing facilities for case representation, indexing, retrieval and even handling digitised multimedia files. The decision for the CBR Shell that best fits the application domain and the instructional strategies to implemented is not an easy task. There are comparisons between the facilities of these tools. For instance, authors such as Althoff (1995) and Watson, I.D. (1997) have compared the performance and capabilities of Shells such as CBR Express, ESTEEM, ReMind, KATE and ART*Enterprise. Further details on the capabilities of these tools and a continually updated review of CBR Shells including some freeware and academic tools can be found at the AI-CBR web site∗. This thesis will not provide an evaluation of CBR Shells for two main reasons: (i) the VECTRA framework does not require a CBR shell, and (ii) due to the fast growing number and capabilities of these Shells. In this thesis, the evaluation of these tools was performed by middle 1995 and currently that evaluation is outdated. For someone ∗

AI-CBR web-site: http://www.ai-cbr.org

Page 163


interested in developing an instructional CBR application relying solely in a CBR Shell it is suggested to consider the training goals, the requirements for case representation, and the instructional strategy, prior to deciding for the most appropriate Shell to handle the training course.

8.3.2 - VR development tools The choice of an adequate VR tool to design the VR environments could not be based on particular attributes of the application domain due to the aimed flexibility of the VECTRA framework in handling different application domains. Moreover, little was known about the particular requirements for representing the VR cases for the VECTRA-SI prototype. For those reasons the decision was made over general criteria, such as: •

cost: the tool could not be expensive due the limited resources allocated to this research nor requiring hardware differently from IBM-PC compatible;

•

designer friendly: due to the limited time scale of the research the requirements in skills for using the tool had to be relatively simple;

•

facilities to integrate with the CBR shells: this factor was required since the initial idea relied in using a CBR shell to retrieve and index the VR cases;

•

open interface capabilities: this was an option since the type of users interaction and the training evaluation required for the domain had not been evaluated at that time. In mid 1995 when the evaluation of the VR tools was performed only two VR

packages for IBM-PC machines were commercially available: Superscape Virtual Reality Toolkit from Superscape (VRT), and WorldToolKit from Sense 8 (WTK). In that time VRT was in its version 4 and WTK in its version 1.1. The VRT consisted of a suite of editors for the creation of VR worlds as described below: •

Shape editor: to create points in space that are connected together to define 2D facets that can be used to assemble 3D objects;

Page 164


•

World editor: to build the VR worlds from the shapes designed in the Shape editor allowing positioning, resizing, applying textures and animations, and using a programming language to build conditions for the interaction with the VR worlds;

•

Image editor: to import, create and edit images that can be applied to objects as textures;

•

Layout editor: to increase interface capabilities providing options such as changing the size of the window displaying the VR worlds, embedding the screen with buttons triggering functions, and instruments simulating control panels;

•

Resource editor: to create dialogue boxes and menus that can be used to get information from the users and trigger animations or programmed functions;

•

Keyboard editor: to assign functions to the keyboard allowing users to control the VR world interaction;

•

SCL editor: to write the functions’ code to be given to the objects so that they could react and inter-react with the users and other objects in the VR worlds. The Superscape tool uses and independent Visualiser for displaying and

interacting with the VR worlds created by the editors that run as an external application. The Visualiser is a freeware tool and allows users that have not acquired the VRT package to run and interact with the creation of VR designers. Another important facility of the VRT is the SCL programming language. SCL stands for Superscape Control Language and is a programming language similar to C++ that allows the creation of routines such as retrieval mechanisms, changing of objects’ attributes and message exchanging between the VR objects. Chapter 9 gives further details on the use of the SCL into the VECTRA-SI prototype. Sense 8 WTK was much simpler and contained about four hundred preprogrammed functions written in C++ allowing the navigation trough the 3D worlds. This tool did not provide editors for the development of the VR cases but requiring external 3D modeller tools. WTK had only a few functions to build simple polygons that could be put together creating complex objects. However, every creation required an external C++ code compiler (such as Microsoft Visual C++ or Borland Turbo C++) prior to displaying the VR world creation. Page 165


An alternative was the use of external 3D modellers such as the DXF format provided by the AUTOCAD and 3D Studio (from Autodesk Inc.). Although it may represent an advantage for designers who are used to CAD tools, each file imported was treated as one single object. WTK allowed to import a combination of independent DXF files to build the VR worlds. However, this approach required an input of geometric co-ordinates placing each individual DXF file into the VR cases. Differently from VRT whose price has gone down, WTK is now at the same price range and containing tools for the design of the VR worlds. Further details on the current capabilities of these tools and other VRML modellers are presented in the Appendix 2 of this thesis.

8.3.3 - The choice for Superscape VRT 4 The VR package used for the development of this application was Superscape VRT version 4.0. It incorporates an environment for building the VR cases and a programming language that allowed the development of the CBR retrieval mechanism and the structure of guidelines for the training sessions. Superscape VRT was more expensive than the WTK and lacked facilities to communicate via DDE with CBR Shells. However, the VRT tool was more powerful in terms of facilities to build the VR cases. Another reason that helped the decision for the VRT was the access to the attributes of each individual VR object via the SCL programming language. A comparison between the capabilities of VRT and WTK for designing the VR cases is presented in Table 8.3.3. This comparison was performed in mid 1995 and at that time the VRT was in its version 4 and the WTK in its version 1.1. Currently, VRT is on version 5.5 and WTK on release 6 and the capabilities of both these packages have been improved.

Page 166


Capabilities

Superscape VRT

Sense 8 WTK

Linking to external CBR shells

VRT was designed to run on DOS but the Visualiser was Windows and functions were available to establish the DDE links.

WTK allows the capabilities of a powerful programming language (C++) for the communication with the CBR shells

Individual objects’ design

The VR objects are build in the Shape Editor and they can be as simple as a 3 point facet or as complex as the developers’ skill and creativity can reach. The Shape editor contains several editing capabilities, such as copying, extrusion, and duplication to speed up the creation of the VR objects.

Virtual world design

VR cases are built in the World editor from the shapes created in the Shape Editor. The World editor allows the grouping of individual objects to build complex objects and thus resizing, colouring, texturing, positioning and animating the VR cases. The capabilities of VRT include the organisation of the object-oriented hierarchies and the attachment of SCL algorithms.

Image and texture editors

Digitised pictures are built in the Image editor and applied to the objects as textures in the World editor. VRT supports a number of image compressing file formats.

There are functions to import and apply image files as textures to objects. The images will always be treated as external files and have to be present with the final executable VR file.

Sound editor

Sounds can be recorded, edited and imported in the Sound editor and attached as properties of objects in the World editor. Sound files are incorporated to the VR world in the World editor.

Sound files are like image files and there are no facilities for recording or editing the sounds. Sounds have to be created and edited by external tools and will be accessed as external files.

Menus and dialogue windows are created in the Resource editor. This Editor provides models that can be used and altered and each menu is part of an object-oriented hierarchy and accessed by SCL functions defined in the World editor.

The menus and other facilities for users input and output rely on the capabilities of the C++ language used to write the algorithms. Although C++ compilers allow facilities to develop menus and dialogue boxes, skill for programming them is required.

User input/output interaction

WTK did not contain facilities for designing the VR objects and relies only on its capability to import files from external 3D modelling tools.

WTK contains functions for handling the VR cases but C++ programming skills are required for designers interested in the WTK. The use of these functions requires passing of parameter to the C++ functions in the unfriendly text editor interface of compilers such as Borland or Microsoft C++.

Table 8.3.3 - The capabilities of VRT and WTK for building the VR cases.

Page 167


Although WTK was also using an object-oriented programming language, the virtual environment was following the architecture of its source package. It means that VR worlds built by 3D modellers such as AUTO-CAD and 3D Studio would require the original package to alter the properties of individual objects. On the other hand, the SCL programming language and the VR worlds in VRT were providing the full capabilities of an object-oriented architecture. Section 8.3 shows that the final decision in this thesis was in using only the VR tool to hold the whole VECTRA framework. An important factor supporting this decision was actually the capabilities of the VRT tool. This tool could also handle external routines developed using other programming languages for situations requiring sophisticated retrieval mechanisms and/or user interface. The VECTRA framework was thus fully developed in VRT and the following section describes the approach used for the VR case representation.

8.4 - The VECTRA framework Following the methodology for the development of the VECTRA framework, this stage deals with the development of the independent mechanisms that are part of the CBR working cycle to be inserted in the VR environment. This stage has been carried out following the incremental and evolutionary prototyping approach that suggests a break down of the system in individual mechanisms so that each can be tested independently and then added to the whole. Mechanisms such as the retrieval engine, the VR case base, the indexes and the user interface should though be designed to interact with one another in such a way to comprise with the actual functional requirements of VECTRA applications. Moreover, the independent mechanisms should also comply with the facilities of the VR technique used to create them. This framework relies on an object-oriented language that is part of the VR package used in this thesis. The object-oriented programming language used in this work is exclusively found in the Superscape VRT. However, other object-oriented languages could be used in to build the VECTRA framework and thus details on the Page 168


particular software package used in this thesis are only discussed when it is necessary to illustrate particular aspects of the VRT tool. Figure 8.4 shows and overview of the VECTRA framework, specifying its main components and their functions. This figure shows that this framework is divided in two main components that are the Index and the VR case repository. These two components contain their own object-oriented hierarchies in independent VR world files accessing each other. The Index component holds facilities such as the retrieval mechanism and the general configuration of both hardware devices for the interaction and the instructional strategy. The Index also holds the features for case retrieval and facilities such as pop down menus and help providing general instructions on system’s capabilities and user’s interface. The VR case repository holds the components required for the creation of the case-base of domain knowledge and involves object-oriented hierarchies holding facilities for case representation, case indexing, instructional strategies and capabilities for navigating the VR cases. There is also a retrieval mechanism that, differently from the one in the Index, can only retrieve the instructional activities inside an individual VR case. This mechanism is useful for applications where individual cases contain more than a single instructional activity. Obviously that this framework is the result of a process of successive investigation of the possibilities to build VECTRA applications. For instance, the possibility to design VR cases containing more than a single instructional activity was only raised during the process of designing the VR cases for the VECTRA-SI prototype (see section 9.4.3). This fact had an influence over the final design of the internal objectoriented structure of each VR case in the VECTRA framework. Further details on the structure for case representation are described in the following section.

Page 169

Chapter 8 – The VECTRA framework VECTRA FRAMEWORK VR CASE REPOSITORY

VR case file Navigation capabilities

Domain components

INDEX

3D human

Physical components

System configuration Interface

Retrieval mechanism

Hardware devices

Case retrieval

Instructional activities

6D aeroplane

Instructional guidelines

Instructional media

Help facilities

Retrieval mechanism

Learning evaluation

VR case indexes

Indexes Case features

Feature weights


Back to Index

MOP features

Script features

Fig. 8.4 – Overview of the VECTRA framework Page 170


8.5 - Case memory structure The structure for case representation in the VECTRA follows the concept of Memory Organisation Packets (MOP) and Scripts, described in Shank (1986; 1996) and Section 2.8 of this thesis. This concept says, for instance, that a construction site with a scaffold structure serves as a MOP for an expert, and that the several activities involved in its inspection constitute what the author calls Scripts. Another use of the Schank’s (1996) concept of MOP and Scripts that has been taken into this work regards the instructional activities in the VECTRA framework. Each case (or MOP) in the VECTRA framework contains a simulation of an on-job experience that is represented in a unique VR case file and each Script represents an instructional activity performed over the VR site simulation. Using again the scaffold inspection domain as example, to instruct the inspection of a particular scaffold structure requires instructing the inspection of several components where each one is seen as an instructional activity. This VR case file is composed of object-oriented hierarchies in a “tree” structure. This structure is illustrated in Fig. 8.5 and shows that each VR case contains independent hierarchies holding facilities such as the physical objects contained in the VR cases, the instructional guidelines, the retrieval mechanism, and the menus and dialogues for getting information from the user.

VR case Instructional guidelines

Instructional retrieval Domain components

Evaluation dialogues

Evaluation dialogues Instructional sequence

Case attributes

Retrieval engine

Anchor Instruction attributes

Fig. 8.5 – Object-oriented hierarchies in the VECTRA framework.

Page 171


In the VECTRA framework these object-oriented hierarchies are created in the Superscape World Editor and each object contains a number of attributes that are automatically given by the tool, such as dimensions, orientation, geometric position, and movements. Although some of these attributes cannot be deleted, the values can be changed and for instance changing the value of a par from “Visible : Yes” to “Visible : No” the object becomes invisible for the user. Apart from the objects representing the physical components of the domain, they are invisible for the user and do not play a role in the animations but are used for controlling the instructional interaction. These objects hold attributes in a value-pair format that can be dynamically changed by the information inputted by users. An example of a dynamic change of an object’s attributes is given in Section 8.6 where an invisible object representing a “dialogue box” receives new values for its attributes and is made visible to get an input from the user. Further details on these object-oriented hierarchies and the role these objects play over the VECTRA framework are discussed in the following sub-sections.

8.5.1 - Case featuring General aspects in choosing the features that properly describe and differentiate the VR cases and Scripts into the CBR repository are discussed in Section 4.4.2. Examples of the features used for the domain of training scaffold inspection are discussed in Section 9.4.4. This section describes the structure for featuring cases in the object-oriented hierarchies of the VECTRA framework independently from the application domain. In the VECTRA framework the VR cases and Scripts are featured independently. This approach allows the retrieval mechanism to search for both cases and instructional activities independently. Case and Script’s features are represented as attribute value pairs over the instances of the objects named Case_attributes and Instruction_attributes as shown in Figure 7.5.1. There is no limits to the number of features describing the cases and their Scripts. Developers are thus allowed to include as many features as necessary to properly describe cases and Scripts. These features are represented in a text format and though the retrieval mechanism is not case sensitive, there are no facilities for Page 172


correcting misspellings or to search for synonyms. In order to avoid possible problems, pop-down menus are provided where users can select the features correctly spelled. Instructional retrieval Case attributes

Retrieval engine

Instruction attributes

Case features Feature1: Value Feature2: Value Feature3: Value Feature4: Value : FeatureN: Value

Script N

Script 1 Feature1: Value Feature2: Value Feature3: Value Feature4: Value : FeatureN: Value

Script 2 Feature1: Value Feature2: Value Feature3: Value Feature4: Value : FeatureN: Value

Feature1: Value Feature2: Value Feature3: Value Feature4: Value : FeatureN: Value

Fig. 8.5.1 – Structure for featuring MOP and Scripts

Case retrieval is performed by the retrieval mechanism within the Index file. The case name and its features are loaded into the memory and passed to the Index file every time the case is retrieved. This upload of case features in the Index file allows the application to keep up-to-date with changes in the case features and with the creation of new cases. The retrieval mechanism inside each VR case searches for the features in every instance of the object named Instruction_attributes. Each instructional activity is featured in an independent instance and requires the creation of one object describing it. This approach allows the retrieval mechanism to search for the attribute value pairs that best matches a user’s description of the instructional activity they are searching for. Moreover, developers can add instructional activities that should thus be accompanied by a new instance where the attributes are set to receive the values describing this new added Script. Further details on the retrieval mechanism inside the VR cases are described in the next section.

8.5.2 – The retrieval mechanism The retrieval mechanism in the VECTRA framework is a computer program written in object-oriented language. The mechanism searches for object’s value pairs that match a description of a past experience simulated by the VR cases. This search Page 173


for objects matching a user’s description is limited to certain types of objects that have been especially created to hold MOP and Scripts’ features. These objects are kept as instances of the object-oriented hierarchies described in the previous section and Figure 8.5.1. There are two levels of search over the applications developed into the VECTRA framework that are the search for cases and the search for instructional activities the cases contain. Using the scaffold inspection domain as example, someone interested in learning how to inspect the attachment of the vertical bars perpendicular to the wall in a Putlog∗ scaffold would have two items to describe: (i) searching for a scaffold structure that involves a Putlog type of structure; and (ii) searching for the instruction on how to inspect this particular scaffold component. The searching mechanism in the VECTRA framework for both case and Script retrieval relies on the nearest-neighbour approach (see Section 4.4.3.1). The code of these algorithms is written in Superscape SCL and this programming language, though similar to C++, is exclusive for VRT users. In order to ease understanding of the code logic and performance, Figure 8.5.2 describes the algorithm for case retrieval translated to English language. The mechanism for Script retrieval follows the same logic. The searching mechanism also allows weighted features and this means that some features of the MOP and Scripts could be of more importance over the retrieval. The option for this retrieval technique is due to the fact that other types of retrieval mechanisms have been tested on the CBR shells ART Enterprise (Brightware Inc.) and ESTEEM (IntelliCorp). These two shells rely on an object-oriented language for case representation and due mostly to the number of cases and the current processing speed of IBM-PC compatible machines, the influence of the different retrieval mechanisms has interfered neither on the time nor on the precision of the retrieval. However, applications involving thousands VR cases and instructional activities may require other types of retrieval mechanisms. Further details on this searching approach are discussed in Section 4.4.3 of this thesis that also provides some insight on the usefulness of different approaches for case retrieval. ∗

Putlog scaffold is a structure that has only one line of vertical bars and utilizes the wall to support the inside edges (BS 5973: 1990).

Page 174


Case Retrieval /* identifying total weight for features in each case */ Loop for X from 1 to Total_Number_of_Cases Loop for Y from 1 to Total_Number_of_Features Total_Weight_Case (X) = Total_Weight_Case (X) + Feature_Weight (Y) /* matching inputted case features */ Loop for X from 1 to Total_Number_of_Cases Loop for Y from 1 to Total_Number_of_Features If Feature (Y) = True Total_Match_Case (X) = Total_Match_Case (X) + Feature_Weight (Y) /* establishing weight for each case

*/

Loop for X from 1 to Total_Number_of_Cases Total_Match_Case (X) = (100 * Total_Match_Case (X) / Total_Weight_Case(X) /* allow retrieval for cases matching more than 50% of the input Loop for X from 1 to Total_Number_of_Cases If Total_Match_Case (X) > 50% Then “display Total_Match_Case (X)and allow retrieval for Case (X)”

Fig. 8.5.2 – The Retrieval algorithm

8.5.3 - Case adaptation Section 4.4.5 shows that the case adaptation plays an important role over the maintenance and updating of CBR applications and classifies the adaptation process in two main types that are automatic and manual. Section 6.6.5 shows that both types of adaptation could take advantage of the access of the object-oriented hierarchies contained in the VR cases. The VECTRA framework does not contain facilities for automatic case adaptation. However, the manual adaptation over the VR cases in the VECTRA framework play an important role over the creation of new VR cases. This adaptation is performed over a set of object-oriented hierarchies that are shared by the VR cases. Case adaptation in the VECTRA framework is reduced to taking these hierarchies to build new cases. For instance, one of the object-oriented hierarchies in the VECTRA framework contains objects capable of moving the viewpoint. Instead of programming new routines to move the viewpoint in the new VR cases, developers could take these objects and only provide the starting and the final viewpoint and the SCL functions in these objects will move the viewpoint. The use of the object-oriented hierarchies is not limited to the standard contents of the VR case in the VECTRA framework. For instance, in the domain of scaffold Page 175


inspection, there are several components of the structure that are repeated for several cases. It means that the first VR case built could involve the creation of complex objects, such as the scaffold tubes, couplers, bolted ties, and gin wheels. Once these objects are created, they can be re-used for other VR scaffold structures. Then the task of designing new cases can be simplified to the positioning of these objects into the new VR case. Figure 8.5.3 shows how the object-oriented hierarchies composing each VR case can be used to help design new cases. In the VECTRA framework designers can take most of the object-oriented hierarchies that are part of previous VR cases to ease the creation of new VR cases. This adaptation differentiates the VECTRA framework from case representation by digitised multimedia files requiring the creation of new files for case adaptation. Standard VR case hierarchy

VR case A

Adapted VR case VR case B

Fig. 8.5.3 - Structure for case adaptation.

There are also object-oriented hierarchies created to represent the visible domain components that could also be taken to build new cases. However, to take the most from adaptation, developers are required to properly organise the hierarchies of domain objects. The VECTRA framework could thus be seen as an approach to organise object-oriented hierarchies for the development of VECTRA applications. An example organising the object-oriented hierarchies representing domain components in an application involving training for scaffold inspection is shown in Section 9.4.3. Page 176


8.6 – Instructional facilities The VECTRA framework provides facilities for the implementation of dialogue boxes and menus to perform the learning evaluation at the end of each instructional activity. The instructional activities into the VR cases depend on the domain and Section 8.4.2 provides details on the scaffold inspection domain. This section discusses the importance of learning evaluation and how it is supported in the VECTRA framework. The effectiveness of the whole training process starts with the evaluation of each instructional activity in the training application. This evaluation tests each instructional activity to determine whether the intended instructional goal has been achieved. Authors such as Rae (1991); Shapiro (1995) and Hall (1997) cited that this is a difficult though necessary task of revision to be repeated until the ratio of correct responses is acceptable. Tests evaluating user learning are an essential part in the design of instructional activities that involves the company offering the training course, application designers, domain experts and trainees. Learning evaluation allows companies and learners to measure the instruction acquired from the tool, the appropriateness of the training provided and the need for further steps to improve both the system’s capabilities and the users’ working skills. Learning evaluation takes place at three main stages that are (i) at the end of each instructional activity in the training tool, (ii) at the end of the training course, and (iii) when users behavioural changes are observed over their on-job activities to be improved. For the organisation served by the training application, the measurement required is over the on-job performances. The aim of training applications is to bring about behavioural change and trainees are expected to do something in a different way from that they use to do the same thing before training. However, the responsibility for the behavioural change evaluation is that of the company’s management. The evaluation at the end of the training course involves the skills that trainees obtained to solve specific problems rather than the process that was used by the experts in the case repository. It is a task that involves criteria to measure trainees’ performance to help decide whether the training tool provided its instructional goal. This evaluation identifies the percentages of trainees that were successful with the Page 177


training course and the appropriateness of the training tool. This task is responsibility of domain experts and reflects the individual’s learning from the training tool. The VECTRA framework provides facilities to evaluate the learning of each instructional activity. Each VR case contains an independent object-oriented hierarchy whose attributes allow designers to present a question, a few possible answers and the actions that each answer leads to. The objects used for evaluation show a window at the end of each instructional activity similarly to Figure 8.6a.

Fig. 8.6a – Evaluation test for instructional activity

These evaluation objects contain slots capable of receiving from two to six possible answers and each one leading to an action such as congratulating users for giving the correct answer or providing instructions about the reasons that took users to choose that specifically wrong answer. An example of passing attributes to an evaluation object is presented in Figure 8.6b that shows part of the input that designers are required to give to the VECTRA application. Error! Objects cannot be created from editing field codes. Fig. 8.6b– Passing parameters for an evaluation object

This evaluation approach allows checking out whether users have learned properly and thus move onto another instructional activity. If the users fail to answer the question, the system could indicate the errors and provide users with facilities to revise and try again. More sophisticated tasks such as re-simulating an instructional action or moving the viewpoint to highlight a specific component are also possible. However, these tasks would require more elaborated algorithms to be developed in the object-oriented language used by the VECTRA framework.

Page 178


Authors such as Rae (1991) Philips (1997), Tucker (1997) and Hall (1997) present a number of methodologies and techniques for learning evaluation. These authors also provide an indication on the types of domains that are best suited by some evaluation methodologies. A common viewpoint is that evaluation is a must for instructional applications and should be considered from the initial development stage. It is unfortunate that evaluation studies of training are somewhat rare. Phillips (1990) cited a survey of management training carried out in 1977 in which only 24% of 3100 executives reported that change in job behaviour was measured and 52% relied exclusively on feedback from trainees. Only 1.8% calculated the return on their training investment and even these were limiting the tests to the measurements of total production rate (Phillips 1990). Independently of the difficulties in choosing the evaluation techniques, designers of VECTRA applications have to identify questions capable of testing user learning. It is a task that involves domain experts and characteristics of the analogical learning provided by past experiences (see Section 2.10). The evaluation for VECTRA applications thus involves the usefulness of the knowledge transfer from past experiences to the actual on-job performances. Further details on the learning evaluation in the VECTRA-SI prototype are discussed in Section 9.6.

8.7 - Synthesis of the chapter This chapter describes the development of the VECTRA framework, from its conceptual stage to its capabilities to help building VR case-based instructional applications. The internal structure of the VECTRA framework and each of its components are described in this chapter, following the prescriptions of the Prototyping Methodology. The resulting framework could be seen as a model that organises objectoriented hierarchies capable of holding and easing the development of interdisciplinary VECTRA applications. The VECTRA framework relies on a case-based instructional approach that imposes limits to the flexibility for the design of the instructional activities and for the type of learning users can take from them. On the other hand, this limited flexibility reduces the amount of work involved in developing instructional applications. In the VECTRA framework, all that is required to build instructional applications is to gather

Page 179


past experiences of the domain, build the VR simulations, point out the instructional activities within the simulations, and provide features describing the cases and their instructional activities. It is difficult to evaluate in which domains the VECTRA framework could be most appropriate suited since only the scaffold inspection domain was evaluated. A general recommendation is that the VECTRA framework offers a greater chance to support domains involving physical representations of spaces and moving viewpoints. Domains involving motor skills or a perceptual sense can also be helped when provided with the appropriate degree of interaction. Further details on the training application for scaffold inspection are given in the following chapter.

Page 180

Chapter 9 - The VECTRA-SI prototype

9.1 – Overview

The previous chapter describes the VECTRA framework and its capabilities and requirements for holding interdisciplinary instructional applications. This chapter describes the development of an application for training on scaffold inspection and thus complements the previous chapter by providing examples of the implementation of an application in the VECTRA framework. The VECTRA Scaffold Inspection (VECTRA-SI) is a prototype and only a few cases have been incorporated at this stage. The cases have been acquired from experts and the processes of gathering, modelling, and implementation of the VR cases in the VECTRA-SI repository are described in this chapter. An analysis of the role the VECTRA-SI plays to its application domain and the comments of the experts on the prototype are described at the end of this chapter.

Page 181


9.2 – The VECTRA-SI prototype The VECTRA-SI prototype was developed to explore the potential of the VRCBR integration, which resulted in the VECTRA framework, for the development of intelligent instructional applications. The main reason behind the prototyping task was to evaluate the type of work involved with the development of an application in the VECTRA framework. Other reasons encouraging the task of prototyping were: •

to evaluate VR capabilities to represent past experiences;

•

to evaluate feasibility of PC machines handling VECTRA applications;

•

to identify the capabilities of the application in terms of training tool for both trainers and trainees;

•

to analyse the potential and weaknesses of the application as an intelligent training tool; and

•

to help promote the feedback from the experts involved in this research. The past experiences of experts on scaffold inspection have been implemented in

the object-oriented architecture of the VR tool used in this thesis. An overview of the VECTRA-SI architecture is described in Figure 9.2 and complements Figure 8.4 by setting the domain components and the instructional activities of the scaffold inspection over the VECTRA framework. The VECTRA-SI architecture follows the VECTRA framework described in Section 8.4 and contains two main components that are the Index and the VR case repository. The Index component works in a similar way to an empty CBR Shell and embodies the retrieval mechanism, the facilities for the general system configuration, and for the allocation of the features describing the VR cases. The VR case repository is the core of the VECTRA-SI application and holds the domain knowledge and the instructional activities in the form of VR simulations of past on-job experiences. Each past experience is represented in an independent VR file that is composed of object-oriented hierarchies holding the representation of the VR cases. Facilities associated to these object-oriented hierarchies are:

Page 182


VECTRA-SI PROTOTYPE VR CASE REPOSITORY

Object-oriented hierarchies Domain components

INDEX

Guided walk-through

Scaffold components

System configuration Interface

Retrieval mechanism

Building components

Hardware devices

Case retrieval

Site components

Navigation capabilities


Free 6D exploration

Guided instruction

Free exploration

Help facilities

Retrieval mechanism

Learning evaluation

VR case indexes

Indexes Case features

Feature weights


Back to Index

Case features

Script features

Fig. 9.2 – Overview of the VECTRA-SI prototype Page 183


•

the VR representation of the physical components of the domain and includes the scaffold structure and the construction site where the past experiences took place;

•

the instructional activities in each VR case and include the domain training requirements, the instructional guidelines, and the learning evaluation;

•

the facilities for moving the viewpoint round the scaffold structure in accordance with the guidelines for the instructional activities;

•

the features of the VR cases and the instructional activities that each case contains; and

•

the mechanism for matching and retrieval of the instructional activities; Currently, the VECTRA prototype operates with three cases of scaffold

inspection where the structures are (i) providing work on the roof top of a two-storey building; (ii) building a two-storey house, and (iii) cleaning and providing small repairs (light work) on the front wall of a two-storey building located in a high-traffic area. Each VR case is a simulation of a real on-site scaffold structure that challenges users’ skills on the various tasks involved in the inspection of health and safety regulations. The VECTRA-SI addresses CBR issues such as case representation, case indexing, case retrieval and case adaptation. These CBR issues are combined with the instructional requirements of the application domain. Further details on the task of instructing scaffold inspection and a comparison with the role the VECTRA-SI can play over this task are described in the following section.

9.3 – The task of scaffold inspection Scaffold inspection is a task that relies mostly on three sources of information that are the regulations described in the British Standards (BS 5973: 1990); field experience of professional inspectors, and books providing guidelines for building those structures. New technologies for scaffold using differently shaped components and made of innovative materials require inspectors to be involved in a lifelong learning practice. Most of these innovations are first seen by the experts on the construction site, thus requiring a theoretical domain background involving structural safety associated with the equivalent old technology. These new technologies may be added to future Page 184


editions of the main reference on scaffold inspection (BS 5973: 1990). However, this reference provides only a code of practice and inspectors cannot declare that a certain structure is not conforming with the regulations because it is using innovative components. Due to the characteristics of the scaffold inspection task, expert decisions rely mostly on empirical knowledge that is composed of common sense and heuristics related to a body of knowledge built from their past experiences. Therefore, past experiences play an important role over the apprenticeship of scaffold inspectors and further details on the way they perform their job, build their learning and the VECTRA-SI approach for training are described in the following sub-sections.

9.3.1 - Experts approach to scaffold inspection Inspection of health & safety regulations on scaffold structures rely on checking whether the components comply with the requirements of the BS 5973: 1990. The inspection aims to identify whether the structure has been properly erected, is safe to work on, and safe for anyone on its surroundings. The inspection task is performed with the experts moving round the scaffold and checking key parts of the structure. The experts do not follow a pre-established sequence when inspecting the health & safety regulations (though some prescriptions do exist) but cover a checklist of key structural components of the scaffold. For instance, when the experts are in a certain position relative to the structure, they inspect all the components that can be seen from that point. This non-sequential and unstructured approach for inspection makes sense to experts who know the requisites and key structural components that have to be inspected. However, novices are not expected to have all the possible irregularities of scaffold structures in their mind from the beginning of their apprenticeship. There is a need to move round the structure to reach sight over these key structural components. This need to move the viewpoint could be simply performed walking round the scaffold when it is relatively small. However, it is common to find scaffold structures whose size and complexity requires inspectors to walk-through the working platforms wearing special protective equipment to reach sight on the scaffold Page 185


components. This walk-through the structures can be a threat for novices in training courses and further details on the training task and the way experts learnt the domain are described next.

9.3.2 – Experts approach to training As an example of real education on scaffold inspection, instructors usually take trainees to real construction sites. As part of the course, different scaffold structures are visited and inspected. Pictures and films of scaffold structures are also used in the training courses. The main reason behind the use of the representation of previous scaffolds in training courses is to emphasise and illustrate the instructions. Their use is then carefully chosen in accordance to the instructional strategy adopted for the training course. A common instructional strategy adopted by trainers relies on firstly providing general knowledge about the principles involving the scaffold inspection and the importance of the task they are going to learn. Learners will then become aware of the situation they are going to face and the difficulties that are likely to happen in real-job situations. When trainees are quite aware of the domain and the tasks they are expected to perform, they will then be taken to on-site scaffold structures. Obviously a training course in scaffold inspection could be completely given only by visiting real on-site structures. However adversities such as weather conditions, availability of real structures, access to sites and safety reasons could make it difficult for the trainers. Pictures and films of scaffold inspections can help by illustrating some concepts but lack the interaction and presentation of aspects of reality that VR and onsite training could support. Despite the type of training course and where it takes place, the issue here is not to lead to the conclusion that only by visiting real on-site scaffold structures novices would not be able to learn about the domain. As described in Section 2.5, researchers such as Gagne (1992) and Schank (1995) cited that there is nothing as effective for instruction as the real experience. However, training activities performed over on-job situations can be dangerous and also require instructional strategies.

Page 186


Actually, the experts that took part in this work had no special courses on scaffold inspection. Their learning was taken from site experiences with the guidance of experienced professionals. Unfortunately in this work has not been possible to compare the efficiency of the training the experts had with the one from a carefully designed course. However, the number of trainees each course can reach and the ease to reuse the training course in the future are factors that give advantages to documented and structured training courses. Some of the experts involved in this work had experience as trainers in scaffold inspection. Therefore, guidelines for the training sessions in the VECTRA-SI have been modelled under the supervision of these experts. Further details on the instructional approach that is part of the VECTRA-SI prototype are discussed in the next section.

9.3.3 – VECTRA-SI training approach The instructional strategy in the VECTRA-SI relies on sequencing the tasks that experts perform inspecting scaffold structures. Section 9.3.1 shows that experts do not follow a pre-established sequence inspecting scaffold structures. However, a sequence for their actions has been modelled to provide the instructional guidelines of the VECTRA-SI. The sequence for the instructional activities has been modelled from the interviews with the experts and from site inspections where the author accompanied the experts. The instructions have thus been organised and sequenced following the professional’s advice. Those professionals agreed with an educational practice suggested by Dean (1992) and Gagne (1992) that relies on providing the instructions from general domain knowledge to more specific activities. The first recommendation of the expert for performing an inspection on a scaffold structure was:

“the first thing to do is to get away from the structure to check whether the vertical components look vertical, the horizontal components look horizontal, and the structure as a whole give an appearance of structural safety”.

Obviously this “appearance of structural safety” has a meaning for the experts that may not be the same for novices. This is the situation where novices are suggested

Page 187


to check the spacing between the components in accordance with the regulation in the BS 5973: 1990. In fact, even the experts recur to this book under certain situations. The difference is that the novices may have to recur to the BS 5973: 1990 more frequently than the experts. The second task in inspecting scaffold structure, as taken from the interviews with the experts, was:

“ to check whether the scaffold is well tied to the building façade and there is no risk for the structure to fall outwards the building. These ties also give some degree of lateral restraint helping to keep the whole structure steady”.

The frequency of points for attaching the scaffold to the building façade depends on factors such as the type of ties being used, the total size of the structure, the type of scaffold and the number of working platforms. Experts have a “feeling” about the requirements on the number and spacing between the ties but for the exact number they also recur to the tables in the BS 5973: 1990. As part of the interviews, the full inspection of a few scaffold structures have been performed and modelled in this thesis. Modelling techniques played an important role in this process providing both documents of the inspection tasks and a source for further discussion with the experts. The modelling technique used at this stage was a list sequencing the inspection tasks, which was easily understood by the experts and provided a document of the inspection where further refinements could be done. Table 9.3.3 shows a list of tasks involved in the inspection of independent tied scaffolds. The sequencing for the tasks in the inspection of scaffold structures, though it does not represent the actual way the experts perform their work, it helps to build the instructional strategy in the VR cases. For a CBR application, it means that the case gathering does not simply involve the representation of past on-job experiences. Rather, it involves the gathering of an instructional past experience, where an expert takes a novice to the site.

Page 188

Chapter 9 - The VECTRA-SI prototype Overall appearance of structural safety Standards Bay-length Bay-width Spacing between transoms Longitudinal bracing Ledger-bracing Foundations Sole-plates Façade attachment Gin-wheels, jib cranes and hoists Distance to the building Working platform Width of working platform Length of the platform Spacing between transoms Board conditions Slope of the platform Gaps between boards Thickness of the boards Board overhang Guard-rails Toe-boards Brick-guards Additional Protection to the public Sheets Lighting Lifting appliances Electrical supplies Hoist ways Ropes Table 9.3.3 – Checklist of activities inspecting scaffold components.

It is expected that domains where the training involves a rigid sequence of activities could be easier to model. However, this unstructured approach for performing inspection is also expected for a wide range of domains. For instance, activities involving inspection, such as structural safety of buildings and bridges also present this non-sequential approach in real situations. The implementation of the VR cases in accordance with the characteristics of the scaffold inspection domain and the instructional activities modelled from the contact with the experts are described in the next section.

Page 189


9.4 – VECTRA-SI case-based instruction Each case in the VECTRA-SI prototype is represented as a VR simulation of an inspection of an actual scaffold structure. Instructional strategies that can be taken from the retrieval of past experiences are described in Section 5.5 and the theory behind this model of human cognition is described in Section 2.9. This section discusses the implementation of the VR cases in the VECTRA-SI. The case repository has been organised in accordance with Schank’s (1982) theory of MOP and Scripts (see Section 2.9 and 8.5). For instance, one of the cases in the prototype describes a site where repairs are being done on the rooftop of a twostorey building. To identify whether the scaffold complies with British Standards regulations (BS 5973:1990), certain tasks have to be performed. One of these tasks is to check whether the vertical bars (technically called “standards”) are centred on top of the soil plates. Any scaffold structure (that is not suspended) must have its bases well centred on top of soil plates. Thus, the task of inspecting if the soil plates are properly supporting the standards is an example of a Script (or instructional activity) that is common for several MOP (or cases). The experts involved in the development process of the VECTRA-SI prototype work for the Manchester City Council and two major scaffold companies actuating in Manchester-UK and the inspection of scaffold structures is part of their professional activity. An extraordinary incident had an influence over the interviewing process: an IRA terrorist bomb exploded in the city centre of Manchester, and the blast damaged several buildings and scaffold structures were required for repairs. This incident left the professionals extremely busy for months and raised an urgent need for trained personnel to help with the inspections. On the other hand, hundreds of scaffold structures became available as possible cases for modelling. A dozen of these structures have been visited by the author of this thesis as part of the case modelling process. At the beginning of the interviewing process the author did not have a knowledge background on the domain. The CBR references reviewed in Chapter 4 indicated that this lack of domain knowledge should not be a problem for developing CBR applications. Moreover, it would avoid any bias over the instructional activities of the application domain. Page 190


Due mostly to the need for modelling the instructional strategy as part of the VR case representation, it is nearly impossible for someone to be involved with the development of VECTRA applications and not learn about the application domain. Further details on the involvement of the author throughout the development process of the VECTRA-SI tool are described in the next sections.

9.4.1 - Case gathering The approach for gathering the cases to build the VECTRA case repository has initially followed the recommendations of CBR developers (see Section 4.4.1). These recommendations suggest that the focus of the interviews should be kept in asking the experts to tell those experiences that somehow have helped building their own domain knowledge. The first contact with the experts proceeded with the author playing the role of the interviewer and explaining the CBR working cycle, the reasons behind the development of the tool and how the case acquisition process was to be performed. This initial stage helped to identify problem areas in inspecting scaffold structures and the implications of the work of professional scaffold inspectors over the construction industry domain. As soon as the actual case gathering process started, it emerged that that the experts seemed to have a tendency to retrieve past experiences that were capable of invalidating occurrences rather then recalling a case capable of instructing an issue. There was thus a tendency for the experts to recall exceptions to the common practice as those were the experiences that provided the learning. Further discussions with researchers in mailing lists∗ showed that this fact was a normal occurrence in knowledge acquisition processes and Jennings (1997) provides further details on how this natural disposition can interfere and be positively used for instruction. The interviews were tape recorded and focused on acquiring cases capable of representing the experts’ approach to scaffold inspection as they were teaching a novice to perform this task. The interviewer played the role of a novice learning the domain and this approach was quite successful for identifying a training methodology. On the

∗

The most active discussion groups commenting the issue were the comp.human-factors and the sci.cognitive.

Page 191


other hand, the interviewing process went off the path indicated by CBR developers of gathering sparse and unconnected cases since the interviews were focused in acquiring cases capable of instructing the professional activities. Another factor that played a role in the case gathering process was the need to relate the cases gathered to the instructional activities in the VECTRA-SI prototype. Further details on the influence of the instructional strategy over the process of case gathering and the development of the VECTRA-SI prototype are described in the next section.

9.4.2 - Case-based instructional strategy The previous section shows that the case gathering process focused on modelling past experiences that could help to instruct the domain. Instructional requirements of CBT applications lead to the fact that simply recording and retrieving past experiences would not be enough for the VECTRA-SI application. CBT requires instructional strategies organising the training sessions, rather than a series of isolated domain cases (see Section 5.7). At this stage the presence of domain experts with training experience proved helpful. It was from the interviews with these professionals that the possibility of sequencing the actions involved in the inspection of scaffolds emerged as the instructional strategy of the VECTRA-SI. The interviews then turned into asking the experts to provide examples of past scaffold inspections and the sequence of the tasks they performed. Another consequence of the sequencing of the instructional activities was that instead of gathering cases containing isolated instructional activities, each case was thus involving all the actions performed over the inspection of a particular scaffold structure. This approach has later proven useful because the task of designing the VR scaffolds into the VR tool was a time consuming task and each VR case simulation could thus include several instructional activities. To enrich the system’s instructional capabilities, the VR cases have been added with sounds and written descriptions of the experts’ actions. The instructions also

Page 192


include moving viewpoints simulating the guidance of an expert taking the users to view the scaffold components. Simultaneously to the movement of the viewpoint, sounds emulating the expert’s guidance are played as part of the VR simulations. The inclusion of sounds and windows containing the verbal descriptions of the experts’ actions was performed by programming with the SCL of the Superscape tool. The insertion of sounds required synchronisation with the viewpoint movement and it was implemented breaking down the sound files in smaller parts and playing at intervals of the viewpoint movement. For instance, a sound file with 5 minutes could be divided in 5 files of 1 minute and each would start and finish between intervals of the viewpoint movement from A to B, as shown in Figure 9.4.2.

5 minutes playing sound file

1 minute

1 minute

1 minute

1 minute

A

1 minute

B Fig. 9.4.2 – Synchronising sounds and viewpoint movements

The written information was simpler to implement and was performed by dialogue windows where the user could read and click the mouse to move onto the next (or previous) dialogue. The difficulty in this task was in choosing the information to be added to each VR case and for each instructional activity. This task was performed with transcriptions of the video-recorded narrative of the experts’ instructions when taking the author to the real scaffold inspections. The VECTRA-SI prototype thus simulates the presence and guidance of and expert instructor. Other facilities have also been provided, such as clicking the mouse over visible VR objects and information such as the dimensions, material and name of the scaffold components (or the type of wall that is part of the building) are shown. Further details on the implementation of the cases into the computer tool are described in the following section.

Page 193


9.4.3 - Case implementation The previous section shows that the cases in the VECTRA-SI repository have been gathered from experiences where the designer of the VR cases (i.e. the author of this work) has accompanied the experts. Due to requirements of training applications for instructional activities, each VR case holds the sequence of the experts’ actions performing the scaffold inspection. This section discusses the implementation of the VR cases into the computer tool. The first step for the implementation of the VR cases involved the modelling of the physical components of the scaffold structure in the VR world builder. The conclusion from this stage was that the building of the VR cases requires more than worded descriptions of experts’ past experiences. Experts do not even recall details such as the dimensions and spacing of the scaffold components that are required for the design of the cases in the VR tool. The VR cases were thus implemented from scaffolds where the author of this work accompanied the expert’s inspection. Video records of the inspections and pictures showing the scaffold components and the construction site have been employed to document the inspections. Four of these pictures, taken from a scaffold structure that became one of the VR cases in the VECTRA-SI repository, are shown in the Figures 9.4.3a to 9.4.3d. Even with the visual documentation of the inspections, the dimensions and spacing of the scaffold components would need to be accessed by educated guesses since it is difficult to measure the actual dimensions from pictures or films. Lead to these requirements for the implementation of the VR cases, the decision was to implement scaffold structures that were still in use and could be revisited.

Page 194


Fig. 9.4.3a

Fig. 9.4.3d

Fig. 9.4.3b

Fig. 9.4.3d

Figs. 9.4.3a – 9.4.3d – Pictures of an on-site scaffold structure.

Once the first case was represented in VR it was given to the experts to check whether the VR simulations had the necessary graphical quality to be used for the instructions. The feedback from the experts viewing the VR scaffold structures highlighted a positive and a negative aspect that influenced the implementation of the VR cases. The positive was that the experts were comfortable with the quality of the VR simulations and they were even capable of identifying constructive mistakes over the VR representation. The negative aspect was that for the building of a VR scaffold structure it was required more than only pictures, films and components’ dimensions but knowledge about the regulations for building real scaffold structures. The process of implementation of the scaffold structures in the VR tool raised the need for domain modelling prior to the representation of the cases in the VR environment. By making explicit an object’s dimensions, positions, dependencies, and

Page 195


its links with other objects, information modelling techniques played an important role helping speed up the creation of the VR cases. This modelling process was only helping to design the VR cases and would not require feedback from the experts but only addressing the regulations in the BS 5973: 1990. The modelling technique was thus chosen considering two criteria (see Section 6.6.2): (i) the capability to handle the information required; and (ii) the developer feeling comfortable with it. The modelling technique used was Express-G and Figure 9.4.3e shows one of the models used to implement a case in VR (i.e. to build a scaffold structure in accordance to the BS 5973:1990). Attachment

Gin wheels

Raising materials

8.2

Boarded

17.1

Transoms and Putlogs

Ladder towers

construction

Attachment

Tab. 3, pp. 33

Spacing 14.3.2

Non-boarded

30.2 Length

Ladder

Attachment

Putlog blades

17.1

14.1

16.1

Independent

Landing

16.2

Scaffold type Attachment

Ledgers

Spacing

Scaffold Working Platform

Ledger bracing

Standards

Joints requirements Foundations

Mobile

10.2

Atachments

Building

10.2

Format

Erection tolerances

Putlog

13.1

10.2

Duty

Components

14.3.2

Height Lenght

Tab. 4.1; pp. 37 selection

Number of platforms

Tab.1; pp.17

Number of boards

Tab.1; pp.17

Bay lenght Lift height

9.5.1

Bay widht

11 Frequency

Ties

Scaffold Structure

Area

Layout

Sheeted

Tab. 1a; pp22

Unsheeted

Tab. 1a; pp22

Tab.1; pp.17 Tab.14; pp.86 Tab.2; pp.33

9.2

Building attachment

Lip ties

9.4.3

Box ties

9.4.2

Through ties

9.4.4

Reveal Ties

9.4.5

Capacity

9.4.3

Attribute Defined-type Numeric/Character

Entity

Select Sub-Type

Fig. 9.4.3e - Case model for further implementation in VR

Modelling techniques have also played an important role in helping to identify the information that needs to be added to the VR cases and instructional activities. Moreover, issues of case representation in CBR, such as the selection of case attributes and the features indexing the VR cases for retrieval, have also been carried out with modelling techniques prior to the implementation into the prototype. Further details on the task of featuring the VR case repository are discussed in the next section.

Page 196


9.4.4 – Featuring cases and Scripts Each VR case contains the complete set of tasks involved in the inspection of a scaffold structure and this approach has influenced the whole development of the VECTRA-SI prototype. Since the nature of the actions performed by the experts determines how the memory of the instructional activities needs to be organised, the task being instructed by the prototype becomes the determinant of its featuring for matching and retrieval. This fact lead to a situation where each VR case contains all it needs to be described and retrieved as an independent inspection. Moreover, in order to support the instructional strategy, where each inspection is performed as a sequence of tasks, it is necessary to feature each of the instructional activities. The approach for featuring cases and Scripts is described in Figure 9.4.4 and shows that cases and Scripts can be retrieved independently.

Fig. 9.4.4 – Accessing case and Script features.

In the VECTRA-SI the featuring process resulted from the combination of three main issues: (i) the CBR paradigm that requires features capable of differentiating the cases in the repository and properly addressing them for retrieval; (ii) the requirements of training applications in terms of using a language and allowing retrieval in such a way that suits a user’s background knowledge; and (iii) the capabilities of the objectoriented hierarchy used to represent the cases in VR. The language for featuring the VR cases and the instructional activities has been chosen between sufficiently abstract to be known by beginners and instinctive for Page 197


users more experienced with the domain. The process of selecting the features has been performed by interviewing the experts, who were advised to keep in mind the instructional objective of the application. The features for the cases include the type of scaffold, the type of work provided by the structure, the type of building, the scaffold dimensions and the characteristics of the construction site. The features for the Scripts have been chosen in terms of describing the scaffold component associated to the action performed for the inspection. Scripts’ features involves items in the inspection such as the soil-plates support, the spacing between the ledgers, the spacing between transoms, the overhang of the boards, and the frequency of ties. Table 9.3.3 shows a list of tasks to perform in the inspection of scaffold structures. The representation of the features for the VR cases and Scripts was performed relying on the object-oriented architecture provided by the VECTRA framework as described in Section 8.5.1. That section also shows that the case features are repeated in the Index file and the reason behind this redundancy is to avoid the need to open and read the contents of each VR file and search for its contents at each time the case retrieval mechanism is used. It could be a time-consuming task slowing down the retrieval process. The algorithm for retrieval and how it has been implemented in the objectoriented architecture of the VECTRA framework are discussed in Section 8.5.2. Further details on the role the retrieval mechanism and how it has been implemented in the object-oriented hierarchies of the VECTRA-SI are described in the next section.

9.4.5 - Retrieval of cases and Scripts The retrieval mechanism is part of the VECTRA framework and its algorithm is presented in Section 8.5.2. That section also shows that the retrieval mechanism for the cases are on the Index component of the VECTRA framework and the mechanism for Script retrieval is part of the object-oriented hierarchies of each VR case. This section describes the task of retrieving cases and Scripts in the VECTRA-SI prototype. There are thus two levels of retrieval in the VECTRA-SI prototype: (i) the case retrieval that brings the VR simulation of a scaffold structure; and (ii) the Script retrieval that brings one of the tasks involved in the inspection of the VR scaffold Page 198


structure previously retrieved. The retrieval of a Script is thus linked to the retrieval of the case where it belongs. For case retrieval the VECTRA-SI loads the Index component and an interface allows users to describe the cases (see Figure A1.3 in the Appendix 1). Script retrieval is only available if the VECTRA-SI has a VR case already loaded and this option brings an interface for input of the Scripts each case contains. The retrieval of both case and Script together is performed with the VECTRA-SI retrieving the case and thus displaying the interface for Script retrieval that each case contains. Figure 9.4.5 shows a flow chart describing the case/Script retrieval process in the VECTRA-SI.

Case and

Index component

Script

Start

Case / Script retrieval

Case

Script

Case description

Index component

VR case

Case description

Script description

Case retrieval

Script description

Case retrieval

Script retrieval

End

Script retrieval

Fig. 9.4.5 – Dataflow diagram of the for case/Script retrieval process.

The retrieval mechanisms and the allocation of the features for both cases and Scripts are standard for applications developed in the VECTRA framework. Section 8.5.1 shows the position of these facilities within the object-oriented hierarchies of each VR case and the Index component. New cases can thus inherit these facilities and the retrieval process will rely on the features of the new cases and Scripts. Further details on the adaptation process for the creation of new cases are discussed in the next section.

9.4.6 - Case adaptation Case adaptation for VECTRA applications is performed over the object-oriented hierarchies of the cases already in the repository and involves two aspects: •

the standard object-oriented architecture of the VR cases contains facilities that are independent of the application domain such as the retrieval mechanism, the allocation for the case/Script features, the routines to move the viewpoint, and the facilities for 2D and 3D navigation; and Page 199


•

the object-oriented hierarchies regarding the application domain and involve VR objects such as the components of the scaffold structure, the components of the construction site and the performances for the instructional activities. The former is described in Section 8.5 and this section describes the adaptation

process for the domain components and how this task has been performed in the Superscape tool. The latter involves the VECTRA-SI domain and it is expected that the characteristics of the VR scaffold objects can also be found in other application domains. Scaffolds are structures that involve thousands components such as tubes, boards, couplers and ties. Despite the number of components that are part of the structure, there are only a dozen that are differently shaped. For instance, the most common component of any scaffold structure is a metallic tube that receives names such as standard, ledger, bracing and transom, in accordance with the position and structural function in the scaffold structure. The implications of this fact to the design of the VECTRA-SI prototype is that only a few different shapes are required to build a VR scaffold structure. Once these shapes have been designed, they can be part of the VR objects library and re-used along in the creation of new cases. For instance, once one shape representing a tube is created it can be re-used as many times as required. Designers can thus associate this shape to the VR objects and only changing the dimensions and positioning of the VR objects in the VR cases. The degree of complexity for the design of VR shapes is due to the number of non-uniform facets required to represent a shape. Shapes such as pyramids and cubes are thus very simply designed, since they use less than ten facets and the edges are geometrically situated. Some of the components in the design of the scaffold structures, such as couplers and ties, have a quite complex shape and require time consuming efforts to be built in the VR tool. The Superscape VRT allows the creation of independent files containing shapes to be shared in the creation of new VR environments. Moreover, the objectoriented hierarchies involving the VR objects of the whole VR scaffold structure or the

Page 200


whole building where the scaffold is attached, can also be shared by VR cases built in Superscape. The creation of the first VR case was very time-consuming but the others were eased due to the re-use of its VR objects. A characteristic of the domain that influenced the design process of each VR case was the number of VR objects required to build the VR scaffold structure. Scaffold structures involve thousands of VR objects and the positioning of each piece in the 3D space of each VR case is an uninteresting and timeconsuming task. The 3D design of the VR cases requires different viewpoints that have to be constantly changed for the positioning of each VR object. The VR case adaptation process was thus performed over the facilities provided by the Superscape tool and once the domain objects have been built, the task of creating new VR cases is eased. Further developments in VR tools should soon provide tools containing more than a single window, i.e. a single viewpoint, for the positioning of the VR objects. These windows showing a synchronised view of the VR environments could also help the development of the users interface that is the issue discussed in the following section.

9.5 – User interface of the VECTRA-SI prototype The interaction with the VECTRA-SI prototype has been designed to cater for different levels of users. Beginners can blindly follow the system’s guidelines and retrieve a pre-defined sequence of cases and Scripts. Users experienced with the domain can retrieve the case and the instructional activity they want to reinforce knowledge. Trainers can use the VR cases to illustrate their lessons with a simulation of a past occurrence on site. Independently from the domain knowledge background, skills to navigate in the 3D VR cases are an asset. Walk through the VR environments driving a mouse, a track ball or a space mouse can be quite troublesome at the beginning. However, users willing to take the VECTRA and navigate through the virtual cases are expected to acquire some skills in navigating through the VR cases. The level of skills required to navigate through the VR environments is higher for trainers that want to move the viewpoint to illustrate their instructions. Further details on user interaction with the VECTRA are described below.

Page 201


9.5.1 The novice interface The VECTRA-SI prototype has been designed to provide instructional guidelines for the training of novices on the inspection of scaffold structures. Training applications, as described in Section 2.7, involve designing the instructional activities. Moreover, CBT must provide the stimulus for self-instruction, the motivation for the users to carry out the course, the instructional strategies and the means for evaluation that learning has effectively occurred. Section 9.4.2 shows that each VR case contains the whole set of actions involved in the inspection of simulated scaffold structures. The guidelines for training the novices rely on a pre-established sequence retrieving VR cases and instructional activities. Each instructional activity emulates the presence of an expert guiding a novice through the scaffold inspection. The instructional guidelines are also part of the object-oriented hierarchy of the VR cases (see Section 7.5) and have been implemented as SCL algorithms that move the viewpoint around the scaffold structure. This moving viewpoint allows novices to have a closer view of the scaffold components from the viewpoint suggested by the experts. Further details on the role of the guided moving viewpoint and an example of a training session with the VECTRA-SI are shown in the Appendix 1. The instructional activities for the novices are provided with sound emulating the voice-guidance of the experts. Users can also access a worded description of the experts’ advice in performing this task. Although it may seen repetitive or over informing users about the task performances, sounds and written information together play a role by coping with different learning preferences. Technically, sound files are quite hard-disk-space consuming and take an average of 90% of the total size of each VR file. The ASCII characters from the written instructional guidance practically do not take any space and once users know what they want, it can be quicker to access than listening to the whole sound file. Moreover, users can let the written information statically in the computer screen and get back to it when desired. Users more experienced with the domain knowledge do not need to follow these guidelines, however they are required to navigate through the VR environment. Further details on the interface for intermediate are presented in the next section. Page 202


9.5.2 - The intermediate interface Section 9.3.1 shows that experts do not follow a pre-established sequence when inspecting health & safety regulations on scaffold structures. This freedom to choose the sequence of the tasks inspecting scaffolds is quite in accordance with the CBR working cycle within the VECTRA framework, where users can retrieve past experiences to help reason on the problem situation they are facing. The VECTRA-SI allows users to retrieve the case they want, according to the nature of their problem. This task relies on the retrieval mechanism and allows users to search for a particular case and/or instructional activity. The algorithm for the retrieval mechanism and its location on the object-oriented hierarchies of the VECTRA framework are presented in Section 8.5.2. The retrieval mechanism performs the search over the features describing the cases and Scripts. Users are thus required to input the description of the case/Script they are willing to retrieve. In order to simplify the search, the features for cases and Scripts are displayed in pop-down menus. This approach has been chosen for two main reasons: (i) to facilitate the matching with the keywords associated as features of each case and Script, and (ii) to avoid problems with misspellings and the language describing the scaffold components. Further developments of the VECTRA framework includes more sophisticated keyword matching capabilities to recognise the different levels of language as well as correcting misspellings over the descriptions. Another type of user that relies on the retrieval mechanism are experts using the VECTRA-SI to illustrate their training session. Further details on the interface for this type of users are described in the following section.

9.5.3 The expert interface The VECTRA-SI can be useful for the experts as a tool to help illustrate their instructions when they are providing a training course. Experts are professionals that have nothing to learn from the VECTRA system. Rather, they could contribute by adding new cases to the prototype’s repository and thus increasing the VECTRA-SI instructional capabilities.

Page 203


Experts are not expected to use the VECTRA-SI instructional guidelines since they would provide the instructional strategies. However, the retrieval mechanism and skills to move around the simulated scaffold structures have an increasing importance. Experts willing to use the VECTRA-SI are required to be able to drive themselves trough the VR cases and this requires skills with the navigational devices. The Superscape VRT tool allows the use of a range of hardware devices for navigation, such as space mouse, track ball, and joysticks. However, different hardware requires configuration for the navigational devices and low-speed processors and graphics card with low capabilities can make it even more difficult to navigate through the VR simulations. At this stage, the VECTRA-SI allows the 3D navigation with the traditional keyboard and mouse. These devices are configured to provide forms of navigation that simulate both an aeroplane flying in 3D around the scaffolds and a human being that can only move around in 2D and look up and down the structures. Both the 3D and 2D types of navigation have been incorporated in the VECTRASI. The default option is the 3D navigation due to the difficulty in providing facilities to allow the 2D type of interaction to move up and down the scaffolds. However, experts can configure the system to use the 2D navigation and show students the scaffold with a higher degree of simulation of the difficulties involved with actual scaffold inspections. Experts can freely walk-through the VR cases searching for scaffold and structural components independently of the system’s guidelines. This approach is especially important for trainers who can use the system prior to visiting scaffold structures accompanying the trainees. Viewpoints such as presented on Appendix 1 could be difficult or dangerous to access on real structures. Moreover, it could be difficult to find scaffold structures containing all the issues in the VECTRA-SI case repository and dangerous to show trainees examples of unsafe structures. Even structures complying with the regulations require special supervision when trainees are involved.

Page 204


9.6 – Feedback of the experts on the VECTRA-SI Early in this chapter is described the development of the VECTRA-SI from the conceptual stage to its implementation. The previous section shows the capabilities of the VECTRA-SI interface to cater for users with a different domain knowledge and how they could benefit from the system. This section describes the aspects of the VECTRASI that have been taken along with the process of interviews with the experts. The development of the VECTRA-SI has never intended to achieve a stage further than of a prototype with a few cases and a flexible framework capable of holding interdisciplinary applications. Some characteristics that have been considered throughout the processes of interviews with the experts and development of the VECTRA-SI were: •

users’ profile – covering the interests of users with a different domain knowledge background and requiring no special computer skills to run the application;

•

instructional capabilities - holding multimedia instructional activities to cater different learning preferences and evaluating the consequences of these instructional media over the amount of work involved for the development of the application;

•

framework flexibility - keeping an attitude towards analysing domain aspects such as the design process of the VR cases and the requirements for instructional strategies that could have an influence over the development of interdisciplinary applications relying on the VECTRA framework;

•

amount of resources - focusing on the evaluation of the workload for the design of the application and capabilities of the framework rather than on the limitations of current hardware capabilities;

•

industrial requirements - evaluating the usefulness of the instructional tool for industrial applications, where training is a constant requirement;

•

long-term usefulness – analysing the system’s capabilities to be updated on its domain knowledge and the profile of the users allowed to do so. The aspects listed above are qualitative and have been taken from the process of

interviews with the experts while developing the application. The author’s conclusions taken from the development of this work and the role the applications can play over

Page 205


CBT and CBR research are described in Chapter 10. The following sections describe the feedback and impressions taken from the experts during the development stages of the VECTRA-SI prototype.

9.6.1 - The experts and the VECTRA-SI The VECTRA-SI prototype has taken the feedback from the experts throughout its development stages. The feedback of these professionals was taken focusing on their impressions with the CBR working cycle and model of cognition, the instructional strategy approached, and the capabilities of the VR technology for providing the simulations. The views expressed by the experts during the process of interviews are summarised below. •

The CBR working cycle in the VECTRA-SI and the training from past-cases approach was easily understood and appreciated by the experts. They have shown neither problem in understanding the working process of the retrieval mechanism nor in providing the features describing cases and Scripts.

•

The graphical quality of the VR cases was good for several tasks in the inspection of scaffold structures. However, some activities such as evaluating the integrity of the tubes due to corrosion or other forms of damage, the integrity of the wooden boards in the working platforms, and the attachment of the ties to the walls, could not be provided by the VR simulations and still require field experience.

•

The capability of the VR technology for entertaining while instructing was appreciated and the experts did not recognise their own struggle in finding their way around the scaffold. Although they were obviously struggling with the mousenavigation they have not noticed their difficulties in getting to some specific viewpoints of the structures.

•

The instructional guidelines with the narrative sounds while moving the viewpoint round the scaffold structure and simulating the experts’ guidance was appreciated. However, the task of synchronising the sound files playing at the right points during the viewpoint movement is time consuming. Different hardware configuration will give a different frame-rate and it can take the synchronisation off the original pace. Limitation on the time for the development of this research was the main

responsible for the lack of an evaluation of the VECTRA-SI with potential learners. Page 206


Moreover, limitations on references providing methodologies to evaluate the CBR-CBT usefulness for learners has also contributed to the lack of an evaluation of the VECTRA-SI with actual trainees in the domain. This is thus a task left for future developments and the following section compares the task of designing VECTRA training courses and classroom training.

9.6.2 - VECTRA-SI and classroom training The literature review on issues such as instructional strategies, preferential learning styles, learning capabilities, training requirements and CBT design, leads to the fact that the more knowledge is gained about these issues, the more there is left to discover. Moreover, the review on ITS and ICAT technologies shows that it is difficult to develop CBT applications capable of dealing with all the cognitive aspects and personal preferences in developing CBT. A consensus taken from the experts involved in this work is that designers for training courses on scaffold inspection should collect as much information as possible on issues such as the usefulness of scaffold structures, the different types of scaffolds, the nomenclature and jargon of the domain, the different types of scaffold components, and the type of accidents involving scaffold structures. This domain body of knowledge will then be organised in such a way that the tutor believes it contains an instructional sequence to provide and facilitate student learning. Different tutors will certainly approach the instructional activities differently. Nonetheless, the work involved in identifying and sequencing the instructional activities is time consuming and there are no guarantees that experts with greater domain knowledge are more capable of elaborating the training plans. Table 9.6.2 has been adapted from Gagne (1975) and presents differences between the instruction the traditional form of training involving groups of students tutored by a teacher, and the VECTRA-SI. Each row of the table presents instructional actions in accordance with each stage of the learning process. Training designers can thus analyse and plan the instructional activities in comparison to the capabilities of the VECTRA-SI prototype.

Page 207

Chapter 9 - The VECTRA-SI prototype Actions of instruction

Traditional instruction

Motivation

-teacher discovers individual motivation; - teacher helps to increase motivation. - teacher communicates objectives.

- learner supplies own motivation; - system’s interface helps to increase motivation. - learners confirm or select objectives from the system.

- teacher make plans to stimulate learners attention.

- students adopt their own attention skills; - system’s design provide attentional set.

- teacher provides the clues for recall; - teacher checks recall of essential items. - teacher provides the learning guidance; - teacher adapts for different groups;

- system provides the clues for recall; - systems provide evaluation for essential items; - learners evaluate their own learning. - system provides the learning guidance; - learners dictate their own learning rhythm.

- teacher encourages learners to use his/her own leads for retention.

- system provides the leads for retention modelled from a group of experts.

- teacher adapts transfer tasks to learners’ capabilities.

- system provides learners with leads for transfer tasks.

- teacher plans the tests to assess learners’ performances.

- system provides tests to evaluate learners’ performances; - learners evaluate their performances. - same as above, just providing results of the performance evaluation.

Informing objectives Directing attention Stimulating recall Learning guidance Enhancing retention Promoting transfer Eliciting performance Providing Feedback

- same as above, just evaluating performances adding the possibility of giving personal advice.

CBT approach

Table 9.6.2 - Comparing instructional activities for classroom and VECTRA-SI training.

The VECTRA-SI proposes an instructional strategy that the experts felt comfortable with and it was easy to model since it contains the simulations of actual past experiences. In spite of these natural sequencing for the instructional activities, there are conceptual issues such as the importance of scaffold structures, the accident rate and the implications of scaffold inspection for the whole construction industry that were not learned from past experiences but from domain publications. There is thus a limitation on the subjects that a case-based training approach can cover indicating that the best training alternative would be an integration of the VECTRA-SI and the experts together providing the training course. Another aspect differentiating the VECTRA-SI from other forms of training rely on the interaction with the VR environment where the past experiences are simulated. The following section discusses the instructional capabilities of the VR cases in comparison with the traditional written description of past experiences.

Page 208


9.6.3 - VECTRA-SI and descriptions of past experiences Topics involving motivation, transfer and retention of skills are central to training and yet no strong theoretical ideas exist to integrate research findings (Wingfield 1979; Cardinale 1994; Tucker 1997). There is a consensus that the skills and knowledge possessed affects how new skills are learned. For instance, Wingfield (1979) cited that Thorndike, as long ago as 1901, cited that learning involves analysing the similarities between tasks and the skills they require. How similarities are defined and measured are issues that have been researched for decades. However, most of the evaluations have been performed under laboratorybased experiments that differ in significant ways from the retention of skills in everyday situations (Annett 1977). Literature concerning the difficulties with the task of learning evaluation is abundant and a brief literature review in learning evaluation from CBT is described in 3.5.9. The development process of the VR cases to build the VECTRA-SI repository is certainly more resourceful than the creation of a case repository with worded descriptions in CBR Shells. However, the issues involving VR capabilities for training, entertainment, memory recall, and interaction with the VR cases that are discussed in Chapter 6 make the VR technology useful for CBR training. Current limitations on the VR tools on their capabilities to design the VR worlds is the main aspect making it more difficult to develop VR cases than worded descriptions. There is still a long way until these two forms of case representation could involve a similar amount of work. However, VR tools and libraries of VR objects are becoming increasingly available and thus closing the gap between the amount of work required to design the VR cases.

9.7 - Synthesis of the chapter CBT applications involve (i) providing stimuli for learners to respond to the training and motivations to carry on the tool, (ii) providing a learning methodology so that learners could move through a carefully designed sequence of progressively complex operations; and (iii) evaluating whether the learners are successfully achieving the training objectives.

Page 209


Current technology allows CBT designers to choose among different instructional media. For instance, CBT applications can be interactive, making use of multimedia such as digitised video, animated images, written material, spoken instructions, and VR. Moreover, CBT can be delivered on floppy disks, CD-ROM or through the Internet and applications can be constantly updated and users can take the tools whenever it is convenient. Whilst different forms of providing the training courses are technically available, designers are responsible for the success and failure of CBT. The process of designing the instructional strategy relying on the of VR representation of past experiences and the sequence of the actions taken by the person who had the experience is a contribution of the VECTRA framework that has been taken into the VECTRA-SI prototype. The design of the VR cases with the current VR tools for PC computers is the bottleneck of the development process of the VECTRA-SI application. Although VR technology has potential to provide instructional applications (see Chapter 6), the building of the VR worlds is a barrier that must be considered for developers willing to use VR instruction. People willing to develop applications using the VECTRA framework and its instructional strategy could follow the description of the implementation of the VECTRA-SI prototype provided in this chapter. A drawback is that to take the objectoriented hierarchies and the VR objects develop in this research requires the use of the Superscape software. The conclusions and future developments to give sequence to the research work developed in this thesis are described in the next chapter.

Page 210

Chapter 10 – Conclusions

10.1 – Overview

Chapters 1 to 6 cover the theory behind the integration between CBR and VR that is at the foundations of the VR case-based instruction proposed in this thesis. Chapter 7 describes the conceptual stages of the VECTRA framework and provides a methodology for the development of VECTRA applications. Chapter 8 describes the VECTRA framework and its capability to provide interdisciplinary instructional applications. Chapter 9 describes the VECTRA-SI prototype, which is an application for training in the inspection of scaffold structures. This chapter closes the thesis and evaluates the work carried out in this research in terms of meeting the hypothesis, aims and objectives planned in Chapter 1 and also presents the limitations and recommendations for future work taken from the development of the VECTRA prototype.

Page 212


10.2 – Review A novel conceptual framework is proposed in this thesis to aid the development of intelligent instructional applications. This framework uses CBR as a technique for the development of AI applications where the domain knowledge is held in a repository of past experiences that can be retrieved to help users learn. The CBR components such as the case-base, the retrieval mechanism, the indexes for case retrieval and the user interface have been built using an object-oriented language for the design of VR worlds. This framework can thus be seen as a CBR in VR where the CBR working cycle and model of cognition have been built with the same programming language used for the design of VR worlds. This framework has been denominated VECTRA and the acronym stands for Virtual Environment for Case-based TRAining. The VECTRA framework can be seen as a Shell for the development of intelligent instructional applications. The word intelligent in this thesis relates to the emulation of a human process of cognition where cases stored in memory provide examples that people can recall and base their reasoning on. This process of human cognition is described in the dynamic memory theory (see Section 2.9), where learning is seen as a process of modification of the mental structures holding the memories of past experiences. The VECTRA framework includes an instructional strategy providing users with a predefined sequence of cases (MOP) and instructional activities (Scripts). Each MOP contains a series of Scripts and the retrieval sequence of the Scripts follows the natural sequence of the experts’ actions when performing their job activities inspecting scaffold structures. Each instructional activity includes a sequence of actions that takes into consideration the different stages of human learning that are also part of the instructional strategy of VECTRA applications. Experiencing and simulating are key aspects of VECTRA applications. VR supports learning-by-doing and the simulation of real experiences it allows provides an interactive environment that helps the understanding of the lessons displayed. VR cases can also contain the voice and viewpoint movements of the expert around the simulated scaffold structures. This is useful to emulate the guidance of an expert looking over the shoulder of a novice trying to perform domain activities.

Page 213


10.3 – Conclusions The hypothesis that drove the development of this thesis was that VR technology could play an important role for AI instructional applications regarding both the CBR model of cognition and interface for case representation. The findings relevant to particular aspects of this thesis that were identified by the author are presented below.

10.3.1 – The CBR – VR integration One of the objectives of this thesis concerned the analysis of the capabilities of the VR technology supporting the model of human cognition behind CBR as a technology for the development of AI applications. The VECTRA framework proves the capabilities of current VR technology by the fact that the whole CBR working cycle was solely developed taking the same object-oriented language used for the design of the VR cases. The VR package used in this research (Superscape VRT) has supplied the facilities for the implementation of the conceptual CBR working cycle within the VR environment. Moreover, this tool provides a 3D interface that allows a real time walkthrough the VR cases and an object-oriented programming language where the capabilities of commercial CBR shells can be implemented. The object-oriented language in the VR tool provides a hierarchical structure capable of holding the concepts of MOP and Scripts from the dynamic memory theory (see Section 2.9) that is at the origins of CBR. The object-oriented language used by the VR tool provides access to the contents of the visualisation. This access allows re-using parts of the cases from the repository and/or libraries (or warehouses) of VR objects and thus facilitates the design of new VR cases. Moreover, individual VR objects hold properties such as dimensions, position in the VR world, shape, colour and motion that can also be used as features to help finding them in these libraries of VR objects.

10.3.2 – VR case representation The user interface is always a concern in the development of computer systems and an innovative aspect of this thesis is the use of VR for case representation in CBR. VR cases can help to simulate the physical objects of the location where the experience Page 214


took place, the actions the experts took to perform their job and the guidance of experts watching over the learner’s shoulder. VR is an alternative to current linked multimedia case representation because of two main aspects: (i) the 3D interactive interface it provides for the representation of case contents; and (ii) the access it gives to the contents of the VR cases, which was performed in this thesis by using the object-oriented language present in the Superscape VR tool. The VR interface provides a visualisation that is not static like a picture or a film and allows for a real-time 3D walk-through the VR cases where different viewpoints, along with sounds and animations, can be experienced. VR also allows for a dynamic change of the properties of the components of the VR cases, thus helping to simulate aspects of the past experience. The task of designing the VR cases can be difficult and time-consuming and can play a major role over the process of choosing the VR tool that best fits the domain requirements. Moreover, the implementation of past inspections of scaffolds showed that the experts’ description of their experiences was not sufficient for the design of the VR cases. Even pictures and films taken on site could only help to form educated guesses for the dimensioning of the VR objects. On-site visits were thus required for the representation of the physical attributes of the domain. The feedback from the experts showed that a top quality simulation of the physical objects of the domain may not be required to provide the instruction of some concepts. It was made clear that VR cases must embody a level of detail in correspondence with the instructional requirements of the domain.

10.3.3 – VECTRA instruction The ability to build 3D interactive simulations where users can access a coherent, concrete, organised and meaningful past experience makes VR a powerful instructional technology. The capabilities VR cases have to help decision-making, maximise information feedback, engage users without the onus of surveillance and provide a lively past experience make VECTRA applications a promising instructional tool.

Page 215


Experiencing is central to using VR case representation and provides a learningby-doing environment where users can interact with the components of the VR cases and learn from the actions taken in the simulated environment. VR cases provide a basis for comparison (analogical reasoning) that includes as many aspects of the real situation as current computer technologies can provide. However, some aspects of reality are still difficult to represent in VR. For instance, aspects from the domain of scaffold inspection such as the degree of deterioration of the wooden boards and the corrosion of the metallic components of the structures could not be properly represented in the VECTRA cases, even though they were two important activities in the domain. VR case representation can support the implementation of instructional strategies such as the one suggested by Gagne (1985) and discussed in Section 7.5 or any of those CBR instructional strategies presented in Section 5.5. However, the VR cases in the repository contain experiences that the experts believed capable of providing the instructions but that do not address learners’ instructional preferences.

10.3.4 – The VECTRA prototype Redmond (1992) stated that CBR applications for training should contain the same kind of situations users encounter on the job and offer a presentation that will be properly kept in the learners’ memory. The same author indicates that one of the challenges in building such CBR systems lies in the ability to provide features capable of allowing users to access the knowledge contained in the cases. The VECTRA uses VR simulations of on-job experiences, thus satisfying Redmond’s (1992) first requirement. For the cases to be kept in the learners’ memory, the VECTRA provides an entertaining VR simulation that complies with Gagne’s (1992) theory of learning conditions where the premise is that instructional goals should be presented in a sequence of instructional actions. The process of featuring cases and Scripts followed the functional approach (see Section 4.4.2) and, during the case gathering process, the experts did not show difficulties in providing those features. Pop-down menus where users can select the features describing the case and the instructional activity they want to retrieve can

Page 216


help to avoid problems related to language and typing misspellings of the users’ inputted descriptions. The retrieval mechanism in the VECTRA framework provides facilities for the use of weighted features. However, in the VECTRA-SI prototype, where the instructional aim focused on making the retrieval easy for the users, weighted features were not needed. The prototype was implemented in Superscape and, although it can run under a shareware Visualiser, it requires the original package for the design of new cases. The hardware requirements to run the applications are high and the smallest configuration consists of at least an IBM-PC machine with a high performance Pentium processor, Microsoft Windows 95 or NT and at least 32 megabytes of memory. Powerful 3D graphics cards with accelerators and 64-MB memory or more can be an advantage. Users of the VECTRA-SI can experience the visit of a scaffold structure and a 3D walk through its components in the safe environment of a computer simulation. The analysis of the experts’ feedback on the VR cases indicates that current VR technology lacks the capabilities to cope with some of the aspects involved in scaffold inspection (see Section 9.6.3). However, the professionals were satisfied with the graphical quality of the simulations and enthusiastic about the ability to navigate through the simulated scaffold structures.

10.3.5 – The VECTRA framework The VECTRA framework provides CBR applications from the very moment their development start, i.e. when the knowledge acquisition process is performed by the gathering of past experiences. However, the interfaces providing the input of casedescriptions, the retrieval mechanism and the search results are the only elements there to remind users that they are actually dealing with a CBR. VR cases can be designed to provide a body of knowledge to support learning for almost any task in any domain. Moreover, there are many aspects of reality involved in on-job experiences that can be properly simulated in VR. However, a previous analysis of the domain aspects that actually need to be taken into consideration can help to identify the amount of work required for the design of the VR cases. Domains requiring

Page 217


the design of complex VR objects and the simultaneous display of a great number of objects demand more resources to build and run the VR simulations. Users with experience in the domain can discard the expert guidance in the VR cases, using the CBR core in VECTRA applications and retrieving only the case and/or Script required to reinforce a particular aspect of their knowledge. Trainers can also use VECTRA applications to help illustrate their tutorials with the VR simulation of past on-job occurrences. The literature review indicates that the cost of the training course is still the factor that most companies rely on when deciding between alternative forms of training (see Section 2.5). The experience gained with this thesis indicates that the VECTRA framework cannot be seen as a low cost option. Nonetheless, possibilities such as to dilute the total cost between users over the years and the comparatively low cost involved in adding new cases to the existing repository to update the system capabilities can help to price down VECTRA applications.

10.4 – Recommendations The author has identified three main areas of research that would facilitate the development of further VECTRA applications. The recommendations for future work in each of these areas are described in the following sub-sections.

10.4.1 - The CBR model of cognition The development of algorithms to perform case adaptation using the objectoriented hierarchies used to represent the VR cases (machine learning algorithms) could provide VECTRA applications capable of improving themselves with the automatic addition of new cases in the repository. Algorithms capable of recognising the user’s descriptions of the MOP and Scripts in natural language could also be incorporated in future VECTRA applications. Another recommendation for future developments on the CBR model of cognition is to work on applications to run on the WWW. VR cases could be designed and kept on different servers and shared by users and developers from different locations. A framework capable of holding VECTRA applications on the WWW is thus an issue for further work.

Page 218


10.4.2 – Design of VR cases The possibility to create libraries of VR objects that could be maintained by governmental and private institutions and shared between VECTRA developers could also facilitate the development process of VR cases. These libraries could also include DLLs allowing the synchronisation of the sounds played with the viewpoint movements and allowing to attach animations to VR objects. Another recommendation is to elaborate criteria to evaluate the capabilities of commercial VR tools taking into consideration the domain requirements for VR case design prior to choosing the VR world builder. These criteria would also help to evaluate the feasibility to develop VECTRA applications.

10.4.3 – VECTRA instructional capabilities Currently, if learners do not understand the instructions, all the system can do is to repeat using exactly the same instructional activities. Unfortunately, the capabilities of current CBT applications and the VECTRA framework do not compare to those of experienced tutors in terms of flexibility of the instructional strategies. Including facilities to cope with this situation, similarly to the Pedagogical and Student Module in ITS, is a suggestion for future developments that is also commonly found in literature related to ICAT and CBT applications. The devices used to navigate and interact with the VR simulations present distinct capabilities and appeal differently to users according to their personal preferences. Moreover, in this thesis, no provision was made to use immersive or augmented VR interfaces. It is suggested that future developments should include these extra capabilities of the VR interaction. For instance, augmented reality has been pointed out as holding a great potential for training applications (see Section 6.3.3). A methodology evaluating domain requirements and the capabilities of these devices is also a recommendation for future work that could help to evaluate the possible tradeoffs between learning and simply having fun with the VR cases.

10.5 – Future research The author has identified areas of research that would carry forward the research work in this thesis. From the beginning, limitations related to the capabilities of the prototype and its results were expected. For instance, the evaluation of the users’ Page 219


learning and their enthusiasm to take the tool was part of the initial project. Nevertheless, this task was left for future developments, mainly due to the limited number of references providing access to similar work in the subject and to the time allocated to this research. A recommendation for future research is to carry out an evaluation with learners on the instructional capabilities of VECTRA applications. The object-oriented hierarchies that hold the contents of the VR technology had a major influence over the development of the VECTRA framework and opened a range of future research areas related to the CBR model of cognition. For instance, VR objects have properties that are inherently part of the visualisation, such as dimensions, RGB colour, position and shape. A similarity assessment between different VR objects could be performed over these properties rather than over textual labels describing them. This raises a potential subject for research in AI such as the comparison of similarities between VR objects based on their shapes and other visible properties. One may say that training courses should first consider the domain characteristics rather than propose VR case-based instruction as a potential asset for training. The author of this thesis agrees, however, the literature review did not provide an instructional methodology capable of insuring interdisciplinary training effectiveness. A methodology allowing to identify the actual capabilities of case-based training is therefore an issue for future CBR research. One of the lessons taken from the implementation of the VECTRA was that the higher the requirements for visual details, complex shapes and instructional activities in the VR cases, the more difficult the VR case design task. The development of modelling techniques capable of making explicit domain aspects such as objects’ behaviour, dimensions, positions and links with other objects prior to the implementation

of

the

VR

cases

is

also

an

issue

for

future

research.

Page 220

Appendix 1 – The VECTRA-SI interface

Page 221


The VECTRA-SI prototype allows two different types of interface: •

for novices where the system provides a predefined sequence of cases and instructional activities; and

•

for domain experts or intermediates where the system allows users to take the retrieval mechanism and search for the case and/or Script they want to see. This option is available from a dialogue box in the viewing area as soon as the

system is loaded in the VRT Visualiser as shown in Figure A1.1. Further changes on the option for the interface from the menu at the bottom by selecting the Config option.

Fig. A1.1 – First screen of the VECTRA-SI prototype

The option for the novices’ interface leads to an interaction where the users are basically required to click on the dialogue boxes continuing or repeating the instructions. Rather than describing this type of interface and its menus, this Appendix shows the experts’ interface for such reasons as: •

it shows the interaction with the retrieval mechanism and the features describing cases and Scripts;

•

it shows the instructional activities that could also be taken from the novices’ interface; and

Page 222


•

it shows viewpoints that are also part of the novices’ interface. The next step in the interaction with the VECTRA-SI prototype bring a dialogue

box where users can choose the type of retrieval they want to perform, as shown in Figure A1.2.

Fig. A1.2 – Options for case/Script retrieval

The option for case retrieval brings the pop-down menus where the users can choose the features describing the cases they want to retrieve, as shown in Figure A1.3.

Fig. A1.3 – Choosing case features

Page 223


Once the case features have been chosen, all the cases in the repository will be listed accompanied by they corresponding matching with the inputted descriptions as shown in Figure A1.4. Obviously this choice is only possible due the small number of cases in the repository. The matching level and the retrieval algorithm are described in Section 7.7.3.

Fig. A1.4 – Retrieval for the case that best match the inputted features

Users can then choose the case for retrieval, which contains a VR scaffold structure as shown in Figure A.1.5.

Fig. A1.5 – VR case showing a scaffold structure

Page 224


Once the case has been retrieved, users have options such as freely navigate inspecting the scaffold structure, click on Load Case from the menu bar at the bottom to retrieve a Script, or click on the numbers at the menu bar where each one moves the viewpoint around the structure performing the inspection of a certain scaffold component. The option Load Task at the menu bar also allows users to retrieve another case. If this option is chosen, the Index file shown in Figure A1.2. The option Config at the menu bar allows users to define whether they want sounds simulating the guidance of an expert. The numbers at the menu bar represent a recommended sequence of activities performing the scaffold inspection. When one of those numbers at the menu bar is chosen, the system moves the viewpoint showing close-ups of scaffold components as shown in Figure A.1.6. For instance, one of the scripts performs inspection on the overhang at the end of the boards. The system’s guidelines move the viewpoint around the structure, replicating the views that an expert should take on a real site. Once the Script is finished, the number changes its colour, thus indicating the Script has been played. Reading the menus or listening to the expert’s advice, users can take the theoretical information about this task.

Fig. A1.6 – Overhand of scaffolding boards

Page 225


Most of the objects of the scaffold structure hold some additional information regarding dimensions, material nomenclature, etc. This information is accessed by clicking on the right mouse button on top of the virtual object. Thus, each case works as a repository of information concerning the domain of inspection of Health & Safety regulations on scaffold structures. Further details on the type of interfaces the VECTRA-SI prototype allows are discussed in Section 8.5 and a few other screen shots taken from the VR cases in the system’s repository are presented in the following figures.

Fig. A1.7 – VR case of a scaffold structure


Page 226


Fig. A1.9 – View from the roof top of a building


Page 227

Appendix 2 – Capabilities of VR world builders

“It may be that all human beings have the same perception of space at the biological level of perception. But certainly every society uses its space differently, both technologically and artistically” (Bolter 1986, p80)

Page 228

Appendix 2 – Capabilities of VR world builders

In addition to the usual issues of case representation in CBR, such as feature selection and indexing cases for retrieval, this thesis also involves the design of the VR cases in the VR world builders. The creation of the VR cases is a process of 3D design and, as such, there is no a common-sense operation, or universally accepted methodology to follow (Brooks, M.R. 1994). The development of this thesis has shown that knowledge of VR capabilities and their influences over the human process of perception and cognition can help decide whether VR is appropriate for case representation. What can be built in VR is not the only issue to consider for designing VR cases. For instance, some factors such as the users’ interaction with the virtual world, and the way it will be displayed should also be carefully evaluated (Davis 1996). In order to help people interested in designing the VR cases, Table A2.1 lists some VR packages regarding their capabilities to represent aspects of reality. The table thus draws a comparison between the capabilities of VR world builders, so that developers can match their needs and decide for the VR tool that best suits the designers’ requirements for domain representation. The VR packages compared in this Table A.1∗ are: §

Superscape VRT version 5 (http://www.superscape.com)

§

Sense 8 WTK version 6; (http://www.sense8.com); and

§

Integrated Data Systems Inc. IDS V*Realm Builder (http://www.ids-net.com) Table 1 considers only built-in functions of the VR packages, avoiding the need

for skilful programmers to include those aspects of reality in the applications. Currently, most VR packages contain a programming language and thus, in a way, most of the aspects present in this table can be achieved. However, if the aspect requires programming, it has not been included “as supported” in this table. By the time you read this thesis, new aspects of reality may have been included as capabilities of new VR world builders. Other comparisons involving technical issues in

∗

VR

packages,

can

also

be

found

at

the

WWW,

at

sites

such

as

Table 1 can also be seen at

case-based reasoning in virtual reality: a framework ...

case-based reasoning in virtual reality: a framework ...

Suggest Documents

A framework of combining casebased reasoning ... - Wiley Online Library

Virtual Reality Case-Based Reasoning: A Framework for Computer ...

VirtualExplorer: A Plugin-Based Virtual Reality Framework

A Framework to Study Trust in Virtual Reality Davide ...

A Software Framework for Controlling Virtual Reality Avatars via a ...

A Proposed Framework for a Virtual Reality ... - Semantic Scholar

Trust in Virtual Reality

A DBR framework for designing mobile virtual reality learning ...

VirtualExplorer: A Plugin-Based Virtual Reality Framework - IDAV

a virtual reality archaeological framework for the ... - Semantic Scholar

A Novel Mobile Virtual Reality Framework for ...

PhysioVR: A Novel Mobile Virtual Reality Framework

A framework for evaluating business lead users' virtual reality ...

AnArU, a Virtual Reality Framework for Physical Human Interactions

Virtual Reality and Augmented Reality as a

Virtual Reality Framework Development in Malaysian ... - SSRN papers

Augmented Reality & Virtual Reality in the Library

Virtual Reality and Augmented Reality in Plastic

Virtual Reality

VIRTUAL REALITY, AUGMENTED REALITY AND ROBOTICS IN ...

The virtual classroom: a virtual reality enviroment

VIRTUAL REALITY

The virtual bat: echolocation in virtual reality

Reaching in reality and virtual reality: a comparison of movement ...