Integrating Technical and Training Documentation

Workshop ITS 2002 Conference http://www-eurisco.onecert.fr/events/callworkshop.htm Organising committee: Yvonne Barnard, Monique Grandbastien, Robert de Hoog & Cyrille Desmoulins

Yvonne Barnard EURISCO International (European Institute of Cognitive Sciences and Engineering) 4, avenue Edouard Belin, 31400 Toulouse, France Tel. +33 (0) 5 62 17 38 27, Fax +33 (0) 5 62 17 38 39 Email: [email protected] Monique Grandbastien, INA P-G and LORIA Université Henri Poincaré Nancy 1, France Email:[email protected] Robert de Hoog University of Amsterdam, Dept. of Social Science Informatics, the Netherlands Email: [email protected] Cyrille Desmoulins CLIPS IMAG & Université Joseph Fourier Grenoble 1, France Email: [email protected]

Keywords Technical training, authoring systems, human factors, instructional design, learning environments, ontologies, meta-data

Preface In many complex operational domains, such as the operation of transport systems, military systems, aeronautics and maintenance work, the actual work and the training to perform the tasks are separate worlds. Technical manuals, operational manuals and training materials are not developed in a coordinated way and they may differ considerably, although the topics treated are the same. Some of the consequences are inefficient ways of producing material, leading to high development costs. Inconsistencies between different documents can lead to potential safety-related problems. Lack of coordination can result in ineffective ways of training and lack of transfer of training, because procedures at work differ from the ones trained. Re-using technical material for educational purposes, and vice versa, can be a solution, but it should be done in such a way that the specific needs of both processes are well served. The purpose of the workshop ”Integrating technical and training documentation” is to demonstrate and to discuss different approaches with people from different backgrounds, such as computer science, artificial intelligence, human factors and educational sciences, in order to develop new ideas to progress on this line of research. Reusing material for different purposes is the key-theme, addressing both practical questions of methods, organisation, tools and techniques as well as more fundamental questions about the differences between work and training and new approaches to bring these processes together. Several multi-disciplinary approaches are needed to enable the development of innovative methods and tools: •

Advanced computer science techniques for handling (store, analyse, index, search, retrieve) of technical material (text, pictures, diagrams, video, sound).

•

Use of knowledge engineering approaches, such as ontology development.

•

Indexing mechanisms for marking-up material, both for operational and training uses.

•

Structured pedagogical approaches for developing training and authoring training material.

•

The use of advanced multimedia techniques for producing training material.

All papers included in this volume report on work in which a combination of these approaches were used. Developing training material consists of several phases. The papers in this volume all contribute to one or more of theses phases. In order to be able to re-use material, it is necessary to have it in the format of small objects which are indexed in a useful way. Using meta-data and ontologies are the main techniques which are described in several papers. The use of standards plays an important role here. The developer of training material needs to search for suitable technical documentation. Several papers describe search processes, and related indexing mechanisms. Technical material can be re-used in different ways. Fragments from technical manuals can form the raw material for creating training material, but they can also be transformed, such as into spoken dialogue or other multi-media forms. In order to be able to re-use technical material for educational purposes, the developer of training material will need a clear didactic frame (a scenario or a template) in which the material can be inserted in order to create lessons or training material. Innovative methods and tools to support this process are described in several papers. Finally the training material has to be constructed in its definitive form, and provide the trainees with high-quality learning experiences. Examples of the final creation of training material, (re-)using technical material are amply given by the authors of the papers. The research presented in this volume is generally aimed at developing technology for enabling the integration of technical and training material. By using standards, tools and methods, developers of training material are supported in their task. But it is not only a matter of innovative software and emerging standards. The research is also driven by underlying ideas on training and learning. The ultimate aim of integrating technical and training documentation is to produce innovative training material, which brings work and learning closer together.

-2-

Workshop schedule 09.00 – 09.30

Introduction

09.30 – 10.30

Re-using technical material and indexing, paper presentations and discussion

10.30 – 11.00

Hands-on experience with indexing and document analysis tools in small groups

11.00 – 11.30

Coffee break

11.30 – 12.00

Continue with hands-on experience

12.00 – 12.30

Discussion on the experiences with working the tools

13.00 – 15.00

Lunch

15.00 – 16.00

Developing training documents from technical material, paper presentation and discussion

16.00 – 17.00

Hands-on experience with the training scenarios and training material development tools in small groups

17.00 – 17.30

Coffee break

17.30 – 18.00

Discussion on the experiences with working with the tools

18.00 – 18.30

Discussion on future research in the area of integrating technical and training material

-3-

Contents

Producing learning objects with XML-XSLT technology

5

Jean-Pierre David, Cécile Guilloux &Alexandre Flament

Re-using technical manuals for instruction: document analysis in the IMAT project

15

Robert de Hoog, Bob Wielinga, Suzanne Kabel, Anjo Anjewierden, Frans Verster, Yvonne Barnard, Paolo DeLuca, Cyrille Desmoulins & Johan Riemersma

Re-using technical manuals for instruction: creating instructional material with the tools of the IMAT project

27

Robert de Hoog, Suzanne Kabel, Yvonne Barnard, Guy Boy, Paolo DeLuca, Cyrille Desmoulins, Johan Riemersma & Daniëlle Verstegen

A scenario tool for coupling human and computerised expertise in the development of training material for complex equipment maintenance

39

Yvonne Barnard, Cyrille Desmoulins & Monique Grandbastien

Generating Training and Assistive Dialogues for Astronauts from International Space StationTechnical Documentation

49

Gregory Aist & Beth Ann Hockey

Maintenance training design: Less is more

55

Lesley Jacobs, Anja van der Hulst & Stefan van der Stigchel

Personalized eLearning and eCoaching in WINDS

63

Marcus Specht, Milos Kravcik, Roland Klemke, Leonid Pesin & Rüdiger Hüttenhain

On the Social Rational Mirror’s architecture: Semantics and pragmatics of educational interactions Daniele Maraschi, Germana M. Da Nobrega & Stefano A. Cerri

-4-

77

Producing learning objects with XML-XSLT technology Jean-Pierre David*, Cécile Guilloux** & Alexandre Flament** *ARCADE team, CLIPS-IMAG GRENOBLE [email protected] **Centre de Développement ARIADNE de l’UJF à Grenoble

Abstract We present a method for producing learning objects, using the technologies of information and communication, which results from modelling learning activities. The study of two case of learning objects, their expression in the open standard XML and the use of the same tool for authoring them, prove the generic aspect of the chosen approach.

Keywords Modelling of educational activity, learning object, structured documents, learning objects generator, XML, XSL

Introduction We shall speak about learning objects as a hypermedia document which allow the user to make an educational activity. We are interested in the definition of class of learning objects to justify the creation of generators of these objects. The research thus concerns several points: how to obtain a model of learning object?; How to express such a model? and What are the tools which can produce activities accorded to this model? We have the conviction that for creating resources, teachers should be helped by authoring tools which contain an educational methodology. For creating such a software allowing teachers to produce learning objects, our team, having chosen proprietary formats as our first productions, has yet determinedly opted for the interoperability of the data and thus for their expression in open standards. Therefore, the emergence of the XML language in 1998, and the first recommendation for XSLT in 1999 gave possibilities for expressing tree structures and transformation of these structures. The Document Type Definition which defines a class of XML documents allows to express generic models. These standardised languages offer the interoperability of all the produced data expressed through them. We detail first of all the experience acquired through the production of autoevaluation exercises with the generator GenEval. We present then the GenDiapo project which is an application of the method built from GenEval. For this class of documents, an editor has been developed and adapted for authoring too the autoevaluation exercises.

Modelling and producing self assessed exercise Story of the GenEval project In 1994, UJF Teachers of Mathematic and physics created exercises prototype in TooBook, based on a self assessment pedagogical approach.

-5-

These prototypes had been validated with an audience of teachers and students, it was observed that it lacked methodology and tool to facilitate the production of this kind of exercises by teachers themselves. In an other part, the ARCADE team, since the production of its interactive environment for the learning of the algorithmic had be concerned to design authoring tools addressing teachers who were not computer specialists. For these reasons, our team has been involved to design authoring tools within the framework of the European project of Research &Development Ariadne. The « exercise » Learning Object What is a self assessed exercise? The educational activity which has to be modelised is a session of steered works in autonomy for the resolution of a problem. The student is in situation to solve an exercise, with accesses to relevant information to understand studied concepts, with orientations and progressive methodological helps. He can use or not the available theoretical or practical helps, he assesses his work, comparing the complete answer with his own. In this class of learning objects “self assessed steered works ", the content on which the activity is supported must be distinguished of the activity itself, which will be the same on all the contents. The model is described on one hand by the elements which compose it and on the other by the breaking down of the tasks which are offered to the learner. Prototype of the student interface An interface, allowing the student activity according to the data of an exercise, was directly made in html, Java and javascript. One Java applet remembers the time spent on every stage of the activity. Javascript code delays the access to the information, open a window on a definition of the glossary …

Figure 1: screenshot of an exercise. All the information to understand the response is organized to be delivered on demand. A glossary is accessible by hyperlinks on key terms of the domain of the exercise. At the end of every question, the student is invited to assess himself according to the criteria of competence defined by the teacher and at any time, he can consult the outcome of his work in terms of time spent on every part of the exercise, the evaluation which he gave himself on all the criteria and the total results by question. Components of an exercise The exercise is composed by a set of elements, themselves made up by a set of sub elements. A tree of elements represents the whole exercise. The contents of the element outcome builds itself dynamically during the passage of the exercise by a student. -6-

Concepts?

Statement

Glossary

“did you know it?”

outcome

Question1 Method 1 Short answer

Theoretical Practical help help illustration illustration Animation Animation

illustration Animation

Detailed answer illustration Animation

AUTOEVALUATION Method 2 Question2

Figure 2: Components of an exercise.

Task offered to the student The scenario of activity expresses itself by a set of tasks which are offered to him at a given moment. The following scheme describes the tasks which are proposed to the student in the exercise. Reading the statement of the exercise Reading the glossary, the bibliography

Solving a question

Reading the "Did you know it?" rubrik

Choosing a solving method m1or m2 ... Making a quizz on the concept of the exercise

Asking for theoretical help in m1

Asking for theoretical help in m2

Asking for practical help in m1

Asking for practical help in m2

Asking for the answer according to m1

Asking for the answer according to m2

Self assessment

Figure 3 : Task offered to a student which runs an autoevaluation exercise

The scenario is confidentially connected to the structure of the components of the exercise, but reflects additional pedagogical choices. In the set of task presented in the scheme above, the order in which these tasks will be executed by the student is not compulsory, it will be simply suggested by the interface. But the teacher can decide that the kind of help, or the complete answer, are not available during that moment, to force the student to work by himself. The description of this kind of scenario can be expressed in various formalisms of tasks (DIANE, JSD, MAD). One can notice behind these formalisms the existence of a structure of tree which will afterward allow to implement them with XML.

-7-

Using XML to express the model The components of the exercise were expressed in the XML language, specific tags were defined and the treelike brake down follows the oriented object design of the component of the exercise.

Etude statistique de lancers de dé. Jean-Pierre David informatique conception d'applications interactives approche par objets 90 réutilisation REUSE Capacité à réutiliser les objets.

/* Styles exercice geneval2.css */ /* Titre de la page */ .head { font: 18pt sans-serif; font-weight:bold; color: #000060} /* Concepts */ .formulaire {font-family: sans-serif; color: #000060} .valid {text-align: center ; font-size: 14pt; } .return {border: 0pt;text-align: center; color: red; margin-top: 15pt} /* enoncé */ /* texte de l'enoncé */ .statement {font: 16pt serif; margin: 10pt} /* texte des remarques eventuelles */ .nota {font: 14pt serif; margin-left: 10pt} /* illustration */ /* corps de l'illustration */ .illst {color: #ffffff; background-color: #200000; padding: 10px; text-align: center} /* Texte de l'illustration */ .textillst {font: 14pt serif; text-align: center}

Figure 4: XML Description of an exercise and Style sheet for the presentation. The presentation of the components on a web page is described by cascaded style sheet. So, the contents and the structure of an exercise are expressed in a XML file whereas the presentations of the corresponding web pages are described in CSS files, modifiable independently. The description of the class of the exercises GenEval is expressed by a DTD.



Figure 5: DTD extract of an exercise. Creating an exercise in XML Since the emergence of the XML standard, numerous tools appeared for editing XML documents. These editors verify that documents are well formed and valid compared to the DTD which defines their class. For an author, to create an autoévaluation exercise there is only a need to edit its structure and its contents by means of an editor XML

-8-

Figure 6: Creating an exercise in an XML editor Using XSLT for getting web pages Expressing transformation with XSLT To unify the mode of description of a document and its transformation in HTML, we chose to express this one in XSLT language. All these languages are standards recommended by the W3C. Furthermore, the environment of modification of the model (meta environment) and the environment of creation of an exercise can be the same, this demonstrates the power of the current standardization. Any editing tool, on all the software or material platforms, can be used to create and modify geneval.dtd, exo.xml and geneval.xsl files. Furthermore, the same editing tool allows to edit the DTD, the files XML and the files XSLT.

Figure 7 : Part of a XSLT script to transform XML file into HTML file A XSLT processor transforms a XML file, according to the XSLT scripts, for producing the final html code, which will be used through the browsers for the end users. A lot of this kind of XSLT processors are available by freeware and open source.

-9-

Transforming Process to produce an exercise

Enseignant

Editeur XML geneval.css Exo.xml

Publicateur ti

Moteur de transformation

Geneval.dtd

geneval.xsl

Exercice d’autoévaluation 8 navigateur

Etudiant

Figure 8 : Transforming Process to produce an exercise This version of genEval is thus made up by a DTD, by style sheets CSS and of scripts XSLT. It requires not specific tools except a XML editor and a XSL processor. The production process of an exercise is described by the following plan

Modelling and producing learning diaporama The GenDiapo Project A team of computer specialists who want to produce multimedia documents for their students noticed that some part of their documents were duplicated in lecture notes and slides which are difficultly maintainable and must be continuously synchronized and adapted to different audiences. The idea then came up to structure in the same document all the information of the course they deliver, indicating which audience addresses certain parts of the document. In addition, they have to differentiate which part belong to the summary for the slides and which part belong to the development for the lecture notes. So, a model of structured doc ument was defined with a scenario of the activity offered to the student on the slide show by adding contents, index, glossary … This learning object was named hyperdiaporama. The initial text will be marked by levels of abstraction of the information. A lecture note or a set of slides will be produced by the same process than previously, according to the level chosen by the author. The hyperdiaporama Learning object Teaching with a hyperdiaporama The objective to model an hyperdiaporama object is more oriented to the simplification of the teacher’s work than to the originality of the activity of learning it. Indeed, the aim is to define a model of a document able to contain information in a exhaustive way, but which will be delivered in a selective way. So, the same information will be reused or reorganized automatically according to demand, this will introduce more flexibility into the production of educational documents of expositive type.

- 10 -

Prototype of the student interface of a hyperdiaporama

Figure 9: Un écran d’un hyperdiaporama. These web pages allow the teacher to present his course and the learner to navigate sequentially or according to the table, to consult the glossary… Components of a hyperdiaporama A hyperdiaporama results of the transformation of following components, which structure the course:






Elements : they are distributed into two groups: • Hierarchical elements : ♦ Section • Content elements: ♦ Non-terminal elements :
Definition, Demonstration, Remark
Note
Example ♦ terminal elements:
Paragraph, Image,Code, List Category define the audience of each part of the course Detail level: can take the value developed or abstract

The publication for a given audience takes into account the category of elements while the choice of a level of detail for the publication will determine the selection of elements to publish on a paper or on web pages. So, in the same document, the teacher can indicate, for every information, which specific audience he addresses and if he wants to produce a slide show on web pages or a complete course on pdf file. With these functionalities one avoids copying and pasting when producing documents of text or of slide show type, a process we make permanently and which is very difficult to synchronise with the evolutions of the contents. Using open standards for expressing a hyperdiaporama As for GenEval, the structure of a class of documents was described by a DTD, a document produced by the teacher according to the model of this DTD appears in the form of a XML file, the parameters of publication are also defined in a file XML. To produce the final code HTML or PDF, which will be useful in the current navigators or printable, an XSLT processor transforms the document.xml file with the specific scripts written in XSLT. The transformation produces automatically, according to the predefined style of presentation, the hyperdiaporama and the hyperlinks allowing to navigate, as well as the hyperlinks towards the glossary.

- 11 -

GenDoc: a generic authoring tool for producing Learning Objects To facilitate the chain of all the stages of the transformation, from the edition phase to the publication phase, and to avoid the author to work at the level of XML tags, the authoring tool GenDoc was developed. This tool was developed from the Java open source XML editor MERLOT of Channel-Point. Next to it the vision of the tree of XML elements was added a styled view of the whole document.

Figure 10: Screenshot of the authoring tool GenDoc. Adding other Learning object model in GenDoc The modularity of the GenDoc architecture allows its adaptation to a new model of learning object defined by a DTD and their transformation scripts. Practically, a system of plugins allows to adapt the styled view to the DTD and to associate a publication to the model. For the moment, these adaptations must be made by a programmer, because XSL script have to be embedded into java code of the plugins. The next step of our approach should be the design of a meta-tool, for adding more easily new learning object models. So the pedagogical point of view of the teacher on a learning activity, translated into a XML model could be taken into account by our GenDoc tool, without the technical aspects described in this paper, but only by using a friendly user interface.

Bibliographie [1.] Yolaine BOURDA et Marc HELIER « Métadonnées et XML : applications aux objets pédagogiques » TICE2000, Troyes, octobre 2000. [2.] J.P. David, A. Cogne "Mutualisation de Production de documents hypermédias et mise en œuvre pédagogique" New technologies of Information and Communication in Engineering Education and in Industry, NTCIF'98 ROUEN, November 1998. [3.] J.P. David et A. Dutel "Modélisation et réalisation d’un générateur d’exercices hypermédias" Colloque "Hypermédias et apprentissage" Chatenay-Malabry 1996.

- 12 -

[4.] J.P. David, A. Cogne et A.Dutel "Hypermedia exercises prototyping and modelising" Conférence internationale CALISCE'96 San Sebastian, juillet 1996. [5.] J.P. Pernin,V. Guéraud. "MARS : Modèle-Associations-Représentation-Scénario. Un modèle de conception d'applications pédagogiques interactives ", 7èmes journées de l’ingénierie de l’interaction Homme-Machine, Cepadues éditions,Toulouse oct 1995. [6.] J.M. CAGNAT, V. GUÉRAUD, J.P. PEYRIN. "the Arcade Laboratory : an environment to help teach algorithms." (Bulletin SIGCSE ACM, décembre 1990).

- 13 -

- 14 -

Re-using technical manuals for instruction: document analysis in the IMAT1 project

Robert de Hoog*, Bob Wielinga*, Suzanne Kabel* †, Anjo Anjewierden*, Frans Verster*, Yvonne Barnard ‡ †, Paolo DeLuca §, Cyrille Desmoulinsª, Johan Riemersma† * Intelligent Systems Laboratory University of Amsterdam, The Netherlands ‡ EURISCO, France § Tecnopolis CSATA Novus Ortus, Italy ª LORIA, France † TNO Human Factors, TNO Physics and Electronics Laboratory, The Netherlands Address, e-mail: [email protected]

Abstract This paper describes the result of the IMAT project, in particular the tools that were developed for analysing technical manuals. The goal of the project was to build a toolset that can support the re-use of the content of technical manuals for instruction. In many domains where equipment has to be maintained and repaired, the buyer of the equipment has to create the training material, because there are no publishers for textbooks for those limited domains. As a consequence, the major source of content is the technical manual delivered by the vendor. Unfortunately these manuals are most of the time not immediately suitable for training and instruction, leading to tedious and time consuming reworking of the source material. By providing a set of tools that allow the breaking up of the technical manual into smaller fragments, indexing them, storing them in a multi-media data base, the “up front” work needed for re-use can be automated to a large extent, thus saving time.

Keywords re-use, manuals, instruction, ontologies, document analysis

Introduction The IMAT project (de Hoog, Barnard & Wielinga, 1999) had as its main aim the development of a set of products for supporting the effective and efficient re-use of the content of technical manuals for instructional purposes. In addition, the creation of a structure to facilitate the storing and exchange of experience gained while working with this material was taken up. The approach was to take a technical manual as delivered by the 1

The IMAT project (ESPRIT 29175) was a project partly funded by the European Union. Partners in the project were: Intelligent Systems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and Electronics Laboratory (NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Dutch Airforce (NL), EURISCO (F). Apart from the authors, many other people contributed to the project from the organisations listed above. We would like to thank them all for their efforts.

- 15 -

manufacturer of the equipment, break it up into small fragments (homogeneous content) and store these fragments, after indexing them in a meaningful way, in an object oriented database. This database with fragments can be used to retrieve content when creating training material with an authoring environment chosen by the user. The project delivered a collection of inter-operable tools consisting of: •

A database schema for describing fragments

•

A Document Analysis Tool that analyses, fragments and indexes technical manuals

•

A Database Facilities Tool for managing the database, including versioning of fragments

•

An Authoring Environments Interface Tool for retrieving fragments from the database and transferring selected fragments to an authoring environment; also used for additional indexing and annotation of fragments

•

An Ontology Development Tool for creating ontologies in a structured way

•

A set of ontologies for describing fragments, domain specific as well as generic, including an extensive instructional ontology

•

An Instructional Scenarios Tool supporting the creation of skeletal lessons for maintenance tasks. The skeletal lessons can be exported to the Authoring Environments Interface Tool and used for guiding database queries whose results can be stored in the skeletal lesson

The tools were used at three different user sites, each representing different approaches to creating training material and teaching: maintenance of anti-aircraft weapons, car repair and maintenance of traffic control equipment. For a better understanding of the design and workflow of the IMAT tools Figure 1 below can be used.

General Generalworkflow workflow

Scenario Scenario tool tool 105 Jasmine Jasmine 2 OO OO Database Database

Document Document 3 analysis analysis analysistool analysistool

Source document

6

Domain Domain ontologies ontologies

5 Generic //

Data base facilities 2

instructional ontologies

Other Other source source material material

Authoring Authoring 7 2 environment environment interface interfacetool interface interfacetool

Indexing tool

3

Org m . em. 4 tool

Authoring Authoring environment environment

Covered in this paper

Figure 1: Workflow and structure of the IMAT toolset The grey boxes are the tools that were developed in the IMAT project. The yellow boxes are the ontologies, which are at the core of the system, but these are described only briefly for reasons of space. The green boxes

- 16 -

represent commercial-of-the shelf software used. In fact much more was used than shown, but this would clutter the picture. The basic workflow is that a source document is analysed by the Document analysis tool, which breaks it up in small parts (fragments) and indexes these fragments. The fragments are exported to the Muli-media data base from which the fragments can be retrieved with the Authoring environment interface tool. This tool also provides functionalities for additional indexing of fragments and adding “lessons learned” like notations to fragments with the Indexing tool and the Organisational memory/learning tool. The retrieval can be guided from an instructional viewpoint by using the Instructional scenarios tool. Finally the retrieved fragments can be copy/pasted into the Authoring environment chosen by the author of the instructional material. This paper will be describe the set of tools that support the analysis and fragmenting of the technical manual, or the “up front” work that is needed for re-use (the dashed box in Figure 1). The accompanying paper (de Hoog et al., 2002b) will show how the results of the document analysis process can be used for creating instructional material.

The multi-media database For storing fragments from technical manuals a commercial product was used: the object-oriented Jasmine® system available from Computer Associates. It allows the storage and retrieval of text, images, sound and video. During the IMAT project only the text and images facilities were used. Underlying the database is a database schema that is derived from the ontologies that were developed. Figure 2 shows the simple window with which Jasmine can be switched on and off. For most users this is the only visible presence of the database.

Figure 2: Control window for Jasmine

The Jasmine® system has performed well. The only drawback is that there is not yet a SQL comparable standard in the Object Orientation world hat permits the replacement of Jasmine® with another system. As a consequence the IMAT tools are coupled to Jasmine®. This holds in particular for the Authoring Environments Interface Tool, the Indexing Tool and the Organisational Memory and Learning Tool. For document analysis this link is weaker as that tool produces as output an ODQL file that can be loaded by any data base system supporting ODQL. Managing the database consists of creating contexts for applications and managing fragments, which works with the data base facilities tool. An important part of this is the loading of ontologies that are relevant for the context (or domain, e.g., a particular type of car) from which the fragments in the technical manual are extracted. Figure 3 shows how ontologies are loaded for a context. The “Domain topics” and “Domain keywords” are domain specific, all other ontologies are generic and applicable across domains but related to instruction.

- 17 -

Figure 3: Loading ontologies in the Data Base Facilities Tool

The document analysis tool The goal of the document analysis tool is to break up a technical manual into smaller text and image fragments, providing them with an index for identification and exporting the fragments to the Jasmine database. The main window of the tool is shown in Figure 4. Though the document analysis process can be carried out fully automatic, from user evaluations it became clear that some control over and insight in the behaviour of the tool was desirable. The “Set context” button allows the user to identify a context in which the material will be stored in the database (this should have been created by the tool described Figure 3). The “Set document” button makes it possible to select (part of) a document that is going to be analysed. The tool requires that the source document is available in PDF. By clicking on the “Text analysis” button the selected document is automatically fragmented and indexed with topics and keywords. This process relies heavily on the domain and generic ontologies.

Figure 4: Main window of the Document Analysis Tool

The tool pops up a window with the HTML version of the analysed PDF document (see Figure 5).

- 18 -

Figure 5: HTML version of PDF document Through another option the user can also view which fragments the analysis process has identified (see Figure 6).

Figure 6: View on the results of the text analysis process In Figure 6 the coloured lines indicate the fragments that were identified, while the margins show the structural type of the fragment (e.g., item or paragraph). Clicking on the “Image analysis” button in Figure 5 performs the analysis of the images in the source text. This process converts the images to SVG, which are stored in this format in the database. The transformation to SVG makes it possible to search the content of an image (like

- 19 -

lines, boxes, text etc.), something that is not possible with Bitmap images. Figure 7 shows the results of analysing a wiring schema.

Figure 7: Result of analysing a picture

The different colours in Figure 7 indicate what has been recognized, shown in the list of elements in the upper right hand corner. This includes lines, rectangles etcetera. Text is recognized by using OCR techniques. By means of colours it is indicated how reliable the OCR is, as this is not 100% certain for documents having a low original quality. As no mistakes are allowed in technical diagrams, the user must have a clue about the quality of the OCR. The Document Analysis tool includes a search facility with which the user can have a look at the created fragments. Figure 8 shows an example window of this part of the tool.

- 20 -

Figure 8: Fragments search facility of the Document Analysis Tool In Figure 8 a search is performed on the topic “Automatic data processor (ADP) microcomputer”. The topmost part of the window shows that two fragments matching this topic were found. The middle part of the window displays the content of the first fragment. It is important to understand that this text fragment has been automatically extracted and indexed from the original technical manual and that these two fragments can be located in different chapters of the manual. If the user is satisfied with the results of the document analysis process, the button “Build DB” can be clicked and the fragments (text and images) are automatically loaded in the Jasmine data base (if it is running, see Figure 2).

The ontology development support tool Ontologies play a key role in the process, as they are needed for indexing fragments. Support for ontology development is concerned with domain specific ontologies, that is, creating and modifying ontologies which are unique for a particular area, e.g., cars, traffic control equipment, anti-aircraft guns etc.. Three forms of support were available: •

Automatic creation of an ontology from a structured parts list of the equipment under consideration or from the Table of Contents from the technical manual, followed by manual fine-tuning.

•

Creating the ontology “by hand” using the development tool.

•

Modifying ontologies loaded in the database using the database facilities.

As the first type of support is very dependent on the domain, this has been offered as a tailored service only during the IMAT project for the users. The second and third types are available in the tools delivered by IMAT. - 21 -

Creating an ontology “by hand” can be done by using the development tool that can be activated by clicking on the menu item that appears in the dropdown menu under the heading “Tools” in Figure 4, this results in the window shown in Figure 9.

Figure 9: The window of the ontology development support tool included in the document analysis tool. This tool allows for the creation of two different types of hierarchies, Is-a and Part-of (upper part of window). For each concept in the hierarchy its location (“Super” or “Part of”) and other attributes can be defined (left hand side of lower part of the window). These are automatically converted into CML (= Conceptual Modelling Language, right hand side of lower part of window). These CML defined hierarchies are than available to be used in the database schema and the Authoring Environments Interface Tool. Of course, this support tool can also be used to modify ontologies. The other way to modify ontologies is by using the database facilities outlined in Figure 3. When activating the correct function, the window in Figure 10 is available.

- 22 -

Figure 10: Modifying ontologies using the Dat base Facilities tool In Figure 10 it is shown how user remarks can be added to elements/concepts in the ontologies. Taken together, the combination of both ways of working, constitute the ontology development tool support provided by IMAT. From the experience gained in the project, it is clear that creating a domain ontology from scratch is a major task, which requires expertise and time. The preferred solution is to create the domain ontology to a large extent automatically and let it be checked and tuned by an expert. Within the framework of the IMAT project time and effort was lacking to find a more or less generic way of doing this for a wide range of domains. Nonetheless, when a parts list or a well-structured Table of Content of the manual is available, some automation can be achieved, albeit in a customised way.

The instructional ontology In order to focus the work on instruction it was necessary to development an ontology that enables the analysis and indexing of fragments from an instructional perspective. At the start of the project several initiatives in this area were just emerging. During the project the development of the instructional ontology has been done with an eye on the ADL/SCORM, LOM and Ariadne projects. The IMAT instructional ontology is to a large extent comparable with the vocabularies from those quarters, but allows for a more detailed instructional indexing of fragments. The “instructional ontology” is a set of related ontologies. For reasons of space we deal only briefly with the ontologies. Fragment ontology This ontology represents the syntactical view on a fragment, “what it is” in terms of the source document. For example, a fragment “is a picture” or a fragment “ is in BMP format”.

- 23 -

Description ontology Fragment description type is an attribute of a fragment that reflects the perspective from which a fragment was written, and it contains a description of the content of a fragment. For example, a fragment was written from a structural perspective, describing a system’s physical features, and the content concerns a definition of a component. Instructional roles ontology This ontology describes the possible instructional roles a fragment can play in the instructional material that has to be created. For example can be a “picture” (from the fragment ontology), but in the new instructional context it can play the role of an “example” or it can be part of an “exercise”. This notion of “roles” represents the pragmatic aspects of content. It enables re-use of the original content of the manual for instructional purposes. This ontology consists of several components: •

A learning goal describes the desired outcome of an instructional curriculum in terms of knowledge, skills and attitudes that are necessary to carry out certain functions or tasks (e.g., perceptual knowledge normal/deviant behavior).

•

Knowledge is information acquired and stored, in an organised manner, in the mind. Knowledge type stands for the knowledge that is necessary for a learner to appropriate in order to gain the learning goal.

•

Knowledge is communicated to the learner by means of instructional strategies. Instructional strategies refer to the effective way of teaching or to characteristics of an effective learning environment (e.g., procedural knowledge).

•

Instructional strategies are refined into instructional activities. Instructional activities stand for the activities performed by instructor and learner (e.g., coaching).

•

Instructional activities are refined into instructor actions and learner actions. Instructor actions stand for everything the instructor does within a certain instructional activity, while learner actions stand for everything the learner does within a certain instructional activity (e.g., provide explanation). The nouns in instructor and learner actions, represent the way fragments are going to be used in instructional material. For example, the learner action “study illustration” indicates that the fragment that is going to be put in the training material, will be used as an illustration.

Domain ontologies The domain ontologies differ from the ones described above because they are closely tied to a unique piece of equipment. This holds in particular for anti-aircraft and traffic control equipment. For cars the link with specific equipment is less strong, as there are many different instances (brands, types) of the general class of cars. Again these ontologies are only described briefly. Anti-aircraft guns Two systems were included: the Flycatcher anti-aircraft gun and the Hawk anti-aircraft missile. The ontologies for these domains were developed in the IMAT project, using Tables of Content and parts lists. The first step was carried out by a computer program. The output was refined with the support of domain experts. Cars The ontology for cars is a large ontology with hundreds of concepts. An interesting aspect is that it not only contains “pure” car concepts, but also concepts related to the maintenance task like “repairing tool”. Traffic control equipment The traffic controller built by ETRA is a piece of equipment that controls traffic lights. It consists of a physical system (the equipment) and software with which the behavior of the controller can be programmed. As a consequence, the ontology had to reflect this distinction. System ontology

- 24 -

An ontology of system types and an ontology of system locations have been developed to describe systems in a general, domain independent, fashion (e.g., subunit or container). These ontologies are not directly useful or visible to the author of training material, but they are of use in document analysis. Another argument to develop them was to standardize terms used across domains. When comparing instructional ontologies and domain ontologies, it becomes clear that the distinction between the two is relative. For example, the description ontology also deals with functional and structural aspects, as does the system ontology. Likewise, the domain ontologies sometimes contain concepts with an instructional flavor such as “repairing tool” and “programming”. The domain ontologies have played a very important part in the IMAT project. They were the cornerstones for automatically indexing fragments as performed by the Documents analysis tool, which in turn was the main prerequisite for making retrieval possible. As instructional indexing cannot (yet) be done in an automatic way, domain concepts really hold the key to the operational effectiveness of the IMAT toolset.

Conclusion The tools were used in three contexts: maintenance/repair of cars, traffic control equipment and military equipment (rockets and guns). In all contexts the source material was quite bulky and previously most of the creation of training material was done by literally copying, cutting and pasting from the paperbased source. Substantial parts of this source material was processed by the document analysis tool and stored in the multimedia database. Generally speaking the document analysis tools did a good job. Transforming paper into electronic form proved to be a benefit in itself. Substantial time was saved in handling electronic material instead of cumbersome piles of paper. Nonetheless some technical problems are quite formidable. The almost endless variety in document types makes it very difficult to build analysis tools that can be applied to any document that comes along. In the foreseeable future there were always be a necessity for some tailoring of the analysis tool, depending on the manual that must be fragmented. In addition, automatic indexing still has its limits. Automatic indexing with instructional mark up is still in the future and has to be done “by hand”. Without a well-structured parts lists, or table of content the building of a domain ontology can be a substantial amount of work, even if editing tools are available. This is probably less of a problem for technical domains, but it could be an obstacle when one wants to transfer the approach to other, less well-defined, areas.

References ADL, http://www.adlnet.org ARIADNE, http://ariadne.unil.ch Hoog, R. de, Y. Barnard & B. Wielinga (1999). IMAT: re-using Multi-media Electronic Technical Documentation for Training. In: J.Y. Roger, B. Stanford-Smith & P.T. Kidd (Eds), Business and Work in the Information Society: New Technologies and Applications. IOS Press, p. 415 – 421. Hoog, R. de, S. Kabel, Y. Barnard, G. Boy, P. DeLuca, C. Desmoulins, J. Riemersma & D. Verstegen (2002b). Re-using technical manuals for instruction: creating instructional material with the tools of the IMAT project. This volume. IMS, http://www.imsproject.org/

- 25 -

- 26 -

Re-using technical manuals for instruction: creating instructional material with the tools of the IMAT1 project Robert de Hoog*, Suzanne Kabel* †, Yvonne Barnard ‡ †, Guy Boy‡, Paolo DeLuca §, Cyrille Desmoulinsª, Johan Riemersma †, Danielle Verstegen † * Intelligent Systems Laboratory University of Amsterdam, The Netherlands ‡ EURISCO, France § Tecnopolis CSATA Novus Ortus, Italy ª CLIPS IMAG & Université Joseph Fourier Grenoble 1, France † TNO Human Factors, TNO Physics and Electronics Laboratory, The Netherlands Address, e-mail: [email protected]

Abstract This paper describes the result of the IMAT project, in particular the tools that were developed to support the creation of instructional material re-using content from a technical manual. The goal of the project was to build a toolset that can support the re-use of the content of technical manuals for instruction. In many domains where equipment has to be maintained and repaired, the buyer of the equipment has to create the training material, because there are no publishers for text books for those limited domains. As a consequence, the major source of content is the technical manual delivered by the vendor. Unfortunately these manuals are most of the time not immediately suitable for training and instruction, leading to tedious and time consuming reworking of the source material. After storing the fragmented content of a manual in a multi-media database, the author of the training material has to retrieve it and insert it in an authoring environment. This process is supported by tools enabling retrieval, creating instructional scenarios, and classifying and storing experience with the material during training as well as regular maintenance.

Keywords re-use, manuals, information retrieval, ontologies, instructional scenarios

Introduction The IMAT project (de Hoog, Barnard & Wielinga, 1999) had as its main aim the development of a set of products for supporting the effective and efficient re-use of the content of technical manuals for instructional purposes. In addition, the creation of a structure to facilitate the storing and exchange of experience gained while working with this material was taken up. The approach was to take a technical manual as delivered by the manufacturer of the equipment, break it up into small fragments homogeneous content and store these fragments, 1

The IMAT project (ESPRIT 29175) was a project partly funded by the European Union. Partners in the project were: Intelligent Systems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and Electronics Laboratory (NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Dutch Airforce (NL), EURISCO (F). Apart from the authors, many other people contributed to the project from the organisations listed above. We would like to thank them all for their efforts.

- 27 -

after indexing them in a meaningful way, in an object oriented database. This database with fragments can be used to retrieve content when creating training material with an authoring environment chosen by the user. The project delivered a collection of inter-operable tools consisting of: •

A database schema for describing fragments

•

A Document Analysis Tool that analyses, fragments and indexes technical manuals

•

A Database Facilities Tool for managing the database, including versioning of fragments

•

An Authoring Environments Interface Tool for retrieving fragments from the database and transferring selected fragments to an authoring environment; also used for additional indexing and annotation of fragments

•

An Ontology Development Tool for creating ontologies in a structured way

•

A set of ontologies for describing fragments, domain specific as well as generic, including an extensive instructional ontology

•

An Instructional Scenarios Tool supporting the creation of skeletal lessons for maintenance tasks. The skeletal lessons can be exported to the Authoring Environments Interface Tool and used for guiding database queries whose results can be stored in the skeletal lesson

The tools were used at three different user sites, each representing different approaches to creating training material and teaching: maintenance of anti-aircraft weapons, car repair and maintenance of traffic control equipment. For a better understanding of the design and workflow of the IMAT tools figure 1 below can be used.

General Generalworkflow workflow

Document 3 Document analysistool analysistool analysis

Source document

6

Domain ontologies

5 Generic / instructional ontologies

Scenario tool 105 Jasmine 2 OO Database

Data Data base base facilities facilities 2

Other source material

Authoring 7 2 environment interface interfacetool interfacetool

Indexing tool

3

Org m . em. 4 tool

Authoring environment

Covered in this paper

Figure 1: Workflow and structure of the IMAT toolset

The grey boxes are the tools that were developed in the IMAT project. The yellow boxes are the ontologies, which are at the core of the system. The green boxes represent commercial-of-the shelf software used. In fact much more was used than shown, but this would clutter the picture. The basic workflow is that a source document is analysed by the Document analysis tool, which breaks it up in small parts (fragments) and indexes these fragments. The fragments are exported to the Muli-media data base from which the fragments can be retrieved with the Authoring environment interface tool. This tool also provides functionalities for additional indexing of fragments and adding “lessons learned” like notations to fragments with the Indexing tool and the Organisational memory/learning tool. The retrieval can be guided from an instructional viewpoint by using the Instructional scenarios tool. Finally the retrieved fragments can be copy/pasted into the - 28 -

Authoring environment chosen by the author of the instructional material. In this paper we will describe the tools at the “down stream” side of the process (shown by the dashed box in Figure 1), which support the re-use of the stored material from the manual. The accompanying paper (de Hoog et al., 2002a), provides details about the “up front” part: analysing and fragmenting the technical manual and storing the content in a database.

The authoring environments interface tool In order to be able to retrieve fragments from the technical manual that were extracted by using the Document Analysis Tool, the Authoring Environments Interface tool was developed. The tool is used as a mechanism for transferring selected fragments from the database to the “authoring environment” (e.g. MS Word®) a person creating instructional material is using. This can be any application that supports the copy/paste facility in MS Windows®. The start window of the tool is shown in Figure 2.

Figure 2: The main window of the Authoring Environments Interface Tool

The “tabs” in the window in Figure 2 make it possible to search for the four main types of fragment (text, pictures, video, sound), but simultaneous retrieval can be done activating the set boxes to the left under “Search”. The main window is used for displaying retrieved material on the empty canvas. By clicking on “Set criteria” the user can specify what is searched for. This is done with the window in Figure 3.

- 29 -

Figure 3: Window for defining search criteria

The slots in the window in Figure 3 represent the search terms that can be used: topics, keywords, description type, knowledge type and use. The “+” and “-“ push buttons open lists of predefined terms, which are derived from the ontologies, that can be selected by the user. After filling all or only a few of the slots, the user clicks on the “Retrieve” button, after which the search is started. In the “Search log” section, a count is given of the number of fragments matching the query. After the search is over a “Display” button appears, which after being pushed, brings the user back in the main window. This shows a list of retrieved fragments in the upper part, each of which can be displayed in the canvas by activating the associated set box. Figure 4 shows a window in which a piece of text from the traffic controller manual is displayed. Figure 5 displays a picture that has been retrieved.

- 30 -

Figure 4: Display of retrieved text fragment

Figure 5: Display of retrieved picture

- 31 -

For pictures the tool offers possibilities to scale it, to show thumbnails when many pictures are retrieved, as well as selecting part of a picture and re-use it in the authoring environment. When the picture is an analysed diagram present as SVG in the database, search can also be performed on the content of the picture. Thus searching for a particular component with “text” and “picture” selected, will return all fragments (text and pictures) containing this component, even if the bitmap version of the picture was not explicitly indexed as containing this component.Transfer to the authoring environment can be done by using the Clipboard (copy-paste) facility or by storing the fragment in a file and retrieving it later from the file in the authoring environment. This is the basic functionality of the tool. However, it contains additional facilities, which will be described below.

The indexing support tool Originally it was planned to develop a separate tool for indexing fragments. However, when the project progressed it became clear that indexing would take place either in an automatic way or by hand. Following the logic of the workflow, it became clear that indexing by hand should be tied to existing fragments. As fragments are stored in the database and are already indexed on structure and domain topic, the best strategy was to integrate the indexing tool into facilities for accessing and managing the database. As a consequence there is not a separate “indexing tool” available in the IMAT toolset. The indexing facility is integrated in the Authoring Environments Interface tool and the Data Base Facilities. In the Authoring Environments Interface Tool a retrieved fragment can be given additional indexes (“mark up”) using the instructional ontology. This is shown in Figure 6.

Figure 6: Indexing a fragment in the Authoring Environments Interface

In Figure 6 a retrieved picture (not shown) is going to be indexed with a particular knowledge type: perceptual knowledge of structure with visual identification of parts. With this facility any fragment that has been created with the Document Analysis Tool, or has been entered into the database in another way, can be indexed. Indexing can also be done with the Data Base Facilities tool (see de Hoog et al., 2002a).

- 32 -

Both processes are very similar. However, in the Data Base Facilities Tool the person doing the indexing has more discretion than in the Authoring Environments Interface Tool, because also Keyword and Topic indexing can be changed, which should be protected because they are key indexes which cannot be allowed to be changed by any arbitrary user.

Organisational memory/learning tool The organisation memory/learning tool serves the need to store in a systematic way experiences gained with technical documentation and instruction. As these experiences are bound to content, it was a logical decision to couple these experiences to content in the shape of fragments. Broadly speaking two types of experiences can be distinguished: •

experience in using a fragment, during work as well as during instruction

•

detecting mistakes in fragments that should be corrected.

Each type should be handled in a different way. As most experience in using a fragment is welcome, it should be accessible to anyone using the fragment. Consequently the facility to annotate fragment from a use perspective was incorporated in the Authoring Environments Interface Tool, because this is the location where retrieval takes place. Figure 7 shows the window that can be activated in the Authoring Environment Interface Tool for collecting and storing use experiences.

Figure 7: Annotation of a fragment in the Authoring Environments Interface Tool

In Figure 7 there are four categories of experiences that can be entered: usability, usefulness, evaluation and additional information. Any user can enter comments/experiences with this facility and, when a fragment is retrieved by another user, these annotations can be displayed on request (see the “notes” indication in the window in Figure 4). In this way an efficient information exchange mechanism concerning fragments between users of the documentation has been put into place. Though the annotation facility should be accessible for everyone using fragments, this does not hold for updating fragments. All manuals contain mistakes that are discovered during use, but the cycle time between the detection

- 33 -

of a mistake and an official update from the manufacturer is often very long. In the mean time it is necessary to store the mistake without changing the “official” content of the technical documentation. The IMAT Data Base Facilities Tools allows for this, but only for persons authorized to do it. By using the Data Base Facilities tool versions of fragments can be created. A fragment containing a mistake can be edited and the modified fragment stored as a draft. In Figure 8 a text fragment has been retrieved (left hand pane in window). Assume that during using the technical documentation it was discovered that “lamp D3” is wrong and should be “lamp D4”. The authorized person can now change this in the left hand pane and store the corrected fragment as a “Draft Revision” (check boxes, right below). When retrieving the original fragment using the Authoring Environments Interface Tool, this will show the existence of two versions of the fragment: the original one and the draft revision. When the manufacturer sends out an authorized correction the original fragment can be discarded and the draft revision elevated to “original” status.

Figure 8: Creating draft revisions of fragments when mistakes are detected

Together these facilities provide the users with a versatile and powerful environment for codifying and exchanging experiences. This is particularly true when the database runs in a real server-client context which makes asynchronous and geographically dispersed sharing of knowledge about the usefulness and correctness of content of technical manuals, for instructional as well as operational use, possible. Nonetheless a cautionary note has to be struck. One of the original objectives was to achieve automatic updating of fragments used in instructional material created with an arbitrary authoring environment. This turned out to be too difficult as it is impossible to keep track of where fragments end up after they were retrieved from the database. This is the price that had to be paid for flexibility at the authoring end of the workflow. Had the project settled for a very tight integration with one particular authoring environment (say Toolbook® or Authorware®), this could have been realized. However, the drawback, forcing IMAT users to shift from the current authoring environment in use to a new, unfamiliar, one, was seen as too serious.

- 34 -

Instructional scenarios tool During the project the user partners stipulated that in re-using material they felt a need for guidance at a pedagogical level, in particular the structuring of (a set of) lessons according to sound instructional knowledge about teaching maintenance skills. Following this requirement a tool was developed that allows a user to develop a skeletal structure for lessons in a certain domain. Based on theoretical work, part of the construction of a skeletal lesson is done automatically as a choice at one step in the process, constrains the choice later on. Figure 9 shows the interface of the Instructional scenarios tool.

Figure 9: The initial steps in the Instructional scenarios tool

In Figure 9 the top-level bars from left “Learning goal” to right “Material use”, are the sequential steps in constructing a skeletal lesson following the instructional ontology (see de Hoog et al., 2002a). First a learning goal is selected and the tool automatically selects the fitting fragment description (“Definition; Location of component”). Next the user selects a knowledge type (“declarative knowledge, knowledge of structure, composition”) and an instructional strategy (not shown in Figure 9). All these selections are either made automatically or represent a selection from a wider range of options which are constrained by previous choices. After choosing a particular instructional strategy the remaining columns are derived automatically (see Figure 10).

- 35 -

Figure 10: Complete scenario

The content of the column “Instructional activities” contains the skeletal lesson consisting of a sequence of things that have to be done. The column “I/L actions” specifies which kind of activities are expected from instructors and learners during a phase of the skeletal lesson. The final column “material use” indicates what role the material used plays in the related phase. For example, during the “Presentation” activity the material used plays mainly a definitional role. After a skeletal lesson has been identified it can be saved to a file for later reference. Apart from this “stand alone” use, the Instructional scenario tool can also be used in conjunction with the database. The structure constructed in Figure 10 can be transferred to a data base context, which contains the material that could be used to retrieve relevant material. If this option is chosen, the Authoring Environments Interface Tool shows a “bag” with the name that was given to the skeletal lesson (see Figure 2). This “bag” can be opened and then it shows the structure of the skeletal lesson developed using the scenario tool. This is depicted in Figure 11.

- 36 -

Figure 11: Authoring Environments Interface Tool with an open bag

In Figure 11 the “instructional material bag” has been opened showing the different actions and activities defined in the skeletal lesson. The user can now employ the characterizations from the skeletal lesson, e.g., knowledge type and material use, together with a topic or keyword to guide the retrieval, that is setting search criteria just as in Figure 3. The retrieved and selected fragments can now be stored in the relevant sections of the bag, instead of being copy-pasted to the authoring environment. When finished, the bag can be exported to a file structure that reflects the structure of the skeletal lesson. This allows the author of instructional material to access the material at a later date, without the need to start up the database. It should be stressed that this facility is very important as the working of the Authoring Environments Interface Tool assumes that when a new query is fired the results of the previous query are erased. This forces the user to decide on what to do with fragments (copy-paste them to the authoring environment or not) before the next query is started. The bag mechanism provides a “storage” facility with which this decision can be postponed if needed. The Instructional Scenarios Tool (“wizard”) can also be activated directly from the Authoring Environments Interface Tool. This turns out to be a powerful way of guiding people, who are not pedagogical experts, in building good lessons. The link with the database and the Authoring Environments Interface Tool guarantees an easy and natural connection between structure and content, by “pre-fabricating” the kind of meaningful queries that can be entered.

Using the IMAT tools Though this paper is mainly devoted to a description of a part of the tools, we will briefly say something about the application of the tools in practice.

- 37 -

For creating the instructional material several different authoring environments were used: MS Powerpoint®, MS Word®, Toolbook®, and an HTML editor. The size of the training material ranged from 50 – 100 Powerpoint® sheets to a complete Toolbook® application. From the latter a page is shown in Figure 12.

Figure 12: Part of the Toolbook® instruction built with the IMAT toolset

Figure 12 is part of an application that is used at AFPA for instructing learners in the area of car body repair. When measured against expected benefits it was shown that the use of the toolset led to: •

A faster production of training material

•

Less expenses when updating training material

•

Improved consistency between training material and technical documentation

•

New ways of training and teaching, in particular the migration from paper based to computer based training

Moreover the tools were extensively used in the daily practice of the people who created training material and give maintenance courses. When asked, they confirm that the tools have added value in their work.

References R. de Hoog, Y. Barnard & B. Wielinga (1999). IMAT: re-using Multi-media Electronic Technical Documentation for Training. In: J.Y. Roger, B. Stanford-Smith & P.T. Kidd (Eds), Business and Work in the Information Society: New Technologies and Applications. IOS Press, p. 415 – 421.

Hoog, R. de, B. Wielinga, S. Kabel, A. Anjewierden, F. Verster, Y. Barnard, P. DeLuca, C. Desmoulins & J. Riemersma (2002a). Re-using technical manuals for instruction: document analysis in the IMAT project.

- 38 -

A scenario tool for coupling human and computerised expertise in the development of training material for complex equipment maintenance

Yvonne Barnard*, Cyrille Desmoulins** & Monique Grandbastien*** * EURISCO International, France [email protected] ** CLIPS IMAG & Université Joseph Fourier Grenoble 1, France, [email protected] *** INA P-G and LORIA Université Henri Poincaré Nancy1, France, [email protected]

Abstract This paper describes a scenario-based approach to assist authors to produce training material from original technical documents in the domain of complex systems maintenance. This approach offers the authors technical and teaching expertise through the use of ontological indexes, in order to build lesson scenarios and to derive useful fragments from technical manuals to be inserted into training material. The scenario tool we developed is described and its use and rationale are presented, as well as its links to others tools developed in the framework of a European project on integrating manuals and training.

Keywords authoring systems, cognitive approaches, human factors, instructional design, instructional meta-data, ontologies, vocational training

Introduction In ITS and computer based training research it became clear that developing computer programs that can teach effectively requires large amounts of effort from experts from different disciplines. Instructional design methods (e.g. Gagné, Briggs, & Wager, 1992) and tools (e.g. Designers Edge from Allen Communication) can support the developer, but still a lot of human expertise must be put into the design. Our research focuses on the support of training material developers in the field of complex equipment maintenance. Training is a key factor for the competitiveness of industry in general and the high-tech industries in particular. Companies spend significant amounts of resources to provide vocational training tailored to the products and services they produce. This paper provides an in depth analysis of the process for developing technical training documents. Then it shows how each step of the development process can be automated or partly assisted. Such an assistance has been implemented into a set of tools in the IMAT project (de Hoog, Barnard & Wielinga, 1999). Among these tools, we describe the so-called “scenario tool” which provides a computer-assisted framework to create training material from technical documents produced by the manufacturers. Moreover the same environment allows the training document designer to add his own teaching expertise to the selected technical documents, the data base of technical documents is enriched by annotations represented as additional indexes. We conclude on the benefits of this set of tools for designing training documents adapted to the changing needs of the professionals and on the perspectives of a corporate training memory that can be initiated through the use of such tools in companies.

- 39 -

Training for maintenance work Developing educational programs asks for different kinds of expertise such as domain knowledge, pedagogic expertise, and knowledge about the students to be taught. In general education, in domains such as mathematics or history, the different forms of expertise are usually available. However, in specialist forms of training this is much more of a problem. Our research described in this paper is concerned with training for maintenance technicians. Characteristics and problems of training for maintenance technicians are: •

Keeping training up-to-date: technical systems and its components change very rapidly.

•

Dealing with variety: many modern products are on the market with a variety of configurations, there is usually not just one single system available, but systems are more and more customised, asking for different maintenance procedures.

•

Training for small groups: courses for specialists usually have a very limited amount of trainees.

•

Training has to be very specific and is often safety critical: trainees need to learn exactly what to do and how to do it, making mistakes in maintenance tasks can be dangerous or costly.

•

A constant need for training and re-training: nowadays technicians are not only to be trained when they are young, then progress towards expertise, and stay on their job for the rest of their working life. People change jobs more readily, so the company has to deal with a continuous stream of untrained newcomers.

•

No training material on the market: training of maintenance skills is generally not part of the public curriculum and therefore not an attractive market for educational publishers, forcing companies to produce their own training material.

•

More theoretical training has to be combined with practical work: this means that the training site is preferably close to the workshop. Furthermore, maintenance is usually done at local sites, so it would be best at least part of the training could be done locally.

These characteristics lead to the situation where higher demands are put on the quality of training while at the same time less time is available to produce training. Teachers in maintenance are usually experienced technicians themselves, sometimes with additional didactic schooling. Sometimes they work only part-time as an instructor, spending the rest of their time doing maintenance work, or coaching inexperienced technicians at the workshop. These people are scarce. Good, self-explanatory learning material, which can be easily updated, is therefore very welcome in technical domains. Computer-based training, centrally developed and locally delivered seems to offer an excellent solution to several of the problems. However, who has to develop the learning material? Again it comes back to the problem of the availability of experienced technicians, who have to provide the domain and didactic expertise. Developing training material in technical domains furthermore asks for a high level of multi-media use, especially complex graphics, schemes and photos are needed to show the trainees where components are located, how to remove, repair and install them, what the functions are of subsystems etc.

Development of training material Training material for technical training can consist of different kinds of documents, in paper- or in electronic format, such as hand-outs for trainees, syllabi, transparencies, electronic slide presentations (e.g. PowerPoint), courseware and trainer documents. Training documents contain texts, pictures, schemes and diagrams. Also animations or videos of processes can be used. Training documents are usually produced in an electronic form, but can be converted to paper-based end-products which are given to the trainees. The content of technical training documents is usually well structured, focused on the technical system and its components, which the trainee has to learn to maintain. The main part of training documents consists of (Barnard, Riemersma, & de Hoog, 1998): • structural descriptions of the components and their connections • functional descriptions of components • procedures for maintenance and repair • safety measures

- 40 -

• faults and diagnostic procedures The work described here comes partly from the European IMAT-project (Integrating Training And Manuals)1 in which tools and methods for the creation of training material from fragments of the technical manual are developed (de Hoog, Barnard & Wielinga, 1999). The user organisations involved in the project faced all of the problems of maintenance training described before. Training material had to be developed for maintenance technicians in the domains of weapon systems, car repair and traffic systems maintenance. These are very different domains, but with most of the problems mentioned above in common. The partners involved had developed paper-based material and PowerPoint slides, often at high costs in terms of effort, or had not even begun to develop training material, which meant that they had to provide a large amount of support to beginning technicians during their job. The project aimed to provide support for developing training material faster, cheaper and better. To reach these goals, the following principles were applied: •

(Re-)use already existing material and domain expertise;

•

Automate as many parts of the development process as possible;

•

Facilitate centralised development, allowing for easy local customisation;

•

Make collaborative development easy, and minimising the effort of domain experts.

These principles could be used because of the specific nature of technical, maintenance training. In the first place, although training material is hard to find, technical material is always available. Complex systems have huge amounts of technical manuals, both for operations and for maintenance purposes. These manuals are difficult to use, and cannot simply be given to trainees to serve as learning material. However, a large quantity of domain expertise is stored in them, and if one could get it out in a form that is suited for training, much development time could be gained. Technical training is not the same as general school education. It is usually very structured; many of the things that have to be learned follow the same pattern. For example, for each component of a system you have to know the location, its function, its connection with other components, how to remove and replace it. Therefore training, also when given by a teacher in a classroom, usually follows the same patterns, treating many different components, but always in the same way. The learning goals are usually addressed by using a rather limited set of didactic strategies. There is also much communality between technical maintenance training for very different systems. Using these characteristics of technical training, domain expertise from the technical manuals was taken as the basis for development of training material and didactic expertise was standardised and made available. Lessons can be described in the form of scenarios, indicating which steps have to be taken by both trainers and trainees to reach the learning goals. For each step of a scenario, a fragment of technical material can be selected which provides the domain knowledge. For example, when the lesson-scenario describes first to teach the function of a component and next the structural position, the learning material should contain a functional diagram and a picture of the position of the component in the equipment. Because the didactics in maintenance training are a rather structured set, didactic ontologies could be developed (Kabel, 2001). These didactic ontologies served as the basis for the design and implementation of a scenario-tool, providing the developer of training material with didactic expertise and supporting him or her to select the right kind of information on the domain. This scenario-tool is part of the set of IMAT tools, methods and scenarios, developed to assist companies in the transition from paper-based training to a training based on effective and efficient use of available electronic documentation. The tools provide capabilities to analyse and segment electronic documents on the basis of characteristics of graphics and text and the relations between them. Coherent segments of information are stored in a company-wide database. Ontologies providing meta-data about an application domain, together with didactic knowledge about classification of instructional elements and scenarios, are used to index and annotate the database. The IMAT approach also provides support to develop a corporate memory and feedback structure. The database with indexed and, if needed, annotated fragments taken from maintenance manuals, provides the 1

The IMAT project (ESPRIT 29175) was partly funded by the European Union. Partners in the project were: Intelligent Systems Laboratory, University of Amsterdam (NL), TNO Human Factors and TNO Physics and Electronics Laboratory (NL), Loria (F), Renault (F), AFPA (F), ETRA (E), Tecnopolis CSATA Novus Ortus (I), Royal Netherlands Air Force (NL), EURISCO (F).

- 41 -

developer of training material with an easy accessible source for finding the raw material needed for a particular topic. We will not describe in this paper how the database is constructed, but focus on the use of the database by the scenario-tool.

Activities in the development of training documents There are three major steps in the development of training material: 1. Set up a lesson structure In this step the developer creates a structure or scenario in which the content can be filled in. For example a lesson on a component starts with the objectives of the lesson, the location of the component in the system, the function of the component, the possible malfunctions, and the repair methods. The structure can have the form of a template, for example a set of PowerPoint sheets where each of these lesson elements can be inserted. In a large organisation, templates can be made available at a centralised level, taking into account the house style, ensuring that the lessons given at a decentralised level are homogeneous etc. The more standardised and structured these lesson structures are, the easier it is for the developer to look for the right kind of content (Boot & Barnard, 2000). The scenario-tool described in the sections below supports the developer in creating lesson structures. There are several reasons for providing support in the form of scenarios: •

To provide on the one hand standard ways of creating instructional material and organising training in order to make this process more efficient, while on the other hand allowing for customisation according to the specific needs of the user.

•

To provide standard good practice ways of training, especially for those instructors and developers of instructional material who are inexperienced or who have little background in educational science.

•

To show instructors and developers of instructional material how to set up new and innovative ways of training.

•

To work towards further industrialising the process of training development in technical domain.

2. Search and retrieve content In order to fill in the slots in a lesson scenario, the developer has to find the right source of content. This is usually a complex and time-consuming task. For example, it would be very enlightening for trainees to see a picture of a malfunction in a system, but it is not easy to find a photograph of a faulty component. For technical training, much of the content may come from the maintenance or the operational manuals. If the original manual does not contain material that is good enough, sometimes material can be found in the manual of another system, because particular components look alike. In order to shorten the time the developer has to search for content, the IMAT tools take technical manuals, fragment them, index them and store them in a database. The training material developer can now search for fragments on a particular topic (such as component X) or with special characteristics (such as a front view of the component). The IMAT tools allow the developer to search for fragments, using several kinds of indexes. He or she can search on topics or keywords, on descriptions of the kinds of fragments (such as short or elaborate ones), on media (text, pictures, audio or video) and on instructional indexes such as material use and knowledge type. During the search the developer can also index the fragments with these last kinds of indexes, for example if he or she finds a fragment to be used as an example, the tag "example" can be added to this fragment. In this way several developers can build up instructional knowledge. The selected fragments can fill in the lesson scenario slots as defined in step 1. This scenario structure in the IMAT database is called an “instructional bag”. A screendump of the tool with which fragments can be searched is given in figure 1. 3. Process content After the raw material has been gathered, the lesson needs to be further processed in an authoring environment. This can be a very simple environment like a word processor or a presentation program, but it is also possible to use a special authoring language to create CBT or a tool to create web-based training. In any case, the developer has to use his or her creativity and knowledge about presentation techniques to create useful and effective training material. In the IMAT system, the user can directly drag a fragment to a standard Windows authoring environment, but the usual way is to take the instructional bag with the fragments and use these for further processing. The scenario serves as a template for the lesson to be made, the developer knows for which purpose

- 42 -

the fragment was chosen and the actions the instructor and the trainees are supposed to perform on this piece of training material.

Figure 1: IMAT fragment search tool

The scenario-tool In this section we will describe in more detail the working of the scenario-tool. In IMAT this scenario-tool has been developed to guide the developer to set up lesson-structures (Verstegen et al., 2001). The developer starts with a certain topic he or she wants to make a lesson about, for example a part of the motor of a car. The scenario tool supports the developer in defining a scenario by combining pre-defined instructional patterns. The scenario is gradually elaborated by selecting, adapting or creating the scenario elements (learning goal, knowledge type, instructional strategy, instructional activities, instructional steps and actions) (Kabel, Riemersma & Wielinga, 2000, Chen 1995). Depending on the choices of the user, an appropriate selection is offered to user in each step. For example, if the user has determined an instructional strategy, he or she will be provided with the list of instructional activities which belong to this strategy. With the support of the scenario-tool, the developer has to perform the following steps for each topic: 1.

Set a learning goal, to be chosen from a list of learning goals suited for technical training.

2 . Determine the knowledge type the trainee has to acquire, for example perceptual knowledge or performing skills, choosing from a list of knowledge types belonging to the chosen learning goal. 3.

Choose an instructional strategy, for example exploration or modelling, again choosing from a list of strategies belonging to the chosen knowledge type.

4.

Determine a set of instructional activities, the activities the instructor and the trainee have to perform, for example setting goals, asking questions, feedback etc. The list of activities is presented which belong to the chosen instructional strategy.

5.

Define accompanying instructor and learner actions, a further refinement of the instructional activities, for example present a learning goal, present a summary, read an explanation, memorise instructions.

When the developer has made all the choices, he or she is provided with a scenario, a standard lesson structure which contains advice on the kinds of fragments with which the lesson can be build. For example fragments on procedures, tools, or tests. The scenario can also be edited. When the scenario is ready, it can be stored in the database. Next the developer will use it to select and retrieve useful fragments from the database.

- 43 -

Step 1: set a learning goal The scenario-tool starts to provide the user with an empty screen in which the scenario will be constructed stepby-step (figure 2). Most of the work for the user consists of choosing the appropriate items, upon which the tool suggest limited sets of other categories from which to chose.

One column of the scenario

Figure 2: Scenario tool, with empty screen

Figure 3: Learning goal choice

- 44 -

The developer starts with selecting a learning goal from a (collapsible) tree, figure 3. Upon validation of the dialog box, the “Learning Goal” column will be filled in with the item chosen, then the “Fragment Description” column will be filled in with appropriate text automatically and the next column will be available for modifications. Step 2 Determine the knowledge type The next step is to choose the knowledge type (figure 4). The user can modify at any time any previous (and editable) columns, but every change in a previous column may cause an update of next columns. For example, if you change the “Learning Goal” column, every next column will be cleared (unless you have chosen the same item). The knowledge type to be chosen is restricted by the learning goal selected at the previous step. Then if you choose another learning goal, you have access to other knowledge types.

Figure 4: Knowledge type choice

Step 3 Choose an instructional strategy The next step is to fill in the instructional strategy column. The instructional strategies which can be selected are restricted by the knowledge type’s choice. If you choose another knowledge type, you will get different instructional strategies. In this step, one instructional strategy has to be selected. Step 4 Determine a set of instructional activities & Step 5 Define accompanying instructor and learner actions Upon completion, the following part of the scenario will be automatically filled in (figure 5). Instructional activities, instructor and learner actions and material use are filled in with appropriate contents regarding the instructional strategy you have chose. Each cell in the “I/L Actions” is prefixed with “Learner” or “Instructor” depending of the role who performs the action in the scenario. The scenario can be completed by instantiating “Action description” column with free text. This text will not be used for the search of instructional fragments but it will helps the user to complete the scenario with concrete and precise actions related to the specified instruction. After completing the scenario, the tool is able to save it into the database. The scenario can also be exported to an HTML document and printed out. Apart from this “stand alone” use, the Instructional scenario tool can also be used in conjunction with the database which contains the fragments from the technical manuals. The user can now employ the - 45 -

characterizations from the skeletal lesson, e.g., knowledge type, material use and description type, as search criteria, together with a topic or key word. Concretely, he or she can directly launch the appropriate fragment search using a pop-up menu on each action. The retrieved and selected fragments are then automatically stored in the relevant action of the instructional bag, instead of being copy-pasted to the authoring environment. When finished, the bag can be exported to a file structure that reflects the structure of the skeletal lesson. This allows the author of instructional material to access the material at a later date, or to share it with other authors. The link with the database and fragments search tool guarantees an easy and natural connection between structure and content, by “pre-fabricating” for the author the kind of meaningful queries that can be entered.

Figure 5: The filled in scenario

Using scenarios to index fragments An important function of the scenario tool is to get useful fragments that are indexed in different ways, by knowledge type, material use (derived from instructor and learner actions), fragment description type and topic to be taught. It is possible to search in the database for fragments that are marked-up with these fragment attributes. For example one can look for a fragment, which is about component X (topic), about how to take precautions (knowledge type), is a reminder (material use), and has a task/procedural description of a condition (fragment description type). In the document analysis process, the fragment description type is derived automatically to a large extent. Most fragments are therefore indexed on fragment description type. Knowledge type and material use, on the other hand, have to be indexed manually by the training material developer. Therefore, it might be possible that not all attributes have values, so the developer has to search for fragments based on a subset of values. However, the scenario tool can also help authors to index fragments, thus providing more support for a next cycle of development or for another author. In figure 6 the uses of the scenario tool are summarised.

- 46 -

Database of fragments

fragments

requests

scenario with search indexes components

Retrieval tool

Scenario Tool

authoring/ indexing

composing scenario

Figure 6: Scenario Tool and relation with user and other tools There are three ways of initiating the instructional indexing process: •

Standard use

If a developer first starts working with the database of fragments, the knowledge type and material use attributes will have no values. During the selection of fragments, the developer can mark-up the fragments with this information. This means that the knowledge type and material use tags are added to the fragments. So the next time a developer uses the database, he/she can use these search terms. In the IMAT tools, the retrieval and the indexing process are integrated; it is possible to perform the mark-up during the authoring process. •

Inverse use

The developer can use the Scenario Tool to initiate the instructional indexing by searching for relevant fragments for a given scenario without the instructional indexes (because there are none or few ones), keeping in mind these instructional indexes and then index the retrieved fragments with these scenario instructional indexes. He or she can also obviously index without using the Scenario Tool in this “inverse” way. This approach is more driven by the instructional point of view of the development work, in which the main task of the developer is not indexing. •

Extensive use

It is also possible to index (nearly) all the material with instructional mark-up before an authoring process. This might be useful in a situation where a central training centre has a database, which is to be used on location in order to produce training material locally. In this case, someone at the centre is devoted to do the indexing. However, even if these three different ways of initiating the instructional indexing are concretely available, the users’ feedback on the IMAT tools indicate that the extensive use is rarely possible and that on-the-fly indexing (standard or inverse) is preferred. Note that fragments can have more than one value for knowledge type and material use. For example a fragment might be used as an explanation as well as an illustration. Fragments can also be annotated with the experiences in use, such as the usability, clearness, quality etc. of the fragment. To this purpose an annotation system is available, based upon an ontology of organisational learning.

Conclusions In technical training, work and training are often closely coupled. The things trainees are taught during their training should be directly applicable in the actual maintenance work they are going to perform. Furthermore, there is a trend towards more integrated forms of training and working, on-the-job-training and just-in-timetraining. Trainees are not only trained in the beginning of their career as maintenance engineers in a classroom

- 47 -

setting, but more and more training takes place at the workshop during the actual work. People also receive retraining when systems are modified or new systems arrive and are sometimes trained just before a new or specialised task has to be performed in which they are not proficient. By the scenario-based approach we described in this paper we allow developers of training material to re-use fragments as they appear in the technical manual and to face the above mentioned new challenges. They can place these fragments just as they are in the context of the training material, adding explanations if necessary. It is easy to build scenarios which simulate the normal course of maintenance work, instantiated with the “real” text and pictures from the technical manual, thus providing the trainees with realistic cases. In addition, the fragment search, which is very time consuming, is incorporated directly in the core-task of the author of training material, i.e. designing a lesson. By the ontology-based approach that has been adopted for all the tools in the project, we aim at promoting a high level modelling of training activities resulting in the sharing of general and domain specialized teaching expertise. In the IMAT tools an extensive annotation facility is provided, both at the fragment, the scenario and the instructional bag level, which was not further described in this paper (see Barnard & Boy, 2001). It can be viewed from every screen in the tools. This facility allows authors to comment on each others work, and to revise material after feedback from the training or from the workshop. By adding annotations, users can build-up collaboratively knowledge and experience about developing and using training material. By initiating an annotation process through the scenario tool, we hope to help training material designers to be on the frontline for building the coming knowledge society, promoting collaborative and distance authoring and experience sharing. Developers of training material at several training centres have tried out the scenario-tool, but now we need more feedback from users in order to adapt these tools to their changing needs. Finally, we believe that such processes making a systematic and heavy use of digital resources combined with human expertise should help workers in their life-long learning processes in rapidly changing industries.

References Barnard, Y.F., & Boy, G. (2001). Organisational memory. ESPRIT 29175, IMAT report. Barnard, Y.F., Riemersma, J.B.J. & Hoog, R. de (1999). The use of IETM's in training for maintenance engineers. Proceedings of the 21th Interservice/Industry Training, Simulation and Education Conference, pp. 1136-1144. Boot, E.W., & Barnard, Y.F. (2000). A building block method for structured courseware development. In: J. Bourdeau & R. Heller (Eds.) Proceedings Ed-Media 2000, World Conference on Educational Multimedia, Hypermedia & Telecommunications. Charlottesville: AACE, p. 1566-1568. Chen, H. (1995). A methodology for characterising computer-based learning environments. Instructional Science, 23, 183-220. Gagné, R.M., Briggs, L.J., & Wager, W.W. (1992). Principles of Instructional Design (4th ed.). Forth Worth: Harcourt Brace Jovanovich College. Hoog, R. de, Barnard, Y.F. & Wielinga, B., (1999). IMAT: Re-using multimedia electronic technical documentation for training. In Roger, J.-Y. et al (Eds.) Business and Work in de Information Society: New Technologies and Applications, IOS Press, p 415-421. Kabel, S. (2001). The added value of ontology-based instructional mark-up. In: J. Moore et al. (Eds.) Artificial intelligence in education. IOS Press, AI-ED 2001 San Antonio. Kabel, S., Riemersma, J. & Wielinga, B. (2001) Final set of ontologies. IMAT deliverable O3. Verstegen, D., Barnard, Y.F., Kabel, S., Riemersma, J., Desmoulins, C., & Grandbastien, M. (2001). Scenarios and Guidelines, version 2. IMAT deliverable R.IV.3.

- 48 -

Generating Training and Assistive Dialogues for Astronauts from International Space Station Technical Documentation Gregory Aist and Beth Ann Hockey Research Institute for Advanced Computer Science (RIACS), Mail Stop T27A-2, NASA Ames Research Center, Moffett Field CA 94035-1000 USA

{aist, bahockey}@riacs.edu http://www.riacs.edu/

Abstract Human spaceflight requires extensive and thorough procedures that cover a wide range of activities to be performed both in space and on the ground. The technical documentation containing these procedures is a fruitful source for potential training material, since crewmembers and mission control personnel undergo long and extensive training in the course of preparation for particular mission assignments. One of the main goals of the RIALIST group at Ames is to support Human Exploration and Development of Space (HEDS) by applying spoken dialogue technology to training and task performance on the ground and in space. We are moving to address this goal by developing a system aimed at taking technical documentation from the International Space Station and producing spoken dialogues geared towards astronaut training and task support. This system is in the early stages of development – six months as of June 2002 – and is still at the design and early prototyping stage. In this paper, we introduce the scope and nature of tasks performed on board the International Space Station, describe our system architecture, discuss some of the key issues that have arisen in building the early prototypes, and lay out future challenges.

Keywords astronaut training, task support, spoken dialogue systems

Introduction Efficient and effective training is key to human spaceflight. Spaceflight is a complicated endeavor carried out with people and machines operating in tandem. A wide variety of tasks are performed on board the International Space Station. For example, the ISS on-board status report of March 28, 2002 (http://www.spaceref.com/news/viewsr.html?pid=5093) included the following activities shown in Figure 1. Current astronaut task practices include a mix of high- and low-fidelity simulations. There are time delays between training & use. Some procedures are performed frequently, while some are performed quite infrequently. There are lots and lots of manuals. Finally, in some cases, another person is reading the checklist to the person performing the task.

- 49 -

· “live interactive TV exchange with Bath Elementary School in Bath, North Carolina.” · “unloading of Progress 7P, using the IMS (inventory management system)” · “transferr[ing] the Russian radiobiology experiment RBO-2 "Bradoz" from the Progress and install[ing] it in the SM.” ·

“solid sorbent air sampler” activity

·

collecting “FMK (formaldehyde monitoring kit) badges”

·

“Part 2 of the inspection of the docking assembly…”

· “… the second part of the BINS strap-down inertial guidance system update/correction using the VP2 and PUMA viewfinder/sighting devices.” · “reviewed the updated procedures of the MSS (mobile servicer system) operations” ·

“completed a dry run of the S0 installation”

· “In preparation for the 8A spacewalks, … performed a thorough checkout of two SAFER (simplified air for EVA rescue) units” · “prepared the Joint Airlock (A/L) for the coming EVAs, among else unstowing EMU (extravehicular mobility unit) equipment bag and EMU servicing kit, and attaching the equipment bag at a designated location in the A/L, ready for use. EMU reconfiguration and spacesuit sizing is scheduled for tomorrow.” · “Bursch completed the weekly FFQ (food frequency questionnaire) on the MEC laptop, which keeps track of the crew's nutritional input, …” ·

“while Onufrienko did the regular BRPK-1 condensate water separator.”

·

“The crewmembers performed their daily physical exercise.”

· “Today's target areas for the CEO program…” (photography): Lake Eyre, Australia; South Sandwich Islands; North Patagonian Glaciers; South Patagonian Glaciers.

Figure 1. Daily activities on board the International Space Station, March 28, 2002; http://www.spaceref.com/news/viewsr.html?pid=5093.

For each of these types of tasks, there are procedures associated with them. For these procedures, the training should match task; ideally perhaps the same computer assistant should be used for both training and for task performance. We are thus developing a spoken dialogue system for training and task support that is aimed at tasks typically performed on board the International Space Station – with a training deployment envisioned initially, but also directed at eventual on orbit deployment. In related work which includes in-task support, Rudnicky et al. (1996) describe supporting vehicle maintenance with speech interfaces; the Mission Rehearsal Exercise (USC n.d.) aims at supporting “mission-oriented training” for military personnel.

System Architecture We developed a procedure assistant consisting of the following components (Figure 2). First, the speech recognizer (Nuance, using specially compiled language models (Dowding et al. ACL 2001)), recognizes the user’s speech and emits a string of words which are passed to the Gemini parser (Dowding et al. ACL 1993).

- 50 -

Gemini emits logical forms that are passed to the discourse manager (DM). The discourse manager uses a translation function to change resolved logical forms into actions that the procedure manager (back end) can handle. The procedure manager loads a new procedure into memory, and maintains it in memory as a list structure. The procedure manager also navigates between steps in a procedure, and generates task state reports that are then passed to the dialogue manager. For example, a task state report might be [procedure_loaded(camera), “Take pictures of the Earth.”]. The dialogue manager takes the task state report, constructs a combination of logical form (LF) and/or word strings, and passes that to the generator. The generator then constructs strings for the text-to-speech engine (Festival) to emit, based on the command received from the dialogue manager. Each procedure is stored as a separate Prolog file, in order to allow for dynamically switching between procedures.

Speech recognizer

Speech synthesizer

Word strings

Word strings

Parser

Generator

Logical form (LF)

Logical form (LF) and/or word strings Dialogue manager Actions

Task state reports

Procedure manager Procedures Camera Physical exam

… (Other procedures)

Figure 2. System architecture.

Procedure data structure We designed a procedure that walks the user through taking a picture of a view of Earth from space. The procedure is specified as a tree, with each node consisting of a description of the node and a list of its children. The description of the node is written as an unanalyzed imperative. A portion of the camera procedure is shown in Figure 3. Separate left and right lists of nodes are kept so that the current position in the node’s children can be maintained – the head of the right list is always the current position in that list. The procedure is loaded into memory as a Prolog term with the structure of subprocedures left in unexpanded form, and instantiated when needed via a call to procedure/5.

- 51 -

/* procedure(Name, Features, Left, Right, Parent) */ procedure(photograph_the_earth, [description: ’Photograph the earth.'], [], [procedure(unpack_camera, _, _, _, _), procedure(install_battery, _, _, _, _), procedure(install_memory, _, _, _, _), procedure(prepare_camera, _, _, _, _), procedure(take_picture_of_madagascar, _, _, _, _), procedure(unprepare_camera, _, _, _, _), procedure(uninstall_memory, _, _, _, _), procedure(print_pictures, _, _, _, _)], _). procedure(unpack_camera, [description:'Unpack the camera.'], [], [procedure(undo_velcro_strap, _, _, _, _), procedure(unzip_both_sides, _, _, _, _), procedure(remove_camera, _, _, _, _)], _). procedure(undo_velcro_strap, [description:'Remove the velcro strap.'], [], [], _). procedure(unzip_both_sides, [description:'Unzip both sides of the camera case.'], [], [], _). procedure(remove_camera, [description:'Remove the camera from the camera case.'], [], [], _). procedure(install_battery, [description:'Install the battery.'], [], [procedure(open_battery_compartment, _, _, _, _), procedure(remove_battery_from_charger, _, _, _, _), procedure(put_battery_in_camera, _, _, _, _), procedure(close_battery_compartment, _, _, _, _)], _). procedure(open_battery_compartment, [description:'Open the battery compartment.'], [], [], _). procedure(remove_battery_from_charger, [description:'Remove the battery from the charger.'], [], [], _). procedure(put_battery_in_camera, [description:'Put the battery in the camera.'], [], [], _). procedure(close_battery_compartment, [description:'Close the battery compartment.'], [], [], _). procedure(install_memory, [description:'Install the memory in the camera.'], [], [procedure(remove_memory_from_camera_case, _, _, _, _), procedure(put_memory_in_camera, _, _, _, _)], _). procedure(remove_memory_from_camera_case, [description:'Remove the memory from the camera case.'], [], [], _). procedure(put_memory_in_camera, [description:'Put the memory in the camera.'], [], [], _). procedure(prepare_camera, [description:'Prepare the camera for photography.'], [], [procedure(turn_camera_on, _, _, _, _), procedure(remove_lens_cap, _, _, _, _)], _). procedure(turn_camera_on, [description:'Turn the camera on.'], [], [], _). procedure(remove_lens_cap, [description:'Remove the lens cap.'], [], [], _). …

Figure 3. (Part of) the camera procedure. Current Capabilities of the Intelligent Procedure Assistant The intelligent procedure assistant currently has the following capabilities: 1. Load a procedure into the procedure manager. 2. Say again - repeat a prompt. 3. Next step - move to the next step in the procedure.

- 52 -

4. Drill down - unpack a procedure into its component parts. Current procedures available include a procedure to take pictures using a digital camera, and a procedure to conduct the initial portion of a periodic physical exam.

Example Dialogue A hypothetical example dialogue is shown below. (The system can engage in dialogues such as these, although the details may vary.) User: Load camera procedure. System: Camera procedure loaded. First, unpack the camera. User: Now what? System: Now, install the battery. User: How do I do that? System: To install the battery, first open the battery compartment. User: Please explain. System: No more information available about how to open the battery compartment. User: Say that again. System: Open the battery compartment. User: Next step. System: Remove the battery from the charger. User: What now? System: Put the battery in the camera. …

Key Issues We discuss some of the key issues facing us as we develop this intelligent procedure system. Handling long and detailed procedures. Some of the procedures we are working with may be quite detailed, and extend for many steps. Thus we have implemented lazy evaluation of procedures - instantiate only what you need of each procedure, and load the rest on demand. Such long procedures also present demands on the dialogue system to maintain the user’s awareness of where he or she is in the procedure. Representing the task structure of procedures. We chose to use lists of lists to represent task structure, because procedures may contain other procedures and this structure facilitates moving from step to step or drilling down to another level of detail. Representing the meaning of a procedure. Right now, we use an unanalyzed imperative as the first take on what a procedure means; eventually, we would like to use logical form as representation of meaning of procedure. Logical form would enable us to construct the system to handle more sophisticated use of the intenal meanings of the procedures. For example, a user could check off an item by paraphrasing it: “Okay, I’ve opened the camera case.” Matching semantics of the software to objects in the real world. This grounding problem is mostly solved if a person is executing the procedure – we can rely on him or her to figure out what the number 7 wrench in compartment 23 is. On the other hand, grounding objects becomes quite interesting if human & robot(s) share execution. Perhaps a robot might ask the human to ground an object by pointing to it, for example. Or, if items are already barcoded, a robot could read the barcodes to confirm that an object it had selected was the one referred to by the intelligent procedure assistant.

- 53 -

Future Challenges We now discuss some future challenges. These range from the simple to the complex. Multiple domains / procedures. The intelligent procedure assistant should be able to work with procedures from a variety of areas, as laid out in Figure 2. Challenges here include access to a wide-coverage lexicon, and Shared execution of procedures. In some cases, multiple people or robots would be using the system, necessitating further research into spoken dialogue design. Translating procedures into Prolog clauses. What if procedures are written in multiple formats? – an issue other workshop attendees may face as well. Should we translate from one format to another? Such an arrangement might be needed if the original procedures are fixed, but for newly written or revised procedures, we would prefer to have them written in a standard format (such as XML) and then write a single translator that would take those procedures and generate Prolog code such as that shown in Figure 3. Markup mechanisms. Mechanisms for marking up documentation for use by the dialogue system include getting lexical items suitable for use by the grammar out of the technical documentation. System improvement from prior performance. Finally, we would like to use data from training runs to improve the system for later training and future deployment.

Conclusion One of the main goals of the RIALIST group at Ames is to support Human Exploration and Development of Space (HEDS) by applying spoken dialogue technology to training and task performance on the ground and in space. We are moving to address this goal by developing a system aimed at taking technical documentation from the International Space Station and producing spoken dialogues geared towards astronaut training and task support. In this paper, we have introduced the scope and nature of tasks performed on board the International Space Station, described our system architecture, discussed some of the key issues in this system, and laid out future challenges. Furthermore, workshop participants will be provided a hands-on experience with our system, which converts a technical procedure written as a set of logical clauses into a spoken dialogue interaction aimed at astronaut training and support.

References 1. J. Dowding, B. A. Hockey, J. M. Gawron, and C. Culy. 2001. Practical issues in compiling typed unification grammars for speech recognition. ACL 2001. 2. J. Dowding, M. Gawron, D. Appelt, L. Cherny, R. Moore, and D. Moran. 1993. Gemini: A natural language system for spoken language understanding. ACL 1993. 3. Rudnicky, A. I., Reed, S. D., and Thayer, E. H. 1996. SpeechWear: A mobile speech system. http://www.speech.cs.cmu.edu/air/papers/speechwear.ps 4. USC. N.d. Mission Rehearsal Exercise. http://www.ict.usc.edu/misreh.html

- 54 -

Maintenance training design: Less is more Lesley Jacobs, Anja van der Hulst & Stefan van der Stigchel TNO-FEL P.O. Box 96864, 2509 JG The Hague, The Netherlands e-mail: [email protected]

Abstract Maintenance training, nowadays, primarily consists of the use of real equipment as single training device. In this paper, arguments will be presented in favour of the use of medium fidelity type trainers (e.g. CBT) over high fidelity type trainers such as the equipment itself. Such medium fidelity trainers not only cost less, they also provide an environment allowing learners to be more actively involved in troubleshooting practice, they allow more learners to be involved simultaneously and the application of such facilities consumes less of the instructors time. All in all, in our opinion ‘lesser’ training facilities, such as CBT and computer based simulations, provide better means to facilitate training for troubleshooting skills.

Keywords Maintenance training, Troubleshooting, Computer Based Training, Fidelity, Case based training

Introduction Few fields are more traditional than that of maintenance training. Where operational tasks are trained more and more with generic trainers, games, or Computer Based Training (CBT), a great majority of maintenance tasks still are trained on the ‘real thing’, the real operational equipment. Instructors have always used the real equipment for troubleshooting practice and for mechanical adjustment tasks and see no other ways of providing trainees with adequate practice. However, using real equipment is neither a practical nor a very cost effective choice for training, for several reasons; 1) The ‘real thing’ is generally extremely expensive, with usually as a consequence that too few systems are procured for training. 2) The limited training systems that are available have a capacity of just one student at a time (low bandwidth). For the rest of a class remains a passive status of on-looker. 3) On top of that, troubleshooting practice on the real systems makes high demands on instructor capacity. Each malfunction has to be inserted individually by inserting a fault or faulty component. This consumes a lot of instructor time and trainer time– and thus furthermore reduces the time available for troubleshooting practice. All these factors make that students receive very little hands-on practice. The instructor may show the effects of some malfunctions but the student gains very little experience in the actual troubleshooting. If troubleshooting practice was to be provided with CBT based facilities, an entire class could be involved in practice simultaneously. Students would be able to practice as much as they need, in their own pace and the underlying system theory about function, structure and behaviour could be directly related to that practice. But, CBT can never equal the fidelity level of the real apparatus.

- 55 -

So the question addressed in this paper is: is it really needed to provide a 100% fidelity training environment with the real apparatus? To gain an answer, in the context of the BOOT1 project we reviewed quite a bit of literature describing empirical studies after transfer of training of low and medium fidelity maintenance trainers (see: van der Hulst, van den Nieuwenhuyzen and van der Stigchel, in progress). In this paper we will document the outcomes of this literature review and come up with a number of general requirements for facilities for maintenance training. On the basis of those requirements and a more thorough analysis of the knowledge and skills involved in troubleshooting we will provide guidelines for the design of facilities for maintenance training.

100% fidelity for maintenance training, is it really needed? It certainly isn’t. In describing fidelity requirements for maintenance training, we’ll limit the discussion to troubleshooting. This maintenance task can be seen as the systematic tracing of faults in equipment. The task requires a thorough knowledge of the functioning of equipment and its separate components. It requires knowledge about the structure of the system. It requires knowledge about the incidence of certain faults and the behaviour that is related to those faults. Students have to learn to apply this knowledge by actually trying to trace simulated malfunctions. For this they need training systems and the question is, must these training systems be the full fidelity real equipment? In several studies the effects of lowering fidelity in training simulators for troubleshooting have been investigated. Authors typically distinguish between functional and physical fidelity of the simulators. Physical fidelity in maintenance trainers refers to the degree to which the components of simulated system ‘look like’ those of the real equipment. The medium/low physical fidelity representations used in the studies are generally (photo)graphical or symbolic replications of components. Functional fidelity refers to the degree to which the simulated components ‘act like’ those of the real equipment. Medium/low functional fidelity is behaviour not alike the actual behaviour. It can, for instance, be the absence of behaviour where the real system would display deviant behaviours. In fidelity experiments, researchers compared the transfer of training of training on the real equipment and that of training with medium or even low fidelity trainers. Several studies indicate that lowering physical fidelity only marginally affects student performance. Cicchinelli, Harmon, Keller and Kostenette, (1980) for instance, compared a three-dimensional medium physical/high functional fidelity trainer for electronic equipment troubleshooting with training on the actual equipment. It was observed that the performance of students trained on either of the trainers was exactly the same. A second experiment by the same authors included yet another (two dimensional) medium physical fidelity trainer again revealed no differences in learning outcome when compared to the actual device based training. The same observation was made by the several groups that measured training effectiveness of an medium physical/high functional fidelity general purpose trainer for maintenance (Spangeberg, 1974; McGuirk, Pipe, and Miller, 1975; Wright and Campbell, 1975). On the other hand, Allen, Hays and Buffardy (1986) notice a small but significant effect on one of their performance measures in an experiment on troubleshooting an electromechanical device. Lowering functional fidelity, however, seems to severely affect student performance, as was shown in an experiment conducted by Allen, Hays and Buffardi (1986). In an experiment focusing on troubleshooting an electro-mechanical device, they varied three (low, medium, high) levels of functional and physical fidelity, leading to nine experimental conditions. They conclude that functional fidelity ‘is a very potent determinant of performance’. Decreasing levels of functional fidelity were associated with substantially longer solution times. 10 experiments by Rouse and Hunt (1984) with the so called TASK/FAULT system for teaching diagnostics of electronic networks have shown that training for troubleshooting can be enhanced with low physical and medium/high functional fidelity computer based simulation. The notion of ‘mixed fidelity’: low to medium functional fidelity simulations when mixed with real equipment practice reduce the need for high physical fidelity hardware in training electronic switchboard maintenance. Diesel generator troubleshooting in a nuclear power plant has been studied and it was shown that mixed fidelity simulation can provide effective training. An experiment by Macdonald et al. (1983) with three levels of functional and physical fidelity for electronic circuits

1

BOOT is a Dutch acronym and stands for ‘decision support for the selection of facilities for education and training’.

- 56 -

and two proficiency levels showed no significant differences between experimental and control groups, only proficiency levels matter. For the training of troubleshooting one might conclude that computer based simulations can be as good as the ‘real thing’. Many of the above mentioned experiments used symbolic representations of the components (medium physical fidelity) which didn’t seem to affect the learning outcome significantly when compared to training with the real equipment. Functional fidelity is essential though. Indeed, when simulating equipment for troubleshooting training, it is essential to have a simulated component display highly realistic behaviour. In conclusion, the empirical literature reveals that a medium physical fidelity level approach can be as good as a full fidelity approach. This implies that for the training of troubleshooting essentially the use of the real operational equipment is not a necessity. One might seriously consider alternatives in the form of CBT while using symbolic representations of the components of the equipment. With the CBT’s advantages of high bandwidth, active involvement of the students and limited investment of instructor time. A ‘medium fidelity approach’ might provide more educational value for definitely less money. In the following sections we’ll elaborate on the ‘medium fidelity approach’ and come up with some concepts for the instructional design of ‘medium fidelity’ facilities for maintenance training. To get to indications for the instructional design (ISD) of training equipment for troubleshooting we’ll firstly outline the nature of maintenance task, and thus the knowledge and skills to be targeted.

Cognitive aspects in maintenance tasks Typical maintenance tasks include: operational skills to use the equipment, testing the operational status of the equipment, access a certain component in the equipment, disassembly, repair of components and troubleshooting in case of malfunctioning equipment. The latter is probably the most challenging maintenance task, also from a training point of view. As indicated before we will limit the discussion to troubleshooting. Troubleshooting requires a thorough knowledge on the structure and the functioning of the equipment, system behaviour and occurrence of faults (including fault cause), fault tracking, tracing and isolation. Troubleshooting mostly takes place during corrective maintenance, this means after the occurrence of a malfunction. To a smaller extent troubleshooting occurs also during preventive maintenance tasks and reconfiguration tasks, for instance when a certain component gets replaced by a new component type. In accordance with Halff (1990b) we would like to make a division in three cognitive components that are used during troubleshooting tasks. The three components are mental models, procedures and fault isolation or troubleshooting skills. Of course, the latter should be the most important component to convey in a training method for troubleshooting. However, mental models form the basis for reasoning about the structure and functioning of the equipment, especially during troubleshooting and different kind of procedures also play an important role in troubleshooting. So, all three types of cognitive components enhancing maintenance skills should be addressed in the design of a training method for troubleshooting. I. Mental models In short, troubleshooting is regarded as a complex cognitive skill. Mental models seem to be particularly important for the performance of complex cognitive skills that encompass reasoning about processes or devices such as in troubleshooting or fault isolation (van Merriënboer, 1997). Therefore a special focus in training methods for troubleshooting should lie on the teaching of mental models. Mental models of equipment refer to cognitive structures used to reason about the equipment being maintained (Halff, 1990a). In terms of equipment a mental model should include the structure of the equipment, the function and the physical appearance of the equipment. The strongest advantage of mental models lies in the fact that they provide a representation of the equipment in the working memory of a maintainer that can be "run" to understand the equipment, solve problems or predict events. In the case of troubleshooting a mental model forms the basis for hypothesis testing during fault isolation. II. Procedures As second cognitive component procedures were mentioned. Procedures in maintenance vary from standard procedures for checking or (dis)assembly of components as listed in job aids or checklists, diagnostic procedures, to emergency procedures that must be performed under high stress conditions. Essentially, - 57 -

procedures are series of actions to be carried out in a predefined order. Sometimes a procedure may have certain choice points where a particular sequence of actions is chosen. Despite the many differences in procedures within and across maintenance domains, two general thoughts come to mind when thinking on maintenance and maintenance training, how to handle procedures and how to handle the huge amount of (different) procedures. It is important to realise that, regardless of the differences, procedures influence the existing knowledge structures and therefore the mental model of the equipment. III. Troubleshooting skills The third and most important cognitive component consists of troubleshooting skills. In literature, there is a fairly coherent picture of skilled troubleshooting (Halff, 1989). When a fault mode in one or more of the components of the equipment occurs, the troubleshooting task is to decide upon a sequence of actions that isolate and repair the malfunctioning components of the equipment. The actions in the solution sequence can be defined by the following possibilities: observe the outcomes of some components, observe the state of some components, manipulate the state of some components and repair or replace certain components (Halff, 1990b).

Instructional design, a medium fidelity approach The question now rises to what extent a medium fidelity type of training such as CBT could provide a good learning environment for troubleshooting. I. Mental models A method suggested to help learners develop good mental models is to provide various conceptual models of the equipment during training (Hagemann, Mayer & Nenninger (1998) in Allessi & Trollip, 2001). Such models can be represented in the form of component or block diagrams, goal-plan models, causal models, concept maps, hierarchies, etc. An essential aspect of the development of mental models is to guarantee the consistency of the conceptual material presented to the students in training and the technical documentation to be used during operational practice. Lack of consistency may well lead to confusion and consequently towards inadequate mental models. Therefore it is essential to reuse as much of the conceptual information as available in the documentation of the equipment. Many elements such as component and/or block diagrams shall be readily available, other conceptual material may need to be derived from the manual, hereby trying to maintain a direct link to the original documentation. A second essential aspect of the development of mental models is to provide dedicated support to help understand the relation between the various models. One may use a symbolic representation to reason about possible faulty components, but, once this component is found, a realistic graphical representation is needed to actually locate the faulty component. Hence training should help convey an understanding of the relations between the various conceptual models. With this we have come to the discussion of the media nature to be used for such training. Real equipment cannot convey any conceptual representations and as such is not an optimal mean to build a mental model. Hence, such conceptual representations are usually added on paper. Paper based methods (usually) provide conceptual models by means of block diagrams, equipment specifications and static graphics. CBT and computer based simulations not only provide better means for displaying conceptual models e.g. by providing multidimensional component overviews, dynamic graphic displays, video and schematic representations, but also for the practice of causal reasoning, exhibition of complex functions of components and display of normal and faulted behaviour of components. On top of that multimedia technology allows easy linking of the various elements of conceptual models (see the previous guideline). This suggests that multimedia technology, with the excellent capacity for animation, diagrams and the like, has great potential for supporting the development of mental models (Allessi & Trollip, 2001). II Procedures The most common method to train procedures is by applying a drill and practice method. That is, parts of the procedure are demonstrated or described to the student and subsequently, students are supposed to do it

- 58 -

themselves and are required to practice that part until mastery. Again, the parts are integrated until the full procedure is mastered. Practice for (dis)assembly procedures is generally done on the real equipment1. Some test or diagnostic procedures, however, can well be taught by means of medium fidelity simulations . That is, interactive symbolic representations of the equipment with an underlying simulation model that is able to display both correct and faulty behaviour of the components concerned. Such a simulation has as advantage that it may be augmented with dedicated facilities to help analyse students’ performance. An example of such facility is the CAI system (Kuiper, 1995) that incorporates a norm model (the correct procedure) that is used to analyse student performance. With such a facility it is possible to provide dedicated feedback and, if necessary, remediation. These instructional measures can hardly be provided while using the real equipment. A note should be made in regard to procedures with strong sensory-motor aspects. For these types of skills, sufficient hands-on experience with the real equipment remains the only way to obtain the knowledge and skill required. When using (interactive) representations of the equipment, again one should strive for consistency with those used to establish mental models. In this manner one can easily relate the merely theory oriented instructional material necessary to build mental models with the practice needed to master procedures. As with the construction of mental models, in training for procedures it is again essential to guarantee the consistency of the procedures presented to the students and those described in the technical documentation. Most technical documentation incorporates test and diagnostic procedures. When such documented procedures are embedded in computer based simulation, it should be possible to visualise the relation between a step in a documented procedure and an action on the simulation. One might, for instance, think of highlighting a step in the documented procedure on a student action on the simulation or vice versa. III Troubleshooting Troubleshooting skills are considered as a mixture of highly knowledge based methods and weak context independent skills (Halff, 1990b). Troubleshooting skills are usually taught trough several case based troubleshooting exercises, and practice forms probably one of the most effective methods for troubleshooting training. One of the most challenging parts in training methods for troubleshooting is probably the sequencing of problems or cases to be used in the training environment. Especially for troubleshooting skills density of practice is important. Exactly here several advantages of CBT and computer simulations over training on the real equipment come forward. By using CBT, the bandwidth is much broader, more students can work actively with the CBT instead of watching passively how a teacher or fellow student operates on the real equipment. In case of safety procedures CBT offers safe mean to obtain some hands-on experience before going into real practice. Furthermore, the real task environment often serves as training on the job facility for the training of troubleshooting skills, and these skills will only be obtained depending on the faults and malfunctioning of the equipment in the present work environment. CBT and computer simulations offer the possibilities to train a much broader scope of troubleshooting exercises. Finally, a case based training method seems very well suitable for the training of troubleshooting, since troubleshooting is a special form of skilled problem behaviour. CBT could even contain a whole library of cases where either the instructor or a learner could select a range of troubleshooting cases, for example from simple to complex, troubleshooting cases on certain components, rare or regular troubleshooting cases, etc. Arrived at the step of case based troubleshooting CBT, one step further could be an adaptive case based training environment for troubleshooting, where the training environments sets up a sequence of troubleshooting cases, adapted to the educational and contextual level of the learner. Here too we plea for consistency. CBT implies medium level fidelity. The student interacts on a simulation underlying a graphical or symbolical representation of the operational equipment. Ideally even this simulation should be consistent with other instructional material in its presentation of the equipment to the student. The symbolic representations used in building a mental model, the interactive one for procedure training as well as the equipment representation within the simulation should be as consistent and similar as possible.

1

Although examples are available of fruitful assembly practice on 3D models embedded in CBT.

- 59 -

Integration of mental models, procedures and troubleshooting skills The final question now rises how to integrate the training of mental models, procedures and troubleshooting skills into a coherent set of training methods. A first obvious step would be the creation of proper mental models. For this step theory, conceptual maps and the main or basic procedures of a system should be presented to the learner. Also a basic introduction to the real system can be included during this phase of training. Note should be taken on the fact that whenever the real system is used in training the system must be handled as much as possible under the student’s control of operation. A second step would include the use of CBT generated cases. Depending on the maintenance domain, it is possible to create a curriculum of troubleshooting cases, varying from simple to complex, component specific or otherwise useful case categories for the specific domain. By creating ‘learning modules’ around a certain topic, it will be possible to create small loops from theory, practise and testing of knowledge of skill. In this type of training environment, the real equipment will only be used in those situations when it is necessary from an instructional point of view, instead of using the equipment as single and main training device.

Potentials Of course the mere use of medium fidelity simulations or CBT can never replace the real hands on experience, especially for sensory-motor skills. However, training methods for troubleshooting seem to be very well suited for medium fidelity type trainers such as CBT and computer simulation. Arguments in favour of this statement, include: •

the effectiveness studies mentioned in the first part of the paper, i.e. medium physical fidelity does not mean less transfer of training when compared to high fidelity trainers;

•

cost effectiveness;

•

high bandwidth;

•

from passive to active learning;

•

less time demands on instructors (shift from instruction to coaching);

•

possible embedding of technical documentation;

•

presentation advantages for conceptual modelling;

•

training of procedures in a safe environment;

•

possibility to train all troubleshooting cases or a larger set of cases then possible in real life.

A final argument includes the huge amounts of technical documentation available in the maintenance field. The technical documentation of the equipment forms almost always the direct basis for the input and creation of the training materials. Especially in the maintenance field there is a strong trend from paper based documentation to (interactive) electronic technical manuals. It goes almost without saying that all the advantages mentioned in this paper for the use of CBT in troubleshooting training hold even stronger when the source of technical documentation is not paper, but electronically available source material. It certainly shall be easier to keep the CBT element derived from the documentation up-to –date. Promising work on the integration of (electronic) manuals and training in the field of maintenance includes several research projects, such as the European IMAT project (http://imat.swi.psy.uva.nl/) and the work in the field of classification standards for electronic technical manuals (http://navycals.dt.navy.mil/ietm/ietm.html). The two highest classes of interactive electronic manuals, IETM class 4 and 5, provide several possibilities to integrate and link the electronic documentation directly to CBT. Thus enhancing many of the advantages mentioned in this paper for medium fidelity trainers over high fidelity as training method for troubleshooting.

- 60 -

Conclusions Based on the literature review, our work1 in the field of maintenance training and troubleshooting for the Dutch Navy and Royal Netherlands Airforce we come to the conclusion that medium fidelity based training methods can provide a good educational and cost effective training method for troubleshooting. Many experiments on high versus medium fidelity trainers were conducted during the eighties and one can conclude that this kind of experiments certainly were not an important research topic during the nineties. However, the conclusions in favour of medium fidelity trainers for troubleshooting derived from these studies would probably hold even stronger when conducted today, since the technical possibilities of CBT have evolved enormously since then.

References Allen, J.A., Hays, R.T. & and Buffardi, L.C. (1986). Maintenance Training Simulation Fidelity and individual Differences in Transfer of Training. Human Factors, 28(5), 497-509. Allessi, S.M. and Trollip, S.R. (2001). Multimedia for learning (3rd ed.). Allyn & Bacon, Massachusetts. Cicchinelli, L.F., Harmon, R.A., Keller, R. and Kostenette, J. (1980) Relative cost and training effectiveness of the three-dimensional simulator and actual equipment. (TECH. Rep. AFHRL-TR-80-24). Brooks Airforce Base, Tex. Air force Human resources Laboratory. Halff, H.M. (1989). Prospects for automating instructional design. Arlington, VA: Halff Resources, Inc. Halff, H. M. (1990a). Teaching troubleshooting procedures. Arlington, VA: Halff Resources, Inc. Halff, H. M. (1990b). Automating maintenance training. Arlington, VA: Halff Resources, Inc. IETMs: http://navycals.dt.navy.mil/ietm/ietm.html IMAT: http://imat.swi.psy.uva.nl/ Kuiper, H. (1995). An instructional support system for training simulators. PhD thesis. (ISBN 90-9008918-7) McGuirk, R.D., Pieper, W.J. and Miller, G.G. (1975). Operational Tryout of a General Purpose Simulator (Tech. Rep. AFHRL-TR-75-13). Brooks Air Force Base, TX.: Air Force Human Resources Laboratory. McDonald, L.B., Waldrop, G.P., & White, V.T. (1983). Analysis of fidelity requirements for electronic equipment maintenance. (technical report NAVTREAQUIPCEN 81-C-0065-1) Orlando, FL: Naval training equipment center. Merriënboer, J.J.G. van, (1997). Training complex cognitive skills: a four-component instructional design model for technical training. Englewood Cilffs, New Jersey: Educational Technology Publications, Inc. Rouse, W. B. and Hunt, R. M. (1984). Human problem solving in fault diagnosis tasks. In Rouse, W. B., editor (1984). Advances in Man-Machine Systems Research Vol. I. Greenwich, CT: JAI Press Inc., pp 195-222. Spangeberg, R.W. (1974). Tryout of a General Purpose Simulator in the Air National Guard Training Environment (Tech. Rep. AFHRL-TR-74-92) Brooks Air Force Base, TX: Air Force Human Resources Laboratory. van der Hulst, A., van den Nieuwenhuyzen, N. & van der Stigchel, S. (in progress) BOOT kennisbasis; Richtlijn voor opbouw, validatie en beheer. Den Haag TNO-FEL rapport. Wright, J. and Campbell, J. (1975). Evaluation of the EC-II Programmable Maintenance Simulator in T-2COrganizational Maintenance Training (Rep. No. NADC=75083-40). Washington, D.C.: Naval Air Systems Command.

1

TNO participates in consortia (e.g. the European Esprit project IMAT) and conducts research projects on the re-use and integration of technical documentation for maintenance training, medium versus high fidelity simulation research and future possibilities towards the integration of IETM’s and CBT or e-learning.

- 61 -

- 62 -

Personalized eLearning and eCoaching in WINDS Marcus Specht, Milos Kravcik, Roland Klemke, Leonid Pesin, Rüdiger Hüttenhain Fraunhofer Institute for Applied Information Technology FIT Group for Information in Context (ICON) D-53754 Sankt Augustin, Germany {specht, kravcik, klemke, pesin, huettenhain}@fit.fraunhofer.de

ABSTRACT This paper introduces one approach to e-Learning describing the Adaptive Learning Environment (ALE) for web-based authoring and learning. The system provides a new methodological approach to design education on the web. ALE will be used to build a large knowledge base supporting Architecture and Civil Engineering Design Courses and to experiment a comprehensive Virtual University of Architecture and Engineering Design in the project WINDS. Here we present the basic concepts of the system, outline the system architecture and show the possibilities for adaptive learning and instruction in two pedagogical approaches called expository and exploratory learning. The system combines classical structuring of learning materials based on reusable and sharable learning objects with and indexing approach for browsing the learning materials based on a alternative structure we call the course index. The course index is a collection of important concepts for a course subject or a whole domain.

Keywords Web based learning, educational hypermedia, interactive learning environment, adaptive systems, e-Learning

Introduction Web-based Intelligent Design and Tutoring System (WINDS) [1] is an ongoing European project1 with the objective to implement a learning environment integrating an intelligent tutoring system, a computer instruction management system and a set of cooperative tools. We have developed the current version of the system and 21 European university partners prepare more than 20 on-line courses that should be used by students from spring 2002. The main objective of WINDS is to contribute to the reorganization of the pedagogical, cultural, and functional aspects of design education at the university level. The traditional approach to design teaching shows some frequent problems that increase learning time and reduce knowledge retention. WINDS will provide support instruments for a pedagogically more adequate approach to design teaching. As far as contents are concerned, the system will focus on the problem of cultural integration. The main benefits of WINDS courses are seen as •

general advantages of e-learning, such as worldwide access to learning material and contact to teachers, independent personal working times, well- tested material reused

•

open European University for Architecture, Construction Management, and Environmental Engineering

1

Web based intelligent design system (WINDS) is EU funded project in the 5th framework of the IST programme # IST1999-10253.

- 63 -

•

focus on teaching professional skills (not only textbook knowledge) by means of specialized tools and the integration of best practices

•

support for cooperation among students (peers) as well as between students and teachers

•

special support for course authors, such as semi- automatic index creation, and the facilitation of creating learning objects with a corporate WINDS style from templates

From a pedagogical point of view the WINDS platform mainly supports: •

the easy creation of flexible learning objects, which can be reused in a variety of learning contexts and different curricula

•

the flexible and personalized creation of goal based learning materials and courseware comparable to [2]

•

the semiautomatic connection of multiple learning activities concerning design artefacts and dynamic learning objects (individual learning, cooperative learning)

•

the connection of a closed knowledge corpus provided by the teacher, with the open and dynamic learning resources in the WWW, and a living repository of knowledge from the community of learners.

•

the integration of workplace software [3] and meta-cognitive learning tools [4]

•

various relations between learning objects

•

a highly detailed learner model and history supporting personalized coaching and tutoring activities

For reaching those goals we have taken into account several of the existing standards and semi standards on learning objects, learner models and their storage and exchange for example [5, 6]. We can build on our experience with development of adaptive hypermedia environments like ISIS-Tutor [7], ELM-ART [8] and ACE [2]. ALE produces individualized courseware for the students depending on their current state of knowledge, their preferences and learning styles. The author specifies the metadata according to which such adaptation methods like additional explanation, prerequisite explanation, comparative explanation, explanation variants or sorting can be implemented, taking into account the user model. In this way, the system can adapt the sequence of learning objects according to the chosen learning strategy. For instance, a concrete example can precede or complement an abstract statement if the student needs it. To reduce the cognitive overload of the learners various annotation techniques are implemented. As a difference to earlier adaptive learning management systems ALE also integrates the possibilities of using methods of user modeling and data mining for building expertise networks and supporting human resources management in knowledge management for organizations. High quality expert finding and colearner finding algorithms can build on detailed user and learner models from ALE. This paper describes an integrated authoring and learning environment implemented in the WINDS project for a virtual university for teaching and learning architecture, construction management, and environmental engineering. First we introduce the fundamental concepts and learning objects used in the system. Then the main ALE components and some examples of interaction and adaptation are described. Finally we summarize the paper and outline future perspectives.

Learning Objects Considering the amount of information that will be collected in the WINDS university storing and collecting the material in a reusable way is essential. Based on the analysis of several approaches to formulation of reusable learning objects and sharable content objects (for example [9]) we defined the WINDS learning objects to fulfil several purposes: •

to be the basis for a consistent content generation by the WINDS system

•

to enable for a maximum of reusability of learning objects

•

to allow for a maximum of flexibility for the dynamic generation of online lessons and courses

Besides being highly reusable and compliant to ongoing standard definitions like SCORM 1.2, IMS, and LOM the learning objects had to

- 64 -

•

reflect the variety of pedagogical approaches applied by the teachers and authors in the WINDS university

•

give the possibility of applying a wide spectrum of methods of adaptive hypermedia like adaptive navigation and adaptive presentation

The WINDS content management and authoring tool supports authors in building independent learning objects and structuring them in a default course hierarchy. Nevertheless the underlying knowledge representation and storage allows for reusing and handling learning objects independently and combining them in new dynamically generated courses later on. This assures that the system is able to produce individualized courseware for students depending on their current state of knowledge, their preferences and learning styles. Based on the current definition of learning objects the system allows for generating adaptive educational hypermedia courses with personalized curriculum sequences and personalized content selection on a level of units and pages, and track all interactions of learners and teachers with those learning objects. The basic building blocks in the WINDS system should provide the basis for consistent content generation with maximum flexibility for dynamic generation of online courses. This means the key feature required from these building blocks is their reusability. Following these aims four basic types of learning objects are defined in WINDS: •

Course Units are top-level elements that have only subunits but no super-units.

•

Learning Units provide the means for course structuring.

•

Learning Elements are the basic chunks of information with templates for different pedagogical purposes. Specifically we use Paragraphs - the contents of the course, Exercises - the practice tasks, Tests - online assessment. The authors compose each learning element as a sequence of atomic parts that are reusable basic data components, e.g. text, image, audio, video.

•

Index Terms are the fundamental terms of a common glossary for a course.

With course units, learning units, and learning elements course authors can create an arbitrary structured hierarchy of learning objects which also represent the default learning path for the students. The course authors can also specify relations between learning objects based on a subset of the Dublin core recommendations [10]. These include prerequisite relations, part_of, and related_to relations between course units, learning units, and learning elements. All these learning objects can be associated with a subset of the Learning Object Metadata defined in the IEEE LOM specification [5], which is currently reworked to be fully compliant with the SCORM 1.2 specification.

- 65 -

Figure 11: Comparison paragraph in edit mode Learning elements as the leafs of the course hierarchy are built of single content blocks. A content block can either be a text segment (ASCII, HTML, other), a URL, or a file that is uploaded to the server. By uploading files authors can easily integrate existing materials like animations, video, images, audio and other binary formats into learning elements. Figure 1 shows a learning element in editing mode with three content blocks: two pictures as examples and one text content block for describing differences and similarities between the two images. Figure 2 shows the same paragraph in view mode like the student will see it. Every content block has a pedagogical role what gives additional possibilities for reusing and even recompiling dynamic learning elements from content blocks of different pages. By taking into account the pedagogical role of each content block also pedagogical patterns for learning elements can be identified. As shown in figure 1 and figure 2 the author created a page that compares two examples and explains something in an additional content block that has the role of fact.

- 66 -

Figure 12: Comparison paragraph in view mode Additionally all learning objects in a course unit will be linked dynamically by the underlying index defined by the course authors. The index reflects a common understanding of the domain and the subject matter. The index terms are described by a term name, a description, synonyms, and relations to other terms. Figure 3.1 shows an editing form for an index term with several relations to other terms. The available relations are definable by an administrator of the system and the indices are usable by all authors for an easy import of pre-existing indices from other courses and adaptations to individual definitions. When the author publishes a learning object or previews it in view with terms mode the system parses all materials in this learning objects and stores occurrences of index terms in the database. This allows giving the learner different approaches for working through the learning materials.

Figure 13: Editing an index term By combining different methods for structuring content in learning objects and by relations to index terms the learning management system basically the structure of the WINDS learning objects supports two main pedagogical approaches: •

Expository learning, where the learners are guided though a curriculum and are supported with coaching components for self directed learning.

•

Exploratory learning, where the learners do not follow the default learning path or some adapted learning path through a curriculum but explore an alternative structure that is somehow orthogonal to the learning unit structure.

- 67 -

Figure 3.2: Browsing the course with index terms Additionally to learning objects the teachers define and to the index terms that can be developed by the course authors cooperatively each course can have a set of external document and web resources connected. This allows the authors to integrate online materials into their courses and though give a reference to further materials or online examples. Like the learning objects of a course all documents are indexed with the index terms of the course. In the course player the students can therefore always browse other learning objects and external documents where the current index terms occur.

WINDS Modules A user of the WINDS system can have one or more of the following three roles assigned: •

Student

•

Author (teacher)

•

Administrator

Each role has access to one special environment – a student to the learning environment, an author to the authoring environment and an administrator to the administration environment. Additionally each user can access Communities, Tools, News and Help. Collaboration The learners are not only passive receivers of knowledge but can themselves contribute to the corpus writing private or public annotations, providing feedback to the author, or discussing course related issues with colleagues and teachers in a discussion forum. In this way students can answer some questions themselves and can cultivate their ability to formulate problems and give argumentations.

- 68 -

The main tools provided by the system for collaboration purposes are: •

BSCW folders [11] as a common place to store and exchange documents, support common projects and submit homework.

•

Discussion forums created by authors for each course.

•

Chat room for synchronous communication.

An important feature of the system is the awareness support: the user can see which other students have registered for a certain course and even who is currently working in it. Course Assessment and Homework Students can be assessed when they solve tests and exercises. Tests are evaluated immediately and automatically. Various types of tests like single- and multiple-choice, picture hotspots, gap fillings and matching pairs are provided. More complicated exercises (e.g. design tasks or Bayesian network exercises) are carried out with special external tools, which should be installed locally on the student’s computer or downloaded as an applet. The result should be submitted as homework to the student’s folder.

Authoring Environment The main components of the authoring interface [13, 14] are the navigation tree and the content frame. The navigation tree gives an overview of all the courses authored and additionally all courses, learning elements, content blocks, index terms, documents and users in the system can be searched. A course has three main folders: •

The Units folder includes all the learning units and learning elements forming the structure and content of the course. The contents can be nested in arbitrary depth and can contain arbitrary learning elements. Authors can provide only text based readings of only exercise or different tests and can link and combine them in learning units.

•

The Index is at the beginning a universal department index prepared in advance and chosen by the author who can enhance it. Administrators can create master indices that can be imported into single courses. Furthermore the systems allows teachers to export their indices to a standardized XML format that enables the system to directly integrate external documents or application in which theses index terms are used as annotation tags. As one example in WINDS we use 3D VRML worlds with integrated references to index terms which allows the students to browse from a specified point in a VRML world to index terms and to the related learning objects.

•

The Documents are supplementary sources of information relevant to the course. Specialised portals with up-to-date materials can be referenced here. The possibility to view external documents enables the system to include highly dynamic information into the courseware. In the further development of the system this will allow students and teachers to monitor important sources of information on the WWW and integrate it directly with a standardized curriculum.

The Content Catalog contains all the learning elements created by the author in the system. These materials can be reused.

- 69 -

Figure 4: Editing a learning unit in the WINDS authoring environment

In the content frame the following forms are provided and are accessible via different tabs: •

Edit to change the structure and content of the course. The learning objects can be added, moved, copied, disconnected or deleted within the course structure. The author can publish the current learning unit and copy it to a buffer as well as edit the learning elements.

•

View to display the learning objects in the same way like students will see them.

•

Terms (just for learning elements, not for learning units) to show the content of learning elements or (external) documents with index term entries highlighted and displaying the corresponding term descriptions on demand.

•

Relations to specify prerequisite units and alternative versions in other languages other relations can be added.

•

Meta Data to provide information related to Learning Object Metadata [5] (general, life cycle, technical, educational, rights).

The course index can also be used to view external documents and learning elements with index terms highlighted and hyperlinked to the term’s explanation. Therefore by browsing from learning elements to index terms and then to external documents, back to index terms and so on, we expect the students to have individualized learning paths based on their interests. Nevertheless the system can keep track of all inspected index terms, learning objects, and external documents to follow the students learning process. Cooperative development of a shared understanding When creating an online course with the authoring tools in ALE the system especially supports the development of a shared understanding of the index terms of the current course and of other courses. By using the content catalogue the authors can explore and search the complete index terms database. This allows for finding all related index term definitions and explore their curricular context and also to contact and discuss relevant definitions with other authors. Additionally by previewing external documents with the index terms of a course highlighted similarities and differences can be highlighted by the course authors. Beside searching for related index terms the system supports the reuse and search for courses, learning elements, content blocks, and experts with a wide variety of criteria. This allows teachers to effectively search the content database for all kinds of objects and contact points to create their online courses.

- 70 -

Adaptive Learning Environment The WINDS learning environment is designed to support various learning strategies. Paragraphs contain materials for expository (explanatory) education. Discovery learning is encouraged not only by hyperlinks but also by index terms and their interconnection with learning objects and external documents. Collaboration facilities promote constructivistic learning approaches. The presented content can be adapted to the current knowledge level of the student on the base of the User Model and the structure of learning objects. The user model contains for each user all his or her educational events, called user episodes. These episodes include events of different types: •

all the user’s actions with the learning objects

•

all evaluations of completed tests made by the system

•

teacher’s reviews of homework submitted by the student

•

self assessment and self estimation of learners

On the base of all these episodes the system can infer conclusions about the user’s knowledge and goals. These inferred episodes are stored in the user model as well.

Figure 14: Viewing a learning unit in the WINDS learning environment

Thus the user model always reflects the current state of the user’s progress. The information is available both for the teacher to control the student’s study process and for the system to adapt the course presentation and navigation for the student. When a user logs in he or she enters Course Registration with the list of all published courses where the registered courses are distinguished and the ratio of the seen as well as tested study materials are indicated. These bar charts are displayed also when the student works on the course to illustrate the study progress. The student can display also more detailed Course Overview showing the structure of the whole course with the current states of all learning objects - interaction history, tested knowledge, and user’s readiness. This overview is also permanently available for the student. From the Course Registration page the student can also access a list of all the seen learning elements together with the date of the last access for each of them. The learning environment in WINDS consists of the following parts:

- 71 -

Learning Object Display The main part of the screen is occupied by the content of the current learning object. Its complete path in the course tree is displayed together with navigation buttons to support orientation and navigation in the course. A learning element can be displayed with or without emphasized index terms hyper-linked with their explanations. Content blocks in learning elements may be represented by texts, images, hyperlinks, multimedia, or special formats (e.g. Word, PDF, PPT). Some of these content blocks can be displayed as icons activating additional window possibly with a note or more detailed information. Annotated Course Structure In another frame of the WINDS learning environment the course structure overview is displayed. It supports student’s orientation in the course and helps to choose suitable learning objects for the next study providing annotations for the learning objects. Learning Object States The states of the learning objects in WINDS can be considered from several points of view or in several dimensions. The system distinguishes three of them: interaction history, tested knowledge, and user’s readiness. Additionally some other characteristics can be defined depending on the followed objectives, e.g. current task or context. Interaction history and tested knowledge can be expressed by quantitative information defining the extent of the seen learning object or the tested knowledge, user’s readiness is nominal information and depends on the prerequisites specified by the author and on the interaction history of the user. Annotation Techniques To represent various states or dimensions of an information space various visual emphasis techniques can be used. Some studies (e. g. [12]) of visual perception recommend the following visual attributes for different kind of information: •

•

Nominal information: o Texture o Color (Hue) o Position Quantitative information: o Length o Position

Currently we have a preliminary annotation solution using icon and text annotations. Another possible solution in the WINDS system is to represent the various states of a learning object in different parts of one icon by different colors. Other applicable annotation techniques include hiding and presentation without hyperlinks. These can be also student’s options. Course Index With each learning object corresponding index terms are displayed. Index terms provide means to interrelate heterogeneous course contents and to find individualised paths through the learning materials. The WINDS course index component maintains the index terms together with their respective descriptions, synonyms, different types of relations (e.g. is_a, part_of, related_to) between terms, their occurrences in the course materials as well as in external documents. The student can access all this information about a specific term by choosing it in the Index frame. The index component can retrieve and highlight occurrences of index terms within the course materials as well as within registered external documents (such as web sites). External Documents External documents relevant to the course domain explain in more detail some specific issues or provide up-todate information like specialized portals do. Such external documents serve also as resources for homework and - 72 -

projects. These materials can go into more details than the course or give alternative views of the domain. In the Documents frame the student can choose an external document related to the current learning object and view this document either with or without emphasized index terms. Feedback and Recommendation Area In the feedback and recommendation area the users can see what their progress is on the current topic and how much of the current learning unit they already have successfully worked on. Additionally several recommendation strategies will be implemented in the next months which supports learners in recommendation on course navigation (prerequisite warnings, next step recommendations), learning style support (learning material preferences, learning activity selections) and cooperation initialisation (co-learner finding, expert finding and tutor support).

Coaching Facilities In ALE a pedagogical agent was implemented to give individualized recommendations to students dependent from their knowledge, their interests and their media preferences [15]. The implicitly given teaching goals were promoting and retaining learner motivation and evoking students’ interest even in lessons they otherwise might have avoided. Beside recommendations of learning material to study the recommendations of the pedagogical agent where extended to propose cooperative activities and contact to other students that are willing to support others. Table 1 shows some type of recommendations given by the agent:

Table 1: Recommendations given by a pedagogical agent in ALE Student request

Agent’s selection and teaching strategy

Anything interesting here?

Find a learning material of the current unit, corresponding with student’s preferences. Following this proposal, the student may acquire the current unit by means of a type of learning material that he or she claimed to be especially convenient.

Anything else?

Find a learning material of the current unit, to complete knowledge about this unit. This proposal draws the student’s attention to a learning material of the current unit that has not yet been seen.

What is related?

Find a unit related to the current unit and corresponding with student’s preferences. This proposition may attract the student’s interest because it shows the idea of the current concept in the light of a global domain the student has claimed to be especially interested in.

Show me something new!

Find a completely new, unrelated, unvisited chapter. A student, who is bored or annoyed by the section he or she is just working at, may like to follow this link.

Contact advanced student!

Find a student that declared that he/she is willing to support others and has excellent knowledge in the current unit or lesson.

Discuss students!

Find students that are willing to cooperate and those have a similar knowledge of the current unit by comparing the overlay model of the current lesson.

with

other

Another feature attached to the agent was the visualization of the learner model. Learners could access a bar chart display of the current knowledge model in the system. From the bar chart they could directly jump to tests about a section to proof that they have better knowledge than assumed by the system. In the experiment done with ADI and the user model visualization the usage of the agent components were voluntary and the learners rarely used this feature. Therefore additional evaluations have to be done. The overall results showed that students following the learning material recommendations of the agent gained a better understanding, had more fun, and worked more intense [15].

- 73 -

Cooperative knowledge collection and adaptive co-learner finding With the integration of student workspaces into the ALE framework new courses can only consist of a given curriculum structure prepared by a teacher while the contents and learning materials are completely collected by the students. This allows teachers to implement basic courseware that is integrated with the growing knowledge of the learner community and develops towards a living repository of knowledge which includes frequently asked questions, best practice solutions, and open space courseware. Students can ask the assistant for online co-learners and topic experts. Furthermore with every presentation of a learning unit the currently available students that are willing to cooperate are shown to the student. The selection of appropriate co-learners is based on a comparison of the student’s knowledge model and the other students’ knowledge model. Students that are willing to cooperate with others can get a contact point to very similar students or to very different students in sense of there curriculum expertise. This can lead to very heterogeneous learner groups that consist of topic experts in selected areas or to very homogenous learner groups that consist of experts on the same topic. The expertise of a student on a certain topic is measured with four main criteria: The number of wrong test answers when working on knowledge tests, the time to answer sufficiently enough knowledge tests, the collection of additional learning material in student workspaces that has high request rates by many other students, and the overall score in the curriculum. The learner groups can work in discussion forums, synchronous moderated chat sessions or communicate via integrated email.

Conclusions and Further Research In this paper we gave an overview of the current state of the ALE system. Its authoring platform supports reusability of the learning objects. The learning environment presents them as adaptive educational hypermedia courseware. One of the primary objectives of the WINDS project is to make the authoring process relatively simple and to support sharing of resources. Currently several tens of authors test the usability of the system and with their comments and suggestions help us to make the system more advanced and reliable. The potential of such a system is high. The created learning objects can be delivered in a variety of ways, from classical web-based training to combinations of classroom events and online seminars, or even as personalized books. Beside the possibility for cross media publishing and the flexible combination of learning objects in individualized curricula the indexing system and the connection of the learning objects allows for personalized coaching of students. In the next few months we will especially extend the coaching and functionality for pedagogical agents supporting the students while learning. Different tutorial and navigation support strategies will be implemented that allow the teacher to adapt the adaptive method to the target group of his/her course and his/her pedagogical framework. For enabling open exploration of contents the system will mainly support on a level of index terms that where already seen or explored by a learner and give recommendation on the next steps. In more strictly focused training settings the systems will keep the student close to the default path of a curriculum based on the teacher’s specifications.

References [1] WINDS project: http://www.winds-university.org/ [2] Specht, M.; Oppermann, R.: ACE, Adaptive courseware environment, New Review of Hypermedia and Multimedia, 1998/4, p. 141-161. [3] Nemetschek AG, The O. P. E. N. platform, Technical white paper. 1999, Nemetschek AG: Munich. [4] Giretti, A. ASA: A Conceptual Design- Support System. Engineering Applications Of Artificial Intelligence, 1997. 10(1): p. 99- 111 [5] LTSC IEEE: Draft Standard for Learning Object Metadata, IEEE P1484.12/ D6.1, 18 April 2001. [6] Aviation Industry CBT Committee: AICC Guidelines http://www.aicc.org/pages/down-docs-index.htm, AICC.

- 74 -

&

Recommendations

for

CMI,

[7] Brusilovsky P., Pesin L. "An intelligent learning environment for CDS/ISIS users". Proceedings of the Interdisciplinary Workshop on Complex Learning in Computer Environments (CLCE 94). University of Joensuu (Finland), May 1994; p.29. http://cs.joensuu.fi/~mtuki/www_clce.270296/Brusilov.html [8] ELM-ART project. Online at http://www.psychologie.uni-trier.de:8000/projects/ELM/elmart.html [9] Cisco Systems, Inc. Reusable Learning Object Strategy, Version 3.1. Online at http://www.cisco.com/warp/public/- 10/wwtraining/elearning/learn/whitepaper_docs/rlo_strategy_v3-1.pdf, 22 April 2000. [10] Dublin Core Metadata Initiative (DCMI) Documents, http://dublincore.org/documents/, DCMI. [11] Bentley, R., Appelt, W., Busbach, U., Hinrichs, E., Kerr, D., Sikkel, K., Trevor, J. and Woetzel, G. (1997), "Basic support for cooperative work on the World Wide Web", Int. J. Human-Computer Studies, pp. 827846. [12] Bertin, J. Semiologie Graphique. Paris: Editions de l’Ecole des Hautes Etudes en Sciences (Les reimpressions). 1967/1999. [13] M. Specht, M. Kravcik, L. Pesin, R. Klemke (2001): Authoring Adaptive Educational Hypermedia in WINDS. Proceedings of ABIS2001, Dortmund, Germany, October 8-10, 2001. [14] M. Specht, M. Kravcik, L. Pesin, R. Klemke (2001): Integrated Authoring Environment for Web Based Courses in WINDS. Proceedings of ICL2001, Villach, Austria, September 28, 2001. [15] Schoech, V., M. Specht, et al. (1998). ADI – An Empirical Evaluation of a Pedagogical Agent. World Conference on Educational Multimedia ED-MEDIA98, Freiburg, Germany.

- 75 -

- 76 -

On the Social Rational Mirror’s architecture: Semantics and pragmatics of educational interactions Daniele Maraschi, Germana M. Da Nobrega, Stefano A. Cerri LIRMM, Laboratoire d’Informatique, de Robotique et Micro-électronique de Montpellier 161, rue Ada – 34392 Montpellier – Cedex 2 – France {maraschi,nobrega,cerri}@lirmm.fr

Abstract The paper presents the overall architecture developed in two different educational application contexts allowing to manage both semantic and pragmatic aspects of student and teacher interactions with the tutoring system. A methodology will be exposed along with some other project considerations. All the discussion will be exploited presenting in parallel two practical applications.

Introduction Starting from a general overview of existent tutoring systems having an interface on the Web, we can observe that the methodology used to design them follows mainly the classical approach of Web site development. We think this kind of approach does not take really in consideration pedagogical requirements coming from the teaching experience. In this article we describe our approach taking into account innovative technologies to develop dialogue based tutoring systems on the Web interface. In such a context, we have developed two applications deployed in the domain of e-commerce. From the design perspective, both the applications follow the guidelines of our approach, while from the functional objectives, they are essentially complementary. The combined solution offers powerful and innovative tools in the new dialogue oriented Web interaction for pedagogical purposes. The first application considered was developed in the context of a European Project [1] with the main goal to provide a course to learn the e-commerce vocabulary in the English language by students coming from the East of Europe. The ambition of this type of application, as explained in [2], is manifold: allow to capitalise from dynamically available Web Information, not just from locally generated learning resources; allow as much as possible Agents to engage both in dialogues with learners and with teachers-developers-experts that wish to enrich, modify, customise etc., on the fly the XML documents generating dialogues with learners; transform the current HTML Web technologies into fully Conversation Oriented, server-based dynamic technologies in order to multiply the potential use world-wide of produced CALL material. The second application is a Web-served learning environment, called PhiInEd, to assist both the planning and the execution phases of a course. The work within PhiInEd is based upon educational theories that privilege autonomous learning from experience, by trial and error, founding the discipline Discovery Learning. The server has been experimented in a student class on Business Contracts. In a complementary paper [10] we report the experiment. In this paper we introduce PhiInEd through a hypothetical example, and we present the main components of its architecture.

P.R.I.M.A Architecture overview We have formalised an architecture, called P.R.I.M.A. (Program-centered Reusable Interactive Multimedia Architecture) describing both the fundamental software components and their organisation. We can identify

- 77 -

three main temporal phases: a dynamic Information acquisition meant to maintain a continuously updated information repository in order to produce Knowledge; the structuring of Information into Knowledge phase meant to organise and to provide semantical notations to the Information collected; and the Knowledge reuse phase meant to be the preferential source to build learning materials. In all of these phases, other mechanisms are considered. For example, since all of them have to provide a user interface in order to interact with the user, the organisation of such interfaces is described by the Logic, Content and Style (LCS) methodology (Figure 1) that modularise a single HTML page into three different files, one structuring the page content, the XML, another one structuring the layout on the Web browser, the XSL and the last one structuring the page interaction in Java. The same methodology has already used to develop an XML course starting from the correspondent offline version on CD-ROM [3].

Figure 1. The LCS methodology

One of the advantages is to associate in one-to-many relations different components: for example, many Web page contents described in different XML files can be associated to the same XSL file providing a uniform layout, or in the other hand, the same content described by an XML file can be displayed in different way on the Web browser. Another mechanism considers the web interface itself ( in this case the XML/XSL page ) the message structure of the dialogue between the user and the cognitive agent managing the interaction. This means that the Web page have to allow the same expressiveness freedom to the two ways of communication: from user to the agent and viceversa. Moreover, they have to facilitate the pragmatic comprehension by explicitly labelling of such messages. Basic framework Since our intention is to provide an easy access to the platform via the Web, we have chosen HTTP as the basic protocol, but this is not enough since, as stressed in [4], HTTP is a stateless protocol which make difficult to track the user session and hence to remember the history of the user interaction. So we have developed the platform using the Java Servlet framework that allows to deploy applications listening on the same socket port of the Apache web server and, at the same time, consider a sequence of web interaction like a whole user session. This provides the main advantage to build next Web pages using information coming from the past and not only from the last page visited, as in the case of many scripting languages. Session tracking is important both from the pedagogical point of view, for example to monitor the student progression, and from dialogue point of view since it allows to maintain the partner dialogue knowledge, as justified in [5]. The Servlet framework is also used to implement the LCS methodology explained before allowing to mix the Logic component with the Content and Style ones; this mean, in other words, to program dynamically interaction of an XML and/or an XSL page. The XML programming allows to manipulate dynamically the page content proposed to the user; for example an XML file structuring an exercise will contain, along with other information, the solutions, but is not really useful to show them immediately to the student. In analogue way, XSL programming allows to program the page layout; this can be useful, for example, to adapt the page content to a particular learning strategy that the system considers more efficient for the student.

- 78 -

Serendipitous Information acquisition We state as a precondition that the Course content, view as a set of information, is already available on the Web taking into account also the advantage of Web dynamics allowing to access to a continuously updated information source. For this reason, we have placed some searching Agents, based on the Madkit agent platform [6], collecting the content of Web pages (Figure 2) coming from serendipitous queries to most common web search tools, like Google [7] and Altavista [8], and saved in a database.

Figure 2. Serendipitous Information search by Agents

We have placed a filtering parser that transform the HTML page in a XML-like tree structure, called DOM, and select only the text content nodes discarding those concerning the images and other multimedia types as shown in (Figure 3).

Figure 3. Structure parsing of an HTML page

- 79 -

Structuring Information into Knowledge All the information collected in HTML pages stored in the database is essentially not structured. For this reason, a person with a particular role, called the Semantic Author, is charged to attribute semantic notations to the information collected. An XML web editor was implemented and presented along with the HTML page preview in order to allow the Semantic Author to do the process manually. The XML web editor, developed in [9], allows to define XML templates describing the component structure of a document; such a structure can be modified immediately in the case the Semantic Author considers it not semantically appropriated for the HTML page content shown on the left-hand side of Figure 4. For each HTML page a new XML document is generated describing semantically the information. In this way, a consistent set of XML documents is stored in a different database.

Figure 4. The XML web semantic editor

- 80 -

Knowledge reuse Once a significant collection of XML files is set up, it is possible to reuse such Knowledge asking for precise meaningful document components in order to built the content of a new Web page. In this context, a user playing a different role, called the Course Author, is charged to built a new set of XML documents composing a new Course content for the domain of e-commerce. Only the resulting XML pages will be used to deploy the final Web application along with the layout described by XSL files and the user interaction implemented in Java. Of course, in the case of changing the goal, correspondingly also the criteria used to structure the new XML pages necessarily change, but neither the Course Author process nor the Semantic Author process described above can be automatically managed by the architecture. This means that human contribution will be always necessary in order to end successfully the process.

The PhiInEd application

Figure 5: Phases of a course supported by PhiInEd.

Within the server PhiInEd, two phases of a course are currently taken into account, namely, Planning and Running (Figure 5). Planning a course consists on the elaboration of a Plan by the one who administrates the course, to which we refer as the Teacher. Running a course consists on the execution of the Plan, by the ones who follow the course, to which we refer as the Learners, guided by the Teacher. The result of the work in the Planning phase, called Plan, is composed of a sequence of Lessons to be studied by Learners. Executing a Plan for the Learners means to study the Lessons from the Plan. We consider the existence of a Plan as a requirement for a course to be run within PhiInEd. The result of the work in the Running phase is called a Reasoning Framework, that represents knowledge to be used by a machine in order to assist humans to reason in the corresponding knowledge domain. In a complementary paper [10], we detail each phase, as well as report an experiment carried out with a class of twenty-seven students in Law. The server PhiInEd has been designed in such a way to support both the individual work and the communication between participants of a course. Therefore, the main components of the server's architecture are: the Framework for Individual Work (FIW) and the Message Manager (MM). Hereafter we detail each one.

- 81 -

The Framework for Individual Work The Framework for Individual Work (FIW) supports, as suggested by its name, the work that each participant of a course is supposed to perform by himself. For the Teacher, the FIW supports the elaboration of the Plan during the Planning phase, while for the Learner, the FIW supports the elaboration of the Reasoning Framework, as a result of the study of Lessons from the Plan. Let us explain, by means of a hypothetical example, the rationale underlying individual work. Let us suppose that the subject of study for Learners is the so-called trading conditions. Learners are instructed to play the role of a lawyer who intends to get assistance from a machine, in order to analyse, verify, and write contracts on trading conditions. According to the business, they are informed that it is necessary to distinguish, for instance, products from services. Also, products considered dangerous are supposed to be treated differently from those considered normal, and so on. Let us now suppose that Learners find out that they would need distinguished assistance, in cases they deal with trading conditions for sellers, or buyers. Maybe buyers of dangerous products intend to impose to sellers the responsibility of the transport. On the other hand, maybe sellers do not want at all lawyers to consider such a constraint when writing their trading conditions. The example is illustrated in Figure 6: the same discourse could be used by lawyers (Learners) to write both sellers' and buyers' trading conditions, but the discourse is constrained differently according to each context of interest (seller or buyer).

Figure 6: (Hypothetical) Trading Conditions in different contexts: buyers and sellers.

According to the example, the resulting Reasoning Framework is composed of: (i) the terms, in a hierarchy, establishing the discourse, (ii) the contexts that are defined by a number of logical constraints among those terms, and eventually (iii) the documents that are classified according to contexts, and are supposed to respect the constraints of their corresponding contexts. One important feature of a Reasoning Framework is the dialectics between documents and contexts. From the one hand, existing documents may be used as a source of information in order to define contexts. From the other hand, once they are defined, contexts may “accept" or not arriving documents. By accept we mean that a document verifies the constraints in a context. Also, one might think in a context as something evolving continuously, possibly due to coming documents. In order to actually assist a human to reason in a domain starting from the Reasoning Framework, a machine should provide him to build and to revise contexts, as well as to verify/create documents with respect to contexts. At this point, we are able to present how individual work is actually provided by the FIW. This is accomplished by means of FIW's three main sub-components, namely, Terms, Folders, and Documents. The component Terms provides the edition of hierarchies of labels. Functionalities like the addition/suppression/movement of labels are provided. A label created through this component as a part of a hierarchy, has the same name of the component itself: a Term. Like the component Terms, the component Folders provides the edition of labels organised as hierarchies. However, a label created through this component as a part of a hierarchy is called a Folder. A Folder represents

- 82 -

what we have called above a context. The Folder components provide as well the creation of Constraints, as logical relations among Terms.

Figure 7: Document Description in Terms Hierarchy.

The component Terms provides the verification/creation of a document with respect to the Constraints in a Folder. One must point out what Terms he has observed as Present or Absent in the case of an existing document, or what Terms he wants to be Present or Absent, in the case of a document being created. Let us recall the example about trading conditions. Suppose that the Terms observed as Present in a certain document are “dangerous" and “Buyers transports". The verification of such a document with respect to the Sellers' context from our example is shown in Figure 7. The two Terms are highlighted because the context “rejects" both Terms simultaneously Present. By reject we mean that the Constraints in the Folder are violated (Figure 8).

Figure 8: Folder representing Sellers' context and its Constraints.

- 83 -

The list of Terms together with their Present/Absent values is called a Description. In PhiInEd, we are particularly interested in documents from Web pages. The structure that represents a document in the system is called a Document. A Document is then a list composed of a Description, a URL, a Name, and Comments. Name is a string identifying the Document. Comments is also a string. The creation of a Document is provided by the component Folders, since the user must point out in which Folder he wants to classify the Document. The maintenance of a Document is provided by the component Documents. Data generated through the FIW component are all stored as a tree in an XML file. The Message Manager The Message Manager (MM) is composed of two sub-components, namely, Sender/Receiver and History Manager. The Sender/Receiver is responsible for the composition and sending a message from a participant to another, as well as for retrieving the new messages arriving for a participant. The composition of a message provides the attachment of the XML file with the Reasoning Framework. Sending a message causes the storage of message information in a Repository containing all the waiting messages. Whenever a participant asks the retrieval of waiting messages, the pending messages that are addressed to him are removed from the Repository. The Sender/Receiver communicates with the History Manager, which is responsible for storing/retrieving the messages of a participant. Whenever a message is sent, message information is also stored in the History of the sender. Whenever a message is received by a participant, message information is stored in his History as well, after being removed from the Repository by the Sender/Receiver. The History of a participant is organised as a sequence of Dialogues (Figure 9).

Figure 9: Dialogues in a Learner’s History.

- 84 -

Conclusions The goal of the paper is to show that enhancing Web technologies we would profit from several opportunities offered in order to: •

use Information available on the Web is a necessary complement to the generation of ad hoc proprietary learning material;

•

transform the resulting learning – authoring process into dynamic and conversational ones, in order to bypass the current limitation of a page-centred client dominated view.

In the first application Information is transformed into Knowledge by the creation of XML files in which semantics is organised through XML tags. In the second application Learners use Web pages as a social information source to produce explicitly organised knowledge resulting in a Reasoning Framework. The complementarity of the two applications lies on the fact that in the PhiInEd application the course content is supposed to be achieved in HTML pages composing the Plan. The methodology presented in the first application can be applied in order to compose the content Lessons of a Course.

References [1]

Larflast EU Project. http://www.larflast.bas.bg/

[1]

Cerri, S. A.; Dikareva, S.; Maraschi, D.; Trausan-Matu, S.; "Web Server Based Architectures for Language Learning: LARFLAST Agents generating CALL Dialogues". Conference on Computer Assisted Language Learning - The Challenge of Change, University of Exeter, September 1- 3, 2001.

[2]

Maraschi, D., Cerri, S.A., Martinengo, G., “CD-ROM -> Web-Generated : reverse engineering of an interactive Multimedia Course”. IEEE Int. Conf. On Advanced Learning Technologies (ICALT). Okamoto, T., Hartley, R., Kinsuk and Klus, J. P. (eds.). Madison, Wisconsin, USA. 6-8 August 2001. pag. 203-204.

[2]

Kinsuk, Han, B., Hong Hong and Patel, A., “Student Adaptivity in TILE: A Client-Server Approach”. IEEE Int. Conf. On Advanced Learning Technologies (ICALT). Okamoto, T., Hartley, R., Kinsuk and Klus, J. P. (eds.). Madison, Wisconsin, USA. 6-8 August 2001. pag. 297-300.

[3]

Cerri S. A., “Shifting the focus from control to communication: the STReams OBjects Environments model of communicating agents", Collaboration between human and artificial societies, vol. 1624 in Lecture Notes in Artificial Intelligence, J. Padget, Ed., Springer-Verlag, Berlin Heidelberg, New York, pag. 71-101, 1999.

[3]

Gutknecht O., J. Ferber, "Madkit : A generic multi-agent platform", AGENTS'00 : 4th Int. Conf. on Autonomous Agents, Barcelona, Spain, June 2000; ACM Press, pag. 78-79, 2000.

[4]

Google web search: http://www.google.com/

[5]

Altavista web search. http://www.altavista.com/

[6]

Wittman P., "A Methodology for porting traditional CBT Courses on the Web", Master Thesis, Univ. of Montpellier II, Montpellier, France, 2002.

[7]

G. M. da Nóbrega, S. A. Cerri, and J. Sallantin. On the social rational mirror: learning e-commerce in a web-served learning environment. In Intelligent Tutoring Systems 6th International Conference, ITS 2002 Biarritz, France, June 2002. (submitted).

- 85 -