An XML-based Approach for Web-based Self ...

An XML-based Approach for Web-based Self Assessment Nathalie Poerwantoro, Bernd Krämer FernUniversität Hagen LG. Datenverarbeitungstechnik Philipp-Reis-Gebäude Feithstrasse 140 D-58084 Hagen-Germany ([email protected], [email protected]) Abstract: In this paper we describe an effective way to create a web-based self assessment tool. Our approach uses an exercise applet that reads and extracts questions from their descriptions in XML format and presents them to the students depending on students’ preferences. A simple authoring tool is used for adding questions and corresponding hints to the exercise environment. Questions can be accompanied by additional media such as video, picture, sound or animation applets.

Introduction In the past, distance education has always suffered from a lack of opportunities for hands-on experimentation and access to physical objects, models, simulations and instruments in computer and engineering labs. Today, computers in education and networked learning environments offer new ways of discovery learning through some unique forms of interaction with virtual objects, online laboratories at home, learner-directed animation of realistic processes, and light-weight software tools. In networked learning environments interactivity allows students to manipulate input data, parameters and thresholds of real-time simulations or animations. Learners may also influence the progress and output of process or algorithm visualizations. The project group at FernUniversitaet Hagen is exploiting some of these possibilities with the development of a series of exercise environments (Mogey 1997) that let students apply and experience what they have learned to predefined problems or problems of their choice. Thereby we expect to raise the attractiveness of the learning material and enhance the students’ motivation, especially that of distance learners. In the design of these environments we have attempted to associate learning objectives with appropriate multimedia solutions (Williams 1997) with respect to different teaching functions such as motivation, presentation of new structures, practicing new behavior, modeling, exploration by trial use and error, or application of new techniques. To achieve a high degree of platform independence, ease of use and portability and to remain independent from proprietary formats, it was decided to rely on HTML, XML (Harold 1998, Idris 1999) and Java (Horstmann 1998, Sun 1998) as core technologies for implementation of the exercise labs. Since the introduction of Java programming language we can see the use of applets in traditional hypertext systems as well as in an Intelligent Tutoring Systems. Java applets are employed very often to explain complex processes to deepen the understanding of difficult parts of educational documents. However, the use of Java applets does not have to be restricted to simulations or visualizations of specific problems. Many examples show that applets can be used for exercises and tests and therefore offer more flexibility than common media. Another advantage of applets is the possibility to accept parameters so that they can be tailored to different learner types. It should be possible to find common learning oriented descriptions for different users of the exercise applet and to make the exercise applet itself adaptable to specific needs. That way only one version of the applet would be necessary for many different kinds of users.

A Generic Exercise Environment After having developed several exercise environments by hand, we realized that we need many more exercises for the students to make the self assessment effective. Developing an exercise applet for each specific topic was not very effective if we want to generate a lot of questions for different topics. Another possibility is to create HTML based questions with applets as companion, which are often found in web-based exercises. The downside of these HTML questions is that they are static, which means each time the exercise is restarted the students will receive the same set of questions. What we were looking for is the ability to create a dynamic exercise environment, that can pick a random set of questions from a question pool, adapt to the student’s preferences and that is easy to use for the tutors creating the questions. This approach would also separate the questions from the applets and use the applets in conjunction with the questions in an exercise environment. The applets should KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc

show an animation or a simulation of a certain topic, and the question can be asked in relation not only to an applet but can also relate to other media such as video, picture, sound. We chose to develop an XML based exercise environment since XML is ideal to describe the question structure, the exercise structure and the structure of a student’s profile. The idea was to describe each question in XML and let those questions be parsed by the exercise environment applet and be presented in an appropriate format depending on the question type and companion media during the exercise. That way a large amount of question pool can be generated more easily. The configuration of an exercise, such as how many sections or how many questions, what kind of question, etc. is also described in XML and parsed into an exercise specification object used by the exercise environment applet at the beginning of an exercise. The exercise environment covers both formative and summative assessment types (Bull 1996, Williams 1997). This approach provides flexibility to authors, since any applets that the authors find useful for the exercise can be added any time to the applets pool as a companion to the questions. The same rules apply to pictures and video clips. With this flexibility comes a big task of putting a lot of questions into the exercise environment. To give a picture of it let’s take an example of an exercise for one topic with one difficulty level, five question types (true/false, multiple choice, text gap, set ordering, matching) with five questions for each type. That will result in at least 25 question files and 50 picture files and not to forget the hints and explanations for errors and companion applets. In order for the exercise environment to select a different set of questions each time the students restart the exercise, more than 25 questions have to be provided to enable a random pick out of the question pool. As soon as enough questions are available, the exercise environment can give the students an adaptive and interesting platform to do self assessment. Currently for evaluation purposes we have created about 150 question files, 150 pictures, a VRML scene, some companion applets and JavaScript animations for 6 topics, which in turn consists of several subtopics. The exercise environment consists of 3 modules or applications:

a) Question generator

Ø

A Java application with forms to create a question. Five of the most common question types in Computer Based Training (CBT) can be generated such as: true/false, multiple choice, fill text, matching and set ordering. An XML Document Type Definition (DTD) was defined to describe the structure of questions. Important parts in a question description include: Ø Domain: defines to which domain or topic the question belongs. Ø Create: defines the date of creation. Ø Level: defines the difficulty level of the question. 5 levels are currently supported. Ø Language: defines the language for the question. Ø Subtopic: defines the subtopic or subchapter of a question. Ø Question type: defines the question type which has to be one of the 5 described above. Ø Question: defines the elements that describe a question such as Ø Problem statement: defines the question asked. Ø Answer options: defines the answer options in question types multiple choice, matching. Ø Keys: defines the answers to the question. Ø Hints: defines the hints, each hint is related to an answer option and the hints consist of 3 levels, starting from a general hint to a detailed hint. The exercise environment will show the hints depending on the selected difficulty level. All 3 levels of hints will be shown for exercises in the first 2 lower difficulty levels, 2 levels of hints will be shown to those in the medium difficulty level and 1 hint will be shown to those in the last 2 higher difficulty levels. Hints are only available in formative mode of the exercise. If the student chooses to do the exercises in exam mode (summative), no hints will be shown. Ø Reason: defines the ‘why’ explanation of the answer Ø Show media: defines the media related to the question. The media can be a picture, a video clip, audio , applet. Picture list: defines a list of picture needed for the question type matching and set ordering.

Example of a question description:

MQnetwork_medium0 test question chapter 14.2 25/07/99 network medium de KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc

networks MQ Weisen Sie die entsprechende FDDI-Komponente dem jeweiligen Bild zu. Eine groessere Darstellung erhalten Sie durch anklicken des entsprechenden Bildes. PHY SMAP SMT MAC PMD 2 3 0 4 1 text Physical Layer Protocol none .... Die Pysikalische Ebene (PMD,PHY) uebernimmt die Anpassung an eine optische Faser. Das Station Management (SMT) hat die Aufgabe der Steuerung,Ueberwachung und Verwaltung der angeschlossenen Stationen und des Netzes. MAC entscheidet, welche Station Zugriff auf den Ring hat none test/pictures/fddi1.gif test/pictures/fddi2.gif test/pictures/fddi3.gif test/pictures/fddi4.gif test/pictures/fddi5.gif b) Exercise generator A Java application for creating the overall exercise specification. The exercise is described using the following fields: Ø Domain: defines the topic of exercise. Ø Level: defines the difficulty level of the exercise. Ø Number of sections: defines the number of sections in the exercise, 5 is the maximum since there are only 5 question types available. Ø Section Description: specifies each section by defining: Ø Section type: defines the question type of this section. Ø Number of question: defines the number of questions in this section. Ø Total question: defines the number of available questions in the question pool. If the number of questions needed in an exercise section is smaller than the available questions in the question pool, the exercise tool will select the questions randomly from the question pool. The exercise is less predictable and becomes more challenging since the students do not get the same questions each time they start the exercise. Ø

Description of test mode:

KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc

Ø Maximum error in relation to minimum solved questions. These 2 numbers are needed to define the maximum error rate allowed to pass the exercise in exam mode. Ø Time limit can be defined for exam mode. c) Exercise environment applet A Java applet which is the environment (Fig. 1) that the students will work with. A student profile is loaded by the applet at the beginning of the exercise. The student’s profile gives the information of selected exercise topic, subtopics, language and difficulty level. The applet then loads the selected exercise scenario and parses it using the IBM XML parser. Using the extracted information about the exercise, the applet prepares the exercise and loads the defined questions one after another. The exercise environment creates a random number for each question category in order to provide variation each time the exercise is restarted. In order to create a correct random number, the exercise environment checks the number of available questions on the server. If the number of available questions is less than requested in the exercise specification, then the exercise environment adjusts the question number for this exercise by reducing it to the number of available questions. Thus preventing the error of creating a random number of a nonexistent question. The XML file describing the exercise will not be modified, since more questions can be added to the question pool in the future, so next time the exercise is started , there could be more questions available than requested in the specification. After all random numbers for each question category have been calculated, each question is loaded, parsed and shown when the student arrives at the specific question. There are two methods to navigate in the exercise: sequential navigation using the forward and backward buttons and free navigation by using the navigation tree. The navigation tree gives more freedom to jump from one question to any other question in the exercise. If a certain media is related to a question such as an applet showing an animation or a simulation, a video clip, an audio file or a picture, then it is shown together with the question. Questions can either be answered or skipped. Skipped questions are counted as unanswered errors. Hints are also provided and can be called during the exercise. Each possible answer option has 3 levels of hints starting from a general one to a more detailed one. The number of hints that can be shown to the student depends on the difficulty level of the exercise. No hints are available in exam mode. In exam mode a time limit can be activated which limits the time given to the student to finish the exercise. When the time is up the exercise will automatically end and no further answers will be accepted. Students can submit the exercise any time and get the result of the exercise. The result of the exercise has an HTML format and is shown in a separate window. Student’s answers are listed together with the correct answer. If the question author provides the question with reason, in case of error, this reason text will be shown. A summary is provided at the end the result report showing error rate and achieved points. Each question number on the result window is linked to the exercise environment applet through an interface. That way students can reload the corresponding question by clicking on the question number in the result window, instructing the exercise environment applet to reload and parse that particular question. A log file is created at the end of the exercise and the student’s profile is also updated. This profile shows how many exercises a student has done and how well.

KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc

Figure 1: overview screenshot of the exercise environment applet

Example of a student’s profile:

jw12345 John Washington teacher 35 image hierarchical consult low de communication layer3 QOS 2 0.8 1.0 0.2 0.0 1

0.4 0.6

KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc

Conclusion and outlook In this paper we described an XML based exercise environment applet, a simple question generator tool and an exercise specification generator tool to create interactive web based exercises for self assessment. Our approach extends the usual static HTML-based exercises and provides more variation and a higher number of questions in an easy and efficient way. At the moment the exercise environment is being integrated into a CD-ROM based multimedia technology lessons. This material will be distributed to buyers of the book and students of the course Multimedia-Technologie (Steinmetz 1998) and will be evaluated for usability and learning effectiveness. References: Cay S. Horstmann, Gary Cornell (1998). Core Java 1.1 Part I , Sun Microsystems Press, Mountain View, Calif. Cay S. Horstmann, Gary Cornell (1998). Core Java 1.1 Part II, Sun Microsystems Press, Mountain View, Calif. Elliotte Rusty Harold (1998). XML Extensible Markup Language, IDG Books Worldwide, Foster City, CA Gary Williams, Liz Smith, Tom Lawson, Bill Lord, Deri Hadler, Roger Murphy, John Merkel (1997). Choosing the type of assessment, Center for Enhancing Learning and Teaching (CELT). Gary Williams, Liz Smith, Tom Lawson, Bill Lord, Deri Hadler, Roger Murphy, John Merkel (1997). A guide to the educational use of multimedia, Center for Enhancing Learning and Teaching (CELT). Joanna Bull (1996). Computer based assessment: some issues for consideration, CTISS Publications. Nazmul Idris, Sun Koh (1999). XML and Java Tutorial, The Bean Factory, LLC. Nora Mogey, Helen Watt (1997). The use of computers in the assessment of student learning, Learning Technology Dissemination Initiative. Ralf Steinmetz (1998). Multimedia-Technologie, Springer-Verlag, Berlin, Heidelberg, second edition. Sun Microsystems Inc. (1998). Java Media Framework Programmer’s Guide, 1998

KRAEMER:DVT-Server-Backup:kraemer:kraemer:PR:papers-old:000123ED-Media.doc