parsmg skills (both parts-of-speech and functional constituents) with first-year language ... conceived äs a senes of tests/exercises with minimal feedback (nght/wrong ... keeps score, recycles incorrectly answered items, and provides explanations and ... built-in user manual, and a short-cut to page l of the theory book. The ...
COOL/CALP: Computer-Assisted Learning to Parse in Dutch Vincent J van Heuven Project Group for Computer Assisted Learning Faculty of Arts, Leiden University Abstract COOL/CALP 1 was developed to diagnose and remedy deficiencies m traditional parsmg skills (both parts-of-speech and functional constituents) with first-year language students at Dutch universities and teacher trammg Colleges Though conceived äs a senes of tests/exercises with minimal feedback (nght/wrong response), students can explore explanations and examples through hypertext links at any time COOL/CALP was developed m 1993-1995 at the Linguistics Department of Leiden University It was used m the two succeedmg years and evaluated on the basis of students' responses to a wntten opmion poll We will outline the pedagogical and technical pnnciples underlymg the programme, and discuss the results of the formal evaluation l
Introduction
In 1993 the Department of Linguistics at Leiden University was asked by the Board of the Arts Faculty to develop a CAL-apphcation that would • enable students to test and improve their knowledge of grammatical terminology and elementary parsmg skills m o parts-of-speech (verb, noun, adjective, etc ), o grammatical functions (or constituent analysis, i e subject, predicate, direct/mdirect object, adjunct, etc ), • provide minimal feedback response correct vs wrong only, • provide consultation of theory and explanation through hypertext links In order to make the product maximally accessible to students, the hardware requirements were to be minimal, such that the programme could • run on even the simplest IBM-compatible PC (8088 with 640 Kb, smgle floppy dnve, monochrome Video monitor), • be entirely held m RAM (fast response, no delays while loadmg modules from disk), 'COOL is an acronym for Dutch Computer Ondersteund Ontleden Leiden The programme is freely available on diskette, send e-mail to pgcoo@rullet leidenumv nl I thank Maarten Hijzelendoorn for his comments on an earher version of this chapter COOL/CALP was developed by Martin Honcoop (courseware author), Maarten Hijzelendoorn (programmer) Teun Hoeksta and Cnt Cremers (theoretical linguists) The present author acted äs the project coordmator An extended version of this paper is accessible through the WWW pagc of the project group (http //wwwlet leidenumv nl/coo/home html)
COOL/CALP: CoMPUTER-AssiSTED LEARNING το PARSE IN DUTCH
75
• fit 360 Kb (i.e., can be contained even by a single b\ floppy diskette), • optionally support more advanced features (hard-disk Installation, color screen, mouse, network version) 2
Description of the programme
COOL/CALP is a concise text-only CAL authoring framework. It provides the possibility to present exercises in multiple choice format, provides feedback, keeps score, recycles incorrectly answered items, and provides explanations and theory through hypertext links. Exercises, tests and theory are input in plain ASCII and can be edited by the courseware developer. Editing courseware is rather cumbersome äs the results of the editing can only be viewed when the Programme is executed. The framework was written in Turbo Pascal. The learning contents of COOL/CALP were decided on in consultation with Modern Language teachers at the various departments. The set of part-ofspeech types and subtypes is listed in table la; table Ib lists the inventory of grammatical constituents that are covered by COOL/CALP. Functional constituents are defined in COOL/CALP äs strings of words that can precede the finite verb in a main clause (e.g. through topicalisation). The finite (and the verbal part of the predicate) itself is therefore not treated äs a functional constituent. COOL/CALP mainly deals with the constituent analysis of simplex sentences. The analysis of compound sentences (two or more coordinated main clauses, or a main clause with one or more subclauses) is only briefly touched upon, and explained through analogy. For each item in table l the theory book, provides a series of (one to seven) help screens that are accessed from the exercises through hypertext links. 3
Flow through session
The logic of the flow through a typical Session is outlined in figure 1. The states (circles) in the diagram are the most important screens that the student secs while working with the program. The arcs represent decisions made and/or actions performed by the student, taking him/her from one state to another. We will now discuss the various stages and comment on the philosophy of the Programme. DOS-screen. Since the programme is launched from äs well äs exited to the PC's operating System (DOS), the DOS-screen is both the initial and terminal state in the diagram. At the Start of a COOL/CALP-session the user first chooses whether the programme will run in EXAM MODE or in EXERCISE MODE. In the exam mode access to the theory book is disabled. Main menu. The student then enters the COOL/CALP main menu. This menu presents various fields, very much like click-on buttons in a Windows user interface. For first-time users there are buttons that open up the built-in user manual, and a short-cut to page l of the theory book. The
VINCENT J. VAN HEUVEN
76
A. Parts-of-Speech type and subtype 1. Verb • main verb • copula • auxiliary o of aspect (perfect) o of voice (passive) o other (modality, causality) 2. Noun • count • mass • proper name 3. Adjective • attributive use • predictative use • grades of comparison 4. Adverb 5. Preposition • simple • compound • stranded
6. Conjunction • coordinating ~ • subordinating ~ 7. Pronoun • personal • possessive • relative • demonstrative • interrogative ~ • indefinite • reflexive • reciprocal 8. Interjection 9. Numeral • cardinal • ordinal 10. Article
B. Functional constituent type 1. 2. 3. 4.
Subject Direct object Indirect object Prepositional adjunct
5. 6. 7. 8.
Adjectival adjunct Adverbial adjunct Predicative adjunct Nominal part of predicate
TABLE l A. Part-of-speech distinctions made in COOL/CALP. B. Functional constituents distinguished in COOL/CALP
COOL/CALP: CoMPUTER-AssiSTED LEARNING το PARSE IN DUTCH
77
blocket! in exarn mode
FIGURE l Flow through a typical COOL/CALP-session. States in the network represent programme screens, arcs denote choices made and/or actions performed by the user.
78
VINCENT J. VAN HEUVEN
crucial buttons on the main menu, however, are clicked in order to choose between the PARTS-OF-SPEECH vs. FUNCTIONAL CONSTITUENTS mode. Assignments directory. When the type of assignment has been decided on, the Student sees a directory of assignments that he can choose from. The notion assignment is used here äs a cover term for a series of 60 parsing problems. For both parts-of-speech determination and for functional constituent parsing, there are two such assignments to be used äs exams (accessible in exam mode only), and four to be used äs exercises (accessible in exercise mode only). The assignments have been compiled such that all parts-of-speech (sub-)types and all functional constituent types are equally represented, allowing efBcient diagnostic testing. Exercise screen. Once a specific assignment has been started, the Programme enters the exercise screen. At the top of the screen a sentence (randomly selected) is displayed with a single word or constituent highlighted. Underneath the test-sentence eight (functional constituent labels) or ten (parts-of-speech main types) buttons are displayed, and the Student is asked to select the appropriate button. Follow-up screens appear when an initial response category has subtypes (cf. table 1). Feedback screen. The feedback screen is, in fact, identical to the exercise screen but it now contains a feedback-line that reports to the Student whether his response was right or wrong. In either case the Student is offered a choice to consult the theory book (see below). Whether or not the theory book is consulted, the Student will then have to make (at least) one other attempt; he cannot leave the feedback screen unless and until he has provided the correct solution. Once the correct response has been given, the Programme returns to the exercise screen (i.e., the feedback message disappears) and shows the next item. In exercise mode, any item that was not answered correctly, is recycled. Score statistics screen. The score screen, when invoked, informs the user in absolute numbers and in percentages of the number of (i) correctly answered items, (ii) incorrectly answered items, (iii) correct Solutions given away by the System. The number of items remaining, i.e. including (ii) and (iii), in the assignment is presented last. For each part-of-speech type and subtype äs well äs for each functional constituent label the distribution of answers in (i), (ii) and (iii) is given, together with a short-cut to the theory book. Thus, the score statistics screen can be used äs a personal guide through the theory book. Theory book screen. When the theory book is accessed from the feedback screen, it presents the first page (screen) of the text (defmition and basic examples) associated with the reponse button just activated. When the student has selected an incorrect button, he will be able to work out for himself by reading the definition and comparing the examples provided by the theory book that this choice does not apply to the problem he was
COOL/CALP: CoMPUTER-AssiSTED LEARNING το PARSE IN DUTCH
79
asked to solve (note that the test item remains visible until the correct solution has been given). Whenever the Student, while reading through the theory screens, encounters terms that are marked äs hypertext links, he can digress, and later work his way back to earlier screens using the escape-key or by pressing the right-hand mouse button. The programme returns to the main menu once all the items in an assignment have been answered correctly. Alternatively, the user can terminate the assignment prematurely. In either case, score statistics will be presented first. From the main menu, the Student can call new assignments or exit to DOS. 4 Introduction into the curriculum; opinion poll COOL/CALP was basically developed in the academic year 1993/94. At the end of this year a beta version of the product was distributed over the Modern Language teachers, who were asked to test the programme during the summer break. Those colleagues who had worked with the programme were generally impressed by its speed and robustness. However, they were dissatisfied with the contents of the programme. As a result of these complaints the product was not released to students, but given a complete overhaul in the next year. The set of grammatical categories was greatly reduced, and came to the set presented in table 1. The theory book was partly rewritten and simplified by the present author in close collaboration with one of the Modern Language teachers 2 . In the summer preceding the academic year 1996/97 some 500 copies of the partly rewritten product were sent to our prospective first-year language students, with an accompanying letter in which we advised the students to work through the programme. Also, students were explicitly reminded of the need (not Obligation ) to work with COOL/CALP by the present author, when he taught the Introduction to Linguistics course, which is compulsory for all first-year students of a Modern Western Language. At mid-term an evaluation form was handed out to and received back from 105 students in the Linguistics course who attended the lectures on that particular day. Roughly half of the questions addressed the user interface of COOL/CALP: • • • •
How useful do you find the built-in hypertext manual? How easy is it to find your way through the hypertext theory book? How user-friendly do you find the programme äs a whole? How clear do you find the screen lay-out (e.g. are screens not too crowded with text)? • How clear is the flow through the Session, i.e. how easy is it to navigate through the programme? The other questions bore on the contents of COOL/CALP: • How useful do you find the central exam session (if applicable)? 2 Thanks are due to Wilma van Straalen of the French department at Leiden University who generously donated her time and effort to this task.
VINCENT J. VAN HEUVEN
80
RESULTS COOL-POLL OCT1996
-P v^·
ΓΑ»
,=.«·
J^
^
·*' ΧΛ
^
