An Interactive Course-Support System for Greek Trude Heift {
[email protected]} Janine Toole* {
[email protected]} Paul McFetridge {
[email protected]} Fred Popowich† {
[email protected]} Stavroula Tsiplakou {
[email protected]} Department of Linguistics, Computing Science†, and Natural Language Laboratory* Simon Fraser University, Burnaby, B.C., Canada V5A1S6 Abstract This paper reports on a course-support system for English learners of Greek which is built around an Intelligent Language Tutoring System (ILTS). Meaningful interaction between the learner and the system is achieved by a number of means: First, students provide natural language input rather than selecting exclusively from among pre-defined answers. Second, the ILTS uses Natural Language Processing to analyze the learner’s input and to provide errorspecific feedback. Third, the system contains a Student Model which keeps a record of students’ performance histories. The information about each learner in the Student Model determines the level of specificity of feedback messages, clues, and exercise difficulty. In addition to vocabulary and grammar exercises, the system also contains oral dialogues with translations, a glossary, and cultural information. The system is designed for introductory learners of Greek and is implemented on the World Wide Web.
1. Introduction As the Web begins to mature and technologies for adapting existing content in language learning converge with increased bandwidth, the desire grows to develop uniquely Web-based applications. Convenient access, currency and variety of material, and integrated multi-media can extend traditional language instruction, but truly new forms require a more profound degree of interactivity than provided by conventional forms. Interactivity has become a key term in Hyper/Multimedia and although it is used extensively, its definition and scope is still not precisely determined. However, Laurel [1991] provides a useful definition by making a distinction between frequency, range, significance, and participatoriness. With respect to frequency and range, we may count the number of times a user can interact with a system and how many choices are available, respectively. Significance and participatoriness both require a qualitative analysis to determine how the user’s choices affected matters and to which degree the user is participating in ongoing system decisions [Ashworth 1996]. But to achieve a high degree of interactivity with respect to significance and participatoriness, the software requires intelligence to consider user’s individual differences. For example, students learn at a different pace and have different sets of skills. These student variables need to be considered for a system to be highly interactive. The course-support material described in this paper differs from others in that it is built around an interactive Intelligent Language Tutoring System (ILTS). Interactivity is achieved in a number of ways. The system provides a wide range of exercises covering vocabulary and grammar practice. For some tasks, students provide natural language input rather than selecting exclusively from among pre-defined answers. The tasks are independent of each other and the exercise types range from drill-and-practice to more game-like tasks. In addition, the system contains oral dialogues, a glossary, and cultural information. Significance and participatoriness is achieved through Natural Language Processing (NLP) and Student Modeling. The system
consists of a grammar and a parser which analyzes student input and provides error-specific feedback. The system also maintains a Student Model which keeps a record of students’ ongoing performance, alters system decisions accordingly, and provides learner model updates. In the following, we will first discuss the goals of the system and address the grammar, parser, and Student Model components. We will then describe the modules involved in analyzing student input. Further we will illustrate the exercise types of the system and provide examples of learner modeling for each of them. Finally, we will conclude with suggestions for further implementations.
2. Goals and System Requirements The goal of the ILTS for Greek is to provide meaningful and interactive vocabulary/grammar practice for English learners of Greek. Meaningful tasks and interactivity require intelligence on the part of the computer program. Unlike existing course-support systems which use simpler grammar practice and feedback mechanisms, the ILTS for Greek emulates two significant aspects of a student teacher interaction: it provides error-specific feedback and allows for individualization of the learning process [see Heift, 1998]. In example [1] the student provided an incorrect Greek sentence: (1) *Ολος ο κοσµος µαθαινεις Ελληνικα. Ολος ο κοσµος µαθαινει Ελληνικα. The whole world is learning Greek. In such an instance, the system will detect an error in subject-verb agreement and, in addition, will tailor its feedback to suit the learner’s expertise. Tailoring feedback messages according to student level follows the pedagogical principle of guided discovery learning. According to Elsom-Cook [1988], guided discovery takes the student along a continuum from heavily structured, tutor-directed learning to a point where the tutor plays less of a role. Applied to feedback, the pedagogy scales messages on a continuum from least-to-most specific guiding the student towards the correct answer. There are three learner levels considered in the system: novice, intermediate, and advanced. For example (1), the novice will receive the most detailed feedback: “You have made a subject-verb agreement error.”, while the intermediate learner will be informed that an error in agreement occurred. In contrast, the advanced learner will merely be told of an error in the sentence. The central idea is that the better the language skills of the learner, the less feedback is needed to guide the student towards the correct answer. This analysis, however, requires: a) an NLP component which can analyze ill-formed sentences, and b) a Student Model which keeps a record of the learner. The NLP component consists of a grammar and a parser. The system is written in LOPE, an ALE style extension of Prolog [see Carpenter and Penn, 1994]. LOPE is a phrase structure parsing and definite clause programming system in which the terms are typed feature structures. Definite Clause Grammars (DCGs), like other unification-based grammars, place an important restriction on parsing, that is, if two or more features do not agree in their values, a parse will fail. However, in a language learning system, these are the kinds of mistakes made by learners. To parse ungrammatical sentences, the Greek grammar contains rules which are capable of parsing ill-formed input (buggy rules) and which apply if the grammatical rules fail (see also [Schneider & McCoy, 1998], [Liou, 1991], [Weischedel, 1983], [Carbonell & Hayes, 1981]). The system keeps a record of which grammatical violations have occurred and which rules have been used but not violated. This information is fed to the Student Model. The Student Model is a representation of the current skill level of the student. For each student the Student Model keeps score across a number of error types, or nodes, for example, grammar or vocabulary. For instance,
the grammar nodes contain detailed information on the student’s performance on subject-verb agreement, case assignment, clitic pronouns, etc. The score for each node increases and decreases depending on the grammar’s analysis of the student’s performance. The amount by which the score of each node is adjusted is specified in a master file and may be weighted to reflect different pedagogical purposes. The Student Model has two main functions: First, the current state of the Student Model determines the specificity of the feedback message displayed to the student. A feedback message is selected according to the current score at a particular node. A student might be advanced with regard to vocabulary but a beginner with passive voice constructions. Hence, a feedback message about vocabulary would be less detailed than a feedback message about passive-voice constructions. Second, the difficulty of the exercises presented to the student is modulated depending on the current state of the Student Model. For example, if a student is rated as advanced with respect to vocabulary, then some of the vocabulary exercises are made more challenging. The same thing can be found with some of the grammar exercises. In the following, we will discuss the error-checking mechanism performed by the system in analyzing student input.
3. Error-Checking Mechanism In addition to the grammar and the parser, the system contains additional error-checking modules which get evoked when processing a student answer. For example, consider the task in (2a) where the student was asked to make a sentence with the words provided. (2a) ισπανικα µιλαω Ν_κη (2b) Νικη µιλας ισπανικα niki – speak – Spanish The student answer given in (2b) contains a number of mistakes: an error in subject-verb agreement, a spelling mistake, and a missing article. While the grammar will detect the agreement error, the remaining mistakes will be discovered by additional modules of the system. Figure [1] illustrates the modules of the Natural Language Processing system. The first module in analyzing student input is a spell check. During the spell check, the system also extracts the base forms of each word from the grammar. The uninflected words are needed to determine whether the learner followed the task, that is, whether the student answer contained the words which were provided. Such errors cannot be determined by the grammar and the parser because the parser can only judge whether a sentence is correct or not. In defining an exercise, we store possible answers of a given task and the Answer Check module determines the most likely answer (MLA) the student intended. The Answer Check module further matches the extracted base forms with the MLA. If any of the words in the task are not contained in the student answer, the system will report an error. The following two checks, Extra Word Check and Word Order Check, refer to additional words in the student answer and errors in word order, respectively. While these two checks are commonly handled by the grammar [Schwind, 1995], preliminary testing of our system showed that the system performs faster if these two error types are treated outside the grammar. Naturally, the speed is also influenced by the grammar formalism used.
The Grammar Check is the most elaborate of the modules. Here the sentence is analyzed by the parser according to the rules and lexical entries provided in the Greek grammar. Currently, the grammar covers a wide range of grammatical concepts, from early beginner constructions (verb ‘to be’ and the concept of null subjects) to fairly advanced structures (passive voice). Development is still ongoing to achieve a complete coverage of Greek grammar. Student input
Spell Check
Answer Check
Extra Word Check
Word Order Check
Grammar Check
Match Check
Feedback Message to the Learner Figure 1: Error-checking Modules The Match Check looks for correct punctuation and capitalization by string-matching the student answer with the MLA. By this time in the evaluation process, it is very unlikely that the sentence still contains any errors other than punctuation or capitalization. If the sentence passes the Match Check successfully, the sentence is correct. If not, an error is reported to the student. The system is organized in a way that if a module detects an error, further processing is blocked. As a result, only one error at a time will be displayed to the learner. This was implemented mainly to avoid overloading the student with extensive error reports in the case of multiple errors. According to van der Linden [1993], displaying more than one feedback message at a time makes the correction process too complex for the student. After correcting the error, the student restarts the checking mechanism by clicking the CHECK button. In the last section, we will briefly describe the six exercise types implemented in the system and discuss the role of the Student Model for each.
4. Exercise Types The system consists of three vocabulary and three grammar exercise types: Guess the Word, Find the Word, Which Word is Different, Word Order Practice, Fill-in-the-Blank, and Build a Sentence, respectively. Student
performance in each exercise contributes to the Student Model. However, the nodes that get updated for each exercise differ. In addition, there are two ways in which this information is used by the system: 1) feedback messages tailored to student expertise 2) difficulty of exercise presented to student In the following, we will discuss the nodes which get updated and illustrate the feedback message and exercise difficulty modulation for each exercise type. Vocabulary Guess the Word This is a slight variation of the vocabulary game ‘Hangman’ where students guess a word by entering one letter at a time. To avoid making this a purely guessing game, a clue is provided for each word. This clue may be the English equivalent of the answer, or a picture or sound representing the answer. For each correct word, there is an increment for the vocabulary node, for each word missed, the system registers a decrement. In this exercise type, the feedback messages are the same for all users. However, advanced students, that is, students with a high vocabulary score get fewer tries to complete the task. Find the Word In this exercise type, students need to find several words in a grid. The system increments the vocabulary node for each word found and decrements for each incorrect guess. The maximum increments and decrements per game are equal to the number of words in the table. The feedback messages are the same for all students. The exercise difficulty is modulated: advanced students receive clues in English, while other students receive clues in Greek. Which Word is Different This exercise displays a number of words all except one of which belong to the same category. The student task is to identify the one which differs from the others. The divergent word may differ syntactically, semantically or pragmatically from the remaining words. There is an increment for each correct word selected, for each incorrect selection a decrement is recorded. The feedback messages are the same for all students and there is no difficulty modulation. Grammar Word Order Practice In this exercise, students practice Greek word order with a ‘drag and drop’ task: words have to be arranged in an appropriate order to form a grammatical Greek sentence. For this exercise type, the word order node in the Student Model is updated. For a sentence with incorrect word order, the system records a decrement, else an increment. There is no feedback or difficulty modulation for this exercise. Fill-in-the-Blank The student’s task here is to complete sentences by filling in any blanks that appear in the example. The Student Model records each grammar node that is detected in the student input. For example, if students are asked to supply the correct conjugation of the verb, then the system records an increment/decrement for subject-verb agreement. The feedback messages are modulated according to the level of the learner. Also, advanced learners obtain more difficult tasks. For instance, novice and intermediate students will find only one blank per example. Advanced students may be presented with examples that contain more than one blank.
Build a Sentence In this exercise type, students are provided with a set of words. Their task is create a grammatical Greek sentence using all the provided words. This was illustrated earlier in example (2). All grammar nodes activated during the processing of the student’s input are updated in the Student Model. An increment is recorded if this aspect of the student’s answer was correct. A decrement is recorded for any grammatical errors made by the student. Feedback messages reflect the current state of the learner’s expertise, as represented by the Student Model. There is no difficulty modulation. It becomes apparent that the modeling process has to be decided for each exercise type independently. Not only does it depend on the task but also the possibilities the exercise offers in adjusting the level of difficulty.
5. Conclusions In this paper, we discussed an interactive course-support system on the WWW. Interactivity is achieved by a wide range of exercise types which are arranged in an independent order. NLP and Student Modeling also contribute to the interactivity of the whole system by providing error-specific feedback and adjusting to different learners. The Student Model is based on the performance history of the student. The information about each learner is recorded in the Student Model and determines the level of specificity of feedback messages and exercise difficulty. Preliminary testing of the content of two modules and the overall functionality of the system has taken place. While the performance of the system is accurate with respect to feedback and student modeling, student responses are currently being analyzed and results will be reported during the presentation. In the meantime, the system is being further developed by adding more content and implementing further grammatical constructions.
References Ashworth, D. [1996]. Hypermedia and CALL. In Pennington, M. C. (ed.). The Power of CALL. Athelstan Publications, Houston, USA. Carbonell, J. G. and Hayes, P. J. (1981). Dynamic strategy selection in flexible parsing. Proceedings of ACL 19th Annual Meeting. 143-147. Carpenter, B., and Penn, Gerald. (1994). The Attribute Logic Engine: User’s Guide, Version 2.0. Computational Linguistics Program, Carnegie Mellon University, Pittsburgh. Elsom-Cook, M. (1993). Student Modelling in Intelligent Tutoring Systems. The Artificial Intelligence Review, 7 (3-4): 227240. Heift, T. [1998b]. Designed Intelligence: A Language Teacher Model. Ph. D. Dissertation, Simon Fraser University, Canada. Liou, H.-C. (1991). Development of an English Grammar Checker: A Progress Report. CALICO Journal, 9: 27-56. Pennington, M. C. [1996]. The Power of CALL. Athelstan Publications, Houston, USA. Schwind, C. B. (1995). Error Analysis and Explanation in Knowledge Based Language Tutoring. Computer Assisted Language Learning, 8 (4): 295-325. Schneider, D. and McCoy, K. [1998]. Recognizing Syntactic Errors in the Writing of Second Language Learners. Proceedings of the 17th International Conference on Computational Linguistics. Van der Linden, E. (1993). Does Feedback Enhance Computer-Assisted Language Learning. Computers & Education, 21 (1-2): 61-65. Weischedel, R.M. and Sondheimer, N.J. (1983). Meta-Rules as a Basis for Processing Ill-Formed Input. American Journal of Computational Linguistics, Special Issue on Ill-Formed Input, 9 (3-4): 161-177.
Acknowledgements We would like to thank the Greek Ministry of Education for generously supporting this project.
