the alice system a workbench for learning and using language

THE ALICE SYSTEM A WORKBENCH FOR LEARNING AND USING LANGUAGE Lori S. Levin, David A. Evans, and Donna M. Gates Abstract: ALICE is a multi-media framework for ICALI programs that is being developed at Carnegie Mellon University. It is not a single instructional program, but rather a set of tools for building a number of different types of ICALI programs in any language. The central components of ALICE are (1) a set of Natural Language Processing (NLP) tools for syntactic error detection, morphological analysis, and generation of morphological paradigms, (2) a set of on-line text, video, and audio corpora that serve as sources of realistic, in-context examples, and (3) an authoring language that allows teachers to configure the NLP tools and excerpts from the corpora into ICALI programs. This paper describes the NLP components of ALICE and the role of excerpts from corpora in treating student errors. Keywords: ALICE project, ALICE system, Argentine movie, artificial intelligence workstation, authoring system, automated error detection, CLARIT system, cognitive research, computational linguistics, detectable rule-governed errors, error analysis feedback, error detection using NLP, foreign-language video-disk, GENKIT sentence generator, ICALI program, indexed text, inflectional paradigm generator, intelligent tutoring system (ITS), language acquisition, language corpus, left-associative morphology program, lexicon, LISP program, machine-readable knowledge base, morphological analysis, morphological generation program, natural-language parsing, natural-language processing (NLP), NLP helping students, on-line dictionary, on-line reference capability, on-line text, pattern matcher, Spanish writing assistant, speech-act situations, student interface, text base, Tomita parser, unification based grammar rules.

CALICO Journal, Volume 9 Number 1

27

I. A General Overview of ALICE ALICE is a multi-media workbench for users and learners of foreign languages that is being developed at Carnegie Mellon University. It provides a variety of facilities, including a framework for designing ICALI programs. In this sense ALICE does not offer a single instructional program, but rather a set of tools for building a number of different types of ICALI programs m any language. The ALICE facilities include tools and resources for (1) syntactic parsing and pattern matching with error detection, (2) morphological analysis of inflected words and generation of inflectional paradigms, (3) on-line dictionaries and other reference material, (4) indexing and retrieval of examples from corpora such as videodiscs, stored audio material, or on-line texts, and (5) an authoring system that allows teachers to combine and configure ALICE tools and resources into instructional programs. A program using the ALICE tools for NLP and for accessing corpora and reference material is called an application. The types of applications that ALICE can support include grammar and vocabulary drills, games and simulations, reading and writing assistants, and open-ended learning environments. We have two prototype applications—a Spanish writing assistant implemented in Lisp and Supercard on a Macintosh-II personal computer and a Japanese grammar exercise program implemented in LISP and cT on a Decstation 5100.1 The design of the ALICE system includes some components that are already implemented on a full scale, some implemented in prototype form, some simulated by hand, and some not yet implemented in any manner. This paper describes both what is currently running and what is planned for the future. II. ALICE Components In this section we describe the components of ALICE and indicate the state of current implementation. This information is summarized in Figure 1, which gives a schematic representation of the components of the system. Elements in solid lines are implemented; elements in dotted lines will be implemented during the next phase of the project. Collections of text, audio, and video corpora are a central component of ALICE. In the prototype Spanish Writing Assistant, the corpora consist of a movie on videodisc and several paragraphs of on-line text taken from textbooks and magazines. We would eventually like to have a library of corpora for each language, including a range of materials appropriate to different levels of expertise. We also hope to be able to take advantage of resources, such as on-line video collections, which are planned for electronic libraries.


28

The corpora are accompanied by indexes of lexical items, grammatical constructions, speech act situations, and other topics. The indexes can contain commonly sought items or can be customized to suit specific curricula. Through an authoring system, it is possible for a teacher to choose sets of examples for presentation to students. The sets of examples can be chosen to illustrate situations in which a word or construction is used appropriately. For example, a Spanish teacher might select a set of examples illustrating the use of the subjunctive mood or the verb estar (be). Alternatively, the examples can be chosen to illustrate appropriate ways to perform a speech act in a given situation. For example, a teacher could select a set of examples that illustrate various ways


29

of ending a phone call with a friend or of apologizing to a boss for being late. We believe that facilities for exploiting examples in their natural contexts offer many advantages to an ICALI system. First such facilities make it very easy to find realistic examples of language use. A computer can store an extensive set of examples—many more than a person can have at his or her fingertips. Of course, the computer cannot replace a teacher as a communication coach. But the teacher can make the instruction of communication more effective by choosing good examples, annotating them with explanations, and designing an ICALI program that makes them available to students. The teacher can even use system-accessed examples in class for lessons in vocabulary, culture, and discourse situations (introductions, shopping, business meetings, etc.). Second, the facilities are designed to present examples to the student in the native context—excerpted from the videodisc or on-line text where they are found. In ALICE, we show a few minutes of a video or several paragraphs from an on-line text so that the student can see the context that determines that a lexical item,, construction, or particular way of performing a speech act is appropriate. We feel that presenting examples in context makes them more natural and realistic than examples that are displayed out of context in dictionaries and textbooks. Finally, the facilities in ALICE for indexing and selecting examples permit a teacher to turn virtually any on-line text or videodisc—for example, movies, newspapers, or novels—into effective instructional material, even if the material was not developed with computer-assisted language instruction in mind. Thus the growing amount of available on-line text and the growing numbers of foreign-language videodiscs can automatically become vast reservoirs of information for foreign language instruction. In the current ALICE system, all corpora are indexed by hand. The indexing procedure for videodisc involves someone watching the video and writing down the track numbers of scenes where various words, constructions, and discourse situations occur. Consequently, the indexes are small; in the Spanish Writing Assistant, approximately 60 excerpts from the corpora are accessible through indexes. In Section 5.1 we describe our plans for incorporating CLARIT, an automated indexing and retrieval system. Automated indexing should facilitate the creation of much larger indexes in a full-scale version of ALICE. One important role of the corpora in ALICE is to provide realistic examples to accompany reference material such as bilingual translation dictionaries, monolingual learners’ dictionaries, and other types of explanations and notes on various aspects of a foreign language. Instead of extracting an example from a text (or video or audio recording) and reproducing it in a dictionary entry, the dictionary entry can contain a pointer to the location of the example in its source. When the student accesses an example, a surrounding portion of the corpus is also displayed. The reference material in the Spanish Writing Assistant consists of a small dictionary and a small grammar menu. The dictionary contains about 50 entries, a few of which have pointers to excerpts in the corpora. A grammar menu provides access to a


30

collection of annotated examples of the use of both the present subjunctive mood, and direct and indirect object pronouns. The dictionary and grammar menu entries have been specially chosen to reflect the new vocabulary and grammar that students are learning at the point in the term when they use ALICE.2 Figures 2 and 3 illustrate how examples from corpora are combined with dictionary entries in the Spanish Writing Assistant. Each dictionary entry consists of several hypertext cards—a definition card for each sense of the word, example cards, culture cards, and picture cards. The definition cards show the citation form of the word, the part of speech, and a definition in Spanish. An English definition is available if the “show English” button is selected (this feature can be suppressed by the instructor). Figure 2 shows a definition card for one sense of denunciar (denounce). Each example card displays one sentence that contains the head word of the dictionary entry. An English translation of the sentence is available if the “show English” button is pressed. There are two video buttons on an example card, one that plays a short segment of video starting with the example sentence and one that plays a longer segment of video illustrating the context that leads up to the example sentence. In addition, the teacher can annotate the example with notes, which are available to the student by clicking on the “Notes” icon. Figure 3 shows an example card for the word denunciar (denounce). A picture card contains a video button and, optionally, a scanned-in picture. The video sequence associated with a picture card shows the object or action described by the lexical item, for example, a scene showing a political demonstration for the word


31

manifestación. A culture card contains a video button and notes that give some cultural background on the head word of the lexical entry. The video sequence associated with a culture card shows people talking about the object or action described by the lexical item, without necessarily using the word or showing it. The purpose of the picture and culture cards is to provide background on the meaning of the word as it fits into the foreign language culture. The current version of ALICE does not contain an authoring system. However, work will begin on an authoring system in Fall 1991. Our plans call for the authoring system to give the teacher the ability to create dictionary entries and other reference material, scan the indexes of the corpora and select excerpts to include in lessons, use the indexing software to index new examples in the corpora, format menus of annotated examples (for example, a menu of excerpts for the corpora that illustrate the use of the subjunctive mood) and on-screen tutorial information, and write interactive lesson scripts involving formatted exercises. A teacher using the authoring system will create programs in an authoring language. The controller is an interpreter for authoring system programs. It translates authoring system programs into screens that appear in the student interface. The controller also executes student commands for such tasks as searching the dictionary, checking grammar, executing menu selections, and playing video excerpts.


32

The student interface consists of a system of windows and menus for editing, grammar checking, using reference material, playing video excerpts, and running application programs. The current student interface consists of (1) an editing window where the student can write a text, (2) pull-down menus (Macintosh style) that allow the student to check grammar, analyze an inflected word, generate an inflectional paradigm, or look words up in a dictionary, (3) a hypertext learners’ dictionary with facilities to play indexed portions of a videodisc that illustrate lexical usage, and (4) menus of scenes from the videodisc that illustrate culture, use of grammatical constructions, and speech acts. In addition to the reference material and corpora, which students are free to examine and teachers are free to modify, ALICE includes a number of Natural Language Processing (NLP) components which students and teachers never observe directly (unless the teachers are trained in computational linguistics). ALICE’s current NLP programs consist of a sentence parser, a morphological analyzer, and a morphological generation program. In the Spanish Writing Assistant and in the Japanese grammar exercise application, NLP helps students with the syntax and morphology of a foreign language by checking for errors in grammar, identifying the citation forms of inflected words, and generating inflected forms of words. Semantic and pragmatic processing could be added to support more ambitious natural-language understanding or to check semantic and pragmatic constraints in order to improve error detection. The NLP components of ALICE are described in greater detail in Section 3. Each NLP program is a general rule interpreter. In order to process the syntax and morphology of a language, the NLP program must be supplied with machine-readable knowledge bases that describe the syntax and morphology of a language. The knowledge bases include grammar rules, morphology rules, and a lexicon that gives the syntactic and morphological properties of each word. We have extensive rules for analysis and generation of inflectional morphology for French operating on a 50,000 word lexicon, and a fairly complete inflectional morphology for Spanish operating on a small lexicon. There is a small set of grammar rules for each language (amounting to perhaps 50 rules each); syntactic coverage is limited in the current prototype systems. Given the resources described above, it is possible to create many types of application programs using NLP and realistic, in-context examples from text and video corpora. Part of the challenge is finding practical ways to use these resources in foreign language curricula. Through conversations with language teachers, we have arrived at the following suggestions for using the ALICE components in various types of assignments and classroom activities. In a grammar exercise application, natural language processing and morphological analysis can be used for error-detection. Morphological analysis can also help students look up inflected words in a dictionary, and indexed text and video corpora can be used for presenting examples of grammatical constructions. As a film teaching aid, ALICE can facilitate quick access to excerpts from a film, including individual scenes and


33

segments that illustrate culture, grammar, and vocabulary. Students can also be invited to explore such segments in an open-ended learning environment. In conjunction with a word processor, ALICE can serve as a writing assistant with fastaccess, on-line reference sources that include dictionaries, grammar information, and illustrative examples from indexed corpora. Parsing can be used to detect sentence-level errors and identify typical trouble spots, and morphological analysis can help students lookup inflected words in a dictionary. Generation of inflectional paradigms also helps students identify correct inflected forms. Fast, on-line reference capabilities can also be used with an on-line text that the user is reading. In this case, morphological analysis can help the novice reader identify unknown inflected forms of words. Finally, ALICE can be used for many types of in-class lessons, serving as a source of examples of vocabulary, grammatical constructions, and speech acts. III. Natural-Language Processing in ALICE This section describes the NLP components of the ALICE prototype. All programs, lexicons, and rules described here are implemented and working. Section 5 discusses future plans for augmenting the NLP capabilities. There are three NLP tools in the current version of ALICE: a morphological analyzer, an inflectional paradigm generator, and a sentence parser. Each of these tools consists of a general rule interpreter and knowledge bases of words and rules from various languages. In describing the NLP components, we will use examples from the Spanish knowledge bases, which include a core lexicon, a set of morphological rules for analysis, a set of morphological rules and inflectional templates for generation, and a set of grammar rules for parsing. III.1. Morphological Analysis The approach we take to morphological analysis distinguishes two types of morphology rules: allomorphy and concatenation. Allomorphy rules create alternate forms of morphemes. For example, the stem busc (from buscar (to look for)) has an alternate form busqu which is used with a suffix that begins with e as in busques. Inflectional suffixes also have allomorphs. For example, the initial unstressed i of the verb suffixes –ieron and –iendo will change to y after a vowel as in the suffix –yeron (cayeron (they fell)), or will be deleted after the palatal consonants ll and ñ to form –eron as in bulleron and gruñeron. Concatenation rules state the order in which morphemes can combine to form words. The Spanish concatenation rules we have written specify only the attachment of inflectional suffixes to nouns, adjectives, and verbs. Verbs may be followed by tense/agreement suffixes, whereas nouns, adjectives, and passive verbs are followed by number and gender suffixes. Some verb forms also allow the attachment of enclitic pronouns (e. g., dígamelo (=diga + me+lo, tell me it)). The morphological analyzer is


34

based on a Left-Associative Morphology program (Hausser 1989). Under this approach, all allomorphs of each stem are created when the lexicon is loaded into Lisp and are stored in an allomorphy lexicon. The purpose of calculating and storing allomorphs is to speed up morphological analysis. Using the allomorphy lexicon, the morphology program proceeds from left to right through a word, making sure that each morpheme is a possible continuation from the previous morpheme and that each morpheme is the correct allomorph in that context. Lexical entries exist at three stages: the core lexicon, the allomorphy lexicon, and the final output of morphological processing, when all of the morphemes in a word have been identified. At each stage, lexical entries have the format (SURFACE FEATURESTRUCTURE STEM). SURFACE is the surface form of a word or stem. SURFACE in a core lexical entry for a regular word is a citation form—infinitive for verbs, singular for nouns, and singular masculine for adjectives. In the allomorphy lexicon, SURFACE is a stem that is created by allomorphy rules. In the final output of morphological analysis, it is the concatenation of all morphemes found in the analysis of the word. FEATURESTRUCTURE is a set of feature-value pairs containing morpho-syntactic information about a word (e.g., verb conjugation class, noun gender). STEM is the citation form of a word that remains the same for all inflected forms. Thus both soy and era, which are SURFACE forms of the same verb, have the STEM ser (be). The knowledge bases used in creating the allomorphy lexicon are the core lexicon and a set of allomorphy rules. The core lexicon should ideally contain only one entry to represent each paradigm—for example, the infinitive of a verb, the masculine singular of an adjective, or the singular of a noun. However, it must actually also contain any forms that are not predictable from a rule. Figure 4 shows core lexicon entries for buscar (look for), soy (be), dormir (sleep), and teng (have). These illustrate different types of core lexicon entries. The entry for buscar is a regular verb entry. The canonical form (infinitive for verbs) is used as the SURFACE and STEM form for all regular verbs. The feature-structure contains the syntactic features (TANS and COMP-TYPE) and the part of speech (CAT). 3 Whereas buscar does have stem allomorphs, these do not have to be mentioned in the core lexicon entry because they are completely predictable from the spelling of buscar. The verb dormir, in contrast, undergoes a stem-vowel alteration that is not completely predictable from its spelling and that results in the three stem allomorphs: dorm-, duermand durm-. (STEM-CHANGE O-UE-U) in the lexical entry of dormir indicates that allomorphs can be generated by applying an allomorphy rule that changes o to ue and u. The remaining two lexical entries in Figure 4 contain an additional feature ALLOFLAG. This feature has two purposes. For a stem like teng, an allomorph of tener (have), it marks the suffixes that are possible continuations of the allomorph. For a word like soy, which is a completely inflected form of ser (be), ALLO-FLAG has a value AGRn (for n=1 to 50). AGRn indicates two things to the morphological analyzer—first, that no


35

("BUSCAR" ((CAT V) (TRANS 2) (COMP-TYPE NO)) BUSCAR) ("SOY" ((CAT V) (TRANS 1) (COMP-TYPE AP) (ALLO-FLAG AGR1)) SER) ("DORMIR" ((CAT V) (STEM-CHANGE O-UE-U) (TRANS 1) (COMP-TYPE NO)) DORMIR) ("TENG" ( (CAT V) (TRANS 2) (COMP-TYPE NO) (ALLO-FLAG (*OR* +SFX1 +SFX7+ SFX8+ SFX9+ SFX10+ SFX11+ SFX12+ SFX44+ SFX45+ SFX47)) (ENCLITICS 0) (CONJ-CLASS ER)) TENER) Figure 4: A Sample of Core Lexicon Entries suffixes should be added to this word because it is already a complete word and second, that a set of inflectional features should be added to the FEATURESTRUCTURE part of the entry. In this case, the features will indicate first-person singular subject, and present tense. Two additional features in the lexical entry of teng are CONJ-CLASS and ENCLITICS. The conjugation class (CONJ-CLASS ER) is present in the core entry so that the appropriate class of suffixes is added to the stem. (For regular verbs, this feature and its value are added by the allomorphy rules when allomorphs are created.) The featurevalue pair (ENCLITICS 0) indicates that this stem does not allow enclitic pronouns (the stem allomorph that would be needed for enclitics is téng- as in téngalo). Allomorphy rules (which are actually just Lisp programs) operate on core lexicon entries and produce an allomorphy lexicon that contains all allomorphs of the items in the core lexicon. This allo-morphy lexicon is created when the lexicon is loaded into Lisp. Figure 5 shows an allomorphy rule. This rule applies to verbs such as buscar that end in

(1) ((ends-in surf ace “CAR") (2) (list (replace-end surface “CAR” “C”) (3) (make-new-cat cat '(form core conj-class ar) ‘(*OR* +sfx1 +sfx2 +sfx3...)) ...) (4) (list (replace-end surface “CAR" “QU") (5) (make-new-cat cat '(form core conj-class ar) '(*OR* +sfx7 +sfx8 +sfx9 ...)) ...)))) Figure 5. An Allomorphy Rule


36

("BUSC” ((CAT V) (TRANS 2) (COMP-TYPE NO) (CONJ-CLASS AR) (ALLO-FLAG (*OR* +SFX1 +SFX2 +SFX3 +SFX4 ... )) ) BUSCAR) (“BUSQU” ((CAT V) (TRANS 2) (COMP-TYPE NO) (ALLO-FLAG (*OR* +SFX7 +SFX8 ...)) +SFX47 +SFX10 +SFXll +SFXI2 +SFX25)) (CONJ-CLASS AR)) BUSCAR) Figure 6. Examples of Derived Allomorphs c and are of the –ar conjugation class. The rule states that if a word ends in –car, two allomorphs should be created, one ending in c (line 2) and one ending in qu (line 4) (e.g. busc- and busqu- respectively). The rule also states that a new feature-structure for each allomorph should be created (lines 3 and 5) containing information about the form, the conjugation class, and the new list of suffixes that can concatenate with the allomorph. The allomorphs created by this rule are shown in Figure 6. The feature-structures for these allomorphs each have the feature ALLO-FLAG, whose value indicates which suffixes are possible continuations. During morphological analysis, a word such as busco is broken down into a stem allomorph and one or more suffix allomorphs. Lexical entries of stems such as busc contain syntactic features that do not change when the word is inflected, for example, gender for nouns and transitivity for verbs. Lexical entries for suffixes such as -o contain inflectional features such as number, person, tense, and mood. When the morphological analyzer recognizes a word (by searching a trie-structure, as described below) as a particular stem followed by a particular suffix (e.g., busc- followed by -o), it creates a lexical entry that contains all of the syntactic features of the stem and affix. Figure 7 shows the lexical entry for the suffix -o. Figure 8 shows the output of morphological analysis applied to busco. The entry for busco consists of the surface form BUSCO, the

("O” ((CAT V-SFX) (ALLO-FLAG AGR1)) *SUFFIX*-l) Figure 7 (above). The Lexical Entry for the Suffix -o Figure 8 (below). The Result of Morphological Analysis of Busco ("BUSCO” ((ALLO-FLAG AGR1) (MOOD IND) (FORM FINITE) (TENSE PRESENT) (AGR ((NUMBER SG) (PERSON 1))) (CONJ-CLASS AR) (TRANS 2) (COMP-TYPE NO) (CAT V)) BUSCAR)


37

feature-structure (with the inflectional features corresponding to AGR1) and the citation form of the word, BUSCAR. It also contains the feature-value pair (ALLO-FLAG AGR1) to prevent the attachment of additional inflectional suffixes. For efficient storage and search, the allomorphy lexicon is stored in a triestructure in Lisp. The trie-structure is a tree that contains an alphanumeric character used in a Spanish word at each node. Words beginning with the same series of characters will share part of the trie-structure. For example, because the first three characters of the words banana and banco are the same, the triestructure organizes the lexicon so that these two words would have the same search path through the trie-structure up to the fourth character. Figure 9 shows a representation of the part of the trie-structure containing several words including banco and banana. The path for banco is highlighted. During the process of dictionary lookup, the morphological analyzer must identify all of the morphemes (stems and suffixes) in the input string. This is done by searching through the trie-structure lexicon using the input word. The nodes that mark the end of a morpheme in the trie-structure have a lexical entry associated with them. Thus, when one of these nodes is encountered, the processor has located a morpheme. When a stem is found, the processor uses the remaining characters of the string to begin a new search through the trie-structure looking for more morphemes. When the input is consumed, the process stops. At each point where a morpheme is found, the processor consults the concatenation rules to see if the morpheme is a permissible continuation of the preceding morpheme. The concatenation rules also specify the FEATURE-STRUCTURE and STEM that result from the concatenation of the particular morphemes. The morphological analyzer and the Spanish rules and core lexicon provide complete coverage of Spanish inflectional morphology for verbs, including accent placement in

((CAT V) (ROOT BUSCAR) (TRANS 2) (CONJ-CLASS AR) (COMP-TYPE NO)) Figure 10. Features Common to all Forms of Buscar


38

non-finite verbs with enclitic pronouns (e.g., notifiquesemelo (notify me about it)). III.2. Paradigm Generation The paradigm generation capability of ALICE enables a student to see inflected forms of a word. To start paradigm generation, the student selects a word and specifies which forms he or she wants to see. The first step in paradigm generation is to send the selected word through morphological analysis to determine its citation form. For example, if the student selected busque, the morphological analyzer would determine that it was a form of buscar. Then, a feature structure containing only features of the citation form (i.e., with no inflection features) is created. Figure 10 shows the features of the citation form for buscar: (CAT V) indicates that it is a verb; (ROOT BUSCAR) indicates that it is a form of buscar; (TRANS 2) indicates that it is transitive; (CONJ-CLASS AR) indicates that it belongs to the conjugation class which takes the –ar inflectional suffixes; and (COMP-TYPE NO) indicates that it has no other complements. Part of the Spanish knowledge base for paradigm generation is a set of fifty feature structures, each representing one member of a verb paradigm. These feature structures contain only inflectional information. The next step in paradigm generation is to retrieve a subset of the fifty feature structures representing the inflected forms that the student wants to see. If the student requested “present tense indicative”, the feature structures shown in Figure 11 would be retrieved. Each feature structure contains the feature FORM who value indicates the finiteness of the verb, the feature AGR whose value expresses agreement information, the feature TENSE, and the feature MOOD whose value indicates whether the verb is subjunctive or indicative. We use the GenKit sentence generator (Tomita & Nyberg 1988) as the principal naturallanguage processor to generate inflectional paradigms. GenKit takes a feature structure (“BUSCO” “BUSCAS” “BUSCA” “BUSCAMOS” “BUSCAIIS” “BUSCAN”) Figure 12. Paradigm Generation of the Present Tense Indicative of Buscar


39

as input and produces a natural-language construction (word, phrase, or sentence) as output. In order to generate members of the paradigm, the feature structure of the requested root word in Figure 10 is unified with each of the inflectional feature structures in Figure 11. Then GenKit uses a knowledge base of rules to generate Spanish words from the resulting feature structures. The rules instruct GenKit to select the correct stem allomorph and concatenate it with the correct inflectional allomorph. The result is a set of Spanish words (Figure 12).4

;;GRAMMATICAL PARSES ;; ( (x2 tense)) ((x2 syn-error subj-verb-agr) > (x1 value)) ((x2 syn-error subj-verb-agr) > (x2 value)) ... (x0 = x2) ((x0 subj) = x1) )) Figure 13. An Example of Grammar Rules

Figure 14. An Example of Grammar Output ((S-TYPE DECLARATIVE) (SUBJ ((AGR ((GENDER MASC) (NUMBER PL) (PERSON 3))) (DET ((AGR ((GENDER MASC) (NUMBER PL) (PERSON 3))) (CAT DET) (ROOT EL) (VALUE LOS) (REF DEF) (FORM LOS))) (CAT N) (ROOT SOLDADO) (VALUE SOLDADOS) (COUNT +) (ANIMACY 1))) (SYN-ERROR ((SUBJ-VERB-AGR (*MULTIPLE* TORTURABA SOLDADOS INDICATIVE IMPERFECT)))) (AGR ((NUMBER SG) (PERSON 3))) (MOOD IND) (FORM FINITE) (COMP-TYPE (*NOT* NP AP X)) (ROOT TORTURAR) (OBJ ((CASE ACC) (CLITIC PRE) (CAT CL1) (ROOT PRO) (VALUE LA) (AGR ((NUMBER SG) (PERSON 3) (GENDER FEM))))) (TRANS 20 (CAT V) (VALUE TORTURABA) (ALLO-FLAG +SFX15) (TENSE IMPERFECT) (CONJ-CLASS AR))


40

III.3. Parsing and Error Detection Parsing is carried out by the Tomita parser (Tomita et al. 1988) along with two knowledge bases—a set of Spanish grammar rules, including rules for parsing erroneous sentences, and a set of feedback templates that are filled in when errors are detected. Figure 13 shows two unification-based grammar rules for parsing sentences, both consisting of a noun phrase followed by a verb phrase. Each grammar rule consists of a context-free phrase structure rule and a list of equations. The equations are instructions to the parser to build a feature structure (set of feature-value pairs) for the sentence. In Figure 13, both rules contain the context-free phrase structure rule indicative) ((x2 syn-error subj-verb-agr) > (x2 tense)) ((x2 syn-error subj-verb-agr) > x1 value)) ((x2 syn-error subj-verb-agr) > x2 value)) These equations are used for adding information to the feature structure about the subject-verb agreement error. They cause the following line to appear in the feature structure in Figure 14.


41

(SYN-ERROR ((SUBJ-VERB-AGR (*MULTIPLE* TORTURABA SOLDADOS IMPERFECT INDICATIVE)))) SYN-ERROR is the feature used to keep track of syntactic errors, and SUBJ-VERB-AGR indicates that the type of error is subject-verb agreement. The value of SUBJ-VERB-AGR is a set of symbols that represent the relevant information in the sentence that is to be used in the error message, including the verb, subject, verb’s tense and verb’s mood. In order to provide feedback to the student, a template for subject-verb agreement errors is retrieved (Figure 15) and filled in (Figure 16). The resulting message is displayed on the student’s screen.

(subj-verb-agr "The verb *x1* does not agree with the subject *x2* in the *x3* *x4* tense." 4) Figure 15 (above): An Error Template Figure 16 (below): An Example of Error Analysis/Feedback "The verb TORTURABA does not agree with the subject SOLDADOS in the IMPERFECT INDICATIVE tense."

Currently, the grammar has 73 rules and detects errors in agreement, verb forms, complementizers, contractions, the personal a, and NP case. Sentences such as those given in Figure 17 are typical of sentences that are covered by the current grammar. To conclude this section, we note three caveats about the prototype error detection mechanism described here. First, the approach that we have taken is one of error prediction. Special grammar rules are written for parsing sentences with frequently occurring errors. When the ALICE parser encounters an error that was not predicted and included in the grammar, the parse will fail. However, no diagnosis can be given in such cases because the parser will also fail on many grammatical sentences that are not covered by the grammar. Second, we are simply detecting sentences that are ungrammatical and not diagnosing the source of the error. In many situations it is possible to tell that a rule was broken, but a diagnosis cannot be obtained without knowing what was intended. Third, this mechanism for error detection is not very robust. It will only detect errors in sentences that it can parse completely. Future plans for a more robust error detection mechanism are discussed in Section 5.2.


42

Figure 17. Sentences Illustrating Grammar Coverage 1. Accent Errors (1) Él estudia mucho. El estudia mucho. He studies a lot. 2. Agreement with Possessive Adjectives (2) José busca sus apuntes. José busca su apuntes. José is looking for his notes. 3.Subject-Verb Agreement (3) Pepe Martinez estudió en México. Pepe Martinez estudié en México. Pepe Martfnez studied in Mexico. (4) El estudiante y yo vamos a estudiar los apuntes. El estudiante y yo voy a estudiar los apuntes. The student and I are going to study the notes. (5) También toca la guitarra y canta muy bien. También toca la guitarra y cantar muy bien. He also plays the guitar and sings very well. 4. Contraction of de and el (6) Busco el radio del avión. Busco el radio de el avión. I am looking for the radio of the airplane. (7) Él no quiere la foto del estudiante. El no quiero el foto de el estudiante. He doesn't want the photograph of the student. 5. Wrong Marker on Embedded Clause and Missing Personal a

(8) No queremos mirar al estudiante. No queremos a mirar el estudiante. We don't want to see the student.


43

IV. Error Treatment in ALICE Having described the components of ALICE, we now turn to a discussion of the types of student errors that can be effectively treated using the ALICE facilities for NLP and retrieval of excerpts from corpora. Error treatment in ALICE consists of error detection and/or exposure to correct forms. In this section we will discuss which types of errors are treatable by each of these two methods and which types of errors are not easily treated in ALICE. The treatability of an error in ALICE depends in part on whether it is an error in a rulegoverned or rule-elusive construction. For our purposes, there are two factors involved in classifying a phenomenon as rule-governed or rule-elusive: the extent to which linguists have been able to formulate complete rules that account for all occurrences of the phenomenon and the extent to which applying a rule depends on semantic, pragmatic, or cultural judgments that are themselves not clear cut. Of course, no construction is completely rule-governed or completely rule-elusive. Errors in constructions that are more rule-governed are more easily detected by parsing, whereas errors in constructions that are less rule-governed are not easily treated by parsing with error detection because is order to recognize something as correct or incorrect, it is necessary to have a procedure for distinguishing the two. Errors in less rule-governed constructions may not be automatically detectable in a student’s writing, but that does not mean that they are not targetable at all in an ICALI system. If the student can be made aware of a potential problem, his or her attention can be directed to examples of correct usage. The assumption is, of course, that exposure to correct usage will help eliminate the error. For errors in more rule-governed constructions, exposure to correct usage might not be as important. A final factor in the treatability of rule-elusive errors is the extent to which examples of correct usage can be automatically indexed in the text, audio, and video corpora. If identifying relevant examples in the corpora depends on subtle features of the structure of discourse whose automatic detection is beyond the state of the art, then treatment of errors in such constructions cannot be targeted by ALICE in the near future. In the remainder of this section, we turn to some examples of actual errors from essays of third-semester Spanish students. (Cf. Figures 18, 19, 20, 2, and 22.) We will comment on the degree to which each error can or should be treated with error detection or exposure to correct usage. Subject-verb agreement is largely rule-governed. The example in Figure 18 shows a typical error. For a student or a parser to check subject-verb agreement requires finding the subject and the verb and making sure that their morphological features match. (This does not, however, tell you why they do not match. A mismatch could be caused by a missing accent mark, a typographical error, or ignorance of the correct verb form.) This is usually possible for a student to calculate either consciously or unconsciously, allowing for a few exceptional cases. This is usually also possible for a parser unless some unanticipated sentence structure prevents it from finding the subject or the verb. As a result, many subject-verb agreement errors are detectable by parsing and it may CALICO Journal, Volume 9 Number 1

44

not be necessary to treat such errors by exposure to several correct examples; subject-verb agreement errors are almost always performance errors and a simple reminder about agreement might suffice.

(9) Los soldados la torturaba. The soldiers tortured her. ⇒ Torturaba should be torturaban. Figure 18. A Subject-Verb Agreement Error

The passage in Figure 19 illustrates, among other errors, awkward use of subject pronouns in a student essay. Since Alicia is the discourse topic, references to her should occur as null-pronouns rather than as the overt pronoun ella. Unlike subject-verb agreement, dropping of subject pronouns is a very rule-elusive phenomenon—to our knowledge no one has ever written a complete rule that describes exactly when to express subject pronouns overtly. Even if such a rule existed, it would contain clauses such as “express the discourse topic as a null pronoun” or “use an overt pronoun for emphasis.” For a student or a natural-language processor to apply such rules would depend on being able to pick out the discourse topic or identify the need for emphasis—procedures that are, themselves, not clearly rule-governed. As a result, awkward usage of subject pronouns is not detectable by parsing. Since dropping of subject pronouns is rule-elusive, it can be more effectively demonstrated by exposure to examples of correct usage than by explanation of an approximate rule. Thus it should be suitable for treatment by exposure to examples from the corpora. However, in order to index relevant examples automatically, we would have to be able to identify discourse topics and resolve reference of overt and null pronouns. This type of analysis is not feasible on a large scale with current NLP technology and, therefore, we do not consider treatment of dropped subject pronouns to be targetable in ALICE. Figure 19. Awkward Use of Subject Pronouns In the following passage, Gaby is Alicia’s adopted daughter. Alicia found out that children were being sold around the time that her husband Roberto adopted Gaby. Roberto refused to tell Alicia how Gaby was adopted. Native speakers of Spanish feel that at least the italicized instances of the pronoun ella referring to Alicia should be dropped. (10) “Alicia empezó a pensar de Gaby y si ella era una nina desaparecida. Ella le preguntó a Roberto de Gaby, pero Roberto no le contestó directamente. Así Alicia empezó a buscar información de Gaby o su nacimiento. Finalmente, ella encontró a la abuela de Gaby. Ella trajo a la mujer a su casa, y Roberto estaba muy enojado cuando llegó a la casa. Ella finalmente se dió cuenta de la verdad de Gaby, y entonces ocultó a Gaby de Roberto. Alicia began to think about Gaby and whether she was a ‘disappeared’ child. She asked Roberto about Gaby, but Roberto didn’t answer her directly. So Alicia began to look for information about Gaby or her birth. Finally, she found Gaby’s grandmother. She brought the woman to her house and Roberto was very angry when he cam home. She finally realized the truth about Gaby and then hid Gaby from Roberto.


45

The examples in Figure 20 involve lexical selection errors. The verbs in these sentences do not express the meaning that the student intended. (Selection of a wrong noun seems to be much less frequent.) Some errors of this type are obviously the result of translation from English or the result of less-than-full understanding of a dictionary definition of the word. When a lexical selection error results in a subcategorization error, it may be detectable by grammar rules used in parsing. For example conocer que...(know that...) is easily detectable as an error because conocer should never occur with a complement clause.5 In other cases, a lexical usage error might be detectable, provided that the detection procedure has access to simple semantic selectional restrictions. For example, sobrevenir (happen/ensue) applies to natural disasters and other things not caused by humans. However, in most cases, lexical usage is rule-evasive, depending on context-sensitive semantic and pragmatic factors such as whether knowledge comes from recognition or understanding (relevant to distinguishing conocer from saber (know)) or whether a condition is temporary or permanent (relevant to distinguishing estar from ser (be)). Mastering the rule-elusive aspects of lexical selection requires developing sensitivity to the shades of meaning that distinguish lexical items. This can be enhanced by exposure to examples from corpora illustrating lexical choice in various situations. Since indexing a text by its lexical items falls within the state of the art, we can automatically extract many examples of the use of each lexical item. Thus some lexical selection problems are treatable by error detection and most are treatable by exposure to examples of correct Spanish sentences. Like the selection of verbs, the selection of prepositions presents problems for students. The first two examples in Figure 21 are types involving subcategorization errors that are Figure 20. Problems in Selection of Verbs (11) Alicia conoce que Gaby no es su hija, pero Roberto le dijo que la madre la regaló. Alicia knows that Gaby is not her daughter, but Roberto told her that the mother gave her away. ⇒ Saber should be used to express ‘knowing a fact.’ (12) Una lucha sobrevino. A fight ensued. ⇒ Sobrevenir is used for the sudden occurrence of events not caused by humans (e.g., earthquakes). (13) Era furiosísimo. He was extremely furious. ⇒ Estar (not ser) is almost always used with emotional states. (14) “La Historia Oficial” trató de la realización de la personaje principal (Alicia Ibañez) que todo no estaba como lo parece. “The Official Story” dealt with the main character’s (Alicia Ibañez’) realization that everything was not as it seemed. ⇒ Ser should have been used instead of estar.


46

Figure 21. Problems in Selection of Prepositions (15) Cuando él se peleó a Alicia, estuve segura que Roberto no era un esposo cariñoso y simpático. When he hit Alicia, I was certain that Roberto was not a caring and kind husband. ⇒ A should be en. (16) Así Alicia empezó buscar para información de Gaby o su nacimiento. Thus Alicia began to look for information about Gaby or her birth. ⇒ Buscar does not take the preposition para before its complement. (17) Un día a la reunión de amigas de Alicia, se encuentra a Anita, su amiga mejor. One day at a reunion of Alicia’s friends, Alicia finds Anita, her best friend. ⇒ A in the first error should be en. A in the second error should be con. (18) Por ejemplo, ella vivía en un buen país abajo de una situación ideal. For example, she was living in a good country under an ideal situation. ⇒ Abajo de should be en.

treatable by parsing. The second two examples are more rule-elusive cases treatable by exposure to correct usage. Tense and mood can be a source of errors (as given in Figure 22), especially if the inventory of tenses and moods or their distribution is different in the foreign language than in the student’s native language. Here again, errors are treatable by a combination of error detection and exposure to correct examples. Errors that can be detected include phenomena like (a) subjunctive mood in complements of verbs that never take subjunctive complements and (b) indicative mood in complements of verbs that always take the subjunctive. In other cases, choice of mood or tense depends on what meaning is being expressed or how a sequence of events is being presented. Such choices are rule-elusive because students must decide whether uncertainty or persuasion is being expressed (relevant to the choice of subjunctive or indicative mood) or whether an event in the past is being described a continuing or as a completed event (relevant to the choice of imperfective or preterite). It is within the state of the art of indexing to extract examples of verbs in various tenses, moods, and aspects. Therefore, tense, mood, and aspect are targetable for treatment by error detection in a few cases and by exposure to correct examples in the remaining cases. Our observations on the targetability of errors in ICALI systems are summarized and presented in Figure 23. We clearly do not wish to suggest that each type of error fits completely into one or another category; as we argue above, examples of virtually every error type can be assigned to every category. Rather, we mean to suggest that prototypical or expected types of errors of the sort we identify above will be generally classifiable as in the matrix in Figure 23.


47

Figure 22. Problems in Tense and Mood (19) Le menciona a Alicia que el govierno venda los niños de los prisioneros políticos. She (Anita) mentions to Alicia that the government sold the children of political prisoners. ⇒ The subjunctive should not have been used in the subordinate clause. (20) Ella también sospechaba que Alicia tenía una hija de una de las desaparecidas. She also suspected that Alicia had a daughter of one of the ‘disappeared’ women. ⇒ The subjunctive should have been used in the subordinate clause. (21) Todavía, Alicia no fue cierto que la vieja era la abuela verdadera. Alicia still wasn’t sure if the old woman was the true grandmother. ⇒ The imperfect tense should have been used since todavía implies a continuation of action. (There is also a lexical error here.) (22) Alicia no supo de dónde Roberto adoptó a Gaby pero, como una buena esposa, nunca preguntó a él decir la verdad. Alicia did not know where Roberto adopted Gaby but, like a good wife, she never asked him to tell the truth. ⇒ Supo—preterite tense (meaning ‘to find out’)—should be sabía—imperfect tense (meaning ‘to know’). (23) También, si Alicia sale de Roberto, ella podría casarse con un hombre honrado, cariñoso, y bueno en general. Also, if Alicia leaves Roberto, she will be able to marry an honest, caring, and generally good man. ⇒ The future tense should have been used in the subordinate clause to get the correct sequence of tenses.

We conclude this section with a note on the treatment of errors through exposure to correct examples. This method is designed for addressing aspects of language that are somewhat rule-elusive and, therefore, cannot be detected reliably by an error-detecting parser. Because these errors cannot be automatically identified in a student essay, we might ask how students will know to direct their attention toward examples from the corpora. The answer, we believe, is that students can be made aware of errors without complete and comprehensive error detection. This can happen in several ways. Students who are reflective about their own language acquisition process might realize that they are not sure about the appropriate use of various words and constructions and might actively seek out examples. Other students—less reflective, perhaps—may learn, in part, because they have specific reasons to access the corpora and reference material. As a lesson-specific heuristic, the instructor can advise students to check their use of trouble-spot words and constructions before handing in their essays. The students can be told to check their use of problematic words and constructions by looking at examples in which they are used correctly. In the future we expect to automate troublespot identification. This would differ from error detection in that it would show the student words and constructions that tend to be misused. For example, instead of detecting errors in the use of the subjunctive mood, the NLP programs would point out instances of the subjunctive mood. The student would then determine which of the trouble spots are correct and which are erroneous with the help of examples from the


48

corpora. As a general rule, when an error is detected, the feedback will suggest review of examples from a menu that illustrate proper use of the construction. Since we cannot detect all errors, the student will be advised not only to check the sentence in which the error was detected, but also to scan the essay for similar errors. This strategy was used in the MINA project and was found to be effective (Hull 1986). Thus, to draw a student’s attention to a rule-elusive error, it might suffice to parse or pattern-match with a heuristic rule that detects only a small percentage of instances of the error. V. Future Plans for Natural Language Processing In the near future, we expect to expand our NLP capabilities to include automated indexing of text and video corpora and more complete robust parsing and error detection. This section describes our plans for NLP programs that are not yet incorporated into ALICE. V.I. Indexing and Retrieval In order for corpora to be useful as sources of examples, we must be able to locate examples automatically. What is needed is an indexing program that can locate instances of lexical items, grammatical constructions, various topics of discussion, and various types of speech acts. In addition to indexing texts and scripts of videos, we will also index “meta-text”, or commentary on the text written by a native speaker. The meta-text might, for example, describe what is going on in a scene (e.g., argument, business deal, date) and describe contextual cues that a novice speaker might not notice (e.g., the style is polite, formal, informal, slang, etc.). Non-text corpora—such as pictures without writing or speaking, or video and audio tracks for which no script is available—may constitute the bulk of material eventually available to an ICALI workstation. Because non-text corpora can have meta-text associated with specified segments, the same indexing tools for text will facilitate access to information in non-text sources. Figure 23. Matrix of Problem Types Rule-Governed Phenomena

Rule-Elusive Phenomena

Directly Targetable in ICALI Systems

Simple Syntactic Phenomena (e.g., S-V Agreement, etc.)

Not Directly Targetable in ICALI Systems

Semantically-Based Morphological, Syntactic Phenomena (e.g., verb form in ‘factive’ complement clauses)

Locally-Determined Lexical Choices (e.g., lexical selection—verbs and prepositions; idioms; collocation) Globally-Determined Lexical or Syntactic Choices (e.g., structure of discourse, use of anaphora, shifts in reference and time, etc.)


49

We expect that all real text and all associated meta-text in the corpora will be indexed by the CLARIT indexing and retrieval system (Evans 1990, Evans et al. 1991). The CLARIT system supports a variety of approaches to indexing that will help us control access to the language corpora. At the simplest level, CLARIT can provide an index of morphemes (free and bound) and morphological features such as past tense, subjunctive mood, and nominative case. CLARIT can also perform a partial parse of the text and extract noun phrases which can then be used as index keys. Future versions of CLARIT will include indexing of other types of phrases as well. At a higher level of complexity, CLARIT can retrieve texts that are about a specified topic, where a ‘topic’ is characterizable in terms of associated text or terminology. In order to do this, CLARIT uses a collection of thesauri. Each thesaurus is simply a list of terms that are typical of some particular topic. For example, there could be a thesaurus of terms related to politics or a thesaurus of terms commonly found in greetings. To analyze the content of texts, several steps are required. First, lists of morphemes and noun phrases are extracted. These lists are then compared to thesauri on various topics. Each thesaurus produces a profile for each text based on a ranking of the terms in the text. After such processing, a text can have multiple profiles—one for each thesaurus (e.g., “politics,” “greetings,” etc.) it is compared to—which provide a perspective on the text. Each profile consists of a ranking of terms in the text that match or partially match terms in the thesaurus. The ranking of terms is based on their frequency in the text and ‘fit’ to terms that are known to be characteristic of the domain. These rankings can be used to retrieve texts on different topics. For example, to find texts about assassinations, one can search for texts whose politics profiles include high rankings for terms related to assassinations. The rankings also help to distinguish between texts that are superficially about the same topic, or texts where target topics (indicated by terms) are only superficially represented and those where the topic is well represented. We can also use CLARIT to search for examples of discourse-level features, such as whether examples of formal speech exist in a text. As with general profiles, this ability is supported by processing a text and comparing results against a thesaurus—in this case, a thesaurus of formal speech. The terms that match or almost match terms in the thesaurus are ranked according to frequency in the text and ‘fit’ to terms that are known to be characteristic of formal speech. Texts with a high number of matches and nearmatches to terms in a formal speech thesaurus can be assumed to contain examples of formal speech. In general, such an approach can be extended to ‘tag’ texts for many kinds of discourse-levels or linguistic features. V.2. Extensions of Parsing and Error Detection Using NLP to flag errors has numerous advantages, such as flexibility, extensibility, and comprehensiveness. However, many kinds of errors in natural language will be difficult to detect, and even simple sentences may present multiple possible analyses. Indeed,


50

error detection is hard even for people. Therefore, any ICALI system that employs error detection must find a practical means of exploiting NLP without falling victim to its limitations and without relying on it exclusively. Our future plans for increasing the robustness of error detection, therefore, involve a combination of strategies and techniques. In formulating our error-detection strategies, we take advantage of two observations about student behavior. First, it is not necessary to find every error (in a category) to get students to correct their mistakes. Second, if students are asked to check for mistakes f a likely type, they will be successful in finding them. Thus, our future strategy for automated error detection is based on the idea that error detection can be effective even when only a small fraction of the errors in a text are detected. Once the student is sensitized to these errors, he or she can look for other similar errors that were not detected. This strategy has two heuristics: 1. Detect Errors with a Low False-Positive Rate, and 2. Flag All Known Trouble Spots Preemptively. Examples of low false-positive rate errors (in our grammar) include gender agreement and accent placement. Examples of trouble spots (for Spanish) include subjunctive mood, preterite and imperfect past tenses, and lexical items such as those illustrated in the samples from student essays in Section 4. Bearing in mind the distinctions between targetable/non-targetable and rulegoverned/rule-elusive phenomena (Section 4), we will match techniques to problem types. For simple syntactic phenomena, we will employ focused NLP-based error detection (of the sort we have illustrated in Section 3). For more complex constructions and for locally-governed lexical choice problems, we will employ pattern-matching with heuristic rules. (Notes that pattern matching can provide a back up even for NLP detectable rule-governed errors.) Finally, for non-targetable phenomena, we will employ predictive checking—asking the student to search for possible errors given a collection of examples that illustrate the error type—and directed exploration of resources—asking the student to search the corpora for specified situation types, which will be relevant to the usage choices the student may be asked to make in an assignment. We will implement two tactics for error-detection using NLP: 1. Parse with Relaxed Constraints Written into the Grammar Rules; and 2. Use Pattern Matching to Focus Processing. The parser will parse expected sentence structures with or without anticipated errors. Sentences that are garbled in some unexpected way, or that are grammatical but are not anticipated by the grammar rules, will simply not be parsed. We are not working on recovering from failed parses. Even if part of the sentence succeeds before the parse fails, we will not be able to detect errors in the part that succeeded.


51

Pattern matching is not entirely adequate for natural-language processing because natural-language syntactic generalizations depend on the grouping of words into constituents, not just on their superficial linear order. For example, subject-verb agreement depends on the head of the subject constituent’s agreement with the verb will not work infallibly. Checking the agreement of the first noun and first verb will also not work. The pattern matcher will not attempt to analyze sentence structures completely, but will work on the basis of heuristics. It might be able to trap simple cases of subject-verb agreement that are easily caught with linear order of words (e.g., pronoun-verb). Figure 24 gives examples of patterns that can be used by a pattern matcher to detect errors. The patterns are in the notation of regular expressions. Figure 24. Examples of Heuristic Rules for Pattern Matching. • conoc* que Find a word that starts with the letters conoc and is followed by the word que. • de el Find the word de followed by the word el. • factive-verb (not que)* que (not verb)* subjunctive-verb Find a factive verb followed by any number of words that are not the word que followed by the word que followed by any number of things that are not verbs followed by a subjunctive verb.

The role of the pattern matcher will be to detect errors in sentences that cannot be parsed completely. Because it will use heuristics that are based on word order, the pattern matcher will not detect a high percentage of any given type of error, especially since the patterns must be written conservatively to avoid matching false positives. Pattern matching can be effective in error detection if it is, in fact, true that it is only necessary to point out a few errors of each type. VI. Second-Language Acquisition and Instructional Design for ICALI In addition to being a platform where students can explore a foreign language, we hope that ALICE will also be a vehicle for extending our understanding of how people learn foreign languages. We plan to use such deeper understanding to make improvements in the system. We conclude this paper by offering suggestions for bootstrapping an ICALI system by incrementally revising it based on observations of how students interact with it. The observations will primarily be in the areas of second-language acquisition and instructional design for ICALI.


52

ICALI brings into the student’s experience many new types of instructional material that were not available in the traditional language curriculum. Because interaction with a computer might be different from interaction with a teacher, a traditional language lab, or a textbook, we might have to discover new principles for design of effective instructional material for ICALI. Conveniently, ICALI differs from other instructional media in that the system that delivers instruction can also be used to collect data about its own effectiveness. This can be done by collecting keystrokes to follow the student’s sequence of actions and by storing student essays and other student input. Analysis of this data will reveal whether students are using the system in the way that the system designer anticipated, whether frequency of certain types of foreign language errors decrease over time, and whether use of certain foreign language constructions increases over time. The design of the ICALI system can be adjusted accordingly. In addition to collecting data on the effectiveness of instructional design, it is also possible to use an ICALI workstation as a source of second-language data that can contribute to the field of second-language acquisition in general. A computer-assisted instruction system can be an ideal medium for collection of second-language data because the data can be collected during the learning process itself, rather than during an isolated psychology experiment. A specific area of instructional design and second-language acquisition that we hope to explore with ALICE is the balance between exposure and explanation in secondlanguage acquisition. The design of ALICE implicitly assumes that errors will decrease as a result of exposure to text or to video excerpts that illustrate proper usage. This assumption follows the received wisdom that exposure contributes more to language acquisition that do explicit explanations of foreign language constructions. However, we are reluctant to assert that explicit instruction is useless. It may be that instruction speeds the process of acquisition by directing the learner to notice language phenomena that are difficult to observe without training or well-defined linguistic categories. By creating explicit, conscious approximations to what will ultimately be unconsciously acquired, the student may facilitate his or her language acquisition. Thus, it will be desirable to collect data from students using ALICE in order to measure the effectiveness of explanations. On that basis, it may even be possible to construct a language of explanation that does not lead to misunderstanding—in particular, a metalanguage optimally suited to the language experience that students gain through examining the corpora. Research issues such as these underscore questions of ALICE development and design. Should the corpora be annotated with explanations? What terminology is appropriate when describing rules, annotations, or errors? Should students be allowed to explore corpora freely? Should they only be allowed to see carefully constructed contrast sets of examples? How should such sets be collected and built? Such design questions can only be addressed with empirical results from studies of ALICE in actual use. Ultimately, ALICE itself is the tool we need to broaden our understanding of ICALI design and to guide the development of future generations of ALICE.


53

Credits and Acknowledgements The ALICE Project is co-directed by David A. Evans and Lori S. Levin. The Spanish Writing Assistant Program was written by Donna M. Gates at the Center for Machine Translation at Carnegie Mellon University, using NLP components developed at the Center for Machine Translation and the Laboratory for Computation Linguistics. Susan Polansky (Modern Languages Program, CMU) designed the Spanish Writing Assistant application for use in her third semester Spanish class. Ted Fenton of the University Teaching Center has also given us valuable advice on instructional design. Programming support for the Macintosh was provided by the School of Humanities and Social Sciences. The French and Japanese ALICE systems, not described in detail here, are being developed in the Laboratory for Computational Linguistics. Kathy Baker, Laurent Delon, Anna Espunya-Prat, Greg Fox, Steve Handerson, Mitzi Morris, Noriko Nagata, and Sono Takano-Hayes have assisted with various tasks in the ALICE Project. We have benefited from discussions with Nina Garrett and members of her 1989-90 “Integration of Theory, Teaching, and Technology” seminar at CMU. Willard Daetsch has been an enthusiastic and constant supporter of our efforts. We would also like to thank Alan Bailin, Robert de Keyser, and David Farwell for helpful comments on an earlier version of this paper.

NOTES 1

At present, AI workstations are used for development of the linguistic knowledge bases and NLP programs. 2 The Spanish Writing Assistant was designed to be used toward the end of the third semester of college Spanish for an assignment that involves writing an essay about the Argentine movie The Official Story. 3 TRANS and COMP-TYPE describe the verb’s categorization. The value of TRANS indicates whether a verb is intransitive, transitive, or bitransitive. The value of COMPTYPE describes the verb’s casual and predicative complements, if any. CAT is a feature that takes a syntactic category such as noun or verb as its value. 4 Accents are internally represented as numbers following the accented letter. “Busca1is” in Figure 12 is the internal representation of buscáis. 5 The words conocer que could also result from omitting words from the constructions conozco al que rompió la ventana (I know he who broke the window) and conozco el que me has dado (I know that which you have given me). Though for beginning students these are less obvious sources of errors than confusion of conocer and saber.


54

REFERENCES Evans, David A. 1990. “Concept Management in Text via Natural-Language Processing: The CLARIT Approach.” Working Notes of the 1990 AAAI Symposium “Text-Based Intelligent Systems.” Stanford University, 93-95. Evans, David A., Ginther-Webster, Kimberly, Hart, Mary, Lefferts, Robert G., Monarch, Ira A. 1991. “Automatic Indexing Using Selective NLP and First-Order Thesauri.” RIAO ’91, Autónoma University of Barcelona, Barcelona, Spain. 624-644. Hausser, R., 1989. “Principles of Computational Morphology,” Technical Report, Laboratory for Computational Linguistics, Carnegie Mellon University. Hull, G. 1986. Using Cognitive Research and Computer Technology to Improve Writing Skill (Grant 830-0355), Learning Research and Development Center, University of Pittsburgh. Tomita, M., Mitamura, T., Musha, H. & Kee, M. 1988. “The Generalized LR Parser/Compiler Version 8.1: User’s Guide,” CMU-CMT-88-MEMO, Center for Machine Translation, Carnegie Mellon University. Tomita, M. & Nyberg, E.H., 3rd, 1988, “Generation Kit and Transformation Kit Version 3.2: User’s Manual,” CMU-CMT-88-MEMO, Center for Machine Translation, Carnegie Mellon University.

AUTHORS’ BIODATA Lori S. Levin received a B.A. in Linguistics with a minor in Mathematics from the University of Pennsylvania in 1979 and a Ph.D. in Linguistics from M.I.T. in 1986. Currently, she is a Research Scientist at the Center for Machine Translation at Carnegie Mellon University and a member of the faculty of the Computational Linguistics Program. Levin’s major research areas are the structure of the lexicon in Lexical Functional Grammar, knowledge-based machine translation, and intelligent language tutoring systems. David A. Evans received an A.B. in German Intellectual History (Geistesgeschichte) in 1971, a B.S. in Mathematical Sciences in 1975, and a Ph.D. in Linguistics (with specialization in Computational Linguistics) in 1982, all from Stanford University. He is currently Associate Professor of Linguistics and Computer Science at Carnegie Mellon University, with appointments in the Departments of Philosophy and Computer Science. He is also Director of the Laboratory for Computational Linguistics and Director of the Computational Linguistics Program. Evans’ research includes aspects of linguistics, cognitive science, and artificial intelligence, especially natural-language processing and knowledge representation. His work focuses on information processing, medical cognitive science, and intelligent language tutoring systems.


55

Donna Gates received a B.A. in Hispanic Languages and Literature with a minor in Linguistics from the University of Pittsburgh in 1984. She is currently a Research Technologist at the Center for Machine Translation at Carnegie Mellon University. Her major interests are Spanish linguistics, knowledge-based machine translation, and intelligent language tutoring systems. AUTHORS’ ADDRESSES Dr. Lori S. Levin Center for Machine Translation Carnegie Mellon University Pittsburgh, PA 15213 [email protected]


Dr. David A. Evans Laboratory for Computational Linguistics Carnegie Mellon University Pittsburgh, PA 15213 [email protected]

56