using a visual palette of letters associated with icons and synthetic ... integrated in and interfaced with common system functions (e.g. access to a printer and the underlying file .... and so the phonetic quality in detail is low. This is true for ...
Paper submitted to ITS 2000, Montreal, June 19-23, 2000
"Today’s Talking Typewriter" Supporting Early Literacy in a Classroom Environment Frank Tewissen, Andreas Lingnau, H. Ulrich Hoppe Dept. of Computer Science and Education, University of Duisburg Lotharstr. 65, 47048 Duisburg, Germany {tewissen, lingnau, hoppe}@informatik.uni-duisburg.de
Abstract In this paper we present a specific phonics based approach to supporting early literacy. The application system is cooperative and embedded in a "computer-integrated classroom". It provides an interactive writing environment using a visual palette of letters associated with icons and synthetic speech feedback for the writing results. Ongoing developments enhance the intelligent support modules for the initiation of partner or group work and for the provision of context sensitive individual feedback. Keywords: early learning, roomware, cooperative systems, interactive learning environments
Introduction Today, early learning in primary schools is often characterised by self-controlled work in rich and pleasant learning environments. The concepts of such rich and stimulating learning environments and less teacher centred classroom activities are rooted in the general ideas of modern pedagogy (e.g. [5]). Children work in different partner and group constellations in classroom environments, that present a "menu" of different activities, tasks and cooperation modes. Children select from these "menus", i.e. they choose the task that they want to carry out on a day. Teachers prepare these rich task or activity environments, and assist and manage the distributed and cooperative activities in the classroom. Our project "Networked Interactive Media In Schools" (NIMIS, cf. [8]), as part of the European ESPRIT subprogramme "Experimental School Environments", aims at supporting early learning in primary schools by augmenting the classroom physically, media-wise and procedurally using networked digital media. So far, it is the most complete and consequent implementation of the idea of a "Computer-integrated Classroom" or CiC (for a first notion, cf. [2]) that we have developed (see Fig. 1 ). As for the content of learning, the project is targeted at supporting literacy-oriented activities in the age group of 5-8 years old. In this context, three applications have been developed based on an integrated desktop environment for young children. One application ("T’rrific tales") aims at supporting collaborative story telling in a cartoon format. A second application ("Teatrix") aims at promoting collaborative acting in 3D scenarios. In the following we focus on a third application also integrated in the CiC framework, "Today’s Talking Typewriter" (T3). T3 provides a phonics-based learning environment for the acquisition of initial reading and writing
skills. Using T3, children can freely and flexibly form their own words with appropriate degrees of cooperation support and intelligent help. The T3 application (and all NIMIS applications) is embedded in distributed classroom activities, i.e. it is not designed as a stand-alone tool but as a customisable and user-adaptive tool integrated with a network of computers and human agents.
Environment NIMIS’ CiCs aim at supporting scenarios that are common in the existing practice of the primary schools associated to the project. They envisage a smooth interplay of human and electronic interaction; environments and software have been designed with this concept specifically in mind. For this reason the software should be flexible and usable in a variety of modes (individual, pair or group). Intelligent support may be used as appropriate. Teachers and children may decide that children remain in one place for an activity or they may choose to move around.
Fig. 1
A NIMIS CiC: First graders and teachers in action
Embedment The hardware and furniture in the NIMIS CiCs has been specially designed or selected under the consideration of existing classroom culture. A big interactive and height adjustable screen replaces the chalkboard. Pen-based input facilities (interactive LC displays integrated in special tables) provide intuitive interfaces for the children. The computers as such are almost invisible (hidden or placed in a separate chamber), information technology is a means and not the subject of learning in the CiC framework. Development of software applications for early learning needs to take into account open forms of learning and interacting, and the varied roles teachers and learners in different contexts.
From a long term perspective, educational tools, material and media will be rated according to their usability in these rich creative environments. We think that the development of a computer support for these scenarios should consider some central demands in a certain order of priority. It should • be conceived as an integrated set of tool-like interactive applications conceptually embedded in existing classroom activities, • be easily customisable by the teachers, • have flexibly adjustable and robust cooperation modes, • provide individual intelligent feedback, and • provide intelligent support for group and partner work. The T3 application is embedded in a specially designed software desktop for children. It is integrated in and interfaced with common system functions (e.g. access to a printer and the underlying file system). Media objects like sounds, images or words can be easily exchanged between different NIMIS applications via drag and drop operations. From a conceptual perspective, the tool character of an application is the first and most important design aspect. The software has to present itself similar to real physical learning material: Ready to use, easy to use, and adaptable to the concrete context of usage. Also the design of software, hardware and furniture ("roomware" cf. [11]) must be subordinate to curricular goals, and not vice versa.
Method T3 takes up and extends a well known method "Lesen durch Schreiben" (LDS, "Reading through Writing"), propagated mainly by J. Reichen (cf. [10]). This method is used in Switzerland and currently becomes more and more popular in Germany, too. The most important benefit of this method is, that children acquire reading and writing skills with their own pace applying already existing knowledge about characters, phonemes, vowels, etc. Independent from the different stages of cognitive development of the children in a class, each child can immediately start writing words or even smaller sentences. The method is called that way, because children start with the writing of words before they have the ability to read. Usually a child that just began with LDS can not read what he or she has written a few minutes ago. Initial reading skills are not trained. Experiences show that, after an individually varying amount of time each pupil starts to read without any external guidance. Typically they start with the recognition of specific words (e.g. their names, the word for the kind of animal they have, etc.). Every child in this phase is stimulated to read by the teacher. Usually after this point in time the child does not apply the method any longer. What comes next in the curriculum, is the transition to the orthographically correct spelling and the enhancement of motor skills by learning handwriting and writing with a fountain pen. In later school stages the children are stimulated by their teachers to write their own stories, to exchange them with partners, to review stories of others and to correct and rewrite the texts. Finally they publish their stories in special lessons for the whole class. Common to all applications of the method is the necessity of knowing the images and sounds on the phoneme table. Children must be sure about the presented images and the words and sounds related to these images before they start using it. This is especially important for some vowels in German because their are no unique and simple words that start with e.g. the umlaut "ö". Furthermore it is crucial for non native speakers to be sure about the phonetic value of the images and the characters because most of them will be different in their mother tongue.
A restriction of the method must be mentioned: The method can only be used in languages with a strong phonetic spelling, like most of the Roman languages or Turkish or German, for example. Unfortunately this is not true for English or French. The pronunciation of e.g. the sequence of letter "ough" in English has many phonetic variations (e.g. "tough", "though", "ought"). Therefore, the application of the method can only partially replace and support the traditional way of learning reading and writing in these languages.
Usage According to the LDS method, children start to write words and later simple sentences by composing each word from single letters. Having a phonetic representation of a word in mind, the children try to compose the written word letter after letter. The basic tool that is used to recognise, write and remember the shape of characters is a table made up by characters and images (phoneme table). Each phonetically important letter is listed on these tables with its upper and lowercase variants accompanied by an image representing an object that begins with this letter. Fig. 2 shows the visualisation of the letter and the associated sound "a" together with two images that represent the two variants of the German sound. In this case the images of the apple and the ant are standing for the short and long pronunciation of the letter "a".
Fig. 2
Phoneme representation
Each sound of a word about to be written is now checked against the table. If an appropriate sound is found, the symbol is written by the child. Actually this "symbol" is a letter, but this is not the children’s point of view in the early stage. For example the children might be searching for an "i" sound and scanning through the table they may find an image of an island. Once they recognise, that "island" starts with the sound "i" (which is also true for the German "i" and "Insel") they know that they have to write the symbol, i.e. the letter right beside the island image to "write" this specific sound. Depending on the methodology the teacher follows, they might get immediate feedback about the phonetic correctness from the teacher or they first finish the complete word. Using the T3 application the children can select letter sounds and associated letters and pictures from the phoneme table and combine them using a drag and drop technique. Combined with pen-based input facilities, the handling is very natural and similar to printing letters or selecting real physical letter objects from boxes. To copy or to write a letter the letter itself or the associated image has to be dragged from the table and dropped into the workspace they write in. This workspace replaces the normal paper material that they would use without computer support. Fig. 3 shows the main parts of T3’s user interface. The German phoneme table on the right side (using different colours for consonants, short and long vowels), a page of prepared words together with their visual representation, and the result (most of the words are already phonetically correct) of a child in the writing board on the left. Different to the non-computerised way of writing with this method, the children do not need to create the letter on their own (i.e. motor skills needed for writing characters are not trained). The effect of this feature depends on the curricular embedding; there is no negative effect of any kind, if the motor skills are trained in subsequent stages, e.g. by preparing for a so called "Fountain Pen License".
Auditive Feedback One of the most important features of T3 is, to provide almost immediate audible feedback of the children's writings. In the non-computerised classroom practice the important feedback
cycle starts with the exact pronunciation of the writing by the teacher. Using T3, children can press the speak button whenever they wish to hear the complete text or the single word that they were most recently working on. T3 uses a Text To Speech (TTS) system to provide the feature of phonetic feedback. TTS systems are designed and implemented for speaking and emphasising correctly written texts inclusive punctuation in different languages. The main focus of the developers are computer based information systems e.g. hotlines and phone orders. In first place they provide a fast and easy understandable speaking. Second order goals concern more human-like voices and sophisticated mechanisms to enhance moods, gender or ages of the synthetic speakers.
Fig. 3
T3’s user interface: Children’s work, the phoneme table and a word bank page
The requirements for a TTS system which will be used in the classroom is different from the requirements for such activities. The results of writing processes in T3 are usually single words which are not spelled correctly. A TTS system normally cannot handle this input. The main goal for the application of a TTS system was, to give the pupils an idea of what they have written and make them able to detect failures on their own. T3 provides an enhanced text to speech (TTS) pronunciation to ensure that words are spoken in a phonetically correct way. A filter between the user interface and the TTS system has been implemented that uses many of the typical TTS parameters, like stress, speed, pitch or dynamic volume to make sure that it pronounces the children’s writings in a phonetic correct way. The exact pronunciation of very simple single words or even single characters is seldom necessary
and so the phonetic quality in detail is low. This is true for several TTS systems that have been tested. Nevertheless TTS systems fortunately also "understand" (i.e. they have an interface for) a phonetic transcription to trigger the underlying system more directly. Additionally T3 supports a number of word tables (see Fig. 3 ). Word tables enable children without an idea of what to write to chose or get inspired by these words. Also the teachers can prepare special thematic pages with a couple of words related to the overall topic of the environment’s learning offer. With a simple click on the word images, the word is spoken and transferred to the workspace. Children can also reject a selection or place several images of a word in their workspace. Once an image has been inserted the system knows about the word that the child is going to write and the child can compare the pronunciation of the correct word and the pronunciation of their own spelling. This pre-selection can also be used by intelligent support in different ways.
Collaboration Collaboration takes place in a classroom, also without computer support. However, computerised material, particularly shared workspace systems, introduce new forms of collaboration. In our approach to facilitating classroom collaboration, we have distinguished the following “working modes”: Table 1
material symmetric asymmetric
task symmetric asymmetric same material, same material, intentionally same roles different roles, e.g. peer helper distributed or extended material, distributed or extended material, intentionally same roles different roles, e.g. peer helper
ì
ó ö
All participants share the same material and tools and have the same goal. Different roles
may appear locally during the collaboration but are not predefined or anticipated by the system. Technique: synchronisation of children’s workspaces All participants share the same material and tools but pursue different goals. Different roles may be assigned initially or may evolve from different previous, as e.g. in reciprocal teaching ([9]). Technique: Same as but although the synchronisation mechanism does not need to know about the distributed roles, the intelligent support module may assess and exploit this information. Participants may use different material but have the same goal and role. A common application of this mode is, to split up the necessary material between the participants to induce collaboration. This can be characterised as a "jigsaw" design (cf. [1] for the origin of the idea, [3] for a technical approach). It is especially effective for introducing computer supported collaboration to novices. Technique: Distribute material (i.e. removing parts of the individual user interface) and synchronise workspaces. Participants may use different material. The material could be distributed (i.e. split up) between the application instances or additional material such as complete words, phrases or sounds could be introduced for some participants. This applies, e.g., to human tutoring scenarios in which the tutor has access to and makes use of supplementary collections of material. Additional circumstances to these different cases of task and material distribution are prepared exercises (i.e. workspaces initialised with certain collaborative tasks) and collaboration outside the system, e.g. joint work of two children with the same application instance on one computer.
ó
ì
ö
Intelligent Support As for intelligent support in a collaborative classroom scenario, we have to distinguish individually oriented and collaboration oriented support. On the individual level, T3 provides the child with visual and auditive feedback, and here intelligent support is based on implicit assumptions about the (relative) correctness of results and the knowledge of the individual user. Collaboration support is mainly focused on intelligently parameterised group formation (cf. [4], [12]). The external embedment and presentation of intelligent feedback to the user) can vary from changes inside the same environment the user is interacting with (e.g. changes in the user’s workspace) to virtual creatures acting as an interface between the user and the machine. In the NIMIS project we follow two different approaches of the visualisation and integration of intelligent support. The first approach is more implicit, i.e. the intelligent support module (or intelligent agent) acts for example as a (virtual) group member taking the role of a tutor, the role of a peer learner, the role of a player in a game-like application or the role of a mediator (cf. [6]). The technical communication with intelligent agents is based on the same principle of shared activity spaces as the person-to-person communication. The agent acts on predefined rules triggered by modifications in the environment itself. Also the agent’s feedback is integrated in the environment, i.e. it represents its results of analysis in smooth changes, movements and highlighting effects in the environment itself. We believe, that following this implicit approach we gain the additional feature of intelligent support without losing the explorative tool character of the application. In an explicit approach, animated or personalised agents appear in the form of artificial creatures that possibly talk to the users by speech synthesis or canned phrases. Although the appearance of creatures may be more disturbing and interrupting as in the implicit scenario, we hope to find a strong motivational aspect when children work with artificial creatures. Which kind of initiative - an active, agent-triggered or a passive, user-triggered - is more suitable for the children in the age group addressed here is still an open question. Fig. 4 shows an activity diagram of the usual practice of the LDS in a German primary school and the additional means for intelligent support by the system. The diagram consists of activity frames (diamond shapes) and attached changes of activities (thick arrows) and is based on discussions with teachers and initial classroom observations. The elliptical shapes contain descriptions of possible actions of intelligent support modules.
Correctness Words are treated as correct or incorrect with respect to their phonetically correctness. Correct spelling is not an issue in the beginning when using the LDS method. Omitted word endings or quiet consonant-vowel combinations are not important, as long as the phonetic "skeleton" sounds similar enough to the standard pronunciation of the word. Applications of the LDS method differ in the following aspects because of different strategies, stages of development or simply different ways of pedagogic integration: • phoneme table (different sets of characters and images, order of vowels and consonants) • correctness of written words (partial application of the method, different local accents) • accompanying feedback while writing (allow or forbid phonetic mistakes on input) • final feedback after a word is finished (simple yes / no feedback, active support)
Developing an idea
Proposing meaningful words Searching, selecting and preparing material Speaking and animating partial text
Writing with phoneme table
T3 or teacher reads loud Pointing out mistakes, missing or wrong sounds T3 or teacher emphasises phonetic mistakes
Polishing the writing, storage
Archiving written texts and learner profiles
Fig. 4
Typical activity frames when using LDS
Typical German mistakes (compared to the correct spelling) in the beginning are • missing spaces between words • missing vowels (e.g. "Bll" instead of "Ball") • mix phonetically similar consonants up (e.g. "Bur" instead of "Buch", German for book) • missing quiet consonants that modulate sounds of vowels (e.g. "Keze" instead of "Kerze", German for candle) • missing "h" and double consonants to emphasise long and short vowels (e.g. "Bal" instead of "Ball")
Examples While the collaboration in synchronised workspaces is already implemented, ongoing developments of the T3 application will enhance the intelligent support modules for the initiation of partner or group work and for the provision of context sensitive individual feedback. Fig. 5 gives an impression of how the representation of an error detection in a child’s T3 environment will look like. Here the mistake is easy to detect, but the judgement and the kind of feedback is based on the child’s history, i.e. based on former activities with the T3 application. In the case shown here, the system detected the goal word "Ball" and highlights the missing "a" by simply moving the existing characters and generating a gap:
Bll ⇒ B ll Fig. 5
Visualisation of an error detection
A more sophisticated intelligent module could propose a partner for group work based on former exercises (see Fig. 6 ). For example a module that supervises the networked activities in the classroom might find out that children had similar problems related to a certain word.
Depending on the individual settings for an application instance, the system could now give the hint, that a couple of other children have written the word, too.
Fig. 6
Classmates’ solutions for the German word "Zitrone" (lemon)
Now, the child can select and hear a variant or select a word to work with or he or she decides to ask a child for help on this specific word. This could be a collaboration outside the system (i.e. meet with the child at one workplace) or inside the system within a shared workspace and further intelligent support. The underlying architecture for these support features already exists. The functionality will be applied in the NIMIS CiC in close cooperation with our evaluation team and teachers mid of February 2000 together with the application of the already existing collaboration modes of T3.
Implementation T3 and the NIMIS desktop environment is completely written in Java and it uses a general Java-based XML format for all persistent data. The user interacts with the application in almost all situations via drag and drop (e.g. copying letters from the phoneme table). Teachers can customise T3 and prepare exercise sheets with an extended application version based on the same Java Bean component. Text can be typed in using a keyboard, other multimedia items (e.g. sounds, images) can be integrated from a file system by simple drag and drop operations. The collaboration is based on a generic Java toolkit for coupled objects, called MatchMaker. The former C++ version has already been used in applications of the approaches presented in [7] and [3]. The new Java based version of MatchMaker now provides a high level API to access object-wise coupling and flexible dynamic remote method invocation based on Java’s RMI and Reflection mechanism (cf. [13]). The audible feedback is provided by an external text-to-speech (TTS) system that has been interfaces to Java. Before T3 submits the text to the TTS it is parsed through a Prolog based converting algorithm. Here, special phoneme control sequences are used to emphasise certain parts of a word or to prevent the TTS system from spelling every single character in unknown or misspelled words. The Prolog interface and the dynamic remote method invocation also serve as the gateway to the intelligent support architecture and its modules. All intelligent support functions are based on modules with standard mechanisms of user and group modelling, which are plugged into a framework system develop from the architecture discussed in [7]. This significantly simplifies the integration of further modules and the adjustment of existing algorithms to young children.
Experiences and Perspectives Pupils from a German primary school have been working with the T3 since August ’99. The application is embedded in the normal curricular activities of a class of first graders. Handling
and integration in the (also non-computerised) classroom activities is easy and robust. Children and teachers are satisfied with the software pronunciation, which allows pupils to get immediate feedback whether or not their written word is spelled (phonetically) correctly. The current version can be used in a collaborative mode, in which children share their workspace with a (possibly) split phoneme table (case and in table 1), for example one child may keep the consonants while the other one gets the vowels. The application is still going through design and re-design phases, mainly based on the children’s and teachers’ feedback. Additionally the following features are currently being prepared for practical use:
ì
• integration of a visual three dimensional head (showing the movement of the mouth to increase the number of easily distinguishable characters) • systematic use of synchronous collaboration modes in the classroom (accompanied by interviews and observations for evaluation purposes) • visualisation and smooth integration of intelligent analysis results • suggestion of pairs or groups for collaboration (controlled by teacher) The T3 application relieves the teacher from the mechanical work of speaking written results of the children by request. Furthermore, the added value lies in the increased freedom of the teacher to support children with special needs of many kinds. For the children, the application presents itself as a passive interactive learning tool, to experiment with. Although not yet very concrete, also some additional effects have already been recognised: For example the children use the computerised feedback more often than the teacher's feedback in a normal classroom. This may be taken as a hint to a more explorative way of learning.
Acknowledgements Parts of this work refer to the Esprit project No. 29301, "NIMIS". We thank our NIMIS partners, namely from CBLU Leeds, INESC Lisbon and MediaWorld Bad Lippspringe, for the good and constructive cooperation, and especially the teachers and pupils of the associated German primary school "GGS Kirchstraße" for their creative input.
References 1. Aronson, E., Balney, N., Stephan, C., Sikes, J. & Snapp, M. (1978). The jigsaw classroom. Beverley Hills CA: Sage. 2. Hoppe, H.U., Baloian, N. & Zhao, J. (1993). Computer support for teacher-centered classroom interaction. In Proc. of ICCE 1993, Taipei, Taiwan. 3. Hoppe, H.U., Gaßner, K., Mühlenbrock, M. & Tewissen, F. (2000). Distributed Visual Language Environments for Cooperation and Learning: Applications and Intelligent Support. To appear in Special Issue in the Group Decision and Negotiation Journal, vol. 9, no. 3. 4. Hoppe, H.U. & Plötzner, R. (1999). Can Analytic Models Support Learning in Groups? In Dillenbourg, P. (ed.). Cognitive and Computational Approaches, Pergamon 1999, Amsterdam, Netherlands. 5. Montessori, M. & Hunt, J. McV (contr.) (1989). The Montessori Method. Paperback reissue edition, Schocken Books. 6. Mühlenbrock, M. & Hoppe, H.U. (1999). Computer Supported Interaction Analysis of Group Problem Solving. In Proc. of CSCL 1999, Stanford: CA.
7. Mühlenbrock, M., Tewissen, F. & Hoppe, H.U. (1998). A Framework System for Intelligent Support in Open Distributed Learning Environments. In International Journal of Artificial Intelligence in Education, vol. 9. 8. NIMIS, Esprit funded i3 Project (No. 29301) (1998). Networked Interactive Media In Schools. http://collide.informatik.uni-duisburg.de/Projects/nimis/ 9. Palincsar, A.S. & Brown, A.L. (1991). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. In Cognition and Instruction, 1, 1984. 10. Reichen, J. (1991). Lesen durch Schreiben. (German teacher’s guide to "Reading through Writing"), Heinevetter Lehrmittel Verlag. 11. Streitz, N., Geißler, J. & Holmer, T. (1998). Roomware for Cooperative Buildings: Integrated Design of Architectural Spaces and Information Spaces. In Proc. of CoBuild 1998, Darmstadt, Germany. 12. Supnithi, T., Inaba, A., Ikeda, M., Toyoda, J. & Mizoguchi, R. (1999). Learning Goal Ontology Supported by Learning Theories for Opportunistic Group Formation. In Proc. of AIED 1999, Le Mans, France. 13. Tewissen, F. (1998). Java MatchMaker Online Documentation. http://collide. informatik.uni-duisburg.de/Software/Docs/JavaMatchMaker/, Dept. of Computer Science and Education, University of Duisburg.
