Wooldridge, Michael ; Fisher, Michael (1993). Executable Temporal Logic for Distributed AI. Proceedings of the. Twelfth International Workshop on Distributed ...
Using Virtual Worlds to Evidence Singular Cognition During the Learning Process Enrique David Espinosa, Gabriela Meza Puesto, Nancy Corlay Rojas Instituto Tecnológico y de Estudios Superiores de Monterrey ITESM-Campus Ciudad de México, {eespinos, gmeza, nrojas}@campus.ccm.itesm.mx Abstract. One of the roles AI now plays in educational software products is expert reasoning over task models on open learning domains. None of these is typically related to cognitive state modeling. We sustain that monitoring of single-actor activities suffices to evidence a singularly-generated cognitive learning process. Thus, an AI characterization is feasible, and useful for mixed objectivistconstructivist, using a virtual learning environment as a tool. Rather than focusing on pure structural domain transmission, we evidence the occurrence of cognitive (i.e. Constructivist) phenomena leading to skill acquisition. In a mixed virtual education situation, a 3D space is key to represent multiple conditions and worlds along which the student travels, so that personalized disatance coaching towards Instructivist goals can be more efficient, as she better understands her student’s constructive learning situation at times when she is away. We present the Constructivist Animated Arena in VRML, part of the Open Temporal Inductive Math Environment (COTIME) we are testing at the High School level for the acquisition of basic math skills at the Monterrey Institute of Technology.
1. Introduction The role AI plays in educational software design and construction is in constant evolution as underlying educational frameworks and paradigms change. It has been realized that the modelization of the learning process outweights the characterization of domain knowledge (Akhras & Self, 1998; Espinosa & Ramos, 1998; Espinosa & Ramos 1999). Yet another important trend has been to identify the importance of underlying & alternative educational paradigms. Purely instructivist [human, and recently machine] tutors, relying on knowledge transmission, are still mainstream in classrooms around the world (Latchman, Salzman & Gillet, 1999). Therefore, domain models and task models are being complemented with cognitive state models (Andriessen & Sandberg, 1999) and expert machines to handle these (Murray, 1999). According to such classification, however, human cognition is only considered as part of an ontology when it comes to learning environments where collaboration enters the scene. We maintain that fulfilling pedagogic goals in an Instructivist coursewsare always motivate the human being to make use of an autonomous cognitive history and data, both of which are interpreted in particular ways depending on a person’s singularity (Espinosa & Ramos, 1996c). What this means is that human behavior is not a consistent model, nor is fully formalizable, given its non-predictive, non-monotonic, and holistic, nature. This is the very nature of human introspective capability. It is the result of the spoken language, as well as its decoding. Therefore, ignoring its corresponding state transformation during single-student media operation is equal to assuming that a default (i.e. ideal) behavior is fully characterizable when performing task modeling on a single person. In our perspective, this does not hold. In contrast, we work in the intersection of behaviorism and cognitivism, and show that monitoring of single-actor activities suffices to evidence the total/partial existence, or incomplete/flawed occurrence of, a singularlygenerated cognitive learning process evidencing learning. We now turn to explain the basis for such an argument. Our work in Instructivist-applied Constructivist educational software has evolved from the realization of a recursive approach to designing observable and interpretable, but non-Constructivist Instructional Graphs called Educational Measurement Instrument (EMI) (Espinosa, & Ramos, 1996a; Espinosa, Boumedine & Chirino, 1996b), to the formalization of cognitive phenomena in non-monotonic, and temporal logic terms (Espinosa & Ramos, 1997). The EMI Model monitored instructivist (i.e. behaviorist) data that uncovered real patterns of conduct during classroom activities and enabled us to watch for screens and lessons that were incorrectly, partially, or seldom, used, from the Objectivist standpoint. This provided data to help computer tutors compare this conduct to ideal ones. The next step has been to incorporate cognitive capabilities to EMI, thus addressing Constructivism. Cognitive data is far more hard to discover than ideal behavior, so an ontology is required to characterize it, without trying to “mimick” the complete process inside the computer. Therefore, AI plays a key role in providing for better mechanisms to uncover subtle events in students’ conduct, but not to reproduce them in formal terms. In this case, Temporal Modal Logic served as a vehicle to datamine through Cognitive State
1
Modeling. The learning process is thus addressed, although issues on Instructional Design are still present. This evolution has been clearly consistent with the mainstream in computer-asisted education, as reported in (Andriessen & Sandberg, 1999), in the sense that “intelligent” behavior of a software agent, or similar expert system, has not been proven a conclusive gadget for effective learning in non-experimental situations per se (Derry & Lajoie, 1993). Throughout the process, we have deepened our research into finding ways of discovering, and recording, of the precise moments in time, and their related circunstances, in which the the cognitive events leading to a particular student’s learning strategy (Cognitive Equilibration, Contradiction and Structure Construction : Twomey, 1996), actually occurr. We know that they evolve as constant, dynamic processes, contributing to unique, and sometimes unpredictable, ways of building knowledge. However, given the structure of current educational programs, observation resulting from direct contact with the student is severely limited with respect to the full 24-hour day. Most of the learning process is intractable to the instructor. An automated tool allowing her to evidence the learning process, for subsequent (i.e. virtual) personalized tutoring, would thus be desirable. Evidence of the occurence of such phenomena could be detected by a Temporal Inference Engine (TIE) working around the clock, on the Internet. Such a tool could then be used to help her visualize the learning history of any student, in graphical (VRML-type), temporal (the student entered lesson A before lesson B after she completed assignment C), and cognitive terms (she retained the concept of tangent after she abstracted the 3D spatial notion of line), allowing her to coach the students more efficiently, and personally. A Constructivist Animated Arena (CAA: Espinosa & Ramos, 1999) lets the student wander at her own pace so that the goals are reached in [properly limited] holistic manners, which the TIE constantly scans and logs, so that “smart” (not intelligent) monitoring fit well, using temporal modal logic. Section 3 deepens into temporal logics and their implementation in our project. This paper presents partial, and simplified, results of actual classroom behavior upon using our software, and attempts to demonstrate that a mixed approach to virtual education accounts for better results when modeled as an open environment. The end-result is the Constructivist and Open Temporal Inductive Math Environment (COTIME) program, a Java-VRML system currently used in the High School program at the Montrerrey Institute of Technology in Mexico City. We provide conclusions, limitations on our work, and future trends we will follow using this new technology.
2. Constructivist Animated Arena (CAA) in the COTIME tutor In (Espinosa & Ramos, 1998) we developed a way to [temporally] listen to the student during courseware usage. At that moment, we used an agent-oriented paradigm for modeling Reactive Systems (Benyon & Murray, 1993), called Concurrent METATEM (Wooldridge, 1992 & 1993). In our current implementation, the declarative construct remains, but is used within the following model. Our aim is to fill in as much as possible of the temporal holes in the instructional graph. We therefore re-model the graph, and make it an open environment. A maze was developed using VRML. The instructivist goal is to visit all the lessons embedded in the maze. The student will learn the concept of Parabola, and its associated formalisms. Each lesson is a freely usable playground for the student to experiment with graphs, equations and other learning tools. We also included review quizzes so we could test the soundness of the detected learning processes with respect to the question-and-answer paradigm. We therefore applied the notion of a Constructivist Animated Arena (CAA: Espinosa & Ramos, 1999). In this case, the maze serves as a CAA (witness Figure 2). Recall Figure 1. The instructional graph containing the temporal holes formerly described can now be traversed in a mixed fashion. The edges of the graph still hold, but inside each vertex now contains an internal maze of possible events, each one of them related to an Educational Knowledge State (EC). Figures 4 & 5 show [possibly] typical computer screens for math learning.
3. Temporal Information Measurement Instrument (TIMeI) We now depict a model that allows us to monitor the student in her constructivist (i.e. free–path through the CAA) session. Let EC be an Educational Knowledge State (EC). Such a state is not predefined, as in
2
other instructional-based ITS’s. It is dynamically built as a result of inferring a cognitive status, from a set of low-level GUI events – the only means to communicate to and from current user interfaces. Let:
Figure 1 : CAA Maze and embedded lessons
EC = { I1 + I2 + I3 + I4 + …….. + In} where It are inferences derived from events in time, then Ij = ( PCP => PCF + R ) In section 1 we stated that our technology was a Temporal Inductive Math Environment. The TIMeI framework deals with modal temporal operators to establish a formal relationship between the user’s actions and the induction process. Let PCP be the precondition in the past-time portion of the inductive rule. Recall Figure 2. Whatever happened in past lessons
t i (b )t f is pre-condition, and evidence, of a
learning status. therefore: PCP = { OMP Action [ OL OMP Action]* } where PCP a set of events in the past. Now, and the postcondition in the future (PCF), the actual inference, is directly based on the PCP, and related to what might happen, cognitively and pedagogically. Suppose
t f (a )t i in Figure 2.
let: PCF = { OMF Action [ OL OMF Action]* } where PCF a set of events in the future. The PCF arguments depict cognitive information resulting from analyzing the event history (i.e. the perceivable [past] cognitive actions, which are strictly related to the instructional design exhibited by the user interface. At this point, we start referring to time using modal operators, as follows: OMP = Modal operator1 in the past: OMF = Modal operator in the future:
} {
(Strong Last), ◆ (Was), ■ (Heretofore), S (Since), Z (weak Since). (Next), ◊ (Sometime), (Always), U (Until), W (Unless).
and: 1
For an in depth treatment of modal operators, witness (Benthem, 1995)
3
Action = 1st order predicate. OL = Logical operator (and/or/not) Such information is used to define coaching strategies and messages, depending on the interaction format. Now, R is a set of [instructionally] defined coaching statements {R1, R2,….Rn} let ECT be a total state of student knowledge, and E be an evaluating object, so: ECT = { EC1 + EC2 + ….EC3}, where ECt = Knowledge state at time t. E = (ECT, Data, R), where Data = Low-level events performed by the student (e.g. mouse clicks, drag-drops, etc).
As claimed before (Espinosa & Ramos, 1998 & 1999) human student autonomously emits about her own acts, according to her perception of the GUI. The GUI-generated actions are all the software receives as working material, so it must try to make sense out of them, without an instructional graph and its associated properties. For this to happen, the cognitive approach to learning must be addressed, so an inference engine was built to work on the following temporal rules:
}(¬solveLesson(3)) => { (¬solveLesson(4))
The last thing that happened was that she did not solve lesson 3, so the next thing that will happen is that she will not solve lesson 4. It happened that she did not solve lesson 7, ◆(¬solveLesson(7)) => (¬solveLesson(8)) so it must happen that she cannot solve lesson 8. The last thing that happened was that she (solveLesson(3)) => (solveLesson(4)) solved lesson 3, so the next thing that will happen is that she will solve lesson 4. It happened that she solved lesson 7, so it ◆(solveLesson(7)) => (solveLesson(8)) must happen that she can solve lesson 8. ◆(¬solveLesson(6)) => ((¬solveLesson(7))W It happened that she did not solve lesson 6, so it must happen that she cannot solve (¬solveLesson(5))) lesson 7 unless she solved lesson 5.
}
{
Table 1: Inductive rules and modal operators These operators work on the analysis of linear history records that provide evidence of Cognitive State evolution, but are general-purpose, so they alone cannot describe the Piagetian scenario. Due to space limitations, we do not show the complete logic mechanism that actually detects the times when the key phenomena: Cognitive Equilibration, Contradiction and Structure Construction (Twomey, 1996), actually occur. These are covered in depth in (Espinosa & Ramos, 1999). Evaluating objects (E), which contain states of student knowledge, as evidenced by inferences over individual actions, do not contemplate a specific rule-based determination of specific hard-coded cognitive phenomena. Rather, they convey the clues for the instructor to make sense of the specific knowledge state. A proper interpretation of these data will most likely be complemented with quizzes and other traditional evaluation procedures, as stated before. We now proceed to describe how these evaluating objects behave, and are implemented.
4. Open Temporal Inductive Mathematics Environment (COTIME) On entering the tutor, the student is presented with a Virtual World (see Figure 2).
4
Figure 2: Virtual World Entrance When sufficient events have been collected from the student, a relational query engine implements the rules in the previous section. For example, rule (¬solveLesson(3)) => (¬solveLesson(4)) is implemented as:
}
{
If exist (select ∂1 ∂2 from table actions where condition = ¬solveLesson(3)) THEN If exist (select ∂1 ∂2 from table actions where condition = ¬solveLesson(4)) THEN it is inferred that the condition was satisfied Else other inferences must be matched against Endif Else Go try the next inference rule. Endif These relational-query inferences run along the way for all the lessons in the CAA Maze, as they are entered (the first one on Figure 3) and traversed (witness Figures 4-5). All events are captured by a series of Java applets reminiscent of our old EMIspectors (Espinosa & Ramos, 1996a). A JDBC engine stores the events in a Hash Table, which is later exported to an Excel spreadsheet that looks as shown in Figure 6. Once tabulated, the data can be manipulated by an inference engine, from which the relational queries are executed.
5. The Inference Engine The processed data looks like that shown in Figure 6. A Constructive Evaluator System (witness Figure 7) now processes the data, using the TIMeI framework described before, and used in two modes: Inline tutoring, in which messages are sent to the student the old-fashioned ITS style, and offline analysis, in which the instructor can review personalized reports of student activity, along with complementary suggestions by the system (witness Figure 8). The instructor is ten free to interpret the information as desired. Offline analysis is the personalized & virtual mode for coaching we presented in section 1.
5
Figure 3: Lesson inside the CAA
This dual-operation mode constitutes the centerpiece of the allowance for an open environment, but a departure from the old ITS scheme, and a corresponding allowance for coaching the students more efficiently, and personally.
6. Conclusions Upon use of the COTIME system, a close relationship between the suggestions the system emitted, and the level of learning evidenced by quizzes and traditional examinations was observed. We have tested the software with more than 200 students at the time we write this paper. Witness Figure 9. Three students (0, 1 & 2) are being analyzed. For student #0 had problems solving all scanned lessons, while student #1 encountered problems, which he solved at the end, and student #2 encountered almost no problems. The number of keystrokes and mouse events is indicated in each case. It is possible for COTIME to report specifics on keystrokes per screen, and many other inferences (not shown here) are issued on request. In 63% of the cases, the inferences were correct, that is, the students actually performed what the inference had predicted. The instructor discovered that offline coaching using the reports shown in Figure 8 had evidenced reality in 58% of the coaching cases. Many students actually perform poorly when presented with computerized learning tools. These people tend to play around with the system, but are not actually learning anything. In about 30% of the cases, the system was confused by an incoherent sequence of events. This still proved useful, since the instructor was able to detect such flawed reports, and personally infer that the student was having problems with the course. In 8% of the cases, it proved impossible to make out a diagnostic of the reasons why a student was performing poorly. In this study, we found no evidence of higher-level performance (e.g. people with above-average math skills who show no need for a computerized tutor).
6
Figure 4: Lesson in the CAA
Figure 5: Raw inference data Finally, we report that the maintenance of a large event sequence database proves hard to maintain over the web in an academic environment. Several key administration decisions will have to take place if a sound information system is to be kept operational around the clock.
7
Figure 6: Constructive Evaluator System Unfortunately, due to space limitations, it is impossible to describe the full temporal logic scheme from which further temporal logic analysis of the user’s performance is executed. It is at this point where detailed inferencing allowed us to detect Equilibration, Contradiction and Structure Construction in a Pagetian manner. These are currently under revision for a broader, more in-depth publication.
Figure 7: Personalized reports of student activity
8
Figure 8: Student Behavior Report
7. Limitations, and Further Work One obvious drawback at this point in time is the effort required to fine-tune specific temporal-modal rules for specific instructional objectives. The interpretation process varies from instructor to instructor, so problems of style arise when two or more teachers interact in a single group, or when two or more groups of students are confronted with differences in their respective instructors’ coaching styles. The use of a Reactive scheme both for the design of a CAA and for the temporal declarative objects still depends on reactive behavior with respect to the human agent. This limits the amount of cognitive data presently obtained on real-time operation. On the other hand, the limited semantics provided by standard event-driven I/O devices makes is necessary to conduct research on User Modeling (Grasso, 1997; Linden, 1997; Adelheit, 1997), which include Ergonomics, Human Factors and User Interface Design, besides the strands subject of treatment throughout this paper. But the strongest limitation relies of the fact that all temporal reasoning still depends on the student sitting in front of an internet-enabled [personal] computer, which not all of our student have access to. There still are temporal holes at times where the machine is not accessible. We believe that, as Mobile Computing digs its way into homes of students, these holes will eventually vanish. Finally, future work will dig into making COTIME a true collaboration environment, in which expert reasoning among software agents will take place, once a cognitive and constructivist-rich negotiation scheme is powered by temporal representations of multiple knowledge types and/or domain expert systems as in (Murray, 1999). The interdisciplinary nature of this work makes it inherently challenging. We still have a long way to go in this direction. However, preliminary results derived from our prototypes reveal that the constructive approach is theoretically and physically sustained, as long as it is applied in conjunction with Instructivist practices. The pedagogical strands to follow will therefore lead us to a continuum drifting away from Objectivism and drawing close to Constructivism.
9
8. References Adelheit Stein, Jon Atle Gulla, and Ulrich Thiel (1997). Making Sense of Users' Mouse Clicks: Abductive Reasoning and Conversational Dialogue Modeling.. Sixth International Conference on User Modeling. Chia Laguna, Sardinia, Italy. June 1997. Akhras, Fabio and Self, John (1998) Modeling the Process, not the Product, of Learning. Keynte address paper at the 4th World Congress on Expert Systems. WCES98. Mexico City, Mexico. March 1998. Allen, J.F. & Koomen, J.A. (1983). Planning using a temporal world model. Proceedings of the 8th IJCAI, 41-47. Andriessen, Jerry & Sandberg, Jacobjin (1999). Where is education heading and how about AI?. International Journal of Artificial Intelligence in Education (1999), 19, 130-150. Benthem, J van (1995). Temporal Logic, in "Handbook of Logic in Artifical Intelligence and Logic Programming", Gabbay, Dov M; Hogger, C.J.; Robinson, J.A., Eds. (1995). Oxford Science Publications. USA 1995. 242327. D. Benyon, D. Murray (1993). Adaptive Systems : From Intelligent Tutoring to Autonomous Agents. KnowledgeBased Systems Volume 6 Number 4 December 1993. 197-219. Derry, S.J. & Lajoie, S.P. (1993). “Computers as Cognitive Tools”. Hillsdale, New Jersey: Lawrence Erlbaum Associates. 1993. Dick, Water (1991). A Instructional Designer's View of Constructivism. Dick, Water. Educational Technology, 1991, 31(5), 41-44. Enrique Espinosa, Fernando Ramos (1996a). Intelligent, Agent-Based Virtual Education using the Java Technology Proceedings of the Third International Conference on Intelligent Tutoring Systems ITS96. IEEE/ACM. Montreal, Canada. June 1996. Springer Lecture Notes in CS 1086. pp 270-278. Espinosa, Enrique & Ramos, Marc Boumdine & Ivonne Chirino (1996b). A Formal Approach to the EMI Model and Case Study. Proceedings of Ed-Media96. AACE. 100-106. Espinosa, Enrique & Ramos, Fernando (1996c). A Study for Modeling Creativity and Discovery in Intelligent Agents based on the Human Unconscuious. 2nd International Workshop on Creativity and Cognition. LUTCHI Research. Loughborough, UK, 1996. 23-29. Espinosa, Enrique ; Ramos, Fernando. (1997). The RSI Temporal-Cognitive Model for Believable & Introspective Interpretation in Inteligent Tutoring Systems. Intelligent Information Systems. Professor H. Adeli, Editor. IEEE-IASTED 1997. 271-277. Espinosa, Enrique ; Ramos, Fernando. (1998). Dealing with Temporal Holes in Instructional ITS's. Proceedings of the Workshop Proceedings on Current Trends and Applications of Artificial Intelligence in Education. Gerardo Ayala, Ed. 4th World Congress on Expert Systems. Mexico City, Mexico. 1998. 17-24. Espinosa, Enrique & Ramos, Fernando (1999). Inteligent Agency and Tutoring: The Importance of Being Timely. Special Issue for Intelligent Agents for Computer-Based Educational Systems. International Journal of Continuing Engineering Education and Life-Long Learning, (IJCEELL), ISSN 0957-4344. Floriana Grasso (1997). Using Dialectical Argumentation for User Modelling in Decision Support Systems. . Sixth International Conference on User Modeling. Chia Laguna, Sardinia, Italy. June 1997. Latchman H.A, Salzman, Ch & Gillet, Denis (1999). Information Tecchnology Enhanced Learning in Distance and Conventional Education. IEEE Transactions on Education, (42)4. November 1999. 247-254. Linden, Steve Hanks, and Neal Lesh Greg (1997). Interactive Assessment of User Preference Models: The Automated Travel Assistant. Sixth International Conference on User Modeling Chia Laguna, Sardinia, Italy. June 1997. Murray, Tom (1999). Authoring Intelligent Tutoring Systems: An Analysis of the State of the Art. International Journal of Artificial Intelligence in Education (1999), 19, 98-129. Pelavin, Richard ; Allen, James. (1986). A Formal Logic of Plans in Temporally Rich Domains. Proceedings of the IEEE, 74(10). October 1986. 1364-1382. Twomey Fosnot, Catherine, Editor (1996). "Constructivism: Theory, Perspectives & Practice". Teachers College Press, USA 1996. Wooldridge, Michael ; Fisher, Michael (1992). Specifying and Verifying Distributed Intelligent Systems. the Manchester Metropolitan University Wooldridge, Michael ; Fisher, Michael (1993). Executable Temporal Logic for Distributed AI. Proceedings of the Twelfth International Workshop on Distributed Artificial Intelligence (IWDAI-93), Hidden Valley, PA, May 1993
10
