Goal-based Modelling of the Learner Behaviour for Scaffolding ...

Proceedings of the 7th International Conference on Computer and Information Technology (ICCIT-04). Dhaka, Bangladesh.

Goal-based Modelling of the Learner Behaviour for Scaffolding Individualised Learning Instructions Mehdi Najjar

André Mayers

Philippe Fournier-Viger

Dept. of Computer Science University of Sherbrooke. Sherbrooke.QC, Canada. email: [email protected]

Dept. of Computer Science University of Sherbrooke. Sherbrooke.QC, Canada. email: [email protected]

Dept. of Computer Science. University of Sherbrooke. Sherbrooke, QC, Canada email: [email protected]

Abstract People are active participants in the world with purposes and goals they want to attain, especially in a didactic context where learners are interacting with distant teaching/learning environments. In this paper, we describe an experimental validation for the explicit introduction of goals into our knowledge representation model. Here, the term ‘goal’ refers to a mental representation of an intention to accomplish a task. Our model has been embodied within an algebra virtual laboratory dedicated to boolean reduction problem solving. The lab is offered to learners with the opportunity to solve problems at their own pace and to adjust the difficulty level of the problems. A behaviour simulator constructs an internal cognitive mapping of the learner’s problem solving process for each case attempted. This mapping has the potential to serve as the basis for individualised learner instruction. Inspired by cognitive theories, our computational cognitive modelling of the learner behaviour treats goals as a special case of knowledge that represents intentions behind the user actions. During learning, the energy employed by learners to achieve their goals distinguishes them from any simple form of knowledge. This distinction involves particular treatment for goals. Keywords Computer Aided Education, Interactive Learning Environments, Learner Modelling, Learner Behaviour Simulation, Personalised Tutoring. INTRODUCTION In spite of an increasing development of virtual and distant applications which use the advantages of multimedia and the Internet for distance education [10,11,21,25], learning by means of such tutorial tools would be more effective if they were specifically adapted to each user needs. Teaching would be well adapted if the ‘artificial’ tutor could identify the correct acquired knowledge and determine its mastery level. Training could be more personalised if the tutor was also able to recognise the erroneous learner knowledge and suggests the planning of a suitable sequence of pedagogical activities to improve significantly the cognitive level of the learner [2,6]. For those reasons, it is necessary that the user model – that the system formulates in its internal represen-

tation – would be as closer as possible to the real learner behaviour when it is put in an e-learning context. For a powerful and effective teaching purpose, various cognitive theories offer to model the human process of knowledge acquisition and handling during the learning activities. Several cognitive architectures show that learners are active participants in the learning environment due to goals they want to attain during the problem solving tasks [5,20]. We use the term ‘goal’ to refer to a mental representation of an intention to accomplish a task, to achieve some specific state of the world, or to take some mental or physical action. Like any other mental representation, goals are capable of becoming automatically activated by environmental features. In fact, the specific state to be reached is generally described using an algorithm, a procedure allowing the satisfaction of that goal. Very often, a goal must be set aside temporarily and then resumed later. Suspending a goal might be necessary, for example, if it requires subgoals to be achieved first, as in hierarchical problem solving [14] and means-ends analysis [19]. If a task is too complex to be performed directly, means-ends analysis is a ‘tool’ that can be used to decompose it into subgoals. If these subgoals are also too complex to achieve directly, they must be decomposed in turn. Eventually the system reaches a subgoal tractable enough to be achieved directly. Early studies of higher-level cognition and intelligence, where goals first appeared on the analytical horizon, were focused more on the flow of behaviour in response to task structure than on human memory and its constraints [8]. Special goal structures became a more explicit assumption with the development of the ACT [3] and Soar [20] cognitive architectures. These approaches demonstrate that the energy employed to achieve goals distinguishes them from any other form of knowledge. This distinction involves a different treatment for goals in the human cognitive architecture. Goals can be seen as a special case of knowledge that represents intentions behind actions of the cognitive system. Learner’s reasoning depends crucially on her/his goals and how likely s/he thinks actions will be successful to achieve them [5]. Other researches [22,23] have demonstrated that the usage of an interface that exposes possible goals and their subgoals enables learners to solve problems faster, more accurately and so results in better learn-

ing. They also showed that reification of goals in calculus led to improvements.

generally taught to undergraduate students on first cycle of higher education. Preliminary notions, definitions and

Figure 1: A complexity level 5 example, proposed by a learner, in the examples section.

The focus of this paper is on describing an experimental validation for the explicit introduction of goals into our knowledge representation model. This model is embodied within an algebra learning environment dedicated to boolean reduction problem solving [17]. We propose to model the learner knowledge according to a psychological approach that explain the human cognitive activity in terms of memory subsystems and their processes. In our model, the learner goals are treated explicitly and reified as a particular form of knowledge. We believe that our proposed knowledge structuring – due to many similarities with that used by learners when they are put in a learning situations – facilitate the identification of correct and/or erroneous acquired knowledge and can help the tutoring system to select a suitable planning of pedagogical activities, specific to each learner. The remainder of the article is organised as follows. First, we present our virtual learning environment. Next, the knowledge representation model, incorporated within the lab, and the goals explicit treatment are described; followed by experimental results and discussion. Finally, we conclude by announcing future work. THE LEARNING ENVIRONMENT The virtual laboratory (VL) demonstrates a problem solving organisation that attempts to model the learner cognitive activity during the task accomplishment. Our subjectmatter domain is the algebraic boolean expressions and their simplification by means of reduction rules, which are

explanations (available, in the lab, through sections exploration via buttons clicking) constitute a necessary knowledge background to approach the boolean reduction problem.In the preliminary notions section, rules of boolean simplification are stated and presented to learners. By choosing particular rule in a combo-box menu, for example, the disjunction rule of a proposal "a" with the truth constant "False" ((a | F) = (a)), or the De-Morgan rule applied to a conjunction of two proposals (~ (a & b) = (~a | ~b)), the latter is posted with a brief formal definition (in French). In the explanation section, hints and thorough explanations on the boolean reduction rules suitable for usage are provided. In the examples section, examples are given. Those are generated randomly with variable degree of difficulty (from 1 to 5) chosen by the learner. Learners can also enter, by means of a visual keyboard, any boolean expression they want and ask the system to solve it. Figure 1 illustrates a complexity level 5 example proposed by a learner. The problem solving steps and the applied rules are displayed on a blackboard. Examples show optimal solutions to simplify boolean expressions and are provided to guide the learner during the problem solving, which begins by clicking on the exercise button. Exercises, also with variable complexity levels, give practice opportunities to the learner to practice tasks. Via the visual keyboard, learners reduce an initial boolean expression (generated randomly) by choosing suitable simplification rules to apply in

the order they want. Although various tutorial strategies are to be considered, we use actually the "Cognitive Tutor" strategy [4], implemented within several intelligent tutoring systems and which its effectiveness has been largely proven [1,7].

revisiting previously explored components for additional ideas or by reviewing prior knowledge and explanations. THE KNOWLEDGE REPRESENTATION MODEL

Figure 2: A complexity level 4 exercise, done by a learner, in the exercises section.

Consequently, in the case of erroneous rule choice (or application) on any of the sub-expressions, forming the initial given expression, the system notifies immediately the learner and shows her/him (i) the selected subexpression, (ii) a correct rule to reduce it, (iii) the resulting simplified sub-expression and (iv) the current state of the global expression. Figure 2 shows the resolution steps, made by a learner to reduce a complexity level 4 expression, in the exercise section. Our belief is that learners should not only learn theoretical contents and concepts of the domain but also how to handle their knowledge and the related skills in a practical world. For this purpose, our design enables learner to explore the environment in dynamic interaction with the context as if they were really in a classroom with a teacher monitoring. Because the VL is designed with the knowledge linked through buttons, learners are able to determine their own learning paths through the materials which can be viewed in any desired sequence. This provides learners a flexible learning environment for a non-guided exploration and assisted problem solving. Indeed, learners can access all components – the domain knowledge and problem solving activities – in non-linear ways. They can seek information by different manners. For example, by orienting themselves to the case study by retrieving detailed case information, by

Our knowledge representation structures are inspired from those of the human memory. The principal characteristic of this memory is carrying out categorisation, generalisation and abstraction processes [9]. According to cognitive psychology theories, it has been argued that knowledge is encoded in various memory subsystems depending on the way in which their contents are used and/or handled. Each subsystem presents a particular type of knowledge such as, semantic [18], procedural [3] and episodic knowledge [24]. Our model regards semantic knowledge as concepts taken in a broad sense. Thus, they can be any category of objects, relations or functions. Moreover, we subdivide concepts in two categories: primitive concepts and described concepts. The first is defined as a syntactically non-split representation; i.e., primitive concept representation can not be divided into parts. For example, in figure 3, which shows a boolean reduction expression, symbols "a" and "b" are a non-split representation of the corresponding proposals. On the other hand, we define described concepts as syntactically decomposable representations. For example, the expression "a & V" is a decomposable representation that represents a conjunction between proposal "a" and the truth constant "V" (true), two primitive concepts. The symbol "&" represents the conjunction logical operator (AND), and is a primitive concept. In this way, the seman-

tic of a described concept is given by the semantics of its components and their associations (which take those components as arguments to create the described concept).

generic knowledge (goals) instantiation, retrieved from semantic memory. Episodic knowledge is organised according to goals.

Figure 3 : Semantic components of a two proposals logical implication. For example, as shown in the diagram illustrated by figure 3, in the expression "(a &V) → b", symbols "a", "b" and "V" are associated to primitive objects (proposals and truth constant), the symbol "&" is associated to primitive function (the conjunction) and the symbol "→" is associated to primitive relation (the implication). Finally, "(a & V) → b" is a described object having three components: "a & V", "→" and "b" and it represents a logical implication between two proposals. In opposition to semantic knowledge, which can be expressed explicitly, procedural knowledge is inferred by a succession of actions achieved automatically – following internal and/or external stimuli perception – to reach desirable states [3]. A procedure can be seen as a mean of achieving a goal to satisfy a need, without using the attention resources. In a boolean algebraic context, substituting automatically "~V" by "F", making abstraction to the explicit call of the truth constant negation rule, can be seen as procedural knowledge which was acquired by the repetitive doing. In our approach, we subdivide procedures into two main categories: primitive procedures and complex procedures. Executions of the first are seen as atomic actions. Those of the last can be done by sequences of actions, which satisfy scripts of goals. Each of those actions results from a primitive procedure execution and each of those goals is perceived as an intention of the cognitive system. Episodic memory preserves temporal relations allowing the reconstruction of previously experienced events as well as the time and context in which they took place. In our approach, the episode representation is based on particular

Each episode specifies a goal that translates the learner interest and gives a sense to the underlying events. These events are sub-episodes that correspond to subgoals realisations. The latter are specified by the procedure used to achieve the main goal. Thus, executions of procedures are encoded in episodic memory and each goal realisation is encoded in an episode. In this way, the learner episodic memory stores all facts during the training activities [15]. THE EXPLICIT TREATMENT OF GOALS In our model, a goal is described using a relation as follows (R: X, A1 A2 .. An). This relation (R) allows to specify goal "X" according to primitive or described concepts "A1, A2, .. An" which characterise the initial state. In a teaching context, stress is often laid on methods that achieve the goal rather than the goal itself; since these methods are in general the object of training. Consequently, the term ‘goal’ is used to refer to an intention to achieve the goal rather than meaning the goal itself. Thus, procedures become methods carrying out this intention, which are noted (R: A1 A2.. An). To underline the intention idea, the expression representing "R" is an action verb. For example, "reduce (a, &, ~V)" and "substitute (~V, F)" respectively mean: the intention to find a simplified form of expression "(a & ~V)", and the intention to replace the "V" truth constant negation "~V" by the equivalent "F" truth constant. Thus, a goal can be seen as a generic function where the procedures play the role of methods. As mentioned above, executions of procedures are encoded in episodic memory. Each goal realisation is encoded in an episode. Thus, episodic knowledge is organised according to goals; i.e., each episode specifies a goal that gives a sense to the un-

derlying events. These events are sub-episodes that correspond to subgoals realisation, which are specified by the procedure used to achieve the main goal. In our model, subgoals are not determined systematically, in a static way, starting from the learner main goal. Generally, the complex procedure "P", which achieves a given goal "G", determines number and order of "G" subgoals, whose each one can be achieved, in turn, by a procedure (called, in this case, a "P" sub-procedure).

Table 1: Main parameters of the experiment Complexity Number of exercises Number of students

1 4 10

2 4 10

3 5 10

4 6 10

5 6 10

procedure handles primitive and/or described concepts such as rules, proposals, logical operators and truth constants. We have stored the trace of resolution, adopted by

1

2 3

4 Figure 4 : An episodic knowledge XML file.

EXPERIMENTAL RESULTS To test and experiment our approach, we asked students in computer science and in mathematics, who attend the courses "MAT-113" or "MAT-114" dedicated to logic calculus, to practice the reduction of boolean expressions using our software. An assisted training, aimed at familiarising them with the various components and available tools in the lab, was offered. Then, we left them practising individually in the learning environment. By this test, our interest was the computational modelling of each learner knowledge and the analysis of its corresponding behaviour during a problem solving task in a real e-learning situation. Although the system proposes to solve boolean expressions which are generated randomly (depending on a difficulty degree chosen by the learner), in order to clearly compare the learners behaviours, we forced the system to provide common problems to learners. For each complexity degree (from the easy level to the very complex one) and for each test, the same boolean expression is provided to all. Parameters of this experiment are reported in Table 1. In the learners resolution processes, each step correspond to a transition realisable by means of a primitive or complex procedure applied to satisfy a goal or subgoal. This

each learner. For each student and each exercise made, the system notes the procedures used as well as the cognitions created and handled. Since every procedure is called to achieve a goal, the collected data allow deducing goals (and their subgoals) formulated during the boolean reduction process. For example, figure 4 illustrates the XML file containing an episodic knowledge of a learner who has made the complexity level 1 exercise "reduce (a & V)". Here, for more clearness, the XML format is reproduced graphically via XMLSPY. At first, the behaviour simulator starts by creating cognitions which forms the boolean expression (label 1). i.e., a proposal "a", a conjunction logical operator "&", a truth constant "V" and the expression "a & V" which results from their association. Then, the intention to reduce the expression is created (label 2). This goal was realisable by means of a primitive procedure (label 3) which gave rise to the final result (label 4). ANALYSIS AND DISCUSSION Empirical results confirm that, in practice, if the task is relatively complex, there can be various correct manners to do it. This engenders different behaviours in achieving the same task. For example, when the learner goal is to reduce a given boolean expression, satisfying this goal amounts to obtain, at the final state, the simplest expression which is

equivalent to the initial one. Various employed methods, which depend on the practice level and the concepts mastery, can lead to the desirable final state. Many cognitive tutoring systems [1,13] are designed to support different solution paths and have explicit rationales for recognising the employed strategies leading to the goals. In our experiments these strategies are the different methods employed by learners in reducing the same expression. The choice of a complex procedure "P", which accomplishes an initial given goal "G", depends of the learner’s practices and preferences when s/he achieves a task. Therefore, the number and order of the applied procedures depend on the selected complex main procedure. This means that goal realisation can be made in various ways, by various scenarios of procedures execution sequences. This last constraint implies – as mentioned before – that number and order of goals (and their subgoals), translating the learner needs, are not modelled statically in advance [12]. We are convinced that those goals are a special case of knowledge which describes a desirable final state to be reached [3]. In this mind, we propose to treat goals explicitly to reify them as particular semantic knowledge which is totally distinct from knowledge that represents objects, relations and/or functions. The dynamic aspect is also seen in the non- predefined combinations between occurrences of concepts and the procedures handling them. In fact, primitive units of semantic and procedural knowledge, chosen with a small level of granularity, are used to build complex knowledge entities which are dynamically combined – to create a new knowledge – in order to represent the learner cognitive activity. The stored traces of this activity represent a content formed by episodic knowledge which is specific to each learner. In other words, semantic and procedural common knowledge are combined to form a new specific episodic knowledge. When a learner makes an error, satisfying the goal that s/he wished to accomplish was realised by means of an erroneous procedure. This error results from bad interpretation of the situation, causing a choice of procedure which (i) can be correct but whose application cannot be done in the current context or (ii) is invented and completely false. The procedure is regarded as erroneous if the final result obtained by the learner is different from that of the tutor. In this case, the procedure will be labelled (within an episode in which the erroneous result is stored) as a "procedureerror" which has a unique identifier and which will lead to formulate a set of valid procedures that the learner should have used to achieve the goal. At this stage, learning and mastering these correct procedures will be one of the immediate objectives of the tutorial strategy. More precisely, as the episode containing the ‘procedure-error’ comprises an instance of the goal, the set of valid procedures which satisfy it will be deduced starting from the goal prototype. This last also contains the didactic resources necessary to teach these procedures which, if they are complex, specify subgoals whose each one contains its own didactic re-

sources. In this way, the tutor easily conceives an ordered sequence of valid procedures allowing the correct accomplishment of any goal. Particularly, those for which the learner has failed. We do not use empirical analysis of learners problem solving within our interactive learning environment to show that there are multiple correct manners to realise tasks. In fact, this is not a new conclusion and is not our purpose. But, our idea is that since there are multiple correct ways to solve a complex problem, then it would be more effective to tailor a tutorial strategy – for a specific learner – which is based on the particular way chosen by that learner. This specific strategy will allow to retrieve and recover the knowledge that s/he handled when attempting to achieve her/his goals. CONCLUSION We have described our computational cognitive-based modelling of the learner behaviour and showed the benefits of our approach which treats goals as a special case of semantic knowledge that represents intentions behind actions of the learner cognitive system. We are actually investigating a new idea for integrating pedagogic and didactic knowledge in our computational model and elaborating new mechanism for measuring qualitatively and quantitatively the understanding degree of the material. We are also experimenting with reusable knowledge techniques to provide suitable and efficient knowledge use in similar contexts. These additional original aspects will be detailed in future papers. REFERENCES [1] Aleven, V. & Koedinger, K. (2002). An Effective Metacognitive Strategy : Learning by doing and explaining with computer-based Cognitive Tutor. Cognitive Science. 26(2) – 147-179.

[2] Anderson J. R. (1989). Psychology and intelligent tutoring. Proceedings of the 4th international conference on AI in Education. Amsterdam, Netherlands.

[3] Anderson J. R. (1983). The architecture of cognition. Cambridge: MA: Harvard University Press.

[4] Anderson, J. R., Corbett, A.T., Koedinger, K.R. & Pelletier, R. (1995). Cognitive Tutors: Lessons learned. Journal of the Learning Sciences. 4 (2), 167-207.

[5] Anderson, J. R. & Lebiere, C. (1998). The Atomic Components of Thought. Mahwah, NJ: Lawrence Erlbaum Associates.

[6] Anderson, J. R., Bothell, D., Byrne M.D., Douglas, S., Lebiere, C. & Qin, Y. (2002). An Integrated Theory of the Mind. Psychological Review. 111(4). 1036-1060.

[7] Corbett, A., Mclaughlin, M. & Scarpinatto, K. C. (2000). Modeling Learner Knowledge : Cognitive Tutors in High School and College. Journal of User Modeling and UserAdapted Interaction (10) : 81-108.

[8] Ernst, G. W. & Newell, A. (1969). GPS : A case study in generality and problem solving. NY : Academic Press.

[9] Gagné, R.M, Briggs, L. & Wager, W. (1992). Principles of Instructional Design. (4th edition), NY: Holt & Winston.

[10] [10] Heermann, D. & Fuhrmann, T. (2000). Teaching physics in the virtual university: the Mechanics toolkit, in Computer Physics Communications, 127(1), 11-15

[11] Lintermann, B. & Deussen, O. (1999). Interactive modelling of plants, IEEE Computer Graphics & Applications 19 (1)

[12] Mayers, A. & Najjar, M. (2003). Explicit Goal Treatment in Representing Knowledge within Intelligent Tutoring Systems. Proceedings of The 6th International Conference on Computational Intelligence and Natural Computing. Cary, North Carolina, USA. pp:1685-1689.

[13] Merrill, D. C., & Reiser, B. J. (1994). Scaffolding effective problem solving strategies in interactive learning environments. Proc. of the 6th Annual Conference of the Cognitive Science Society. pp: 629-634. Atlanta, USA.

[14] Miller, G. A., Galanter, E. & Pribram, K. H. (1960). Plan and the structure of behavior. New York : Holt.

[15] Najjar, M. & Mayers, A. (2003a). A Computational Cognitive-Based Approach to Represent Knowledge within Intelligent Tutoring Systems. Proceedings of The 3rd IEEE International Conference on Advanced Learning Technologies. Athens, Greece. pp: 66-71.

[16] Najjar, M. & Mayers, A. (2003b). Dynamic User Modelling through Virtual Course : Toward a Cognitive Approach to Represent Knowledge for E-Learning. Proceedings of The 8th AACE World Conference on E-learning. Phoenix, Arizona, USA. pp:1725-1728.

[17] Najjar, M. & Mayers, A. (2004). Using Human Memory Structures to Model Knowledge within Algebra Virtual Laboratory. Proceedings of The 2nd IEEE International Conference on Information Technology in Research and Education, London, UK. pp: 155-159.

[18] Neely, J. H. (1989). Experimental dissociation and the episodic/semantic memory distinction. Experimental Psychology: Human Learning and Memory, 6, 441-466.

[19] Newell, A. & Simon, H. A. (1972). Human Problem Solving. Englewood Cliffs. NJ : Prentice-Hall.

[20] Newell, A. (1990). Unified theories of cognition, Harvard University Press, Cambridge, Massachusetts.

[21] Rzepa, H & Tonge, A. (1998). VChemlab: A virtual chemistry laboratory. Journal of Chemical Information and Computer Science, Vol 38, pp1048-1053.

[22] Singley, M. K. (1990). The reification of goal structures in a calculus tutor: Effects on problem solving performance. Interactive Learning Environments, 1, 102-123.

[23] Scheines, R. & Sieg, W. (1994). Computer Environments for Proof Construction. Interactive Learning Environments. 4(2), pp 159-169.

[24] Tulving, E. (1983). Elements of Episodic Memory. Oxford University Press, New York.

[25] Wells, L.K. and Jeffrey, T. (1996). LabVIEW for Everyone: Graphical Programming Made Even Easier, Prentice Hall Eds., NJ.