Designing Affective Computing Learning Companions with Teachers as Design Partners. Sylvie Girard
Hilary Johnson
HCI Research lab, Department of Computer Science, University of Bath,
HCI Research lab, Department of Computer Science, University of Bath,
BA2 7AY BATH, UK,
BA2 7AY BATH, UK,
[email protected]
[email protected]
ABSTRACT There is a growing interest in studying the potential of including models of emotion in Embodied Pedagogical Agents (EPA), included in Computer-Assisted Learning (CAL) software. Children’s understanding and response to emotions matures alongside their cognitive development. Any model of emotions embedded in an EPA will impact children’s responses, and use of the EPA characters. Therefore EPA design should include both user characteristics and the pedagogical purposes behind the CAL system. This paper presents the participatory design of an EPA’s affective responses to children’s interaction with mathematical software. The participatory design sessions were performed with teachers as partners in designing the affective learning components. The results of a pilot study on children’s responses to the emotional sequence defined in the model will be introduced, as a separate study. Finally, a plan of future research is presented to further validate the model’s potential for CAL systems, including the integration two software learning applications constructed during the design sessions with teachers.
Categories and Subject Descriptors K.3.1 [Computers and Education]: Computer uses in Education, H.1.2 [User/Machine Systems]: Human Factors
General Terms Design, Experimentation, Human Factors.
Keywords Affective computing; CAL systems; Embodied Pedagogical Agent; Emotion for learning; Participatory design; learning companion.
1. INTRODUCTION Research in educational technology has been increasingly interested in the interplay between emotions and learning, with the aim of defining emotions relevant for learning [13]. Studies have investigated the relationship between students' affective states and their use of learning environments [20]. Computational emotional Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. AFFINE’10, October 29, 2010, Firenze, Italy. Copyright 2010 ACM 978-1-4503-0170-1/10/10...$10.00.
responses influence both how people appropriate products, and their long-term usage [11]. However, whilst research has investigated the impact on learners of pedagogical agents as a medium to induce emotions [19], opportunities for improving the design and interpretation of EPAs and affective learning environments, remain. This work is in line with a research project investigating the possible use of affective components in the design of OpenLearner-Modeling (OLM) tutoring systems for primary-school children. OLM software comprise tutoring systems where the students have access to their user-models [4]. It aims at investigating the use of different OLM components, with Embodied Pedagogical Agents (EPA) as an interaction medium, on children’s learning, motivation, and development of metacognitive skills. In order to enable comparisons between the different studies performed in this project, a single model of EPA’s affective responses to software interaction needed to be designed as closely as possible to the software pedagogical aims. In order to ensure a final product was usable in classes afterwards, the model was designed with the help of primary school teachers, responsible for classes of age relevant children. The teachers were part of the design process as participatory-design partners, therefore fully involved within the design process. After reviewing the state of the art on emotions for learning and their inclusion into computer entities, our model of EPA’s affective responses to user interaction is presented, along with its design process, and theoretical grounding. Then, the results of a study on children’s reaction to the affective model in educational context will be presented, before exposing the future research considerations. The results about the potential of the model to improve learning achievements are not presented in this paper, but can be found in [7], the current paper introducing the model and a first pilot study of children’s understanding and use of the model.
2. AFFECTIVE EPAS FOR LEARNING There is a growing body of research in the interplay between emotions and learning in various educational Computer-AssistedLearning applications. EPAs have been used as an interaction medium to portray emotional responses, with interesting results as to their potential to increase student motivation in a tutoring application [5]. Lester [14] used EPAs to praise and portray sympathetic emotional displays, while Biswas et al [3] used human-like traits to promote empathy and intrinsic motivation in learning-by-teaching systems. Motivated by research in the psychology of emotions, several associations of emotions have been tested, according to the emotional development stage of user participants. Kort et al [13] proposed a possible set of emotions
relevant to learning, along five axes of emotional representation: anxiety-confidence, boredom-fascination, frustration-euphoria, dispirited-encouraged, terror-enchantment. Another aspect of affective computing when applied to learning is how models of emotion should be activated within a learning environment. Oatley and Johnson-Laird [17], argue that positive and negative emotions are activated (respectively) by the belief that some goal will be achieved or threatened. Similarly, Ortony, Clore and Collin’s (OCC) theory [19] states that the valence of emotions (positive to negative) depends on the desirability of events. In this research, the agent’s reaction to user’s interface actions will follow the OCC approach, with an emotion set inspired by Kort et al’s work [13] and Girard & Johnson’s [9] findings for our end users’ age group.
3. THEORETICAL UNDERPINNING The participatory design approach with teachers as design partners, described in the next section, led to a compromise in emotional responses for the model between theory and practice. For example, a succession of wrong answers was identified by teachers as being a source of moderate to high frustration level in children of low mathematical abilities. Spurred by Klein et al’s advances in reducing frustration [12], it led to the definition of motivational and empathic responses from the tutor, represented in Figure 1 as the emotional states comforting and empathic. Most of the emotions and emotional states selected to be part of the model arise from Kort et al’s [13] set of possible emotions for learning. However, some emotions (sad, amused, inspired, awed) were added from the results on emotions [9], by their potential capacity of recognition and integration by child users. A last emotional state added to Kort et al’s set is empathy, which has been shown to be potentially useful for keeping the learner motivated [5, 14]. The use of empathy corresponds to Batson et al’s view on attribution theory [2] when confronted with user’s boredom or frustration. The EPA’s display of empathy portrays an awareness of ‘blocked’ goals and a willingness to help from the agent. According to Batson et al, the empathic display may cause the student to understand the agent is attempting to help and will make the student more likely to follow its guidance. The emotional state of empathy is also used for medium and low level learners when they fail repetitively to answer questions correctly. Following the cognitive disequilibrium theory [15], at the beginning of the activity, the agent will be more comforting and encouraging so that the student can return to a state of equilibrium, causing a reduction in confusion. However, if the learner persists in failing the activity while trying to answer (not just clicking on whichever button is the closest, or at random), the state of the agent will become more empathic toward the learner’s attempts. The agent is acknowledging the child’s attempts to reach their goals and direct them out of the negative state of emotions before they give up entirely. The artificial intelligence production rules forming the emotional responses for each learning situation described in Figure 1 are articulated according to the following principle: the emotional response of EPAs should be context sensitive and adaptive to the learner’s level and abilities in answering the problem to maximize learning. The applications considered are OLM software, where the teachers can define abilities and pedagogical expectations to promote an individualized learning experience. Whilst the learning goals can be further individualized by teachers at a later
stage, each child initially belongs to a ‘level’ for the learning of this particular mathematical domain, initialized by teachers before software deployment in classes. This level is inspectable and editable by teachers as the student progresses, through the student’s learner model. Three levels are defined, differentiating students of low, medium, and high-level abilities. This approach of dividing pedagogical expectations and emotional support for children was used in class by all teachers involved, and has been exploited in the literature, for example in the AutoTutor project [5]. It can be summarized here in the following way: reassuring the low-level and medium-level children in case of failure, congratulating them on their success, and pushing the high-level students to try their best. High-level students’ EPA responses were chosen in order to encourage children to give an answer, rather than playing with the system as a game [1]. On the contrary, responses for low-level learners contain the highest level of encouragement and supporting behavior from the EPA. The teachers wished for the EPAs to try and capture the child’s attention when presented with instructions as to how the activity works, and mathematical method of answering the problem. For this reason, the EPA’s response ranged from polite interest to captivated by the instructions given; and inspiration as the method is revealed to mirror the understanding process of the mathematical skill to acquire. Alongside EPA’s responses to student’s answers of the activity’s questions, another state of the interaction was defined: a period of inaction from the student when asked to give an answer. The defined EPA’s response was inspired by the cognitive disequilibrium theory [2], and supported by high levels of recognition of the state of boredom by children corresponding to our final user’s age group, as described in [9]. The principal cause of inactivity in such applications as witnessed by teachers, seems to be boredom. To bring the student out of this negative state of learning, a design decision was made to confront the student with the emotion itself, in the hope that the EPA looking bored would produce a positive reaction from the child. A second emotional response was associated with this state of inaction: when a longer amount of time has passed without any interaction from the student on the software, s/he is considered as having problems with this particular question. In response, the EPA acts as puzzled and prompts him/her to try answering once more, or pass the question if s/he really cannot find the solution.
4. Designing the Model 4.1 General design process The participatory-design sessions took place during the development of the two OLM systems, with one teacher from an English school in the UK (when developing DividingQuest for English children aged 9-11), and two teachers from a French school (where Multipliotest was created for French 7-11 yrs). A first model was undertaken [7] with the English teacher. In order to build a model satisfying the pedagogical needs of both French and English children aged 7 to 11, the model evolved through iterative design loops, each loop producing a model to be completed and validated by the both design partner’s party: the English teacher, and then the two French teachers working together. The definition of the learning situations to be considered for an EPA’s emotional response, in relation to this type of application (OLM drill-and-practice) was performed in the 1st sessions with teachers and children, when designing the
pedagogical role, look and feel of the DividingQuest and Multipliotest. All emotional responses were created from results in emotional feedback for learning and observation notes taken from classroom observation featuring the affective interaction between teachers and children. The results of these sessions produced a set of interface scenarios, in the form of sketches of child-interaction with the system. During the early participatorydesign sessions with each teacher, the design partners used said sketches of student’s interaction with the system, to define the corresponding responses from the EPA in the form of a written representation of the emotional response in a text box (in French and English). The teachers were asked to rate the usefulness of this emotional response for the pedagogical role within the software, and provide one (or several) alternative(s) response(s) if needed, with a description of how such responses could be better associated with the pedagogical goal. In the next iteration of design, all samples of answers from the first session were compiled in order to present teachers with the alternate possible emotional answers for each situation. The teachers compared the various rationales for using each answer, towards finally agreeing on the definition of a single emotional model. At first there was some cultural differences encountered in relation to child coping strategies when encountering a learning activity. The child teacher was more pronounced in creating a model with a more challenging experience for children with a borderline knowledge, whereas the French teachers were adamant that some higher level of encouragement should be portrayed, to avoid a larger number of children “giving up”. After several sessions on the subject between the design partners and a closer look on the studies showing the results of one strategy or another on children’s behavior, the model was finally defined, with the following objective: reassuring the low-level and medium-level children in case of failure, congratulating them on their success (with the presence of empathy and satisfaction), and pushing the high-level students to try their best (with emotions such as disappointment when the results were too low).
changes with time, and is better represented as a range rather than a single valence position on the axis. For example, when the user is given an instruction, the EPA seems interested, which is considered as a positive emotion of low intensity, represented by a
4.3 Software characteristics A single affective response model was formed for the EPAs of all interfaces used in the evaluations of the research project, due to the similarities in nature and architecture of the two OLM software applications constructed, DividingQuest and Multipliotest [8]: both are mathematical drill-and-practice applications, separated by the number of activities of increasing levels of difficulty. Each activity has the same sequence of events, as illustrated in Figure 1. Once they choose an activity, children are presented with instructions and have access to a screen reminding them of the mathematical content needed to complete this activity, allowing practice prior to beginning the activity itself. Once they have begun however, they lose the access to basic mathematical help (e.g. revising the mathematical tables, …), and have to answer a predefined number of questions. For each question, they are presented with instant feedback on their result. At the end of the activity, the EPA gives them their total result related to this skill and advises them on the next activity to choose.
4.4 EPA’s Affective Response Model for Drilland-Practice applications The EPA responds emotionally to the child’s interaction with the activity in the following way: At first, the EPA is interested when presented with the instructions, and points out several important points to pay attention to. This is inspired by the understanding process of the mathematical skill to acquire, presented in the method of resolution of the problem.
The choice of splitting the emotional response of EPA’s according to the level of encouragement the child should encounter, was the design partner’s choice, supported by the results of research studies in the literature [5]. The selection of emotions was a long iterative process, where experience and theory were contrasted for the learning benefit of children, in order to create a model of learning at different ability levels. All emotions are supposedly recognized by children this age, from the results of the literature, and should hold a specific impact on the child’s interaction with the software and learning process, which should help keep the child engaged in the activity and choose the learning strategy of activity choosing that is best for him/her.
For each question, the EPA progressively becomes bored after a period of inactivity on the interface. After an amount of time defined by the teachers, the EPA acts puzzled and prompts the child to answer or pass the question s/he could not answer. Results of each question brought the following initial responses: ·
o For a child considered able to do the activity with ease (high level): inspired for good answer, unpleasant surprise for medium, and disappointed for a wrong answer.
·
o For a child considered borderline to do the activity (medium level): amused-satisfied for good answer, unpleasant surprise when mistakes were made, and empathic when the wrong answer was given.
4.2 Model representation
·
In Figure 1 the emotional responses of EPAs are represented according to their valence with a Traffic-Lights System (TL) comprising colored squares (negative: red, - ; neutral: orange; positive: green, +). The TL metaphor, commonly used in UK primary school to represent the acquisition of knowledge skills, is instantly recognized by French and English children aged 7 to 12, as representing how positive or negative an emotion might be [9].
o For a child considered unable to do the activity yet (low level): amused-satisfied for a right answer, empathic when the answer is not quite right or definitely wrong.
Those responses change in intensity within the activities: when a child has a succession of good answers, the EPA will be more and more satisfied instead of amused. Similarly, for medium and lowlevel children, a succession of wrong answers will bring a stronger emotion of empathy and encouragement from the EPA.
The squares at the top of the emotional boxes represent the range in valence of a response. As dynamic representation of emotions in EPAs, the valence of the animations is indeed not constant, but
Finally, results after completing each question of the activity are a choice between three states “WIN”, “GOOD TRY”, and “LOSE”, according to the number of questions correctly answered and the
Figure 1: EPA’s affective response model within the OLM systems
personalized number they had to get right, taken from their usermodel:
our children’s age group, however not enough data was collected to present viable results.
·
o For a child considered able to do the activity with ease (high level): amused-satisfied for a win, unpleasantly surprised for a good try, and disappointed if s/he failed the test completely.
5. EXPERIMENTAL STUDY 5.1 Pilot study: testing the validity of the model
·
o For a child considered borderline to do the activity (medium level): satisfied when succeeded, comforting when a real effort has been made, and sad for a failed test.
·
o For a child considered unable to do the activity yet (low level): awed for success, hopeful for a good try, and empathic in case of failure.
In this section an exploratory study is introduced, which results will guide future research. For this reason, the study was performed on small numbers of participants per condition, in order to gain some insight on children’s reaction to the model while answering some research questions. Therefore, as statistics would not mean much with such small numbers, the results are presented with actual data in the form of percentages.
4.5 Validation of the emotive representation The visual representation of EPA’s emotional states in the DividingQuest and Multipliotest software were designed with English and French children as design partners, using two metaphors of learning in the design of the affective components [9]. The good recognition of such emotional states was tested during the development of an affective self-reporting tool, Sorémo [9]. The results showed a good level of understanding of the visual representations in terms of emotions represented, with the presence of the Traffic Light system or ‘colors’ reinforcing the understanding of an emotional state. The understanding of the what the emotions mean within the context of the pedagogical application was tested in another pilot study and seemed usable by
5.1.1 Aim of the study and procedure In order to validate the affective model of EPA’s responses, a pilot study was conducted on 27 English and 27 French children aged 9 to 11 years, analyzing their emotional reaction and the impact on learning of the various emotional strategies involved. It aimed at answering the following research questions: - Does the choice of emotional response in EPA impact the child’s learning strategies definition, help-seeking behavior according to the child’s knowledge prior to the activity? - Does the choice of emotional response in EPA impact the child’s emotional responses according to their knowledge level prior to the activity?
The pilot study consisted of a learning session of 30 minutes on the respective software (DividingQuest for the English children, Multipliotest for the French children). A preliminary experiment, not presented in this paper, was performed using the model embedded in the DividingQuest and showed a significant learning gain when comparing a version with the affective EPA to a version with no EPA, as well as improvements in terms of helpseeking behavior, motivation to use the application, and user’s engagement in learning activities. Two software, designed during the participatory-sessions with teachers, were used: DividingQuest [7] and Multipliotest [8], were created to investigate the impact of using Affective EPAs as interaction medium with the user-model of OLM systems. Children’s interaction with the software were recorded, as well as their emotional reactions to software interaction via webcam recordings, built into the computers used for the experimentation, and expected to be inconspicuous for the child-users.
5.1.2 Study design The study was designed as a 3 group between-subject design, with each child working on one unique user-interface. The two groups of 27 children were each represented by 9 children of each level (low, medium and high) of the mathematical skills involved in the software (divisions for the English, and multiplications for the French children). The children considered with “low” divisions skills were dividing into three groups, each group being submitted to one of the Affective Model simulation of EPA’s responses: one where the student was considered able to do the activity, one for borderline students, and one where the student was not considered able to perform the tasks correctly. The same repartition of interface per child occurred in all the groups aforementioned. Each interface illustrates on strategy of emotional responses to EPAs, as described in the model in Figure 1: the “able” interface includes emotional strategies initially designed for the more skilled students, while the “borderline” interface features emotional responses designed for medium-skilled children, and the “unable” interface for poorly-skilled students. Each interface was seen and manipulated by 9 children, of mixed abilities (3 of each level).
5.1.3 Results Three types of information are considered within this study: the impact of an EPA’s emotive intervention on children’s learning strategy, help-seeking behavior, and the user’s emotional reaction. The emotional reactions were singled out from the video stream recorded, and compared with Ekman’s FACS [6] to identify the underlying emotions. When the children were interviewed, they were asked to corroborate the identification of emotions by looking at the video recording of their session and to explain their reaction at this particular moment of interaction.
5.1.3.1 Learning strategies Several different types of behaviors could be observed, and classified according to Roll et al’s meta-cognitive ACT-R model of students’ learning strategies [21]. One strategy is looked at here in particular, corresponding in terms of student’s goal as “Least Effort Oriented”: the child begins an activity, tries a few questions, and then does not attempt to finish it, but tries other activities instead, one after the other. Two types of children appear to choose this learning strategy when confronted by pedagogical software: low ability students
who find the activities too difficult and “give up”; and high ability students who find the activities too easy and “flutter” from one activity to another in order to find one more challenging. More strategies of this sort were found in the low and moderate students working on the ‘able’ interface than the high level students, or the low and moderate students on other interfaces. When interviewed on the subject, children stated that the lack of understanding and empathy of the EPA when they failed to give the right answer discouraged them from trying the activity further. A second tendency has been revealed for the presence of this behavior: highly skilled students used this strategy more in the unable interface than in the others, when they tried the “easiest” activities to try on. However, as children were free to choose the activities they wished to work on, not enough data was registered in this pilot study to see the impact on the learning strategies chosen.
5.1.3.2 Help seeking behavior Low and moderate leveled children working on the borderline and unable interfaces asked for help more often than those working on the able interface (Avlow=6.7, Avmod=4.5, Avhigh=2.4). The display of help-seeking behavior appeared mostly after an EPA’s display of empathic or comforting behavior. However, no difference could be found in the number of times the highly skilled children asked for help between the three interfaces.
5.1.3.3 Emotional Aspect A greater level of frustration was identified in the “able” and “unable” interfaces. Respectively, more lower and higher level students were found to express frustration during the game activities than the other children working on this interface. On interviewing the children, it appears that the high level (66%) and some moderate level students (33%) working on the low level interface found the emotional display to be too compliant and too comforting, lacking in challenge, especially in the easiest activities of the learning platforms. On the other hand, the low level children working on the most challenging interface (100%) seemed to had trouble keeping up, and felt discouraged by the character’s display, whereas children of low level working on the borderline and unable interfaces expressed a low level of frustration and appeared more motivated to keep working on an activity until they understood its principles.
5.1.4 Conclusion The observations made during this pilot study appeared to give results in line with the theoretical grounding of the model: - Children’s learning strategies definition and help-seeking behavior when confronted to a particular EPA’s emotional response strategy vary according to the child’s knowledge prior to the activity. - Children’s emotional responses to software interaction when confronted to a particular EPA’s emotional response strategy vary according to their knowledge level prior to the activity. The differentiation of EPA’s emotional response to user interaction according to his/her acquisition of skills level can provide users with adaptive learning. The children seem to have understood the emotions chosen and their representations, and it seems to have some impact on the decisions they made as to software interaction and learning acquisition. It appears that the choice in responses, when the interface tested corresponded to the child’s level, help to keep children more motivated, and avoid ‘fluttering’ learning strategies to some extent, when the appropriate emotional strategy is used. Some children from the
low level group stated that after working on the unable or borderline interfaces, felt that the EPA had taught them something, and felt more secure in answering such mathematical problems.
6. CONCLUSION AND FUTURE WORK In this article, we are interested in the design of computational models of emotions for use in computational agents embodied in intelligent tutoring systems. We present a model of emotions developed with teachers as participatory-design partners, and based on current research in affective computing. The model has been especially designed for drill-and-practice mathematical OLM systems with child users aged 7 to 12 years. The pilot study hereby presented validates the use of different emotional strategies according to the teacher’s expectations concerning the child-users, defined by their current level of skills concerning the application pedagogical content. It appears that the choice in responses, when the emotional strategy corresponds to the on defined by teachers for the child’s level, help to keep children more motivated, and avoid ‘fluttering’ learning strategies to some extent. The next step is to use the model within learning software to investigate the benefits of using an EPA with such emotional strategy in comparison to more traditional software architectures when considering learning benefits, changes in learning strategy and help-seeking behavior, and use/recognition of the underlying user model in OLM systems.
7. REFERENCES [1] Baker, R., de Carvalho, A., Raspat, J., Aleven, V., Corbett, A.T., Koedinger, K.R.: “Educational Software Features that Encourage and Discourage "Gaming the System"”. AIED 2009: 475-482. [2] Batson, C., Turk, C., Shaw, L., & Klein, T., “Information function of empathic emotion: Learning that we value the other’s welfare..” Journal of Personality and Social Psychology, 68, 1995, 300-313 [3] Biswas, G., Schwartz, D., Leelawong, K., Vye, N., and TAG-V, “Learning by teaching. A new agent paradigm for educational software”, Appli. Artif. Intelligence special Issue “Educ. Agents – Beyond Virutal Tutors”, vol. 19, 2005. [4] Dimitrova, V., McCalla, G., & Bull, S., "Open Learner Models: Research Questions (Preface of the Special Issue, Part 1)." International Journal of Artificial Intelligence in Education, 17(2), 2007, 83-87. [5] D'Mello, S. K., Scotty D. Craig, Amy M. Witherspoon, Bethany McDaniel, Arthur C. Graesser: “Automatic detection of learner's affect from conversational cues”. User Model. User-Adapt. Interact. 18(1-2), 2008, 45-80
[8] Girard, S. and Johnson, H. (2008). Towards Guidelines for Designing Open-Learner Modeling Tutoring Systems: Mixing Colors and Emotions. In the proceedings of the WEITS workshop, ITS conference, Chicago, IL, pp. 44-54. [9] Girard, S. and Johnson, H., Designing affective educational software products: Sorémo, a new method for capturing emotional states. Journal of Engineering Design, Special Issue on Design and Emotion, 2009. 20(5), 2009, p. 493-510. [10] Haake, M. and Gulz, A., A Look at the Roles of Look & Roles in Embodied Pedagogical Agents - A User Preference Perspective. International Journal of Artificial Intelligence in Education, 19(1), 2009, 39-71. [11] Heraz, A., Frasson, C., “Detecting the Affective model of interplay between emotions and learning by measuring learner’s brainwaves.” WECITS’2008 (pp55-62). ITS 2008. Montreal, Canada, June 2008. [12] Klein, J., Moon, Y., and Picard, R., “This computer responds to frustration”. Interacting with computers, 14(2), 2002, 119140. [13] Kort, B., Reilly, R., and Picard, R.W., "An Affective Model of Interplay Between Emotions and Learning: Reengineering Educational Pedagogy-Building a Learning Companion," In Proceedings of International Conference on Advanced Learning Technologies (ICALT 2001), August 2001, Madison, WI. [14] Lester, J.C., Towns, S.G., Callaway, C.B., Voerman, J.L., and FitzGerald, P.J., “Deitic and emotive communication in animated pedagogical agents” in J.Cassel, S.Prevost, J.Sullivan, and E. Churchill, (eds.), Embodied Conversational Agents, MIT Press: Cambridge, 2000, 123154 [15] Mandler, G., “Mind and Body: Psychology of emotion and stress.” New York; Norton. 1984 [16] Nowak, K. & Rauh, C., Choose your ‘buddy icon’ carefully: the influence of avatar androgyny, anthropomorphism and credibility in online interactions. Computers Mediated Communication, 11(1), article 8. [17] Oatley, K., Johnson-Laird, P.N., Towards a cognitive strucutre of emotions. Vol. 13, 7, 1987, pp. 29-50 [18] Ortony, A., Clore, G.L., Collins, A.: “The cognitive structure of emotions”. Cambridge University Press, Cambridge, M.A, 1998. [19] Piccard, R.W. and Daily, S.B., “Evaluating affective interactions: alternatives to asking what users feel”. CHI Workshop on “Evaluating affective interfaces: Innovative approaches”. Portland., 2005.
[6] Ekman, P., Friesen, W., 1978. Facial Action Coding System : A technique for the measurement of facial movement. Palo Alto, CA :Consulting Psychologists Press.
[20] Rodrigo, M.M.T et al., “Affect and Usage Choices in Simulation Problem Solving Environments”. In Artificial Intelligence in Education. R.Luckin et al (Eds.) IOS Press, 2007. pp145-152.
[7] Girard, S., 2010. Traffic-Light and Smiley Faces: Do children learn better with affective OLM software?, PhD dissertation, Department of Computer Science, University of Bath.
[21] I. Roll et al., A Metacognitive ACT-R Model of Students’ Learning Strategies, J.C. Lester et al. (Eds.): ITS 2004, LNCS 3220, pp. 854–856, 2004. Springer-Verlag Berlin Heidelberg 2004
