An Affective Learning Environment for Java Ramón Zatarain Cabada, María Lucía Barrón Estrada, Francisco González Hernández, Raúl Oramas Bustillos. Instituto Tecnológico de Culiacán
[email protected] II. Abstract— We present Java Sensei, an Intelligent Learning Environment (ILE) for learning Java programming. The ILE is formed by an affective tutoring system working in a Web environment. The tutoring system was implemented under different learning methodologies like problem-solving for the pedagogical module, knowledge space for the expert module, and overlays for the student module. The main contribution of this work was the integration of cognitive and affective information in a behavior graph which is used by the pedagogical agent to show emotion and empathy to the student.
JAVA SENSEI
THE Java Sensei ILE consists of seven components (figure 1): Affective Framework, Student or User Module, Pedagogical Agent, Adaptation Module, Teaching (Pedagogical) Module, Expert Module and the ITS (Intelligent/Affective Tutoring System) Engine. Next, we describe each of the components.
Keywords: Intelligent Learning Environments, Intelligent Tutoring Systems, Affective Computing, Programming Languages, Neural Networks, Fuzzy Systems.
I.
INTRODUCTION
The teaching and learning of computer programming is a universal challenge for all colleges and universities in any country. Some factors that contribute to this challenge are: the teaching methods employed by the instructor, the study methods employed by the student and his/her abilities and attitudes, the nature of the art of programming, and the psychological influence that the student suffers from society [1-4]. Many researchers have proposed and developed methodologies and tools to help students learn programming. For example the ACM has SIGCSE (Special Interest Group on Computer Science Education) where issues related to the development, implementation and/or evaluation of programs and curricula are discussed [5]. Today, in the field of Artificial Intelligence, Intelligent Tutoring Systems (ITS) try to simulate a human tutor to provide personalized instruction taking into account not only cognitive aspects of students [6] but also affective elements [7]. This paper presents the implementation of an Intelligent Learning Environment (ILE) for learning Java programming. This tool, named Java Sensei, attempts to create a flexible and interactive web-based environment that considers the cognitive and affective states of students. The paper is organized as follows: section 2 describes the design and implementation of system components. Results and application testing are shown in section 3 and conclusions and future work are discussed in Section 4.
Figure 1. Architecture of ILE JavaSensei Affective Framework: In the Affective Module we created an emotional framework based on AutoTutor [9] where we implemented a fuzzy logic system and a neural network back propagation for emotion recognition. The emotional framework represents a student's emotional and cognitive model and is composed of input and output variables for the fuzzy logic system. In Table 1 we can see each variable of the emotional framework and next a brief description of each one. Table 1. Emotional Framework. Student Module Current Emotion Previous Emotion Global Student Skill Quality of Current Answer
Tutor Actions Feedback Empathetic and emotional responses Facial expression Intervention
Student (User) Module: The student model represents the cognitive and emotional part of the student. It consists of two emotional and two cognitive aspects. • Current Emotion: The current emotion of the student when answering an exercise. The emotion is obtained through the emotion recognition application. The possible values are: Happy, Angry, Sad, Neutral, and Surprise. • Previous Emotion: The student previous emotion. • Global Student Skill: The proficiency of the student regarding the course. • Quality of Current answer: Is the number of errors that the student makes in one exercise. Tutor Actions. The actions of the tutor represents the emotional response of a human tutor and represent the output of the fuzzy logic system that is used by the component of the Pedagogical Agent and the Example-Tracing Engine. Both will be explained later. • Feedback: This is a message that will give a positive, negative or neutral perspective to the student regarding their progress during the year. It represents the feedback that gives you a human tutor when you want to know if you're right or wrong in your academic progress. • Empathetic and emotional responses: These are phrases received by the student depending on their emotional state. The sentences will depend on the student's current emotional state and aims to give the feeling of understanding that gives a human tutor. This is not an input variable for the fuzzy logic system, since the value of a facial expression is used. • Facial expression: This is the facial expression of the Pedagogical Agent. • Intervention: This value represents whether the Pedagogical Agent must perform an intervention or not on the student.
The emotion recognition system was built in three steps: the first one was an implementation to extract features from face images in a corpus used to train the neural network. The second step was the implementation of the neural network. We used the Java-based algorithms implemented in Weka [12] to implement classification by using a neural network (feed-forward method). The third step integrated extraction and recognition into a fuzzy system, which is part of the ITS. For training and testing the neural network, we used the corpus RAFD (Radboud Faces Database), which is a database with 8040 different facial expressions that contains a set of 67 models including men and women. Once the emotional state is extracted from the student, the emotional state is sent to the fuzzy system. Figure 3 shows the steps followed in the recognition and affective feedback from the system. In the first step the ILE receives the data and the student's collection of images from the browser which are later sent to the ILE. In the second step the ILE starts a cycle with several student faces being sent to the neural network, and decides, at the end of the cycle, the current emotion. This current emotion is obtained based on the most frequent student emotion. In the third step the ILE send the updated student model to the fuzzy system which perform the fuzzy rules to output the tutor actions. In the last step, the tutor actions are sent to the browser to build the actions of the pedagogical agent. Pedagogical Agent: The pedagogical agent is responsible for transmitting messages that a human tutor would perform. For this, it has been provided with both facial expressions and dialogues that can communicate to the student. These expressions are based on facial expressions investigated by Ekman [13]. Positions of the eyebrow, mouth, and eye opening are taken into account. These positions are entered as parameters in the FaceGen Modeller [14] software. In Figure 2 we observe the four facial expressions made by the pedagogical agent of Java Sensei. These emotions were based on the pedagogical agent working with AutoTutor [10].
Fuzzy System and Neural Netwkork The fuzzy logic system seeks to represent the way a human tutor acts when he/she faces various scenarios related with cognitive and emotional situations of a student [10]. These situations have to do with the predefined actions. The fuzzy system uses the emotional values obtained by the neural network which recognizes the affective state of the student when he/she works with the ILE Java Sensei. The fuzzy logic system works with four input fuzzy variables and three output ones. A total of 144 fuzzy rules were constructed. To define these rules we use Fuzzy Control Language, a standardized language for creation of fuzzy rules. The implementation was done with jFuzzyLogic [11].
Figure 2. Pedagogical Agent of JavaSensei
Figure 3. Affective Recognition and Feedback Adaptation Module: The ILE must be able to adapt to student [8]. The ILE collects the rating produced by the student concerning the exercises and resources that the student visits. The rating of one exercise is measured using Likert scale [15] where one indicates that you totally disagree with the exercise and five indicates that you totally agree. At the beginning all the exercises are rated with a value of two. We used a recommendation system created with Apache Mahout [16]. This tool helps to find the next problem the student must solve. The system implements the K-Nearest Neighbor (k-NN) approach and the Pearson Correlation. As more users are obtained, the rating system generates the best recommendations and thereby it is more adapted to the changes. Pedagogical Module: The system uses the pedagogical model known as "problem solving" [6] where the student learns as he solves problems with a certain structure. The system uses three types of strategies for solving problems. Type 1 is used to evaluate theoretical concepts with “truefalse” exercises. Type 2 questions present a complete program or piece of code and asked the student to determine the output of the code. Type 3 is a combination of the two previous types and allows you to create more complex exercises involving "n" number of steps to reach the solution. Expert Module: This model contains expert knowledge that the student wants to learn and practice. The system represents the knowledge using JSON files (a variation of XML files). Knowledge representation uses the knowledge
space theory [20]. The expert model represents six basic skills that students must master to be modeled by a graph representing the Knowledge Space. These skills are: Introduction to Java, Variables and calculations, selection, iteration, methods, and arrays. ITS Engine: An ILE combines hypermedia techniques with other methods used by ITS. We applied Example-Tracing [17, 18] in order to implement the ITS in Java Sensei. Example-Tracing ITS have the advantage of providing stepby-step guiding to the students and providing with multiple strategies to the problem solution, including optimal or suboptimal solution or miss-conception management. ExampleTracing exercises can be easily adapted with the pedagogical model implemented with “problem solving” and they are represented by a graph that is traversed by the student (figure 4). One of the main contributions of our work is the integration of cognitive and affective information in a behavior graph which is used by the pedagogical agent to show emotion and empathy to the student. Figure 4 shows part of a behavior graph. Next, we describe the algorithm to implement the affectiveexample-tracing [19]: a) The ILE load the JSON file with the corresponding exercise. b) From the JSON file, a behavior graph is created and the initial step is executed.
c) As students traverse the graph (by clicking with the mouse), different options can be performed and different paths are traversed. The options are: i. Initial step (IS) ii. Error step (ES) iii. Optimal step (OS) iv. Sub-optimal step (SOS) v. Final Optimal step (FOS) vi. Final Sub-optimal step (FSS) d) Every step triggers two events: i. Actual Emotion Evaluation. ii. Tutor Action Evaluation.
Figure 5. Affective Example-Tracing III.
RESULTS AND COMPARISONS
The system has been tested with students in our research lab. Next, we show (figure 6) a small session of a student browsing in the ILE. To access the ILE, the student must use a browser with a PC, laptop, or mobile device (e.g. a smartphone), and a Facebook account allowing the use of the Web cam. The top part of the example shows a problem with the student making a mistake, at the end of the code. In the bottom part of the problem the student finish successfully the exercise.
Figure 4. Behavior Graph
Every time a student chose some of the options in the graph, the engine call the fuzzy logic system to get the tutor action (feedback, intervention, etc.) and to get the emotion information so the pedagogical agent can build the graph. Figure 5 illustrate an example. At the left we can observe how the student solved the problem in an optimal way. At the right the student made a mistake.
Figure 6. Session of a Student
For the evaluation of our system we took as reference the seven features that distinguish a modern ILE or ITS from traditional instructional systems Woolf [6]. Table 2 shows the system evaluation with a metric of Likert values 0 to 5, where 0 represents the features that will be added in future versions or are not covered, 1 to 4 represents an incomplete feature and 5 is a complete feature. The system was compared against ITSs ASK-ELLE [21], JavaTutor [22], SQLT-Web [23] y ELM-ART [24]. Table 2. Comparison against other ILE or ITS systems. Characteristic Java ASKJava SQLTSensei ELLE Tutor Web Generality 3 3 1 1 Student 3 3 3 3 Model Expert Model 5 5 5 5 Mixed 2 2 1 1 Initiative Interactive 5 5 5 5 Learning Instructional 2 3 3 3 Model Self2 1 1 1 improvement Total 22 22 19 19 assessment
[2]
[3]
[4]
ELMART 1 3
[5] [6]
5 1
[7] [8]
5 3
[9] [10]
1 19
According to the total assessment, Java Sensei achieves a score of 22 points, tied with ASK-ELLE, and with a higher score than the others ILEs/ITSs. We still are working to include Mixed Initiative and Self-improvement features. IV.
REFERENCES [1]
CONCLUSIONS AND FUTURE WORK
The ILE has been implemented to be acceded from different platforms (operating systems, web browser, computers, etc.) with open source code in programming languages and software libraries like Java, Javascript, MongoDB, JSON, jFuzzyLogic, Apache Mahout, Weka, jQuery, jQueryMobile among others. Right now, a group of programming Java professors is starting to test the ILE with students from our university (Instituto Tecnológico de Culiacán). For future work, we want to include multimodal affect recognition, including identification of emotion by text, voice, and EEG devices. We also want to work with emotions more learning centered (engagement, boring, frustration, etc.). Last we are developing a new version oriented to more collaboration work among students and authoring new exercises by instructors.
[11]
[12] [13] [14] [15]
[16] [17]
[18]
[19]
[20] [21] [22]
ACKNOWLEDGMENT
[23]
The work described in this paper is fully supported by a grant from the DGEST in México.
[24]
Matthíasdóttir, Á. How to teach programming languages to novice students? Lecturing or not. in International Conference on Computer Systems and Technologies-CompSysTech. 2006. Jenkins, T. On the difficulty of learning to program. in Proceedings of the 3rd Annual Conference of the LTSN Centre for Information and Computer Sciences. 2002. Gomes, A. and A.J. Mendes. Learning to program-difficulties and solutions. in International Conference on Engineering Education– ICEE. 2007. Bernard, M. and E. Bachu, Enhancing the Metacognitive Skill of Novice Programmers Through Collaborative Learning, in Metacognition: Fundaments, Applications, and Trends. 2015, Springer. p. 277-298. SIGCSE. 2015 [cited 2015]; Available from: http://www.sigcse.org/. Woolf, B.P., Building Intelligent Interactive Tutors. 2009, Amherst, Massachusetts, Estados Unidos: Morgan Kaufmann Publishers. Picard, R.W., Affective computing. 2000: MIT press. Brusilovsky, P. and E. Millán. User models for adaptive hypermedia and adaptive educational systems. in The adaptive web. 2007. Springer-Verlag. D'Mello, S., R.W. Picard, and A. Graesser, Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 2007(4): p. 53-61. D’Mello, S., et al. AutoTutor detects and responds to learners affective and cognitive states. in Workshop on Emotional and Cognitive Issues at the International Conference on Intelligent Tutoring Systems. 2008. Cingolani, P. and J. Alcalá-Fdez, jFuzzyLogic: a java library to design fuzzy logic controllers according to the standard for fuzzy control programming. International Journal of Computational Intelligence Systems, 2013. 6(sup1): p. 61-75. Hall, M., et al., The WEKA data mining software: an update. ACM SIGKDD explorations newsletter, 2009. 11(1): p. 10-18. Ekman, P., Facial Expressions. 1999. Inversions, S., FaceGen modeller (Version 3.3)[computer software]. Toronto, ON: Singular Inversions, 2008. Likert, R., A method of constructing an attitude scale. Scaling: a sourcebook for behavioral scientists. Chicago: Aldine, 1974: p. 233243. Owen, S., et al., Mahout in action. 2011: Manning . Aleven, V., et al., A new paradigm for intelligent tutoring systems: Example-tracing tutors. International Journal of Artificial Intelligence in Education, 2009. 19(2): p. 105-154. Aleven, V., et al. The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains. in Intelligent Tutoring Systems. 2006. Springer. Kumar, R., et al. Comparison of Algorithms for Automatically Building Example-Tracing Tutor Models. in Educational Data Mining 2014. 2014. Doignon, J. P., & Falmagne, J. C. Knowledge spaces. 1999. Springer. ASK-ELLE (Haskell Tutor): Gerdes, A. (2012). Ask-Elle: a Haskell Tutor. PhD thesis, Universiteit Utrecht. JavaTutor: Wallis, M. (2011). JavaTutor-A Remotely Collaborative, Real-Time Distributed Intelligent Tutoring System for Introductory Java Computer Programming-A Qualitative Analysis. SQLT-Web: Mitrovic, A. (2003). An intelligent SQL tutor on the web. International Journal of Artificial Intelligence in Education, 13(2), 173-197. ELM-ART: Brusilovsky, P., Schwarz, E., & Weber, G. (1996, January). ELM-ART: An intelligent tutoring system on World Wide Web. In Intelligent tutoring systems (pp. 261-269). Springer Berlin Heidelberg.
