An Affective and Web 3.0-Based Learning

An Affective and Web 3.0-Based Learning Environment for a Programming Language 1

Ramón Zataraín Cabada * [email protected]

1

María Lucía Barrón Estrada [email protected]

1

Francisco González Hernández [email protected]

1

Raúl Oramas Bustillos [email protected]

2

Carlos Alberto Reyes-García [email protected]

1

Instituto Tecnológico de Culiacán, Division of Research and Postgraduate Studies Juan de Dios Bátiz 310 Pte. Col. Guadalupe, Culiacán, Sinaloa, CP 80220, México 2

Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE) Luis Enrique Erro No. 1, Sta. Ma. Tonanzintla, Puebla, 72840, México

*Corresponding Author Abstract. We present a Web-based Environment for learning Java Programming that aims to provide adapted and individualized programming instruction to students by using modern learning technologies as a recommender and mining system, an affect recognizer, a sentiment analyzer, and an authoring tool. All these components interact inreal time to provide an educational set where the student learn to develop Java programs. The recommender system is an e-learning 3.0 software component that recommends new exercises to a student based on the actions (ratings) of previous learners. The affect recognizer analize pictures of the student to recognize education-centered emotions (frustration, boredom, engagement, and excitement) that are used to provide personalized instruction. Sentiment Text Analysis determines the quality of the programming exercises based on the opinions of the students. The authoring tool is used to create new exercises with no programming work. We conducted

two experiments to evaluate the learning System. The results of the evaluations showed that engineering students using our tool had…. Keywords: Web 3.0; Intelligent Learning Environment; Educational Applications

1. Introduction Learning a first programming language is a major challenge for a student of computer science or a related field (Soloway & Spohrer, 2013; Hoc, 2014). Some factors that contribute to this challenge are: the teaching methods employed by the instructor, the study methods employed by the student and his/her abilities and attitudes, and the nature of the art of programming. On the other hand, new advanced learning technologies related to what is called the Web 3.0 are actively used in the teaching scene (Hendler, 2009; Morris, 2011; Barassi & Treré, 2012; Vásquez-Ramírez et al. 2016). The combination and integration of these new web technologies like Linked Data (Bizer et al. 2009; Heath & Bizer, 2011), Big Data (Manyika et al. 2011), Semantic Web (Berners-Lee et al. 2001), and Educational Web Mining (Romero & Ventura, 2010), with new fields and application of artificial intelligence, affective computing, and education pedagogy, has created new powerful learning environments with modern interfaces for many knowledge fields. This new area has been called E-Learning 3.0 in many works (Rubens et al. 2012; Dominic et al. 2014; Hussain, 2012) marking a difference with past Web 2.0 educational systems where Social Networks played a very important role. In this paper, we present Java Sensei, a Web-based Intelligent Learning Environment (ILE) designed for learning Java programming (Cabada et al. 2015). Our learning system has an Affective and Intelligent Tutoring System (ATS) working in a web-based environment (http://javasensei.ddns.net/). The Affective and Intelligent Tutoring System adapts its content individually according to preferences of students and a student model, either changing its graphic user interface or presenting different learning material. In order to achieve this goal, Java Sensei implements a recommendation system which uses a rating method where each rating represents the user preference. The rating method uses a general algorithm which performs a comparison between a current user and other users in the database. The rated elements are exercises and help items which are saved in data server. Both elements receive ratings from all users in order to know the preference of each user. Java Sensei uses the

technique “Collaborative Filtering” (Schafer et al. 2007) by taking advantage of what a Web system is in real time. The learning environment also contains a facial expression recognizer to identify emotions in students, a sentiment analyzer to identify the quality of the exercises based on the dialogues (texts) of the students, and a fuzzy system for working the reasoning and inference of the learning (expert) system. The paper is organized as follows: section 2 describes related work of the main topics. Section 3 presents the research methodology, Section 4 shows and explains the software architecture of the Web-based system. Section 5 shows implementation of affect and intelligence into the system. Section 6 presents the experiments and results produced with the system. Conclusions and future work are discussed in Section 7. 2. Related Work This section describes research work in areas related to our investigation. 2.1. The Web 3.0 and Learning E-Learning 1.0 had learning or educational objects available online. It allowed students to read learning material stored in the Web. This learning material was created by authors of web pages (Richardson, 2005). E-Learning 2.0 allowed students not only to read learning material but also to write/create it (Richardson, 2005; Downes, 2005; Vásquez-Ramírez et al. 2014a). Users/Students also became the authors of the collective intelligence on the Web. Social Networks were the main tools to allow users/students to participate as authors of the knowledge in the different learning environments. In addition to allowing students to read and write learning material, in an E-Learning 3.0 environment, students collaborate with each other (Wheeler, 2011). Artificial Intelligence plays a crucial role in this type of learning environments by providing new methods and techniques to organize the web pages so they can be structured as a semantic web. Users (Students) can see the Web not only as data and information but as knowledge that can be accessed from a learning environment. Today, there have been many implementations of learning environments which use different technologies related to the concept of the Web 3.0. Kurilovas et al (2014) presents an analysis of interconnections between students’ learning styles, their preferred learning activities, relevant teaching / learning methods, and Learning-Object types in a Web 3.0 virtual learning

environment. Personalization of learning styles is implemented by special ontologies. In (Dietze et al. 2012), authors introduce a general approach for a Linked Education environment and describe a general approach to use existing TEL (Technology-Enhanced Learning) data on the Web by allowing to treat it as Linked Data in the educational domain. Paredes-Valverde et al (2015) propose a natural language interface that allows non-expert users to access semantic knowledge bases through the formulation of queries in natural language. The approach uses a special domain-independent ontology. In order to test the method they conducted an evaluation in the music domain using LinkedBrainz. VásquezRamírez et al (2014b) presents a system called AthenaTV to generate Android-based educational applications for TV, working with the scheme used by Google to develop interfaces based on interface design patterns established in Google TV. 2.2. Affect Recognition in Facial Expressions There are currently many works related to obtaining the current emotion in facial expressions, and they implement different methods of feature extraction and classification. For example, Khandait et al. (2011) uses a neural network with back-propagation for the classification of emotions such as astonished, neutral, sad, upset, frightened, happy and angry. They work with morphological image processing operations such as SUSAN Edge Detection. For the training of its neural network they use the JAFFE database. Gangwar's emotion recognizer (Gangwar et al. 2011) uses a neural network for the feature classification. To obtain the features of the face, the method uses Matlab Toolbox Image processing. The output of the neural network in this recognizer is a vector of 6 values, where each one indicates the similarity of the processed image with each emotion. The algorithm always selects the emotion with the highest value. De et al (2015) has a process of extracting facial features where they use an HSV color model to detect the face in the image. After that, they use PCA to reduce the size of the face dimension. Table 1 shows the main features of these recognizers as the classification method, the method used for feature extraction, and the precision rate for emotion prediction. Table 1. Affect Recognizers for Facial Expression. Reference

Classification

Khandait et al. 2011

Neural Network

Fature Extraction Method SUSAN edge detection

Rate 95.2 %

Gangwar et al. 2013

Neural Network

Image toolbox matlab

88 %

De et al. 2015

Euclidean Distance Comparation

Eigenfaces

85.38 %

There are also many valuable surveys about the state of art in face expression recognizers. One of the most recent works is from Sariyanidi et al (2015) that covers a good number of recognizers not only for basic emotions but also for non-basic emotions. It also covers different aspects of face recognization such as face registration, spatial representation, spatiotemporal representation, and dimensionality reduction. 2.3. Modern Learning Environments for Programming Languages Modern research on intelligent learning environments or intelligent tutoring systems (ITS) for programming languages has focused on different programming languages like Java, C, C#, or PHP. Wiggins et al. (2015) implemented the JavaTutor System, an affective tutoring system that uses natural language dialogues for human-to-human or human-to-computer dialogues using machine learning techniques. The system takes into account cognitive aspects, and applies the ACT-R theory of cognition for knowledge representation. Affect focused on nonverbal behavior obtained through bodily expressions and detection is performed by different hardware tools. CSTutor is a tool (Hartanto & Reye, 2013) to help students learn programming in C#. The tool incorporates anchored instruction for the domain representation of the programming language and game theory, instead of affect recognition, for motivation improvement. PHP ITS was developed to teach students to program in the PHP scripting language (Weragama & Reye, 2012). The tutoring system makes an analysis written by the student and translates it into AST (abstract syntax tree) code. To check the correctness of a program, the tutoring system analyzes whether all predicates in the goal for a particular task are present in the final state. Through this method, the student writes the solution of the task defined in its own way, as long as the final state of his/her program meets the specified target. CTutor (Kose & Deperlioglu, 2012) is an intelligent learning environment (ILE) where students perform exercises programming in C. The idea of this system is to facilitate the effectiveness of learning when students navigate through complex problems and thereby provide an advanced education system to enhance the learning process. This ILE allows teachers to create new programming exercises and manage them from the

system interface. The domain in the system is adjustable for the solution of each exercise. The tool uses a constructivist pedagogical orientation where the authors give special weight to feedback as the main source of information; moreover, its domain is formed on a constraint-based model. Describir características generals de Java Sensei que lo diferencían de otros sistemas Next, table 2 shows a comparative analysis between 11 different types of learning environments and Java Sensei. Tabla 1. Comparative Analysis of Learning Environments (LE) and Java Sensei. Learning Environment

Year

Description and Language

JITS (Sykes & Franek, 2003) BITS (Butz et al. 2004) j-LATTE (Holland et al. 2009) JavaGuide (Hsiao et al. 2009) ITS PHP (Weragama & Reye, 2012) DebugIT (Carter & Blank, 2013) CSTutor (Hartanto 2014)

2003

ITS for learning Java that generates feedback based on the student's responses. It uses the ACT-R theory. It provides a guide to find the most suitable subjects that must be studied to learn programming in C++. It is a constraint-based tutor for Java that teaches a subset of the Java programming language. It is a system that guides students to select the most appropriate questions about Java programming. The ITS analyzes the student code (PHP) and translates it into abstract syntax tree (AST). ITS to teach programming using the concept of code debugging.

CTutor (Kose & Deperlioglu, 2012) NooLab (Neve et al. 2012)

2012

Intelligent Code Tutoring System

2015

2004

2009

2009

2012

2013

2014

2012

ITS to help students learn C#. It uses the concept of anchored instruction, where teaching and learning activities are designed around a story or situation that includes a problem or issue. The domain module in the ITS can be adapted to the students. It uses a constraintbased model to help students coding C programs. ILE used in distance education with a constructivist teaching model. NooLab is focused on programming in Javascript language. It uses the concept of visual communication for the understanding of

Type of LE ITS

ITS

ITS

ITS

ITS

Differences with Java Sensei It uses AI techniques to generate feedback just like Java Sensei. It uses Bayesian networks to track the student knowledge. Java Sensei does not use a constraint-based model. Similar to ITS BITS. Java Sensei does not have something similar. There is no similar technique in Java Sensei.

ITS

There is no similar technique in Java Sensei.

ITS

In Java Sensei we have not implemented anchor instructions for the tutor. Instead we use Example Tracing. Java Sensei does not use a constraint-based model.

ILE

ILE

In NooLab the user writes code, in Java Sensei the code is only evaluated.

ILE

There is no similar technique in Java Sensei.

(Kadge et al. 2015) The JavaTutor System (Wiggins et al. 2015)

2015

large datasets and thus facilitate their programming. It produces a Flowchart. It is an ILE for Java with multimodal affect recognizing that takes into account cognitive and affective aspects of students using different hardware tools to recognize their affective state.

ILE

It is similar to Java Sensei with the difference that this system handles dialogs for affect recognizing.

There are different advantages of Java Sensei over these similar systems. One crucial advantage that Java Sensei has over other systems is that our ILE has the capability to recognize learning-centered emotions, unlike other ITS and ILE that recognize only basic emotions (i.e. anger, fear, sadness, happiness, disgust, and surprise) which are not generally relevant to the process of learning (D’Mello et al. 2008; Lehman et al., 2008). This represents a weakness in the other systems since emotions such as engagement, frustration, boredom, or confusion (non-basic emotions) dominate in a learning environment. A second advantage is that Java Sensei is focused on the improvement of good programming practices. For example, when the student finds a solution to a problem, the system recommends a better or optimal solution. Another advantage of Java Sensei is a recommendation system that uses the student model to customize exercises and assistance depending on the type of user (student). Regarding the use of technologies, the main advantage of Java Sensei is that it can be used on any kind of computer (PC, laptop, mobile) that supports a browser and has a camera and a microphone installed, which is common today. Unlike other similar systems, Java Sensei needs no special sensors for emotion recognition. 3. Research Methodology For the development of the project, the Rational Unified Process (RUP) methodology was used in combination with a methodology for designing and evaluating software architectures that satisfy both functional and quality requirements. The project development process was based on the development cycle of a software system. This process contains the next stages: Requirements Analysis. The purpose of this stage of development was to establish the function and quality requirements that our system must fulfill. Some activities were:



Study of the different types of intelligent learning environments with a focus on collaborative, affective, mobile, and web-based learning.



Research of intelligent tutoring systems with recognition and multimodal handling of affection and sentiment.



Study of programming languages, database handlers, and libraries with a focus on the Web, database, and artificial intelligence to be used in the implementation of the entire project.

Specification of use cases. This stage aimed to establish the different requirements that each of the scenarios used to achieve the proposed goals. It also establisheed the communications that were carried out between the actors of the system. Design of the Software Architecture. In this stage, the goal was to distribute the system's functionalities in various components that interacted with each other to achieve the general functioning defined in the functional requirements, as well as to determine the architectural model that best adapts to the established quality requirements. Implementation of components. The purpose of this stage was to implement each component of the software architecture according to its specifications and design. Integration of components. The functionalities of all the components (intelligent tutoring system, recommender system, individual recognizers of emotions or affection, etc.) were integrated in one, the Web 3.0-Based Learning Environment. Experiments and Analysis. A set of experiments and/or assessments with students using the environment were carried out. The results of these experiments were then analyzed. The experiments were performed first with a group of graduate students and then with a group of engineering students in computer systems. 4. The Design of Java Sensei In this section the design of the software architecture and the Web interfaces of Java Sensei are presented and explained. Java Sensei has a layered design in order to organize its modules and components. This layered design allows scalability and easy maintenance because its tasks and responsibilities are distributed. Information is exchanged among every layer,

module, and component. This architectural style was proposed due to its ability to add new features, so future changes such as new methods of emotion recognization can be carried out without affecting other components. It also enables a better control over data persistence, allowing the addition of more repositories to process thousands of students' photos and log data.

Each responsibility and description are defined below. Figure 1 illustrates the

organization and structure of the layers in the software architecture.

4.1 Presentation Layer This layer contains the components that interact with users. Interactions can be made through screen, mouse, keyboard, or any type of input. It is the layer where the users interact. The components of this layer are responsible of several tasks such as generating web content, building web user interface, and controlling the information flow in the system. The modules are described as follows.

Figura 1. Architecture of Java Sensei Authoring Tool and Example-Tracing Interpreter This module contains the authoring tool which works to create exercises from a web interface and the Example-Tracing Interpreter that provides step-by-step guide to students and multiple strategies to the problem solutions, including optimal, sub-optimal, and missconception solutions. In the first case, it is the teacher/instructor who performs the activity. The interface shows an easy-to-use tool wherein an instructor can create new exercises. The module contains two components. The first one allows the creation of new exercises from scratch. The second component allows Java sensei to follow the student's response and compare it to the correct solution. The Authoring Tool Interface is shown in figure 2.

Fig 2. Graphic user interface of authoring tool. The main task of the interpreter is to perform the Example-Tracing Approach (Aleven et al. 2009; Aleven et al. 2006). The module integrates a tree of steps which contains metacognitive and emotional data. The system will execute the Example Tracing algorithm to traverse the graph, selecting the nodes on each step, according to the input of the student. The type of available steps (nodes in figure 2) are initial step (P.I.), error step (P.E.), optimal step (P.O.), sub-optimal step (P.S.O.), final optimal step (P.F.O.), and final sub-optimal step (P.F.S.). Web User Interface Generator This module contains five components responsible for making web user interfaces. It receives data which is used by the module components. Components have different tasks depending on which type of interface will show. The component Exercises generates a structure which contains a question with multiple choice answers that are presented as buttons. The component Pedagogical Tutor creates a visual representation of a pedagogical agent wich can show different types of texts and facial expressions. Currently, it has four facial expressions (delighted, surprised, empathetic, and skeptical), and two types of texts

(metacognitive and emotional texts). The component Resources builds a visual representation of help items for the user. It shows diverse contents such as videos, codes, explanations, and images. The component Course Content shows the content of a course that a student can take. It shows the progress of covered and uncovered topics. The visualization is through a collapsible menu divided by main topics which contain ten lessons each. The last component of this layer is Recommendations. It shows users a menu of exercise recommendations. This menu contains a set of buttons which represent top-rated exercises. The menu is different for every user because ratings vary among users. When the user selects one button, the exercise is shown. Figure 3 shows an example of recommendations provided by the system to help the student.

Fig 3. Exercise recommendations for users when they enter the help menu Facebook login is a third party component which allows avoiding implementing a custom login. System login component provides user information as metacognitive and emotional data, advances in the course content, reviewing of test scores, exercises, and help item ranking. 5. Integrating Intelligece and Affect in Java Sensei Tutoring System In this section we present with details the implementations of the components related to the Afective and Intelligent Tutoring System. The components were organized into two big layers of the software architecture. The tutoring layer and the intelligent layer.

5.1 Tutoring Layer This layer contains all modules related to the Intelligent Tutoring System including the recommender system. The modules are described below. ITS Module This module was implemented using the problem-solving approach wherein students learn by following a clear structure and using different solutions. The solutions are presented repetitively until the student gets a complete understanding of the problem. There are three types of strategies for solving problems. The first strategy evaluates theoretical concepts using exercises with “true-false” options. The second strategy shows some code that represents a partial or complete program and asks students to define the output of the code. The third strategy allows to create more complex exercises that include a finite number of steps to reach the solution. Domain Management. This module contains expert knowledge that the student wants to learn and practice. Knowledge representation uses the knowledge space theory (Doignon, 1999). The expert model represents six basic skills that students must master to be modeled by a graph representing the Knowledge Space. These skills are introduction to Java, variables and calculations, selection, iteration, methods, and arrays. Student Management. This component stores the cognitive and emotional information of the student. The stored information is Current Emotion (the current emotion of the student when answering a Java programming exercise); Previous Emotion (the student’s previous emotion); Global Student Skill (the proficiency of the student regarding the course); and Quality of Current answer (the number of errors that the student made in one exercise). Recommender System Module This module manages part of the adaptation on the system. The module adapts its content individually according to student preferences. Those preferences are ratings of elements in the system performed by students. Recomendations change for each user according to given ratings. Each rating represents the student preferences (Ricci et al. 2011). The component implements a collaborative filtering (CF) approach for a learning environment in the Web, where it is possible to receive ratings from different students in different platforms (PC or

mobile devices). Users (students) give a rating to elements and these ratings affect predictions or recommendations. This type of rating is known as explicit feedback (Koren & Bell, 2011). In order to generate a representative recommendation, the system predicts the rating of an element using ratings from other users with a similarity in older ratings of other elements (Desrosiers & Karypis, 2011). Next, we explain the implementation of the recommendation system. The recommender system follows a general algorithm to create recommendation items. The first step is collecting preferences from a database and saving preferences into a file. The file contains rankings from every exercise and help item, solved and used by the student. This file contains the key properties to build the system recommendations which are: student identification, item (exercise and help) identification, and the rating element. The second step of the system is finding similar users. In this step, we need to measure the similarity among all users in the database. In order to achieve it, we used Pearson correlation. The correlation coefficient measures how two sets of data fit on a straight line; named best-fit line because it tries to come as close to all the items as possible. The correlation is computed according to equation 1.

𝑃𝐶 (𝑤, 𝑢) =

∑𝑖(𝑟𝑤,𝑖 −𝑟̅𝑤 )(𝑟𝑢,𝑖− 𝑟̅𝑢 )

(1)

√∑𝑖(𝑟𝑤,𝑖 −𝑟̅𝑤 )2 √∑𝑖(𝑟𝑢,𝑖 −𝑟̅𝑢 )2

Where w and u represent the two users or items for which the coefficient is being calculated, i is an item, the two r´s are individual ratings from w and u for i, and 𝑟̅𝑤 and 𝑟̅𝑢 are average ratings for user (or item) w and u, respectively. The third step of the system is ranking people. That means that it is necessary to save all calculated similarities in a data matrix. Also, we need to save the top similarity for every user (with respect to other users). For practical reasons, we only use the top five ranking for every people. Finally, the fourth step is to generate recommendation items. One example of the process is presented next (and using table 1): a) Rankings from top five people are extracted from the database. These values are columns ex 1, ex 2, and ex 3 in table 1.

b) Every item is multiplied by the similarity value from the user. These values are columns S.xEx 1, S.xEx 2, and S.xEx 3 in table 1 too. c) All S.x fields are added and the result is written in row Total. d) Exercises with more grades have a major value. In order to normalize this problem (partially evaluated exercises), every similarity from users who ranked the exercise is added. The result is written in row called Sim. Sum. e) Every value of field Total is divided by the value of field Sim. Sum. The result is written in row Total/Sim. Sum. This value represents the prediction for the user.

Table 1. Example of rating for one person as demonstration. Person

Similarity

Ex 1

S.xEx 1

Ex 2

S.xEx 2

Ex 3

S.xEx 3

Person 1

0.98

3

2.94

1

0.98

3

2.94

Person 2

0.75

5

3.75

2

1.5

0

Person 3

0.23

4

0.92

5

1.15

0

Person 4

0.9

2

1.8

Person 5

0.12

5

0.6

3

0

3

2.7

0.36

1

0.12

Total

10.01

3.99

5.76

Sim. Sum.

2.98

2.08

2

Total/Sim. Sum

3.35

1.91

2.88

5.2 Intelligent Layer This layer keeps components responsible for processing and making decisions with data generated by the system. The components inside this layer take intelligent actions in order to perform tutoring activities. This layer receives requests from the Tutoring Layer and collects data from the Web Content Layer. Fuzzy Logic Module This module contains a fuzzy-logic system for implementing the reasoning and inference of the learning (expert) system. Fuzzy rules were built considering student emotions and other pedagogical aspects such as the quality of answers or global ability. In total there are 495 rules. Fuzzy sets and fuzzy rules were written using Fuzzy Control Language (FCL) and the implementation was supported with library JFuzzyLogic (Cingolani & Alcalá-Fdez, 2013).

Emotional values are extracted from modules Facial Emotion Recognition Module and Sentiment Text Recognition; the values are produced from photographs of students (face expressions) taken from the ILE interface and from text dialogs entered by students. The cognitive values are generated by calculating the completed tasks (Java exercises) and incorrect/correct answers in the ILE. The tutor can produce output variables wich are reactions and actions; some of the reactions are positive/neutral/negative feedback messages about the progress of the student. The trasmition of messages to students is done by a pedagogical agent. The agent shows different facial expressions and dialogues in order to communicate with the student. There are three types of answers: Feedback (messages with a positive, negative or neutral perspective regarding the progress during the year); Empathetic and emotional responses (phrases received by the student depending on their current emotional state); Intervention (it represents whether the Pedagogical Agent must perform an intervention or not on the student). Linguistic or fuzzy variables were established by two teachers with expertise in java. The variables are a representation of a state; in the case of cognitive states, they are represented by values bad or good; in the case of emotional states, a linguistic variable represents each emotion (frustration, boredom, engagement, and excitement). Sentiment Text Recognition In Java Sensei we used Sentiment Text Analysis to determine the quality of the programming exercises based on the opinions of the students. With this information, the instructor will be able to know how to improve the programming exercises based on the student’s opinion. To perform the Sentiment Analysis it is necessary to have a data corpus labeled with at least two categories (positive and negative). In the case of Java Sensei, the TASS corpus (Liu, 2012) is used. Figure 4 shows the class diagram for the module SentiTextSensei that is responsible for performing the Sentiment Analysis in the system. To implement the Sentiment Analyzer, we have to perform a cleanup of the data in the corpus. First you have to convert all the text to lowercase letters; second, punctuation must be eliminated; third, stop-words are eliminated; and finally the emoticons are translated to an equivalent text. Class CorpusProcessing is responsible for performing these tasks through the methods loadCorpus(), cleanCorpus(), and processCorpu (). Class CorpusProcessing

generates a new data file or Corpus. This file is used as input for class TrainCorpus which is responsible for converting the texts of the corpus into a series of numbers by extracting their most relevant features using method featureExtractor(). With method setClassifier(), a Nayve Bayes classifier algorithm is used to train the corpus using method trainModel() that generates a new file with the already-trained corpus. We used the Naïve Bayes Classifier because it is the simplest and most widely used classifier. As mentioned by Liu (2012), a Naïve Bayes classification model computes the posterior probability of a class, based on the distribution of the words in the document. The model works with the Bag of Bows (BOWs) feature extraction which ignores the position of the word in the document. It uses Bayes Theorem (Equation 2) to predict the probability that a given feature set belongs to a particular label P(label | features): 𝑃(𝑙𝑎𝑏𝑒𝑙 |𝑓𝑒𝑎𝑡𝑢𝑟𝑒𝑠) =

P(label)∗P(features|label) P(features)

(2)

P(label) is the prior probability of a label. P(features|label) is the prior probability that a given feature set is being classified as a label. P(features) is the prior probability that a given feature set occurs. Finally, class SentimentPredictor receives the already-trained corpus and the text to be evaluated. With method predictor(), the sentiment analysis is performed and the polarity of the text (positive or negative) is obtained.

Fig. 4. Class Diagram for module SentiTextSensei .

Facial Expression Recognition Module

In order to recognize facial expressions in the learning environment, we decided to build our own facial-expressions database or corpus. The database should contain faces expressing emotions directly related to learning. Every face in the database is labeled as one of the education-centered emotions (frustration, boredom, engagement, and excitement). This set of emotions have been proven to always be present during the process of deep learning (Baker et al. 2010; Bosch et al. 2015). In order to accomplish this task, we used an electroencephalography (EEG) technology called Emotiv Epoc. The process of creating the database is shown in figure 5. Emotions were captured during one learning session where a student solved a problem by coding a Java program using a Java environment (1). Meanwhile, the Emotiv EPOC device captures brain activity (step 2-left) of the student and a webcam takes a photograph of her every 5 seconds. Every student photograph is labeled with an emotion obtained from the Emotiv EPOC device (step 2-right). Both objects (photograph and emotion) are saved into the facial expression Database (step 3). At the end we obtained a database containing a total of 7,019 photographs.

Fig. 5. Methodology to create the face expression database. Figure 6 shows the class diagram of the module for emotion recognization. The class diagram reflects 3 activities of the System: corpus creation, classifier training, and emotion recognition. The relationship between processes and classes is explained below:

As previously stated, in the process of creating the corpus the student photographs taken by a webcam are associated with an emotion obtained from the Emotiv Epoc device. However to save the results to a database, we need to create a data structure that can save them. The class CorpusDBObject represents this result containing emotion and photography as its attributes. The class CorpusCollection is an administrator of a set of objects of type CorpusDBObjects, and the attribute listPhotos represents that set. The method addCorpusObject() adds objects of type CorpusDBObjects to the set and the method saveDatabase() saves the whole set to the database.

Fig. 6. Class Diagram for module FacialEmotionRecognition. For the training process we defined the interface ITraining which contains the following methods: loadData() to load the photographs with their respective labels; configure() to set the classifier configuration parameters; train() to run the training process; testScore() to evaluate the classifier and set a reliability score, and saveModel() to save the training to disk. The class TrainingSVM has an inheritance relationship with ITraining and uses a Super Vector Machine (SVM) algorithm. The attributes data that represents the already loaded photographs, classifier that is the classifier as a model, and extractor which obtains face features like eyes, forehead, nose, and eyebrows. In the recognition of emotions we need to go through two important steps: feature extraction and class prediction. The class Recognizer contains the method getEmotion() which receives the photograph and returns a result that is the current emotion. The attributes classifier and

extractor are needed to perform the recognition process. The class FeatureExtractor has two attributes which represent the photograph and the feature extraction method. The method getFeatures() provides the features of the photograph and uses both attributes. The class ClassifierSVM performs the prediction with the method predict() and loads the model created in the training with the method loadModel(). Our emotion recognizer was implemented based on the method local binary pattern (LBP) (Happy & Routray, 2015). In this technique, the recognizer identifies emotions by dividing the user’s face in active facial patches. The algorithm for emotion classification was implemented with a support vector machine (SVM). Whe used scikit-learn (Pedregosa et al. 2011) as a support tool to create the SVM. 5.3 Web Content Layer This layer contains four repositories: Sentimental Text Corpus: contains the TASS Corpus described above. Fuzzy logic rules (FCL files): contains rules defined with the fuzzy control language so that this repository is repeatedly used by the Intelligent Layer. Resources repository: This repository saves files that are used by other layers. Exercises repository: They are files needed to build interfaces and get recommendations in the Presentation Layer as well as the Tutoring Layer. 5.4 Data Layer The system has a MongoDB database containing information on the Student and the Domain Management Modules which in turn manages the requests of these components for reading or updating data. 6. Experiments and Results In order to test the ILE Java Sensei, we conducted two evaluations with students from the Instituto Tecnológico de Culiacán. Both evaluations were carried out with a group of engineering students in computer systems and mechanical engineering. 6.1 Evaluation of Technology Acceptance We selected the Technology Acceptance Model (TAM) to investigate the impact of technology on user behavior. TAM (Davis, 1986) is one of the most accepted theories in e-

learning acceptance studies (Teo, 2009). We discussed the relationship between Java Sensei and Perceived Usefulness (PU), Perceived Ease of Use (PEU), Perceived Enjoyment (PE), Attitude Toward Using (ATU), and Intention to Use (ITU). This leads to the following hypotheses: H1. Perceived ease of use (PEU) will positively affect perceived Usefulness (PU). H2. Perceived ease of use (PEU) will positively affect perceived enjoyment (PE). H3. Perceived ease of use (PEU) will positively affect attitude through activity (ATU). H4. Perceived enjoyment (PE) will positively attitude through activity (ATU). H5. Perceived usefulness (PU) will positively affect attitude through activity (ATU). H6. Perceived usefulness (PU) will positively affect attitude through activity (ATU). H7. Perceived enjoyment (PE) will positively affect intention to use (ITU). H8. Attitude through activity (ATU) will positively affect intention to use (ITU). A seven-point Likert-type scale ranging from (1) ‘‘strongly disagree” to (7) ‘‘strongly agree” was used to answer the ten questions in the seven constructs of the questionnaire. Participants. We asked 43 engineering students to complete a preliminary questionnaire of 10 items. There were 14 women and 29 men, all of them of Mexican origin. Participants’ ages ranged from 19 to 22 years old, with an average age of 21 years old (SD=N). Procedure. The study was divided in two sessions. In the first session, the students were using the tool under teacher supervision. This session last two hours and was carried out in a computer science laboratory at the Instituto Tecnológico de Culiacán. After this session, for two days students were able to enter and use the tool without supervision. The last day the students completed the usability survey. The time to complete the survey was 10 minutes. Data analysis and results.

The questionnaire statements of the survey and their descriptive statistics are presented in Table 2. All mean values are within a range of 4.88 and 5.77. The standard deviation range was from 0.93 to 1.47. Table 2. Results of Survey with mean scores and standard deviations. Questionnaire statements Perceived usefulness (PU) PU1. The use of Java Sensei can help me improve my academic performance in Java programming courses. PU2. Java Sensei is useful for learning the basics of programming in Java.

M

SD

5.53

1.20

5.81

0.93

5.81 5.40

1.24 1.45

5.53 4.88

1.24 1.47

5.26 5.42

1.27 1.22

5.77

1.19

5.51

1.30

Perceived ease of use (PEU) PEU1. The Java Sensei user interface is easy to use. PEU2. Interacting with Java Sensei is easy because it does not require much mental effort.

Attitude toward using (ATU) Java Sensei ATU1Using Java Sensei in the classroom is a good idea. ATU2. Learning Java with Java Sensei is more interesting than a traditional class.

Perceived enjoyment (PE) PE1. I enjoyed learning to program in Java with Java Sensei. PE2. It was fun learning Java programming with Java Sensei.

Intention to use (IU) IU1. I would like Java Sensei to be applied with other programming languages. IU2. I would recommend Java Sensei to all my friends.

A reliability analysis was completed in order to confirm the internal validity and consistency of the items used for each variables. A Cronbach alpha was calculated for the statements belonging to each construct in the research model. Cronbach alpha values from 0.6 to 0.7 are considered the lower limit of acceptability. Table 3 shows the reliability of the measurement scales. The obtained Cronbach alpha for Perceived enjoyment and Intention to use are at a very satisfactory level, as shown in Table 3. In the case of perceived usefulness, perceived ease of use and attitude toward Java Sensei, the values are in the limit of acceptability. Table 3. Cronbach’s alpha (Reliability). Variable Perceived usefulness Perceived ease of use Perceived enjoyment

Cronbach’s alpha 0.66 0.65 0.92

Attitude toward Sensei Intention to use

Java

0.67 0.77

From the results obtained, we can see that the PE factor is excellent so we can infer that the students enjoyed using the Java Sensei tool and that the intention to use the tool is acceptable to good. On the other hand, a moderately acceptable relationship can be observed between the PEU, PU and ATU factors suggesting that the user interface of the system should be improved to be easy to use. To verify all hypotheses (H1 to H8), regression analysis was applied to study the relationship between pairs of the variables defined in our research model. All test were conducted using a two-sided alpha level of 0.05. The results of the regression analysis are presented in Table 4. Table 6. Summary of hypotheses testing. Hypothesis

Dependent variable

Independent variable

Coef

R2

p-value

*H1 H2 *H3

Perceived ease of use Perceived ease of use Perceived ease of use

0.30 0.51 0.32

0.28 0.50 0.30

0.0001 0.0000001 0.0001

H4

Perceived enjoyment

0.68

0.67

0.00

H5

Perceived usefulness

0.46

0.44

0.000001

*H6 H7 H8

Perceived usefulness Perceived enjoyment Attitude through Java Sensei

Perceived usefulness Perceived enjoyment Attitude through Java Sensei Attitude through Java Sensei Attitude through Java Sensei Intention to use Intention to use Intention to use

0.39 0.58 0.64

0.37 0.57 0.6

0.000009 0.00 0.00

Figure 7 shows that hypotheses H1, H3, and H6 are rejected since their R2 values indicate that there is no correlation between the dependent and independent variables. That is, a direct relationship between perceived ease of use and perceived usability was not detected, and ease of use has no relation with attitude when using the tool. In addition, perceived utility of the tool does not influence the intention to use. On the other hand, from the statistical regression analysis we found that the students consider that the tool is easy to use (PEU) and therefore we can determine that it influences students to continue using the tool (ITU) since they enjoy learning when using the tool (PE, PU).

Figure 7. Results of regression analysis.

6.2 Evaluation of Academic Performance The second assessment focused on the learning gain aspect when a student is using the tool. Our research is focused on answering the following question: Is there a significant difference in the averages of student scores when using the Java Tutor tool? Our null hypothesis (H0) is that the student by using the tool will not improve the gain in learning: the alternative hypothesis is that the tool will improve the gain in student learning (H1). Participants. The participants were 36 computer system students of the Instituto Tecnológico de Culiacán of which 26 (72%) were men and 10 (28%) were women. Participants were between 18 and 23 years of age. Procedure. The experiment used a comparison design of two paired samples. Two tests were designed. A pretest before the application of the tool with a duration of 25 minutes and a posttest after using the tool with a duration of 25 minutes. The topics evaluated were: basic principles of Java, variables, and selection staments. After the students took the pretest, they had a session using the tool under teacher supervision. This session last two hours and was carried out in a computer science laboratory at the Instituto Tecnológico de Culiacán. After this session, for two days students were able to enter and use the tool without supervision.

Data analysis and results. Figure 8 shows a bar chart which represents the frequency histograms of the student's grades. In the X axis of the graph the student's scores are shown, which are grouped in ranges of 10 in 10. The Y axis shows the frequency obtained from the students that are in that range. Within

Frequencies

each bar, the individual frequency of each range is added. 12 11 10 9 8 7 6 5 4 3 2 1 0

11 9 8

8 7

7 6 5

PreTest PostTest

3 2 1

2

2

1

0 0

0

0

0

10

20

30

40

0 50

60

70

80

90

100

Scores

Figure 8. Student grades in the pretest and postest.

In the pretest, the highest distribution is in ranges 5, 6, 7 and 8. While in the posttest scores are grouped in ranges 7, 8, 9 and 10 that are clearly greater. To validate the scores of both tests, a Student's t-test was calculated for paired samples with an alpha value of 0.05 and twotailed test. The results of the test are presented in Table 7, which also contains the arithmetic mean, the standard deviation, and the value t. Table 7. Results of pretest and posttest. Groups Pretest Posttest Difference

Count 36 36 36

Mean 56.6944444 76.6666667 -19.9722222

Std Dev 17.4729496 12.8062485 17.2137588

Std Err

t

df

Cohen d

Effect r

2.8689598

-6.96148556

35

1.16024759

0.76200321

With the results of the previous table we obtain the p-value to know if there are significant differences in the means of the paired sample. The results of the test are shown in Table 8.

Table 8. Results of p-value. One Tail Two Tail

p-value 2.142E-08 4.2839E-08

t-crit 1.68957246 2.03010793

lower -25.7965203

upper -14.1479242

sig yes yes

Table 7 shows a significant difference between the two arithmetic means with a total value of 19.97. Table 8 indicates a p-value