Data Mining of Computer Game Assisted e/m-learning ...

Data Mining of Computer Game Assisted e/m-learning Systems in Higher Education Petar Jurić*, Maja Matetić** and Marija Brkić** * Primorsko-goranska **

County, Department of Information Technology Services, Rijeka, Croatia University of Rijeka, Department of Informatics, Rijeka, Croatia [email protected], [email protected], [email protected]

Abstract - One of the major challenges in the implementation of e-learning systems is motivating students to use it. E-learning systems are mostly unadapted to mobile platforms because they do not use responsive web design and progressive enhancement based on browser, device or feature detection. This paper presents the evolutionary cycle, current status and potential development of new platforms for e/m-learning in higher education. Educational computer games that serve as a motivational element in e/mlearning are presented, as well as the data structures that can complement the existing models of e/m-learning. The University of Rijeka uses MudRi e-learning system, which is based on Moodle open source software. This system collects data on the activity of participants in the educational process, which is suitable for data mining. A preliminary research on the correlation between the activity in forum discussions and the course pass or fail grade is carried out. The possibility of using data mining based on the data structure of the Moodle system and the models of e/mlearning and learning through play is described, aiming at finding optimal ways of learning and optimal learning results. Finally, a model of integrated system user interaction on which our future research will be based is presented.

I. INTRODUCTION Technologies available through new generation mobile devices, such as smartphones and tablets, are increasingly used as a primary platform for Internet access. These devices are suitable for learning on the go, and their capabilities often surpass desktops and laptops, primarily because of the sensors such as accelerometers, gyroscopes, magnetometers, and GPS which desktops and laptops lack. Using them for learning can be fun and available wherever there is mobile signal. Unfortunately, to the best of our knowledge, no statistics exists on the number of students in Croatia who use these devices. Therefore, in our future work we plan to investigate this issue thoroughly. Mobile learning (m-learning) systems can directly and indirectly generate and store large amounts of data. Direct data refer to the learning materials such as video lectures, presentations, forum posts, etc. Indirect data are user activity logs, as well as user locations when mobile phones are used for accessing the system. Motivational element is an important part present in all forms of the educational process. The main motivation of students in the educational process of higher education is based on collecting the required number of points with

868

regard to the course requirements, regardless of the content and the form. Since learning with reduced motivation results in lower exam achievements, reaching the highest level of creative knowledge in students is not possible without high motivation. The use of serious and educational computer games can increase the level of interactivity in e/m-learning systems. When students master educational content via computer game elements, the level of their motivation is raised because learning becomes fun. Learning through computer games is an upgrade of e-learning. Therefore, the same advantages and disadvantages presented in the subsequent section relate to them, but with the positive effect on motivation, and negative effect on the time needed to create the system. Data mining techniques are used to analyse data obtained through these systems. By performing data mining in e-learning systems, new methods for specific types of information related to education are developed [1]. In addition to the data contained in the user profile and system check-in, actions taken by the student based on the available content can also be monitored. II. E-LEARNING SYSTEMS There is a number of e-learning definitions. Elearning is often seen as an upgrade of distance learning [2] and computer-assisted learning. All of these categories are characterized by the lack of face to face interaction between the student and the teacher and by some kind of technology (usually a computer) that is used for communication as well as for accessing educational content [3]. Advantages and disadvantages of e-learning are described in Table I [4]. From our point of view, elearning systems can be described as: 

classic – fixed (desktops) or portable (laptops, which are portable, but not mobile in the sense that in general they cannot be used while walking or standing)



mobile – can be used while moving.

We strongly believe that e/m-learning on mobile devices can provide equivalent or even better content experience compared to conventional devices, i.e. desktops and laptops, ever since 2008 onwards, or more precisely, since the advent of smartphones and tablets.

MIPRO 2014/CE

TABLE I. Student Advantages Interactivity

Disadvantages Alienation

Availability

Need for selforganization

Independence from group

Poor motivation

Self-assessment and progress tracking

Rare physical contact with teacher

ADVANTAGES AND DISADVANTAGES OF E-LEARNING SYSTEMS Teacher Advantages Automatic system adjustment to the student on the basis of prior knowledge Availability

Disadvantages Less personality and possibilities for intervention

Less activities involving education and more involving moderation Besides IT resources, Communication with the other material resources student is primarily are less necessary written Students' feedback Rare physical contact with teacher

A. Classic systems for e-learning Classic systems are adapted for use on devices with large screens and high resolution. They are most commonly supported in web environments (an Internet browser is sufficient for use) of the following operating systems: Windows, Linux and Mac OS. The keyboard and the mouse, and lately the touchscreen, are used as primary input units for interaction with these systems. B. Systems for mobile learning With the invention of PDAs (personal computer assistants) and mobile phones with small screen resolution, m-learning comes into existence and makes it possible to use the e-learning system anytime and anywhere. In the past six years m-learning has become linked to smartphones. A smartphone is a mobile phone with a mobile operating system that provides standardized interface and platform for application development. It has a built-in camera, a web browser, location services support, and the ability to play multimedia. The most common mobile operating systems are Android and iOS, followed by Windows Phone, Symbian, BlackBerry OS, Bada and others. The touchscreen is used as a primary input unit for interaction with the smartphone system. A tablet uses a mobile operating system and is basically a smartphone with a giant screen; the possibility of making telephone calls is set aside in favour of the facilitated interaction with the device. With regard to the education system, m-learning is more represented in higher education because most of the students have their own mobile devices. Up to this point of time, courses have not yet been developed in accordance with the capabilities of technology. Most of the applications of m-learning available at universities include news, and calendars with lecture schedules, instead of the content to be learned [5]. Although using m-learning on devices that support new technologies is conducive to students in comparison to the conventional e-learning, exam results might be weaker [6]. The reason for this is usage on the move or in situations where the environment can adversely affect concentration. Therefore, it is necessary to consider additional elements, such as learning through computer games, in order to gain more attention and put more focus on the content. Furthermore, the fact that all new mobile

MIPRO 2014/CE

Educational institution Advantages Unlimited number of students

Disadvantages Not all students and teachers have IT skills

Availability

Dependence on IT

Lower cost in the long Poor influence on run usage and motivation Quick course adjustment

Long-term development

operating systems support multi-touch (up to 10 fingers simultaneously) and identifying the orientation of the screen needs to be taken into account. Hence, it is necessary to use new methods and technologies in developing systems for mobile e-learning. To achieve greater unification in developing elearning systems, the technology such as HTML5 ecosystem can be used, which tends to reduce the problem of fragmentation among and within mobile and classic devices. III. COMPUTER GAME-BASED LEARNING Computer games are interactive software applications that have fun as the basic objective while users engage in them. By using multimedia, networking and other technologies, they lead users to solve problems and achieve goals in a virtual environment [7]. Unlike computer games, whose primary purpose is fun, learning through computer games should include the following: 

motivational elements but not necessarily fun,



active participation in problem solving,



providing feedback,



adjustment of the level of knowledge,



clarity in the presentation of the objectives and learning content,



presentation of content that can be replicated in the virtual world, while drawing parallels with the real world,



scalability to a large number of simultaneous users.

Computer games with the above listed elements are called educational games. In addition to educational, there are also serious computer games with fun as a motivational element. These are primarily regular games with educational elements, e.g. simulations. The two areas are often intertwined in order to achieve good balance of educational content load and fun. According to Prensky [8], the new generation of students born in the last quarter of the 20th century is developing different cognitive styles in comparison to the previous generation (Table II).

869

TABLE II. NEW PREVAILING COGNITIVE STYLES IN STUDENT LEARNING: GENERATIONS WHICH GREW UP WITH COMPUTER GAMES New cognitive styles

Old cognitive styles

Twitch speed

Conventional speed

Parallel processing

Linear processing

Graphics first

Text first

Random access

Step-by-step access

Connected

Standalone

Active

Passive

Play situations

Work situations

Payoff

Patience

Fantasy

Reality

Technology-as-friend

Technology-as-foe

Related studies point to the benefits of designing educational content through computer games, because students have shown several times higher interest, i.e. the time spent using the content has been recorded as follows: 7-8 times higher compared to the textual content, and 3-4 times higher compared to the audio/visual content [9]. The disadvantage is reflected in the demanding and time-consuming preparation. According to students, motivational elements to use computer games in higher education sorted by relevance are the following [10]: 

challenge – appropriate level of difficulty, achievement of multiple objectives for the coming victory, constant feedback, unpredictability,



curiosity – optimal information dosing,



cooperation – helping each other to achieve common goals,



competition – comparison of results with other users,



control – ability of choice and perception of consequences,



recognition – sense of satisfaction with the achievements,



imagination – identification with the role.

Learning based on computer games can have a positive impact on learning [11]. The content is interesting not only because students have grown up with computer games [12], but also because it is an alternative way of learning which can be more efficient than the classical one [13, 14]. Educational computer games have an action-based scenario. The most common question is how to adjust the game 'plot' when the virtual world needs to suit the real one. In certain scenarios students can work together to solve problems, while in others they can play against the

870

computer or compete with each other through competitive games in order to achieve better results [15]. Computer games can teach computer science students to develop and apply algorithms for solving problems, and to simulate, evaluate and detect logical errors [16]. Previously mentioned studies on using computer games for learning, although of a newer date, do not take into account the variety of mobile platforms, and are therefore restricted to a specific range of compatible devices. This means that the problem presented at the end of the previous chapter manifests itself here as well, and it can be largely solved in a similar manner, i.e. by the development of HTML5 compatible games run in an Internet browser. IV.

SYSTEMS FOR COMPUTER GAME E/M-LEARNING AND DATA MINING

Data mining forms the main component and an integral part of the process of knowledge discovery in large amounts of data [17]. Knowledge discovery in data comprises the following tasks: data cleansing (deviation and inconsistency detection), integration (in cases where data from multiple sources can be combined), data selection (only data relevant for the analysis), data transformation (consolidation in the appropriate form, and related operations), data mining (detection of patterns in the data), pattern evaluation (determination of patterns that represent the desired result, i.e. the knowledge that wants to be revealed), and knowledge presentation (using visualization and representation techniques). E/m-learning and computer game learning are based on the interaction between the student and the application, and thus generate a variety of data suitable for data mining. Although e-learning systems hold large amounts of data, they are primarily designed to support learning rather than the analysis of the stored data [18]. Moodle System (Modular Object Oriented Development Learning Environment), on which MudRi system of the University of Rijeka is based, is one of the most used open source systems for e-learning. Moodle version 2.5 includes over 200 tables in its database structure [19]. These tables store configuration settings, user profiles, courses, access and activity data, etc. Forums rely on eight tables. In Moodle, user authentication is performed by password. The problem with the data from e-learning systems is their interdependence which violates the assumption that majority of machine learning algorithms are based on. For example, forum answers can build on previous ones. Previous studies on data mining in elearning systems [20] include the following techniques: 

regression – predicting the time students will spend logged onto the system, or predicting to what extent students will be satisfied with the educational institution,



grouping – establishing models of students who work in similar conditions and recommending activities the student has not yet used which would help him or her in the acquisition of knowledge depending on the predisposition,

MIPRO 2014/CE



classification – predicting students’ final grades, according to their activity in the system,



association courses.

rules

–

recommending

elective

By taking into account the data produced by modern m-learning systems, the database can also record geolocation to find out how often the system is used with regard to the location. Furthermore, if geolocation is recorded periodically, at short intervals, it can be verified whether the system is actually used on the go. Such systems may include all of the data structures from elearning systems. By using the available APIs, social networks such as Facebook and Twitter can be integrated into the system. Therefore, current conventional communication through forum discussions can be partially or completely replaced. Data mining of texts can be used for analysing student activity on social networks and its effect on the results. Interaction model of the system for computer game e/m-learning and data mining is presented in Fig. 1. It is evident that the new m-learning systems tailored for smartphones and tablets are related to the classical elearning systems in a way that e-learning has become a subset of m-learning, while with the old mobile devices and associated systems this relation is reversed (Fig. 2). Accessing e-learning systems via mobile devices makes it possible to identify and track how users access content through certain platforms, and therefore makes it possible to offer appropriate design (a single column or two columns instead of multiple columns, etc.) or adapt to the screen resolution. In educational computer games places where most mistakes are made can be annotated by following the paths of information and interaction with the interface [21]. The start time, exit time before the end of the game (the student has not passed the whole content) and end time (the student has passed the whole content) can be recorded. Various data can be studied: how many times has the game been played, has the student who passed the whole game come back to it again, etc. Students

access the content through interface

E-learning

m-learning e-learning educational computer games

Figure 2. Display of data systems for learning with computer games compared to systems for e/m-learning

Students can be distracted in using the system when accessing it with mobile devices which have the incoming call feature (tablets do not usually fall into this category), SMS, MMS, e-mail, and other messages. Due to its complexity, data mining may be performed by system administrators or by teachers who create ecourses [22]. The results may help teachers (to find ways to improve the level of acquired knowledge of their students, and to improve the development of e-courses), students (if the system recognizes their level of knowledge and their behaviour, more time-efficient learning can be offered or learning can be supplemented with additional content sources and activities), and administrators (to achieve optimal server load and network resources parameters). Knowledge discovery in different data systems that form the basis of m-learning can be particularly challenging and complex in initial preparatory activities. The reason for this is the diversity of data which is mostly unstructured (they require special processing methods and techniques, including natural language processing which can be particularly challenging in cases of microblogging service where the number of characters is limited and messages frequently use combinations of abbreviations and sign language). A further problem lies in various purpose services (e.g. SMS, MMS, geolocation) which are adapted to aggregate data and integrate it into a single unit.

Educational computer games

M-learning

data system is the source of content which is adapted to discovered knowledge, saves user interaction

discovered knowledge is evaluated and stored in the data system

Discovered knowledge

Data mining File system: cleaning, integration, selection, transformation

access the content through interface

Teachers and administrators

access to data and report system directly or via a platform

Report system

conduct an in-depth analysis of the data system; discovered knowledge is stored and can be a source of new analysis

database and logs, social networks

data patterns are discovered and knowledge to adapt and improve the system is generated

Figure 1. Representation of the user interaction model with data mining of computer game assisted e/m-learning in higher education

MIPRO 2014/CE

871

V. PRELIMINARY RESEARCH The analysis presented in this paper includes student activity in forum discussions of the MudRi system and their final pass or fail grade. The simplest way to measure user activity relates to the measurement of the number of posts of a particular user. Mostly users write a post, which is mainly a query, and others give answers to the query. The number of posts can often lead to false conclusions. For example, a user can write only two posts, one may be a detailed answer to a complex problem, and the other only a thank-you note. Using the previously mentioned method, these posts would be rated equal, so a correction is applied which takes into account the average size of posts and the total number of characters in the post. This in turn can again lead to false conclusions because one may copy and quote the entire post as an additional source of information, while someone else may only add a link. Users who give answers to the questions are considered experts in a particular area, and those who ask questions are considered seekers of knowledge. The user type can be observed through the time of posting, so forums can be classified according to their prevailing activity time: morning, weekend, night, etc. This approach can be combined with the aforementioned post counts. This study analyses the data from two generations of students enrolled in the course Programming 1 at the undergraduate study of Informatics at the University of Rijeka. The research related to the generation 2010/11 includes 72 students, and the one related to the generation 2011/12 includes 76 students. Students who have posted at least once to a forum, i.e. who have been actively engaged in the discussion are further on referred as Forum members. The number of posts is 145 in 2010/11, and 86 in 2011/12. Forum members are those students who have written at least one post in one or both of the discussion forum categories. Namely, the observed forum discussions can be grouped by content, and described as follows: 



Forums with lectures and exercises handle notifications of problems and seeking help to solve problems, and issues related to the content of the script, homework and exams. Generation 2010/11 includes 79 posts, i.e. 54 % of the total, and generation 2011/12 includes 21 post, i.e. 24%. Forums with student tutorship mainly relate to the confirmation of arrival to the tutorship. They include 66 posts in 2010/11, i.e. 46% of the total, and 65 posts in 2011/12, i.e. 76%.

To calculate the correlation between students who actively post on forums and those who do not in regard to whether they pass the course, a statistical Chi-square test with one degree of freedom is employed (1) using the data from Table III.

2 

872



 f 0  f t 2 ft

(1)

TABLE III.

COURSE PASS/FAIL GRADE COUNTS OF FORUM MEMBERS AND NON-MEMBERS YES

NO

Passed course Forum members with lectures and exercise Forum members with student tutorship All Forum members Non-members

2010/11

2011/12

2010/11

2011/12

18

8

6

a1-2010/11 22 a2-2010/11 28 a3-2010/11 13 c2010/11

a1-2011/12 24 a2-2011/12 26 a3-2011/12 20 c2011/12

b1-2010/11 10 b2-2010/11 13 b3-2010/11 18 d2010/11

1 b1-2011/12 6 b2-2011/12 6 b3-2011/12 24 d2011/12

By replacing the cell values in Table III with letters a, b, c and d, we obtain: 2

N  N   ad  bc   2  2  . a  c  b  d  c  d  a  b

(2)

The null hypothesis H0 set in the Chi-square test assumes that there are no differences in passing the course between different categories of Forum members and non-members. The working hypothesis H1 assumes the dependence between these values and categories. The level of significance is set to P=0.05 with Yates's correction, due to the small number of posts in some categories, i.e. less than 10. According to the results shown in Table IV, the null hypothesis is not validated in five out of six monitored cases. To sum up, there is a statistically significant correlation between the course pass or fail grade and the activity in forum discussions. The working hypothesis is not validated only in the case of Forum members with student tutorship in generation 2010/11. The reason for this may lie in the purpose of the forum for student tutorship; it is not clear whether the students attend tutoring because they need supplemental tuition or advanced tuition. Generation 2010/11 has an additional motivating factor: they are invited to post the errors identified in the script, and to propose corrections. Thereby, the number of students actively participating in forum discussions is higher in that generation. Text mining in that group might identify and differentiate students who provide assistance and solutions from those who seek help. TABLE IV.

CHI-SQUARE TEST RESULTS OF FORUM MEMBERS AND NON-MEMBERS

Forum members’ categories Forum members with lectures and exercise Forum members with student tutorship All Forum members

χ2

P

2010/11

2011/12

2010/11

2011/12

4.744

4.048

0.0294

0.0442

3.564

7.456

0.0591

0.0063

3.985

8.494

0.0459

0.0036

MIPRO 2014/CE

VI. CONCLUSION AND FUTURE WORK The evolution of e-learning and m-learning systems is in progress since the Internet is increasingly accessed via smart mobile devices (phones and tablets). Simultaneously with the development of e/m-learning systems, various studies are conducted regarding how to increase the use of these systems, since distance learning reduces direct communication among participants in the educational cycle. Consequently, this situation is demotivating in most cases. Different methods are used to increase the motivation; one of the most present in today's field of research is the application of educational and serious computer games. With data mining in e/m-learning a personalized approach to each student can be achieved in a way that it can optimally adapt to both, students who possess sufficient knowledge as well as students who have difficulty in acquiring new material. This is done either by accelerating the path of learning or by offering additional learning sources. In addition to the impact on the quantity and quality of content, this enables new experience in the acquisition of knowledge, which is particularly interesting to new generations of students. Further research in this area might involve the following stages of development: creating the application and the associated data system in HTML5 ecosystem for easier use on smart mobile devices, processing certain topics (e.g. programming) through computer games and knowledge acquisition through levels, and integration with the existing data from the MudRi system. Data mining can provide us with a range of answers on the relation between students' activity in the system and their final grade. The efficiency of the optimized learning model can be verified by using the model for processing different generations of students, as well as different courses. Previous studies on data mining in e/mlearning systems mainly relate to a particular educational institution or a course. This raises the question of generalization. Hence, it would be interesting to conduct a study that would synergistically involve both, similar and different universities in Europe and worldwide.

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15] [16]

[17] [18]

ACKNOWLEDGMENT

[19]

This research has been supported under the Grant No. 13.13.1.3.03 of the University of Rijeka.

[20]

REFERENCES [1]

[2]

[3]

[4]

C. Romero and S. Ventura, “Educational data mining: A survey from 1995 to 2005,” Expert systems with applications, vol. 33, no. 1, pp. 135-146, July 2007. J. L. Moore, C. Dickson-Dean, and K. Galyen, “e-Learning, online learning, and distance learning environments: Are they the same?,” Internet and Higher Education, vol. 14, no. 2, pp. 129135, March 2011. M. Ally, “Foundations of educational theory for online learning,” in Theory and practice of online learning, 2nd ed., T. Anderson, Ed. Athabasca, Canada: AU Press, 2008, pp. 15-44. “E-learning: advantages and disadvantages,” Innovative learning technologies [Online]. June 13, 2011. Available: http://www.innovativelearningtechnologies.com/?p=80. [Accessed: January 27, 2014].

MIPRO 2014/CE

[21]

[22]

J. Cheon, S. Lee, S. M. Crooks, and J. Song, “An investigation of mobile learning readiness in higher education based on the theory of planned behavior,” Computers & Education, vol. 59, no. 3, pp. 1054-1064, November 2012. F. Martin and J. Ertzberger, “Here and now mobile learning: An experimental study on the use of mobile technology,” Computers & Education, vol. 68, pp. 76-85, October 2013. S. Tang, M. Henneghan, and A. E. Rhalibi, “Introduction to games-based learning,” in Games-based learning advancements for multi-sensory human computer interfaces: Techniques and effective practices, T. Connolly, M. Stansfield, and L. Boyle, Eds. Hershey, PA: IGI Global, 2009, pp. 1-17. M. Prensky, “The games generations: How learners have changed,” in Digital game-based learning, Minnesota: Paragon House, 2007, pp. 35-65. F. Fong-Ling, S. Rong-Chang, and Y. Sheng-Chin, “EGameFlow: A scale to measure learners' enjoyment of e-learning games,” Computers & Education, vol. 52, no. 1, pp. 101-112, January 2009. T. Hainey, T. Connolly, M. Stansfield, and E. Boyle, “The differences in motivations of online game players and offline game players: A combined analysis of three studies at higher education level,” Computers & Education, vol. 57, no. 4, pp. 2197-2211, December 2011. L. S. Sica, A. D. Veneri, and O. Miglino, “Exploring new technological tools for education: Some prototypes and their pragmatical classification,” in Methodologies, tools and new developments for e-learning, E. Pontes, Ed. Rijeka, Croatia: InTech, 2012, pp. 108-128. D. Oblinger, “The next generation of educational engagement,” Journal of interactive media in education, vol. 8, pp. 1-18, May 2004. I. Kovačević, M. Minović, M.. Milovanović, P. O. Pablos, and D. Starčević, “Motivational aspects of different learning contexts: My mom won't let me play this game...,” Computers in Human Behavior, vol. 29, no. 2, pp. 354-363, March 2013. N. J. Whitton, “An investigation into the potential of collaborative computer game-based learning in Higher Education,” PhD Thesis, Edinburgh Napier Univ., Edinburgh, Scotland, 2007. R. Lawrence, “Teaching data structures using competitive games,” IEEE Trans. Edu., vol. 47, pp. 459-466, November 2004. C. Kazimoglu, M. Kiernan, L. Bacon, and L. Mackinnon, “A serious game for developing computational thinking and learning introductory computer programming,” Procedia – Social and Behavioral Sciences, vol. 47, pp. 1991-1999, February 2012. J. Han and M. Kamber, Data mining: Concepts and techniques, 2nd ed. San Francisco, CA: Morgan Kaufmann Publishers, 2006. A. Kruger, A. Merceron, and B. Wolf, “A data model to ease analysis and mining of educational data,” in Educational Data Mining 2010, 3rd International Conference On Educational Data Mining. Pittsburgh, PA, 2010, pp. 131-140. “Database schema introduction,” Moodle [Online]. Available: http://docs.moodle.org/dev/Database_schema_introduction. [Accessed: January 31, 2014]. C. Romero and S. Ventura, “Educational data mining: A review of the state of the art,” IEEE Trans. Syst. Man Cybern. C, Appl. Rev. vol. 40, pp. 601-618, November 2010. C. S. Loh, “Information trails: In-process assessment of gamebased learning,” in Assessment in game-based learning, D. Ifenthaler, D. Eseryel, X. Ge, Eds. New York: Springer, 2012, pp. 123-144. C. Romero, S. Ventura and E. Garcia, “Data mining in course management systems: Moodle case study and tutorial,” Computers & Education, vol. 51, pp. 368-384, August 2008.

873