Design adaptive learning system using metacognitive

Design adaptive learning system using metacognitive strategy path for learning in classroom and intelligent tutoring systems Khafidurrohman Agustianto, Adhistya Erna Permanasari, Sri Suning Kusumawardani, and Indriana Hidayah Citation: AIP Conference Proceedings 1755, 070012 (2016); doi: 10.1063/1.4958507 View online: http://dx.doi.org/10.1063/1.4958507 View Table of Contents: http://scitation.aip.org/content/aip/proceeding/aipcp/1755?ver=pdfcov Published by the AIP Publishing Articles you may be interested in Development of Students’ Metacognitive Strategies In Science Learning Regarding Nuclear Energy AIP Conf. Proc. 1263, 29 (2010); 10.1063/1.3479885 On Development of an Adaptive Tutoring System for Calculus Learning AIP Conf. Proc. 1247, 276 (2010); 10.1063/1.3460237 OPUS One: An Intelligent Adaptive Learning Environment Using Artificial Intelligence Support AIP Conf. Proc. 1247, 215 (2010); 10.1063/1.3460231 DCBITS: Distributed Case Base Intelligent Tutoring System AIP Conf. Proc. 1007, 162 (2008); 10.1063/1.2937603 The Use of Multi‐Agent Systems to Build Intelligent Tutoring Systems AIP Conf. Proc. 627, 340 (2002); 10.1063/1.1503703

Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

Design Adaptive Learning System Using Metacognitive Strategy Path for Learning in Classroom and Intelligent Tutoring Systems Khafidurrohman Agustianto1, a), Adhistya Erna Permanasari1, b), Sri Suning Kusumawardani1, c), Indriana Hidayah1, d) 1

Department of Electrical Engineering and Information Technology, Gadjah Mada University, Jl. Grafika No.2 Kampus UGM Yogyakarta, Mlati, Sleman, Yogyakarta, Indonesia a)

Corresponding author: [email protected] b) [email protected] c) [email protected] d) [email protected]

Abstract. Adaptive learning (AL) is a learning process focusing on students’ personality through of recommendationbased learning and inquiry-based learning. This research intends to create a system capable of giving AL recommendation in the form of learning path (LP) in which LP will then be used as a recommendation for the teacher in constructing a learning process. In this research, LP was obtained through the identification of students’ metacognitive with the use of Metacognitive Awareness Index (MAI) respectively. The system itself was developed using Naïve Bayer Classifiers (NBC) and rule-based system to construct the LP. Accuracy of the machine learning for learner style module was of 97.25% and black box test indicated that the system was functionally acceptable. From these results, it can be concluded that the system was functionally acceptable and capable of representing an expert (seeing as it could produce an output that conformed to the expected condition). An expert in education had declared the LP was acceptable or in accordance to the design suggested by education experts. The developed adaptive learning path was proven to be better than the learning path in the preceding research.

INTRODUCTION Engineering education is one that any part must be developed in order to establish an independent country, especially in technology, it is imperative for educational institutions to construct a learning process with its correspondence to the students’ characteristics in mind because each student has different needs and characteristics such as prior knowledge, intellectual level, cognitive traits, and learning styles [1-2]. This type of learning process is known as personalized learning approach or adaptive learning (AL). Substantially, AL is a learning process focusing on students’ personality through of recommendation-based learning and inquiry-based learning [2]. This research intends to model the students by using Data Mining (DM) in a machine learning system. DM is a process of extracting hidden patterns, information [3], relationships, structures, and estimating values of the objects of the data [4], the results of DM can be used to support decision-making [3-5]. DM in educational case is called as Educational Data Mining (EDM). EDM is part of the Learning Technology (LT), LT is kinds of technology to support learning [6]. EDM emerged as a new paradigm to designing the models, tasks, methods, and algorithms to explore data from the world of education. The goal is to understanding the patterns and make predictions about the characteristics of learners, awards, domain knowledge, assessment, educational facilities, and the implementation of education [4,7-8]. EDM has a great potential, especially since education is one of the most important priorities in human life [4], which is referred to as a new paradigm for improving quality, efficiency, and

Advances of Science and Technology for Society AIP Conf. Proc. 1755, 070012-1–070012-6; doi: 10.1063/1.4958507 Published by AIP Publishing. 978-0-7354-1413-6/$30.00

070012-1 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

the achievement of the education system. This study uses the EDM to model the metacognitive of students to determine the Learning Path (LP) also called learning models. Metacognitive is a knowledge of the ability to think themselves, including awareness in controlling learning, correcting mistakes, analyze how effective the strategies used, replace the habit or strategy when required [9-11], and the cognitive ability of students. Learning model can be interpreted also as a pattern that is used to formulate the curriculum, organize the material, and provided the guidance to the teacher in the classroom [1]. So it is important to determine the model used by the teacher in the learning process. Knowledge of metacognitive types of students makes the teacher is able to evaluate certain types of learning models that should be used for specific learning materials, knowing the mechanism of a particular teaching method, assignment forms, forms of evaluation, and assessment. This system based on the concept of Blended Learning (BL), BL is an effective learning model today [12]. This model is a merging of the learning model by using merging several models, such as the merging of face to face learning model with internet-based learning model, or merging learning model supported a technology with models with different technologies, with the aim of creating the most efficient learning environments [12]. BL in the application applied by merging of online and traditional learning, technology and media to delivery the learning materials [13-15]. The study aims to develop an adaptive system using metacognitive by merging the physical classroom with e-learning using LP. In this research, LP was obtained through the identification of students’ metacognitive with the use of Metacognitive Awareness Index (MAI) respectively. The system itself was developed using Naïve Bayer Classifiers (NBC) and rule-based system to identify ILS and construct the LP.

RELATED WORK Student modeling to find the students' characteristics is one that any part of research on the students activity [2], such as Kardan et al. [16] that using the technique of algorithms two levels AL called ACO-Map, the first level of the algorithm is to see patterns of cognitive of the students, and the second level using the ant colony optimization to find LP based on Ausubel Meaningful Learning Theory, in order to obtain the output of concept folders for each group based on their needs. Jugo et al. [17] using the DM to make the AL, this study makes recommendation based on patterns derived from the domain of knowledge of students. Moga et al. [18] create a new design in making the domain module to create interaction between the individual and learning modules, that conducted the design based on Euclidean Distance Between a Learning Object and The Student's Ability (EDALO) and Reusable Learning Object Matrix (RLOM), while the domain of the modules itself an important part of Affective Tutoring System (ATS)/ITS. Seghroucheni et al. [19] create a system that can cope with learning path of the student who did not pass the exam, with calculating similarities between the natural activities of the students learn with what has been done. In this research, the system create LP, LP was obtained through the identification of students’ metacognitive with the use of Metacognitive Awareness Index (MAI) [20] respectively. The system itself was developed using Naïve Bayer Classifiers (NBC) and rule-based system to identify ILS and construct the LP. Identification of metacognitive obtained from the students was then used as variable in LP rule-based system which then produced an output in the form of guidance for a learning process.

DISCUSSION Object of previous studies on student modeling such as understanding the learner behavior [21], determines the environmental effectiveness of the education system [22] or measuring the success of instructional efforts evaluation of the structure of the learning content [23], effectiveness in the learning process and predict the performance of the students [24], review of adaptive feedback for teachers [25], and analyzing of possible interactions with students using machine learning techniques, the result is the extraction of interesting patterns of student behavior and used to assist teachers in assessing learning [2]. Research on modeling student not only see from cognitive aspects but has grown more complex [21][26]. Then this study using metacognitive aspects.

p A B

pB A u p A p B

(1) [27]

070012-2 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

V1(KM)

…

V8(KM)

TABLE 1. NBC Test Result V1(RM) … V8(RM) KM

RM

Tool

Expert

0.7 0.7 … 0.6 0.6 0.6

… … … … … …

0.0 1.0 … 0.0 1.0 1.0

0.9 0.9 … 0.9 0.9 0.9

0.3 0.1 … 0.3 0.0 0.1

RM RM … RM KM RM

RM RM … RM KM RM

… … … … … …

0.7 0.3 … 0.7 0.3 0.3

0.0 0.1 … 0.0 0.0 0.1

The research methodology is shown in Figure 2, the first stage of research is the literature review that used to determining the state of the state-of-the-art of research topic and composing research instruments. The next step is analyzing the data, the result of this process is a knowledge base. Data analysis in this research using KMean Clustering and validated by experts. The next step is to classifying data by using the Naïve Bayes Classifier (NBC) shown by Equation 1. NBC implementation conducted by the research begins with the transformation of the questionnaire’s data, by change the range of 0-50 became 1-5. The values then divided into two classes based on the type of class, then the data is stored for later comparison. In the implementation of NBC, system produces a value (‫)݀݋݋݄݈݁݇݅ ݔ ݎ݋݅ݎ݌‬, the value is used as a determinant of posterior value. In the implementation of the research to make changes in the formula NBC, by eliminating the value of evidence, it is based on the assumption that the value of the same events can be removed. For example, a value that has same denominator, the value of the denominator can be removed, so just compare the value of the numerator. rxy rxy

ri ri

^n¦ X

n¦ XY   X ¦ Y

`^

 ¦ X 2 n¦ Y 2  ¦ Y 2 74 u 51482  26 u 13550

2

`

74 u1107  26 74 u 2507074  1107 2

2 k ­° ¦ V i ½° 1  k  1 ®°¯ V t 2 ¾°¿

¦V i

74 ­ 45 ½ 1 74  1 ®¯ 350.799 ¾¿

Vt2

2

2

45 2

0.882

(2)

0.33015

¦ total 

¦ total 2

74

74

13550 2 2507074  74 74

(3) 350.799

The population used in this research was the 2014/2015 undergraduate students of DTETI Universitas Gadjah Mada (UGM) with 90 people taken as sample and 73 of respondents for data trainer. The next step is to validity and reliability test, the validity shown by equation 2, reliability shown by equation 3 using SPSS, and the data shown in Table 1. The next step is to determine the class of the data, this study using Simple KMean Clustering with supervision by education experts. The next step is to perform student modeling, this step using NBC to model student's metacognitive. The last step is determining the accuracy of system by using comparison with expert result (ground check). In this research, the system create LP, LP was obtained through the identification of students’ metacognitive with the use of Metacognitive Awareness Index (MAI) [20] respectively. The system itself was developed using Naïve Bayer Classifiers (NBC) and rule-based system to identify ILS and construct the LP. Identification of metacognitive obtained from the students was then used as variable in LP rule-based system which then produced an output in the form of guidance for a learning process. The result of processing expert form learning path, which is expected in accordance with the characteristics of the students. In forming LP, this study uses a rule-based system are shown in Table 2.

070012-3 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

TABLE 2. Decision Table of Learning Path Metacognitive Condition

Declarative Knowledge Procedural Knowledge

1 0

0 1

Declarative Knowledge Procedural Knowledge Conditional Knowledge Planning Information Management Monitoring Debugging Evaluation

1 1 1 0 0 0 0 0

0 0 0 1 1 1 1 1

Action

Expert System: Learning Path

Learning Path

Student

Lecture

-----------------

Metacognitve

FIGURE 1. Use Case

Literature Review

Questioner

Data Collection

Algorithm Test

Machine Learning Accuration Test

Metacognitive: MAI

Validity Test

Data Test

Metacognitive Strategy Test

Reliability Test

Cluster SKMeans: Weka

Accuracy and Validation Test

Implementation and Validation

Learning Path Learning Path Validation:Expert

Validaiton: Expert Analysis and Discussion KB Metacognitive

Conclusions and Suggestions

FIGURE 2. Research Method

070012-4 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

TABLE 3. Metacognitive Component [28] Description

Sub Variable

Indicator

Knowledge of metacognitive

Declarative knowledge

Knowledge of the skills, resources capabilities of a person as a learner.

Procedural knowledge

Knowledge of how to implement learning steps

3, 14, 27, 33

Conditional knowledge

Knowledge of why and when learning steps used

15, 18, 26, 29, 35

Planning

Plan learning process, setting goals, allocating resources for learning priorities

4, 6, 8, 22, 23, 42, 45

Information management

Skills and strategies to efficiently process information (such as organizing, elaboration, summarizing, focusing election)

9, 13, 30, 31, 37, 39, 41, 43, 47, 48

Monitoring

Assessment of learning or the use of strategies

Debugging

The use of strategies to improve/check for errors in the understanding and implementation/achievement.

1, 2, 11, 21, 28, 34, 49 25, 40, 44, 51, 52

Evaluation

Analyzing the achievements and the effectiveness of the strategy after learning process

Regulation of metacognitive

Items

and

5, 10, 12, 16, 17, 20, 32, 46

7, 19, 24, 36, 38, 50

IMPLEMENTATION Interactions between the two entities of the system shown in Figure 1, where the user entity interaction of the two processes, namely learning path which requires metacognitive characteristic of students. User interactions made by students, user should login to see the results of the identification. While teachers interact with, teachers requires to sign in for identification. This study built a system to determine the learning path. System is web-based system, that using the programming language PHP and MySQL database and implement methods Rule Base as a method of decision-making. The implementation result shown in Figure 3, it is the user interface of the learning path of the system. The LP describe the detail of the educational proses minute to minute, and also detail about what should teacher and student doing.

FIGURE 3. Learning Path

CONCLUSION In this research, LP was obtained through the identification of students’ metacognitive and learning styles with the use of Metacognitive Awareness Index (MAI) respectively. The system itself was developed using Naïve Bayer Classifiers (NBC) and rule-based system to construct the LP. NBC was chosen to model the students’ metacognitive because of its processing speed and high accuracy, as well as its efficiency and wide usage in predicting user preference for e-learning. To construct the LP, rule-based system was chosen because the data used in this research was deterministic data that was suitable for rule-based system, a system representing a set of rules to obtain solution/conclusion from various problems and situations. Identification of metacognitive obtained from the students

070012-5 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06

was then used as variable in LP rule-based system which then produced an output in the form of guidance for a learning process. The population used in this research was the 2014/2015 undergraduate students of DTETI Universitas Gadjah Mada (UGM) with 90 people taken as sample. Results pre-analyst data indicates that the data can be accept statistically, it is based on two tests were performed using SPSS 19 is test bivariate to see the validity of each item to the total score, and the reliability test using Cronbach's Alpha that showed 88.1%. The acceptance test was done by utilizing a black box. The number of respondents for said functionality test was 10. Meanwhile, the tests for conformation and advantage of the output were done by 3 education experts. Accuracy of the machine learning for learner style module was 97.25%, and black box test indicated that the system was functionally acceptable. From these results, it can be concluded that the system was functionally acceptable and capable of representing an expert (seeing as it could produce an output that conformed to the expected condition). An expert in education had declared the LP was acceptable or in accordance to the design suggested by education experts. The developed adaptive learning path was proven to be better than the learning path in the preceding research.

REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28.

Suprijono, Cooperative Learning Teori dan Aplikasi PAIKEM vol. 36 (Pustaka Pelajar, Yogyakarta, 2012). E. Gaudioso, M. Montero, and F. Hernandez-del-Olmo, Expert Syst. Appl. 39, 621–625 (2012). M. S. B. PhridviRaj and C. V. GuruRao, Procedia Technol.12, 255–263 (2014). A. Peña-Ayala, Expert Syst. Appl. 41, 1432–1462 (2014). J. Han, M. Kamber and J. Pei, Data Mining: Concepts and Techniques (Elsevier Inc., 2011). G.-Z. Liu, N.-W. Wu, and Y.-W. Chen, Res. Dev. Disabil. 34, 3618–28 (2013). R. B. Sachin and M. S. Vijay, “A Survey and Future Vision of Data Mining in Educational Field,” in 2012 Second Int. Conf. Adv. Comput. Commun. Technol. (IEEE, 2012), pp. 96–100. C. Romero and S. Ventura, Expert Syst. Appl. 33, 135–146 (2007). M. M. Spada, B. Langston, A. V. Nikčević, and G. B. Moneta, Comput. Human Behav. 24, 2325–2335 (2008). S. Sánchez-Alonso and Y. Vovides, Comput. Human Behav. 23, 2585–2595 (2007). G. Schraw and D. Moshman, Contemp. Educ. Psychol. 19, 460–475 (1994). T. Unit, Fundamentals of Blended Learning (University of Western Sydney, Sydney, 2013). K. Thorne, Blended Learning vol. 36 ( Kogan Page, London, U.K., 2003). F. Alonso, Br. J. Educ. Technol. 36, 217–235 (2005). J. Bersin, The Blended Learning Handbook vol. 36 (Wiley, New York, 2004). A. Kardan, “A Novel Adaptive Learning Path Method,” in The 4th International Conference on e-Learning and e-Teaching, ICELET 2013 (IEEE, 2013), pp. 20–25. I. Jugo, B. Kovačić, and V. Slavuj, “Using Data Mining for Learning Path Recommendation and Visualization in an Intelligent Tutoring System,” in 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (IEEE, 2014), pp. 26–30. H. Moga, F. Sandu, A. C. Stanca, and O. M. Machidon, “Computing of Educational Paths based on the Euclidean Distance Ability-to-Learning Object,” in 2013 11th Roedunet International Conference (RoEduNet) (IEEE, 2013), pp. 1–8. Y. Z. Seghroucheni and A. Mohammed, “Exploitation of the recommendation systems in the calculation of the learning path,” in 2014 5th International Conference on Information and Communication Systems (ICICS) (IEEE, 2014), pp. 1–6. L. G. Magaldi, Revista Canaria De Estudios Ingleses 61, 73-86 (2010). E. Kurilovas, I. Zilinskiene, and V. Dagiene, Comput. Human Behav. 30, 550–557 (2014). A. Guney and S. Al, Procedia - Soc. Behav. Sci. 46, 2334–2338 (2012). O. Akdemir and T. a. Koszalka, Comput. Educ. 50, 1451–1461 (2008). T. Adeyinka and S. Mutula, Comput. Human Behav. 26, 1795–1805 (2010). M. Thurlings, M. Vermeulen, T. Bastiaens, and S. Stijnen, Educ. Res. Rev. 9, 1–15 (2013). K. Chrysafiadi and M. Virvou, Expert Syst. Appl. 40, 4715–4729 (2013). Akella, Alan, A. Downey, A. Gorm, A. Lincoln, Arauzo, and Arichnad, Naive Bayes classifier. 2010. G. Schraw and D. Moshman, Educational Psychology Review 7, 351-371 (1995).

070012-6 Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 192.30.84.115 On: Mon, 07 Nov 2016 23:35:06