Available online at www.sciencedirect.com
ScienceDirect Procedia - Social and Behavioral Sciences 116 (2014) 1594 – 1598
5th World Conference on Educational Sciences - WCES 2013
The Development of an O-NET Score Forecasting System Sageemas Na Wichian a *, Suwimon Wongwanich b, Patharawut Saengsiri c a
College of Industrial Technology, King Mongkut’s University of Technology North Bangkok, Bangkok 10800, Thailand b Faculty of Education, Chulalongkorn University, Bangkok 10330, Thailand c Thailand Institute of Scientific and Technological Research, Pathum Thani 12120, Thailand
Abstract This research proposes the design and development of a monitoring system which is a computer program that forecasts the ONET scores (Ordinary National Education Test) of grade 6 primary school students. The evaluation of the system's ability to forecast O-NET score is satisfactory. As for the result forecast of the grouped incremental learning, the system compares the test data with existing data, using Gaussian calculation to convert distances into membership ranks that are used as grouping criteria. The functional requirement test and the usability test reveal fair efficiency of the system. © 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and/or peer-review under responsibility of Academic World Education and Research Center.
Keywords: forecasting system, O-NET;
1. Introduction Ordinary National Educational Test (O-NET) is a standard test evaluating the academic performance of affiliated schools, so that they are up to the same national standard. The knowledge and ideas of students at Grades 6, 9 and 12 are tested (National Institute of Educational Testing Service, 2010). O-NET scores of students are considered along with their applications for higher education. O-NET is also one of the criteria for assessing the quality of learners outside the basic educational system (Pornprasert, 1998). Despite the importance of O-NET, students and related parties did not know which scores or levels are appropriate for students to further their education in different educational institutions. Appropriate scores are usually predicted from the scores of students from previous tests. Accurate score prediction is necessary. The prediction of scores in different fields is in the form of alerts or monitoring systems. Data is collected and activities are followed up. Then data is processed by computer programs. At present, alerts are extended and improved so less labor is required. However, there are drawbacks to these techniques since each has different features. The implementation of alerts or monitoring systems in education is new. Schools and students are able to see trends of average scores of schools in the future. This is useful for schools and students wanting to prepare themselves for higher education, in improving the curriculum and for correcting teaching errors. This research aims to develop a system of clustering schools for monitoring purposes. An *
Corresponding author name: Sageemas Tel.: +6-686-050-0508 E-mail address:
[email protected]
1877-0428 © 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
Selection and/or peer-review under responsibility of Academic World Education and Research Center. doi:10.1016/j.sbspro.2014.01.440
Sageemas Na Wichian et al. / Procedia - Social and Behavioral Sciences 116 (2014) 1594 – 1598
1595
incremental learning algorithm based on Mahalanobis distance (ILM) was integrated with a genetic algorithm working on the neural network concept and never-forgetting learning. The Mahalanobis distance technique and the Gaussian radial basis function are employed to design membership values (Komkhao, 2007; Meesad, and Yen, 2001; Sageemas, Suwimon, and Patharawut, 2011). Students, teachers, parents and schools in Grade 6 were able to realize O-NET score trends in the future. 2. Literature Review Incremental learning facilitates the creation of an unprecedented learning algorithm and storage of data types learnt in the past. Therefore, incremental learning creates an active algorithm that is efficient and resource-effective. This does not mean data is duplicated in memory. However, at least some part of the data was duplicated in the last node. Therefore, incremental learning leads to natural learning and saves time and resources in its processing. Examples of incremental learning clustering are COBWEB clustering and incremental learning, based on the Mahalanobis distance (ILM). The ILM algorithm is based on incremental learning clustering. It can work with both clustering and classification. Developed by Komkhao in 2007, this algorithm is based on the neural network concept and supports learning both with and without teachers. The ILM employs the incremental learning concept without forgetting preexisting knowledge, and utilizes Mahalanobis distance and Gaussian radial basis function to design membership values (Komkhao, 2007). Nevertheless, the efficiency of the ILM algorithm depends on how data is distributed. Important parameters are the covariance matrixes and distance rate. An ILM algorithm is composed of: 1) a learning phase with a learning algorithm and an incremental learning algorithm; 2) a predicting phase with a prediction mode algorithm. 3. Design and System Development Methods 3.1. Data Preparation 1. Data of the Ordinary National Educational Test (O-NET) in this research was acquired from the National Institute of Educational Testing Service (Public Organization). 2. Data used to create a school clustering model for monitoring purposes was data from the Ordinary National Educational Test at the second phase of Grade 6 students in Bangkok during 2007-2008. The data was divided into three groups, A, B and C, according to the clustering criteria of the National Institute of Educational Testing Service (Public Organization). Group A data: the top 15 O-NET scores. Group B data: 16 – 689 O-NET scores. Group C data: the last 15 O-NET scores. The 2007 data was prepared as data for a training set and the 2008 data was prepared as data for a testing set. 3.2. Determining Parameters Suitable for GA ILM Algorithm
1596
Sageemas Na Wichian et al. / Procedia - Social and Behavioral Sciences 116 (2014) 1594 – 1598
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
Figure 1 Working Process of GAILM Determine 30 chromosomes and the size of 20 genes. Randomly identify primary populations including covariance matrix and distance. Create a model using the ILM algorithm and the training set. Test the model using the training set. Use the genetic algorithm. Select new populations. Select the top five populations from populations in Item 5. Use the roulette strategy to select 25 genes. Perform crisscrossing or strain crossing by using the random technique and changing one position. Mix two genes using the random technique. Send values acquired from the genetic algorithm to the ILM model. Calculate strength from precision rates. Place precision values of each round in the matrix table, ranging from high to low precision. Then select the parameter acquired from the highest precision value to develop a clustering program for monitoring purposes.
3.3. System Design and Development The scope of system development was designed, based on relevant data. The system to be designed includes users and system data, which are Home, Prediction, Policy, Q&A and Contact Us. Users input scores to acquire the predictions in web browsers. 3.4. Design of tools to estimate efficiency of the system 1.
Methods and processes of creating an evaluation form a) Study the evaluation form and then select, adjust, add and correct it, so that it is consistent with the developed system. b) Determine a sample group in the evaluation that is related to the O-NET such as 30 students, teachers, schools and parents. 2. Estimation criteria or standards The researchers created an evaluation form including questions with multiple choices and a five-level rating scale divided into two aspects: functionality testing and usability testing. 3. Recognition of the efficiency of the system Average scores of the tested group were considered. In order for school clustering to be considered effective, the average scores must be higher. Criteria to determine the efficiency of the system comprised a fivelevel rating scale and a five-level quantitative scale. 4. Results Implementation of the school clustering system aims to estimate risks from average O-NET scores of students or schools at the second phase (Grade 6). The user interface design was in the form of a web application as seen in Figure 2.
Sageemas Na Wichian et al. / Procedia - Social and Behavioral Sciences 116 (2014) 1594 – 1598
1597
Figure 2 System Access Page When the user selects the Prediction page, they must fill in data of the predicted average scores of Thai, Mathematics and Sciences of the schools. After pressing Prediction, risk levels will be calculated on a meter. The meter indicator shows risk levels. The higher the level, the riskier it is. A round graph also shows the acquired scores as seen in Figure 3.
Figure 3 Risk Prediction Screen
1598
Sageemas Na Wichian et al. / Procedia - Social and Behavioral Sciences 116 (2014) 1594 – 1598
Assessing the quality of the system was based on evaluating the values of system users such as teachers and students in the functionality and usability tests. Percentage (%), means () and standard deviation (S.D.) Evaluating results showed that the developed system’s functionality and usability was at a high level ( Forty-five percent of students stated that the data they received could be used to further their education and 39% stated that they needed to test the precision of the real and predicted scores. O-NET data changed when subjects were added and clustering criteria was readjusted. A dynamic system should be developed to be consistent with changing O-NET results and integrated into a system that can analyze various kinds of data and predict scores of different schools in the Association of Southeast Asian Nations (ASEAN), in preparation for future free trade area projects. Acknowledgements This research is part of the "Research and Development of Driving Strategies for the Second Decade of Education Reform Policy” Project. All authors would like to thank not only the Thailand Research Fund (TRF) for funding this research (RTA5380011), but also the College of Industrial Technology of KMUTNB for partial financial support, and The National Institute of Educational Testing Service (NIETS) for dataset.
References Komkhao, M. (2007). An Incremental Learning Algorithm Based-on Mahalanobis Distance. Master Thesis of Graduate College, King Mongkut’s Institute of Technology, Thailand. Pomprasert, D. (1998). Comparison of National Educational Test and Provincial Educational Test Scores of Mathematics Grade 6 Students in the Academic Year of. Master's Degree Program in Education, Measuring and Evaluating Educational Outcomes, Ramkhamhaeng University, Thailand. Meesad, P. and Yen, G. Gary. (2001). An Effective Neuro-Fuzzy Paradigm for Machinery Condition Health Monitoring. IEEE Transactions on systems, man, and cybernetics – part B: Cybernatics. 31(4), (August 2001): 1567-1572. National Institute of Educational Testing Service (Public Organization). (2010). Guidelines for Grades 6 and 9 Students Taking O-NET Tests, Bangkok: Paboonma. Sageemas, N.W., Suwimon, W. and Patharawut, S. (2011). Clustering Model for Surveillance System using ILM Algorithm. ICIC Express Letters, An International Journal of Research and Surveys. 5(12), (December 2011): 4409-4414.
