UbiLearn2010
Impacts of adaptive reflection prompt on learners’ reflection levels in a u-learning environment
Nian-Shing Chen Department of Information Management, National Sun Yat-sen University, Taiwan
[email protected]
Gwo-Jen Hwang Department of Information and Learning Technology, National University of Tainan
[email protected]
Sheng-Wen Hsieh Department of Management Information System, Far East University, Taiwan
[email protected]
Yu-Rei Chang Department of Information Management, National Sun Yat-sen University, Taiwan
[email protected]
Abstract: In recent years, a lot of educational scholars have put their research focus on ubiquitous learning (u-learning) environments operated through mobile devices and wireless networks. Many studies have found u-learning to be effective in increasing learning efficiency, and as a result, varied learning activities are continuously being developed for u-learning scenarios. The process of reflection has been found to be important for increasing learners’ learning performance. However, learners find it difficult to reflect in u-learning environments where the majority of learning tasks take place in outdoor learning environments. Therefore, this study tries to connect outdoor learning with u-learning environments and the use of reflection prompts to increase learners’ reflection process. The purpose of this study is to explore whether reflection prompts with adaptive learning styles have potential to increase learners’ reflection. 39 elementary school students participated in the experiment. Four learning scenarios were developed according to their learning styles (reflective or active) and reflection prompts (group discussion or personal prompts). The participants were randomly assigned to the two learning scenarios (adaptive and non-adaptive), and divided into four groups. The results show that the reflection levels of the adaptive learners are higher than those of the non-adaptive learners. Hence, the connection of u-learning environments with reflection prompts is found to be helpful for learners. Keywords: u-learning, reflection, learning style
1. Introduction
UbiLearn2010 In recent years, the convenience and effectiveness of the mobile devices have attracted educational scholars to broadly apply them in teaching. Through the mobile devices and the wireless Internet technology, learners can be assisted and facilitated to learn in the real world environment, the paradigm commonly known as ubiquitous learning or u-learning (Hwang, Kuo, Yin, & Chuang, 2010). U-learning provides a highly portable medium for learning and enables learners to interact with other learners through mobile devices. Previous studies have suggested that u-learning has the potential to increase learning efficiency (Chu, Hwang, Huang, & Wu, 2008; Hwang, Yang, Tsai, & Yang, 2009 ). Moreover, scholars have argued that reflection plays an important role in the learning process; many studies have found that reflection can effectively increase learning performance (Mcnamara, 2004, 2006). Instructors typically ask questions and provide feedback to facilitate the development of learners’ reflection ability (Chi et al., 1994). The current higher education trend also emphasizes on improving learners’ reflection ability in order to prepare them to be adaptable to the changing environment (Chen, Wei, Wu, & Uden, 2008). Hence, how to improve learners’ reflection ability is an important issue to be discovered. In the outdoor learning environment, where majority of u-learning tasks take place, learners learn through interacting with the environment. In traditional outdoor learning situations, it use to be very difficult, if not impossible, for the instructors to be aware of each learner’s learning status. Instructors used to spend consideration amount of time to gather learners and prompt them to reflect in outdoor learning activities. Besides, learners rarely reflect in the gathering process. U-learning can provide an excellent mechanism for the instructors to build a u-learning based outdoor learning environment through mobile device and wireless internet, and to design reflection prompt mechanism to direct learners’ thinking in the learning process, fostering a real time and efficient interaction process as a result. Instructors in such u-learning environments have possibility to record learners’ learning status and provide real time feedback. Studies have concluded that if instructors’ teaching styles fit learners’ learning styles, there is a direct positive effect on learners’ learning attitude and performance (Felder & Silverman, 1988). With the advance of information technology, this conclusion conforms to the strategies of adaptive learning (Brusilovsky & Maybury, 2002). Adaptive learning approaches enable teaching contents to be customized according to learners’ learning styles. In u-learning environments, instructors can provide adaptive learning by building learners’ learning profiles through mobile devices and delivering personalized learning contents and activities according to each learner’s learning style and learning status. The focus of this study is therefore on the exploration of the influence adaptive reflection questions could have on reflection levels of learners in u-learning environments.
2. Methodology 2.1 Research Variables Since reflection prompt can improve students’ reflective ability (Chi et al., 1994), this study increase learners’ reflection level through reflection prompt. Moreover, Felder and Silverman (1988) suggested that if the teaching activities can fit learners’ learning styles, it will increase their learning performance. Therefore, this study designs adaptive reflection prompt to help learners reflect. In figure 1, the learning styles include active and
UbiLearn2010 reflective, and the reflective prompts include group discussion and personal prompt. If the reflective prompt can fit learners’ learning styles, it will increase their reflection levels. On the other hand, if the reflective prompt cannot fit learners’ learning styles, it will not increase their reflection levels. Learning styles and reflective prompts are used as the two independent variables in this study. Reflective prompts are used to raise learners’ reflection level in the u-learning environment. According to the processing learning styles, this study makes the two mechanisms of reflective prompts, group discussion and personal prompt. Based on learning style theory, the reflective prompt of group discussion is suitable for learners with active learning style since they prefer to study through experiment and group discussion (Felder &Silverman, 1988). In addition, the reflective prompt mechanism of personal prompt is suitable for reflective learners who prefer to learn through thinking. Hence, this study assumes that reflective prompt mechanism of group discussion and personal prompt can each help the active and reflective learners have higher learning performance.
Figure 1. research framework There is only one dependent variable: reflection level. According to the reflection level table of Bain et al. (1999), the researchers classify learners’ reflection levels by their reflection contents to understand their reflective levels. 2.2. Experiment Design The grouping is shown in Table 2. Group A are learners of active learning style with reflection prompt mechanism of group discussion. Group D are learners of reflective learning style with reflection prompt mechanism of personal prompt. These two groups are adaptive learning groups. On the other hand, group B and C are non-adaptive learning groups. Table 1. Experiment Group Design
Learning style
Active learning style Reflective learning style
Reflection prompt Group discussion Personal prompt A B C D
This study focuses on u-learning environment and the outdoor curriculum of primary education. Therefore, this paper conducts an experiment at Cheng Kung elementary school, which is located in Tainan County..The experiment procedure includes the pretest experiment, the learning activities in u-learning environment and the posttest experiment. At the first stage, the fifth grade learners take the pretest which includes the measurement of learning styles and pre-requisite knowledge, and the pretest of reflective levels. The questionnaire of learning style measurement is based on Felder and Soloman’s ILS scale (1997) and translated into Chinese. The test of pre-requisite knowledge is to measure participants’ related basic knowledge of butterfly ecology.
UbiLearn2010 The pretest of reflective levels includes having the participants read the teaching material of butterflies and answer the reflection questions, and identifying participants’ reflective levels according to their reflection contents. In addition, since there is a need of PDA mobile device in u-learning environment, the participants receive PDA using instruction. At the second stage, the participants use mobile device to study butterfly ecology in the butterfly theme park. At this stage, the four learning lists of butterfly life-stages, egg, larva, pupa and adult, are presented on PDAs. In the butterfly theme park, the participants check the characteristics of the life-stages on PDA touch panels. Since it is hard to observe the eggs, the egg characteristics are provided on PDAs. Then, participants receive the reflection prompt mechanism with group discussion or personal prompt. In the reflection prompt mechanism with group discussion, the participants are grouped and provided with reflection questions to do group discussion. In the reflection prompt mechanism with personal prompt, the system gives every participant a reflection prompt and a hint every two minutes to help the participant think. Finally, the third stage is the posttest experiment which includes the posttest of reflective levels and the measurement of learning performance. The posttest of reflective levels is to provide the reflection prompts for the participants to fill in their reflection contents, and then estimate their reflective levels according to their reflection contents.
3. Research Results There should be 40 participants in this experiment, but one of them could not be present at that experiment day. Hence, there are 39 participants in the experiment. The students are fifth grade elementary students in two different classes Since the participants are the fifth grade learners at Cheng Kung elementary school, and the experiment is done at the normal course, the experiment schedule has to be cooperated with their curriculum. The reflection prompt mechanism is that one class proceeds reflection prompts with group discussion, and the other one proceeds reflection prompts with personal prompt. It is important to examine if the two classes have the same pre-requisite knowledge of butterflies, thus the participants fill out the pretest questionnaire with butterfly pre-requisite knowledge. This questionnaire is designed by the experienced instructors of Cheng Kung elementary school. Because the experiment sample is small, nonparametric test is used to examine if there is significant difference of butterfly pre-requisite knowledge between the two classes. T-test is used to analyze the butterfly pre-requisite knowledge of the two classes. The p-value of the data is 0.463 > 0.05. Therefore, it can be concluded that there is no significant difference between the butterfly pre-requisite knowledge of the two classes. This study refers to the reflective level estimation standards of Bain et al. (1999) to analyze the reflection contents of the participants and to understand that after different reflection prompts mechanism, what the difference is among the reflection contents of learners with varied learning styles. In addition, the researchers discuss and grade the reflection contents of the participants with the trained professionals. Then, Kappa test is used to examine the pretest and posttest reflection contents in the experiment. The Kappa value in pretest and posttest are 0.919 and 0.893. The Kappa value of this study is higher than 0.8 which shows good consistence. The
UbiLearn2010 final score is negotiated by both the researchers and the professionals. This study uses independent t-test to check if there is significant difference between the reflective level increase of the adaptive and the non-adaptive groups. Hypothesis 1: In the u-learning environment, the reflective level of the learners with adaptive reflection prompt mechanism is significantly higher that of the non-adaptive learners. First, through the Levene’s Test (p-value=0.998), it is found that the adaptive and non-adaptive groups are homogenous and the variances of the two groups are not significantly different. Since the experiment sample of every group is small, the data is tested by Kolmogorov-Smirnov Z. The p-value of the data is 0.117 which is higher than 0.05, thus the study data can be examined by t-test. Then, the increased reflective levels of the adaptive and non-adaptive group are examined by t-test. It can be seen from Table 2 that the reflective level increase of adaptive group is significantly higher than that of the non-adaptive group. Therefore, hypothesis 1 is supported. Table 2. Kappa test results Group sample size mean std. t-value df. p-value Adaptive 19 1.89 1.050 -2.882 37 0.007 Non-adaptive 20 0.95 0.994 Hypothesis 2: In the u-learning environment, the reflective level of the active learners provided with adaptive reflection prompt mechanism is higher than these provided with non-adaptive reflection prompt mechanism. The data is analyzed by Mann-Whitney U test. It can be seen that the p-value of the data is 0.092 which is higher than 0.05. Therefore, there is no significant difference between the reflective levels of the active learners provided with adaptive and non-adaptive reflection prompt mechanisms. Although the reflective level of the adaptive group is higher than the non-adaptive group, the result is not significant. The reason leads to non-significant result may be as the following. Theoretically, learners with active learning style prefer learning by group discussion. However, since the teaching environment in Taiwan has been spoon-fed education for a long period of time, active learners become shy of or afraid of expressing their opinions and not enthusiastic to participate the learning activities of group discussion. Therefore, instructors can not only teach the students and have them acquire the knowledge, but also need to design teaching activities of group discussion for students to gain knowledge from their peers and get the experience of group discussion. Hypothesis 3: In the u-learning environment, the reflective level of the reflective learners provided with adaptive reflection prompt mechanism is significantly higher than those provided with non-adaptive reflection prompt mechanisms. The data is analyzed by Mann-Whitney U test. The p-value of the data is 0.044 which is smaller than 0.05. Therefore, providing reflective learners with adaptive reflection prompt mechanism can increase higher reflective level than providing them with non-adaptive reflection prompt mechanism. If instructors can know that the students are learners with reflective learning style in advance and provide them with personal hints in the teaching prompt, it will be useful to help learners reflect. The study result shows that most of the participants provided with reflection prompt mechanism increase more than one reflection level. In the research, the system automatically provides learners with reflection prompts mechanism and helps them reflect. Instructors can spend more time to design the reflection prompt mechanism, make learners think the learned knowledge in class, and provide them with reflection prompt, thus learners can increase their learning performance through the reflection learning activities.
UbiLearn2010
4. Conclusion This study designs an adaptive reflection prompt mechanism to help learners practice reflection in the outdoor u-learning environment. The mechanism is made according to the learning styles of the learners (active or reflective). The system provides active learners the reflection prompt mechanism of group discuss and provides reflective learners the reflection prompt mechanism of personal prompt to train their reflection and increase their learning effects.。 This research found that the increase of reflective level of adaptive group is significantly higher than that of the non-adaptive group. Therefore, it can be said that the adaptive reflection prompt mechanism can really help learners with different learning styles to reflect. This result provides instructors more standards to refer to. Instructors can examine learners’ learning styles in advance and provide them adaptive reflection prompt strategies (group discussion or personal hint) to help learners have higher learning performance. This paper is a pilot study; hence the sample number is not enough. The future study of adaptive learning in the u-learning environment may increase the sample in the experiment. The past studies pointed out that there are many factors affect learning, such as IQ, sex and personal characteristics; with the development of mobile technology and wireless Internet, the future study of adaptive reflection prompt mechanism should also concern these factors to make the adaptive learning more complete and to implement it in real learning environment.
Acknowledgements This work was supported in part by the National Science Council (NSC), Taiwan, ROC, under Grants NSC 98-2631-S-024-001
Reference Bain, J. D., Ballantyne, R., Packer, J., & Mills, C. (1999). Using journal writing to enhance student teachers’ reflectivity during field experience placements. Teachers and Teaching, 5(1), 51-73. Brusilovsky, P., & Maybury, M. T. (2002). From adaptive hypermedia to the adaptive web. Communications of the ACM, 45(5), 30-33. Chen, N. S., Wei, C. W., Wu, K. T., & Uden, L. (2008). Effects of high level prompts and peer assessment on online learners' reflection levels. Computers & Education, 52(2), 283-291. Chi, M. T. H., de Leeuw, N., Chiu, M., & Lavancher, C. (1994). Eliciting self- explanations improves understanding. Cognitive Science, 18(3), 439-477. Chu, H. C., Hwang, G. J., Huang, S. H., & Wu, T. T. (2008). A knowledge engineering approach to developing E-libraries for mobile learning. The Electronic Library, 26(3), 303–317. Felder and Silverman, 1988. Learning and teaching styles in engineering education. Engineering Education, 78(7), 674-681. Felder, R. M., & Soloman, B. A. (1997). Index of learning styles questionnaire. Retrieved February 20, 2010, from http://www.engr.ncsu.edu/learningstyles/ilsweb.html. Harvey, L., & Knight, P. (1996). Transforming higher education. Buckingham, UK: Open University Press.
UbiLearn2010 Hwang, G. J., Yang, T. C., Tsai, C. C., & Yang, J. H. S. (2009). A context-aware ubiquitous learning environment for conducting complex experimental procedures. Computers &Education, 53(2), 402–413. Hwang, G. J., Kuo, F. R., Yin, P. Y., & Chuang, K. H. (2010). A Heuristic Algorithm for planning personalized learning paths for context-aware ubiquitous learning. Computers & Education, 54(2), 404-415. McNamara, D. S. (2004). SERT: Self-explanation reading training. Discourse Processes, 38(1), 1-30. McNamara, D. S., O’Reilly, T. P., Best, R. M., & Ozuru, Y. (2006). Improving adolescent student’ reading comprehension with iSTART. Journal of Educational Computing Research, 34(2), 147-171.
