The Perception of Learning in Asynchronous Online

Available online at www.sciencedirect.com

Procedia - Social and Behavioral Sciences 83 (2013) 776 – 780

2nd World Conference on Educational Technology Researches – WCETR2012

The perception of learning in asynchronous online discussions: A scale development study Erdi Okan Yılmaz a *, Halil Yurdugül b a

b

Uşak University, Distance Learning Vocational College, Uşak 64200, Turkey Hacettepe University, Computer Education and Instructional Technologies, Ankara 06100, Turkey

Abstract In this research, the scale development study of the perception of learning in the discussion environments was done in order to present the perception of learning of the students who were in a learning process asynchronous online. In the scope of this scale development study, firstly, the perception of learning was identified, secondly the scale item pool was developed by scanning the existing literary field and the items in the item pool were re-edited by expert opinion, and finally the tool of candidate measuring was stated out. Afterwards, in the discussion environments developed by researchers, the applicability of the scale was attained as 7.96 and its reliability was attained as 0.93. Furthermore, a rubric-type scale for the aim of convergent validity checked the message quality of the messages in the discussion environments, and it was found out that there was a 0.52 type of relation between the message quality and the perception of learning levels. © 2013 Theand Authors. Published by Elsevier Ltd. Openofaccess BY-NC-ND Selection peer review under responsibility Prof.under Dr. CC Hafize Keser.license. Selection and/or peer-review under responsibility of Prof. Dr. Hafize Keser Ankara University, Turkey

Keywords: Online discussions, perception of learning, scale development.

1. Introduction In the face-to-face learning environment, the students’ achievement is connected with learning sources (such as learning tasks, instructional materials, etc.) which are controlled by teachers in classrooms. Because of this, in educational studies the students’ achievement indicators are determined by formal and/or informal assessments. The indicators (such as achievement measures) deal with dependent variable in statistical models. However, in online learning environment, unfortunately the control of learning sources by educators is difficult. Hereby, in online learning the self-assessment scales are used instead of achievement tests to determine levels of students’ learning. Alternatively, the perception of learning is used instead of success conception, and self-assessment scales are used instead of achievement test. The perception of learning, presented here, is not approached within the context of summative assessment, but within the context of formative assessment. Online discussion environment is a web based virtual environment aiming to provide the interaction of the students with the content, teachers and classmates (Brower, 2003). Online discussions are a part of learning

* Corresponding author. Erdi Okan Yılmaz Tel.: +90-276-221-2148 (3370) E-mail address: [email protected]

1877-0428 © 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license. Selection and/or peer-review under responsibility of Prof. Dr. Hafize Keser Ankara University, Turkey doi:10.1016/j.sbspro.2013.06.146

Erdi Okan Yılmaz and Halil Yurdugül / Procedia - Social and Behavioral Sciences 83 (2013) 776 – 780

777

environments to promote students’ collaboration and learning in the educational process. These environments are based on students’ interactions and generally, it can be a component of learning management systems (LMS). According to Lonn & Teasley (2009), learning management systems are web-based systems providing material sharing between teacher and students, exchanging feedback and communicating online. West, Waddoups & Graham (2007) cite that besides their intended uses such as organizing, managing and getting course materials, the learning management systems have a vital function to provide the interaction between student and teacher. It is important to evaluate students’ learning in online discussions environments. Centra & Gaubatz (2000) presented in their research that, in evaluating the success in online learning environments, although formal assessment system is accepted as the prime quality valid measurement, it does not reflect learning outcomes accurately. Wu & Hiltz (2004) states out that, it is not clear how the discussion environments affect students' success. According to Wu & Hiltz, the measuring of students' success in discussion environments can be generated by self-assessment tools as informal methods instead of formal methods. By the way, it is difficult to evaluate students’ knowledge and ability in online discussion environments. Especially, it is difficult to administer the formal assessment methods and performance assessments methods in online discussion environments. Collecting, transferring and learning the knowledge in online discussion environments is important (Stodel et al., 2006). In evaluating students, it is important not only to evaluate what they learn but also to evaluate the quality of discussion environment outputs. In online discussion environments, student success should not be merely focused on. In these environments, also through a quality evaluation, the factors such as experience and interaction of the students and their perception of learning should be focused on (Peterson et al., 2001). In order to display the success of online discussion, it is necessary to analyze the perception obtained (Ryan & Scott, 2008). On learning perception in informal learning environments, especially online discussion environments, there are a few researches (Levenberg & Caspi, 2010). On the contrary, of informal learning, in formal learning the evaluation of student success is quite easy. In formal learning achievement tests are enough to evaluate student success, whereas in informal learning different evaluating methods are used such as learning perception focusing on the experimental aspects of learning (Levenberg & Caspi, 2010). This study aims to investigate students’ perceptions of learning in order to identify their learning performance in online discussion environments. Lately, student achievement in distant education researches has been measured increasingly by students’ perceptions of learning as self-assessment method. By the widespread use of e-learning environments in higher education, the researches focus on students’ experiences and learning outcomes through their experiences (Frederico, 2000; Liaw, 2002; Song et al. 2004; Ellis et al. 2008; Lee & Tsai, 2011). Informal learning is a process resumed by the learner. Therefore, there is not a structured teaching program and it does not end with a total evaluation. In informal learning, there is no other way to evaluate the learner except learning perception process (Levenberg & Caspi, 2010). In order to display the potential success of learning management systems, learner and teacher perceptions should be conceived (McGill & Klobas, 2009). Certainly, self-assessment scores of the students’ perception of learning are not as reliable as academic achievement test scores. On the other hand, students’ perception of learning scores both can provide a clue about students’ learning experience and can be used for redesigning the learning environments. This study tries to develop a scale for identifying students’ perception of learning in online discussion environment through analyzing their interactions. 1.1. Researches about Students’ Perception of Learning Ellis & et al. (2008), in literature, there are a few researches studying on learning experiences in online discussions. Fich & Hiltz (1999) stated out in their research that, combining group work with asynchronous learning networks improve students’ perception of learning. Hanson & Robson (2004) found out that the learning management systems develop learning perceptions positively. Wu & Hiltz (2004) researched if the online discussions improve the students’ perceived learning or not. In their study applied to 116 students, they stated out that online discussion environments improve students’ perception of learning. Ellis & et al. (2008) in their research on student experiences

778


in discussion environments found out that they managed better learning in online discussion environments. Lonn & Teasley (2009) in their research on student perception related to the use of learning management system found out that the use of communication technologies improve learning perception. Lee & Tsai (2011) compared the learning perception in online learning environments to formal face-to-face learning environments. They found out the perceptions in online learning environments are higher. At the end of their literature scan on the best ways of online discussion applying methods, Roehm & Bonnel (2009) found out that the role of teachers and educational institution are important and providing useful experiences by creating interaction is vital. 2. Methods Appropriate to the steps of scale development studies; a) The perception of learning was identified, b) The scale item pool was developed by scanning the existing literary field, c) The items in the item pool were re-edited by expert opinion, d) The tool of candidate measuring was stated out. Afterwards, 90 students were enrolled in the discussion environment developed by researchers which is a part of learning management system (LMS) and throughout one semester they interacted with the content of LMS, besides they discussed the topics in online discussion environment. At the end of the semester the students’ perception of learning in online discussion environment was measured by the scale developed. The reliability and validity of the obtained findings were investigated. Firstly, the factorial validity of the scale was analyzed. Furthermore, for the aim of convergent validity, students’ messages that were posted to the discussion environment were evaluated by a rubric-type scale and they were called quality of contribution, and the correlation between the quality of contribution and the perception of learning was investigated. For the interrater reliability of the quality of contribution scale, the incoming messages were graded according to two expert rubrics. Cohen’s kappa statistic was determined as 0.84 between the data sets obtained from two experts. 3. Findings and Results The statistical correlation between the perception of learning and the item points obtained from the scale is called a measuring model. For the estimation of this statistical correlation, firstly, principal component analysis (PCA) was applied on the data set. According to results of PCA, the first eigenvalue of data set was determined as 7.96. In addition to this, explained total variance was 72.30%. Results of PCA indicated that the data set was unidimensional. Because a process of scale development is also a volitional process based on hypothesized relation, we were preferred the confirmatory factor analysis and structural equating models instead of exploratory data analysis such as PCA. The perception of learning was handled with the quality of students’ contribution in discussion forums as the proof of convergent validity. Quality of contribution has seven indicators: a) know validity, b) cognitive effort, c) meaning, d) sample-references, e) clarity, f) collaboration and g) redirect. In Figure 1, it was given the estimated factor loadings of items in the scale. All of the factor loadings had higher and statistically significant values. The data-model fit indices were at the satisfaction levels (GFI=0.94; CFI=0.94; RMSEA=0.03). Obtained findings showed that 11 items in the scale evaluated the perception of learning in discussion forums unidimensionally. Also, in Figure 1, findings related to convergent validity were presented. Between the level of students’ perception of learning and the quality of students’ contribution in discussion forums, a correlation at 0.52 levels was determined. This correlation value was at medium level. This value was expected on a higher level as the students with higher perception of learning were supposed to contribute in discussion forums more. But, ineffective contribution of some students with higher perception of learning might have caused this. Therefore, obtained correlation value can be taken as the proof of convergent validity. In additional to these findings, internal consistency reliability of this scale was measured by Cronbach's Alpha Coefficient as 0.93. This value is quite high.


779

4. Conclusions The perception of learning is a construct to be considered while determining about learner or identifying the effectiveness of learning environment. Discussion forums are also learning and interacting environments. In this research, the psychometric properties of the scale developed for the assessment of the students’ perception of learning in discussion forums were investigated. According to the obtained results, this scale has the quality to identify the perception of learning in discussion forums embedded in specially the LMS. References Brower, H. H. (2003). On emulating classroom discussion ia a distance delivered OBHR course: Creating an on-line learning community. Academy of Management Learning and Education, 2(1), 22-36. Centra, J. A. & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? Student Instructional Report –SIR II. Ellis, R. A., Goodyear, P., Calvo, R. A., Prosser, M. (2008). Engineering students’ conceptions of and approaches to learning through discussions in face-to-face and online contexts. Learning an Instruction, 18, 267-282. Fich, R. B., Hiltz, S. R. (1999). Impacts of asynchronous learning networks on individual and group problem solving: A field experiment. Group Decision and Negotiation, 8, 409-426. Frederico, P. A. (2000). Learning styles and student attitude toward various aspects of network-based instruction. Computers in Human Behavior. 16, 359-379. Hanson, P., Robson, R. (2004). Evaluating course management technology: A pilot study. Boulder, CO: Educause Center for Applied Research, 24. Lee, S. –Y, W., Tsai, C. C. (2011). Students’ perceptions of collaboration, self-regulated learning, and information seeking in the context of Internet-based learning and traditional learning. Computers in Human Behavior, 27, 905-914. Levenberg, A., Caspi, A. (2010). Compairing perceived formal and informal learning in face-to-face versus online environments. Interdisciplinary Journal of e-Learning and Learning Objects. Volume 6. Liaw, S. –S. (2002). An Internet survey for perceptions of computers and the World Wide Web: Relationship, prediction, and difference. Computers in Human Behavior. 18, 17-35. Lonn, S., & Teasley, S. D. (2009). Saving time or innovating practice: Investigating perceptions and uses of Learning Management Systems. Computers & Education, 53, 686-694. McGill, T. J., & Klobas, J. E. (2009). A task-technology fit view of learning management system impact. Computers and Education, 52(2), 496508. Peterson, J. Z., Hennig, L. M., Dow, K. H., & Sole, M. L. (2001). Designing and facilitating class discussions in an Internet class. Nurse Educator, 26(1), 28-32. Roehm, S., Bonnel, W. (2009). Engaging students for learning with online discussions. Teaching and Learning in Nursing, 4, 6-9. Ryan, J. & Scott, A. (2008). Integrating technology into teacher education: How online discussion can be used yo develop informed and critical literacy teachers. Teaching and Teacher Education. 24, 1635-1644. Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. (2004). Improving online learning: Student perceptions of useful and challenging characteristics. Internet and Higher Education, 7, 59-70. Stodel, E. J., Thompson, T. L., & MacDonald, C. J. (2006). Learners’ perspectives on what is missing from online learning: Interpretations through the community of inquiry framework. International Rewiev of Research in Open and Distance Learning, 7(3). West, R. E., Waddoups, G., & Graham, C. R. (2007). Understanding the experiences of instructors as they adopt a course management system. Educational Technology Research and Development, 55(1), 1-26. Wu, D. & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Network. 8(2), 139152.

780


Appendix

Through online discussion forums … …my critically thinking skills on subjects expanded. …I understood the relations among concepts related to subjects. …my social sharing about subjects increased. …I learned the knowledge related to content better. …my communication skills expanded. …I realised what I learned. …my learning increased. Online discussion forums … …helped me to learn this subject. …afforded me a new point of view on the subject. …helped me to associate with the subjects on discussion matter. In online discussion forums … …learning was a pleasure.

Strongly Agree

Agree

Neutral

Disagree

Strongly Disagree

Perception of Learning in Asynchronous Online Discussions Environments (Scale)