Interactive Technology for Enhancing Distributed Learning: A Study on ...

Interactive Technology for Enhancing Distributed Learning: A Study on Weblogs Andreas Holzinger

Michael D. Kickmeier-Rust

Martin Ebner

Research Unit HCI4MED Medical University Graz Auenbruggerplatz 2/V A-8036 Graz, Austria +43-316-385-3883

Department of Psychology University of Graz Universitaetsplatz 2/III A-8010 Graz, Austria +43-316-873-9554

WG Social Learning Graz University of Technology Steyrergasse 30 A-8010 Graz, Austria +43-316-873-8540

a.holzinger@ hci4all.at

michael.kickmeier@ uni-graz.at

martin.ebner@ tugraz.at

ABSTRACT In this study, it was investigated whether, and to what extent, Web 2.0 technologies, actually Weblogs, can be a suitable instrument for enhancing the practice of distributed learning. In educational settings, which are based on traditional lectures many students begin serious study shortly before the exam. However, from previous empirical research, it is known that the practice of distributed learning is much more conducive to retaining knowledge than that of massed learning. A 22 factorial design (within – repeated measures) with pre-test and post-test in a real life setting was applied; the study lasted for the whole summer term 2007. Participants were N=28 computer science undergraduates of Graz University of Technology. We randomly assigned them to two groups of equal size: The experimental group given the Weblog treatment are referred to as Group W; whereas the control group with no access are referred to as Group C. Students of group W were instructed to use the Weblog for developing their paper and studying during the lecture and they were requested not to reveal their group affiliation. The results showed that performance scores of group W were significantly higher than that of group C. This demonstrates that Weblogs can be an appropriate instrument to supplement a classical lecture in order to enable deeper processing of information over a longer period of time, consequently resulting in enhanced learning performance.

Categories and Subject Descriptors K3.1. [Computer Use in Education]: Collaborative learning, Computer-managed instruction (CMI)

General Terms Experimentation, Human Factors, Theory

Keywords Web 2.0., Weblogs, Distributed Learning, Massed Learning, Learning Performance.

1. INTRODUCTION AND MOTIVATION Research on new media in educational settings is an extremely broad area: emerging technologies enter education quickly and often find application well beyond their original intention [3], [9], [12]. There is a increase in educational technologies and the influence of these new technologies is relatively high, however, we must never forget that learning is both a basic cognitive and social process, and that education cannot be replaced by technology [12]. Previous research in new media provided inconsistent and sometimes even contradictory results on effects of different types of media and their learning performance [10], [11]. For example, in recent years, Web 2.0 technologies and social software have increasingly influenced the educational practice in schools and universities [8], [13]. Especially Weblogs can be used to support education by encouraging reflective practice [2], [3] and enabling student-centred approaches [15]. However, empirical research is only sparsely focussed on the measurable success of such Weblogs [7] and research in reallife settings is missing. Consequently, our central question within the present work is whether Weblogs can contribute towards addressing, and possibly overcoming the well-known and common problem of student learning strategies, that is, massed (or crammed) learning. In typical educational settings, where a longer period of teaching is completed with some kind of test or exam (e.g., traditional classroom lectures), we frequently observed that students start serious learning rather shortly before the exam; this may be a week, maybe days, or even hours. During this final period students learn literally day and night and, amazingly, they can acquire a substantial amount of information, which at least suffices to pass the exam. Such learning strategies are, from both a psycho-pedagogical and educational perspective, problematic. A large body of research yielded that the efficiency and, above all, retention over a longer period of time is significantly impaired by such learning behaviour [17], [18], [5], [16].

2. PSYCHOLOGICAL BACKGROUND © The Author 2009. Published by the British Computer Society

Learning and practising strategies, preferences, or styles can be defined and characterized in a variety of ways, taxonomies, and classifications. Probably one of the most simple, yet important classifications refers to the temporal intensity of learning, that is, the classification in massed versus distributed learning (or practising). For example, based on a thorough analysis of the literature, Lee and Genovese [14] defined those concepts in terms of pauses between learning and practising trials.

HCI 2009 – People and Computers XXIII – Celebrating people and technology

309

A. Holzinger et al.

Consequently, massed learning refers to a behaviour wherein the learners attempt to acquire as much knowledge or as many skills as possible within a limited period of time and literally without any intermittent pauses. In turn, the concept of distributed learning refers to the learning strategy of allocating learning trials over a larger time interval, including prolonged breaks and rests. Essentially, results of previous research indicate that distributed learning is superior to massed learning (see e.g., [14], [4]).

Basically, after the first successful logon, a Weblog is provided to each single user where data entries can be made. In addition to text entries, any multimedia content can be published online. The platform allows the creation of communities for a specific topic. Students and teachers attending such a community (so called members) are allowed to blog within the community blog and exchange their opinions (co-operative blogging).

Baddeley [1], for example, conducted a study, teaching a large number of postmen to type. The results indicate that one hour learning sessions on a daily basis reduced the number of necessary training hours and enabled the participants to increase typing performance more rapidly than massed learning sessions. A more recent study by Childers and Tomasello [6] with a different user group summarizes the gist of the matter. However, the amount of content that can be learned in a massed way is amazing and when comparing the learning performance of massed learning and distributed learning right after learning sessions, it is probable that the performance will be more or less identical; however, when comparing retention after a longer period of time, distributed learning results in substantially better performance, most of all in a deeper understanding. Of course, the effects of massed versus distributed learning are moderated by diverse further variables, for example the type of learning (cognitive versus motor skills), the complexity of the learning material, or meaningfulness of the content.

We carried out a longitudinal study over a complete semester in the real-life setting of a typical lecture: “Applied Human– Computer Interaction” (TUG LV 706.046), held by the first author. The study started on 26th February 2007 and ended on 25th June 2007.

3. METHODS AND MATERIALS 3.1 The Weblog used Since 2006, Graz University of Technology (approximately ten thousand students) is using a social networking environment [8] based on the open source product ELGG (http://elgg.org). A socalled Blog Sphere was installed and adapted according the requirements of the university. Students and teachers are able to establish digital identities and connect to each other, collaborate with them and discover new resources through their connections. Figure 1 shows the community blog of the lecture. The layout can be recognised by blog users as fairly standard, with a typical student contribution on the left side, while search possibilities and links to the weblog and bookmarks are on the right side – together with a list of the members. The original development of the ELGG was initiated in 2004 by Ben Werdmuller and Dave Tosh. It has a strongly educational background and fulfills most requirements of universities, schools, and scientific institutes.

3.2 Experimental Setting

During recent years, it was interesting to observe that students did not do most of their work continually during the lectures, as recommended by previous learning research, but – as could be observed in other lectures as well – at the very end, shortly before the deadline; where they then worked night and day. This was the motivation for checking the hypothesis: when we provide a weblog for one group, and require them to contribute to this weblog continually during the whole semester, and tell them that this is of equal worth as the traditional paper writing, this will encourage continuous learning.

3.3 Participants A total of 28 students participated: 4 females and 24 males. The average age was 24 years (SD = 1.33), the youngest participant was 22, the oldest 27. All participants were undergraduate computer science students and therefore familiar with Web 2.0 technology in general, and Weblogs in particular.

3.4 Procedure The students were randomly assigned to one of two equally sized groups, an experimental group (group W), who were required to use their personal Weblog related to the lecture and the subject matter during the semester. In the control group (group C), students were not affected by the study and could perform regularly. The students of group W were instructed not to communicate the Weblog usage to group C. Internal logging data showed that all students used the Weblog to some degree. At the beginning of the semester all students were provided with the general learning and study strategy inventory (LASSI, [19]), which is a questionnaire with ten sub-scales that assesses specific learning and study strategies, predominantly focussing on meta-skills. In addition, a basic learning styles inventory in the German language (HALB test) was presented at the beginning of the semester. To assess the student’s prior knowledge, a knowledge test scale was conducted, including ten dichotomous items, eight multiple-choice items, and three Likert scales. To assess learning performance during the semester, a written exam was conducted at the end of the semester. The exam required the students to write and present a paper about a specific topic. The exam consisted of 20 correct/incorrect judgements, 20 multiple-choice questions, five free-answer items, and three problem solving tasks. The written paper made 70% of the grading, the writing exam 30%.

4. RESULTS AND DISCUSSION Figure 1. TU Graz Learnland – Community Weblog

The main focus of this paper is whether learning performance increases with a more distributed learning strategy by using (or being required to use) Weblogs in comparison to regular strategies. In order to ensure a clean distribution, the control group were monitored in the classroom to avoid any social

HCI 2009 – People and Computers XXIII – Celebrating people and technology

310

Interactive Technology for Enhancing Distributed Learning: A Study on Weblogs

networking and requested to limit their learning activities to the tools provided. For the statistical analysis, we rely on two knowledge tests, which were of course identical in the pre-test and the post-test; first a ten item dichotomy (yes/no) judgment test (T1) and second, an eight item multiple choice knowledge test (T2). Each of the multiple choice items had four alternatives, where at each item none, one, or more of the alternatives could be correct, to decrease the probability of guessing. As expected, no differences in the test scores were found in the pre-tests at the beginning of the semester. In group W the mean score was 2.31 (SD = 1.11) for test T1 and 0.92 (SD = 1.44) for test T2. Similarly, in group C the mean score was 2.77 (SD = 1.01) for test T1 and 1.15 (SD = 1.95) for test T2 (Figure 2). A normal distribution test (Kolmogorow-Smirnov) proved the normal distribution of the data. Subsequently, an analysis of variance (ANOVA) revealed non-significant differences between both groups for T1 (F(1, 24) = 1.227, p = .279) and T2 (F(1, 24) = .118, p = .735). At the end of the semester, a clear divergence of results between the experimental group and the control group could be seen. In the Weblog group W the mean score was 14.31 (SD = 2.14) for T1 and 15.23 (SD = 4.15) for T2. In the control group C the mean score was 12.33 (SD = 2.67) for T1 and 12.08 (SD = 4.19) for T2 (Figure 2).

Figure 3. Learning performance for both groups (W, C) and both tests (T1, T2) In addition, we asked the students for their opinion about the quality of existing learning material in the field of the lecture (i.e., applied human-computer interaction). The average quality rating on a percentage scale (where 100% means best quality) was 73.92 (SD = 13.89), which is a quite good judgment. Moreover, we asked the students whether they thought that multimedia material is more beneficial for learning than simple textbook lessons. The average rating, again on a percentage scale, was 58.92 (SD = 18.80), which is a medium value, yet quite alarming for educational designers. Overall, the judgments in these questions did not influence the participants’ learning performance; we found no correlations (Pearson correlation) with learning performance in T1 (-0.16) and T2 (0.28).

5. CONCLUSION The presented work supported our hypothesis (section 3.2.) in that the use of Weblogs does facilitate and stimulate a more distributed learning behaviour. This result provides an indication that Weblogs can be an appropriate instrument to improve learning performance by supplementing traditional lecturing. Figure 2. Test scores for both groups (W, C) and both tests (T1, T2) at the beginning and the end of the semester. For further analyses, we used the learning performance, which was computed as the difference between post-test and pre-test scores. In group W the mean learning performance was 12.00 (SD = 2.16) for T1 and 14.31 (SD = 3.71) for T2. In group C the mean learning performance was 9.38 (SD = 2.87) for T1 and 10.46 (SD = 2.83) for T2. For both blocks of items, group W yielded a higher learning performance (Figure 3). An ANOVA revealed on the 5%-level significant differences between the experimental group and the control group for both tests (T1: F(1, 24) = 6.881, p = .015; T2: F(1, 24) = 5.917, p = .023).

The students who were required to use Weblogs during the semester did perform significantly better than the students who relied on their usual learning strategy. However, at the moment it is unclear whether this result is due to effects of a more distributed learning or whether publishing and discussing in Weblogs facilitates a deeper learning.

6. FUTURE OUTLOOK In future work, we will extend the analyses, particularly with respect to the effects of learning styles and learning strategies. A promising and important focus will be the students’ behaviour and the effects of regular pauses, social networking and the distribution of the learning over longer periods. In addition, future analysis will investigate specific effects on long term retention.

HCI 2009 – People and Computers XXIII – Celebrating people and technology

311

A. Holzinger et al.

7. REFERENCES [1] Baddeley, A. D., Essentials of human memory, Psychology Press, East Sussex, 1999. [2] Beale, R., Blogs, reflective practice and student-centered learning, http://www.bcs.org/upload/pdf/ewic_hc07_sppaper1.pdf, last visited: 2009-05-07 [3] Beale, R., Mobile blogging: experiences of technologically inspired design, CHI '06 extended abstracts on Human factors in computing systems, ACM, Montreal (CA), 2006, 225-230 [4] Bloom, K. C. and Shuell, T. J., Effects of massed and distributed practice on the learning and retention of second-language vocabulary, Journal of Educational Research, 74, 4, (1981), 245-248. [5] Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T. and Rohrer, D., Distributed practice in verbal recall tasks: A review and quantitative synthesis, Psychological Bulletin, 132, 3, (2006), 354-380. [6] Childers, J. B. and Tomasello, M., Two-year-olds learn novel nouns, verbs, and conventional actions from massed or distributed exposures. ; 38(6), 2002., Developmental Psychology, 38, 6, (2002), 967-978. [7] Du, H. S. and Wagner, C., Weblog success: Exploring the role of technology, International Journal of HumanComputer Studies, 64, 9, (2006), 789-798. [8] Ebner, M., Holzinger, A. and Maurer, H., Web 2.0 Technology: Future Interfaces for Technology Enhanced Learning?, In Stephanidis, C., ed., Universal Access to Applications and Services. Lecture Notes in Computer Science (LNCS 4556), Springer, Heidelberg, Berlin, New York, 2007, 559-568. [9] Ebner, M., Kickmeier-Rust, M. and Holzinger, A., Utilizing Wiki-Systems in higher education classes: a chance for universal access?, Springer Universal Access in the Information Society, 7, 4, (2008), 199-207. [10] Holzinger, A., Kickmeier-Rust, M. and Albert, D., Dynamic Media in Computer Science Education; Content

Complexity and Learning Performance: Is Less More?, Educational Technology & Society, 11, 1, (2008), 279290. [11] Holzinger, A., Kickmeier-Rust, M. D., Wassertheurer, S. and Hessinger, M., Learning performance with interactive simulations in medical education: Lessons learned from results of learning complex physiological models with the HAEMOdynamics SIMulator, Computers & Education, 52, 2, (2009), 292-301. [12] Holzinger, A., Nischelwitzer, A. K. and Kickmeier-Rust, M. D., Pervasive E-Education supports Life Long Learning: Some Examples of X-Media Learning Objects, http://www.wccee2006.org/papers/445.pdf, last visited: 2009-04-18 [13] Klamma, R., Chatti, M. A., Duval, E., Hummel, H., Hvannberg, E. T., Kravcik, M., Law, E., Naeve, A. and Scott, P., Social software for life-long learning, Educational Technology & Society, 10, 3, (2007), 72-83. [14] Lee, T. D. and Genovese, E. D., Distribution of practice in motor skill acquisition: learning and performance effects reconsidered, Research Quarterly for Exercise and Sport, 60, 1, (1988), 59-65. [15] Motschnig-Pitrik, R. and Holzinger, A., Student-Centered Teaching Meets New Media: Concept and Case Study, IEEE Journal of Educational Technology & Society, 5, 4, (2002), 160-172. [16] Ruch, T. C., Factors influencing the relative economy of massed and distributed practice in learning, Psychological Review, 35, 1, (1928), 19-45. [17] Underwood, B. J., 10 Years of Massed Practice on Distributed Practice, Psychological Review, 68, 4, (1961), 229-247. [18] Underwood, B. J. and Ekstrand, B. R., Effect of distributed practice on paired-associate learning, Journal of Experimental Psychology, 73, (1967), 1-21. [19] Weinstein, C. E., Palmer, D. R. and Schulte, A. C., Learning and Study Strategies Inventory (LASSI), H&H Publishing, Clearwater (FL), 1987.

HCI 2009 – People and Computers XXIII – Celebrating people and technology

312