Usage of mobile self-assessment to support a continuous ... - ThinkMind

eL&mL 2011 : The Third International Conference on Mobile, Hybrid, and On-line Learning

Usage of mobile self-assessment to support a continuous learning process Thomas Sch¨uler, Karsten Morisse Dept. of Engineering and Computer Science University of Applied Sciences Osnabr¨uck, Germany Email: [t.schueler, k.morisse]@hs-osnabrueck.de

Abstract—In this article we describe the usage of mobile self-assessments to support a blended learning context in higher education. The students have the option to attend selfassessments and use all other provided learning media on their mobile devices. We describe the didactical setting in which the self-assessments are embedded, the development of a mobile application for using the assessments and the implementation and evaluation of it in the current term at our university. Further, we emphasize the benefit of developing special applications to support mobile learning. Keywords-Mobile Assessment; Blended Learning; Continuous Learning;

I. I NTRODUCTION The usage of e-assessments is motivated by several facts. E-assessments may be evaluated automatically and therefore provide immediate feedback for the learner. Once prepared, they are easily replicated and used in following courses without extra work for the lecturer. E-assessments are a good way to structure and evaluate the students learning behavior. The Bologna process in Germany has lead to a higher amount of examinations for the students. As a result, the number of examinations and assessments that have to be accomplished within a university increases. Usage of electronic systems may be a way to reduce the amount of time needed to organize and evaluate those. In higher education e-assessments and examinations can be used for different purposes and in different contexts. Within this article, we focus our view on the usage as a tool for self-assessment for the students. Within our didactical setting the self-assessments are offered every week in our learning management system. They cover the topics that have been discussed in the preceding week and are only accessible for a short time-period. However, participating in the self-assessments is seen as an optional service for the students, i.e. they don’t have to participate in them to be admitted to the final examination. Providing self-assessments in a didactical setting can help to work against a well known danger of e-learning in general: If the teaching material is offered in an electronic way, the students are ensnared to download and gather everything, but work with it just right before the final examination. Learning continuously during the semester involves a very

Copyright (c) IARIA, 2011.

ISBN: 978-1-61208-120-5

stringent time management, which many students overburdens. These circumstances lead to the procrastination effect: the learning process is postponed, because the teaching material is always available later and more pressing tasks are preferred. [8] sees in electronic self-assessments a very useful and helpful tool for knowledge control by the students themselves. If they are offered continuously and in short time-frames during the semester, they can help activating a continuous learning behavior. In the last few years, electronic devices are becoming more mobile and this may be a crucial factor to support the adoption of e-learning by students. One of the main benefits of e-learning is often said to be the independence of time and location. This only fully applies, when the devices that are used to access e-learning content and tools are portable and convenient to use at any place. While the hardware of mobile devices is getting more powerful, it is now timely to deliver applications for specific e-learning tools. To make our learning scenario truly mobile, a special development was needed to enable working with the self-assessments on a portable device. The remainder of this paper is organized as follows: Section 2 of this paper presents our evaluated blended learning approach which uses self-assessments as a regular service and in which video lectures substitute the real lecture. Section 3 presents the development of an application for mobile self-assessment on Apple iPhone and iPod touch devices. In section 4 we describe the implementation and evaluation of this tool in didactical settings in higher education contexts. The paper closes with a conclusion and some final remarks. II. B LENDED LEARNING BASED ON LECTURE RECORDINGS

We are using a blended learning approach based on video lectures for several years. The development of the concept and an evaluation of it are published in a series of papers ([2], [3], [5], [4], [6]). Within this section, we summarize the didactical modules of that concept. It is used in the lecture Audio- and Videotechnology in a computer science study program. The course provides insight in the technical basics of the media types audio and video in the context of computer science. In addition to the theoretical part there is a practical part in which the students have to do some

46

eL&mL 2011 : The Third International Conference on Mobile, Hybrid, and On-line Learning

practical work with audio and video, e.g. the production of a short movie. Over the years we have been using different technologies for the distribution of lecture recordings, namely liveand on-demand video streaming, synchronized on-demand presentation and podcasting [2]. In the beginning, lecture recordings have been provided supplementary to the classical classroom lecture. Since 2007 the podcast has completely substituted the classroom lectures. At present, the didactical structure of the concept combines podcast lectures with livecoaching, electronic self-assessment and practical sessions to support the students learning process (see [3]). The podcast episodes are published in the Apple iTunes Music Store and are publicly available for everybody. Therefore the students can access them easily and by means of mobile devices use them anytime and anywhere. Additionally, the lecture slides have been annotated in high detail and are available as PDFdocuments corresponding to each of the podcast episodes. Both are linked together by integrating the URL of the slide in the corresponding podcast episode. The modules of the didactical concept are explained in the following (see also figure 1). Results of a thorough evaluation can be found in [5]. A. Podcast There are more than 40 podcast episodes for this course publicly available in Apple’s iTunes Music Store. The content of these podcasts had been recorded during preceding semesters. However, each recorded lecture underwent an intensive postproduction process (chapter marks, dividing into small episodes, integration of animations). They are available anytime and, using portable podcast-player, also anywhere. However, to give some structure troughout the course, a time-schedule on a weekly basis is available. When using this technology, the learning-process can be self-organized to a high degree. Each episode of the podcast offers a fine-granulated access to the content by offering chapter-marks and can be used platform-independent. Each episode includes a reference to a PDF-document with a more detailed explanation of the content. B. Live Coaching Live-coaching is a weekly meeting of the students with the lecturer. Instead of pure content presentation sessions, these meetings are organized by means of coaching to support the students’ learning process. Certain aspects of the content are discussed, misunderstandings are clarified and practical issues are introduced. Herein, the students have the possibility to ask content-specific questions to the corresponding episodes of the podcast. If there are too many questions, the content is presented in more detail by the lecturer. Within the coaching sessions links to the practical part of the lecture are also given.

Copyright (c) IARIA, 2011.

ISBN: 978-1-61208-120-5

C. Electronic self-assessment To support a continuous learning process of the students, an electronic online assessment system called VIPS is used (see [1]). Within this system there are about 10 to 20 questions about the content of each podcast episode or logical block of episodes. For each week of the semester there is one set of questions corresponding to the specific content of that week. Each set is only available for a short time frame (e.g. two days). These electronic assessments are an essential part of the didactical concept and are organized by means of self-assessment. The students are free to use the assessments to evaluate their learning process. This way the assessments are an important motivation to work with the recordings during the semester. It is up to the students’ choice, if they process the assessments every week or if they just work on a few of them. However, a bonus system urges the students to work continuously with the assessments. If they reach more than 50 per cent of the points overall, they can collect bonus points for the final written examination. D. Practical Work Weekly lab hours accompany the theoretical part of the lecture. Within this part, the students have to work on a media project, e.g., production of a short movie or the production of a podcast. This active learning or learningby-doing approach strengthens the theoretical facts of the lecture. Usually, this work is done in small groups of up to three or four students and the result of their work has to be presented at three milestones (storyboard, rough cut as a first version, final version of the movie). These lab hours take half of the overall workload and are used to gain practical experience in video and audio production. E. Final Examination To collect credit points for the module, the final examination, which is usually a written examination, must be passed successfully. III. M OBILE S ELF -A SSESSMENTS In this section we describe the idea and development of an application to provide self-assessments on mobile devices. For now, this application is only available on iPhone and iPod touch and works with the mentioned VIPS online assessment system at our university. With this application the online assessments (see chapter II-C) can be attended anywhere. A. E-learning on mobile devices Above we described a blended learning concept, which we use already since 2007. In the last few years, electronic devices were becoming more mobile and, especially with the introduction of iPhone and iPod touch, more powerful and versatile. Many of the benefits of e-learning are only truly visible, when provided on mobile devices. The option to

47

eL&mL 2011 : The Third International Conference on Mobile, Hybrid, and On-line Learning

Figure 1.

Modules of the blended learning concept

attend classes via video does not mean a true independence of time and space, when one is bound to a stationary computer. It is also often not that easy, to use a laptop in a crowded bus or even likely, to carry it around all day in search for a sunny spot on the campus chill-out lawn. And in the odd case that it’s really happening, energy is low after 30 minutes video. Clearly, e-learning is still often limited by technology and infrastructure. Therefore, after experimenting with modern mobile devices, we decided to pay some attention to the possibility of using our learning concept on these. For the beginning we decided to use Apples’s hardware. The lecture recordings and the annotated slides already use standardized data formats, which are provided by most player hardware. Podcasting was introduced with Apple’s mobile devices and was adapted by many other manufacturers. The PDF-format is common for electronic documents. Both can be accessed on iPhone and iPod touch as well as on other mobile devices. The VIPS assessment system is designed as a plugin for the Stud.IP Learning Management System [10] and is not available on mobile devices as a native application. Our goal was to develop an application for this missing part of our concept. B. assessMe - a mobile VIPS application The development of the assessMe iPhone and iPod touch application was started in late 2009. Recently we released a first version, which is limited to be used within our university’s structures. This version will not be available in the Apple iTunes App-Store, which will be considered for upcoming versions. The application integrates with the Stud.IP VIPS assess-

Copyright (c) IARIA, 2011.

ISBN: 978-1-61208-120-5

ment system, so that the same data is provided for mobile as well as for stationary users. It may be synchronized to the server, allowing to switch between devices. assessMe only displays the data and handles to solve the assessment questions (see figure 2 and figure 3). The users’ responses are then transferred to the server on a sync command and are processed further only there. This way complicated algorithms are used only where enough computing power is available and granted. Moreover usage of storage space on the mobile device is low and it is easier to stay in sync. These are general guidelines for the development of mobile applications. Unfortunately today it also means that the feedback to the assessments is only accessible on the web after the automatic correction was initiated by the lecturer. There is no option to sync this feedback to the mobile application. However, this may also be implemented in the future. The interface design was kept simple in order to first evaluate functionality. By now 6 basic question types are supported: • single-choice • multiple-choice • open textbox • cloze exercise • yes-no question • association Each of these types may consist of text and pictures for the question and may be supported by an optional hint. The application shows all active assessments for a student’s Stud.IP account sorted by courses in which they are offered. Transfer to and from the server is initiated with corresponding buttons.

48

eL&mL 2011 : The Third International Conference on Mobile, Hybrid, and On-line Learning

Figure 2.

List of available assessments sorted by courses

The application uses XML as the technical basis for description and transfer of the assessments and answers. The XML structure for the assessments is build by the server according to the IMS QTI (Question and Test Interoperability) specification (see [9]) and parsed by the mobile application to an internal object structure. On transfer of the solutions, the answers are wrapped by an XML structure and transferred to the server. XML is a common description language, that most programming languages and applications are able to handle. With QTI the interoperability of the application between systems is supported in general. Universally, the design of interfaces is an important issue when developing an application that should integrate itself in other systems. To gain most of it, it’s advisable to use standards so that the application itself is usable in different contexts. We therefore tried to develop an application, that is open enough to support different online assessment systems. However we didn’t have enough resources yet, to completely reach this goal. Our application is limited to the VIPS system at our university, but a second development phase has already been started. After the first successful implementation of assessMe in our local learning concept, the goal is now to support different systems on the outside while using the same UI and data-structure on the inside. IV. I MPLEMENTATION AND E VALUATION IN THE CURRENT TERM

Our blended learning scenario was developed over the years within the course Audio- and Videotechnology. Since 2007 we made no changes to it and gathered practical

Copyright (c) IARIA, 2011.

ISBN: 978-1-61208-120-5

Figure 3.

UI of a single-choice question

experience with quite different groups of students of the media computer science program. Overall we gained much positive feedback from students and colleagues as well. Also, the average results of the final examination has been improved. In the last years, smart-phones were becoming more and more popular and many of our students started using the podcasts on these devices. We were wondering, if usage of the e-Learning services improves, when provided on mobile devices. The first idea for the development of the assessMe application was to fully support the usage of the blended learning scenario from section II on mobile devices. Now that we finished the first development phase, we are going to implement it in the current semester. The course Audioand Videotechnology will be held according to the blended learning concept in the winter term 2010. To carry on with it’s evaluation, we will track the learning behavior of this term’s attendees. They are forming two groups: One using mobile devices (iPhone or iPod touch) throughout the semester and one using less portable devices. To balance the groups, some students are provided with iPod touch devices from the university. Students of both groups are asked to fill a learning diary where they mark their learning activities for this course. This will be differentiated by place of learning and used mediaform of our concept. To keep focus on our blended learning concept we won’t further differentiate learning activities with other media-forms like textbooks or the WWW. A blank sheet of the learning diary is shown in figure 4. The data of the diaries will be supplemented by qualitative

49

eL&mL 2011 : The Third International Conference on Mobile, Hybrid, and On-line Learning

VI. ACKNOWLEDGEMENT Thanks to our developer Dennis B¨ockmann for his great work on assessMe. Thanks to all participants in the current studies. R EFERENCES [1] P. Huegelmeyer, R. Mertens, M. Schroeder and H. Gust. Integration des Virtuellen Pruefungssystems ViPS in die Lehr/Lernplattform Stud.IP, Proceedings of the Workshop on eLearning 2005, HTWK Leipzig, 11.-12. pp. 187-196.

Figure 4.

Blank sheet of the learning diary

data. Students of both groups should answer a questionnaire at the end of the course, where we ask for their subjective experience with the blended learning concept. This way we hope to gather information about the importance of supporting mobile devices within our scenario. Positive feedback from students of preceding semesters made us believe that self-assessment should be provided in other courses, too. Especially in courses where a wide field of knowledge need to be memorized, students will benefit from a continuous learning process. In the current semester, we therefore provide self-assessments also in the course Theoretical Informatics. We distribute the assessMe application among participants of this course and further among other users of the Stud.IP VIPS plugin in our university. This way we have a reasonable group of betatesters. Their experiences will be evaluated by means of a bug tracking system and based on simple quantitative questionnaires. V. C ONCLUSION After several semesters of using the lecture concept described in section II, we realized the need to fully support mobile learning. Therefore we developed the application assessMe, which allows to solve the Stud.IP VIPS assessments on mobile devices. The application was recently released in a first version, which is available only for our universities infrastructure. A second development phase was already started with the goal, to release a more universal version on Apple’s iTunes Store. The usage of the assessMe application is evaluated in the current semester in our blended learning scenario as well as among other users of the assessment system. The results of this evaluation will be published in the near future. With our work we emphasize the possible benefit of supporting mobile devices in approved e-Learning scenarios in general.

Copyright (c) IARIA, 2011.

ISBN: 978-1-61208-120-5

[2] M. Ketterl, R. Mertens, K. Morisse and O. Vornberger. Studying with Mobile Devices: Workflow and Tools for Automatic Content Distribution. World Conference on Educational Multimedia, Hypermedia & Telecommunications (ED-Media 2006), Orlando, FL, USA, Juni 2006. S. 2082-2088. [3] K. Morisse and M. Ramm. Teaching via Podcasting: One year of Experience with Workflows, Tools and Usage in Higher Education. Proceedings of ED-MEDIA World Conference on Educational Multimedia, Hypermedia & Telecommunications 2007 (pp. 2081 - 2088). Chesapeake, VA: AACE. [4] K. Morisse and M. Ketterl. User Generated Web Lecture Snippets to Support a Blended Learning Approach. ED-Media - World Conference on Educational Multimedia, Hypermedia & Telecommunications, 2009, Honolulu (USA). [5] S. Wichelhaus, T. Schueler, M. Ramm and K. Morisse. More than Podcasting: An evaluation of an integrated blended learning scenario. Proceedings of ED-Media - World Conference on Educational Multimedia, Hypermedia & Telecommunications, 2008 (pp. 4468 - 4475). Chesapeake, VA: AACE. [6] S. Wichelhaus, K. Moeller, T. Schueler, M. Ramm and K. Morisse. Weg von der klassischen Frontalveranstaltung - Podcasts, Live-Coaching und Onlinetests als integrale Veranstaltungselemente in der Lehre. Delfi 2008 - 6. e-Learning Fachtagung der GI, S. 209 - 220, Lecture Notes in Informatics, Vol. 132, Gesellschaft fr Informatik, Bonn, 2008. [7] S. Wichelhaus, T. Schueler, M. Ramm, K. Morisse. Medienkompetenz und selbstorganisiertes Lernen - Ergebnisse einer Evaluation. In S. Zauchner, P. Baumgartner, E. Blaschitz, A. Weissenbaeck (Hrsg.): Offener Bildungsraum Hochschule, S. 124 - 133, Waxmann-Verlag, M¨unster, 2008. [8] K. Wannemacher. Computerbasierte Pr¨ufungen. Zwischen Self-Assessment und Abschlussklausuren. In E. Seiler Schiedt, S. K¨alin, C. Sengstag (Hrsg.): E-Learning - alltagstaugliche Innovation?, S. 163 - 172, Waxmann-Verlag, M¨unster, 2006. [9] IMS Global Learning Consortium - Question and Test Interoperability Specification http://www.imsglobal.org/question/ [10] Stud.IP LMS http://www.studip.de

50