IMCL Template

IMCL2008 Conference

16-18 April 2008, Amman, Jordan

A Mobile Quiz System for Assessment of the Learning Process Mohammed Otair1, Adnan Tawfiq1, A. Y. Al-Zoubi2 and Akram Alkouz2, 1

Arab Academy for Banking and Financial Sciences, Jordan, 2 Princess Sumaya University for Technology, Jordan.

Abstract— Elearning systems usually encompass an assessment component tailored toward PC- based with Web access but are customized neither for mobile student nor for mobile devices. A mobile quiz system (MQS) running on a Personal Digital Assistant (PDA) to support the learning process is proposed to enable instructors to create an interactive Web-based quizzes which can be delivered to an Internet-connected computer equipped with a browser. Students can access and perform quizzes anywhere anytime using the PDAs. The MQS, designed to accommodate faculty and student interfaces, was developed, implemented and tested successfully.

Index Terms— Mobile Learning, Mobile Quiz, Mobile Devices, PDA.

I.

INTRODUCTION

Mobile learning is defined as the ability to learn anytime and anywhere using handheld devices with direct access to learning material [1]. Mobile learning has thus received a great deal of attention in recent years and is currently viewed as the courier of the next wave of learning due to the rapid growth in wireless networks and mobile devices, for example, one in every six persons in the world today owns a mobile phone [2]. However, mobile applications seem natural to digital natives such as incoming students, but not for the vast majority of instructors. Consequently, evaluation of the learning process based on mobile applications presents special challenges at student, instructor and institutional levels [3]. Improvements need to be made on the efficiency and interactivity of existing examination systems such as multiple-choice quizzes which are based on short message text (SMS) and Wireless Application Protocol (WAP) [4-7]. In such systems, the questions are sent as SMS messages by the instructor to the student who answers the questions back by an SMS to the instructor. Additionally, mobile examination systems must utilize the full capabilities of the rising 3G and 4G networks. In the other words, WAP and SMS mechanisms are no longer valuable enough to improve the quizzes systems and the learning process. Examination of several mobile learning and teaching applications that are currently being deployed and evaluated reveals that mLearning can only compliment eLearning. This can be achieved by creating an additional

channel of access for mobile users with mobile devices such as hand phones, PDAs and pocket PCs. In this paper, a new mobile quiz system (MQS) is the proposed. The system, which may operate on either PC or mobile platforms, consists of an interface for the instructor and another for the student. The quizzes are prepared at the instructor interface and then uploaded into the database server. The student undertakes the quiz on a PDA or a computer connected to the Internet. The interface at the student (client) side calculates and shows the grade. Finally the grades and answers are made available to the instructor when the students complete the quizzes. The proposed MQS also includes features such as student login, viewing quiz for each course, multiple choice quiz questions, error validation and results display. II.

MOBILE DEVICES AND LEARNING

Mobile learning (mLearning) is focused on eLearning environments that implement wireless transmission and mobile devices such as PDAs, mobile phones, laptops and tablet PCs [8]. In other words, mLearning is simply learning that takes place with the help of mobile devices [9-11]. Mobile devices, for example, PDAs and “smartphones,” are increasingly pervasive, especially in student populations [10]. In some parts of the world each student has more than one such device. The pervasive nature of these devices provides an unprecedented opportunity for creation of learning applications. As with paper and books, we can increasingly expect that our students will (or can be mandated to) have personal access to a mobile device to complement their traditional learning activities [3]. Furthermore, it was noted that in excess of 90% of students agree that a computer is useful for general study with special emphasis on document creation and searching for information (e.g., Google [12]). They also note that a mobile phone is useful to assist with university studies to send or receive text messages (84%) and as a personal organizer (60%) [12]. Mobile learning is a field which combines two very promising areas, mobile computing and eLearning. Mobile learning is intended as any form of learning and teaching that use mobile devices in a mobile environment. On the other side of mobile learning, we have eLearning, i.e., every educational process assisted by computers

The 3rd International Conference on Mobile and Computer Aided Learning, IMCL2008, www.imcl-conference.org

1

IMCL2008 Conference

through the use of networks, and Internet in particular. Mobile learning is often considered as the future of learning, or as an integral part of any other form of educational process in the future. A mobile learning educational process can be considered as any learning and teaching activity that is possible through mobile tools or in settings where mobile equipment is available [13]. There are three main forms of PDA mobile technologies: Palm operating system (running on 65% of the PDA market), Pocket PC from Microsoft, and Smart phones [14]. A PDA is actually a computer based handheld device that incorporates personal organizer tools and has the ability to easily exchange information with a desktop computer [15]. PDAs are pocket-sized computers. PDAs are extensible, with optional hardware (e.g., keyboards and wireless networks) and software (e.g., word processors, databases, bilingual dictionaries, flash-cards). PDAs cost US$100-500, but most programs are free, so running costs are zero [16]. One of the PDAs major strengths is that unlike a Laptop or Tablet PC a PDA can be easily carried at all times [15]. Furthermore, early evaluations of learning with mobile devices have suggested that a wide range of students respond to learning with mobile devices favorably. For example, 90% of teachers in a study of 100 Palmequipped classrooms reported that handhelds were effective instructional tools with the potential to impact student learning positively across curricular topics and instructional activities [14]. A recent study suggests that student work habits can be categorized into several components, the key elements relevant here are [17]:  Pervasive and integrated, where students use a wide range of technologies to suit individual needs;  Personalized, students are highly pragmatic, moving between computers, books and the internet seamlessly;  Social, technology facilitates a 'community of practice' for networking, sharing ideas, asking for help and feedback (from each other).  Interactive, students have a preference for byte sized chunks of content than can be stored on handheld devices and accessed 24/7;  Transferability, access to information is seen as ubiquitous, and simple, students expect to have the same levels of access and interaction in courses;  Time, students are increasingly using technology to remain connected and synchronized. As mentioned above, the learning process can take place in different settings, i.e. in a lecture room, in a coffee place or at home. For these different settings, there exist a variety of possible learning scenarios that have to be supported by collaborative applications [18]:  All participants meet in one room, so all members are physically present. This can be used for teaching scenarios as well as for interactive meetings, such as brainstorming.

16-18 April 2008, Amman, Jordan  All participants are physically distributed, potentially mobile, and are communicating with collaborative applications. This can be used to let physically distributed people interact and share the same workspace and/or applications.  A mixture of both scenarios: some participants meet in one room physically, and some other participants, physically distributed, are connecting to this meeting via collaborative applications. As mentioned earlier, with PDA, an instructor or student can do interesting things: take notes, calculate, sketch ideas, collect data, access resources, manage activities and, with the right hardware, even access the Internet wirelessly. Handheld computers can offer unique benefits to students and teachers. Students can have a personal, portable device ready-at-hand for individual or collaborative learning activities, wherever they go. Students can use handheld computers to collect data in the field, to learn vocabulary words while waiting to be picked up after soccer practice, or to self-quiz during a long car ride [19]. On the technological side, we have seen some developments that can give a boost to mobile learning. On the one hand, there is the upcoming usage of GPRS and UMTS; this makes it possible to send/receive data at a higher speed. On the other hand, there is a large increase of functionality and usage of mobile devices especially with PDA. One of the latest developments is the support of TCP/IP, http-protocols within WAP2.0. This makes mobile internet applications possible and gives access to general web page formats [19]. III.

MOBILE QUIZ SYSTEM (MQS)

The proposed mobile quiz system (MQS) is based on a client-server architecture where the server can be accessed by an instructor while the students act as clients. The MQS provides sets of multiple choice questions that the students can complete online. After they upload their attempt, they are told which questions were answered correctly (or not). A wireless application is simply one that runs on a wireless device. Some wireless applications are embedded or hardwired within the host device. These are typically small, standalone ROM-based wireless applications, which are not easily alterable. They may or may not require connectivity across a wireless network to a remote server. Other wireless applications are device downloadable applications, which either run off a remote server or run within the device itself. The developed wireless applications (i.e. MQS) typically belong to this second group. The proposed MQS was first applied to a mandatory database course, with over 100 STUDENTS typically enrolled each semester, in all information technology majors in the Arab Academy for Banking and Financial Sciences, Jordan. In addition, all of the students possessed mobile devices such PDA. The core idea was therefore centered on how to use PDAs in such a way to


2

IMCL2008 Conference

overcome challenges of large classroom learning with broad applications such as quizzes. The main benefits of an online PDA quiz (i.e. MQS) are: 1. Saving thousands of paper-printing per lecture; 2. Efficient, automatic marking of grades of multiple choice questions on online quizzes; 3. Better utilization of resources when instructors are more focused on teaching rather than on administrative efforts (e.g. paper printing). In other words, this in return will save effort, time and resources. In addition to the three significant benefits of MQS, there are six activities can that be made through the assessment in the learning process using MQS as follows:


be 6 (Offset =0.2*33), and variable I is equals to 67 (increased by Skip value (i.e. 34+33)). In summary, the displayed questions will be 14, 63, and 73 respectively. Thus, there is a low possibility to duplicate the same three questions of this quiz for any two students.

 instructors set and upload quizzes using any Internetconnected computer equipped with a browser (to be stored on the database server);  instructors give the permission for students and only the authorized students can take the quiz;  authorized students download quizzes into their PDAs using MQS;  authorized students complete quizzes ;on their PDAs;  authorized students upload answers using MQS;  authorized students receive feedback with their final grades. Each quiz consisted of a set of multiple-choices questions, which in turn consists of a question, a correct answer and three wrong answers. In general, there are two types of multiple-choices questions:  an incomplete statement followed by several answer choices;  a question followed by several answer choices. The answers for every question are saved. With this, the progress of the students can be monitored. Only the administrator of the quiz system can view the result. The questions appear in groups of N on each PDA which are randomly generated to prevent any duplicate questions between neighboring students, if they are located in one room, where N is the number of questions of the quiz which is determined by the instructor in advance. For instance, the instructor can submit 100 questions and select only 3 (for M) questions that must be answered by each student. To explain how an MQS generate random questions for each student, consider the flowchart in Fig. (1), in which RND is a built-in function written in Visual Basic, designed to generate a random number between 0 and 1 (excluding 1). To know how the flowchart works, let us take the following example: Assume that RND function generates 0.4, 0.9 and 0.2 respectively. From the following flowchart Skip will be 33 (100/3), at the first iteration the Offset variable will be 13 (Offset=0.4*33)), then the first question that will appear is question number 14 (13+1, 1 is the current value of I). In the second iteration I variable will be 34 (increased by Skip value (i.e. 1+33)), and the Offset will be 29 (Offset =0.9*33)), then the second question that will appear is question number 63 (34+29, 34 is the current value of I). The third question that will appear is question number 73 (67+6) because the Offset value will

Figure 1. A flowchart showing the mechanism of generating random questions. IV.

TECHNOLOGY ARCHITECTURE OF MQS

The software tools and technology architecture used to develop MQS focused primarily on the platform of PDA/phone devices running the Pocket PC operating system. The quizzes use ASP.Net and C# to implement their interactivity, but students do NOT need to know anything about these languages in order to use the developed system. All they need to do is enter the data for their quizzes (questions, answers, etc.), and press a button. In the developed system the Web pages for the instructors are created, and they can then upload their quizzes to the servers. Moreover, the students can submit their quizzes using their mobile devices such as PDAs. On the client side, the applications run in the mobile terminal’s browser. This thin-client approach has the following key advantages: location and device independence of application logic, centralized software maintenance and data management: a browser plug-in (for PDA and Smartphone) in combination with a server platform. PDA (Pocket PC operating system) with Internet access via wireless LAN at Arab Academy is available for all students (and instructors). Table 1 show the targeted device for the clients:


3

IMCL2008 Conference

Class PDA


TABLE 1 TARGETED DEVICES Device Operating System IPAQ 5450

PocketPC 2003

Browser Pocket IE

With the availability of high bandwidth wireless channels such as 3g-telecommunication infrastructure and wireless LAN, MQS is becoming more feasible. a summary of the technology architecture that can be used by MQS is as follows as shown in fig.2:  Server OS: Windows/Linux /Unix/Mac;  Application: MQS;  Client/PDA OS: Pocket PC / Windows Mobile/Palm/Symbian/Notebook/ PC;  Supported PDAs and/or Phone from HP, Dell, Acer, Mitac, Dopod, O2, Palm, Treoand, ect;  Maximum concurrent connections: “Unlimited”.

Figure 2. Block diagram of the mobile quiz system at the client side. The choice of which PDA should be used is personal since what suits one person may not suit another. The personal qualities of the device are reinforced by a combination of customization of the device by the installation of applications and the content of the data stored on the device. The following devices are useful for the developed system and can be used without any problems:  GSM/GPRS cellular phones;  PDA with Pocket PC/Windows CE;

 Smart-phones;  UMTS telephones;  Tablet PCs. V.

IMPLEMENTATION OF MQS

The mobile quiz system is packaged into a server and can be easily downloaded into the mobile device. The Quiz system Admin web page written in ASP.Net language allows the faculty to conveniently enter the quiz questions, the number of questions, and give the permission for authorized students into a SQL server database file. At the other side, the client side written in C# language allows students to take up a quiz posted by a faculty member over the PDA. The MQS is divided into two main sides: 1. A Server side (accessed by faculty and authorized instructors): which is a web-enabled application. 2. Clients Side (used by Students): which is a wireless based (handheld device) application. 1 Server or Faculty Side The server side application was to develop a web based wireless system to be used by community of authorized users (administrator and instructors). The faculty is provided with a username/password to log onto the QMS faculty Admin Web page. The web page allows the faculty to view any of the registered student quiz results or post a new quiz. The username and passwords for each student registered by the faculty, then the students can login from their PDAs to take and view quiz score. To create a new quiz the faculty enters a course name and the number of questions of the quiz that must be entered (by the instructor) and that must be answered (by the students later on). Then the quiz questions along with the correct choice of answer are entered one by one as shown in Fig. (3). Moreover, an instructor can evaluate each student by reviewing his grade and his answers to questions. Also he can improve his next quizzes by analyzing the students’ answers to each question.

Figure 3. Entering questions by instructor at the server side.


4

IMCL2008 Conference


5.2 Clients or Students Side The wireless connection system facilitates the communication and monitoring process of students during the quizzing process. The student needs to download the program of MQS into a PDA or smart phones. Once the application is successfully installed into the mobile device the student registers with the faculty (enters a valid username/password to take the quiz). The application allows student to take the quiz once and the score of the quiz will be taken as the final score. The total time for taking the quiz and the number of questions is set by the faculty. Each question consists of 4 choices and the student selects a choice from the radio-buttons and clicks on the finish button at the last question. To move on to the next question the student clicks the next button. The main activities and functions on the clients’ side are as shown in Fig. (4):  Log in into the system to take a quiz;  Select the quiz for specific course to attempt;  Select the correct answers (based on the number of questions);  Summary of the quiz result.

The students need to log in into MQS application to subscribe and quiz download onto the PDAs. When the student is logged-in into his PDA, he/she will choose the course name and then select the correct answers for each question for this quiz. He/she will see the first of the questions and can answer the first question by clicking on one of the answer radio buttons; keep choosing answers until getting the correct one. When a question is answered, the student can move on to the next one using the Next buttons under the question, as we seen from Fig. (4) (a, b, c, and d).

Figure 5. The mobile quiz result. At the end and once the quiz is complete by clicking on the Finish button, the student answers are compared with the correct answers and the student is given the result immediately, as shown in Fig. (5). The result is then sent back to server for review.

VI.

(a)

(b)

(c)

(d)

CONCLUSIONS

The importance and acceptance of mobile learning systems by both students and faculty is becoming increasingly important. The advantages offered by such systems are accessibility, mobility, and collaboration. The opening of the educational process for continuous and lifelong learning through mobile learning is therefore of paramount importance. The results obtained show that mobile devices can extend a learning environment and increase the flexibility of the learning process. The Mobile Quiz System proposed, creates a personalized, interactive eLearning experience particularly for large classes, through the use of wireless Personal Digital Assistants (PDAs), resulting in superior learning outcomes. This mechanism is not limited to taking multiple-choice quizzes and also can be used for other activities like making elections, taking tests with descriptive or short answers, and other forms of examinations and evaluation mechanisms.

Figure 4. Mobile quiz interfaces accessed via a PDA.


5

IMCL2008 Conference


REFERENCES

AUTHORS

[1] Kinshuk, and T. Goh, “Mobile Adaptation with Multiple Representation Approach as Educational Pedagogy”. In Uhr W., Esswein W. and Schoop E. (Eds.) Wirtschafts-informatik, (1) 747764, 2003. [2] M. Shahreza, “An Improved Method for Steganography on Mobile Phone”, WSEAS Transactions on Systems, Vol. 4, Issue 7, pp . 955-957, 2005. [3] D. Vogel, D. Kennedy, D. and R. Kwok, “Do Mobile Device Applications Lead to Learning?” Proceedings of World Conference on Educational Multimedia, pp. 555-564, 2007. [4] M. Shahreza, “Taking Multiple-Choise Quiz by SMS on Mobile Phones Including Analyzing Property”, Journal of Computers, Vol. 2, No. 2, 2007. [5] L. Bollen, S. Eimler and H. Hoppe, “The use of mobile computing to support SMS dialogues and classroom discussions in a literature course”, Proceedings of IEEE International Conference on Advanced Learning Technologies, Finland, pp. 550–554, 2004. [6] A. Stone, J. Briggs And C. Smith, “SMS and interactivity some results from the field, and its implications on effective uses of mobile technologies in education”, Proceedings of 2002 IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2002), Sweden, pp. 147-151, 2002. [7] A. Tretiakov . and Kinshuk “Creating a Pervasive Testing Environment by Using SMS Messaging”, IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2005), Japan, pp. 62-66, 2005. [8] E. Mikleia, “Mobile Devices to be applied as supporting tools in Research Methods Class for Undergraduate Students”, Proceedings of the 7th international conference on Learning sciences, pp. 918919, 2006. [9] J. Attewell, “Mobile learning: reaching hard-to-reach learners and bridging the digital device”, In: G. Chiazzese, M. Allegra, A. Chifari, S. Ottaviano, ed. Methods and Technologies for Leaning. Southampton: WIT Press, 361-365, 2005. [10] P. Dawabi, “Using mobile devices for the classroom of the future. Learning with mobile devices, research and development.UK: Learning and Skills Development Agency”, pp 55-60, 2004. [11] A. Trifonova and M. Ronchetti, “Prepare for a bilingualism exam with a PDA in your hands”, W I T Transactions on Information and Communication Technologies, Vol. 34, pp 343-347, 2005. [12] G. Kennedy, K. Krause, K., T. Judd, A. Churchward, and K. Gray, “First year students' experiences with technology: Are they really digital natives?” Australia: The University of Melbourne. Retrieved 26 October 2006, from the World Wide Web: www.bmu.unimelb.edu.au/research/munatives/natives_report2006.p df. [13] L. Colazzo, A. Molinari, M. Ronchetti, and A. Trifonova, “Towards a multi-vendor Mobile Learning Management System”, Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, pp. 2097-2100, 2003. [14] Ufi/learndirect and Kineo, “Mobile Learning Reviewed, Technology and Learning, sharing best practice”, Retrieved 25th September 2007, the World Wide Web: www.ufi.com/home/section5/10_goodPracticeGuides/Mobile_learn ing_reviewed_final.pdf. [15] J. Trinder, J. Magill and S. Roy, “Portable Learning and Assessment: Towards Ubiquitous Education”, International Journal of Electrical Engineering Education, 2005. [16] C. Houser, P. Thornton, P. and D. Kluge, “Mobile Learning: Cell Phones and PDAs for Education”, Proceedings of the International Conference on Computers in Education (ICCE’02), pp. 1149-1150, 2002. [17] G. Conole, M. de Laat, T. Dillon, T. and J. Darby, “JISC LXP: Student experiences of technologies”, Milton Keyes: Joint Information Systems Committee (JISC). Retrieved 5 December 2006, World Wide Web: www.jisc.ac.uk/media/documents/lxp_project_final_report_nov_06.pdf. [18] O. Brand, L. Petrak and M. Zitterbart, “Learning and Collaborating with Mobile Clients”, International Conference on Computers in Education, pp. 295-296, 2002. [19] J. Schreurs, “M-learning using PDA’s and our supporting LOMS”, International Journal of Computing and Information Sciences, 4(2). p. 72-80, 2006.

Mohammed A. Otair is with Computer Information Systems, Arab Academy for Banking and Financial Sciences, Jordan Adnan Tawfiq is with Computer Information Systems, Arab Academy for Banking and Financial Sciences, Jordan Abdullah Y. Al-Zoubi from Princess Sumaya University for Technology, Jordan. Akram Alkouz From Princess Sumaya University for Technology, Jordan.


6