2014 IEEE 14th International Conference on Advanced Learning Technologies
Comparison of Adaptive E-learning Mobile and Web-Based Software Applications Effectiveness a
Muhammad Adnan
a,b ,
Hamid Mukhtar
b
Kohat University of Science and Technology (KUST), Kohat 26000, Pakistan b National University of Science and Technology (NUST), Islamabad 44000, Pakistan Email: 10msitmadnan,
[email protected]
system uses mobile app called ClassGuide that through timely personalized reminders, informs students about exam, quizzes, assignments and presentations details. In the second approach, the E-learning software system sends timely messages to students regarding their attendances, assignments submissions, quizzes, teachers meeting time, class activities and overall semester schedule using web-based application. As compared to the interactive mobile app, web-based SMS application is adaptive, less interactive and it does not require daily and regular entries from the user side. Only concise, personalized and relevant messages are sent to the user’s mobile and user memory load is kept reduced as much as possible. We have tried to reduce the information gap between the students and the teachers as mobile phones are ubiquitous and the fastest and easiest way of communication nowadays. We now briefly discuss the persuasive technology, its effectiveness and the way it can improve human learning behavior. Persuasive technology is a branch of Human Computer Interaction (HCI) and its aim is to change the thinking and behavior of users in constructive and positive manner without any deception and coercion [3] [4]. Persuasive technology is very common with health, politics, safe driving, education, environment and public relation as its key areas [5] [6] [7]. Nowadays, mobile phones are ubiquitous and a key platform for persuasive technology and personalized learning to be used in them [8] [9]. Our research paper is organized as follows. Section I is about introduction to our research work. In section II details about the Android mobile app called ClassGuide and Web-based SMS mobile application are discussed. Section III is about the survey, usage, results and testing process of software applications and section IV concludes our research work and gives further future direction to our research.
Abstract—Our research studies the impact of different adaptive learning strategies on improving students study behavior. We have developed an e-learning software system that assists students to improve their study behavior from two different perspectives. In both of the perspectives, our e-learning system has different functionalities and implements different adaptive learning techniques. In the first approach, e-learning software system uses mobile app that through automatic reminders and different screen widgets, persuades the students to stay in touch with study and keep themselves up to date about different class activities. In the second approach, e-learning system uses automated webbased SMS application for automatically sending timely and related messages to different students in order to keep them aware of their study progress and different class activities. Both approaches were thoroughly tested on undergraduate students and at the end interesting results were revealed about different adaptive and persuasive techniques that would help educationalist and software engineers in streamlining the process of mobile learning in future. Index Terms—E-learning, Adaptive, Personalized, Smart Phones, Mobile Persuasion, Persuasive strategies, Study Behavior, Persuasive text messages, Reminders
I. I NTRODUCTION Currently different universities have employed an open source e-learning software platform system called Moodle which is very effective and fruitful [1]. By using Moodle, teachers can easily manage different courses, take online attendance and create online discussion groups. Students, on the other hand, can submit assignments and presentations and can download different study materials online. Students can also view their quiz, exam and assignment marks online. It has been observed that due to the one-size-fits-all nature of Moodle, all students are treated equally and their is no social appreciation for good students. Secondly, most of the students do not interact with Moodle on daily basis. Mostly the students access the Moodle during exams days while at the beginning of the semester, access rate is very low [2]. Our research integrates web component, mobile component and personalized persuasive strategies to effectively encourage students to study from the start of the semester and to stay up to date about different class activities. Our e-learning software system tries to encourage personalized learning and reduces information gap between the students and the teachers through mobile phones and web. E-learning software system uses two approaches for keeping students in touch with their academic studies activities. In the first approach, e-learning software 978-1-4799-4038-7/14 $31.00 © 2014 IEEE DOI 10.1109/ICALT.2014.30
II. S TUDY D ESIGN The objective of our research is to streamline the process of persuasive mobile learning and to find out what persuasion strategies are most effective in increasing students study behavior. For finding effective persuasion strategies and techniques it is important to first test them thoroughly and then apply them on general masses in order to change their thinking and behavior in a positive way. In the next section both personalized persuasive applications, 67
(a) Assignment Submission Activity
(b) ClassGuide Activities List
(c) Widget Screen
(d) Note Taking Activity
Fig. 1: ClassGuide Interface Snapshots their working, design and how they are different from each other is discussed.
B. Automated Web-Based SMS Application Automated web-based SMS application assumes that all students are on the same stage at the beginning of a semester. With the passage of time automated web-based SMS application runs automated queries on online database where every student’s attendance, quiz marks, exam marks, presentation marks and assignment marks are stored. Then based on students progress, different persuasive messages are sent on their mobile phones. Unlike ClassGuide which uses local mobile generated notifications as reminders, the automated web-based SMS application automatically sends different short mobile messages to students on daily, weekly and monthly bases based on their class performance. Automated web-based SMS application is programmed in such a way that short messages are sent automatically to students mobile phones and no human intervention is needed. For example, students who are weak in academics are warned about the likely consequences if they continuously show poor performance. On the other hand students showing improvement in academics are appreciated and encouraged to further improve their academic performance. Similarly, students who are very good right from the start of semester are constantly praised and admired through short messages. Short messages become more persuasive when the performance of good students is shared among other students for appreciation and social acceptance. This way other weak and improving students are also persuaded to concentrate on their studies and get admiration from faculty and other students. Through automated web-based SMS application, students always stay in touch with the current semester activities and all updates are provided to them in time. Automated web-based SMS application sends messages based on the architecture shown in figure 2. Apart from sending automatic short messages to students the automated web-based SMS application also allows students to send queries from their mobiles for any needed information. This feature of automated web-based SMS application is important as sometimes a particular student may not
A. Android Mobile App ClassGuide First we developed ClassGuide app, an interactive and diverse Android mobile app that assists and facilitates undergraduate students in increasing their study behavior. With ClassGuide app a student can maintain the time table of semester and customize it to his or her own preferences. Students are also allowed to keep record of their assignments, presentations and quizzes. But ClassGuide app is not like just another organizer application. The moment a particular student enters data about class time table, assignments, presentations or quizzes ClassGuide app will remind him/her about particular activity if he/she forgets it. The other persuasive technique in ClassGuide app is mobile screen widget that daily retrieves data about class time table, assignments and presentations and shows it to a student on his/her mobile’s main screen. With ClassGuide app students can increase their presentation skills where they can record their daily, weekly or monthly video presentations dates and times wise. Students can rate their presentations, compare them and also can record them again for more improvement and perfection. ClassGuide app also allows students to record audio lectures of teachers, tag them and consult them later during their studies. These audio lectures are recorded subject and lecture wise so that a student can easily find and consult them. With the help of the ClassGuide app, students can take short notes where students can write and also draw anything on mobile screen and save it as a picture. In using ClassGuide app, a student has complete authority over its different functionalities e.g. a student can adapt it and customize it to his own personal preferences. Figure 1 shows different ClassGuide app snapshots related to assignment submission activity, ClassGuide activities list, widget screen and note taking activity. Next we discuss automated web-based SMS application.
68
Fig. 3: Bulk message sending activity by a teacher
Fig. 2: Architecture of Web-based SMS application and its working get class updates due to no mobile signals or student’s cell phone being switched off when automated updates are being sent. Students can query automated SMS system in different ways. They can fetch data related to upcoming examination by simply typing exam as keyword in their mobile phone message box and sending it to the SMS server. Similarly data related to timetable, quizzes marks, assignment marks and student’s class position can be fetched by typing respective keywords like time table, class name, marks quiz, marks assignment, my position in mobile phone message box and sending that to SMS server number. This way, if a particular student does not get automated updates regarding class activities from automated web-based SMS application he or she can also manually query the automated web-based SMS application about any relevant information. The other persuasive feature of automated web-based SMS application is communication between teachers and students via mobile phone messages. Here the most important thing about teachers and students communication is the privacy of their mobile phones numbers. Communication between teachers and students become more persuasive when both sides need only names for communication rather than mobile numbers. Automated web-based SMS application always hide mobile numbers of teachers and students during communication. Teachers can also send broadcast messages which is received by the whole class through automated web-based SMS application interface as depicted in Figure 3. Similarly teacher can communicate with a particular student via short message telling him or her about meeting time or other important information.
Fig. 4: Web-based SMS application and ClassGuide app Comparison also based on constructs of TAM. Mainly there are four construct of TAM called perceived usefulness (PU), perceived ease of use (PEOU), Attitude and intention to use (ITU). Since its inception, TAM model is extensively used in estimating and accepting different types of software applications related to health, education, web-based learning systems and mobile applications. Likert scale type questions were used to gather data from students about ClassGuide app and automated web-based SMS application effectiveness. In Likert scale there were seven options for every question out of which a student can select only one particular option. Weightage of 1 was given to strongly disagree, weightage of 2 to disagree, 3 for some what disagree, 4 for neutral, 5 for some what agree, 6 for agree and 7 for strongly agree option. Cronbachs alpa method was used to evaluate the internal reliability of both questionnaires. In order to estimate the acceptance level of ClassGuide Android app and automated web-based SMS application we used path analysis, regression analysis and correlation analysis on the data collected from both questionnaires. Eighty students (75 males and 5 females) from BS 1st semester and BS 3rd semester of Department of Physics, Kohat University of Science and Technology, Kohat, Pakistan were selected for testing the effectiveness and usability of ClassGuide app and automated web-based SMS application. First it was made sure that all of them had Android mobile and all of them were given two hours short training related to ClassGuide app functionality and usage. It was decided that students will have to use both software’s for thirty days for their acceptance
III. S URVEY D ESIGN The intention of carrying out the survey was to streamline the process of personalized persuasion and to find which persuasion techniques are more effective in improving students study behavior. We have used Technology Acceptance Model (TAM) [10], to test the acceptance level of ClassGuide and automated web-based SMS application. The questionnaires related to ClassGuide and automated web-based SMS application are
69
TABLE I: Web based SMS application Correlation Matrix SMSappPU Pearson Correlation 1 SMSappPU Sig. (2-tailed) N 75 Pearson Correlation .308** SMSappPEOU Sig. (2-tailed) .007 N 75 Pearson Correlation .413** SMSappAttitude Sig. (2-tailed) .000 N 75 Pearson Correlation .215 SMSappItu Sig. (2-tailed) .064 N 75 Correlation is significant at the 0.01 level (2-tailed). Correlation is significant at the 0.05 level (2-tailed).
SMSappPEOU .308** .007 75 1
SMSappAttitude .413** .000 75 .435** .000 75 1
75 .435** .000 75 .291* .011 75
75 .455** .000 75
SMSappItu .215 .064 75 .291* .011 75 .455** .000 75 1 75
TABLE II: ClassGuide Correlation Matrix Pearson Correlation ClassGuide PU Sig. (2-tailed) N Pearson Correlation ClassGuide PEOU Sig. (2-tailed) N Pearson Correlation ClassGuide Attitude Sig. (2-tailed) N Pearson Correlation ClassGuide Intention to Use Sig. (2-tailed) N Correlation is significant at the 0.01 level (2-tailed).
ClassGuide PU 1 75 .322** .005 75 .301** .009 75 .139 .235 75
ClassGuide PEOU .322** .005 75 1 75 .386** .001 75 .219 .059 75
and effectiveness. Both software applications were tested and evaluated for their effectiveness by those eighty students for thirty days and at the end two separate questionnaires were given to them related to the usage and applicability of both applications. Out of those 80 students, 5 students’ data was screened out due to missing data, outliers and data repetition. Interesting results were revealed about both mobile persuasion strategies which may help streamlining the process of mobile persuasion.
ClassGuide Attitude .301** .009 75 .386** .001 75 1 75 .172 .139 75
ClassGuide Intention to Use .139 .235 75 .219 .059 75 .172 .139 75 1 75
statistically significant positive correlation as compared to the constructs of ClassGuide app. Figure 5(a) and figure 5(b) shows the observed variable path analysis (OVPA) structural model for automated WebBased SMS application and ClassGuide app respectively. In figure 5(a), the rectangular boxes represent means for SMSappPU, SMSappPEOU, SMSappAttitude and SMSappItu items while the circles e1, e2 and e3 are the error terms. Similarly in figure 5(b), the rectangular boxes represent means for ClassGuide PU, ClassGuide PEOU, ClassGuide Attitude and ClassGuide ITU. The circles e1, e2 and e3 in figure 5(b) are error terms associated with the means of corresponding items. The directed arrows in both figure 5(a) and figure 5(b) are standardized regression weights between pair of items. The stronger regression weights in figure 5(a) shows that automated web-based SMS application items tends to have more impact on end users perception about it. Finally, the internal Cronbach’s reliabilities of the questionnaire for PEOU, PU, Attitude and ITU related to automated web-base SMS application were 0.87, 0.81, 0.86 and 0.90 respectively. Similarly, the internal Cronbach’s reliabilities of the questionnaire related to ClassGuide app for PEOU, PU, Attitude and ITU were 0.78, 0.85, 0.89 and 0.93 respectively.
A. Survey Analysis Figure 4 indicates students’ level of agreement towards automated Web-based SMS application as compare to ClassGuide app with PU, PEOU, Attitude and ITU constructs. The graph in figure 4 clearly shows that students were more oriented towards automated Web-based SMS application as compared to ClassGuide app. From correlation analysis, the constructs of automated Webbased SMS application are strongly correlated as compared to the construct of ClassGuide app as represented in table I and table II respectively. PU (Perceived Usefulness) of ClassGuide app has statistically weak correlation with PEOU (0.322), Attitude (0.301) and ITU (.139). As compare to ClassGuide app PU, overall PU of automated Web-based SMS application has more correlation strength with PEOU (0.308), Attitude (0.413) and ITU (0.215) constructs. Data in two correlation matrices in table I and in table II clearly shows that the constructs of automated Web-based SMS application have
B. Survey Discussion Most of the students were of the view that automated webbased SMS application was simple, effective and useful. Also automated web-based SMS application frees them from all
70
(b) OVPA For ClassGuide App
(a) OVPA For SMS Application
Fig. 5: Observed Variable Path Analysis Structural Model for Web-Based SMS and ClassGuide Application
made more effective by integrating online adaptive lessons on it web interface where every student can view his/her own personal study progress. In addition, taking online course competition from group of students may further motivate them in improving their course knowledge.
responsibilities and worries, giving students right information at the right time. Furthermore, socially appreciating good students performance through SMS motivated other students to improve their study performance. On the other hand, students view about ClassGuide app were assorted. They were of the opinion that ClassGuide app requires so many input data and daily mobile interaction. The other worry was that ClassGuide was increasing their memory load and responsibilities. Thus instead of becoming a facilitator, they were controlling ClassGuide which was difficult task to perform on daily basis. Some students also appreciated ClassGuide app for giving them timely reminders, persuading them for class preparation, improving their presentation and communication skills but they were also of the view that ClassGuide app was also increasing their memory load during its usage. Standalone mobile app that persuades students to bring improvement in their study behavior could not be effective as most of the students use mobile app for entertainment and games. Furthermore, a student having lazy behavior will feel frustration when using ClassGuide in very precise and accurate way. On the other hand, automated web-bases SMS application is smarter and knows what and when to send relevant information to students mobile phones. This way, with the help of automated webbased SMS application, students feel free, open and utilizes their maximum time in improving their weak study areas.
R EFERENCES [1] M. Trust, “Moodle.” https://moodle.org/. Accessed July 7, 2013. ˇ [2] M. Minovi´c, V. Stavljanin, M. Milovanovi´c, and D. Starˇcevi´c, “Usability issues of e-learning systems: Case-study for moodle learning management system,” in On the Move to Meaningful Internet Systems: OTM 2008 Workshops, pp. 561–570, Springer, 2008. [3] R. Gass and J. Seiter, “Persuasion, social influence, and compliance gaining,” Recherche, vol. 67, p. 02, 2010. [4] S. Consolvo, E. Paulos, and I. Smith, “Mobile persuasion for everyday behavior change. captology media,” 2007. [5] P. Beatty, “Designing persuasive technologies for human factors engineering: An alternative classification to the triad of captology,” in Persuasive 2008: Third International Conference on Persuasive Technology: Poster Proceedings, pp. 98–101, 2008. [6] A. J. Obrien, C. Alfano, and E. Magnusson, “Improving cross-cultural communication through collaborative technologies,” in Persuasive Technology, pp. 125–131, Springer, 2007. [7] M. Foth, C. Satchell, E. Paulos, T. Igoe, and C. Ratti, “Pervasive persuasive technology and environmental sustainability,” in Proc Pervasive 08 Workshops, 2008. [8] B. Fogg, Mobile persuasion: 20 perspectives on the future of behavior change. Mobile Persuasion, 2007. [9] D. Berdichevsky and E. Neuenschwander, “Toward an ethics of persuasive technology,” Communications of the ACM, vol. 42, no. 5, pp. 51–58, 1999. [10] F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS quarterly, pp. 319–340, 1989.
IV. C ONCLUSIONS AND F UTURE W ORK We showed that learning can be enhanced by sending simple text messages carrying personalized information on the right time to students cell phones. From adaptively point of view, software applications that are easy to use and reduces students memory load are more effective. Furthermore, Socially enabled technologies of web and mobile phones that appreciate the good performance of a student in text message and send that text message to entire class cell phones can have great impact on increasing student study behavior. In future automated web-based SMS application could be
71
