a comparison and evaluation of open source learning ...

ISBN: 972-99353-6-X © 2005 IADIS

A COMPARISON AND EVALUATION OF OPEN SOURCE LEARNING MANAGMENT SYSTEMS∗ Jamil Ahmad Itmazi PhD. Student Computer Science Department, Granada University, Spain

Miguel Gea Megías, Patricia Paderewski, F.L. Gutiérrez Computer Science Department ETSI, Granada University, Spain

ABSTRACT eLearning is a new tool introduced to enhance the traditional learning system; it is becoming an integral part of the learning tools used by every school, business, colleges, and universities. The core of eLearning solutions are LMS or CMS (learning/course management systems), which help the universities automating the managing of learning events. Some of LMS are commercial Software, while others are free Open Source LMS (OS-LMS). The universities give more attention for OS-LMS packages, which they are becoming interested part of the eLearning landscape and they are competing with packages of commercial Software. Studying and analyzing LMS is a complicated task, and the few serious studies to evaluate and compare the LMS were not sufficient, especially the OS-LMS, which is given less attention in those studies. One goal of this article is to emphasize the necessity of more analysis and studies of LMS which involve OS-LMS. In this article, we provide a brief introduction to LMS, listing some of them, choosing two of OS-LMS (Moodle and Ilias) to study and evaluate then we start the comparison and evaluation to every package with introduction, features, different views and weaknesses. Our discussion will cover the basic information about every package: its name, link, source, version, structure, and explain its features. KEYWORDS LMS, Moodle, Ilias, eLearning.

1. INTRODUCTION The learning process needs mechanism to present the knowledge (from different resources), interact with it and share it with others. In this context, eLearning is becoming an important tool to support the learning system to achieve its goals. eLearning (as an ICT supported education) became hot topic in the 1990's after the spread of the Internet. Although it has a relative short history, it is becoming an important part of the learning system in our society. The majority of the universities adopted eLearning within its learning system. The eLearning is “the use of new multimedia technologies and the Internet to improve the quality of learning by facilitating access to resources and services as well as remote exchanges and collaboration", [Pla01]. The eLearning is based on special purpose information system called LMS- Learning Management System, and it is been widely used on universities. However, the importance of eLearning in the university’s environment does not mean that it will replace the traditional Learning mode. In other words, “eLearning is not intended to replace face-to-face classroom training but it can be used to enhance the traditional training”, [APO03].

∗

This research is partially supported by a R+D project of the Spanish TIN 2004-08000-C03-02.

80

IADIS International Conference on Applied Computing 2005

2. LEARNING MANAGEMENT SYSTEMS 2.1 Background of LMS LMS is the main software of the eLearning solutions “that automates the administration of training events. All LM systems manage the log-in of registered users, manage course catalogs, track learner activities and results, and provide reports to management”, [BRAa]. An LMS may include additional functions”, Some experts use additional terms to LMS, e.g. Portal Learning, which some experts consider it as another name of LMS; while others consider it as broader in functionality than an LMS, [ELe01]. Other experts use Learning Content Management Systems, which is a combination of typical LMS features with the additional ability to implement a standards-based learning object structure, incorporating the storage, controls and methodology, [ELe01]. The key functions of a LMS are [ANT02]: • Manage courses and programs. LMS • Provide and administer course registration. • Track student registration, access and progress. elearning solutions • Manage learning administration and reporting. • Enable financial tracking and control of learning. Figure 1. The LMS within e-learning • Provide course scheduling and administration. The current software market of LMS packages are “probably has at least 200 products already”, [Wil02]. The LMS are increasing very fast; The Brandon Hall report (www.brandon-hall.com) reviewed 27 LMS in 1997-1998, 59 LMS in 2000 and 70 LMS in 2002. CUE (2003) state that, by 2003, “there are now almost 200 providers, but the market is expected to consolidate”, [BRAb], and the last Brandon-Hall resource estimates that by 2003, the LMS sector will represent about 10% of entire eLearning market. Some of LMS are commercial Software, e.g. WebCT , eCollege and LearningSpace while others are Open-Source, e.g. Moodle, , ILIAS, and Claroline .

2.2 Open Source LMS Open source software (OSS) is becoming an interesting sector of the entire software landscape; it spread out and in competition with other proprietary (commercial) software. In short, OSS is a software that you can freely: access, use, share, modify, and redistribute the Source Code, [Ope]. The OS-LMS is very suit to the universities learning sector due to: - OSS is a good solution to control their software and format it upon their need. - The cost of using licence is almost nothing. - OSS licence permits any change, modify and improvement of the LMS´s without any pay. - Some big organizations adopted OSS, e.g. NASA's Center switched from Oracle to MySQL.

2.3 List of LMS: According to [BRAb] there are now almost 200 LMS. Table “1” lists the more recent and powerful LMS/CWS (Course website software), according to Google@. Table 1. List of CWS [Goo], accessed on 1/3/2004. • BSCW - http://bscw.gmd.de/ • Fle3 - http://fle3.uiah.fi • Blackboard - www.blackboard.com/ • WebCT - www.webct.com/ • Moodle - http://moodle.org/ • eCollege.com - www.ecollege.com • TopClass - www.wbtsystems.com/ • ARIADNE Project-www.ariadne-eu.org

• Open Knowledge Initiative-http://web.mit.edu/oki/ • Manhattan Virtual Classroom-http://manhattan.sourceforge.net • Generation21 Learning Systems - www.gen21.com • Convene Learning Platform - www.convene.com • ATutor - www.atutor.ca/ • Jones e-education - www.jonesadvisorygroup.com/standard. • Knowledge Environment - http://kewl.uwc.ac.za • Interact - http://cce-interact.sourceforge.net/

81

ISBN: 972-99353-6-X © 2005 IADIS

• IBT-Server - www.time4you.de • Claroline - www.claroline.net/ • CyberProf - www.howhy.com/home/ • OLAT - www.olat-zentrum.unizh.ch/ • dotLRN - http://dotlrn.org • IntraKal - www.anlon.com/

• Virtual-U - www.vlei.com • ILIAS - www.ilias.uni-koeln.de/ios/index-e.html • Lotus LearningSpace-www.lotus.com/home.nsf/welcome/ • Knowledge Forum - www.knowledgeforum.com/ • Janison Toolbox - www.janison.com.au/janison/default.asp • Online Instructor Suite-www.onlinecoursetools.com/products.asp

Also there is a list of OS-LMS in link http://richtech.ca/, [Les] and in [Com03].

2.4 Why Moodle and Ilias? We choose two famous OS-LMS; Moodle and Ilias for studying and analyzing, due to the flowing factors: 1- Ilias and Moodle are OSS. 2- Some recommendations about them; where according to [Com03:2; CUE03] ILIAS and Moodle are in the top three candidates, the Austrian Ministry of Education [Kri03], Fraunhofer Institute [Ber02] recommended using Ilias and [Kam03; Gob04:13; Yor03; Rey03] somehow recommended Moodle. 3- The credit (reputation) of both systems are high, where 1026 sites from 75 countries are using Moodle, while there are 115 sites from 18 countries are using Ilias (1) . 4- Moodle has been translated to 30 languages and Ilias to 16 languages, while some commercial software, like WebCT has been translated to 14 languages and BlackBoard to 11 languages (2).

3. THE ANALYSIS [Edu] site is listing 68 LMS, and performing an on-line comparison between those LMS upon 42 Tools (features and characters); we extract the results of Ilias and Moodle which are shown in Table “2”. The “*” symbol means that the tool is exist; otherwise, the tool is absent. Table 2. General comparison of the characters of ILIAS 2.3.8 and Moodle 1.1 [Edu] Tools 1. Discussion Forums 2. File Exchange 3. Internal Email 4. Online Journal/Notes 5. Real-time Chat 6. Video Services 7. Whiteboard 8. Bookmarks 9. Calendar/Progress Review 10. Orientation/Help 11. Searching Within Course 12. Work Offline/Synchronize 13. Groupwork 14. Self-assessment 15. Student Community Building 16. Student Portfolios 17. Authentication 18. Course Authorization 19. Hosted Services 20. Registration Integration 21. Automated Testing and Scoring

ILIAS * * * *

* * * * * * * * * *

*

Moodle * * * *

* * *

* * * * * * *

Tools 22. Course Management 23. Instructor Helpdesk 24. Online Grading Tools 25. Student Tracking 26. Accessibility Compliance 27. Content Sharing/Reuse 28. Course Templates 29. Curriculum Management 30. Customized Look and Feel 31. Instructional Design Tools 32. Instr. Standards Compliance 33. Client Browser Required 34. Database Requirements 35. Server Software 36. Unix Server 37. Windows Server 38. Company Profile 39. Costs 40. Open Source 41. Software Version 42. Optional Extras

ILIAS *

Moodle * * * * *

*

*

* *

* * * * * * * * * * * * *

* * * * * * * *

1 Ilias (www.ilias.uni-koeln.de/ios/info-e.html), Moodle (http://moodle.org/sites/), on 15.4.2004 2 Moodle (http://moodle.org/download/lang/-27/3/2004), Ilias (www.homer.ilias.uni-koeln.de/iliasdoc/doc/html/8_4.html), WebCT (webct.com/intl/viewpage?name=intl_download_plugin) and BlackBoard (blackboard.com/worldwide/gb/en/ml.htm) on 4.4.2004

82

IADIS International Conference on Applied Computing 2005

3.1 Moodle 3.1.1 Introduction Name: Moodle, Version: 1.1.1, Last release: 10th Sep.2003, Link: http://moodle.org/. Source: Moodle.com© company by the developer Martin Dougiamas. License: is available as OSS GNU General Public License. Moodle (Modular Object-Oriented Dynamic Learning Environment) is one of a LMS (CMS) for producing Internet-based course Web sites. It has been designed to support modern pedagogies based on social constructionist theory, and includes activity modules such as forums, resources, journals, quizzes…etc. It is runs without modification on UNIX, Linux, Windows, and support some databases (e.g. MySQL).

Figure 2. The main screen of Moodle

3.1.2 Features Moodle is an active and evolving product. This page lists some of features which contains [Moo]: Promotes social constructionist pedagogy (collaboration, activities, critical reflection, etc). 1- Suitable for 100% online classes as well as supplementing face-to-face learning. 2- Simple, lightweight, efficient, compatible, low-tech browser interface. 3- Easy to install on almost any platform that supports PHP. Requires only one database. 4- Course listing shows descriptions for every course on the server, including accessibility to guests. 5- Courses can be categorised (allows a choice of 3 course formats) and can be searched. 6- Emphasis on strong security throughout. Forms are all checked, data validated, cookies encrypted etc. 7- Most text entry areas can be edited using an embedded WYSIWYG HTML editor. [Jen04] mention those additional characters: 8- Moodle is based on PHP and MYSQL (both are OSS). 9- Easily upload course resource, or link to other websites, and will track.

3.1.3 Views of Usability The Current User Community is very wide and it is highly credited; there are 1026 sites from 75 countries, using Moodle, these organizations are actively participating in making improvements to this system, also Moodle have a high popularity, because it is translated to 30 languages (one version for every language). The Evaluation of Moodle could be seen from different perspectives: - User: we have tested both approaches: the student and the teacher approaches, it was noticeable that this system is easy to be used where only little training is needed to use it effectively. - Programmer and developer: it is very interesting to them, because Moodle is open source, and so it is free to re-programming, modifying…etc. Also it is programmed in PHP, support MySql and Apache; all are open, active and power. It can be install and run without modification on UNIX, Linux and Windows. - Organization and administration: Moodle is OSS; therefore, the Organization can customise it to their local needs, and can develop it without any cost of licenses. Also there are more than 1000 organization are using, developing and sharing it. Also they can be sure about the security {the source code is reviewable!}.

83

ISBN: 972-99353-6-X © 2005 IADIS

3.1.4 Weaknesses According to table “2”, it is clear that Moodle has 34 capabilities out of 42. The following table “3” lists some of missing features. Table 3. Features absent from Moodle 1.1 • Calendar features - there are plans underway to integrate this feature for version 1.3. • Bookmark – this tool manages creates, saves, shares, displays and updates links of Internets, it allows students

to return to important sites easily within their Groups / Classes or outside their course on the web. • Work Offline - it enables students to work offline. Sometimes students download course content and they access

content on a CD-ROM. A course placeholder automatically returns to the location in their course where they were working the last time they logged off [Jen04]. • Groupwork – a tool that organizes a set of student within group, it provide services to facilities achieve specific tasks or projects and support cooperative learning. The released 1.2.1 supports basically this Feature. • Student Community Building- a tool related to Groupware that allows students to establish teams (groups or clubs) to encourage and support the growth of student collaborative, studying, friendships and partnerships. • Recommendation systems - tools provide personalized and related suggestions about items that students will find interesting, which can intelligently obtain information from profiles of: teachers, students, Courses...etc • Standardization Tools (SCROM standard) - very important tools which allow knowledge to be publish, distribute, share, tracking. Moodle will have basic support for SCORM in their upcoming version 2.0 • Instructional design tools/Course templates- are tools that help instructors to create the initial structure for an online course and to create learning sequences. Moodle –somehow- support those tools weakly. • Private Email (and messaging) - This tool may be allow users to send / receive email to/from outside sources and Notifies users of new mail, announcements, discussions, quizzes, content, grades and assignments. • Content Sharing/Reuse – these tools enable specific content created for one course to be conveniently shared with another instructor teaching a different course perhaps even at a different institution. [Edu]. • Curriculum Management- provides students with customized activities based on prerequisites, prior work, or results of testing. It similar to the tools used in student services as part of providing academic advising, [Edu]. • Video Services - enable video conferencing and enable instructors to run stream video from within the system. • Whiteboard Tools- tools used by instructors and learners in synchronous modes (virtual classroom).

3.2 Ilias 3.2.1 Introduction Name: Ilias, Version: 2.4.4, Last release: 28th Feb.2004, Link: www.ilias.uni-koeln.de. Source: It was part of the VIRTUS project in the University of Cologne, and is now also worked on by the Sal. License: GNU. ILIAS (Integrated Learning, Information and cooperAtion System) is a web-based LMS; it consists of tools for learning, authoring, information access and co-operative work, thus presenting an integrated environment for learning and teaching on the Internet. ILIAS authors can create entire courses within a team and publish them on the web. Students can create groups to work through learning material and communicate with each other. ILIAS is OSS, developed using PHP, MySQL and Apache to work under UNIX and Linux.

Figure 3. Example of Ilias Personal desktop screen.

84

IADIS International Conference on Applied Computing 2005

3.2.2 Features In general, we can list the main features of ILIAS: • Personal desktop for each user. • Learning environment with personal annotations, tests, glossary, print function, search engine. • Communication features like news system and discussion forums. • Group system to organise group members and resources. • Authoring environment (Editor) to create courses. • Context-sensitive help system for learners and authors. • System administration interface. Users play different roles such as system administrator, authors, tutors and students. The system administrator puts the actors together by creating groups and assigning the access rights, privileges and resources needed: mailboxes, forums, discussion groups…etc. [Bas03].

3.2.3 Views of Usability There are 115 sites from 18 countries using Ilias which give it good credit and it gains a good evaluation in some recent studies. Also, Ilias has a good popularity, because it was translated to 16 languages. The Evaluation of Ilias could be seen from different perspectives: - User: Moodle is easier than Ilias, according to [Com03], even though, Ilias somehow is easy to use, I used it as a member of the Demo version on their site and it needs a little training to use it effectively. - Programmer and developer: Ilias is very interesting to them, because it is OSS, so that it is free to reprogramming, modifying…etc. Also it is programmed in PHP, support MySql and host at server Apache; all of them are open, active, power. Upon Ilias’s site, “ILIAS is developed for Linux and Sun Solaris. With small changes it can also be used under Windows or Mac” (www.ilias.uni-koeln.de/ios/source-e.html), but it did not works correctly under Windows XP when I tested it. Ilias somehow not easy to install, it need many software with the latest versions, and long procedure is needed to install it. - Organization and administration: Ilias is OSS; therefore, the Organization can customize it to their local needs and can develop it without any cost for licenses. This system also is used by more than 115 organization, they always develop it and share these developments. Also they can be sure about the security because the source code is reviewable. According to [CUE03] “There is no module for competency assessment. The system is set up with no clear links to reports in the administrative view”.

3.2.4 Weaknesses According to table “2”, Ilias has 26 capabilities out of 42. The table “4” lists some of missing features. Table 4. Features absent from ILIAS 2.X • • • • • • • • • • • • • • • •

Recommendation system – See table “3”. Standardization Tools (SCROM)-See table “3”. ILIAS is planning to support it in Version 3.0 Instructional design tools/Course templates- See table “3”. Windows OS– It does not work correctly under Windows. Student Portfolios – tools allow students to use area related to groupware to display their work, display their personal information and may be they can create a personal home page. Hosted Service- these tool help to work without provide any hardware, but Ilias do not provide host CMS Content Sharing/Reuse - See table “3”. Curriculum Management- See table “3”. Registration Integration - tools are used to add/drop students from an online course. Administrators and/or instructors use registration tools while students can use them when self-registration is available. Course Management-it allows instructors to control the progression of an online class through the course material. Online Grading Tools - help instructors mark and provide feedback on the student work. Student Tracking - is the ability to track the usage of course materials by students, and to perform additional analysis and reporting both of total and individual usage. File Exchange-allows learners to upload files from their local computers and shares these files with others. Video Services - See table “3”. Whiteboard Tools- See table “3”. Real-time Chat – a conversation between people over the online Internet (the same time).

85

ISBN: 972-99353-6-X © 2005 IADIS

4. CONCLUSION LMS -as the core of eLearning- is very important to the university, furthermore, the OS-LMS seem to be a good solution due to many reasons. From our comparison and evaluation process of the two OS-LMS (Moodle and Ilias), we found that both systems have good features and good usability issues. We also highlighted some weakness of them, those weaknesses are a motivation to the hundreds of programmers and developers –a round the word- to fix them and add the necessary features.

ACKNOWLEDGEMENT I would like to thanks the agency of (AECI), which give me the Scholarship of my PhD study and to the Excellence Doctoral Program of Lenguajes y Sistemas Informáticos Dpt. of the Granada University (http://lsi.ugr.es/doctorado)

REFERENCES [ANT02] ANTA, 2002. Flexible Learning Fellowship Change Management Plan, Strategy 2001, Nancye Stanelis (South

Australia), Aug.2002.Online: www.flexibleLearning.net.au/fellowships/ [APO03] ©APO, 2003. Multimedia and eLearning: a new direction for productivity promotion and enhancement. Asian

Productivity Organization. Tokyo. ©APO. 2003. ISBN: 92-833-2344-0. [Bas03] Bastanzo, A. et el., 2003. Open Source Software on Web-Based Education, exposed by GDED in the IV Foro

Internacional de Software Libre desarrollado en Porto Alegre (Brazil). [Gob04] Goba, S. et el., 2004. Online Course Material Interoperability and Tutorial Module for Moodle. Technical Report

CS04-24-00, University of Cape Town. http://pubs.cs.uct.ac.za/archive/00000179/ [Ber02] Berger, K., 2002. Recommendations for the choice of an eLearning platform within the scope of the

INNOPUNKT program, new learning made in Brandenburg, ©Fraunhofer IPK 2002. Germany. [BRAa] BRANDON-HALL.COM, ELearning Glossary of Terms. www.brandonhall.com/public/pdfs/glossary.pdf [BRAb] Brandon-hall.com, 2001. Learning Management and Knowledge Management: Is the Holy Grail of Integration

Close at Hand? California: brandon-hall.com, www.brandon-hall.com. [Com03] The Commonwealth of Learning and 3waynet Inc. 2003. COL LMS Open Source, Jun.2003.www.col.org [CUE03] CUE, 2003. Learning Management Systems for the Rest of Us, White Paper Open Source Learning Management

Systems. A research report, Corporate University Enterprise, Inc, 1.May.2003 [Edu] EduTools, WCET - the Western Cooperative for Educational Telecommunications. www.edutools.info [ELe01] ELearnity, 2001. Learning management and portal, Version 1.0, ©2001 eLearnity Ltd. Online: www.sharpstream-training.com/Knowledge/ELearning/LMPortals.pdf [Goo] Google, (Reference>Education>Instructional Technology>Higher Education>Course Website Software) [Jen04] Jensen, J., 2004. White Paper: Is This the Best Way to Develop, Deliver and Manage Online Training, Feb.2004.www.americastrainingpartners.com/projects/ATP/addtl_files/best_way_html/best_way.htm [Kam03] Kameron, S., 2003. Online classrooms for FREE?!, A Review of Free Online Learning Management Systems (LMS), TESL-EJ (TESL Electronic Journal), Vol.7. No.2, Sep.2003. [Kri03] Kristöfl, R. et al, 2003. Evaluation von Learning Management Systeme, (BMBWK), bildung.at. www.virtual-learning.at/publikationen/Evaluation-LMS-Schulen.pdf [Les] Les Richardson, Open Source Software Development. //richtech.ca. [Ope] OpenSource, 2004. Open Source Initiative ©. Online: http://opensource.org, [accessed on 1/3/2004]. [Pla01] The eLearning Action Plan., 2001. The eLearning Action Plan: Designing tomorrow’s education, Communication from the Commission to the Council and the European Parliament, COM, 172 [Rey03] Reynolds, R., 2003. Open Source Courseware--Evaluation and Rating, XPLANA under Category: Emerging Technologies. www.xplana.com/whitepapers/archives/Open_Source_Courseware. [Wil02] Wilson, D., 2002. The future of learning management, learning technologies© Principal Media Ltd. Online: www.learningtechnologies.co.uk/magazine/article_full.cfm?articleid=19&issueid=5§ion=1 [Yor03] Yordanova, L. et al, 2003. Development of a Web – based Course on Informatics via Open-source Software Package MOODLE, Proceedings of CompSysTech’2003.

86