Editorial Manager(tm) for Educational Technology Research & Development Manuscript Draft Manuscript Number: Title: Selection of LMS Based in a Pedagogical Approach Article Type: International Review Corresponding Author: Dr Jesus Salinas, Ph.D. Corresponding Author's Institution: University of Balearic Islands First Author: Jesus Salinas, Ph.D. Order of Authors: Jesus Salinas, Ph.D.; Jesus Salinas, PhD
Manuscript (must not contain author information) Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Selection of LMS Based in a Pedagogical Approach Jesús Salinas University of Balearic Islands Campus UIB. Edif.. Guillem Cifre. Ctra. Valldemossa km. 7,5. 07122 Palma. Baleares (Spain) Phone number:+34 971 173074 Fax:+34 971 173190
[email protected]
Barbara de Benito University of Balearic Islands Campus UIB. Edif.. Guillem Cifre. Ctra. Valldemossa km. 7,5. 07122 Palma. Baleares (Spain) Phone number:+34 971 172569 Fax:+34 971 173190
[email protected]
This work was presented at 2008 AECT International Convention Abstract The implementation of Virtual learning environments depends on decisions related to the educational design and on other decisions related to technology itself and the selection of the most appropriate system and tools. This selection requires thinking about instructional design principles, knowledge of the existing technology and their educational possibilities. Sometimes when an institution plans to implement a virtual learning environment, the person responsible does not have knowledge of both the pedagogical and technical areas. Usually the selection of LMS is based principally upon technical criteria. Because we believe that technology must be used to facilitate the teaching/learning process, we designed a tool made for making decision concerning the technology required for implementing learning strategies, keeping in mind three points of view: pedagogical, technical and organizational. For this, we tried to determine the indicators and criteria users and organizations
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
should keep in mind when selecting the most appropriate technologies for the high quality elearning experience.
Keywords LMS, selection tool, virtual learning environment, e-learning Abbreviations LMS Learning Management System ICT Information and Communication Technologies CMC Computer Mediated Communication LVSV Virtual Learning Management Systems Validation Laboratory
Introduction Nowadays through Internet, teachers find many tools at their disposition for the teaching - learning process. With the proliferation of these tools, the problem is more in determining which tools are the most appropriate for obtaining specific educative goals. Following the same line of thinking as McGreal, Gram & Marks (1998), Spector, Wasson & Davidson (1999), Cook (2000), we believe that taking into account the instructive design principles based on networks and the knowledge of both the technical characteristics of the tools and the pedagogical aspects, would contribute to selecting the most appropriate software for the distribution of educative experiences based on networks. However, selecting a Learning Management System is very complex due to the great number of tools developed specifically for the distribution and management of virtual environments aimed at learning, and also due to the continous advances made and the upgrading of new tools which this field is generating. The dynamic use and the increase of the ICT in learning, the expansion which is now happening with the results of a multitude of experiments, the emerging models of teaching, the immense quantity of tools and applications developed, the lack of knowledge about the educative possibilities of these, the disregard of functionalities and technical possibilities and the lack of criteria for evaluating and selecting tools, has led us to this research.
Description of the Research Process The aim of this research was the design and validation of a tool helping the selection of a LMS for a Virtual Learning Environment. Because the choice of tools cannot be made only by using one criteria, being there so many elements to take into account in this environment, we have based ourselves on the multi-criteria theory as our base method for designing the instrument. This study was conducted as a design research (van den Akker, 1999; Reigeluth & Frick, 1999; Reeves, 2000), identifying the indicators and criteria which need to be taken into account when selecting tools and designing the instrument to help in the making decisions. Phases of the research: 1.- The first phase consisted in the identification of the criteria quality which defines virtual teaching- learning environments. It was done, on one hand, by reviewing literacy, experimenting LMS and doing research concerning tools and virtual learning environment implementation. On the other hand, it was necessary defining the indicators and attributes which needed to be taken into account in the selection of LMS. For this, we made a functional analysis of both: the technical possibilities of the tools and the relation to learning contexts. In relation to the functional analysis about technical possibilities, we made a meta-analysis (de Benito, 2006) of comparatives studies or reports about tools. This allowed us to know the technical and pedagogical possibilities of the tools, and we also made a questionnaire that was sent to professors with e-learning experience. For the functional analysis in relation to the learning context we used the questionnaire mentioned above, which was aimed to analyze the different learning contexts in virtual environments. (de Benito & Salinas, 2005; de Benito & Salinas, 2006, Salinas & de Benito, 2007) 2.- The second phase was the design, development and validation of the prototype. Once validated by experts the instrument consisted in 20 questions regarding to organizational, pedagogical,
2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
technological and economical aspects of a virtual learning environment (de Benito & Salinas, 2007).
Identification of Indicators and Criteria for Selection For identifying the indicators and criteria which need to be taken into account when selecting tools, we use the functional analysis of the technology in relation to the learning contexts. This functional analysis included: - Analysis of didactic techniques used by the teachers - Analysis of didactic techniques in relation with implemented teaching model - Analysis of didactic techniques in relation to the type of studies - Analysis of didactic uses of technology - Matrix of didactic functions of the tools - Functional map of tools made by the meta-analysis of comparatives studies or reports about platforms and by a questionnaire sent to professors with e-learning experience. (de Benito, 2006) Some of the results obtained from the functional analysis revealed which didactic functions the tools presented. Studying the different strategies implemented, allowed us to find out which tools were used more often in virtual environments, how they were used and also allowed us to make a matrix of the didactic functions of these tools.
Analysis of Didactic Techniques The didactic techniques in use by CMC have been widely analysed by various authors. The number of techniques in use is large and even more if we consider the variables which can be incorporated. So in order to be able to analyse the different didactic functions of these tools we decided to select and group together those techniques which we found were used by the greatest number of teachers. The techniques analysed are in graphic 1, where we can observe that the most often didactic techniques used were: availability of material and learning resources for the student (85%of teachers), followed by tutorials (79%) and the search and recovery of information (73%). The least often used techniques were those based on simulation, games and roll playing. 1 2 100
3
90
4 5 6
80 70 60
7
50 40
8 9 10
30 20 10
11
0 1
2
3
4
5
6
7
8
9
10
11
12
13
12 13
access to material search and recovery of information online presentations, conferences consulting experts demonstration debates and discussion groups simulation, games and roll playing social communicaction group activities case studies, problem solving working on projects/ web quest individual tutoring group tutoring
Graphic 1 : Percentages didactic techniques used. We also analysed the didactic techniques in relation with the teaching model, we observed that the techniques are almost the same independently of the teaching model. Though teaching distance model also gave importance to group tutoring. Table 1. Resume of the most used didactic techniques in relation with the teaching models Face-to-face Blendend At distance
3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
- Access to material 82%
- Individual tutoring 89%
- Search and recovery of information 74%
- Access to material 78%
- Individual tutoring 65%
- Debates and discussion groups/symposium, round table or panel 72%
- Access to material and individual tutoring (100%) - Search and recovery of information, debates and discussion groups, symposium, round table or panel (83%) - Group tutoring (78%)
Analysing the techniques in relation to the type of studies in which they were implemented (Health Sciences, Experimental Sciences, Social and Law Sciences, Technical and Humanities) we observed that access to material and the search and recovery of information were the mostly used techniques no matter the studied subject. The teachers using the greatest number and variety of techniques are those who study experimental sciences followed by those who study social and law sciences. Online presentations, conferences or consulting experts were used by approximately 50% of professors of experimental sciences, technical and humanities, whereas consulting experts were hardly used at all by those in the health science studies. Demonstration techniques were used very little in general (37% of all the professors who answered the questionnaire) although it is to be noticed that they were the least used in technical teaching. Debates, discussion groups etc. were the mostly used techniques; only 60% implemented them sometimes. As to the subjects in which they were most used, experimental sciences came first (80%) followed by technical studies (78%), health sciences (66%) and social sciences and humanities (60 and 50% respectively). Simulations were the generally least used though 60% of the experimental sciences professors said they used them. Informal exchanges although it cannot be called a proper didactic technique, we decided to include it since50% of professors say they use it. Group activities, such as case studies, problem solving, etc., are used by more than 50% of professors in social and law studies and experimental sciences. Work on projects is a technique which has more followers every day, however it is still one of the least used from what came out from the questionnaire. The subjects in which it is seemed the mostly applied were social and law sciences, followed by health sciences. This is the least used technique by experimental sciences professors say.
Didactic Uses of Technology The tool’s use is another aspect of the analysis of didactic techniques. In order to analyse these, we divided the tools into three groups : communication, (mail, chat, forum, instant messaging, news, video conferences), organisational (job assignment, calendar, student groups, authoring tools, brainstorming, voting) and those related to access of information (shared applications and files, links to URL, FAQ, conceptual maps, shared navigating, electronic blackboard). Keeping in mind the above classification, we made graphics 2, 3 and 4 which show the tools used by professors in each didactic technique and its percentage of use. We observed that the tools which facilitated communication were the most generally used. Graphic 2 show that mail and forums are way ahead from the others and are used in almost all didactic techniques. Mail, as is logical, is used for individual tutoring, access to information and case studies. The technique in which they are least used is for online presentations, conferences, debates and discussions in which the forums tool is mostly used. Another techniques used in which a considerable percentage of professors use forums is in the creation of social spaces for informal exchanges, and even if in a menor degree, in proposing group activities, simulations, games or roll playing, as well as in accessing information, case studies and problem solving. The next most used tool is the chat. This is most used in consulting experts and as a social space. Video conferencing is mostly used for online presentations and conferences; for demonstrations and for consulting experts. News is a less used tool, it is mostly used in accessing material (30% of professors), more then for the search and recovery of information, online presentations, consulting experts and demonstrations (used by 15 to 20% of professors). In last place, instant messaging is the least used tool and the one less avalaible in platforms. Those who say they use it, is for working on project technique, as well as in individual tutoring and informal exchanges.
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1 mail
2
chat
3
4
forum
5
6
7
8
instant messaging
9
10
news
11
12
13
video conferences
Graphic 2: Percentage of professors using communication tools in didactic techniques Concerning the tools which we grouped in the organisational category, the most used are: task assigning, calendars and student groups. Job assignment is mainly used in the techniques to access material and in work projects, followed by search and recovery of information and case studies. The Calendar is used in the majority of techniques, especially in accessing material. Student groups are mainly for group activity proposals, work projects, simulations and roll playing. The least used, leaving aside individual tutoring, is consulting experts. The authoring tool is used for demonstrations, access to material and search and recovery of information. One should notice that it is not used in the work project techniques or group activities. Brainstorming and voting are hardly used at all. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1
job assignment
2
3
calendar
4
5
6
7
student groups
8
9
10
authoring tools
11
12
13
brainstorming
voting
Graphic 3 : Percentage of professors who use the various organisational tools Last we analysed the tools which facilitate access and interchange of information. Of the 3 groups of tools, this is the group which is generally the least used. Those who most use them, with some differences, are links to Internet, followed by shared files. Links are mostly used for access to material and demonstrations. Following they are used, by 50% of professors for search and recovery of information and work projects. The technique in which it is least used, leaving aside tutoring, is for the social space. Shared files are used in work projects, access to material and proposals of group activities, less for consulting experts. Shared applications are the next most used tool, mainly for access to material and simulations. Conceptual maps are used only for access to material. For last, FAQS, shared navigation and the electronic blackboard are hardly used at all with the exception of the use of the shared blackboard for online presentations, conferences and the shared navigation which has been used by a greater number of professors for simulations.
5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1
2
3
4
5
6
7
8
9
10
11
12
13
shared applications
shared files
links
FAQs
conceptual maps
shared navigating
electronic blackboard
Graphic 4: Percentage of professors using tools to facilitate access and interchange of Information in didactic techniques.
Matrix of Didactic Functions of the Tools The results of the study allowed us to make a framework for didactic use of the tools in function of how often they were used in the various didactic techniques. 1
access to material
2
search and recovery of information
3
online presentations, conferences
4
consulting experts
-- not used
5
demonstration
*
6
debates and discussion groups
7
simulation, games and roll playing
8
a place for informal exchanges
9
group activities
10
case studies, problem solving
11
working on projects/ web quest
12
individual tutoring
13
group tutoring
used by < 15%
** used by 15-25% *** used by 25-50% **** used by 50-75%
Table 2. Matrix of Didactic Functions of the Tools
6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Design, Development and Validation of the Instrument As mentioned before, the tool design was based on the multi-criteria theory. In our case, the criteria were divided in four dimensions: organizational, pedagogical, technological and economical. Graphic 5 illustrate the process followed to create the instrument.
7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Functional analysis
Criteria selection
Meta-analysis of comparatives estudies
Functional analysis in learning context
Functional map
Decision matrix
Determining criteria and attributes
Building elenction group
Decisión matrix
Prototype elaboration
Item selection and elaboration of flux diagram
Validationn
Validation
By experts
Formative evaluation
Final version of the instrument
Table 3 shows the different criteria organized by organizational, pedagogical, technological and economical dimensions. Table 3. Criteria used in the instrument TECHNICAL AND HUMAN RESOURCES COST (COMMERCIAL OR FREE PLATFORM) HOSTING TYPE OF INSTITUTION (HIGHER EDUCATION, NETWORK UNIVERSITIES, VIRTUAL CAMPUSES, ETC.) EXPERIENCE FROM THE INSTITUTION BEHALF IN ELEARNING USE OF PLATFORMS INSTITUTIONAL PROJECT KNOWLEDGE OF THE TEACHERS IN THE USE OF VIRTUAL ENVIRONMENTS. SUPPORT FOR TEACHER’S AND STUDENT’S NEEDS GEOGRAPHICAL AND TIMETABLE NUMBER OF STUDENTS STUDENT MANAGEMENT EDUCATIONAL METHODOLOGY (DISTANCE, BLENDED,…) METHODS AND STRATEGIES PEDAGOGICAL RESOURCES EVALUATION PROCESS RE-USE OF PEDAGOGICAL RESOURCES IMPORT/EXPORT
economical economical economical organizational organizational organizational organizational organizational organizational organizational organizational organizational/ technological pedagogical pedagogical pedagogical pedagogical technological technological
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Each criteria has several attributes associated which are interrelationated (see graphic 6 for the relation existing between them)
Graphic 6. Relationship between key criteria Once defined the criteria with the attributes the instrument consisted in20 connected questions. Each answer has different options which are associated to a particular attribute. With the answers given a report is generated with the most suitable technological configuration in relation with the needs of the virtual learning environment. This report has an online version, a print version and an exportable file format so that can be modified by the user and where he can add his preferences.
Conclusion The use of the design and development methodology allowed us to design an instrument for LMS selection based on organizational, pedagogical, technological and economical criteria. These results, although we are aware that generalising these requires caution, provide some first indications of how the LMS are being introduced in higher education in Spain. The caution is due to that while the technological aspects can be considered standardized, educational and organizational aspects are strongly conditioned by the cultural and institutional context of each country. Nevertheless some reflections on the results can be applied (or examined) to similar situations in other regions. To contribute in the technologies development used in education, it is necessary to continue doing research projects like this, aimed to find answers to the design and the implementation of innovative methodological strategies in accordance with the possibilities which TIC offer us. This will allow defining new learning models based on virtual learning environments and will contribute to the advances on tools which give support to the emerging models. Thanks to the goals achieved in this research the future advances aim to validate the instrument by real users and also to connect the results obtained through the questionnaire with the data base tools of LVSV (Virtual Learning Management Systems Laboratory) of the University of Balearic Islands. References Cook, K.C. (2000): Online professional communication: Pedagogy, instrucctional design, and student preference in Internet-based distance education. Business Communication Quarterly 63 (2) 106-110 de Benito, B (2006): Diseño y validación de un instrumento de selección de herramientas para entornos virtuales basado en la toma de decisiones multicriterio. Tesis doctoral. Universitat Illes Balears.
9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
de Benito , B. & Salinas, J. (2005). Situaciones didácticas en los entornos virtuales de enseñanzaaprendizaje (EVEA) en la enseñanza superior: elaboración de un instrumento de análisis. Paper presented in EDUTEC05. Formación del profesorado y Nuevas Tecnologías. Santo Domingo, República Dominica. de Benito , B. & Salinas, J., (2006). Análisis de situaciones didácticas en los entornos virtuales de enseñanza-aprendizaje (EVEA) en la enseñanza superior. Paper presented in EDUTEC06. La educación en entornos virtuales: calidad y efectividad en el e-learning. Tarragona, España. de Benito, B. & Salinas, J. (2007). A tool for Selection of LMS Based in a Systems Padagogigal Approach. Poster. IADIS International Conference e-Learning 2007 Proceedings. IADIS, Lisbon (PORTUGAL) Mcgreal, R. et alt. (1998). A Survey of New Media Development and Delivery Software for Internet-Based Learning. [http://telecampus.com/developers/environment/ index.html] .[acces april 1998] Reeves, T. C. (2000). Enhancing the Worth of Instructional Technology Research through “Design Experiments” and Other Development Research Strategies. International Perspectives on Instructional Technology Research for the 21st Century Symposium. New Orleans, LA, USA. Reigeluth, C. Y Frick, T. (1999): Formative research: A methology for Creating and Improving Design Theories. En Reigeluth, C. (Ed.): Instructional-Design Teheories and Models. A New Paradigm of Instructional Theory (Vol. II). Lawrence Erlbaum. Mahwah (NJ), USA. 633-652. Salinas, J. & De Benito, B. (2007). Analysis of the Didactic Functions of Technology in Online Formation Systems. Paper presented in IADIS International Conference e-Learning 2007 Proceedings. IADIS, Lisbon (PORTUGAL) Spector,M.; Wasson,B. & Davidoson,P.(1999): Designing collaborative distance learning environment for complex domains. In Collis,B. & Oliver,R. (Eds.): Proceedings of Ed-Media 1999: World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Seattle, Washington, June 19-24, Charlottesville, VA: Association for the Advancement of Computing in Education, 323-329 van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, N. Nieveen, R. M. Branch, K. L. Gustafson, & T. Plomp, (Eds.), Design methodology and developmental research in education and training (pp. 1-14). The Netherlands: Kluwer Academic Publishers.
10
