for LOs evaluation during their management for e-learning systems. ... and technical requisites to reuse them, according to this we suggest the next evalua-.
Applying CIPP Model for Learning-Object Management Erla M. Morales Morgado1, Francisco J. García Peñalvo2, Carlos Muñoz Martín1, and Miguel Ángel Conde Gonzalez1 1
2
GRIAL Research Group - University of Salamanca Computer Science Department / Science Education Research Institute / GRIAL Research Group - University of Salamanca {erlamorales,fgarcia,carlosmm,mconde}@usal.es
Abstract. Although knowledge management process needs to receive some evaluation in order to determine their suitable functionality. There is not a clear definition about the stages where LOs need to be evaluated and the specific metrics to continuously promote their quality. This paper presents a proposal for LOs evaluation during their management for e-learning systems. To achieve this, we suggest specific steps for LOs design, implementation and evaluation into the four stages proposed by CIPP model (Context, Input, Process, Product). Keywords: Metadata, Learning objects, Knowledge Management.
1 Introduction In pedagogical area, there is an old model CIPP (Context, Input, Process and Product) attuned to evaluate the teaching/learning process into four stages. The stages mentioned above can be applied for LOs management in order to promote their quality during their life cycle. To achieve this, we suggest specific things to take into consideration for each evaluation stage. Section 1 explains context evaluation where we suggest key issues to design LOs taking into account the specific educational situation where they are going to be reused. Section 2 explains input evaluation, attuned to ensure LOs elements are enough to achieve the specific leaning goal. On this basis, we propose a normalization process where it is possible to design LOs taking into account pedagogical and technical issues. In order to ensure LOs quality design, we suggest an instrument for expert evaluation “HEODAR”, which contains rubrics and specific criteria for pedagogical and technical issues. Section 3 explains how we think to promote process and product evaluation, to achieve this we take into account the students’ interaction with LOs and the teachers LOs evaluation as a product using HEODAR, in this way they can be able to improve and reefed the LOs quality continually. Finally it presents our conclusions and plans for the next stages of our work (section 4). M.D. Lytras et al. (Eds.): WSKS 2010, Part I, CCIS 111, pp. 506–511, 2010. © Springer-Verlag Berlin Heidelberg 2010
Applying CIPP Model for Learning-Object Management
507
2 Learning Objects Context Evaluation The opportunity offered by the LOs to be reused, indicating a high probability that they are imported, that is, acquired from external sources. On the other hand, may be the case that the LO requested does not exist and it has created. In both cases to determine the LOs needed in a learning situation, teachers should be based on the learning context on which to select the objectives, content and educational activities. In case they are imported or created, they need to be suitable for the new context in which to reuse them. Williams [8] defines context evaluations that investigate the socio-political, organizational, and other contextual variables associated with the need for learning objects, courses, and support efforts. Taking into account these arguments, we suggest the following things to take into account for context LOs evaluation: Learning goals, contents, learners’ characteristics and didactic strategies. As shows Figure 1, we think it is necessary to normalize LO according to a specific model that aim to ensure their suitable characteristics as a LO and their pedagogical and technical issues [4].
Fig. 1. Context and Input Learning Objects Evaluation
508
E.M. Morales Morgado et al.
In this context, normalization means that although LO instructional design is currently a much-discussed topic, there are certain aspects that must be considered to ensure a quality LO instructional design [4]. The model promotes to consider the minimal required elements to support a LO as a reusable didactic source, where it is needed to establish all the context issues mentioned above. Once we have the LOs design it is needed to valuate if they have the pedagogical and technical requisites to reuse them, according to this we suggest the next evaluation stage: Input evaluation, as we are going to explain in the next section.
3 Learning Objects Input Evaluation CIPP model establish the input evaluation as a way to detect alternative necessities in the context evaluation. According to [8] input evaluations compare alternative inputs or means for meeting the needs identified in context evaluations, including but not limited to learning objects. On this basis, we suggest to apply an input evaluation directed to ensure the LOs have all the elements for their reuse and the didactic elements, which promotes a specific learning goal [2]. •
•
LOs expert evaluation: Nowadays, there are some proposals about LOs quality taking into account a instructional design [8] and their sequence [7]. On this basis, we designed an instrument HEODAR to gather qualitative and quantitative data about LOs [5], [6]. Criteria are based on Pedagogical and Technical Categories, the first one take into account pedagogical and didactic-curricular issues and the last one take into account interface and navigation design. Add suitable metadata: Sound LO management requires the incorporation of reliable metadata, but the viability of the only metadata schema currently regarded as a standard IEEE LOM [1] has been called into question because it uses vast quantities of ill-defined types of data, and some of its metadata categories do not make it clear what kind of information has to be added, thus further complicating the task of LO management [3].
4 Learning Objects Process and Product Evaluation Williams [8] states process evaluations that formatively assess the planning, design, development, and implementation of learning objects and associated efforts to use them, including attempts to adapt instruction based on individual differences as expressed in learner profiles, etc. As we show in this paper, we promote to evaluate LOs planning, design and development into the context and input evaluation taking into account specific metrics. On this basis, as shows Figure 2, in order to evaluate the LO process we suggest to made special emphasis on the students interaction together with the LO content and students/teacher relationship. In this way it is possible to take significant notes for LOs evaluation to improve their quality. As we said before we promotes the process evaluation to valuate the LOs implementation and collect information about the attitude of the students and comments. In
Applying CIPP Model for Learning-Object Management
509
this process teacher can realize if their students are understanding and learning about the specific contents. In order to obtain specific users data we suggest to make a product evaluation through HEODAR that can be use to collect quantitative and qualitative information.
Fig. 2. Process and Product Learning Objects Evaluation
In order to get a balanced LO quality value we suggest to average the final score from each one of the four instrument parts, in this way it is possible to add a specific value to LO metadata into 9.classification IEEE LOM metadata [4]. This category aims to add quantitative and qualitative values in order to search and manage LOs according to their quality.
5 Conclusions Williams rescues the CIPP approach to assess LOs promotes reuse because they are used in processes for a variety of contexts. Into the "Context" evaluation we had mentioned certain aspects to consider in determining whether the LOs to reuse adapting to the new educational situation. To ensure a proper LOs design, we suggest a normalization process where LOs was proposed to classify them according to criteria to different kind of goals and contents to facilitate their management. The classification of
510
E.M. Morales Morgado et al.
the LOs according to the cognitive level allows teachers to facilitate the LOs searching according to their educational goals, which indicate what the student is able to do and estimated the level of complexity. The "Content Type" classification help to better define the type of LO for reuse. The “Input evaluation”, ensures that the LOs met with the interoperability and reusability, and also that the contents of the LOs contain quality criteria. The HEODAR instrument assesses LOs before they are delivered to students seeking to ensure optimal quality of teaching, which will also consider technical issues for interface design. The LOs “Process” evaluation during and after the teaching process can be useful to discuss the initial assessment made by experts. It is suggested to promote students to express their satisfaction with the LOs gaining a qualitative assessment. The "Product" evaluation is the final LOs assessment, at the level of lesson course and, through tools to accurately know the best and worst rated. According to the result suggested, LOs must be classified with a numerical rank according to their quality through element "9.Classification". Through this classification teachers can search the LOs according to their quality and available through the item "9.3.Description" the virtues and possible shortcomings presented by the LO.
Acknowledgments The authors would like to acknowledge the support of the Lifelong Learning Programme of the European Union. Project Reference: 502461-LLP-1-2009-1-ESCOMENIUS-CMP. This project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Also, this work is partially supported by the Regional Ministry of Education of Junta de Castilla y León through project GR47.
References 1. IEEE LOM: IEEE 1484.12.1-2002 Standard for Learning Object Metadata, http://ltsc.ieee.org/wg12 2. Morales, E.M., García, F.J., Barrón, Á.: Key Issues for Learning Objects Evaluation. In: Proceedings of the 9th International Conference on Enterprise Information Systems (ICEIS 2007), vol. 4, pp. 149–154. INSTICC Press (2007) 3. Morales, E.M., García, F.J., Barrón, Á.: Quality Learning Objects Management: A proposal for e-learning Systems. In: 8th Int. Conf. on Enterprise Information Systems Artificial Intelligence and Decision Support Systems, pp. 312–315. INSTICC Press (2006) 4. Morales, E.M., García, F.J., Barrón, A.: Improving LO Quality through Instructional Design Based on an Ontological Model and Metadata. J.UCS. Journal of Universal Computer Science 13(7), 970–979 (2007) 5. Morales, E.M., García, F.J., Barrón, A.: An evaluation instrument for learning object quality and management. In: 10th Int. Conf. on Enterprise Information Systems, pp. 327–332. INSTICC Press (2008)
Applying CIPP Model for Learning-Object Management
511
6. Morales, E.M., Gómez, D., García, F.J.: HEODAR: Herramienta para la Evaluación de Objetos Didácticos de Aprendizaje Reutilizables. In: X Simp. Int. de Informática Educativa (2008) 7. Zapata, R.M.: Calidad en entornos virtuales de aprendizaje y secuenciación de Learning objects (LO). In: Actas del Virtual Campus 2006 (2006) 8. Williams, D.D.: Evaluation of learning objects and instruction using learning objects. In: Wiley, D.A. (ed.) The Instructional use of LOs (2000), http://reusability.org/read/chapters/williams.doc
