Evaluation of Free Authoring Tools for producing ...

4 downloads 54340 Views 832KB Size Report
Tutorial. Exercise and Practice. Applicative. Use of Multimidia & Internet. Development of Multimidia &. Internet ... commercial tools Adobe Flash, Adobe Dreamweaver, Adobe. Photoshop and Corel Draw. In Figure 1 you see the reference -.
> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR< against which the applications results of ATs assessed will be compared, henceforth referred to as the Reference - Learning Object (R-LO). The evaluation of the Authoring Tool is divided into three forms: (a) method for the selecting authoring tools, (b) Method of Assessment of the Authoring Tools and (c) Method for the Evaluation of the Generated Products (LOs). A. Method for Selecting Authoring Tools The first phase of the research begins with the selection of 14 ATs with the following aspects: (a) Open source or Freeware and (b) production of LOs in the SCORM standard. In this first step LOs were produced in the following format: (1) Containing Texts, (2) containing images, (3) Containing Animations (4) Containing Exercises. Then the ATs were evaluated based on the rating model structured on the combination of the researches [4, 5]. Table I presents the model proposed classification.

Category Non Interactive Goals

TABLE I MODEL DE CLASSIFICATION Subcategory Rated Itens Textual

Palin text Website Flash Applet Audio Video

Interactive Goals

Explorative

Conceptual Maps Transaction Graphs Programming

Evaluative

Blackboard Wiki Forum

Educacional Rating

B. Methodology for Evaluating Authoring Tools In order to evaluate the Authoring Tools in terms of usability, the methodology applied by the researchers was heuristic evaluation. In heuristic evaluation, we used the 10 heuristics Nielsen [6] and the Ergonomic Criteria of Bastien and Scapin [7]. For both heuristics, the process applied was first to use ATs to develop an LO as close as possible to the R-LO, and then apply the heurisitcs on each of the tools. 1) The 10 Nielsen Heuristics In this method, the interfaces are evaluated using 10 rules, which provides specific guidelines to measure the usability of the proposed tools. The rules used are: (a) visibility of system status, (2) compatibility between the system and the real world, (3) control and freedom to the user, (4) consistency and standards, (5) error prevention ( 6) recognition instead of recall, (7) flexibility and efficiency in use, (8) aesthetic and minimalist design, (9) help for users to recognize, diagnose and recover from errors and (10) help and documentation.

Questionnaires Learning Time

Coooperative

important item, 3 minimum item.

2) Ergonomic Criteria of Bastien and Scapin Bastien and Scapin [6] propose ergonomic criteria for the development of methods and tools in order to assign human factors in the design, and to evaluate Human Machine Interfaces (HMI). These criteria are regarded as a resource to define and operationalize usability dimensions. The criteria proposed by the authors are: (1) guidelines (2) workload, (3) explicit control, (4) adaptability, (5) mistake management (6) consistency, (7) code lexicon and names and (8) compatibility.

Hypertext Multimidia

2

Tutorial Exercise and Practice Applicative Use of Multimidia & Internet Development of Multimidia Internet Simulation and Modeling

&

Games

We have stabelished the following formula to select the 6 ATs : NF = N * P, where (NF) is the final note, (N) score ranging 0-6 and (P) is the weighing 1-3. The possible values for note (N) are: 0 for ATs with no requirements or not applicable, 1 to ATs that do not meet requirements; 2 to ATs that poorly meet the requirements; 3 ATs that meet about half of the requirements; 4 for ATs that meet most of the requirements; 5 for ATs that meet nearly all the requirements, 6 for ATs that fully meet the requirements. And the possible values for weight (P) are: 1 dispensable item; 2

C. M ethodology for the Evaluation of Generated Products (LOs) An objective and repeatable metric is required for qualitative assessment of the LOs produced. In order to have such metric, we applied the quality criteria for LOs referred in [8, 9, 10]. The criteria proposed by the authors are divided into three hierarchical levels. The first level features four elements, the second level features 16 elements and the third level features 41 elements. 1) Reference Learning Object In order to develop an LO to feature as the gold standard, the team made use of the public material of the Specialization Course in Family Health of UNA-SUS, as well as the commercial tools Adobe Flash, Adobe Dreamweaver, Adobe Photoshop and Corel Draw. In Figure 1 you see the reference LO (R-LO) for the production of LOs in Authoring Tools.

1558-7908 © 2008 IEEE Education Society Students Activities Committee (EdSocSAC) http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR
IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. X, NO. X, MONTH YEAR< REFERENCES [1] [2] [3]

[4]

[5] [6]

[7]

[8]

[9]

[10]

[11]

[12]

[13] [14]

[15] [16] [17] [18]

[19] [20]

Universidade Aberta do Brasil. Disp. em: . Acesso em: 20/04/ 2010. Universidade Aberta do Sistema Único de Saúde (UnA-SUS). Disponível em: . Acesso em 20/04/2010. S. Ainshworth e P. Fleming. Evaluating Authoring Tools for Teachers as Instructional Designers. Comp in Human Behavior. 22: 131-148. Jan. 2006. V. N. Convertini, D. Albanese, A. Marengo, V. Marengo and M. Scalera. The osel taxonomy for the classification of learning objects. Interdisciplinary Journal of E-Learning and Learning Objects. 125–138, 2006. J. A. Valente. O Computador na Sociedade do Conhecimento. UNICAMP/ NIED. 1999. J. Nielsen. Ten Usability Heuristics. Disponível em: . Acesso em 10/04/ 2010. J. M. C. Bastien, D. Scapin. Ergonomic Criteria for the Evaluation of Human-Computer Interfaces. Institute National de Recherche en Informatique et en Automatique, France, 1993. A. Zapata, V. H. Menendez e M. E. Prieto, Discovering Learning Objects Usability Characteristics. Intelligent Systems Design and Applications, International Conference on, 1126-1130, 2009 Ninth International Conference on Intelligent Systems Design and Applications, 2009. A. Segura, C. Vidal, V. Menedez, A. Zapata, M. Prieto. Exploring Characterizations of Learning Object Repositories Using Data Mining Techniques; Springer Berlin Heidelberg, 46: 215-225. Kaambal: Teleaprendizaje y Gestión del Conocimento. Disponível em: . Acesso em 15/03/2010. R. R. Relvão. Estudo Sobre a Utilização e Interoperabilidade entre Conteúdo de Aprendizagem com Diferentes Granularidades. Dissertação de Mestrado em Sistema de Informação. Universidade do Minho, Jun 2006. S. Downes. Learning Objects: Resourse for distance education worldwide. International Review of Research in Open and Distance Learning. 2, 1, 2001. T. Boyle. Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology. 19: 46-58, 2003. J. H. Liu et al. The Design of Learning Objetc Authoring Tool Based on SCORM. Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies. 778-782, 2005. CourseLab. Disponível em: . Acesso em: 31/03/2010. eXe Learning. Disponível em: . Acesso em 31/03/2010. HotPotatoes. Disp.: . Acesso em: 31/03/2010. Microsoft LCDS. Disponível em: . Acesso em: 31/03/2010. MyUdutu. Disponível em: . Acesso em 31/03/2010. Xerte. Disponível em: . Acesso em: 31/03/2010.

1558-7908 © 2008 IEEE Education Society Students Activities Committee (EdSocSAC) http://www.ieee.org/edsocsac

10