checklist-based assessment of learner experience in

CHECKLIST-BASED ASSESSMENT OF LEARNER EXPERIENCE IN NEO-TROPICAL WILDLIFE ELEARNING БАЗИРАНА НА ВЪПРОСНИК ОЦЕНКА НА СТУДЕНТСКОТО ИЗЖИВЯВАНЕ ПРИ ЕЛЕКТРОННО ОБУЧЕНИЕ В ОБЛАСТТА НА НЕО-ТРОПИЧНАТА ФАУНА Stacey Syne1, Alexander Nikov2, Gary Garcia1, Michele Singh1, Sasho Draganov3 1

Department of Food Production The University of the West Indies, Trinidad and Tobago E-mail: [email protected], [email protected], [email protected] 2

Department of Computing and Information Technology The University of the West Indies, Trinidad and Tobago E-mail: [email protected] 3

Department of Industrial Design Engineering Technical University of Sofia, Bulgaria E-mail: [email protected]

Abstract. A lack of dedicated knowledge resource platforms, a large geographic distribution of experts, and the absence of widespread educational material are just a few of the challenges that plague the field of neo-tropical wildlife science. The use of eLearning promises to be an effective, affordable solution whereby the public and experts alike can learn more about these species and their conservation. For this to be successful, eLearning system assessment is necessary and the checklist described in this paper presents a way to do this, focusing on positive wildlife learner experience. When students evaluate an eLearning system with the checklist in a real life learning environment, results can map experiences so that elements of the eLearning system in need of substantial improvement can be identified and remedied with relative ease. By so doing, both ease-of-use and increased satisfaction can be incorporated to improve learner experience. Keywords: user experience, elearning, neo-tropical wildlife, checklist, assessment.

1. Introduction eLearning can be described as all forms of electronic supported procedures of learning and teaching which aim to construct knowledge using experience, practice, and information and communication technology systems [1]. Although the transition from traditional classroom-based lectures to eLearning presents itself

397

as an obstacle [2], if done properly, eLearning gives significant advantages as it allows time and location flexibility, learning to occur as an integrated activity, and the participation of a global audience [3, 4]. eLearning can be personalized by determining the content according to users’ needs, for example, changes in learning material through user input of new experiences and practices [4]. The advantages of eLearning can be of substantial benefit to the study of neotropical wildlife, which are the terrestrial and aquatic non-domestic species that live in tropical areas of Latin America and the Caribbean [5]. Significant threats to these animals exist such as over-hunting, neglect through improper forest management, and deforestation, which in turn, result in problems such as reduced population biomass, reduced species diversity, and possible extinction [5-7]. Despite the steep economic and ecological value of these animals, there is a dearth of information on them, particularly on their species life cycle, physiological states for production, and animal specific needs [8]. An example which demonstrates the vastness and geographic dispersion of the region under study is the Amazon basin which covers an estimated area of 6,000,000 km2 [9]. However, the impact of man on the environment should never be underestimated as over 75% of the Amazon basin can be accessed by foot if people are prepared to walk up to 9 km from an access point, and the greatest impact of hunting usually occurs within 6 km of access points [9]. Compounding the necessity for widespread education and conservation of these species, is the fact that hunters are often prepared to walk much longer than this 6 km mark if required [9]. The promise of eLearning in the field of neo-tropical wildlife brings with it the dissemination of information in this little known field, easily at no or very little cost to researchers, hunters, farmers, students and the general public, with the end result of increased awareness, conservation efforts and significant improvements in wildlife farming practices. Ripple effects envisioned encompass neo-tropical wildlife species with increased population numbers and a high genetic diversity, more available information on the anatomy, behaviour and physiology of these animals, and increased yields to farms which rear these animals. Ultimately, this will lead to a better quality of life through the ecological services nature provides as well as through increased income via eco-tourism, farm sales and their spin-off activities. In order for eLearning in neo-tropical wildlife to be successful, it should offer good learner experience. Learners have a tendency to become frustrated and disappointed when course-related websites are not efficient, easy to learn, effective to use and enjoyable [10]. In addition to this, developers traditionally do not place a high priority on students’ wants, behavior and needs, rather, they focus on their technical know-how [10]. Learner experience comprises components based on how easily users can learn tasks from a site the first (learnability) and subsequent (efficiency) visits, how well they can remember to perform these tasks (memorability), the frequency, severity 398

and recovery from errors, the availability of help, and how pleasant the design is to use (satisfaction) [11]. From a pedagogic standpoint, important aspects to consider are the design of learning activities within the framework of technical, general, academic and context-specific user experience [10]. Furthermore, it must be ensured that learning objectives have been met, there is active student participation, collaborative learning, learning with understanding and that conflict and frustration is reduced [10]. Content-related interactions support meaningful learning, message design describes good information presentation in terms of organization and layout, learning design ensures that sound principles of learning theory are supported and learners are engaged with tasks closely aligned with learning objectives, and media integration describes the use of appropriate media to support instructional design principles [12]. The effective eLearning can be supported with case studies or data, and feedback to provide the learner with information on their current level of achievement [12]. Attentive and careful interface design together with information on how people learn with computers can help to mould eLearning websites with features such as rapid learning, high skill retention, low error rates, consistency, predictability, easy navigation and flexibility [10]. One of the most important factors where more emphasis needs to be placed, is the creation a consolidated assessment methodology to gauge student’s interactions [13]. One such method is a qualitative Weight and Sum Approach which evaluates eLearning systems using factors such as adaptivity, adaptability and personalization through students ranking their experiences from a real life teaching situation on a questionnaire [14]. However, there is a need to employ a more comprehensive method for measuring student experience by a finite number of dimensions. This can subsequently provide a reliable method for assessing students’ perceptions and performance, as well as act as a feedback mechanism which can prompt change for the better. The proposed checklist is just such a method which covers all of these. 2. Checklist Design A checklist for supporting the creation of efficient, effective and affective learner experience through the feedback of students using an eLearning system in neo-tropical wildlife is developed. In fig. 1, the structure of the checklist for wildlife learner experience (WLX) assessment is presented.

399

WLX index calculation

Fig. 1. The checklist structure 400

The checklist comprises 42 close ended questions based on 10 dimensions: sensory design, interface design, navigation design, visual design, learning design, interaction design, personalization design, affective design, reliability /maintainability and satisfaction. Each of the checklist questions are designed to be ranked by the learner against a 5 point Likert scale in order to determine their level of satisfaction or dissatisfaction with the particular feature. The Likert scale is a scale of attitude statements which presents a rating of agreement and disagreement in empirically derived scales, with the central point indicating neutrality [15, 16]. In this checklist, the 5 points from 1 to 5 are Strongly Agree, Slightly Agree, Neutral, Slightly Disagree and Strongly Disagree respectively (cf. fig. 2).

Fig. 2. Assessment scale The sensory design dimension comprises 7 questions (cf. fig. 3) which focus on the visual components of the website, how meaningful their functions were and the extent to which they enhance user experience. The impact of visual stimuli such as color scheme, font and font-size are important to measure, as it influences emotions that can either reinforce or undermine other aspects of design, to negatively or positively impact users’ learning experience [17].

Fig. 3. Example checklist items for dimension sensory design For the next dimension, interface design, 3 questions are incorporated which mainly focus on the prevention of errors. Error prevention is usually the product of either direct support provided to the user such as FAQ’s, or it may be indirect through the efficacious design of the website. Readily accessible help is important

401

to the user as it provides steps to rectify any errors made or resolve any questions about the site’s features and use [12]. The navigation design dimension holds 5 questions which are centered on the ease of the user’s movement through the information architecture, as well as features of the eLearning website that enable this [17, 18]. Navigation design is likely to refer to the use of dropdown and or slide out menus which help to facilitate and manage a high diversity of content, in addition to breadcrumbs which allow the user to have an idea of where they are in a complex website. The visual design dimension comprises 5 questions which focus on the format of course material and how well it supports learning and understanding concepts. The incorporation of slideshows, videos, images, photo galleries and other multimedia in lecture notes are just some of the ways the assimilation and interpretation of information can be improved, with the added advantage that different learning styles are catered for. Questions in this dimension determine if the website is successful in providing a variety of material to learn from. The learning design dimension is based on pedagogic practices which ensure that course material can easily and progressively build understanding and knowledge in an efficient, logical manner. Six questions cover this dimension to evaluate the website in this light. Five questions fall under the next dimension, interaction design, which deals with access to course material and communication features. Questions under this category investigate whether the site sufficiently provides content-related interactions as well as tasks that support learning activities and fill knowledge gaps [12]. This may be in the form of chat rooms and message boards to exchange information or quizzes to test knowledge. Students are also asked how quickly and easily course information can be retrieved. Two questions cover the personalization design dimension, which refers to the extent to which the site’s features can be changed to match the various learning styles of students. This is important as flexibility over the learning process, style and content helps users meet their individual needs [3]. The next dimension is affective design, which focuses on how enjoyable the site is to use as a learning medium, taking into special account the way in which course material is presented. The aim of the 4 questions in this dimension is to discern whether the learning experience was a pleasurable one or not, and identify if the cause originated in the layout, presentation, or content of the course material. Reliability/Maintainability is the next dimension which speaks to how often the site is updated and new developments are shown. This is particularly important in the field of neo-tropical animal science where new information can be vital to the advancement of the field and progress of conservation activities; and 2 questions are dedicated to this in the checklist. The final dimension is satisfaction, where 3 questions ask the user about his/her overall experience and if they are satisfied. This ultimately indicates the

402

success of the website as satisfaction is more or less a holistic description of the learning experience. 3. Wildlife Learner Experience Assessment For assessing learner experience the steps shown in Fig. 4 are proposed. At step 1 and step 2 the checklist dimensions and their relevant items or questions are determined. At step 3, answers of the learner against the checklist questions after using the eLearning system are gathered. At step 4 the data collected at step 3 are analyzed and WLX quantitative index calculated (cf. Fig. 1). At step 5, relevant recommendations for improving learner experience are proposed. 1

2

3

4

5

Checklist dimensions

Checklist items

Data gathering by checklist

Checklist data analysis

Design recommendations

Fig. 4. Checklist-based assessment steps

At step 3 data from learners using the eLearning system is gathered by the checklist. For this purpose a Likert scale 1-5 is used (cf. fig. 2). Prior to the full scale evaluation, 2 pilot tests were carried out to ensure that the methodology was sound, and that both the tasks and checklist questions could be reasonably completed, especially within the timeline available. The WLX assessment consists of a number of tasks students are asked to perform (e.g. find 403

course material within the site, find the text of interest, read it, understand it) and then answer a quiz based on the text. They are instructed to write their start and stop times, and once the quiz is finished, students are asked to rate their learning experiences on the Likert scale (cf. fig. 2) using the checklist questions (cf. fig. 3). Thereafter at step 3, data on the quiz answers, the period of time for the completion of each task, and students’ ratings on the Likert scale for checklist questions are gathered. At step 4 checklist data analysis is performed in order to determine which checklist dimensions are in most need of improvement. At step 5 relevant design improvements of the eLearning system are recommended. By carrying out this process, the students’ experiences are mapped and their needs are explored so that certain elements of the website’s WLX in need of substantial improvement can be identified with relative ease. Thereafter, enhancements can be made where deemed necessary to provide new flexibility, adaptiveness, and to make their interaction with the online material as natural, intuitive and fluid as possible [19]. Several discoveries can also be made to determine; where more appealing styles of information are required in the eLearning website, if new intriguing methods of teaching are needed to keep students interested, and how well are students assimilating and memorizing the information. In so doing, the different learning styles of students can be more greatly taken into account, alongside the incorporation of sound pedagogic practices [19]. The overall result is a comprehensive set of recommendations which the eLearning designer can use as a platform to make site improvements. Without such an assessment exercise, chances are the learner’s experience will continue to suffer from problems related to navigation, orientation and visual appeal, among others [12]. 4. Conclusions A checklist for assessing learner experience in the field of neo-tropical wildlife eLearning is presented. Using the students’ assessments, relevant redesign suggestions for improving the eLearning system are determined. This checklist has the potential to help change the entire way in which neo-tropical animal science and production is taught and viewed by academia and the general public. These species and their native habitats are very often associated with the stigma of sparsely populated rural communities in the lower-income bracket, desolate areas, and remote forests, all of which are not at the forefront of people’s minds in such a fast-paced world. With feedback from students undertaking evaluations using this checklist, the ability to effectively and efficiently revise the site to improve learner experience is greatly enhanced. Distance education can be enabled and recent research, knowledge and the most up-to-date techniques can be shared to highlight the value of these species. Additionally, a well-designed, functional eLearning website solely dedicated to neo-tropical wildlife has potential to put these species in the limelight 404

in a modern, exciting fashion. By refining eLearning in this manner, there is great potential to redefine how these animals are managed, conserved, utilized and protected in the pursuit for sustainable development. References 1. TAVANGARIAN, D., ET AL., Is e-learning the Solution for Individual Learning. Electronic Journal of E-learning, 2004, 2(2):273-280 p. 2. O'NEILL, K., ET AL., Implementing eLearning Programmes for Higher Education: A Review of the Literature. Journal of Information Technology Education, 2004, 3. 3. DONGSONG, Z., ET AL., Can eLearning Replace Classroom Learning? Communications of the ACM, 2004, 47(5):75-79 p. 4. STOJANOVIC, L., S. STAAB, AND R. STUDER. eLearning based on the Semantic Web. in WebNet2001-World Conference on the WWW and Internet. 2001. 5. GARCIA, G. Neo-tropical Animals and Agro- tourism: A Trinidad and Tobago Opportunity. in Conference on Turtle Conservation, Ecotourism and Sustainable Development, University of the West Indies, St. Augustine. July 28-29, 2009. 6. NOGUEIRA, S.S.C. AND S.L.G. NOGUEIRA-FILHO, Wildlife farming: an alternative to unsustainable hunting and deforestation in Neotropical forests? Biodiversity and Conservation, 2011, 20(7):1385-1397 p. 7. PERES, C.A., Synergistic effects of subsistence hunting and habitat fragmentation on amazonian forest vertebrates. Conservation Biology, 2001, 15(6):1490-1505 p. 8. BROWN-UDDENBERG, R.C., ET AL., The Agouti (Dasyprocta leporina, D. Aguti) Booklet and Producers' Manual. 2004, Champs Fleurs, Trinidad: GWG Publications. 9. PERES, C.A. AND I.R. LAKE, Extent of Nontimber Resource Extraction in Tropical Forests: Accessibility to Game Vertebrates by Hunters in the Amazon Basin. Conservation Biology, 2003, 17(2):521 p. 10. KUKULSKA-HULME, A. AND L. SHIELD. Usability and pedagogical design: Are language learning websites special? in Proceedings of ED-MEDIA 2004. 2004. 11. NIELSEN, J., Usability 101: Introduction to usability, 2003. 12. REEVES, T.C., ET AL., Usability and Instructional Design Heuristics for E-Learning Evaluation. 2002. 13. COSTABILE, M.F., ET AL. On the Usability Evaluation of E-Learning Applications. in System Sciences, 2005. HICSS '05. Proceedings of the 38th Annual Hawaii International Conference 2005. 14. GRAF, S. AND B. LIST. An Evaluation of Open Source E-Learning Platforms Stressing Adaptation Issues. in ICALT. 2005. 15. KOMORITA, S.S., Attitude Content, Intensity, and the Neutral Point on a Likert Scale. The Journal of Social Psychology, 1963, 61(2):327-334 p. 16. WUENSCH, K.L. What is a Likert Scale? and How Do You Pronounce "Likert?". 2005 11 March 2014; Available from: http://core.ecu.edu/psyc/wuenschk/StatHelp/Likert.htm. 17. OPEN JOURNALISM ON THE OPEN WEB. Elements of User Experience by Jesse James Garrett. 2011 16 March 2014; Available from: http://www.slideshare.net/openjournalism/elements-of-user-experience-by-jesse-jamesgarrett. 18. GARRETT, J.J., Elements of User Experience, The: User-Centered Design for the Web and Beyond. 2010: Pearson Education. 19. ARDITO, C., ET AL., An approach to usability evaluation of e-learning applications. Universal access in the information society, 2006, 4(3):270-283 p.

405

.

TECHNICAL UNIVERSITY - SOFIA

~ FACULTY OF MANAGEMENT

IV'

§

•

-§::

UNIVERSITY OF VERSAILLES SAINT QUENTIN EN YVELINES - FRANCE

GRADUATE SCHOOL OF MANAGEMENT

SCIENTIFIC AND TECHNICAL UNION

OF MECHANICAL ENGINEER ING

NATIONAL SCIENTIFIC AND TECH NICAL SOCIETY "MANAGEMENT AND ENGI NEERING"

CONFERENCE PROCEEDIN

Volume I

X II INTERNATIONAL

SCIENTIFIC CONFERENCE

"MANAG EMENT AND ENGINEERING '14"

SCIENTIFIC PROCEEDINGS SCIENTIFIC-TECHNICAL UNION OF MECHANICAL ENGINEERING Year XXII, 6/155 June 2014

ISSN 1310-3946 ISSN 1314-6327

DAYS OF SCIENCE AT TU-SOFIA, 2014 JUNE 22-25, 2014 Sozopol, BULGARIA

CONTENTS VOLUME 1 Management ..................................................................................................................... 12 Vasyl Gerasymchuk, Management Mechanism By Organization....................................... 13 Arnoldina Pabedinskaitė, Measuring And Improvement The Quality Of Airport Services ............................................................................................................... 23 Todor Kralev, Lidija Naumovska, Retailing–Mix For Partial Grading Of Success In Retail Stores Operations .................................................................................................................. 33 Panayot Gantchev, Martin Istatkov, Information And Communication Subsystem Structure In Small Enterprises .............................................................................................. 42 Matilda Alexandrova, Options For Balancing The Project Portfolio In Bulgarian ProjectOriented Organizations ......................................................................................................... 52 Ivan Paligorov, Stanislava Kovacheva, Elena Dragozova-Ivanova, Ivaylo Ivanov, Key Factors In The Forest Landscape Management - Parameters Of Their Change And Degree Of Influence ............................................................................................................. 64 Kiril Anguelov, Innovative Methods For Quality Management Of Administrative Services ................................................................................................... 73 Toni Mihova, The Process Of Making Managerial Decisions – Some Approaches And Techniques............................................................................................................................ 80 Irena Dimitrova, Krasimir Enimanev, Guidelines For Improving The Management Of Bulgarian Mobile Operator ................................................................................................... 88 Mimoza Davcheva, Aleksandra Stoilkovskа, Valentina Paleska, Delevoping Strategic Management System Within The Educational Organizations Towards Securing Higher Quality Management .......................................................................................................... 100 Nikolay Zlatov, Ralitza Zlatova-Stojer, The Appropriateness Of A Lifelong Learning (Ll) Framework For Mechatronics Graduates At Lsbu .............................................................. 108 Ivaylo Ivanov, Business Continuity Management In The Bulgarian Industrial Enterprises .......................................................................................................................... 118 Todor Razmov, Risk Analisis In The Preparation Of The Package To Transport Investments Projects ........................................................................................................... 128 Velislava Kostova, Potential For The Development Of Specialized Types Of Tourism In Bulgaria .............................................................................................................................. 139 Yuri Ivanov, Valentin Velev, Peculiarities Of Control Over The Business Of The Operators W&S .................................................................................................................. 149 Elena Dragozova-Ivanova, Evaluation Of The Students For Education In Landscape Architecture ........................................................................................................................ 157 Anatoli Georgiev, Insurance Policies Of The Economic Organization ............................. 164 Vesela Vodenicharova, Malin Malinov, Administrative Services To The Library Information Centre Technical University – Sofia ............................................................... 172 Desislava Shatarova, Tanya Gigova, Strategic Managemant – A Condition For Successful Business .............................................................................................................................. 182 Tanya Gigova, Desislava Shatarova, Small And Medium Business – Opportunities For Development Of The Bulgarian Economy .......................................................................... 192 Natasha Kraleva, Critical Analyses Of The Organization Development As An Organizational Change ....................................................................................................... 202

Industrial Engineering and Operational Management ...................................... 207 Borislav Petrov, Realization Of Universal Timer Through Microcomputer ..................... 208 Borislav Petrov, Realization Of Vending Machine Through Microcomputer ................... 217 Ivan Ivanov, Goritsa Chileva, Maria Gergova, A Subsystem Iteration Approach For Equilibrium Point Of Continuous Time Markov Jump Linear Systems ............................. 224 Mirena Todorova, Application Of Ahp Model For Evaluation Of Carrieer As Service Supplier .............................................................................................................................. 230 Yuliyan Idakiev, Comparative Analysis Of Models For Evaluation Of Service Related Business Processes ............................................................................................................. 240 Svetoslav Dimkow, Remanufacturing Strategy In Contract Furniture Production ............ 247 Svetoslav Dimkow, Digital Manufacturing Strategy Implementation In Contract Furniture Manufacturing .................................................................................................................... 257 Ivan Dakov, Mariyan Genchev, Analysis Of The Approaches For Mass Customization In Industrial Enterprises .......................................................................................................... 267 Ludmil Nedyalkov, Ognyan Andreev, Ivan Dakov, Implementing Lean Production In A Bulgarian Industrial Enterprise ........................................................................................... 277 Sashko Gramatnikovski, Aleksandra Stoilkovska, Konstantin Angelovski, Increasing The Company’S Efficiency Via Supply Chain Management .............................................. 286 Milena Dicheva, Stanislava Kovacheva, Study Opportunities For Implementation Of ErpSystem With Alternative Tourism Supply .......................................................................... 296

Human Resource Management ................................................................................ 304 Scilien Jean-Christophe, Rozin Philippe, Change Management In Restructuring Plans: The Risk Of And "Anchored" Hidden Costs, An Empirical Study In A French Context ... 305 Martial Kadji, Janine Quintin, Role Of Knowledge Management Transfer In The Sustainability Of Communities Of Practice: The Case Of Open Source Actors Of Communities In France....................................................................................................... 324 Sarra Mokaddem, Amina Bécheur, Luciana Castro, Proposition Of An Integrated Analysis Scale For Understanding Brakes In The Adoption Of The Innovation: The Case Of An Eco-Mobility Plateform ................................................................................................ 342 Annie Bartoli, Christelle Perrin, What Managerial Lessons From Social And Solidarity Economy? Particularities And Possible Transpositions In Other Organizations ................ 350 Thierry Côme, Gilles Rouet, The Insertion Of Doctorate Holders In Enterprises: A Managerial Innovation ........................................................................................................ 366 Stilyana Petrunova, Development Of Methods And Instruments For Measuring, Identifying And Managing The Intellectual Capital In Organizations ................................ 380 Konstantin Petkovski, Kire Popovski, Fanche Joshevska, Factors For Good Climate In Educational Organizations .................................................................................................. 390 Stacey Syne, Alexander Nikov, Gary Garcia, Michele Singh, Sasho Draganov, Checklist-Based Assessment Of Learner Experience In Neo-Tropical Wildlife Elearning 397 Nataliya Kireva, Borislava Kolchagova, Human Resource Management’S Mechanisms Of Influence On Organizational Results .................................................................................. 406 Bela Boneva, Influence Between Strategies For Human Resource Development .............. 416 Mina Angelova, Opportunities To Motivate Specialists Through The System Of Human Resource Management........................................................................................................ 423 Neli Nikolova, Impact Of Emotional Intelligence On Motivation And Employability ...... 430