9th International Conference on e-Learning

Proceedings g of the

9th International Conference on e-Learning Technical University y Federico Santa María Valparaiso, Chile 26-27 June 2014

Edited by Dr Teresita Arenas Yáñez, Dr Oscar Saavedra Rodriguez Technical University Federico Santa Maria, Chile and Dr Paul Griffiths Birchman Group, Chile

A conference managed by ACPI, UK

Proceedings of the The 9th International Conference on e-Learning ICEL-2014 Hosted by The Federico Santa María Technical University Valparaiso Chile 26-27 June 2014 Dr Teresita Arenas Yáñez, Dr Oscar Saavedra Rodriguez Technical University Federico Santa Maria UTFSM, Chile and Dr Paul Griffiths Birchman Group, Chile

Copyright The Authors, 2014. All Rights Reserved. No reproduction, copy or transmission may be made without written permission from the individual authors. Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper abstracts were read and selected by the conference panel for submission as possible papers for the conference. Many thanks to the reviewers who helped ensure the quality of the full papers. These Conference Proceedings have been submitted to Thomson ISI for indexing. Further copies of this book and previous year’s proceedings can be purchased from http://academic-bookshop.com E-Book ISBN: 978-1-909507-84-5 E-Book ISSN: 2048-8890 Book version ISBN: 978-1-909507-69-2 Book Version ISSN: 2048-8882 CD Version ISBN: 978-1-909507-88-3 CD Version ISSN: 2048-8904 Published by Academic Conferences and Publishing International Limited Reading UK 44-118-972-4148 www.academic-publishing.org

Contents Paper Title

Author(s)

Page No.

Preface

iii

Committee

iv

Biographies

vii

E-Learning has low Adoption in Chile: Is This Related to Learning Styles?

Teresita Arenas and Paul Griffiths

1

A Flexible Framework for Metacognitive Modelling and Development

Liz Bacon and Lachlan MacKinnon

7

Authentic Assessment in Online Learning: Moving Beyond Text to Celebrate Multimodal Measures of Student Achievement

Wendy Barber, Sherry King and Sylvia Buchanan

15

Addressing the Mathematical Knowledge gap Between High School and First Year University Chemical Engineering Mathematics Course: The Role of Facebook

Moses Basitere and Eunice Ivala

22

Telling Tales: Responding to Challenges in Literacy Competencies Using e-Reader-Based Programs

Paula Charbonneau-Gowdy

31

Factors Leading to Technology Non-Use in Education: Validation of Reversed-UTAUT (R-UTAUT)

Phaik Kin Cheah, Yee Onn Yap, Prabha Unnithan, Priscilla Moses, Fong Wei Diong and Pek Hoon Er

39

Tracing Digital Traces of Phatic Communion in #mooc

Judith Enriquez-Gibson

45

Developing a Learning Management Platform: A Bulgarian Case

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev

50

Extending the Design of Games for e-Learning to Include Affective Computing Elements

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon

59

Communities of Practice: The Impact of Web 2.0 Technologies

Paul Griffiths and Teresita Arenas

67

Negotiating the Triple Helix: Harnessing Technology for Transformation

Maggie Hutchings, Anne Quinney and Kate Galvin

76

An Empirical Survey on the Adoption of e-Learning in Corporate South Africa: Service Providers’ Perspective

Eunice Ivala

86

An Approach in the Design of Virtual Environment for eLearning Based on Usability Studies

Borka Jerman Blažič, Tanja Arh and Andrej Jerman Blažič

95

Engaging Local and Global Learners in an e-Learning Environment - University of Toronto Experiences

Nohjin Kee, Dani Sarkis and Chris Perumalla

104

Virtual Collaboration Competence Requirements for Entrepreneurship Education in Sparsely Populated Areas

Päivi Lohikoski, Matti Muhos and Janne Härkönen

109

E-Learning’s Indispensability in Crisis Management Education

Maria Nathan

118

Crowdsourcing College Examinations Using Technology to Improve Assessment and Higher Learning by Students

Timothy Olson

123

Comparing the Student Experience of Resident and Distance Learning Students

David Owen and Elly Philpott

131

Developing Employability Skills in Humanities and Social Sciences Using the Flipped Model

Brenda Ravenscroft and Ulemu Luhanga

142

Can Seniors’ Quality of Life be Enhanced Through the use of Digital Games? Definition and Measurement

Lise Renaud, Louise Sauvé, David Kaufman and Emmanuel Duplàa

150

The Effect of 3D Learning Tools on Learning Outcomes: A Study of Students’ Interaction With an Online Atlas

Florin Salajan and Anuradha Prakki

157

i

Paper Title

Author(s)

Page No.

Forced Necessity: MOOCs in Post‐Soviet Countries

Daniyar Sapargaliyev

165

A Web‐Based Recommendation System for Engineering Educa‐ tion e‐Learning Solutions

Thorsten Sommer, Ursula Bach, Anja Richert and Sabina Jeschke

169

Location‐Based Object Identification Algorithm for Augmented Reality in Adaptive Mobile Learning

Qing Tan, William Chang and Kinshuk

176

Employing Objective Measures in Search of a Relationship between Knowledge Blogs and Introductory Programming Performance Outcome

Dalize van Heerden and Mac van der Merwe

185

Textbook‐Free Learning: A Framework for Critical Analysis

Roxanne Ward Zaghab and Patricia Beckenholdt

Factors Influencing Users’ Perceived Learning During Online Searching

Xiangmin Zhang, Jingjing Liu, Chang Liu and Michael Cole

200

PHD Research papers

211

Qualifying the Quantified Self ‐ a Study of Conscious Learning

Thomas Kjærgaard and Elsebeth Korsgaard Sorensen

213

Rhizomatic, Digital Habitat ‐ a Study of Connected Learning and Thomas Kjærgaard and Elsebeth Korsgaard Technology Application Sorensen

221

E‐Learning Portal Functionality Assessment With the use of Eye‐Tracking Experiment

Stanislav Popelka, Ales Vavra, Rostislav Netek and Vilem Pechanec

228

Masters Research papers

239

Learner Perceptions on the use of Social Networking Services in Ricardo da Rocha, Pieter Conradie and Education: A Case Study Antoinette Lombard

1490

241

Evaluation of Accessibility and Usability Applied to e‐Learning Platforms in Health

Nuno Dominguinhos

248

Work In Progress Paper

255

SCORM Packaging of 8085 Microprocessor Simulator and its Evaluation on Moodle

Kazuhiro Muramatsu

257

Late Submission papers

263

The role of Open Access and Open Educational Resources: A Distance Learning perspective

Stylianos Hatzipanagos and Jon Gregson

265

Game Based Learning – A Way to Stimulate Intrinsic Motivation Peter Mozelius

272

E‐learning as a way of Connecting Students from Developing Countries to the Global Academic World: A case study on Bolivia, Chile and Barcelona

279

César Yáñez, Marc Badia‐Miró and Anna Carreras‐Marín

ii

Preface This book represents the Proceedings of the 9th International Conference on e-Learning. The conference this year is hosted by The Federico Santa María Technical University, Valparaiso, Chile. The Conference Chair is Dr Teresita Arenas Yáñez, from Technical University Federico Santa Maria UTFSM, Chile, and the Programme Co-chairs are Dr Oscar Saavedra Rodriguez, Technical University Federico Santa Maria UTFSM, Chile and Dr Paul Griffiths, Birchman Group, Chile. The International Conference on e-Learning (ICEL-2014) invites researchers, practitioners and academics to present their research findings, works in progress, case studies and conceptual advances in areas of work where education and technology intersect. The conference brings together varied groups of people with different perspectives, experiences and knowledge in one location. It aims to help practitioners find ways of putting research into practice and researchers to gain an understanding of real-world problems, needs and aspirations. With an initial submission of 95 abstracts, and after the double blind peer review process, there are 29 Academic papers, 3 PhD Research papers, 2 Masters research papers, 1 Work in Progress paper published in these Conference Proceedings. These papers represent research from more than 20 countries including: Bulgaria, Canada, Chile, Czech Republic, Denmark, Finland, Germany, Kazakhstan, Malaysia, Nigeria, Portugal, Slovenia, South Africa, South Africa, Spain, Sweden, UK and USA. A selection of the best papers – those agreed by a panel of reviewers and the editor will be published in a conference edition of the EJEL (Electronic Journal of e-Learning www.ejel.org). These will be chosen for their quality of writing and relevance to the Journal’s objective of publishing papers on a range of topical E-Learning issues. We wish you a most interesting conference. Dr Teresita Arenas Yáñez, UTFSM, Chile Conference Chair Dr Oscar Saavedra Rodriguez, UTFSM, Chile Dr Paul Griffiths, Birchman Group, Chile Programme Co-Chairs June 2014

iii

Conference Committee Conference Executive Dr Teresita Arenas Yáñez, Technical University Federico Santa Maria, Chile Dr Oscar Saavedra Rodriguez, Technical University Federico Santa Maria, Chile Dr Paul Griffiths, Birchman Group, Chile Mini track chairs Dr Susan Crichton, University of British Columbia – Okanagan, , Canada Jeanne Schreurs, Hasselt University in Belgium Dr Eunice Ivala, Cape Peninsula University of Technology (CPUT), South Africa Dr. Mads Bo-Kristensen, Smart University Vejle, Denmark Conference Committee Members: The conference programme committee consists of key people in the e-learning community around the world. The following people have confirmed their participation: If you are interested in joining the committee for this conference, please click the conference committee button on the right of this page. Mohd Helmy Abd Wahab (Universiti Tun Hussein Onn Malaysia, Malaysia); Dr. Peter Aborisade (The Federal University of Technology Akure, Nigeria); Dr. Bulent Acma (Anadolu University, Eskisehir, Turkey); Dr. Chigona Agnes (Cape Peninsula University of Technology, South Africa); Dr. Ali Alawneh (Philadelphia University, Jordan); Prof. Saleh Alhalalat (King Saud University, Saudi Arabia); Lisa Allen (The University of British Columbia, Canada); Naji AlQbailat (Al-Balqa' Applied Uinversity, PAUC, Jordan); Nabeel Al-Qirim (UAE University, United Arab Emirates); Dr. Zahra Rashid Said Al-Rawahi (Sultan Qaboos University, Oman); Kalid Alshahrani (King Fahad Naval Academy , Saudi Arabia); Prof. Abdullah Al-Zoubi (Princess Sumaya University for Technology, Amman, Jordan); Dr. Anca-Olga Andronic (Faculty of Psychology and Educational Sciences, Spiru Haret University, Romania); Dr. Razvan-Lucian Andronic (Faculty of Psychology and Educational Sciences, Spiru Haret University, Romania); Dr Teresita Arenas Yanez (Technical University Federico Santa Maria UTFSM,, Chile); Dr. Ezendu Ariwa (London Metropolitan University, London, UK); Peter Arthur (University of British Columbia Okanagan, Kelowna, British Columbia,, Canada); Dr. William Ashraf (University of Sussex, UK); Dr. Kallol Bagchi (University of Texas at El Paso, USA); Prof. Philip Leon Balcaen (University Of British Columbia, Kelowna, Canada); Karen Barnstable (UBC Okanagan, Canada); Dr. Tshepo Batane (University of Botswana, Botswana); Dr. Gary Bell (London South Bank University, UK); Younes Benslimane (York University, Toronto, Canada); Jennifer Bergh (Eiffel-Corp - (Blackboard partners and resellers), South Africa); Prof. Sonia Berman (University of Cape Town, South Africa); Prosper Bernard (University of Quebec, Canada); dr. Igor Bernik (University of Maribor, Slovenia); Prof. Dr. Amine Berqia (University of Algarve, Faro, Portugal); Karen Bjerg Petersen (The Danish School of Education, University of Aarhus, Denmark); Dr. Patrick Blum (inside Business Group, Aachen, Germany); Dr. Mads Bo-Kristensen (Resource Center for Integration, Vejle, Denmark); David Bond (University of Technology, Sydney, Australia); Prof. Luis Borges Gouveia (University Fernando Pessoa, Portugal); Lynn Bosetti (University of British Columbia Okanagan, Kelowna, British Columbia,, Canada); Dr. Tharrenos Bratitsis (University of Western Macedonia, Greece); Ian Brown (Hong Kong Polytechnic University, Hong Kong); Dr. Sheryl Buckley (Unisa, South Africa); Jekaterina Bule (Riga Technical University, Latvia); Pasquina Campanella (University of Bari "Aldo Moro", Italy); Prof. Phaik Kin Cheah (Universiti Tunku Abdul Rahman (UTAR), Malaysia); Dr. Adeline Chia (Taylor's University, Malaysia); Satyadhyan Chickerur (B V Bhoomaraddi College of Engineering and Technology, Hubli,, India); Chinnapaka Chitharanjandas (Information systems and e business, Bang college of business, Republic of Kazakhstan); Dr. Mohammad Chizari (Tarbiat Modarres University, Iran); Chee-Keong Chong (Universiti Tunku Abdul Rahman (UTAR), Malaysia); Hal Christensen (Christensen/Roberts Solutions, Forest Hill, NY, USA); Dr. Jaesam Chung (Ewha W. University, Rep. of Korea); Prof. Delaine Cochran (Indiana University, USA); Dr. Glenn Cockerline (Brandon University, Canada); David Comiskey (University of Ulster, Ireland); Dr. Caroline Crawford (University of Houston-Clear Lake, USA); Susan Crichton (University of British Columbia , Canada); Johannes Cronje (Cape Peninsula University of Technology, South Africa); Prof. Laura Czerniewicz (University of Cape Town, South Africa); Ramiza Darmi (Universiti Putra Malaysia, Australia); Annemarie Davis (University of South Africa, Pretoria, South Africa); Dr. Pieter De Vries (Delft University of Technology, The Netherlands); Prof. Rhena Delport (University of Pretoria, South Africa); Prof. Jack Dempsey (Univ. of South Alabama, USA); Christina Dinsmore (Southampton Solent University, UK); Dr. Gilles Doiron (Zayed University, Abu Dhabi, United Arab Emirates); Jerome Dooga (University of Jos, Nigeria); Dr. Martina A. Doolan (Univeristy of Hertfordshire, UK); Dr. Laurent Dukan (PHD International, France); Dr. Bulent Gursel Emiroglu (Eskisehir Yolu 20.km. Baglica Mevkii, Turkey); Dr. Judith Enriquez (University of North Texas, USA); Prof. Dr. Alptekin Erkollar (ETCOP, Austria); Prof. Jean-Louis Ermine (Telecom Business School, Evry Cedex, France); Nima Fallah (BETA - Strasbourg University, France); Stephen Farrier (University of Edinburgh, UK); Dr. Omid Fatemi (University of Tehran, Iran); Prof. Corona Felice (Faculty of Medicine and Surgery, University of Salerno, Italy); Dr. Aikyna Finch (Strayer University, Huntsville, USA); Dr. Titi Fola-Adebayo (Fed Univ of Tech, Nigeria); Prof. Joseph Fong (City University of Hong Kong, Hong Kong, China); Marga Franco-Casamitjana (Universitat Oberta de Catalunya, Spain); Daniela Gachago (Cape Peninsula University of Technology, South Africa); Fenella Galpin (Open University, UK); Dr. Grisel Garcia Perez (UBC Okanagan, Caniv

ada); Prof. Henrique Gil (School of Education -Polytechnic Institution of Castelo Branco, Portugal); Dr. Judy Gnarpe (University of Alberta, Canada); Dr. Andrew Goh (International Management Journals, Singapore); Gerald Goh (Multimedia University, Melaka, Malaysia); Dr. Andrea Gorra (Leeds Metropolitan University, UK); Jivesh Govil (Cisco Systems Inc, USA); Dr. Sue Greener (University of Brighton, UK); Dr. David Guralnick (Columbia University and Kaleidoscope Learning, New York, USA , USA); Dr. Rajaram Gurusamy (DMI ST. John the Baptist University, Malawi); Dr. Rugayah Gy Hashim (Universiti Teknologi MARA (UiTM), Malaysia); zuwati Hasim (University of Malaya, Malaysia); Dr. Mahmoud Hassanin (Pharos University in Alexandria, Egypt); Thanos Hatziapostolou (International faculty of the university of sheffield, Greece); Dr. Stylianos Hatzipanagos (King’s College London, UK); Alan Hilliard (University of Hertfordshire, UK); Dr. Eun Hwang (Indiana University of Pennsylvania, USA); Avi Hyman (University of Toronto, Canada); Dr. Amr Ibrahim (American University of Cairo, Egypt); Prof. Rozhan Idrus (Universiti Sains Malaysia, Penang, Malaysia); Dr. Michael Ievers (Stranmillis University College, N. Ireland, UK); Dr. Marina Ismail (Universiti Teknologi MARA, Malaysia); Issham Ismail (Universiti Sains Malaysia, Penang, Malaysia); Rubeina IsmailAllie (Tshwane University of Technology, Gauteng, South Africa); Dr. Eunice Ndeto Ivala (Cape Peninsula University of Technology, South Africa); Sheila Jagannathan (World Bank Institute, Washington, USA); Prof. Dinesh Chandra Jain (SVITS, India); Dr. Jill Jameson (University of Greenwich , UK); Katherine Janzen (Mount Royal University, Canada); Kanthi Jayasundera (Simon Fraser University, Canada ); Amor Jebali (University of Manouba, Tunisia); Runa Jesmin (Global Heart Forum, UK); Phillip Jones (Hong Kong Institute of Education, Hong Kong); Prof. Leila Kajee (University of Johannesburg, South Africa); Prof. Leila Kajee (University of Johannesburg, South Africa); Prof. Konstantinos Kalemis (National Centre of Local Goverment and Administration, Greece); Dr. Michail Kalogiannakis (University of Crete, Faculty of Education, Crete); Pankaj Kamthan (Concordia University, montreal, canada, Canada); Dr. Haijun Kang (Kansas State University, USA); Saba Khalil Toor (T.E.C.H Society, Pakistan); Dr. Mohammad Ayoub Khan (C-DAC, India); Adrian Kirkwood (Open University, UK); Brant Knutzen (University of Hong Kong, Hong Kong); Dr. Yu-Ju Kuo (Indiana University of Pennsylvania, USA); Prof. Reggie Kwan (Caritas Institute of Higher Education, Hong Kong, China, China); Dr. Hok Yin Jean Lai (Hong Kong Baptist University , Hong Kong); Kamaljit Lakhtaria (Atmiya Institute of Technology & Science, India); Paul Lam (Centre for Learning Enhancement And Research,The Chinese University of Hong Kong, Hong Kong, , China); Prof. David Lamas (Tallinn University, Estonia); Dr. Maria Lambrou (University of the Aegean, Greece); Dr. Mona Laroussi (Institut National des Sciences Appliquées et de la Technologie, Tunisia); Debora Larson (Kaleidoscope Learning, New York, USA); Jno Baptiste Laurelle (OISE/ University of Toronto, Canada); Kenneth Lee (Delaware Valley College, Pennsylvania, USA); Stella Lee (Athabasca University, Canada, Canada); Victor Lee (School of Continuing and Professional Studies,The Chinese University of Hong Kong,, China); Christine Levy (Kaleidoscope Learning, New York, USA); Dr. Ken Li (Hong Kong Institute of Vocational Education, China); Dr. Rita Yi Man Li (Hong Kong Shue Yan University, Hong Kong); Dr. Ying Liu (Cambridge University, Uk); Jenny Lorimer (University of Hertfordshire, UK); Dr. Pam Lowry (Lawrence Technological University, USA); Prof. Sam Lubbe (NWU, South Africa); Dr. Grace Lynch (University of New England, Australia); Prof. Lachlan MacKinnon (University of Greenwich, UK); Maria Madiope (University of South Africa, South Africa); Dr. Chittaranjan Mandal (School of IT,IIT Kharagpur, India); Robert Manderson (University of Roehampton, UK); Phebe Mann (University of Reading, UK); Jorge Martins (Information School, University of Sheffield, UK); Dr. Gianina-Ana Masari (Alexandru Ioan Cuza University of Iasi, Romania); Prof. Hassan Mathkour (King Saud University, Saudi Arabia); Dr. Jeton McClinton (Jackson State University, USA); Dr. Cherifa Mehadji (University of Strasbourg, FRANCE); Dr. Sabita Menon (University of West of England, UK); Mandia Mentis (Massey University, Auckland, New Zealand); Dr. Cecilia Mercado (Saint Louis University, USA); Bente Meyer (The Danish University of Education, Denmark); Sunilkumar Mistry (Johnson Group, Ahmedabad, India); Ali Moeini (University of Tehran, Iran); Sahel Mohammad Esa (Kabul Education University, Afghanistan); Dr. Gholam Ali Montazert (Tarbiat Modares University, IRAN); Dr. Begoña Montero-Fleta (Universitat Politecnica de Valencia, Spain); Dr. Jane Moore (Liverpool Hope University, UK); Jolanda Morkel (Cape Peninsula University of Technology, South Africa); Markus Mostert (Rhodes University, South Africa); Molefe Motshegwe (University of Botswana, Gaborone, Botswana); Dilawer Mowzer (College of Cape Town, South Africa); Manabu Murakami (Tokyo University of Science, Japan); Dr. Minoru Nakayama (Tokoyo Institute of Technology, Japan); Dr. Vincent Ng (The Hong Kong Polytechnic University, China); Dr. Dick Ng'ambi (Uiversity of Cape Town, South Africa); Prof. Roger Nkambou (Université du Québec à Montréal, Canada); Grace O’Malley (National College of ireland, Ireland); Prof. Birgit Oberer (Kadir Has University, Turkey); Dr. Maruff Akinwale Oladejo (Department of Educational Administration, University of Lagos, Akoka, Nigeria, Nigeria); Francisca Onaolapo Oladipo (Nnamdi Azikiwe University, Awka, Nigeria); Dr. Roxana Ologeanu (Universite Montpellier 2, France); Smart Odunayo Olugbeko (Adeyemi College Of Education, Ondo, Nigeria); Prof. Abdelnaser Omran (Universiti Sains Malaysia, Malaysia); Dr. Jacinta Agbarachi Opara.In (School of Science, Federal College of Education(Technical),Omoku, Nigeria); Dr. Addin Osman (Najran University, Saudi Arabia); Maria Osuna Alarcón (Salamanca University, Spain); Dr. Ecaterina Pacurar Giacomini (Louis Pasteur University, FRANCE); William Painter (NCC Education Ltd, UK); Prof. Bamidis Panagiotis (Aristotle University of Thessaloniki, Greece); Dr. Shireen Panchoo (University of Technology, Mauritius, Mauritius); Masouras Panicos (Cyprus University of Technology, Limassol, Cyprus); Dr. Arna Peretz (Ben Gurion Univeristy of the Negev, Israel); Dr. Beth Perry (Athabasca University, Canada); Dr. Donatella Persico (National Reserach Council, Institute fo Educational Technology, Italy); Prof. Selwyn Piramuthu (University of Florida, Gainesville, USA); Dr. Michel Plaisant (University of Quebec in Montreal, Canada); Dr. Carmen Prez-Sabater (Universitat Politcnica de Valencia, Spain); Paul Prinsloo (University of South Africa (Unisa), South Africa); Zahra Punja (University of Toronto , Canada); Anne Quinney (Bournemouth University, UK); Dr. Ronald Robberecht (University of Idaho, Moscow, USA); Dr. Melissa Saadoun (MS Institute , Paris, France); Osman Sadeck (Cape Education Department, South Africa); Dr. S.R. Balasundaram Sadhu Ramakrishnan (National Institute of Technology, INDIA); Khalid Saifuddin (Aalborg University, Denv

mark); Dr. Florin Salajan (North Dakota State University , Canada); Gustavo Santos (University of Porto, Portugal); Prof. Chaudhary Imran Sarwar (Mixed Reality University, Pakistan); Prof. Jeanne Schreurs (Hasselt University, Diepenbeek, Belgium); Dr. Nima Shahidi (Islamic Azad University - Nourabad Mamasani Branch, Iran); Dr. Khitam Shraim (Birzeit University, Ramallah, Palestine); Sibongile Simelane (Tshwane University of Technology, Pretoria, South Africa); Dr. Deena Slockett (ADU (Adventist University of Health Sciences), USA); Dr. Keith Smyth (Napier University, Edinburgh, UK); Dr. Yeong-Tae Song (Towson University, Maryland, USA); Dr. Elsebeth Sorensen (Aarhus University, Denmark); Dr. Mark Stansfield (University of West of Scotland, UK); Juliet Stoltenkamp (University of Western Cape, South Africa); Yana Tainsh (University of Greenwich,, UK); Dr. Ken Takeuchi (Tokyo University of Science, Japan); Dr. Jyothi Thallur (University of South Australia, Australia); Dr. John Thompson (Buffalo State College,, USA); Prof. Ramayah Thurasamy (University Sains Malaysia, Penang, Malaysia); Prof. Christopher Turner (University of Winchester , UK); Karin Tweddell Levinsen (Aalborg University, Denmark); Dr. Sapna Tyagi (Institute of Management Studies(IMS), India); Duan Van der Westhuizen (University of Johannesburg, South Africa); Dalize van Heerden (Unisa, Pretoria, South Africa); Linda van Ryneveld (Tshwane University of technologySouth Africa, South Africa); Prof. Dr. Asaf Varol (Firat University, Turkey); Paduri Veerabhadram (Vaal University of Technology, south africa); Dr. Steven Verjans (Open Universiteit of The Netherlands, The Netherlands); Maggy Minhong Wang (The University of Hong Kong, Hong Kong); Dr. Anita Welch (North Dakota State University, USA); Robert Wierzbicki (University of Applied Sciences Mittweida, , Germany); Roy Williams (University of Portsmouth, UK); Shirley Williams (University of Reading, UK); Dr. Noeline Wright (University of Waikato, Hamilton, New Zealand); Daniel Yakmut (Federal University Lafia, Nigeria); Dr. Ruth Yeung (Institute for Tourism Studies, China); Aw Yoke Cheng (UNITAR International University, Malaysia, Malaysia); Dr. Nabil Zary (Karolinska Institutet, Sweden); Mingming Zhou (Nanyang Technological University, Singapore); Gwen Zilm (University of British Columbia Okanagan, Kelowna, British Columbia,, Canada); Dr. Mitra Zolfaghari (Tehran University of Medical Sciences, Iran); Jacek Stańdo (Lodz University of Technology, Poland); Dr HAMID ALASADI (Basra University, Iraq); Matava Matava (University of Toronto, Canada); Dr Daniyar Sapargaliyev (International Academy of Business, Kazakhstan); Dr Mourad Mars (University of Monastir, Tunisia); Jacek Stańdo (Lodz University of Technology, Poland); Dr Aneta Sokďż (University of Szczecin, Poland);

vi

Biographies Conference Chair Dr Teresita Arenas Yáñez is a professional Commercial Engineer, working in the area of Management, Finance and Accounting. She studied Commercial Engineering PUCV and has MA in Finance from the University of Chile and holds the degree of Doctor of the University of Barcelona Spain. She teaches at the Department of Industry, at the Technical University Federico Santa Maria (UTFSM), Finance and Accounting courses. She has had senior positions in the academic administration of the University, as Head of Career and Academic Director of Campus Santiago. In this moment, her research has focused on knowledge management and intellectual capital, has developed some studies in regions, particularly in the V Region of Valparaiso, along with other authors wrote the book "Towards a new concept of Cluster "and has participated in several international conferences related to the subject.

Programme Co-Chairs Dr Oscar Saavedra Rodriguez is a professional Industrial Engineering, working in strategic management and quantitative finance. He studied Industrial Engineering and Master of Industrial Engineering and Finance at the University of Chile, did an Industrial PhD at the Universidad Polytechnic of Valencia in Spain. He is a professor in the Department of Industry of the Technical University Federico Santa Maria UTFSM, teaching courses in Information Systems Management, Project Evaluation, Strategic Management, Data Mining, among others, has had important positions in the university administration, participated MECESUP projects and in the process of accreditation of the university, led the strategic planning process UTFSM and has been a consultant to small and medium enterprises in Chile. He was assessor FONDEF in CONICYT, and in this moment is director of the Master in Business Management UTFSM. He is working on research in applied Business Intelligence Strategic Management, Data Mining in Finance and Project Evaluation, and applications in biotechnology and bio-business. Paul David Richard Griffiths, BSc, MEng, DBA, A.Dip.C is a researcher and practitioner in the strategy, technology, knowledge and intellectual capital (IC) spaces. As a practitioner he helps organisations do their strategic planning, manage information and set up knowledge communities. On the research side, he is interested in the pre-history of IC and how its development connects to the mind. Paul holds a Master’s degree in engineering from Sheffield University; he has been a Humphrey Fellow (Fulbright Commission) at the University of Minnesota; and holds a doctorate in business administration from Brunel University. He is a prolific writer in professional and academic publications and a renowned speaker at conferences and seminars.

Mini Track Chair Dr Susan Crichton has taught in rural and urban K-12 schools. She is a visiting professor with Aga Khan University – Institute of Educational Development, Dar es Salaam, Tanzania and a Fellow of the Commonwealth Centre of Education, Cambridge. Dr. Crichton has worked on development projects in rural western China, and as a consultant with academics in Bhutan and Chile. She is an online mentor for the United Nations Institute for Training and Research (UNITAR) project in Afghanistan. Her research explores innovative uses of technology to foster creativity and imagination. She works with colleagues in challenging contexts exploring ways appropriate technologies can provide access to professional development and learning.

vii

Biographies of Presenting Authors Liz Bacon is President of the British Computer Society (Chartered Institute for IT) 2014, having been deeply involved in the development of her profession for the last twenty years. She has been involved in software engineering and eLearning research for more than 10 years, this includes work on personalisation, serious games, and enterprise development. Wendy Barber is the Director of the B.Ed. Program at the University of Ontario Institute of Technology in Oshawa, Canada. Her research interests lie in Health and Physical Education, and Creating Online Communities. Dr. Barber is a passionate advocate for teacher education, teaches Authentic Assessment and Adult Education, and Psychological Foundations in Digital Technology. Moses Basitere is a Lecturer at the Cape Peninsula University of Technology, Chemical Engineering Department. He is currently lecturing in Mathematics and Physics in the Extended Curriculum Program. His research area is in industrial wastewater treatment and he is also involved in educational research in promoting the use of emerging technology in teaching and learning in higher education. Patricia Beckenholdt is the Academic Director of the Business Administration and Management Studies programs at University of Maryland University College (UMUC), USA. Dr. Beckenholdt earned a B.A. in Business with an emphasis in Finance from the College of Notre Dame, an MBA from Johns Hopkins University, and a doctorate in management from UMUC. Paula Charbonneau-Gowdy is professor of English as a Foreign Language in teacher training at the Universidad Andres Bello, Santiago, Chile. She worked formerly as Senior Advisor in Learning and Technology to the Government of Canada and has published extensively in learning and technology while seeking to push the pedagogical boundaries of technology to promote democratic change. Phaik Kin Cheah is the Dean of the Faculty of Arts and Social Science at the Universiti Tunku Abdul Rahman, Malaysia. Her research interests are in the areas of online instruction, teaching and learning, education policy, and community engagement. Ricardo da Rocha is a lecturer in ICT department at the Vaal University of Technology (VUT). Ricardo holds a BSc: Honours in IT from Northwest University and is currently busy with his Master’s degree in IT titled ‘The use of social networking services to enhance the learning experience and academic performance of tertiary level learners’. Nuno Dominguinhos is a Postgraduate in Technologies and Methodologies in e-Learning and Graduate in Information Technology, and Project Leader in the Regional Administration of Health, Lisbon and Tagus Valley. He has written two articles and presented two communications related with the e-Learning platform at health organizations thesis he has written. Stylianos Hatzipanagos is Senior Lecturer (Centre of Technology Enhanced Learning) at King's College London. His research portfolio includes: learning design, formative and technology-enhanced assessment, computer supported collaborative work, open educational resources, social media and social networking. His output is peer reviewed articles, books, edited journal special issues and he has led and participated in research projects at a national (UK) and European level. Maggie Hutchings, Associate Professor, School of Health and Social Care, Bournemouth University, is an educational researcher/practitioner interested in transformative learning, practice-based and interprofessional education; supervising doctoral students in professional practice development and technology-mediated learning, leading an interprofessional module using technology to connect learners to evidence-based practice for humanising care; supporting innovation in learning, teaching, assessment. Thandokazi Euthodora Ikedinobi holds an MSc: Computing degree with Merit from Coventry University in the United Kingdom. She is currently working at Walter Sisulu University in South Africa as an E-learning Specialist. She intends to further her studies for a PhD in E-learning. Borka Jerman Blažič is working as a head of the Laboratory for Open Systems and Networks at Jožef Stefan Institute, Slovenia and as a full professor at the University of Ljubljana, Faculty of Economics in Slovenia. The main field of applications and research are computer communications, internet technologies, security in networking, privacy and internationalisation of Internet services. Thomas Kjærgaard is a PhD student at Aalborg University, Denmark. He has worked in the field of e-learning and learning in technology rich environments since 2001. He is associate professor and leader of educational ict at The Teachers Collage UCN Aalborg, Denmark. His main field of interest is learning in networks. viii

Lachlan MacKinnon is a Professor of Computing Science (Strategic Development) and Head of Computing at the University of Greenwich, UK. He is deeply involved in national developments of his subject, particularly through CPHC and BCS. His research interests are in computing policy, information and knowledge engineering, smart systems, games technologies, eHealth and eLearning, and computer security. Peter Mozelius has since 1999 been employed as a teacher at the Stockholm University in Stockholm, Sweden. He is currently working in the Department of Computer and Systems Sciences as an IT-pedagogue and researcher. Research interests are in the fields of ICT4D, e-learning, and game based learning. Maria Nathan graduated from the University of Southern California, Los Angeles, United States. She has been a Professor of Management and Business Administration in the School of Business and Economics, Lynchburg College, Lynchburg, Virginia, since 2000. Her areas of research expertise include strategic management, nonprofit management, and crisis management. Timothy Olson is a Senior Lecturer at the University of Minnesota in the Information Decision Science Department. He has been teaching information system courses successfully using e-textbooks and wiki sites for several years. Tim has published several articles and numerous presentations on e-learning, enterprise system implementation and team building projects. David Owen was a Business Manager for the UK power company, PowerGen and now works at the University of Bedfordshire. He is the Course Leader for a Project Management course which has face to face, blended learning and a new online option. He has a PhD in Chemical Engineering and a Warwick University MBA. Chris Perumalla is an Educator in the Department of Physiology and the Director of the Division of Teaching Laboratories in the Faculty of Medicine, University of Toronto, Canada. He lectures in three physiology courses and coordinates physiology laboratory courses, which are designed to translate conceptual learning to hands-on experiences and develop problemsolving strategies. Stanislav Popelka is a PHD student in the Department of Geoinformatics, Palacký University in Olomouc, Czech Republic. His dissertation is focused on the evaluation of 3D visualization in maps with use of the eye-tracking system. He is a member of the ICA Commission on Cognitive Visualization. Brenda Ravenscroft, Ph.D., is Associate Dean in the Faculty of Arts and Science at Queen’s University, and is responsible for teaching and learning initiatives in the faculty, including a large-scale, outcomes-based blended learning project to transform introductory courses, the expansion of online studies through Continuing and Distance Studies, and curricular quality enhancement. Lise Renaud, sociologist and Ph.D. Educational Sciences, is Professor of Social and Public Communication at Université du Québec à Montréal and Director of ComSanté. Her current research programs, supported by a four-year grant of Social Sciences and Humanities Research Council of Canada, focus on digital games and older adults. Florin Salajan is Assistant Professor in the School of Education at North Dakota State University, Fargo, USA, where he teaches undergraduate and graduate courses in the Teacher Education Program. His areas of research interests are European higher education policies, international and comparative education, comparative e-learning, interactive learning technologies, and educational technology effectiveness for teaching and learning. Daniyar Sapargaliyev is Deputy Director of the Center for research and development at the International Academy of Business in Almaty. He received his PhD from Eurasian National University. His research interests include mobile learning and using mobiles in education. He has written publications in the refereed books, journals and conference papers. Thorsten Sommer is a research assistant and Ph.D. student at IMA/ZLW & IfU, RWTH Aachen University in the research group "didactics in the STEM fields". He studied computer science (diploma) and electrical and information techniques (master) and has founded a company for the project "MathComm!", an e-learning platform for higher mathematics. Qing Tan is an associate professor in School of Computing and Information Systems at Athabasca University. Dr. Tan’s background is in computer information systems, control engineering, and aviation navigation. Dr. Tan got his PhD at the Norwegian Institute of Technology in Norway. Dr. Tan also worked in IT industry for over ten years in Canada. Dalize van Heerden is a Junior Lecturer within the School of Computing at the University of South Africa. She started working for UNISA in 1999 and has been teaching programming modules for the past 15 year. Her main research interests include elearning, m-learning and technology-enhanced learning. She is currently working on her Honours in Science Education.

ix

Roxanne Ward Zaghab, DM, CKM, is with the Center for Innovative Pharmacy Solutions (University of Maryland School of Pharmacy) as the Director of the Knowledge Enterprise, a robust online learning platform. As Assistant Professor with University of Maryland University College, she has designed curriculum and taught online in Knowledge Management and Health Care, USA. Xiangmin Zhang is an assistant professor with the School of Library and Information Science, Wayne State University. His research interests include human-computer interaction, personalization techniques, information retrieval, digital libraries, and e-learning. He frequently presents at international conferences and has published extensively in the area of information sciences.

x

E‐Learning has low Adoption in Chile: Is This Related to Learning Styles? Teresita Arenas1 and Paul Griffiths1,2 1 Universidad Santa Maria (USM), Chile 2 Birchman Group, Chile and Visiting Faculty, Henley Business School, UK [email protected] Abstract: Through the authors’ personal experience, it appears that e‐learning courses do not have significant traction in Chile despite there being a robust technology infrastructure and a significant adoption of the internet and broadband. This is particularly notable since Chile’s economy has developed world‐class industrial clusters where many professionals work in isolated locations (e.g., salmon farming, wine production, agribusiness, forestry and mining.) There may be many explanations for this phenomenon, such as the lack of credibility of distance learning amongst employers, the need for learning in a social environment, or other cultural issues. However, the specific purpose of this paper is to explore if there is a relationship between cognitive styles in Chile and an inclination to enrol on e‐learning programmes. The research method is quantitative and applies a learning‐style survey instrument to two groups of graduate professionals. One sample comprises professionals who frequently enrol on e‐learning courses for their professional development, while the other one represents the population of evening students at USM engineering and management school. The result of the study is that, on the one hand, learning styles do not differ significantly for students over 25, whether they tend to take face‐to‐ face courses or distance learnging courses. On the other hand, it emerges that postgraduate evening students improve their learning experience with varied teaching methods. Thus graduate programmes would benefit greatly if they shifted from full face‐to‐face programmes to incorporating some measure of distance learning. It would increase their reach and benefit participants by introducing different teaching methods. Keywords: e‐learning, learning styles

1. Introduction According to statistics shown in the year 2012 by Internet World Stats, the lowest percentage of internet penetration is in Africa, reaching only 15,6%, and the highest percentages are in Europe, with 63,2%, and North America with 78,6%. Latin America shows penetration of 42,9%, and Chile in particular is one of the countries where the internet has penetrated most strongly, reaching 53,9% in 2012. This scenario is quite a promising sign that the country is advancing rapidly towards a knowledge society. Nevertheless, if we analyse internet use, we observe that users prefer to search for entertainment, social networks and related topics (90%),rather than a productive use associated with banking, commercial or other transactions (35%) (Agostini & Willington, 2012). Where internet use in relation with teaching or training programs is concerned, we have not found any significant advances. In Chile, despite this successful scenario in which internet use is among the highest in Latin America, progress has not carried through forcefully enough into the educational sector. The country has been promoting this area through training programs. In 2013, more than 50% of SENCE training programs were carried out via e‐ learning, and in the opinion of businessmen this program has proven quite successful. Nonetheless, undergraduate education in Chile has advanced slowly, and has used the tools of technological information only as a support to formal education, constituting a dual education system. In this system of e‐learning Argentina is the country with the largest number of universities that possess distance learning and virtual th classrooms as a support to formal education. Then come Brazil and Mexico, with Chile appearing only in 6 place. In this regard, some authors claim that cognitive styles and learning styles influence the success of a student when he or she encounters e‐learning methods versus classroom‐based ones. Therefore they could set the standard for the success and permanence of in‐person or virtual study programs. The present investigation therefore intends to identify the learning styles presented by people who use traditional methods of learning, and compare them with the average learning styles of people that use e‐learning methods. In order to do so, we shall use Kolb’s test in this investigation.

1

Teresita Arenas and Paul Griffiths

2. Theoretical background Learning styles, understood as the development of specialized abilities and preferences for learning(D. Kolb, 1984), differ between individuals, because it is the way in which people perceive, interact with and respond to their learning environments. The components of the style of learning are cognitive, affective and psychological elements which are strongly influenced by the individuals’ cultural background. The citation analysis made by Desmedt and Valcke(Desmedt & Valcke, 2004) concludes that Kolb is the most quoted author in the literature of learning styles. For this reason, among the different models of cognitive styles that prevail, Kolb’s model designed in 1984 is the chosen one for use in this investigation. At the Kolb’s model, learning styles are composed of four key components as learning abilities: concrete experience (CE) (A. Kolb & Kolb), abstract conceptualization (AC) (Agostini & Willington), reflective observation (RO) (Abdul‐ Rahman & Boulay), and active experimentation (AE) (Yamazaki, 2010). The CE emphasizes “feeling“ and is interpreted as the ability that involves grasping immediate experience through sensing and feeling, producing tacit knowledge (D. Kolb, 1984); (A. Kolb & Kolb, 2005). In contrast, the AC emphasizes “thinking” and is the ability which, in dialectical relationship with the CE ability, entails generating concepts and ideas as explicit knowledge through logical thinking and analytical reasoning. Although the CE and AC abilities work to create two different types of knowledge, the RO and the AE abilities play an important role in transforming one type of knowledge into the other. The RO requires reflecting on immediate experience in the form of tacit knowledge by observing it from various perspectives within the self. The focus of RO is on an introspective approach to transform tacit knowledge into explicit knowledge by figuring out the meaning of such experience. In contrast, the AE emphasis is on “doing” and generating abilities, which are dialectically opposite to those of RO. It necessitates taking action to test explicit knowledge generated by the abilities of AC. This action then leads to an individual’s new experience as a source of tacit knowledge created by the CE abilities. The AE abilities thereby serve to transform explicit knowledge into tacit knowledge through individual action. Kolb’s model indicates that initially an individual develops the specialisation between: concrete experience (A. Kolb & Kolb) and abstract conceptualization (Agostini & Willington), which he calls the dialectic dimension of learning, and also develops the specialization between: reflective observation (Abdul‐Rahman & Boulay), and active experimentation (AE), which he calls the learning dimension (D. Kolb, 1984). Consequently various combinations can be developed that definitively generate four basic styles of learning; these are the ones that can be seen in table Nº 1. Table 1: Learning style Learning style Logical analytical – theorist (Assimilator)

Learning characteristic Abstract conceptualization (AC) and reflective observation (RO), thinking and watching

Strengths/skills

Strong ability to form theoretical models.

Induction ability. Higher interest in abstract models than in people Strong imaginative skill. Learns by concrete experience (A. Kolb & Kolb) Imaginative type – reflector Cerates new interested in people. and reflective observation (RO) feeling and ideas, (Diverger) sees new perspectives, watching Broad cultural interests

Pragmatic (Converger)

Activist (Accomodator)

Strong impractical application of new ideas. Abstract conceptualization (AC) and active Concentrated on deductive reasoning experiment‐ting (AE) thinking and doing on specific problems. Unemotional. Narrow interests. Ability to realize things. Risk taking. Concrete experience (A. Kolb & Kolb) High performing when needed to react and active experimenting (AE) feeling and doing to changed conditions. Solves problems intuitively.

Source: (Knoskova & Jurkovicova, 2012)

2

Teresita Arenas and Paul Griffiths In recent years these styles of learning have been object of different investigations. While revising the literature we have found various works where Kolb’s method has been used, and would like to highlight those which are more directly related to our investigation. In Columbia, research indicates that the diverging style of learning is predominant in students who have taken 100 % online courses (Romero, Salinas, & Mortera, 2010). In Peru an investigation carried out with undergraduate students using the remote learning system showed that older students (between 26 and 40 years of age) do not present clear differences of styles of learning,whereas younger students (between 18 and 25 years of age) are inclined towards the converger style, meaning that abstract conceptualization (AC) and active experimentation (AE) are predominant. An investigation carried out with students from the University of Qatar, (Manochehr, 2006), showed that studentswith the Assimilator style of learning (those who learn best hrough lecture, papers and analogies) and the Converger style (those who learn best through fieldwork, observing and working in laboratories) didbetterwiththe e‐learningmethod. This means that those who like to learn through thinking and watching, and thinking and doing, would learn better with e‐learning. In addition, students with the Accommodator learning style (those who learn best through simulations and case study) and the Diverger learning style (those who learn best through brainstorming and logs) received better results with traditional instructor‐based learning. Against this background, we can ask: do evening students from the USM in Chile present clearly defined styles of learning? Do these styles concur with the results of other investigations in other countries? Are there differences of learning styles between the group that attends classroom‐based clases and those surveyed who participate frequently in online studies? In the following section, we shall attempt to answer these and other questions.

3. Methodology and results This is an exploratory study, in which the Kolb test has been applied to two samples.The first is made up of students who are actually studying a second professional career in the USM, meaning that they are students who already have a profession and decided to study a second career in a classroom‐based way of learning, during the evening. The second is made up of professionals who work in businesses where they are also being frequently trained online. As mentioned, the objective is to analyze whether different styles of learning exist between people who follow face‐to‐face studies and those whose training involves distance or online learning. Initially we collected 108 answers of which 47 were from businesses and 61 from students of the University (USM). Approximately 30% of the surveys were incomplete, so after a first analysis we were left with a sample of 70 survey respondents, of which 51% were from businesses and 49% from the University. 35% of those surveyed were female and the average age was 33 years old. One hundred percent of the business professionals took online courses during 2013, against only 15% of the students from the University. In order to analyze the reliability of the answers we obtained Cronbach’s Alpha. This proved to be very close to 70%, which is considered to be acceptable. Subsequently, and making use of Stata software (version number 20), we analyzed differences in averages with the aim of testing the hypotheses suggested in the investigation. The following are the principal results. We shall first analyze the bases of the learning styles, which are the learning abilities, namely the characterization of concrete experience(CE) (A. Kolb & Kolb), abstract conceptualization (AC), reflective observation (RO), and active experimentation (AE). We shall consider them in terms of those polled and also in terms of the two groups under analysis. Figure 1 shows that the professionals on average have higher scores in three out of the four learning abilities. The exception is AE (active experimentation). As we can also observe, there are no clear preferences among students for one or other type of learning abilities. This can be explained by the age of the sample (33 years

3

Teresita Arenas and Paul Griffiths old on average) and the fact that all of those studying are doing so for their second degree at least. These findings concur with results found by Blumen in 2011. 17 16,5 16 15,5 15 14,5 14

AE

CE

AVERAGE USM RO

AVERAGE PROF

AC

Source: analysis of survey results Figure 1: Learning abilities When analyzing the difference of averages between the two sample groups, the t‐test did not prove significant, meaning that for each one of the learning abilities the sample showed the same learning abilities, whether they were taking face‐to‐face courses or were professionals who normally use distance learning. We can also observe that the maximum average score is in AE learning skill, meaning the ability of active experimenting, thinking and doing which permits abilities to form, thereby serving to transform explicit knowledge into tacit knowledge through individual action. Resuming the analysis, we shall use the method designed by Kolb to examine the aspects of initial specialization among the sample, in relation to what he called the dialectic dimension of learning. We shall determine whether those surveyed are abstract or concrete (AC ‐ CE) and on the other hand their learning dimension, identifying individuals as active, experimental or reflective (AE ‐ RO). From the analysis of figure 2, a strong inclination emerges among USM students towards active, experimental abilities, just as with the characteristics of those students who take traditional or on‐site courses. It is worth noting that the gaps or specializations are lower among professionals who are not studying: this can be explained by the findings of Blumen in 2011, vis‐a‐vis greater age and work experience. 2 1,8 1,6 1,4 1,2 AVERAGE USM 1

AVERAGE PROF

0,8 0,6 0,4 0,2 0 AC ‐ CE

AE‐ RO

Source: Analysis of survey results Figure 2: Dimensions of specialisation We come finally to analysis of the learning styles. Kolb differentiates four of these: assimilator, converger, accommodator and diverger. Figure 3 shows that the group not studying in the USM reaches higher scores in every one of the styles. In this group, no clear tendency can be noted towards any one style, whereas the USM students tend towards the accommodator and converger styles of learning.

4

Teresita Arenas and Paul Griffiths These results partly concur with empirical evidence from other countries, where it has been found that the accommodator style of learning is characteristic of students in on‐site programs (Manochehr, 2006),(Blumen, Rivero, & Guerrero, 2011).

ASSIMILATOR 33 32 31 30 ACCOMODATOR

AVERAGE USM

29

DIVERGER

AVERAGE PROF

CONVERGER

Source: Analysis of survey results Figure 3: Styles of learning

4. Discussion, conclusions, limitations and further reasearch The findings of this exploratory investigation indicate that, as with the results gathered by other investigators, learning styles are not clearly differentiated in people older than 25 years of age, whether they are taking on‐ site courses or are accustomed to remote learning. Even if differences can be seen in both groups in the graphics, the test of average differences in the four Kolb learning styles is not significant. The prior point makes it even more surprising that in a country like Chile where there is a significant penetration of internet and broadband connectivity, and where remote learning systems are well engrained for corporate training in certain areas, that this medium for teaching has not been adopted for teaching in the post‐graduate university system. These exploratory results appear to indicate that on one hand academics, when teaching evening courses with continuing education students, should use a variety of resources of teaching to offer students the best results. They should not confine themselveselves, for example, to the accommodator style which is seen most often with on‐site students. And on the other hand, given the age differences in the daytime and evening students, they should not use the same teaching techniques when delivering lectures in both groups, since it is quite probable that daytime students display a certain style of learning. The latter should be the subject of future research. It is also important to mention that even though this is not a conclusive study, the findings show that from the perspective of the learning styles of these evening students, the programme could also be taught at a distance or in dual mode, and not exclusively on‐site which is what happens nowadays. This would allow professionals all around the country to enjoy further training opportunities in prestigious teaching establishments. The final point is that in the university postgraduate learning environment, there is still a great deal to investigate. Until now the national emphasis has been put on basic educational levels and secondary school teaching in Chile’s compulsory school system. Even if we have found a few studies at undergraduate level, these encompass young people under 25 years of age: we have not found major studies dealing with learning styles at postgraduate level, which is why this investigation constitutes a real contribution to university teaching.

References Abdul‐Rahman, S.‐S., & Boulay, B. d. (2014). Learning programming via worked‐examples: Relaton of learning styles to cognitive load. Computers in Human Behavior, 30, 286‐298.

5

Teresita Arenas and Paul Griffiths Agostini, C., & Willington, M. (2012). Acceso y uso de Internet en Chile: evolución y factores determinantes. Persona y Sociedad/ Universidad Alberto Hurtado, XXVI(1), 11‐42. Blumen, S., Rivero, C., & Guerrero, D. (2011). Universitarios en educación a distancia: estilos de aprendizaje y rendimiento académico. Revista de Psicología, 29(2), 225 ‐ 243. Desmedt, E., & Valcke, M. (2004). Mapping the learning Styles "Jungle": An overview of the literature based on citation analysis. Educational Psychology: An international journal of experimental educational psychology, 24(4), 445 ‐ 464. Knoskova, L., & Jurkovicova, L. (2012). Learning styles and Digital Literacy for Innovation in Designing E‐learning Courses. International journal of economic practices and theories, 2(2), 81 ‐ 90. Kolb, A., & Kolb, D. (2005). The Kolb learning style inventory ‐ Version 3.1 2005 Technical Specifications. Kolb, D. (1984). Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall. Manochehr, N.‐N. (2006). The influence of learning styles on learners in e‐learnig environments: an empirical study. CHEER, 18, 10 ‐ 14. Romero, L., Salinas, V., & Mortera, F. (2010). Estilos de aprendizaje basados en el modelo de Kolb en la educacion virtual. Revista Apertura, 2(1). Yamazaki, Y. (2010). Impact of learning styles on learning‐skill development in higer education. In I. U. o. Japan (Ed.), GSIM Working paper. Japon.

6

A Flexible Framework for Metacognitive Modelling and Development Liz Bacon 1 and Lachlan MacKinnon 1, 2 1 Department of Computing and Information Systems, University of Greenwich, London, UK 2 Buskerud & Vestfold University College, Norway [email protected] [email protected] Abstract: Research in eLearning and technology enhanced learning (TEL) has predominantly focused on the creation of learning materials in appropriate forms, such as learning objects, the assessment methods that can usefully be applied online, and the delivery mechanisms for these materials, particularly in virtual learning environments (VLEs). In more recent times, research has begun to focus on pedagogical issues, and in particular whether there is some specific model that applies explicitly to online learning situations. Through a number of projects over the last ten years the authors have considered issues of learning style, learning strategy, pedagogy, immersive environments, student engagement and motivation, games‐based learning, adaptation and personalisation. Emerging from this work, and from extensive consideration of the existing research in this area, this paper argues a need to move not only to a different pedagogic model, but also to change the existing structural approach to learning to support the rising demand for online distance learning provision worldwide. Fundamental to this argument is a need to support a heutagogic model of student learning, which requires that the students involved are sufficiently educationally mature to take control of their own learning experience. Whilst within traditional teaching models in higher education there is an explicit aspiration that students will emerge as educationally mature, metacognitive graduates, this is often seen as an outcome of the learning process itself, rather than as a skillset which can be taught and assessed. The paper describes an approach to metacognitive assessment that has already been used to determine the level and skills displayed by students in making selections of learning materials online. Based on this approach, a structural model for online learning support is proposed, using an assessment, feedback and training loop to ensure that students have the level of metacognitive skills necessary to take effective control of their own online learning experience. Keywords: metacognition, learning strategies, heutagogy, personalised learning, technology‐enhanced learning

1. Introduction In recent times, there has been a huge explosion in the demand for online higher education for several reasons:

Higher education in many countries in the developing world far outstrips supply and although governments recognise the need to educate more of their population to help their countries move out of poverty, for most, their ability to deliver this in the near future is limited, however the education is needed now. Whilst many students from developing countries study abroad, the majority cannot afford it. In the short term at least, online distance education offers one solution to help address this issue.

Online education is not only for those studying at a distance. Students now arrive at university immersed in technology and they expect that technology to support them through their education in the same way as does in their everyday lives. Many students take online courses as part of their on campus studies, as reported by Hachey et al. (2012), in the US in 2010, 30% of students took an online course at some point during their college career and the trend is increasing. However, online education is not just about studying an online course it is also about supporting face‐to‐face delivery to provide an enhanced, innovative, enjoyable, immersive and personalised educational experience.

Internet bandwidth, worldwide, has increased by an order of magnitude over the past 10+ years, providing the capacity to offer video etc. to many parts of the world where it previously either wasn’t available or where the bandwidth made it not viable. This additional bandwidth is now providing us with the capacity to completely revise our online educational models to support more effective, efficient and personalised online teaching.

Moreover, it can also be argued that this represents the opportunity to move to a new model for higher education attuned to our technological age, and to optimise the benefits of technology‐enhanced learning to make a significant step forward in education provision. This challenges the currently accepted role for online and distance education as subsidiary and inferior to traditional learning models.

7

Liz Bacon and Lachlan MacKinnon All of the above is good news so exactly what is the problem? The success of online education is, in the majority of cases, considerably worse than a typical on campus educational classroom experience where students are co‐located for their learning. They have been shown to be particularly poor for MOOCs (Massive Open Online Course), the success rate of which is often at less than 10%. There are many reasons for this and the literature has identified many factors, for example: prior experience of online courses; age; gender; ethnicity; motivation; intrinsic interest in the subject; personal and financial issues; learning styles and learning strategies; social and cultural characteristics including the ability to make friends online; the provision of engaging and immersive environments; and the retention of some physical contact with staff and other students i.e. a more blended approach to learning than completely online [MacKinnon and Bacon 2014]. However there is one characteristic that is key to success, without which, a student is unlikely to achieve and will certainly not perform at their best, that is, metacognition. In this context we define metacognition as the knowledge someone has about how they personally learn and can include strategies about when and how to use particular learning techniques, and how to make selections between learning objects and materials, i.e. someone’s “learning strategy”. In face‐to‐face education metacognition is not something that is generally taught, it is somehow expected to emerge as students progress through their education and by the time they reach higher education, it is assumed that students are generally metacognitive however, many are not. Despite this fact, most students pass their degrees in a standard classroom setting. Whilst they might not perform at their best, a traditional classroom style, structured form of education is one they are familiar with and have experienced all their life so most have learned to cope by developing learning strategies to deal with the variety of teaching styles that they have experienced. The world of online education is however, very different. Students are often confronted with a range of materials in a variety of forms such as text, graphics, animation, audio, video and games. Student educated in a traditional classroom setting can encounter several problems with this for example:

They may have limited experience of using these types of materials for learning.

They may be given a free choice of materials to use, which can be confusing.

Because of the nature of the web, learning materials maybe presented in a nonlinear way i.e. there may be more than one sensible route through the learning materials from which they have to choose (Azevedo, Cromley and Seibert 2004; Mulwa et al. 2010). Having been used to a more traditional style of education where the lecturer provides the suggested sequence of materials in order to learn, students can become confused having to decide the order in which to tackle the learning materials.

The result of this type of online education is that it requires students to take more control over their learning, as it requires a more andragogic/ heutagogic, as opposed to pedagogic, model of learning and therefore works best for students who are self‐directed, motivated, well‐organised and strategic in their learning. To quote, Blaschke (2012) says, “Learner autonomy is characteristic of and promoted in distance education learning environments, distance education inherently supports heutagogical practice”. However, in an online environment, where students are faced with decisions about how to learn, weaker, less metacognitive students, are likely to flounder more than in a traditional face‐to‐face, organised teaching environment, as this type of learning can accentuate the problem when metacognitive skills, coupled with the ability to regulate themselves and their learning are required to succeed. That said, having these skills is not a guarantee of success (Schunk 2008). Good students will always generally succeed however they are taught but current educational practices are not developing metacognitive skills in the majority of students to the level required and we are therefore failing in our practice. The rest of this paper first of all reviews the literature on metacognition and then discusses a framework for developing metacognitive learners, which brings together the work of the authors and their PhD students in this area. Before concluding it reflects on the future of online learning given the potential for the development of digital monitoring environments, which could revolutionise our educational models and approach to assessment.

2. Metacognition As discussed above, the definition of metacognition in this context refers to the ability to understand how someone learns and this includes the ability to develop learning strategies to cope with different styles of teaching, understanding how to find a sensible route through a range of learning materials and know which types of learning materials work best, in what order to tackle them etc. As also identified above, this is

8

Liz Bacon and Lachlan MacKinnon something rarely taught in education, students go through the learning process, most having been taught through traditional face‐to‐face models where a teacher stands at the front of a classroom imparting their words of wisdom. This is then typically followed by some practical discussion or application of the learning. These traditional 19th century models have worked reasonably well in the past however they have not served some students well at all who do not learn in that way, or who have been unable to develop learning strategies to progress. They also do not embrace the potential benefits that online learning could offer. Many people often think of online learning a poor second to face‐to‐face teaching, partly because in many cases the pedagogy from the classroom is simply translated badly to the online environment. An online environment can however provide learning experiences that are simply not possible to deliver in a classroom. For example, the use of simulations such as the ability to mix chemicals together and experiment safely as though in a chemistry lab or learning through computer games which can be an immersive, fun and engaging learning experience [Kazimoglu et al. 2012]. Much of the literature on metacognition does not directly address how to develop it in learners. This is however a complex area that can only be summarised here and it should be noted that being metacognitive alone does not offer a guarantee of success (Efklides 2011). Both Efklides (2011) and Boekaerts (1996) argue that metacognition, motivation, affect and volition are all part of self‐regulated learning and the ability to undertake self‐regulated learning is itself a form of metacognition. It also includes the ability to translate knowledge, skills and attitudes from one learning environment to another (Boekaerts, 1999). Efklides (2011) agrees with other authors that metacognition has many facets. There is however, no single agreed definition for these but by way of an example, metacognition could be categorised into the following:

Metacognitive knowledge, which is important for the control of cognition and comprises declarative, procedural and conditional knowledge, self‐efficacy beliefs, theories, achievement goal orientations etc.

Metacognitive strategies or skills, which influence the control of cognition and involve activities such as planning, orienting, monitoring, checking, selecting, revising, evaluating, self‐monitoring and self‐ evaluation. Their purpose being to ensure that a goal is met.

Metacognitive regulation, which regulates the use of metacognitive strategies.

Metacognitive experiences, which are current and past experiences such as feelings of difficulty.

According to Boekaerts & Niemivirta (2000) and Boekaerts (1996), self‐regulated learning involves three layers: regulation of the self, which is about individual choice of goals and resources, regulation of the learning process which is about using metacognitive knowledge, strategies and skills to direct the learning and regulation of processing modes i.e. metacognitive regulation in which cognitive strategies are selected. In addition, metacognition, self‐regulation and self‐regulated learning are all influenced by a multitude of other factors such as personal characteristics, physiological and emotional states etc. The literature has shown a number of ways that metacognition can be developed and supported, more recent literature referring to the development of heutagogic learners who must possess many of the desired characteristics of metacognition, self‐direction etc. if they are to be independent learners (Blaschke 2012). Below summaries a few key examples from the literature of how metacognition can be developed:

Woolfolk and Margetts, (2007), as part of the educational psychology literature have also questioned whether individual metacognitive abilities are as a result of biological differences or different learning experiences, i.e. nature vs. nurture. In terms of biological differences, the research on this is unclear however several researchers have demonstrated that metacognitive skills training and support for self‐ development can help (Wagster et al. 2007 and Gunter et al., 2003), in addition to the fact that students develop metacognitive abilities as part of their usual learning and observation experiences (McInerney and McInerney, 2006).

Azevedo and Cromley, (2004) argue the importance of metacognition. Their research shows that not all students have the ability to regulate and manage their learning, and deploy relevant strategies at the right time, or monitor their own progress etc. The presence of a tutor in a technology‐enhanced environment has been shown to assist with the development of metacognition (Azevedo and Cromley 2004).

Kirsh, (2005) argues the need for a well‐designed learning environment with a good visual design that provides an appropriate structure, well‐written easy to understand sentences which require less cognitive effort to comprehend, ensuring that support tools, such as chat tools, are easily visible to students and

9

Liz Bacon and Lachlan MacKinnon links to learning materials and are not missed, can all make a significant difference to the effectiveness of metacognitive development.

3. A flexible framework for metacognitive development and modelling In this section we describe the development of a framework for metacognitive modelling, which brings together successful research from a number of projects. These research projects and their results are summarised below and the section concludes with a description of the framework explaining how the research is brought together to assess and develop metacognitive skills in students.

3.1 Use of tagging to support the authoring of personalisable learning content This section describes a research project that focused on the personalisation of eLearning platforms. Whilst many learning platforms provide some means of personalisation, analysis of platform features revealed that most of these were fairly superficial e.g. changing the colour of the user interface. Nine of the most commonly used learning platforms were evaluated against a number of personalisation criteria and none of them were considered to offer a truly personalised experience. The analysis included an assessment of the instructor’s ability to monitor learners, manage and sequence course material, a learner’s ability to search for learning objects, whether the system can structure learning materials according to need, and the ability of the system to adapt the user interface, and adapt to a learner’s goals, behaviours and learning styles etc. A conclusion from the analysis and background literature was that a solution based on a tagging system, utilising concepts from adaptive hypermedia systems, was required. A system was developed to allow authors of material to tag learning objects according to their subject, topic, learning style, level, object file type and object resource type. This ensured that multiple representations of learning objects were kept consistent, supported their reuse and provided a mechanism to aid personalised learning (Peter et al. 2010). The focus of the system as developed was for use by authors only, future developments will carry this concept further to enable student tagging for their own use as well as other students. In this research, the discriminatory model used was that of learning styles based on the VARK (Visual, Aural, Reading, and Kinaesthetic Learning) [Fleming 2001] learning style. Whilst the research demonstrated, through expert and author evaluation, that learning styles could indeed be used successfully as a discriminatory type, it was acknowledged that this might not be the best discriminator however, there weren’t any serious alternatives to use in order to discriminate between different types of learning materials. The methodology and tags devised are however flexible and could be applied to any discriminatory concept considered appropriate. This research supports the use of discriminatory tagging in the learning repository as part of the framework.

3.2 Categorisation of learning materials by learning style and their support for metacognitive development This section describes two research projects that have both used learning materials, categorised by a specific learning style model, in different ways to explore the metacognitive skills of students. In the first research project students were asked to answer a questionnaire to determine their learning style. In this case the Felder and Silverman Learning Style Model (FSLSM) (Felder and Silverman 1988) was used. Students were then provided with the learning materials for all of the categories of FSLSM learning styles. Immediately after their learning was complete, they were asked to take a recall test and then two weeks later they were asked to undertake a retention test. The students were then divided into two cohorts, those who chose materials consistent with their learning style (the matched group) and those who chose materials inconsistent with their learning style (the mismatched group). In order to determine which group performed better, an analysis of some external factors that might have affected the learning, such as prior knowledge of the subject, were assessed and taken on board in drawing conclusions from this study. Key findings from the experiment were that statistically significant achievements in learning were demonstrated by both the matched and mismatched groups in different areas. The research showed that the matched group understood their learning style and were therefore able to demonstrate metacognitive skills however, where the mismatched group were also successful in some areas and despite choosing materials inconsistent with their learning style, they had clearly developed appropriate learning strategies to deal with this inconsistency in order to be successful. Analysis of these learning strategies and learning behaviours could prove helpful to other students and can provide support for the unsuccessful students identified in the framework. [Cemal Nat et al. 2011]

10

Liz Bacon and Lachlan MacKinnon The second project researched the impact of learner control on learning in adaptable and personalised eLearning environments. Adaptable systems that allow the user to change certain system parameters and adapt their behaviour accordingly are called ‘adaptable’ Santally (2005). There are many definitions of personalised learning systems, however they should be capable of adapting automatically to changes in an individual’s learning characteristics as the learning experience progresses (Karagiannikis and Sampson 2004). In this research project (Mustafa, 2011), learning materials were developed based on the VARK [Fleming 2001] learning style. Two cohorts of MSc students were divided into two groups randomly during the module registration process. One group was assigned to an adaptable eLearning environment and the other to a personalised eLearning environment. A switchboard model was then developed and used to manage the process of providing relevant materials to students, depending on different aspects of their learning style and whether they were in the adaptable or personalised group. The assessment questions were categorised into three types: Recall, Competency and Understanding. The results showed that a personalised eLearning system performed better in supporting learning that required students to recall and understand what they learned, whereas in the adaptable eLearning system, students showed a marked improvement in the assessment based on competency. Thus key findings from the analysis of the resulting data demonstrated that certain types of learning environment are better suited to certain type of learning behaviour. The outcomes from this research could provide a basis for the future design of eLearning systems, utilising different models of learner control, based on underpinning educational philosophies, in combination with learning preferences, to structure and present learning content according to type. In the context of the framework, this research provides guidance on how to help students choose the most relevant materials for them based on their learning style.

3.3 Use of games‐based learning to support metacognitive development Games are inherently fun, immersive, challenging, engaging etc. and have been used for educational purposes for a long time. This section describes a research project that was used to develop computational thinking skills in the game players with the aim of helping them learn how to program computers. Wing (2011) discusses the concept of computational thinking as including conceptualising, developing abstractions, building algorithms and designing systems etc. The game is called Program Your Robot (http://www.programyourrobot.com/) (Kazimoglu et al. 2012) is a puzzle game, the aim of which is to help a robot to escape from a grid platform by reaching the teleport square which will take players to the next level in the game. To do this, a player has to work out the algorithm to get the Robot from the start to the end square, negotiating various obstacles. An example screen shot of the game is shown in Fig 1.

Figure 1: Program Your Robot game

11

Liz Bacon and Lachlan MacKinnon The game was designed for use by higher education computing students and the results from considerable quantitative and qualitative analysis were extremely positive. Computational skills are a form of abstraction and so help to build metacognitive skills.

3.4 The framework Figure 2 presents an overview of the framework, which outlines the concepts and structures within a system that could be used to test for metacognitive ability. Each stage of the process is briefly described below and shows how the research projects described above can be linked together to provide an approach to assessing and developing metacognitive skills in students: Stage 1a: Students undergo an assessment based on one or more discriminatory learning models such as learning styles however they could also be categorised by other discriminatory models such as religious, cultural or accessibility needs. Stage 1b: Learning objects are tagged based on all the discriminatory models used to assess the students and stored in a suitable learning environment. A learning object could be text, video, a game etc. Stage 2: Students are given complete freedom to use any of the learning objects for their learning. Stage 3: Having completed the learning they are given a recall test, typically within 48 hours of completion and then a retention test typically about two weeks later. Stage 4a: For those students who do not demonstrate a threshold level of metacognition, the learning objects they used are compared to those they would have been expected to use as a result of the assessment in 1a and some guidance is provided. They can then repeat the learning process. Stage 4b: Students who demonstrate a threshold level of metacognition are provided with a heutagogic environment, in which to continue their learning. Ideally the framework learning and assessment take place in a monitored digital environment and the reason for this will be explained in the following section.

Monitored Digital Learning Environment 1a. Assessment of students according to discriminatory learning approach(es)

1b. Learning objects tagged according to discriminatory model(s)

2. Students undertake learning exercise – given free choice of tagged learning objects

3. Students are tested on their learning in two phases: recall and retention

4a. Threshold level of metacognition NOT met: guidance provided

4b. Threshold level of metacognition met

5. Student moves to heutagogic learning environment

Figure 2: Framework lifecycle

12

Liz Bacon and Lachlan MacKinnon

4. Educational models As discussed earlier, if we really want to develop metacognitive, self‐regulated learners then we need to revisit current approaches to online teaching, which tend to employ pedagogic as opposed to andragogic/heutagogic models of delivery that better support the development of truly independent learners (Blaschke 2012). Learning environments / platforms should be seen as a mechanism that frees the academic to develop new materials to support a different style of teaching on a far more individual basis as opposed to a convenient document repository which is their most common use today. The proposed educational model for the framework is based on lessons learned from the games community where the focus is on providing a rich, engaging and immersive environment that allows a game player / learner, to progress at their own pace. In such a learning environment, learning objects are linked in to electronic assessment models and instruments regarded as appropriate for the subject area. Much of the feedback to students would be automated with students receiving near instantaneous feedback on their activities and assessments. Academics would therefore be free to concentrate on helping the students to develop their learning capabilities and subject knowledge by offering guidance on appropriate techniques, additional areas for self‐guided study etc. Given that the educational model is intended to be andragogic/heutagogic, conversations between tutors and students could be focused much more on the professional advice given by the academic on how the student could improve or develop their performance to meet their own learning goals. In this solution, the ability to monitor and capture the student learning experience in digital environments is vital and must then link in to a mechanism that will allow that experience to be quantitatively and qualitatively analysed. Taking this concept further, if we can properly authenticate a student and their experience in the fully monitored, digital environment then we could rethink our approach to assessment. Given the purpose of education is to enable a student to achieve some learning and people all learn at a different pace and in different ways, there is no reason to say that everyone has to be assessed in the same way and at the same time. If our digital environment is able to authenticate use by a particular student then not only could the student submit themselves for assessment when they are ready, but for certain skills where the digital environment is used for their development e.g. software development, we do not need to assess a student if we know that a particular skill has been successfully developed during the learning process, the digital environment can confirm that the student has achieved the desired learning outcomes. This approach would provide a more accurate reflection of a student’s ability, particularly for students who have poor exam technique for example. This approach would also decrease the incidence of cheating and plagiarism and reduce the assessment load for academic staff, freeing them up to provide other types of support to students. It could also provide employers with more accurate information about a student’s abilities and finally, is very likely to be preferable and less stressful to the students.

5. Summary, conclusion and further work There is a massive requirement for higher education provision worldwide, which cannot be met purely by trying to emulate traditional teaching models in an online environment. However, simply providing online materials, even of high quality, does not solve the problem. Being able to assess and develop the metacognitive skills of learners is an important step towards improving outcomes in online learning, because we cannot easily recreate the structured support online, that a traditional face‐to‐face learning experience provides. In order to be successful in online learning students need to be heutagogic learners and we therefore need a mechanism for testing and developing metacognitive skills. This work has brought together the outcomes of a number of research projects into a coherent framework for assessing and developing metacognitive skills in students. Whilst the individual components of the research have been rigorously tested, future work will focus on trials that test all the stages of the framework as a coherent process.

Acknowledgements The authors would like to acknowledge the contribution of their PhD students: Musser Cemal Nat, Ryan Flynn, Cain Kazimoglu, Alan Mustafa and Sophie Peter.

References Azevedo, R., Cromley, J. G. and Seibert, D. (2004) Does adaptive scaffolding facilitate students' ability to regulate their learning with hypermedia? Contemporary Educational Psychology 29 (3), pp. 344‐370.

13

Liz Bacon and Lachlan MacKinnon Boekaerts, M. (1996). Self‐regulated learning at the junction of cognition and motivation. European Psychologist, 1, 100– 112. Boekaerts M. (1999). Self‐regulated learning: Where we are today? International Journal of Education Research 31: 445‐57. Boekaerts, M., & Niemivirta, M. (2000). Self‐regulated learning: Finding a balance between learning goals and ego‐ protective goals. InM.Boekaerts, P. R. Pintrich,&M. Zeidner (Eds.), Handbook of self‐regulation (pp. 417–450). San Diego, CA: Academic. Blaschke, L. M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self‐determined learning. The International Review of Research in Open and Distance Learning, 13(1), 56‐71. Cemal Nat, M., Walker, S., Dastbaz, M., Bacon, L., and Flynn, R. “Impact of metacognitive awareness on learning in a technology enhanced learning environment”. HEA “e” Teaching and Learning Workshop 1st June 2011 held at the University of Greenwich. Efklides Anastasia. (2011) Interactions of Metacognition With Motivation and Affect in Self‐Regulated Learning: The MASRL Model. Educational Psychologist, Volume 46, Issue 1, pages 6‐25. 2011. DOI: 10.1080/00461520.2011.538645. Felder, R. and Silverman, L., (1988) Learning and teaching styles in engineering education. Engineering Education, 78 (7), 674–681. Fleming, N. (2001), “VARK A guide to learning styles” http://www.varklearn.com/english/index.asp Gunter, M.A., Estes, T.H. and Schwab, J. (2003) Instruction: A models approach (4th ed). Boston: Allyn & Bacon. Hachey, Alyse C.; Wladis, Claire W.; Conway, Katherine M. (2012). Is the Second Time the Charm? Investigating Trends in Online Re‐Enrollment, Retention and Success. Journal of Educators Online, v9 n1 Jan 2012. Kazimoglu Cagin, Kiernan Mary, Bacon Liz and Mackinnon Lachlan (2012). “A Serious Game for Developing Computational Thinking and Learning Introductory Computer Programming” Published in Procedia‐Social and Behavioral Journal (ISSN: 1877‐0428), Volume 47, 2012, Pages 1991–1999. Kirsh, D. (2005). Metacognition, Distributed Cognition and Visual Design, in Cognition, education, and communication technology, (Eds.) Peter Gardenfors, Petter Johansson. Mahwah, N.J. : L. Erlbaum Associates, 2005 pp: 147‐180. MacKinnon L. and Bacon L. 2014. “Developing next generation online learning systems to support high quality global Higher Education provision”. In press, chapter 1 in “Innovations in Technology Enhanced Learning” Cambridge Scholars Publishing, scheduled for publication in Jan 2014. McInerney, D.M. and McInerney, V. (2006) Educational psychology: constructing learning. Pearson Education. Mulwa, C., Lawless, S., Sharp, M., Wade, V. and Sanchez (2010) Adaptive Educational Hypermedia Systems in Technology Enhanced Learning: A Literature Review. In proceedings of the ACM Special Interest Group for Information Technology Education Conference. ‘SIGITE 2010’. 7‐9 October, 2010, Central Michigan University, Midland, MI, USA. Mustafa A. (2011). PhD Thesis “Impact of learner control on learning in adaptable and personalised e‐learning environments” Feb 2011. Peter,S., Bacon,E. and Dastbaz,M. (2010). "Adaptable, personalised e‐Learning incorporating learning styles", Journal of Campus‐Wide Information Systems, The international journal of information and learning technology. Vol 27, Issue 2 pp 91‐100. ISSN: 1065‐0741. Sampson, D., Karagiannidis, C., & Kinshuk. (2002). Personalised Learning: Educational, Technological and Standardisation Perspective. Interactive Educational Multimedia 4. Santally, M., Senteni, A., (2005) A Learning Object Approach to Personalized Web‐Based Instruction. European Journal of Open, Distance and E‐Learning. Schunk, D. H. (2008). Metacognition, self‐regulation, and self‐regulated learning: Research recommendations. Educational Psychology Review, 20, 463–467 Eds.), Self‐regulated learning: From teaching to self‐reflective practice (pp. 1–19). New York, NY: Guilford. Wing, J. M. (2011). Computational thinking. In VL/HCC (p. 3). Woolfolk, A. & Margetts, K. (2007). Educational psychology. Frenchs Forest, NSW: Pearson Education Australia. Zimmerman, B. J. (2008). Investigating self‐regulation and motivation: Historical background, methodological developments and future prospects. American Educational Research Journal, 45, 166–183.

14

Authentic Assessment in Online Learning: Moving Beyond Text to Celebrate Multimodal Measures of Student Achievement Wendy Barber, Sherry King and Sylvia Buchanan University of Ontario Institute of Technology, Oshawa, Ontario, Canada [email protected] [email protected] [email protected] Abstract: The purpose of this paper is to examine the specific techniques and teaching methodologies that enabled the authors to create authentic online assessment tools for undergraduate and graduate online courses. The paper examines the challenges and victories of designing innovative online assessment strategies used in graduate and undergraduate courses through Adobe connect and synchronous weekly meetings. This project uses qualitative methodology including online narrative inquiry, digital storytelling and self‐study to analyse the effectiveness of using Digital Moments as a teaching and assessment technique. Although the growth of online learning is quite substantial, our ability to effectively assess it has not kept pace. There are still many who believe that electronic courses are inferior to those that offer face to face contact, and who favor traditional means of using text based methods to assess student achievement. This endemic belief can stem from an appalling experience with online assessment or simply because it is a non‐traditional format that for some may invoke fear, anxiety or even complete disdain for that which is different. Grounded in qualitative theory, the authors challenge traditional notions of online learning and argue that using Digital Moments can be an effective means to improve the quality of online learning experiences. The paper chronicles the role of weekly Digital Moments in the development of relationships within the class and the creation of a deep sense of community. Further to this, it builds a foundation from which authentic assessment, student ownership of learning and peer support can occur. While the digital world speaks in a language of one or zero, assessment cannot be reduced to such simple terms. In order to fully embrace the online learning environment, we cannot limit ourselves to simple text based measures of student achievement. Stepping into this brave new world requires innovation, creativity and tenacity, and the courage to accept that as the nature of knowledge has evolved in the digital landscape, so must our means of assessing it. Assessment in 21C learning is a complex, dynamic, and creative process, requiring qualitative methods to tell human stories. Keywords: authentic assessment, online learning, multimodal assessment

1. Introduction This paper is grounded in the theoretical framework of several authors who identify the parameters for authentic learning environments and authentic assessment tasks. This work will contextualize these elements with reference to particular synchronous online environments. (Reeves, Herrington & Oliver, 2002; McCarthy, 2013; Rosemartin, 2013; Herrington & Herrington, 1998; Bozalek, Gachago, Alexander, Watters, Wood, Ivala & Herrington, 2013). Literature reveals a general consensus about some of the key elements of an authentic learning environment. These include authentic context, authentic tasks, access to expert thinking and modelling of process, provision of multiple roles and perspectives, collaborative construction of knowledge, reflection, articulation to enable tacit knowledge to be made explicit, coaching and scaffolding, and authentic assessment of learning within the tasks. (Bozalek, et al, 2013, p. 631) Reeves et al (2002) add that tasks must have real world relevance, be ill defined, comprise tasks to be investigated over time, examine the task from different perspectives, provide opportunity to collaborate, reflect, be integrated and applied beyond domain specific outcomes, are seamlessly integrated with assessment, create polished products and allow diversity of outcomes. (p. 564) The use of Digital Moments is a robust and valid method of recording digital stories that emerge through these authentic contexts. Connelly and Clandinin (1990) refer to the use of narrative inquiry by stating that “the main claim for the use of narrative in educational research is that humans are storytelling organisms who, individually and socially, lead storied lives” (p. 2). Bullogh and Pinnegar (2001) concur that “writing about teacher education practice may be best expressed in the story form where linearity gives way to a different sense of time, where emotion drives action” (p. 18). Eisner (1997) states that “stories instruct, they reveal, they inform in special ways” (p. 5). Although Digital moments represent an alternative form of data representation and storytellying, they are a new means to do qualitative research in online environments. The

15

Wendy Barber, Sherry King and Sylvia Buchanan validity of such artistic research is supported by Eisner (1997) as he also refers to the importance of paying attention to the aesthetic and artistic elements of qualitative analysis. He states: concerns for verification, truth and precision have led us away from an experiential conception of understanding and toward a verificationist conception of knowledge – something that can be tested, packaged, imparted and sent like bricks across the country to build knowledge structures that are said to accumulate. (p. 7) In later work Eisner refers to the term “educational connoisseurship” (1998, p. 63) to describe a new way of knowing and forming knowledge. This knowledge is considered valid when it demonstrates “structural corroboration” (1998, p. 110) (such as multiple sources of data including videos, words, photos, drawings, social media) and “internal coherence” (1998, p. 113) (such as reflections, peer sharing, peer teaching). This paper reports on the use of Digital Moments not only as a strategy to create a professional learning community, but as a format for students to assess their learning. Implementing Digital Moments as a pedagogical tool encourages the development of trust, motivation, creativity and growth in learning. As an instructional strategy, it allows for many of the parameters in authentic learning environments to exist. Students learn in authentic contexts, do tasks of their choosing, collaborate with others, and have access to peers who share expertise in the particular technology they wish to learn. This creates collaborative construction of knowledge, coaching and scaffolding, and embeds assessment within the learning process. The sharing of each student’s and instructor’s Digital Moment creates a natural log of the individual and collective learning process, and the weekly sharing of stories allows verbal articulation of the learning; it enables the tacit knowledge to emerge as explicit. From this foundation, a natural evolution occurs to allow students to develop and design tasks through which they, along with their instructor and colleagues, would use to assess their learning.

2. Rationale It is clear that learners in the 21C exist in a world that continually redefines itself. The development of new knowledge outpaces our ability to keep up with content, thus many authors have re‐defined the essential skills required of the 21C learner. Several authors concur that these skills include the development of creativity, self‐motivation, innovation, problem‐solving and collaboration skills (McNeill, Gosper & Xu, 2012; Voogt, Erstad, Dede & Mishra, 2013; Kaufman, 2013). Within the digital world, we have a myriad of opportunities to invite students to develop these skills, if the instructor has the courage and tenacity to relinquish some authority, and level the playing field. Expertise no longer resides in one individual in a professional learning community, and so the roles of teacher and learner meld. It is in the development of this safe and trusting environment, envisaged here through the creative implementation of Digital Moments as a teaching and learning tool, that growth occurs. This is what Flavin (2012) refers to as “disruptive technologies” (p. 103). He states that “when digital technologies are brought into the classroom setting, the lecturer may have to relinquish some of their authority, thus impacting on the ‘rules’ and ‘division of labour’ nodes in order to enable enhanced learning” (Flavin, 2012, p. 104). This sharing of ownership in the learning environment has been identified by Cochrane (2012) as one of the critical success factors in mobile learning. He states that features of a successful virtual learning environment include pedagogical integration of technology into the course and assessment, lecturer modelling of the pedagogical use of the tools, creating a supportive learning community, and creating sustained interaction that explicitly scaffolds the development of ontological shifts, that is the reconceptualization of what it means to teach and learn within social constructivist paradigms, both for the lecturers and the students. (Cochrane, 2012, p. 125) The sustained interaction of the individuals’ Digital Moments within the professional learning community is a foundational element within which authentic learning and authentic assessment of that learning can emerge. The varied sources of data collected as Digital Moments (youtube, tweets, photos, poems, drawings) tell the story of the class as it evolves. As Connelly and Clandinin (1990) state, “the central value of narrative as method is its capacity to render life experiences, both personal and social, in relevant and meaningful ways; narrative researchers are compelled to move beyond the telling of the lived story to tell the research story” (p. 10).

16


3. Methodology This research occurred in three phases and was used to analyse the effectiveness of using the pedagogical documentation strategy of “Digital Moments” as an assessment tool. Phase 1: This involved using Digital Moments as an opening activity in online synchronous graduate courses as a way to create a professional learning community. Students in each phase took a course entitled “Authentic Assessment” and there were 20‐25 participants from a variety of backgrounds including education, nursing and health care, and business. The instructor was an Assistant Professor in the Faculty of Education. Classes met three times a week for two hours over a 6 week period in the spring term. The purpose of using “Digital Moments” was to simulate the social and community building network that evolves naturally during the first minutes of a face to face class environment. Each week, 25 pods were created in Adobe connect, and students entered the virtual room ahead of class time to post their Digital Moment. Students were given some exemplars as to what a Digital Moment might look like, (words, phrases, pictures, colours, musical links) but were not limited in their creativity. Anecdotal reflections from students recorded in Blackboard chat rooms, audio recordings of Adobe connect classes and field notes from the professor were collected. Phase 2: Students in this phase began to facilitate others’ uses of new technologies in order to submit their assignments in different formats. Having gained confidence and trust, two important elements of a virtual professional learning community, they began to ask the instructor if they could submit their final assignments using alternate means to text‐based artefacts. While traditional teaching at the graduate level involved submitting text based essays and final papers, students were allowed to fulfil their requirements by using alternative modes (youtube, video, audio, photojournal) as long as the work demonstrated evidence of competence, critical thinking, and was clearly grounded in the literature. At this phase it was noted that the Director of Grad Studies approached the instructor to ensure that text‐based assignments would also be included to fulfil university requirements. Phase 3: Students began to create assessment tools by which they were able to assess their own work and the work of others. These included rubrics, but also included portfolios of their course work, journals and comments by their peers who had witnessed, and often aided in the learning process. These tools were built in social and constructivist ways to ensure that the learning was both meaningful to the learner and relevant to their own professional contexts.

4. Data collection Ethical review was passed and informed consent of participants was obtained. Data were collected via recordings of classes in Adobe connect, including both formal and informal chat rooms for review. Anecdotal information from external professional learning communities created by the students in Linked In and Facebook was obtained. Recordings of classes were kept on a secure server located at the university. Audio and text data were used to analyze how well the strategy worked in terms of students’ perceptions of their online community. Students were asked to maintain weekly comments in Blackboard chat rooms and use this as a journal format to record their observations about their online community. Copies of assessment tools and the links to multi‐modal assignments were stored at the university website. It is also worthwhile to note that after the experiment had completed, several of the graduate students, themselves employed as teachers, have continued to journal with the professor and began to use the “Digital Moments” strategy in their own work environments.

5. Sample assignments and links to data Digital Moments Samples: http://www.youtube.com/watch?v=RJPLvWATbsA&feature=em‐share_video_user Steve N. ‐ Physics: http://www.youtube.com/watch?v=s23NGxRAN9Y Sylvia B. ‐ Art: http://www.youtube.com/watch?v=kO_iXocqFQw&feature=youtu.be Kate D. ‐ Avatar/special education: http://www.youtube.com/watch?v=c_gibuFZXZw Joel C. ‐ Flow state meets digital technology: http://www.youtube.com/watch?v=kduOsBKmy2Q

17


6. Findings and key themes 6.1 Creativity Kaufman (2013) reveals that “school is not simply about tests and ‘checking boxes’ of topics and assignments. Rather, schools today should have a mission of developing students as individuals and igniting their creativity” (p. 79). Students in this project began to unleash the bonds of traditional online courses they had taken, and began to flourish in the freedom of creative practice. At the same time, ironically, they began to take more responsibility for their own learning. Being allowed to choose empowered them to discover the intimate bond between real freedom, self‐responsibility and creativity. While many stated they had been indoctrinated by a culture of marks and grades, many revelled in the return to a natural state of learning, one that allowed freedom, innovation and a deeper level of responsibility than many had taken in some time. In previous online courses, the keeper of knowledge had been the instructor. It took courage on the parts of both instructor and learners, but once out of their educational cage they embraced the wide open fields of knowledge the digital world provided. One student referred to his favourite quote that “wild elephants walk softly in open fields” as a metaphor for feeling free, calm and in his natural learning environment.

6.2 Extended relationships The use of Digital Moments began to take on a life of its own beyond the scheduled class time. Some students created their own learning communities on Facebook and LinkedIn in order to stay in touch once the course had ended. In addition, Twitter feeds were used to follow each other and sustain friendships and learning experiences. These extended connections through technology became a web within which students connected on a personal level, a professional level, both emotionally and digitally. This is evidence that “learners are responding to the new technical and social opportunities with little help from the formal education system” and there is “evidence of deep networking and knowledge building in learners’ informal practices” (Littlejohn, Beetham & McGill, 2012, p. 551). Learning that is situated in digital worlds must also have a social component to be effective. Kearney, Shuck, Burden and Aubusson (2012) concur that learning is a social endeavour. They identify three distinct features of mobile or virtual learning that include “authenticity, collaborations and personalisation” (p. 2). They refer to a socio‐cultural model for virtual learning and the importance of “enhanced collaboration, access to information and deeper contextualisation of learning” (2012, p. 2)

6.3 Teacher‐learner‐teacher role shifts During the course, the roles in this professional learning community became almost indecipherable. While still within the university context, the instructor fulfilled the responsibility to assign grades to students. But in the learning environment, the power differential became almost invisible. The students with expertise in particular technologies took on the role of instructor, the teacher became the learner, thus empowering learners with the confidence to take risks, make mistakes, and ask for help. This supports the notion that 21C learners must be able to think critically, be problem‐solvers and work collaboratively. In particular, for 21C learners in a virtual classroom, they must be able to go beyond the class and use their digital literacy within the st context where they work and live. “It is obvious that not only learners, but also teachers need to acquire 21 st century competencies as well as become competent in supporting 21 century learning” (Voogt, Erstad, Dede & Mishra, 2013, p.408). In order to create authentic learning and assessment tools, teachers need to learn how to design such tasks. McNeill, Gosper and Xu (2012) surveyed academics and found that many continued to target lower order learning outcomes. They state that universities increasingly value the skills such as problem‐solving, critical thinking and creativity, yet the curriculum needs to be designed to support and scaffold development of these skills, and integrating them into assessment strategies has proven a challenge. While new technologies have sometimes been heralded as having the potential to address an apparent gap between the rhetoric of curriulum alignment and assessment practice in universities, academic practice is slow to change, and the uptake of new tools to support the development of higher order skills remains relatively low. (McNeill, Gosper & Xu, 2012, p. 283) This research argues that if Digital Moments can be used to create learning environments that support academics to learn new skills, then they may create more relevant 21C learning outcomes for their own

18

Wendy Barber, Sherry King and Sylvia Buchanan students. In the digital world, it is imperative that teachers, regardless of academic standing, continually redefine themselves as life‐long learners and model this for their students.

6.4 De‐valuing and re‐valuing The implementation and acceptance of arts‐based and creative assessment tools meant a significant ‘unlearning’ and ‘revaluing’ what it meant to demonstrate one’s knowledge. It became important to unpack how each learner had developed their values about the importance or lack of importance of marks and grades versus the value of the learning process itself. Students began to see how the development of friendships and simple human qualities like trust, caring and compassion were the real foundation for creating meaningful learning experiences. It also helped them to begin to trust themselves; they began to believe there was an authentic self in each learner who could choose which direction to go, which tasks were personally and professionally relevant, and which were best left to others. The level of passion and interest became more important than the grade, and this represented a significant shift in values. As Kaufman states “development of these skills is purposefully integrated within core content areas in ways that help students find relevancy in their work, a characteristic central to motivation and learning” (2013, p. 79). Contrary to traditional educational frameworks, wherein the power is centred in the instructor or the institution, this model required a re‐valuing of where the fundamental responsibility for learning resides ‐ within the learner.

7. Discussion The human story remains at the essence of every great learning experience. Using Digital Moments to tell individual stories and create learning communities proved an invaluable teaching strategy to create meaningful learning experiences for students. This sharing of stories, allows for learners to develop empathy, compassion and deeper understanding of each other. As the 21C learning landscape becomes increasingly impersonal, isolated and digital, it is imperative that we continue to use pedagogical strategies such as Digital Moments to preserve the richness of our online learning environments. As Cousins and Bissar affirm, What stories can be told about the fast‐changing world of higher education, and what can we learn from them? Adapting to new situations, conquering fears and overcoming obstacles are familiar storylines, with particular relevance for university lecturers having to introduce new technologies in their working practices. (2012, p. 1) Digital Moments are personal, and help us to create connections in a world where being wired to technology 24‐7 often makes us feel disconnected from those around us. This is the great paradigm of the digital 21C world. Educators need to find ways to reconnect learning in a very human, empathetic and meaningful way. Without this, we cannot ground our problem‐solving in a human context, and we cannot solve the issues we face alone. Rolfe (2012) states the importance of identifying individual pioneers and “understanding the motivations and characteristics of potential users in order to establish strong and sustainable practices” (p. 16). We know that student engagement in online courses is challenging as instructors face a huge inundation of competition from text, you‐tube, Facebook, Twitter and more. Students are wired in, and our instructional strategies need to acknowledge that keeping their attention requires us to use some of the same engagement strategies that are used so successfully by social media, video games and digital environments. Badge, Saunders and Cann (2012) acknowledge that students’ online attention is focussed on these other sites with high activity rates, and that “engagement is more than participation, it requires emotion and sense‐making as well as activity, these social networks are rapidly moving beyond their original purpose and are inevitably becoming part of the learner experience” (p. 2). Thus, to engage students in authentic learning environments, capture their attention and imagination, we need to use social strategies that appeal to students. Based on this captive audience, we can move them towards authentically assessing their learning, using modalities that are not text‐based, but which permeate their world on a moment by moment basis.

8. Conclusion Our digital stories can be effectively used as a strategy to create authentic online learning environments, and to assess student work authentically. This requires us to revisit several of the themes that emerged in this project. First, we need to celebrate and encourage the development of creativity by allowing students to use original and artistic ways to express knowledge; further, they need to be able to create the means to

19

Wendy Barber, Sherry King and Sylvia Buchanan authentically assess that knowledge and the learning of self and peers. Second, we need to acknowledge that the successful creation of the parent professional learning community is often insufficient, and readily gets supplemented by digital communities of practice developed by students. This is evidence of the power of extended relationships among learners, and it also allows for the shift in power from the university instructor to the real world of the student. Using Digital Moments can be a precursor to this shift. Third, the roles of teacher and learner must be interchangeable and fluid. The degree to which the instructor is willing to empower students, risk making mistakes and put themselves in the context of ‘beginner’s mind’ will parallel the trust and empathy in the learning environment. If we are to make it safe for students, we must model a certain degree of vulnerability ourselves, relinquish our post as ‘expert’ despite our academic qualifications, and quite probably re‐learn to have fun with the simple process of learning. Finally, there is a significant de‐ valuing and re‐valuing that occurs in authentic learning contexts. 21C learning environments do not require students to leave behind text‐based measures of knowledge completely. Rather, they acknowledge that text‐ based measures of achievement are insufficient to capture or measure things in a digital world. Ultimately, both learners and instructors must discern what remains ‘real’ in any authentic learning context. Digital worlds provide us with a plethora of options beyond text; we need to become responsible and free users of these alternative means to demonstrate knowledge. Our assessment methods must catch up to the reality of learning in the 21C. Our tool box must expand to include, but move beyond text to celebrate multi‐ modal measures of knowledge. The experience of becoming ‘real’ online was a journey fraught with highs and lows, like any good adventure. It is clear that digital classrooms can provide uniquely human learning experiences. The gaps that were anticipated in getting to know students, creating relationships between students online and designing a safe environment for taking personal risks in learning were not as scary as previously thought. Prior to teaching in this environment, the authors believed that “authenticity in teaching” would be more difficult online. In some respects, it is, but in our unfolding digital world, perhaps we need to use this venue for reaching out to learners more globally. Technology was a powerful tool, but the humanity in the classroom remained untouched as the real driver of the learning experience. It is important to remember that the teacher‐learner relationship cannot be replaced, nor does it need to be replaced by high tech solutions. The two must be woven together in an authentic and meaningful way, with both parties deciding how, when and why to use online environments. Margery Williams’ classic children’s tale of the Velveteen Rabbit sums up this notion: “What is real”, asked the rabbit, “does it mean having things that buzz inside you and a stick out handle?” “Real isn’t how you are made” said the skin horse, “it’s a thing that happens to you. It doesn’t happen all at once. You become. It takes a long long time. That’s why it doesn’t often happen to people who break easily, or have sharp edges or who have to be carefully kept. Generally, by the time you are real, most of your hair has been loved off, and your eyes drop out and you get loose in the joints and very shabby. But these things don’t matter at all, because once you are real you can’t be ugly, except to people who don’t understand” (1991, p. 32). The nature of knowledge has shifted; the nature of assessment lags far behind. Digital access to knowledge will continue to move faster than we can keep pace. Our job as instructors is not to carefully box our students’ knowledge in text based measures, label it securely in a container we feel is safe, and move on. If we limit ourselves to this academic prescription pad, serving our students a traditional dose of only text based assignments, we will remain far behind the digital divide. While not abandoning our history of essays and academic writing, we need to expand this learning and assessment tool box. We need to let students demonstrate their knowledge in a variety of artistic and creative ways that best fit their digital skills and knowledge. This is how we can challenge students to stay engaged, keep curiosity and creativity alive, and fully take the reins of their learning adventure. While academics will continue to refine and redefine how we assess online learning, what is valid, and what works, our students may be far up the road, leaving us behind in a puff of embers while they courageously create their next Digital Moment.

References: Badge, J., Saunders, N. & Cann, A. (2012). Beyond marks: new tools to visualise student engagement via social networks. Research in Learning Technology, 20(16283). doi: 10.3402rlt.v2010/16283 Bozalek, V., Gachago, D., Alexander, L., Watter, K., Wood, D., Ivala, E. & Herrington, J. (2013). The use of emerging technologies for authentic learning: a South African study in higher education. British Journal of Educational Technology. 44(2), 629‐638. doi: 10:1111/jbet.12046

20

Wendy Barber, Sherry King and Sylvia Buchanan Bullogh, R. & Pinnegar, S. (2001). Guidelines for quality in autobiographical forms of self‐study research. Educational Researcher 30(3) 13‐21. Cochrane, T. (2012). Secrets of m‐learning failures: confronting reality. Research in Learning Technology, ALT‐C Conference Proceedings, http://dx.doi.org/10.3402/rlts2010.19186 Connelly, M. & Clandinin, J. (1990). Stories of experience and narrative inquiry.Educational Researcher, (June‐July) 2‐14. Cousins, S. & Bissar, D. (2012). Adapting to the digital age: a narrative approach. Research in Learning Technology, 20(18976), 1‐13. Eisner, E. (1997). The promise and perils of alternative forms of data representation. Educational Researcher, (Aug‐Sept), 4‐ 10. Eisner, E. (1998). The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. Upper Saddle River, NJ: Prentice Hall. Flavin, M. (2012). Disruptive technologies in higher education. Research in Learning Technology, ALT‐C 2012 Conference Proceedings. http://dx.doi.org/10.3402/rlt.v2010.19184 Herrington, J. & Herrington, A. (1998). Authentic assessment and multi‐media: how university students response to a model of authentic assessment. Higher Education Research & Development, 17(3), 305‐322. doi: 10:1080/0729436980170304 Herrington, J. & Parker, J. (2013). Emerging technologies as cognitive tools for authentic learning. British Journal of Educational Technology, 44(4), 607‐615. doi:10.1111/bjet.12048 Herrington, J., Parker, J. & Boase‐Jelink, D. (2013) Social presence and participatory media in authentic learning. In: Open and Distance Learning Association of Australia (ODLAA) 2013 Distance Education Summit, 4‐6 February 2013, Novotel Sydney Manly Pacific hotel, Sydney. http://researchrepository.murdoch.edu.au/13914 Kaufman, K. (2013). 21 Ways to 21st century skills: why students need them and ideas for practical implementation. Kappa Delta Pi Record, 49(2), 78‐83. doi: 10.1080/00228958.2013.786594 Kearney.M., Schuk, S., Burden, K. & Aubusson, P. (2012). Viewing mobile learning from a pedagogical perspective. Research in Learning Technology, 20(14406). doi: 10.3402/rlt.v2010.14406 Littlejohn, A., Beetham, H. & McGill, L. (2012). Learning at the digital frontier: a review of digital literacies in theory and practice. Journal of Computer Assisted Learning, 28, 547‐556. doi: 10.1111/j.1365‐2729.2011.00474.x McCarthy, G. (2013) Authentic assessment: key to learning. In E. Doyle, P. Buckley & C. Carroll (Eds.), Innovative Business School Teaching: Engaging the Millennial Generation. 81‐92. United Kingdom: Routledge. McNeill, M., Gosper, M. & Xu, J. (2012). Assessment choices to target higher order learning outcomes: the power of academic empowerment. Research and Learning Technology, 20(17595) doi: 10.3402/rlt.v2010.17595 Reeves, T., Herrington, J. & Oliver, R. (2002). Authentic activities and online learning. Quality Conversations: Research and Development in Higher Education, 25, 562‐567. Rolfe, V. (2012). Open educational resources; staff attitudes and awareness. Research in Learning Technology. 20(1) 7019. Rosemartin, D. (2013). Assessment for learning: shifting our focus. Kappa Delta Pi Record, 49(1), 21‐25. doi: 10.1080/00228958.2013.759826 Voogt, J., Erstad, O., Dede, C. & Mishra,P. (2013). Challenges to learning and schooling in the digital networked world of the st 21 century. Journal of Computer Assisted Learning, 29, 403‐413. doi: 10.1111/jcal.12029 Williams, M. (1991). The Classic Tale of the Velveteen Rabbit. Philadelphia, PA: Courage Books.

21

Addressing the Mathematical Knowledge gap Between High School and First Year University Chemical Engineering Mathematics Course: The Role of Facebook Moses Basitere and Eunice Ivala Cape Peninsula University of Technology, Cape Town, South Africa [email protected] [email protected] Abstract: Research shows that the South African higher education sector is faced with the challenge of the knowledge gap between the knowledge possessed by school leavers and the knowledge required for first year entry into mathematics courses. Literature shows that some institutions have taken steps to bridge this knowledge gap, while others have continued to ignore the problem leading to high dropout rates. This paper reports on a study carried out at a University of Technology, South Africa, aimed at identifying the existence of the mathematical knowledge gap and evaluating the intervention designed to bridge the knowledge gap amongst students studying first year mathematics at the chemical engineering Extended Curriculum Programmme (ECP). In this study, a pre‐test was used as a diagnostic test to test incoming chemical engineering students, with the aim of identifying the mathematical knowledge gap, and to provide students with support in their starting level of mathematical knowledge and skills. After the diagnostic test, an intervention called the autumn school was organized to provide support to bridge the mathematical knowledge gap identified. A closed Facebook group served as a platform for providing student support after school hours. After the autumn school, a post‐test was administered to measure whether there was an improvement in the knowledge gap. Both quantitative and qualitative methods of collecting data were used in this study. A pre‐test was used to identify the mathematical knowledge gap, while a post‐test was employed to measure whether there was improvement in the knowledge gap after the intervention. Focus group interviews were carried out with the students to elicit their opinions on whether the intervention was of any help for them. Quantitative data was analysed using descriptive statistics, while qualitative data was analysed using inductive strategy. Results showed that all the students in this study had the mathematical knowledge gap as no student in the class scored 50% on the overall pre‐test. Findings further revealed that the intervention played a major role in alleviating the mathematical knowledge gap from some of the students (with a 1/3 of the students scoring 50% and above in the post‐ test), with Facebook playing a crucial role of extending learning beyond the classroom time. We hope that insights generated in this study will be of help to other institutions looking into designing interventions for bridging the knowledge gap. Reasons for lack of improvement in the knowledge gap of 2/3 of the students in this class will be highlighted. Keywords: knowledge gap, extended curriculum programme, descriptive statistics, inductive strategy, diagnostic test, autumn school

1. Introduction In South Africa, engineering programmes require at least two years of university mathematics. The student selection into these courses is based on one’s marks achieved in the final school leaving examination‐the National Senior Certificate (NSC) (Van der Flier, Thijs & Zaaiman 2003). Over the years, students entering university engineering courses struggle to succeed in the first year mathematics (Moyo 2013; Wolmarans et al. 2010). As a result, the South African higher education (HE) sector is faced with the challenge of the mathematical knowledge gap ( mathematical knowledge gap is defined as the lack of smooth transition from high school mathematics to university first year mathematics for students majoring in science, mathematics and engineering due to the shortcoming of both the high school and the first year university mathematics programmes) between the knowledge possessed by school leavers and the knowledge required for first year entry into mathematics courses (Wolmarans et al. 2010). Research shows that some institutions have taken steps to bridge this knowledge gap, while others have continued to ignore the problem leading to high dropout rates (Moyo 2013). This paper reports on a study carried out at a University of Technology, South Africa, aimed at identifying the existence of the mathematical knowledge gap and evaluating the intervention designed to bridge the knowledge gap amongst students studying first year mathematics at the chemical engineering Extended Curriculum Programme (ECP) (The Extended Curriculum Programme gives access to higher education to students from disadvantaged backgrounds who have only met the minimum requirements for entry to university and assists them in developing academic foundations by offering instructions in small classes over an extended period of time, and with more dedicated support). In this study, a pre‐test was used as a diagnostic test to test incoming chemical engineering students, with the aim of identifying the mathematical knowledge gap and to provide students with support in their starting level of mathematical knowledge and skills. After the diagnostic test, an intervention called the autumn school was organized to

22


provide support on the knowledge gap identified. A closed Facebook group served as a platform for providing student support after school hours. After the autumn school, a post‐test was administered to measure whether there was an improvement in the knowledge gap. Using both quantitative and qualitative methods of collecting data, findings of the study showed that all the students in this study had the mathematical knowledge gap as no student in the class scored 50% on the overall pre‐test. Findings further revealed that the intervention played a major role in alleviating the knowledge gap from some of the students (with a 1/3 of the students scoring 50% and above in the post‐ test), with Facebook playing a crucial role of extending learning beyond the classroom time. We hope that insights generated in this study will be of help to other institutions looking into designing interventions for bridging the knowledge gap. To investigate the matter under study, the researchers were guided by the following objectives:

To investigate the existence of the mathematical knowledge gap

To evaluate the impact of the intervention designed to bridge the knowledge gap

2. Literature 2.1 The transition between schooling and first year entry into mathematics courses According to 2005 HEMIS (The Higher Education Management and Information System), the “pool” from which suitable science and engineering students are selected is small. For instance, 20 percent of the 2008 National Senior Certificate (NSC) candidates achieved a pass that allowed them access to admission to degree studies at higher education institutions, and only 7.9 percent of the candidates achieved more than 60 percent in the 2008 NSC mathematics examination‐a requirement to enter engineering programmes (Department of Education 2008). Research suggests that many students in South African schooling system are underprepared for success in higher education (HE), and especially in mathematics and Science disciplines (Badat 2010). Research shows that students entering university engineering courses struggle to succeed in the first year mathematics (Moyo 2013; Wolmarans et al. 2010). This is partly attributed to the mathematical knowledge gap possessed by school leavers and the knowledge required for first year entry into mathematics courses (Wolmarans et al. 2010). The knowledge gap includes: a serious lack of essential technical facility‐the ability to undertake numerical and algebraic calculations with efficiency and accuracy; a marked decline in analytical powers when faced with simple problems requiring more than one step; and most students entering HE no longer understand that mathematics is a precise discipline in which exact, reliable calculations, logical exposition and proof play an essential role (Clark and Lovric 2009). The matric topics that are said to be problematic and not taught in a way that minds the gap are: functions, sequences and series, differential calculus, euclidean geometry, analytical geometry, vectors, complex numbers and statistics. This mathematical knowledge gap encountered by first – year engineering students in mathematics courses is not unique to the South African context (Moyo 2013; Wolmarans et al. 2010). These authors indicate that similar patterns have emerged in, amongst others, the United States. However, the situation in South Africa in regard to mathematics is regarded to be more pronounced than elsewhere (Wolmarans et al. 2010). The effects of the knowledge gap in South Africa are that students’ drop‐out affecting the throughput rates and some students are discouraged from taking mathematics and physical sciences because they are perceived as cognitively difficult. This may have a long‐ term impact on the skills available in these areas in the country. As a result, it is imperative that institutions of higher education find educational strategies that would impact student success (Scott et al. 2007). Hence, many universities in South Africa in response to the fact that schools do not adequately prepare students for what is expected of them when they enter first‐year mathematics, are designing interventions aimed at helping address this deficit (Human et al. 2010). Thus, this paper reports on a study carried out at a University of Technology in South Africa, aimed at identifying the existence of the mathematical knowledge gap and evaluating the intervention designed to bridge the knowledge gap amongst students studying first year mathematics at the chemical engineering Extended Curriculum Programme. Facebook, a social media application was used as part of the intervention.

23


2.2 Facebook for teaching and learning Social media can be defined as a group of internet‐based applications built on the ideology and technology of Web 2.0, which allows the creation and exchange of user generated content (Kaplan & Haenlein 2010). Social media emphasizes active participation, connectivity, collaboration, community and sharing of knowledge and ideas among users (Correa 2013). This technology exists in different forms such as internet forums, web‐logs, social‐blogs, micro‐blogging, wikis, podcasting, social bookmarking and social networks (Mazer, Murphy & Simonds 2007). The case for use of social media for teaching and learning is quite convincing (Badge et al 2012;Ivala & Gachago 2012; Leece & Campbell 2011; Muňoz & Strotmeyer 2010). Badge et al. (2012) contend that by encouraging engagement with social media , students develop connections with peers, establish a virtual community of learners and ultimately increase their overall learning . By participating in a community of learners, students become more engaged with the course content which increases the achievement of popular learning outcomes such as critical thinking. Similarly Junco (2012) reports positive relationship between the use of social networking sites websites and student engagement adding that frequent users of social networking websites participated more often and spend more time in campus organisations than less frequent users. Mazur et al (2007) also indicated that social networks offer opportunities to cultivate the student teacher relationship, which creates a positive learning experience for both parties. Hence social networking services such as Facebook, Twitter and MySpace have gained huge popularity and widespread use in HE globally over the past few years. The reasons for using Facebook in this study was because Facebook is ranked the top social networking site (SNS) in the world with an estimated 1 billion monthly active users and 552 million daily active users on average (Facebook newsroom statistics 2012). Studies by Hargittai (2008), Jones and Fox (2009) and Junco (2012), also indicated that Facebook is the most popular social media web site for college students. Given that Facebook continues to be popular among college students, and that universities are interested in engaging and retaining students, Junco (2012) advises that those working in HE need to familiarize themselves with Facebook (and other such technologies) and to design and support interventions that meet students where they are in order to help them get to where they are going. Reporting on an experiment with Facebook as a teaching and learning tool, Esteves (2012: 6) revealed that: students asked questions related to the course topics by positing on the group’s wall and received answers through “comments” from other members of the class; students shared new media like videos, websites, comic strips, podcast related to web design and publishing, distance education and Facebook as used for learning; and students used the “Chat” feature to discuss class‐rated topics or simply chat casually with their classmates and faculty‐in‐charge. Facebook offers educators an additional venue for connecting with diverse cohorts of students, as well as a different forum for exchange with students in a less hierarchically structured manner. By “meeting students where they are” (e.g.Facebook) educators can capitalize on the existing incentive of participation in personal networks much like the commercial interests take advantage of their potential clients presence in those spaces. In terms of creating a ‘community of learners’ the use of a tool that students embrace in their daily lives, may improve discussion thread postings on course material, while supporting interactivity with students in these informal spaces. Caution however needs to be exercised as literature shows that social network services might also negatively affect learning. For example, Welch and Bonnan‐White (2012) found that many students had difficulty with the technology due to lack of familiarity and that some students were reluctant to adapt to unfamiliar technology and classroom expectations. Social media can also cause miscommunications often because of the limited context available in digital communication and “since the lack of face‐to‐face contact involved in using social technologies leads to limited context, it is incredibly easy, and takes very little ego investment to propagate rumors and harassing content quickly and easily across the internet. Other disadvantages of using Facebook for teaching are privacy concerns and issues of self‐disclosure and identity management (Correa 2013).

3. The context of the study The study was conducted in 2013 academic year at the department of chemical engineering at a University of Technology, in South Africa. The participants of the study were 41 ECP chemical engineering students. The ECP programme has been designed to support students who are enrolled in chemical engineering course with a minimum of 50 percent pass rate in mathematics and physical sciences. As part of the support system, the ECP students take half work load compared to mainstream students (Mainstream students are those whose matric marks are above 50 percent and take six subjects per semester). The ECP programme takes two years in which

24


students are provided with the necessary support to acquire the necessary skills and competences they need to succeed in their studies. The ECP programme is supposed to play a role in filling in the mathematical knowledge gap but in practice that does not happen often. Therefore, in this university, the lecturer responsible for teaching mathematics with support from the department decided to administer a diagnostic test to incoming chemical engineering students to measure the mathematical competence of students upon arrival at university and to provide prompt and effective support to students on mathematical knowledge gap identified. The approach of diagnosing and providing support is based on the trust that incoming students are in principle quite capable of overcoming the initial difficulties and can attain the desired level in a rather short term. The lecturer’s administration of the diagnostic test and designing of the intervention was driven by the concern that students may lack certain conceptual understanding as well as dispositions that are essential in the practice of mathematics and applied mathematics and the fact that the possibilities for genuine education depend on the knowledge and experience already existing within students (level of development) as well as on the students potential to learn (Vygotsky1986). The diagnostic pre‐ test consisted of open‐ended questions, while the post‐test consisted of open‐ended questions with a few multiple choice questions. The areas covered by both the pre‐test and post‐ test were: trigonometry; analytical geometry, fractions and simplifying exponents and calculus. On establishing the existence of the mathematical knowledge gap, the lecturer designed an autumn school (the intervention), with a closed Facebook group being used to extend learning out of the classroom. The intervention was run for one week and it was aimed at : filling gaps in mathematical knowledge and skills, reviewing essential facts and brushing up on calculations and develop appropriate attitude. The reason for this lecturer’s caring about the knowledge gap was to improve student learning; to increase and/or maintain their motivation to study mathematics and to consider a career in maths related field and to use resources effectively. Additionally, the lecturer cared about the transition because he has a responsibility towards his students, as they strive to achieve their university goals to offer resources that are most effective for students’ success so that students who fail in their studies will then know that it was not due to poorly designed transition and initial university offerings (Clark & Lovric 2009). On the social economic level, there is a tendency for greater proportions of the population to become engineers and scientists. As a result, engineering and science study programmes are needed to take students who may in the past not have been considered as suitable for such studies, and then need to educate them to the right level. Secondly, the school system in South Africa is currently struggling to produce sufficient number of students who are well prepared for engineering and sciences course. As a result, university courses need to adapt, and many engineering and science undergraduate programmes in South Africa are already presenting support subjects in addition to ‘core’ mathematics, science and engineering subjects (Human, et al. 2010). In the next sections, the methodology and the results of this study will be presented.

4. Methodology Both quantitative and qualitative methods of collecting data were used in this study in order to ensure triangulation of data and to enhance the significance of the findings by integrating different ways of knowing (Caracelli & Greene 1997).

4.1 Context and participants The study was carried out at the faculty of engineering, department of chemical engineering at a University of Technology in South Africa. The participants of the study were 41 students enrolled for ECP chemical engineering in 2013 and working towards a national diploma. A purposive sampling was used to select the participant in this study (Patton 1990). The students were selected because it was felt that they had rich information gained through their experiences in the intervention (Patton 1990).

4.2 Data collection Both quantitative and qualitative methods of collecting data were used and the data consisted of recordings of three focus group interviews with the students, which were carried out to draw out their perceptions of the benefit of the intervention (autumn school). Quantitative data was gathered through the use of a diagnostic test as pre‐test, used to identify whether students had the mathematical knowledge gap and a post‐test was administered after the intervention to measure whether the intervention had any impact in addressing the

25


mathematical knowledge gap students possessed. The pre‐test question paper consisted of open‐ended questions, while the post‐test question paper comprised of open‐ended and multiple choice questions.

4.3 Data analysis Quantitative data was analysed using descriptive and inferential statistics, while qualitative data was analysed using inductive strategy. Frequencies were calculated to determine whether the students had mathematical knowledge gap and to measure whether the intervention had any impact in addressing the mathematical knowledge gap. A paired sample one tailed P‐Test was executed to test for significant differences in the pre and post‐tests marks. Focus group interview data was recorded on tape and transcribed verbatim. The interviews were analyzed focusing on the identification of conceptual themes and issues emerging from the data, using techniques such as clustering, and making contrast and comparisons (Miles & Huberman, 1994). The researchers were especially interested in moments in the project that could be construed as the focal points for the benefits of the intervention. The participant consent to participate in the study was sought and the purpose of the study was explained to the students. Interview transcripts and student scripts were available for the students to scrutinize. Anonymity and confidentiality was adhered to as promised to the students. And ethical clearance was given by the faculty of engineering ethics committee.

5. Findings and discussion The study aimed at identifying the existence of the mathematical knowledge gap and evaluating the impact of the intervention designed to bridge the knowledge gap amongst students studying first year mathematics at the chemical engineering Extended Curriculum Programmme (ECP) , at a University of Technology in South Africa. Findings are presented under the following categories:

Identification of the existence of the mathematical knowledge gap

Impact of the autumn school in addressing the mathematical knowledge gap

5.1 Identification of the existence of the mathematical knowledge gap Findings of the study showed that students had the mathematical knowledge gap in the four areas assessed, (that is, trigonometry; analytical geometry, fractions and simplifying exponents and calculus), as no student in the class scored 50% on the overall pre‐test (see figure 1).

Figure 1: Pre‐test results The above results confirm the existence of the mathematical knowledge gap amongst the incoming first year chemical engineering students. These results are in agreement with findings by Wolmarans et al. (2010) and Moyo (2013) who reported that the mathematical knowledge gap existed amongst first –year engineering students in mathematics courses in South Africa and elsewhere in the world.

26


5.2 Impact of the autumn school in addressing the mathematical knowledge gap After establishing that the incoming first‐year engineering students had the mathematical knowledge gap, the lecturer designed and implemented an autumn school to teach the four areas tested in the pre‐test. A closed Facebook group was opened to extend learning beyond the classroom. After the autumn school, a post‐test was administered on the first day of the new university term, which commenced after the autumn school. Findings of the post test showed that the intervention played a major role in alleviating the mathematical knowledge gap from some of the students (with a 1/3 of the students scoring 50% and above in the post‐ test and 2/3 scoring less than 50) (see figure 2), with Facebook playing a crucial role of extending learning beyond the classroom time. Further, the pre‐test results showed a zero pass rate, with average results of 23%, which indicates that students lacked understanding on some of the basic mathematical concepts needed to be mastered before they start their first year engineering mathematics. The results of the post‐test, indicated an improvement on student performance with a pass rate of 39%, with an average student mark of 48% (see table 1). To determine whether the pre‐ and post‐test marks were comparable, a p‐test was performed on the pairs of pre‐test and post‐test data. A p‐value smaller and equal to 0.05 (p ≤ 0.05) is interpreted as significant as it indicates a probability of 5% or less difference between the pre‐test and post‐test data sets. The p‐test, showed a p‐value of 2.22E‐11, which is less than 0.05 indicating significant difference between pre‐test and post‐test (see table 1).

Figure 2: Post‐ test results The progress of individual students between the pre‐test and post‐test is represented in the scatter diagram below (see figure 3). Each dot represents the score of a student in the pre‐test and post‐test.

Figure 3: Scatter diagram of students' scores in pre‐test and post‐test

27


Even though 2/3 of the students did not score 50 percent and above, results show that there was increased understanding in knowledge as their marks had improved compared to what they had scored in the pre‐test (see figures 2 and 3). However, it is disconcerting that only a third of the student population entering chemical engineering ECP reached a desirable level of mathematical competence (scoring>50%). Table 1: Comparison of students’ pre‐test and post‐test marks

Pre‐test

Post‐test

End term results

Sample size

41

41

41

No of passes

0

16

31

Pass rate

0

39

76

Mean

23

48

54

Standard deviation

13

16

10

Coeficient of variation

57

33

9

P‐test

2.22E‐11

Quantitative data does not provide the reasons why two thirds of the students did not score 50 per cent in the pre‐test. But reasons are important in order to make informed decisions regarding the impact of the intervention. Therefore, three focus group interviews, each comprising of ten students were carried out with the students to measure the impact of the intervention. Analysis from the focus group interviews showed that all the students found the autumn school of benefit to them as it introduced them to things they did not learn in high school, helped them revise high school work and gave them the basic competences and skills needed to cope with university work, as exemplified in the following quotes: Student C: I think it helped a lot with some of the things we did in school, …we did not do in school.. .like we did not cover most of the things like trigonometry in NCS…So his intervention actually help when it comes to trigonometry. Student E: It was fine… because at first we didn’t know the basics of things we were introduced to like here at university. So the bridging the gap thing gave us the basics so that we can cope well on like doing… the university work. The teaching methods of the lecturer and the use of a closed Facebook group were given by majority of the students as some of the aspects of the intervention which enhanced its usefulness. On the lecturers teaching methods, students said: Student D: It was his teaching method during that time. He had patience…we could bring like any queries that we had to him, especially the difficulties we had during matric…so we had an understanding to that, which helped us step up to be introduced into the university work The Facebook closed group was said to have been helpful in extended learning outside of the class. For example, students indicated that they used Facebook to: Student F: I use Facebook to get notes, my notes, like most of the notes I get it from Facebook and if I want to know what is happening here [on campus] like if we are about to write a test and I am not sure about a thing…then that’s when I use Facebook… Facebook was also used extensively by students to post questions and get answers from their peers and the lecturer, which enhanced their learning as evidenced in the following: Student A: When you do tutorials like Mr Moses gave us or things on the book then you go and ask question on the page and then you get answers and then they also show you steps on how you do it and then you get more understanding like that. You don’t have to wait on something until the following day you just ask on Facebook and then they help you out, then you’ll understand it, then you can do it… Mr Moses can help us and also the tutors and also my classmates Student B: It definitely improved my academic life because as I said, if you study a day before the test you can post like in the group a question and they help you with that question and if the question… comes in the test and then you score marks with that question.

28


Some of the students indicated that Facebook enabled shy students who would hardly ask questions in class to post questions on the Facebook group: Student G: You know sometimes its difficult when you’re sitting in class and you want to ask a question but you are ashamed to ask it because you know …maybe they’re going to laugh at you. But the Facebook is giving you a platform where you ask a question then there’s no one who’s going to laugh at you but you are going to ask a question which you’re going to help the second person who was going to ask that question and if they answer that question everyone is gaining not only you who’s is gaining…which is good The above results show that students felt that Facebook was of benefit to them because they learned from each other’s posts. This is vital because participating in someone’s else’s learning contributes to the growth of that person, ones’ experience as peers tutor becomes a significant component of his\her education process and the best way to test whether we have leant something is to teach it to someone else (Clark & Lovric 2009). Despite the above benefits of the intervention, 2/3 of the students scored less than 50 percent in the post‐ test. Students gave varied reasons for their failure. Some of the reasons given were: forgetting what they learned, being lazy to study, lack of time management; being nervous during exams and struggling with university work. Some of the students echoed the following student’s sentiment: Student E: It is not the lecturers problem… he is doing his job and he is giving extra, but we also have like reach out like put more effort in, the time …we have the responsibility like she says, like we do the certain work, Thursday, we have to go and revise it that day… The above results show that students appreciate the effort the lecturer was putting in teaching them and acknowledged that they needed to take responsibility for their studies. The researchers suggest that balance has to be struck between being too helpful, a short‐term fix, and encouraging students to overcome their own deficiencies and problems (Clark & Lovric 2008), as a well‐intended initiatives at helping students, if lecturers go too far (babying students) can certainly disempower students as learners. Another reason given for failure was that students did not have time to study as they had to travel home and at home they had to do household chores or for some the environment was not conducive: Student B: …Like me I get up at home 8 o’clock, [leave] from here about 4:30 pm because I am using the train from Belville to here [home] from here [Campus] to Kuils River, from Kuils River I take another train. So I don’t have enough time to study… Student C: … like most of the people here are staying in Res [residence] and you have to cook when you get home…you have to do like a lot of things when you get into your home before you study.So some of the things that we do like cooking, washing dishes and everything actually consumes our time of studying… Student D: Where I live I’m surrounded by taverns … So it is always noisy all the time and I’m living with too many people…there are a lot of disturbances…the environment is not conducive for me to study. The above results highlight issues which were not elicited by the diagnostic test but are crucial in impacting student success. The researcher suggest that student background information, their attitude towards mathematics and their learning styles should be included in the diagnostic test in order to enable the lecturer to design a relevant intervention for the students.

6. Conclusion and recommendations Findings of this study showed that the incoming first year chemical engineering Extended Curriculum Programmme (ECP) students at a University of Technology in South Africa had the mathematical knowledge gap and hence, the need for this University to come up with effective strategies for strengthening incoming students’ level of mathematics knowledge and skills in order to enable them to cope and succeed in their university studies. The intervention designed by the lecturer in this study was perceived by all the students as beneficial for their studies. In particular, the lecturers teaching methods and the use of a Facebook group was applauded by the students. However, due to the 2/3 of the students not scoring 50 percent in the post‐test, the researcher agree with Clark & Lovric (2009) in suggesting that an effective diagnostics tool must inquire besides obvious information on mathematics background knowledge and skills, students attitudes towards mathematics and learning mathematics, their motivation, intelligence, learning styles, and their social development in order to enable lecturers to design a relevant and effective intervention . The researchers

29


further suggest that the university need to find ways of assist students living out of residence in order to enable them to have enough time for their studies.

References Badat, S. (2010). “The challenges of transformation in higher education and training institutions in South Africa,” Paper commissioned by Development Bank of South Africa, [Online], http://www.dbsa.org/Research/Higher%20Education%20and%20Training/The%20challenges%20of%20transformatio n%20in%20higher%20education%20and%20training%20institutions%20in%20South%20Africa%20by%20Saleem%20 Badat.pdf. Accessed 16 April 2012. Badge, J.L., Saunders, N.F.W. & Can, N. (2012). “ AJ beyond marks: new tools to visualize student engagement via social networks”, Research in Learning Technology, Vol. 20, No.1, DOI: 10.3402/rlt.v20i0/16283pp1‐14. Caracelli, V. J. and Greene, J. C. (1997). “Crafting mixed‐option evaluation design”, In J. C. Greene & V. J. Caracelli (eds.), Advances in mixed‐option evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Program Evaluation, No. 74, pp. 19‐32.460, Jossey‐Bass, San Francisco, CA. Clark, M. & Lovric, M. (2008). “Suggestion for a theoretical model for secondary‐tertiary transition in mathematics”, Math. Educ.Res. J, Vol 20, No. 2, 25‐37. Clark, M. and Lovric, M. (2009). “ Understanding secondary –tertiary transition in mathematics,” International Journal of Mathematical Education in Science and Technology, Vol. 40, No. 6, 755‐776. Correa, E. (2013). “Social media in online course construction”, In: R.M. Kersner (Ed.), The online university: Building viable learning experiences for higher education, World University Forum book series, USA, Common Ground Publishing LLC. Department of Education (2008). “Abridged report 2008 National Senior Certificate Examination Results”,[Online], http://edulibpretoria.files.wordpress.ocm/2009/01/abridgedreport2008nscexams.pdf. Accessed 1 January 2014. Esteves, K.K. (2012). “Exploring Facebook to enhance learning and student engagement: a case from the University of Philippines (UP) Open University”, Malaysian Journal of Distance Education, Vol.14, No. 1, 1‐15. Human, C., Hofmeyr, A. Human, P., Makae, N. & Van Koersveld, P. (2010). “Strategic and conceptual challenges experienced by first‐year students while attempting to solve problems that require mathematical modelling”, Paper presented at the ASSAF ‘mind the gap’ forum, 21 October 2010, Cape Town, South Africa. Ivala, E. and Gachago, D. (2012). “Social media for enhancing student engagement: The use of Facebook and blogs at a University of Technology”, South African Journal for Higher Education Vol. 26, No. 1. Junco, R. (2012). “The relationship between frequency of Facebook use, participation in Facebook activities and student engagement”, Computers and Education, Vol. 58, No. 1, 162‐171. Kaplan, A.M., & Haenlein, M. (2010). “Users of the world, united! The challenges and opportunities of social media”, Business Horizons, Vol.53, No. 1, 59‐68. Leece, R. and Campbell, E. (2011). “Engaging students through social media”, Journal of the Australia and New Zealand Student Services Association, Vo.38, 10‐15. Mazer, J.P.,Murphy, R. E. & Simods, C.J. (2007). “I’ll see you on “Facebook”: The effects of computer mediated teacher self‐ disclosure on student motivation, affective learning, and classroom climate, Communication Education, Vol. 56, No. 1, 1‐17. Miles, M.B. & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks: Sage. Moyo, S. (2013). “A study of the possible existence, causes and effects of the mathematical knowledge gap between high school and first tear University mathematics programmes and possible remedies for the situation at UNIVEN: A case study”, [online], http://www.assaf.co.za/wp‐content/uploads/2010/10/Mathematical‐gap.pdf. Accessed 1 February 2014. Muňoz, F.M. and Strotmeyer, K.C. (2010). “Demystifying social media”, Journal of Student Affairs Research and Practice, Vol. 47, 123‐127. Patton, M. Q. (1990). Qualitative evaluation and research methods, 2rd edn, Newbury Park, California: sage Publication. Scott, I. Yeld, N., & Hendry, J. (2007). “A case for improving teaching and learning in South African higher education”, Research paper prepared for the Council on Higher Education. Van der Flier, H., Thijs, G.D. & Zaaiman, H. (2003). “Selecting students for a South African mathematics and Science foundation programme: the effectiveness and fairness of school‐leaving examinations and aptitude tests” , International Journal of Education Development, Vol. 23, 399‐409. Vygotsky, L. (1986).Thought and Language, Cambridge, MA: MIT Press. Welch, B.K. & Bonnan‐White, J. (2012). “Twittering to increase student engagement in the university classroom”, Knowledge Management and E‐learning: an International Journal, Vol. 4, No. 3, 325‐345. Wolmarans, N., Smit, R., Collier‐Reed, B. and Leather, H. (2010). “Addressing concerns with the NSC: An analysis of first‐ th year student performance in mathematics and physics”, Paper presented at the 18 Conference of the South Africa Association for Research in Mathematics, Science and Technology,KwaZulu‐Natal.

30

Telling Tales: Responding to Challenges in Literacy Competencies Using e‐Reader‐Based Programs Paula Charbonneau‐Gowdy Universidad Andres Bello, Santiago, Chile [email protected] Abstract: Current worldwide debates on quality standards in education are interwoven with debates on the use of technology. While technology is profoundly changing the way we live, communicate and learn, relatively little seems to be known about its potential to influence literacies. The limited research that exists in this area, although sometimes encouraging, seems to focus on large‐scale investment in laptop programs that have little relevance or hope in being tried or tested in evolving countries where literacy rates are most indicative of the urgent need for practical and affordable solutions. This paper reports on a 4‐month study, part of a longitudinal study that looks at the power of e‐readers to support change in the literacy habits and ultimately the learning cultures of a group of English as a foreign language (EFL) teachers‐in‐training in Chile. Literacy and how to develop it, is a prime concern in Chile. With the need for worldwide citizens to have increased and diverse competencies in meaning making and in negotiating complex information, many Chileans still lack basic reading skills. The future looks even bleaker as successive groups of teachers, who themselves have serious literacy challenges, are entering the system with little hope of affecting change. The aim of the study was to determine if access to low‐cost mobile readers and a social‐learning based, technology‐supported, guided reading program, could reverse this picture for the participants. The study is based on social‐cultural theory in which learner agency, access to funds of knowledge and social interaction are imperative ingredients for developing engaged, life‐long readers. Participatory Action Research (PAR) is used to conduct the inquiry. Working within a qualitative research paradigm, ethnographic tools are employed to uncover how the use of technology influences both the literacy practices and identities of the teachers‐in‐training, from their perspectives. Numerical data from pre‐ and post‐test results are also analyzed and used to support the findings that patterns of literacy development changed. The evolving changes in the way many of these learners come to view their literacy practices supported by technology, combined with their identity construction over the period of the guided program online, have important implications for other learners, educators and policy makers. The findings suggest a new model of literacy education is needed in such challenging contexts, and particularly here in Chile, that involves accessible technology as an important part of the solution to providing democratic possibilities for all learners to develop the reading habits and competencies they require to be active 21st century global citizens. Keywords: literacy, teacher‐training, e‐reader technology, multi‐literacies, 21st century skills

1. Introduction I have never considered myself as a good reader, because I hate reading my whole life. (Reflective Essay Marie Jesus, January 2014) When I started this new step in my life [university studies] and I read the program I knew that I had to adopt new habits. At the beginning it was terrible because I had to read books that were wrote in English. I never read something in Spanish and now I will have to learn to read in English. (Reflective Essay Karina, January 2014) As the new century progresses, the volume and tone of the conversations around 21st century skills being developed in educational institutions at least in the developed world, seems to be rising to a fever pitch. Given the abounding rate at which technology has infiltrated informal as well formal learning sites, it is not surprising that a great deal of this dialogue is debating how technology will help support the development of these so‐ called essential skills. Paradoxically, along with the recognition that technology will be integrally tied to 21st century skill development is that individuals will need the multi‐literacy skills to take advantage of that technology support. In short, taking full advantage of technology for learning and being an effective and productive global citizen in the 21st century go hand in hand. Many evolving nations, and some developing ones, are left on the sidelines of these dialogues. Most are marginalized from such high‐level debates, as they find themselves still struggling with the challenges of promoting the most basic reading and writing literacy skills among their citizens. The comments of Karina and Marie Jesus, two individuals studying English pedagogy in Chile, reflect those literacy challenges. While it is generally known and accepted that economic and cultural development is tied directly to the literacy levels of a country’s citizens (Mingat & Tan,1996; Matear, 2008; Norton, 2010), in many of these countries finding long‐ term effective strategies to face the enormous challenges of promoting basic reading and writing

31

Paula Charbonneau‐Gowdy competencies, has been elusive. And yet, along with English language learning, it is commonly accepted that these basic skills are critical gateways to developing the much‐acclaimed skills of the century on the micro level. On the macro level of a country, these same basic literacy skills, once well‐honed among its citizens, can assure a voice in the global dialogue not to mention support responsible management of one’s national resources in the global market and sustained economic growth. Chile, a growing economy and recently accepted member to the Organization for Economic Co‐operation and Development (OECD), is one such evolving country. While proud of its economic progress over the last ten years, it is also very aware of its deficiencies, especially in education. It has been in the spotlight for several years as students have taken to the streets demanding solutions to these issues (The Economist, April 12, 2012). A high degree of social stratification is partly at the root of these protests and the deficiencies in education they seek to address. Indeed, a 2009 OECD report placed Chile in the bottom quarter of the list of thirty‐one nations in terms of literacy. Reports on English language literacy skill rates project an equally dim picture (SIMCE 2012, Dowling, 2007). At the same time, heavy investments in technology in public education by government have made little dent in these statistics (FP.cl, 2014). The first ever ICT Sistema de Medición de Calidad de la Educación/Education Quality Measurement System or SIMCE, conducted in the country in 2011, revealed that 50% of secondary students in this country have basic to intermediate digital literacy skills and only 3.3% use computers for their learning. Interestingly, Chile leads the rest of Latin America in the numbers of mobile devices. A surprising finding of a 2010 OECD study on the connection between technology use and educational performance revealed that the new digital divide is no longer solely about access, but rather the one existing between those who have the right competencies to benefit from computer use and those who do not. These findings ring particularly true in my own work in an English teacher education university program in Chile. Most students in the program are not lacking access to technology; almost all carry smart phones, ipads, ipods and tablets. Yet their levels of literacy and English skills confine many to using these tools solely for entertainment or social‐networking; i.e. listening to music, watching movies and connecting with friends on Facebook. Teachers, and indeed the students, recognize these shortcomings. Under the circumstances, it would be reasonable to expect that pre‐service teachers in language should be doing all that is possible to develop the advanced language proficiency and literacy skills that they will need as future teachers of language. Yet in my experience, such is often not the case. Many in the country concerned about education view these pre‐service teachers’ literacy skills, or lack thereof, as a litmus test of the hopelessness that pervades the country for the future of educational development in Chile. In an effort to come to terms with this hopelessness, fingers continue to point from within at the multiple roots of its low literacy rates ‐ a stratified, poor quality and economically‐driven education system, generally low rates of education in parents, poorly trained teachers and the high cost of reading materials, are cited most often. And yet, while regular international standardized testing and its results continue to feed this feeling of hopelessness for reversing or even understanding the issues, little is being done in terms of in‐depth educational research in or for this country to re‐direct this hopelessness toward well‐grounded action and to seek local solutions. As one Dean of an Education Faculty in a large private university in Chile, one of the few that conducts research save for the two elite public universities here, recently reported to me: “There is virtually no research being done in this university in education, let alone in technology and its relationship to literacy development.” (personal communication, January 2014). And with reportedly few educational researchers in Chile fluent in English, participation in solution‐generating dialogues with others from abroad with similar interests in literacy and technology (for example, Warschauer, 2006; 2011; Thorne & Black, 2007; Godwin‐Jones, 2010) is limited. It’s a definite catch‐22 scenario. Without this dialogue and the kind of research that it fosters, education, including finding effective solutions to the literacy issues it faces, could easily remain elusive in Chile. This research report, in part, is an effort to spark dialogue in Chile around seeking literacy solutions through the use of technology. The aim of the study was to determine whether a guided program using e‐readers could influence the low reading rates of a group of pre‐service tertiary level teachers. Some headway into understanding the implications of the use of mobile technologies in particular for literacy are being made in the wider research community (Gee, 2003: also see Baron, 2009 for a summary of this area of research). No grounded research in the use of e‐readers, in particular, seems to be available. The dearth of research is understandable perhaps since the popularity of these mobile devices has only recently taken hold within the general public (Godwin‐

32

Paula Charbonneau‐Gowdy Jones, 2010). Since 2010, when interest in e‐readers first began to soar in the public domain, the affordances of these mobile devices have opened up many questions around learning and literacy practices. The goal in conducting the study was to begin to address some of the questions surrounding the use of these e‐readers through the evidence generated with individual pre‐service teachers. Answers to these questions are crucial, especially given the implications that supporting the development of literacy habits of these future teachers can potentially have on others, for the ten teachers in the study, quite reasonably expected to be over one thousand alone in their first year of teaching. The answers to these questions might also inform the field of education more broadly as the border between formal and informal learning becomes increasingly blurred globally and the use of technology outside the classroom is being recognized as influencing what takes place within. The findings could add further insight for other evolving nations that are searching for context‐ appropriate technology solutions to the educational issues they face.

2. The present study: Addressing literacy challenges with technology The study is an attempt to respond to the literacy challenges that a group of future language teachers face as they prepare to become teachers of English as a foreign language (EFL). The connection between literacy development in English as a second/foreign language and in one’s own native language has been well supported by research (Norton, 2010). The principal objective of the study was to determine whether shorter (4‐month) and longer term (9‐month) access to a guided reading program using technology, in this case mobile e‐reading devices, would have implications for them as learners and for their literacy skills. The present report is of the first four‐month pilot phase of the study. The project that was put in place in early October 2013 involved research on ten EFL pre‐service teachers studying in the undergraduate English Pedagogy program in a large private Chilean university. The focus of the study was examining how their literacy development, and they themselves were influenced over time as a result of participating in this guided reading program. The study was conducted as a means of uncovering many kinds of information about the personal literacy experiences of the individuals who chose to participate in the research project. Three questions guiding the research reported here are:

What was the nature of the previous literacy experiences of the participants and how did they perceive their competencies at the outset of the research study?

What is the influence of the reading program, if any, on these participants’ reading habits and their use of language, in this case English?

How has the reading program affected their identities as learners, future teachers and as individuals?

3. The theory: Literacy, social identity and technology Lipka and Siegal (2011) observe that “strong literacy skills are a prerequisite for success in contemporary society”. (p.1874). They are not alone in their observations. Finding ways that are effective in supporting learners in developing strong literacy practices has been the topic of educational research for several decades. While there is still much conflicting debate on how to foster literacy in formal learning sites, most scholars now agree that literacy development is more than just a cognitive process of phonetic decoding. Literacy development is being recognized as a complex, social process embedded in relationships “connected to different practices and preferences among social classes” (Myrberg and Rosen, 2009, p. 696). From early childhood, children are socialized into certain cultural practices around language and literacy that define who they are as a person ‐ their social identities. These initial interactions in the early stages of children’s lives prepare them culturally, socially and cognitively for fundamentally different life roles (Heath, 1983). The practices that they develop through these early experiences can also predict their future scholastic achievement and group membership. Bourdieu’s concept of cultural capital, (2002) offers a theoretical construct to explain how this social stratification occurs. Cultural background, knowledge, preferences, attitudes and behaviours passed down for example by educated or higher classes to their children are rewarded in the school system and thus reproduce societal inequalities. Although there is substantial evidence to support the influence that cultural capital has on students’ literacy practices and scholastic achievement, there is less known about how various mediating factors, can affect and/or alter these processes.

33

Paula Charbonneau‐Gowdy In order to consider mediating processes that might influence the social stratification that occurs in the development of literacy, we need to understand how literacy, in and of itself, functions. Luke and Freebody’s (1999) model explains that in a postmodern world, literacy involves the reader taking on four distinct roles: Breaker of the language codes; Participator in the meanings of the text; User of text functionally and a Critic, Analyzer and transformer of text. The processes that make up Luke and Freebody’s model are interesting to the present study for several reasons. First of all, they help us to understand where learners are on the continuum in terms of their literacy development – areas where they are able to assume the necessary roles and others where they need guided support. The model also reminds us of the highly contextual nature of literacy. In other words, from a sociocultural perspective, literacy is politically, economically, culturally, historically, pedagogically, linguistically and personally charged, and importantly, embedded in relations of power. Warschauer’s has conducted extensive research (2006, 2011) in the use of laptops in schools. His research has provided grounded evidence of the powerful role these personal IT tools can have on literacy development and learning, especially for learners at risk or marked by their SES (socio‐economic status). The substantial findings he has uncovered in the area of multi ‐ literacies have also allowed him to raise the alarm bell that “it takes more than handing a child [or in this case, a future teacher] a laptop to transform education” (p.ix). This warning offers clear signals to educators and policy makers to consider the multiple factors affecting the success of computer technology in influencing learning, but falls short of determining whether e‐readers, with their unique affordances, can be positive influences in support of literacy development. Drawing on the theory of cultural capital and previous studies in literacy and technology, I support the argument that technology, in this case e‐reader technology, depending on its use, can offer encouraging signs of influencing learners’ identity and literacy development. I base my support on the emerging findings from this Phase 1 of a longitudinal study of the literacy experiences of a group of pre‐service EFL teachers in Chile. In the next sections, I explain the 4‐month study and report on the major findings.

4. Phase 1 of a longitudinal study: Pre‐service EFL teachers as readers Phase 1 of the study ran from late September 2013 to January 2014. The self‐selected participants included a group of 10 pre‐service teachers in their third year of an undergraduate English pedagogy program in Chile. Most of the participants had taken part in a course I had taught the previous semester. The participants were told that the study would involve enhanced opportunities to practice their reading skills using technology. Each participant was given an e‐reader and a steady offering of e‐books, of their own choice. Part of the draw for the participants in agreeing to these sessions was the opportunities it afforded for guided linguistic feedback in the weekly individual and/or pair interviews. These interviews along with periodic whole group meetings were a major source of the data collected. In these meetings, the participants discussed the books they were reading at the time and often commented on their evolving literacy experiences. Written journals that these participants submitted as part of their course work, as well as reflective essays they wrote about their literacy experiences at the end of the Phase 1, also accounted for a substantial amount of the data. Results of two term tests and one final test were consulted and analyzed as a means of looking for trends in the data. From my background knowledge of the individual participants and their skill levels as well as from remarks many made to me, I understood that all of these individuals had various literacy challenges. Many were quite open about their lack of interest in reading and about their feelings of inadequacies as a result. In an informal survey, all but one reported having read less than one book in English or Spanish in the 18 months prior to the study. I based my research questions on the assumption that they wanted to change their literacy levels and that they lacked strategies to do so. For this reason, a Participatory Action Research (PAR) design was considered particularly appropriate. Both participants and I as their teacher/researcher were seeking to make changes to their literacy levels that we saw necessary, not only in terms of their linguistic capabilities in English but also for the potential implications that these changes could have for them as individuals and teachers. From my perspective as a teacher with over 30 years of pedagogical experience and from my knowledge of the more recent literature on reading, I also assumed that giving readers choice and refraining from testing them could have a positive influence on their desire to read. Since the study is connected to the learning of English as well as literacy development, I adopted Spolsky’s (1989) argument that practice and exposure to the target

34

Paula Charbonneau‐Gowdy language are essential for making progress in the language and that that exposure can come from the intersection of both formal exposure in the classroom and/or the informal exposure outside, such as reading for pleasure.

5. The analysis: Cultural capital, identity and investment in literacy change The data that was generated over the first four‐month phase was extensive. The data sources included: research notes from 50 hours of weekly face‐to‐face and/or Skype interviews which ranged from 30 to 60 minutes each; 260 pages of writing journals; evaluation results from two term tests that were offered as part of their course, as well as a pre‐ and post‐ standardized tests in the various language skills; participant generated reflective journals; field notes and observations. These data sets were analyzed using standard qualitative methods for themes and patterns. The numerical data generated by the tests was also tabulated and analyzed. In this section, I highlight some of the findings that have been most interesting in Phase 1 of the study with reference to the questions I have posed. All of the participants in the study expressed with strong conviction that they were motivated to improve their language skills. After all, the majority of these individuals were in the Pedagogy Program to become teachers of language. They had already spent time in classrooms in their teaching practice sessions and they were quite cognizant that knowledge of the language was critical for their survival in their profession. Yet in all but two cases, none of this strong motivation lead to reading of any kind in the year and a half prior to the study. At the same time, they willingly chose to participate in the study despite the extra time it would take from their already busy academic, personal, and work schedules, including for many from a two to four hour commute a day. According to the participants, other factors in their motivation were fear of further course failures, the high cost of their education and the burdens that this cost represented for themselves and their families, the tremendous cultural focus on marks and grades, plus the pride around being a first generation family member to attend tertiary education. These factors caused pressure and anxiety to several of the participants, which was evident in their demeanor, especially in class – shy to engage, hesitant about their accuracy when speaking, a reluctance to write and multiple comments and physical signs of anxiety that I witnessed. An important theme that surfaced in interacting with the participants and that apparently added to their anxiety was their early and more recent experiences in literacy in their own language. Few had access in their childhood to a parent who read to them regularly nor to a wide range of books. Many agreed that literacy development in their early schooling was inconsistent and/or “boring”. Some reported teaching approaches were didactic and consisted often of lack of choice of reading materials or old classics that had little relevance to their lives. These practices were accompanied by frequent testing that reflected an emphasis on copying and/or lifting information from text. Six of the ten participants admitted that they rarely read books in their own language for pleasure while the other four did so irregularly. Reading in English was confined only to what was required in their courses and this requirement in itself was often neglected. All of the participants expressed they felt weak in the syntactic and morphological use of the language. Their comments seemed to be corroborated by the disparaging remarks concerning their linguistic skills from colleagues who also taught some of the same individuals in other courses. In other words, the literacy identities, both in English and in Spanish, that the participants projected and that was reinforced by the impressions of others, including their teachers was one of being “weak, poorly skilled, mark driven yet resistant to working to improve.” Their deficiencies in cultural capital, using Bourdieu’s (1994) construct, was being reflected in the learning context in their images of themselves as deficient as well as in their anxiety as they considered their chances to succeed in the program. Bourdieu’s observation that “The sense of the value of one’s own linguistic products is a fundamental dimension of the sense of knowing the place which one occupies in the social space.” (p. 82) seems particularly relevant in explaining why many of the participants saw themselves as marginalized in the teacher‐training program. Over the period of this first phase of the study, as I met with the participants each week for them to tell the stories of what they were reading – to tell their tales, I began to observe subtle and more obvious indications of change in the way many of the participants’ viewed themselves and their subjectivities. As Diana remarked midway through the first four months of the study: I feel more confident with myself when I’m talking because I didn’t used to be like that. I was always afraid of making mistakes. Now I think the book and the reading is like a support to me. (Diana, Group Interview, November 2013)

35

Paula Charbonneau‐Gowdy Diana, in her own words, clearly expresses these changes in her awareness of her increasing confidence to speak in class. Other signs of changes in Diana and many of the others were their increased vocal proactivity in the classroom despite the presence of more advanced others in the group, the visible exhilaration when some in the group received marks that far surpassed what they had achieved previously, increased attendance in class, more regular submission of assignments, as well as expressed enthusiasm for the books they were reading and the prospects of reading the next. Norton’s (2010) construct of investment, which is broader than motivation, helps explain the changes to participants and to their engagement in literacy practices. Partially due to the convenience of the e‐readers and the access they provided to a whole range of books and learning features, the participants began investing in increased and more regular reading. These literacy opportunities as well as those that came from participating in the guided weekly discussions programs led to the learners beginning to recognize as evidenced from their testimonies and my observations that the value of their “cultural capital” was increasing in the classroom and in the context of the Pedagogy Program. With the increase in their individual cultural capital, there was a corresponding change in their learner identities, or their sense of place in that context. This change in identity was obvious in some of the participants’ willingness to speak more confidently in front of others. Their change in engagement in turn served to help some of the participants view themselves as key players in the classroom and at the same time to further their literacy skills even more. Lave and Wenger’s (1991) theory of a community of practice, in which through a process of legitimate peripheral participation (LPP) individuals move from the margins to the centre of interactive communities by being mentored or “apprenticed” by more powerful others, are further theoretical justification for the changes the participants were experiencing. Evidence of changes in this first phase of the study was not confined solely to the participants’ construction of more empowered literacy identities nor to their own testimonies. There were also encouraging linguistic changes that were revealed in the data set.

evidence of more regular and intensive reading of English, an average of 3 books per participants over 10 weeks;

increase in level of length and difficulty of books chosen;

noticeable decline in the number of major errors in weekly journals especially syntactical, although there was a less evident decline in mechanical errors.

With regard to Luke and Freebody’s model of literacy, there was also strong evidence to support that the participants moved through the first 3 stages of the model. Karina’s remark: “During the first pages in the first book that I read, I had to learn what the book was trying to say me… First I read one hour per day with some difficulties, but then I was extending the hour to two hours then three and so on. (Reflective Essay, Karina, January 2014) is just one example. Other examples included the participants who revealed that they regularly spoke to one another in English as a means of practicing and another explained that she was reading regularly to her younger sister. Yet, unsuccessful attempts to encourage a deeper discussion of topics indicated that most of these individuals had not progressed to the stage of critical analyzer. The numerical data that was collected during this first phase of the study involved the results of two term tests in their Language program that coincided with the outset of the study and at the end of three months. Students in the program are tested in all four literacy skills – reading, writing, listening and speaking. A comparison was made of the changes to the grades of the participants as opposed to those of their classmates, the so‐called control group, who did not participate in the e‐reader program. The results of the tests showed little difference when comparing changes in listening and writing. On the other hand, there were interesting differences with regard to speaking and to some extent reading (See figure 1 and 2). For example, there was a 26 % increase in in the average speaking grade of participants on Term Test 2 in relation to their average speaking grades on Term Test 1. On the other hand, the averages for their classmates, the quasi‐control group was 16%. These results are especially encouraging for two reasons. Firstly, generally the participants in the study underperformed on tests for all skills, especially speaking compared to the rest of the class. Secondly, given Vygotsky’s theory that learning is dialogic, that is that cognitive change begins with social interaction, the fact that the participants have shown significant progress in their oral skills suggests that changes to their other skills can be expected but in the longer term.

36

Paula Charbonneau‐Gowdy Table 1: Comparative grades on term tests

Table 2: Pre and post speaking test results on standardized tests

Similar impressive changes to speaking and reading test results were noted for the participants in on pre and post standardized practice tests, the Cambridge University based First Certificate of English practice tests. On these tests, the participants improved their average grades by 20% and 15% in speaking and reading respectively. It is of course difficult to untangle cause and effect for the progress suggested by the numerical data; numbers can be so imprecise when used to understand human behaviour, regardless of the means used to gather them. Yet, if we agree with Krashen (2004) and others who argue that free reading is a major factor in the development of language, and with Vygotsky (1986) that learning is dialogic, then it is clear that the reading strategies the participants were engaged in was certainly a factor in the improvement in literacy skills that was recorded.

6. Conclusion The findings of the study revealed the literacy development of the 10 preservice teachers over the course of the 4‐month initial phase of the study. One may question the connection of technology to the rather impressive results that this first phase revealed given the short timeline. A critical reader might further suggest that an extensive library would serve the same end in supporting the literacy development of the participants. While of course there is some validity to these observations, the point is that prior to the study most of these individuals had not had such an opportunity to rich sources of literacy tools. For most of these individuals there was a lack of reading materials and occasions to use language to discuss what they were reading, if at all, both at home and at school. The significance of this study is that it provided clear evidence that technology, in the form of e‐readers, was able to offer support for reversing to some extent the shortage of culture capital that the participants in the study had and portrayed in their academic studies. These changes could be short lived. Since the end of the study, as many of the students have left the university for the holiday break, there has certainly been a decrease in requests for e‐books and most have not followed up on offers for weekly meetings that were

37

Paula Charbonneau‐Gowdy scheduled on Skype. Yet, the engagement and investment that these learners demonstrated over the course of this first phase of the study is compelling and serves as a very strong reason for continuing. Guthrie (2004) offers a further one. His findings have shown that student engagement is one of the critical factors leading to success in literacy and that, over time, students from low income and low education backgrounds but who are highly engaged readers will substantially outscore more privileged students. The higher percentage of average increase in test results on the part of the participants compared to others in their language course could be one example of this phenomenon. The confident and active learner roles that the participants have assumed in the classroom and that reflect more empowered teacher/learner identities prior to taking part in the literacy program with e‐readers, is another Both Guthrie and Cummins (2007) offer convincing support to arguments that the increased reading that the participants chose enthusiastically to do over most of the four months, their obvious enjoyment in doing so and also in other forms of literacy such as speaking, writing and listening will result in a propensity to pursue literacy activities more actively within the context of their studies and beyond. This is precisely the kind of self‐regulation that is so lacking generally in the Chilean education system. It appears that the affordable and recent technology innovation, i.e. e‐readers, used in the study, combined with opportunities for guided reading support, could offer a potential effective model and be an important part of a pedagogical solution to the literacy woes that this country and others may be seeking. Further in‐ depth studies of the application of the model for institutional pedagogical practice in teaching literacy suggested in this research are needed at all levels of the education system and in variety of contexts both in Chile and internationally to determine its long‐term effectiveness. When it comes to literacy development, the uses of technology to support long‐term solutions are the only ones that matter.

References Baron, N. (2008) Always On, New York: Oxford University Press. Bourdieu, P. (1991) Language and Symbolic Power, Cambridge, MA: Harvard University Press. Bourdieu, P. (1994). Free Exchange. Stanford: Stanford University Press. Bourdieu, P. (2002) ‘The forms of capital’ in Halsey A. H., Lauder H., Brown, P. and Stuart Wells, A. (ed.) Education –Culture Economy, Society, pp. 46‐58, Oxford: Oxford University Press. Cummins, J., Brown, K. and Sayers, D. (2007) Literacy, Technology and Diversity: Teaching for Success in Changing Times, Toronto: Pearson. Darling, J. (2007). English teaching in Chile: A failing grade. October 1‐ Business Chile Magazine. Retrieved from: www.businesschile.cl/en/news/reportaje‐prinicipal/english‐teaching‐chile‐failing‐grade. DF.Cl (2014) Chile’s Financial Journal. Retrieved from: http://www.df.cl/las‐herramientas‐tecnologicas‐y‐metodologicas‐ que‐impactaran‐las‐aulas‐en‐2014/prontus_df/2014‐01‐17/203640.html Gee, J.P. (2003) What Video Games Have to Teach us about Learning and Literacy, New York: Palgrave Macmillan. Godwin‐Jones, R. (2010) Emerging technologies, literacies and technologies revisited. Language Learning and Technology, Vol. 14, No. 3, pp 2‐9. Guthrie, J.T. (2004). Teaching for literacy engagement. Journal of Literacy Research, Vol. 36, pp 1‐30. Heath, S. B. (1983) Ways with words: Language, Life, and Work in Communities and Classrooms, Cambridge, England: Cambridge University Press. nd Krashen, S.D. (2004) The Power of Reading 2 edition, Portsmouth, NH: Heinemann. Lave, J. and Wenger, E. (1991) Situated Learning. Legitimated Peripheral Participation, Cambridge, MA: Cambridge University Press. Lipka, O. and Siegel, L. (2011) The development of reading comprehension skills in children learning English as a second language. Reading and Writing Journal, Vol. 25, No. 8, pp 1873‐1898. Luke, A. and Freebody, P. (1999) A map of possible practices: Further notes on the four resources model. Practically Primer, Vol. 4, No. 2, pp 5‐8. Retrieved at: http://www.readingonline.org/research/lukefreebody.html Matear, A, (2008). English language learning and education policy in Chile: Can English really open doors for all? Asia Pacific Journal of Education, Vol. 28, No. 2, pp 131‐47. Mingat, A. and Tan, J. (1996) ‘The full social returns to education: Estimates based on countries’ economic growth performance’, Human Capital Development Working Papers, The World Bank Group, Washington, DC. Norton, B. (2010). Identity, literacy and English‐language teaching. TESL Canada Journal/Revue TESL DU Canada, Vol. 28, No.1, pp 1‐13. Spolsky, B. (1989) Conditions for Second Language Learning, Oxford: Oxford Univer‐ sity Press. The Economist (2012) Lessons from Students, Retrieved on April 14, 2012: http://www.economist.com/node/21552573?zid=305&ah=417bd5664dc76da5d98af4f7a640fd8a Vygotsky, L. S. (1986) Thought and Language, Cambridge, MA: MIT Press. Warschauer, M. (2006). Laptops and Literacy: Learning in the Wireless Classroom, New York: Teachers College Press. Warschauer, M. (2011). Learning in the Cloud: How (and why) to Transform Schools with Digital Media, New York: Teachers College Press.

38

Factors Leading to Technology Non‐Use in Education: Validation of Reversed‐UTAUT (R‐UTAUT) Phaik Kin Cheah1, Yee Onn Yap1, Prabha Unnithan2, Priscilla Moses1, Fong Wei Diong1 and Pek Hoon Er1 1 Universiti Tunku Abdul Rahman, Kampar, Malaysia 2 Colorado State University, Fort Collins, USA [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] Abstract: The integration of technology in teaching and learning is essential in the development of any nation. It is one of the key areas that the Malaysian government has identified in its plans and policies to digitize and globalize education. Malaysia’s National Education Blueprint 2013 however reported that 80% of school teachers in Malaysia use Information and Communication Technologies (ICT) for less than one (1) hour each week, and students perceive that teachers are not regular ICT users. This study measures the level of resistance of technology use among educators in institutions of higher learning in Malaysia. A review of past research found many studies using the Unified Theory of Acceptance and Usage of Technology (UTAUT) as a model to measure user intention and behavior. UTAUT was consistently cited in its ability explain factors related to technology usage and acceptance. Thus, UTAUT was adopted and the items reversed. This study presents the conceptual idea pertaining to validating the instrument by reversed items. Future research will involve a larger number of respondents. Keywords: UTAUT, reversed UTAUT, online instruction, higher education, non‐adoption, Malaysia

1. Introduction Information and Communication Technologies (ICT) play an important role in Malaysia's economic development and various projects implemented for the development of ICT. There is a positive correlation between growth in a state’s ICT competitiveness and the state’s ability to compete economically and internationally (Igari, 2014). Some of the projects under Malaysia’s plan to boost ICT adoption are the Malaysian Information, Communications and Multimedia Services 886 Strategy (MyICMS 886) which is aimed at providing high speed broadband services and the Multimedia Super Corridor Malaysia 2.0 which is aimed at improving quality of life through ICT‐enabled services (Ministry of Higher Education Malaysia, 2012). A survey of internet development in 28 Asian countries found that the government plays a crucial role in implementing the adoption of ICT (Hao & Chow, 2004). Much effort has been put into the development of ICT infrastructure development in the educational setting by the Malaysian government. Malaysia’s National Education Blueprint 2013 – 2025 stated that RM6 billion will be invested in ICT initiatives for education projects (Ministry of Education, 2012). Meanwhile, Centre for British Teacher (CfBT) Education Trust, in its commentary report on National Education Blueprint 2013, reported that students and academic staff should leverage on ICT more efficiently to boost the learning process in the future (CfBT Education Trust, 2012). Therefore, in line with Malaysia’s Vision 2020 goal to achieve the status of a developed nation, it is predicted more radical changes and improvement in technology usage and acceptance will take place. However, resistance of technology among users has been found to be a key problem among ICT implementers (Rivard & Lapointe, 2012). Educators' ability to utilize ICT is associated with the qualities of coming generations. ICT proved able to boost education levels (CfBT Education Trust, 2012). Affiliated with this statement, one of the visions stated in National Education Blueprint 2013 – 2025 is to normalize the ICT facilities among educators (Ministry of Education, 2012). This helped formulating policies for boosting technology acceptance among educators.

2. Literature review and theoretical context One of the common ICT facilities in a university is the internet portal. The internet portal has go beyond the monotonous function of providing only announcements to multitudinous functions including the sharing of

39

Phaik Kin Cheah et al. learning materials, discussion over a lecture’s contents, etc. In Malaysia, it is common to incorporate the use of ICT in higher education. However, Malaysia’s National Education Blueprint 2013 reported that 80% of school teachers in Malaysia use ICT for less than one (1) hour each week, and students perceive that teachers are not regular ICT users (Ministry of Higher Education Malaysia, 2012). As for educators in institutions of higher learning in Malaysia, they face challenges in adapting and lack skills in integrating ICT in teaching and learning (Cheah, Kok & Chew, 2012). Although they were aware of the technologies and the benefits it brings, they were found to be adamant because of the lack of rewards and recognition from the management, and lacked confidence in blended learning (Cheah, Kok & Chew, 2012). In another study conducted among teachers in Malaysia, it was found that the students’ reasoning skills were not promoted by the teachers when ICT is applied in their teaching practice (Ling & Rohaida, 2013). This problem should be highlighted in both pre‐service and in‐service training courses so that teachers could be directed to apply ICT and emphasize on guiding students through constructivist methods (Ling & Rohaida, 2013). In a study on the use of ICT in Japan, Igari (2014) reported that the country’s innovative infrastructure is not reflected in its application of ICT. Igari (2014) suggested that in order to improve the usage of ICT effectively, it is important to develop facilities emphasizing on users and ensure proactive implementation of ICT policies by the authorities. Lapointe and Rivard (2005) outlined five aspects of resistance which include “subject of resistance”, “object of resistance”, display of resistance, identified threats, and preliminary situations. In response to those attributions of resistance, Rivard and Lapointe (2012) suggested that to overcome user resistance, implementers should identify the actual causes of the resistance behaviors beforehand, identify the right object of rectification, ensure credibility in implementing accommodating and imposing persuasions, and select effective responses to avoid inaction. Those aspects of resistance are significant because they are deemed to affect the prospects of effectiveness and end results (Rivard & Lapointe, 2012). Venkatesh, Morris, Davis, and Davis (2003) proposed the Unified Theory of Acceptance and Usage of Technology (UTAUT) to explain the factors related to technology acceptance and usage. UTAUT proved to be giving more holistic explanation compared to other models since it is compressed of 32 variables from previous researches, which are Theory of Reasoned Action (TRA), Theory of Planned Behavior (TPB), Technology Acceptance Model (TAM), Motivational Model (MM), combined TAM and TPB (C‐TPB‐TAM), Model of PC Utilization (MPCU), Innovation Diffusion Theory (IDT), and Social Cognitive Theory (SCT). All previous model are synthesized into four main effects and four moderating effects in UTAUT (Oye, Iahad, & Nor Zairah, 2012; Venkatesh et al., 2003). Williams, Rana, Dwivedi, and Lal (2011) systematically reviewed 450 journals with UTAUT cited. They found that only 43 actually utilized the theory in examining information systems or information technology related issues, most of the studies only partially used UTAUT in their research while some studies combined UTAUT and other external factors. Williams et al. (2011) proposed that UTAUT needs to be further analyzed in other method rather than merely citing it. Based on this research need, this research further analyzes and validates the UTAUT in reversed items. This study contributes an alternate instrument to obtain ratings of technology resistance. In this study, "technology resistance" is interchangeable terminology for non‐ use of technology.

3. The purpose of study To validate a new questionnaire R‐UTAUT which aims to test the level of technology non‐use in teaching and learning among educators.

4. Methodology All the survey questions were uploaded to a Google document using survey form format. Educators who were available to researchers received the e‐mail link to the online survey form. During pilot testing phase, 31 questionnaires were distributed to educators in UTAR for pilot testing. The data from the pilot test underwent Cronbach's Alpha reliability test. Adjustment on the actual questionnaire will be implemented later based on the results of the pilot test. Therefore, these 31 responses will not be included into the later actual testing phase. Lastly, actual testing would involve 300 lecturers from Universiti Tunku Abdul Rahman (UTAR), Universiti Malaya (UM) and Universiti Sains Malaysia (USM). Data from actual testing would undergo Structural Equation Modeling to confirm the theoretical constructs.

40

Phaik Kin Cheah et al.

5. Instrument In the literature of technology acceptance, there is no questionnaire specifically designed to test the level of technology resistance or level of non‐ use of technology. This research is also aimed to design a specific questionnaire to measure the level of technology resistance by adopting the questionnaires proposed by Venkatesh et al. (2003). Venkatesh et al. (2003) stated that this scale accounted as much as 70% of the variance in technology acceptance. Follow‐up researchers who adopted this questionnaire also reported similar results. UTAUT is proven as a reliable model in assessing technology acceptance and usage. Yahya, Nadzar, and Rahman (2012) used UTAUT to test E‐Syariah portal usage among Syariah users in Malaysia and a score of 0.7 Cronbach’s Alpha was reported indicating acceptable reliability. In another study, Singeh, Abrizah, and Karim (2013) reported 0.864 of Cronbach’s alpha level for the same instrument with a sample size of 108 academic staffs in institutions of higher learning. There are 16 survey questions which relates to the four major factors in UTAUT (Performance Expectancy, Effect Expectancy, Social Influence, and Facilitating Conditions). All the survey questions possessed the possible responses on a 5‐point Likert scale whereby 1 is "strongly disagree" and 5 is "strongly agree." Those four major factors are then linked to prediction of technology acceptance, and behavioral intention of using the system. There are three survey questions in total to test the behavioral intention of using the system. All the items were reversed by two researchers independently. Then the statements were compared. The final version of reversed items was proposed based on the agreement among both researchers. This final version of reserved items was later checked by two English language experts from the Department Languages and Linguistics of UTAR to ensure the face validity. The reversed questionnaire is named R‐UTAUT to specifically test the level of technology resistance.

6. Sample size This research uses the principle given by Roscoe (1975) to decide the sample size. According to Roscoe (1975), multivariate research sample size requires 10 times the number of variables in by the study. The sample size should be kept within 30 to 500 (Abranovic, 1997; Sekaran, 2003). According to Abranovic (1997), 98% of the time a sample size of 500 would not exceed 10% of standard deviation. As mentioned by Davis, Gallardo and Lachlan, (2012), all populations will fall into normal distribution regardless of size. Hence, a sample size of either 100 or 1000 would conform to similar normal distribution as long as the characteristics of sample are similar to the population being studied and the size is not overly small. A few studies have been carried out for determination of sample size for social science research. Krejcie and Morgan (1970) proposed a table for easy reference whereby a sample size of 382 is able to represent a population of 10,000. In general, a range between 300 and 400 is an ideal sample size. Williams, Onsman, and Brown (2010) proposed that sample size of at least 300 is needed for factor analysis. Since this research consists of 19 survey items, the ideal sample size would be 190 (19 x10 = 190) for the actual testing phase. Sufficient sample size is critical to safeguard model identification. Sample covariance matrix S only positive‐ define when the sample size higher than number of indicator, this is the basic assumption of Structural Equation Model (SEM) (Reinartz, Haenlein, & Henseler, 2009). Given the nature of covariance‐based SEM, unlimited increasing of sample size does not impact the reliability of the estimate (Reinartz et al., 2009). This suggested that researchers using SEM need to go beyond of the minimum amount of the proposed sample size to ensure unbiased results. Based on this stand, we decided the sample size should be 300.

7. Ethical consideration This research received consent from all participants that their responses would be included in resulting publications anonymously. All indicators towards participants identity in data was replaced with codenames. If the participants ever felt uncomfortable with the research, they were allowed to withdraw at any time. All the participants in this research were informed the objectives, process, risks and benefits of the study. Official permission of this research was granted by Scientific and Ethical Review Committee (SERC), UTAR.

41

Phaik Kin Cheah et al. Table 1: Statements in R‐UTAUT Items Performance Expectancy 1. I would find the system not useful to my job. 2. My tasks been dragged due to the using of the system. 3. Using the system decreases my productivity. 4. I will have lower chances of getting a raise if I use the system. Effort Expectancy 1. My interaction with the system would be confusing and incomprehensible. 2. It would be difficult for me to become skillful at using the system. 3. I would find the system difficult to use. 4. Learning to operate the system is difficult for me. Social Influence 1. People who influence my behavior think that I should not use the system. 2. People who are important to me think that I should not use the system. 3. The senior management of this business has not been helpful in the use of the system. 4. In general, the organization has not support the use of the system. Facilitating Condition 1. I do not have resource that is necessary to use the system. 2. I do not have the knowledge necessary to use the system. 3. The system is not compatible with other system I use. 4. No specific person (or group) is available for assistance with system difficulties. Behavioral Intention to use the system 1. I intended to use the system less frequently. 2. I predicted that I will use system less frequently. 3. I do not plan to utilize the system more frequently.

8. Result This study recruited 31 academic staff from the Faculty of Arts and Social Science, UTAR. Among those participants, there were 25 female staff and 6 male staff. Their age ranged from 24 years old to 58 years old, the mean age is 32.5 years old. In terms of years of service, it ranged from 0.25 years to 17 years. The mean for years of service among the participants is 3.57 years. Table 2 below provides more detailed data regarding participants in the pilot study. Table 2: Demographic data of participants in pilot study

Frequency

Percentage

Gender

Female Male

25 6

80.6 19.4

Academic Position Tutor Lecturer Assistance Professor Associate Professor

8 19 3 1

25.8 61.3 9.7 3.2

In this preliminary study, Cronbach’s alpha for the 19‐items scale is 0.937, which is above the acceptable criterion for a developed instrument of 0.8 (Carmines & Zeller, 1979; Reinartz et al., 2009). Item means ranged from 2.0 to 3.19 with grand mean of 2.526. This result showed that R‐UTAUT is reliable for further examination in actual testing. Table 3: Item analysis of R‐UTAUT Items PE1 PE2 PE3 PE4 EE1 EE2

M(SD) 2.00 (1.18) 3.03 (1.64) 2.38 (1.56) 2.58 (1.46) 2.45 (1.52) 2.45 (1.38)

Item‐Scale Correlations .80 .35 .76 .72 .65 .62

42

Phaik Kin Cheah et al. Items EE3 EE4 SI1 SI2 SI3 SI4 FC1 FC2 FC3 FC4 BI1 BI2 BI3

M(SD) 2.35 (1.28) 2.29 (1.27) 1.97 (1.11) 3.19 (1.92) 1.81 (1.11) 2.61 (1.38) 2.29 (1.48) 3.19 (1.18) 2.77 (1.76) 2.58 (1.59) 3.00 (1.79) 2.55 (1.61) 2.48 (1.50)

Item‐Scale Correlations .71 .60 .53 .62 .58 .72 .67 .71 .69 .64 .60 .50 .69

Note. PE = Performance Expectancy, EE = Effect Expectancy, SI = Social Influence, FC = Facilitating Conditions, and BI = Behavioral Intention

9. Discussion UTAUT been widely accepted as a validated and predictive model in explaining issues related to ICT adoption and usage (Venkatesh et al., 2013). Restructuring of the survey questions into reversed form is predicted to negatively correlate with original version of questionnaire. Inconsistency of response to regular items and reversed items are termed misresponse (Weijters & Baumgartner, 2012). It proves the need for altering the instrument if there are inconsistent results. According to the result from the pilot test, reversed items achieved high internal consistency with Cronbach Alpha: 0.937. This result showed that R‐UTAUT is capable of measuring the level of technology resistance consistently. In terms of validity, face validity is ensured by multiple experts in related fields. Using reversed items in a questionnaire is a common practice. It helps identify agreement or acquiescence bias (Weijters & Baumgartner, 2012). Weijters and Baumgartner (2012) stated that reversed items served as cognitive “speed bumps” and improve scale validity. Nineteen survey questions in UTAUT instrument do not contain reversed items. UTAUT consistently scored high reliability in previous researches (Venkatesh et al., 2013). However, it still lacks some controlling items to avoid potential agreement bias by participants. Survey items coded in the same direction and similar wording may result in a good alpha coefficient, and participants who respond to this survey may answer based on confirmation bias which misleads the conclusion (Weijters & Baumgartner, 2012). However, Swain, Weathers and Niedrich (2008) proposed that reversed Likert items is problematic as it contributed nearly 20% of misresponse on the average. Hence, careful examination of the use of reverse items is needed. This research aimed to provide an alternate survey that is able to specifically test the factors related to level of technology resistance among educators in higher learning institute of Malaysia. Besides, this study analyzes and validates UTAUT instrument from different perspectives.

Acknowledgements This study was funded by the Universiti Tunku Abdul Rahman Research Fund.

References Abranovic, W. A. (1997). Statistical Thinking and Data Analysis for Managers. Reading, MA: Addison‐Wesley. Carmines, E. G. & Zeller, R. A. (1979). Reliability and Validity Assessment. Berverly Hills, CA: SAGE Publications. CfBT Education Trust. (2012). CfBT Education Trust – commentary on the Malaysian Education Blueprint 2013 – 2025. Putrajaya, Malaysia: Government Printing Office. Cheah, P. K., Kok, J. K. & Chew, W. L. (2012). A qualitative inquiry of online instruction among teachers in Malaysian universities and colleges. International Journal of e‐Education, e‐Business, e‐Management and e‐Learning, Vol 2, No. 2, pp 135‐139. Davis., C. S., Gallardo. H. L., & Lachlan., K. A.& (2012). Straight talk about communication research methods (2nd ed.). Dubuque, IA: Kendall Hunt. Hao, X. M., & Chow S. K. (2004). Factors affecting internet development: An Asian survey. First Monday, Vol 9, No. 2, pp 1‐ 25.

43

Phaik Kin Cheah et al. Igari, N. (2014). How to successfully promote ICT usage: A comparative analysis of Denmark and Japan. Telematics and Informatics, Vol 31, pp 115‐125. Krejcie, R. V. & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, Vol 30, pp 607 – 610. Lapointe, L. & Rivard, S. (2005). A multilevel model of resistance to information technology implementation. MIS Quarterly, Vol 29, No. 3, pp 461‐491. Ling, P. Y. & Rohaida, M. S. (2013). Use of Information Communications Technology (ICT) in Malaysian science teaching; A microanalysis of TIMSS 2011. Procedia – Social and Behavioral Sciences, Vol 103, pp 1271‐1278. Ministry of Education. (2012). Pelan Pembangunan Pendidikan Malaysia 2013 – 2025. Putrajaya, Malaysia: Government Printing Office. Ministry of Higher Education Malaysia. (2012). ICT human capital development. Putrajaya, Malaysia: Government Printing Office. Oye, N. D., A.Iahad, N. & Nor Zairah, A. R. (2012). The impact of UTAUT model and ICT theoretical framework on University academic staff: Focus on Adamawa State University, Nigeria. International Journal of Computers & Technology, Vol 2, No. 2, pp 102‐111. Reinartz, W., Haenlein, M., & Henseler, J. (2009). An empirical comparison of the efficacy of covariance‐based and variance‐based SEM. International Journal of Research in Marketing, Vol 26, No. 4, pp 332–344. Rivard, S. & Lapointe, L. (2012). Information technology implementers’ responses to user resistance: Nature and effects. MIS Quarterly, Vol 36, No. 3, pp 897‐920. nd Roscoe, J.T. (1975). Fundament al Research Statistics for the Behavioural Sciences, 2 Edition. New York, New York: Holt Rinehart & Winston. Sekaran, U. (2003). Research methods for business: A skill buildings approach. 4th Ed. New York, New York: John & Wiley. Singeh, F. W., Abrizah, a., & Karim, N. H. A. (2013). Malaysian authors’ acceptance to self‐archive in institutional repositories: Towards a unified view. The Electronic Library , Vol 31, No. 2, pp 188–207. Swain, S. D., Weathers, D., & Niedrich, R. W. (2008). Assessing three sources of misresponse to reversed Likert items. Journal of Marketing Research, Vol 45, No. 1, pp 116‐131. Venkatesh, V., Brown, S. A., Bala, H. (2013). Bridging the qualitative‐quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS Quarterly, Vol 37, No. 1, pp 21 – 54. Venkatesh, V., Morris, M., Davis, G., & Davis, F. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, Vol 27,No. 3, pp 425–478. Weijters, B., & Baumgartner, H. (2012). Misresponse to reversed and negated items in surveys: A review. Journal of Marketing Research, Vol 49, No. 5, pp 737‐747. Williams, B., Onsman, A., & Brown, T. (2010). Exploratory factor analysis: A five‐step guide for novices. Journal of Emergency Primary Health Care, Vol 8, No. 3, pp 30‐39. Williams, M. D., Rana, N. P., Dwivedi, Y. K., & Lal, B. (2011). Is UTAUT really used or just cited for the sake of it? A systematic review of citations of UTAUT’s originating article. In European Conference on Information Systems (ECIS). Wong, K. T., Teo, T., & Russo, S. (2013). Interactive Whiteboard Acceptance: Applicability of UTAUT model to student teachers. Asia‐Pacific Educational Research, Vol 22, pp 1 – 10. Yahya, M., Nadzar, F., & Rahman, B. A. (2012). Examining user Acceptance of E‐Syariah Portal Among Syariah users in Malaysia. Procedia ‐ Social and Behavioral Sciences, Vol 67, pp 349 – 359.

44

Tracing Digital Traces of Phatic Communion in #mooc Judith Enriquez‐Gibson Faculty of Education, Health and Community, Liverpool John Moores University, UK [email protected] Abstract: This paper meddles with the notion of phatic communion introduced 90 years ago by Malinowski, who was a Polish‐born British anthropologist. It explores the phatic in Twitter within the contents of tweets related to moocs (massive online open courses) as a topic or trend. It is not about moocs though. It is about practices that could easily be hidden or neglected if we let big or massive topics take the lead or if we simply follow the computational or secret codes behind Twitter itself and third party software analytics. It draws from media and cultural studies. Though at first it appears data‐ driven as I submitted data collection and analytics into the hands of a third party software, Twitonomy, the aim is to follow how phatic communion might be practised in a social media site, such as Twitter. Lurking becomes its research method to analyse mooc‐related tweets. A total of 3,000 tweets were collected on 11 October 2013 (UK timezone). The emphasis of lurking is to engage with Twitter as a system of connectivity. One interesting finding is that a click is in fact a phatic practice. A click breaks the silence. A click in one of the mooc website is actually a tweet. A tweet was posted on behalf of a user who simply chose to click without formulating the text and perhaps without knowing that it contains #mooc. Surely, this mechanism is not about reciprocity. To break the silence, users did not use words. They just clicked the ‘tweet button’ on a mooc website. A click performs and maintains connectivity – and Twitter as the medium in attendance in our everyday, available when needed to be of service. In conclusion, the phatic culture of breaking silence in Twitter does not have to submit to the power of code and analytics. It is a matter of human code. Keywords: click, Twitter, phatic communion, social media data, mooc

1. Introduction Social media has increasingly and irreversibly taken centre stage in people’s mode of presence and connectivity. It has been used in every aspect of everyday life, for instance, in the food and agriculture sector (York and Brewster, 2013). It has changed the way we communicate and has open new ways on how we interact with one another. This paper is a continuation of an attempt to keep talking about what I have tried to grapple with in unsystematic ways in the last three years. I will meddle with the notion of phatic communion introduced 90 years ago by Malinowski, who was a Polish‐born British anthropologist. I must mention that he coined the term participatory observation. I will draw closer to the phatic and yet not in a manner at all participatory. So perhaps Malinowski would not be really pleased with me in setting phatic communion on a virtual stage and to make it worse in a social media site called Twitter. I proceed to do this because I have pet‐peeves that relate to the trendy topics of big data and moocs (massively open online courses). I would like to locate or dislocate such topics within educational research or within social science research. I would like to step aside and consider the small and less ‘infographics’ aspects of tweets (posts to Twitter) with #mooc or #MOOC in the 140‐character limit. The sole reason for choosing #mooc is its currency in the discussions related to the future of e‐learning and higher education at large. MOOCs become an important focus to consider big data and the value of phatic communion to pursue and examine an alternative framework beyond a news media model for education and the role of social media sites as social technologies. This paper is really a small talk, both constrained and contained in its arguments. It chose to deal with massive in social media within tweets about moocs. It is not about moocs though. It is about practices that could easily be hidden or neglected if we let big or massive topics take the lead or if we simply follow the computational or secret codes behind Twitter itself and third party software analytics. To attend to the small I had 3000 tweets that prompted me. This may seem contradictory at first. But as tweets and hashtags are brought into view by digging deeper and not stopping at the layer of the massive visual display of text, the phatic in #mooc is examined. In the following sections, I will first of all introduce hashtag in Twitter and how it is in fact a practice of human coding beyond machine learning. Then, phatic communion is defined in the context of big data and moocs. I draw from media and cultural studies and though at first it will appear data driven as I submitted data collection and analytics into the hands of a third party software, Twitonomy, the aim is to follow in the big data the less obvious and the less understood quite linguistically. Finally, the findings are rather small and yet present some useful insight of the role of phatic

45

Judith Enriquez‐Gibson communion in both hashtags as human codes and clicks (i.e. mouse clicks or key presses) as computer code manipulations.

2. Hashtag as human code Launched in 2006, twitter as a social technology provided and elicited differences in its make‐up and making over the years. It’s original tagline, ‘What are you doing?’, made it a popular friend‐following tool for updates and whereabouts without hashtags. The first hashtag was used by the Google interface designer and BarCamp cofounder, Chris Messina, back on 23 August 2007 with a question on Twitter: How do you feel, he asked, "about using # (pound) for groups. As in #barcamp [msg]?" (McFedries, 2013). It was obviously intended for tweeting groups and yet the time it finally caught on was during the San Diego forest fires in October 2007. People tweeted about the emergency using the same hashtag, #sandiegofire (Sutton, et al, 2008). Most of the tweets were said to be 40% ‘babble’ (BBC News, 2009) or daily chatter (Java et al, 2007). In November 2009, its original tagline was changed to ‘What’s happening?’ Roger (2013) suggests this is a significant shift from an ego‐centric (friend‐following tool) to a reporting (sharing news) machine. Hashtags are the primary resource for breaking news, distributed in a much faster rate than news media could manage. Does this mean that the ambient and phatic intimacy of small talk is lost? Is it all about having an audience and being a micro‐celebrity (Marwick & boyd, 2010). This paper argues that phatic communion could not be neglected. Hashtags do not only classify information or news events. They are also used to disclose one’s physical or emotional state (e.g. #happy, #ugly) and situational context (e.g. #tgif). The hashtag is the “smiley” that Twitter doesn’t have. With has[h]tags, you don’t need to scroll through a list to choose the image best representing your emotional state. You can simply invent it on the spot, and combine multiple complex feelings as well as situational context (Arnoux, 2013, online). In short, a hashtag is a human code used to find and exchange a wealth of information whether based on a subject, topic, news, event or feeling. Contrary to what the Twitter team deemed to be more effective to be executed and filtered through the use of machine learning (Gannes, 2010), hashtags have become codes written by Twitter users and not its programmers. As a human code, Twitter and its tweets and users could not be simply quantified into big data and delivered as infographics. For this reason, this paper traces phatic communion in mooc‐related tweets.

3. Phatic communion Phatic communion is defined by Malinowski (1923) as ‘… a type of speech in which ties of union are created by a mere exchange of words’ (p315) irrespective of those words’ informational content, and against a background where ‘the breaking of silence, the communion of words is the first act to establish links of fellowship … ‘ (p 314 cited in Frosh, 2011, p 385). It is a kind of chit chat with the intent to stay in touch or to recognise other’s presence (Coupland and Coupland, 1992). In short, its main purpose is not to carry a message, but to establish presence.

4. The phatic in #mooc Twitter has been described as a reporting machine (Rogers, 2013). After analysing 106 million tweets in 2009, Kwak and colleagues (2010) concluded that Twitter has become a news media, where users broadcast to followers with low reciprocity in following. Low reciprocity was interpreted as a shift from the social to the informational. This assumes that the social is a dyad interaction and requires attention between the follower and the followed. This is rather misleading when it comes to mediated sociality. On Twitter, unlike in facebook, you are not watching the person. Typically, one would say that you are watching their content, but more importantly, their presence or movements. Twitter users do not have to know each other to follow one another. Hence, as clearly articulated by Coupland and Coupland (1992), “[o]ur most instrumental, transactional encounters are pervasively organised around multiple (italics in the original) interactional goals that go well beyond the transmission and reception of factual information. In short, it is quite wrong to isolate a discourse/talk mode that embodies relational closeness as some partial or minor set of communication” (p 211).

46

Judith Enriquez‐Gibson A total of 3,000 tweets were collected on 11 October 2013 using Twitonomy. Twitonomy produces descriptive statistics and a downloadable .csv file which contains time‐stamped tweets with user profiles. Its analysis basically includes 'a priori (tweets) analytics' containing numbers of tweets per day, user mentions, links, tweets retweeted, replies and hashtags. It also shows tweet history plus followers, most retweeted tweets, most replied to and most mentioned. Its analytics did not really matter to me. Tweets were submitted to Wordle (www.wordle.net) to quickly produce a visualisation of frequencies of terms (see Figure 1). The mediated sociality that Twitter exhibits is about the creation and maintenance of connectivity, of presence which is not always about paying attention but about breaking the silence. Connectivity is not equal to reciprocity and it does not necessarily requires content. The main purpose is keeping in touch (Miller, 2008). To keep in touch is not about providing meaningful information. The value of Twitter is not in its content, it is its perpetual attendance in everyday life to provide service when needed. It is within this notion of connectivity with or without words that I venture into phatic practices within Twitter.

Figure 1: Tweet Cloud Figure 1 is a tweet cloud generated from the 3000 tweets containing '#mooc'. It is easily discernible by the size of @iversity that it had high frequency in the tweets and that a sizeable subset of tweets came from the iversity website. In particular, iversity.org/c/6 and iversity.org/c/2 refer to two of its courses, Future of Digital Storytelling and Design 101, respectively. There were 958 mentions of '@iversity' (http://iversity.org). 'I just found the online course..' was the beginning line for 803 tweets, which were tweets generated by clicking the Twitter link within iversity course pages (see Figure 2). @iversity is the European Berlin‐ based MOOC provider. It went online on 14 October 2013 and launched 6 courses.

4.1 A tweet is a click This tweet cloud in Figure 1 could be read and interpreted in such a manner that would suggest that the larger text were the most talked about mooc‐related topics and circulated widely through retweets perhaps. However, this is actually not true at all. A click on the Twitter button in Figure 2 posted a tweet by the user without writing the text herself or himself and perhaps without knowing that it contains #mooc. Here a phatic practice is a click. Breaking silence in Twitter does not need words to establish links. A click would do. Surely, this mechanism is not about reciprocity. To break the silence, users did not use words. They just clicked the ‘tweet button’ on the iversity.org website. In Twitter, I could follow you and you might not even know it and there is no pressure that you should reciprocate and be my follower. This is not about paying attention to a dialectic of the follower and followed. A click performs and maintains connectivity – and Twitter as the medium in attendance in our everyday, available when needed to be of service.

47

Judith Enriquez‐Gibson

Figure 2: Twitter Link on iversity mooc page

4.2 # as presence or connectivity (out‐reach) There were a total of 25 tweets that contained a series of question marks in them. First I thought, these Twitter users were spammers. Then, I realised that Twitonomy was not able to read Japanese characters in the case of the user in Figure 3. It only read the pound (#) sign and #mooc in the tweet depicted in Figure 3. With the third party software’s failure to translate the Japanese characters in the tweet, the Twitter user’s attempt to have an out‐reach to the English‐speaking world is exposed here. In this case at least, #mooc inscribes a form of intentionality in the tweet because the user has explicitly classified or labelled his or her tweet by a specific topic using a hashtag. It increases the user’s findability and in a search for the topic mooc, this user instantly becomes a member by indirect or ambient affiliation with others who do not necessarily speak Japanese or whose text was lost between the search results and the download to a .csv file using Twitonomy. Here is another manifestation of the phatic communion on Twitter. It was not used as a channel for understanding between the followed and the follower. It is a mere expression of fundamental human sociability among multiple others and perhaps a desire to a need to stretch one’s connectedness – an out‐ reach to the English speaking others or even a break‐out of the Japanese language.

Figure 3: A Japanese tweet with #mooc

48

Judith Enriquez‐Gibson

5. Human code vs. computer code Beer (2012) has suggested that there is value in enacting social science research through by‐product data and social media aggregators. And at any point that we feel they become too prescriptive we can always opt to employ alternative research approaches and perhaps engage with social media ethnography (Postills and Pink, 2012). Beer (2012) speaks of a critical social science that is willing to expand its reach and domain into the context of massive social media data, which he argues to be very much part of everyday life. He suggested that the power of algorithms would allow us to find patterns in unfathomable data sets. For me, visualisation provides hints and traces of what may be worth investigating or following not because of aggregates and lines of computer code, but because of a critical social science relies on the human ability to code phatically. As with thousands of tweets to code and other social media data, we must not only engage with twitter or any other social media sites as a container of tweets generated through lines of computer code. Instead, we must attend to its multiple coded possibilities as a reporting tool or as a space for direct audience engagement or phatic communion.

6. Closing The intent of this small venture in big data was not to follow content or user. Not really. After submitting collected data to Twitonomy, I went about exploring the act of following itself as permitted by web‐based data and sites. The emphasis of following is to engage with Twitter as a system of connectivity. Its tweets are performed without the conscious attention or intention of their speakers, or the attentive engagements of their recipients. In short, inattention is not a deficit or a disconnection, instead it is an ecological achievement of mediated sociality – one can disengage from media text without losing one’s connections (followers). Phatic communion is not about dialogue, it is about the connectivity of being (t)here (Enriquez, 2013) with or without words, with or without language.

References Arnoux, D. (2013) “Hashtag: Where Did This #phenomenon Begin and Why Do We #love it (but only on Twitter)?” [online], Lifehacker.org, http://www.lifehack.org/articles/communication/hashtag‐where‐did‐this‐phenomenon‐begin‐and‐ why‐love‐but‐only‐twitter.html. BBC News (2009) “Twitter tweets are 40% 'babble'” [online], http://news.bbc.co.uk/1/hi/8204842.stm. Beer, D. (2012) Using social media aggregators to do social research. Sociological Research Online, Vol 3 [online], http://www.socresonline.org.uk/17/3/10.html. Coupland, J., Coupland, N., & Robinson, J. D. (1992) " How are you?": Negotiating phatic communion1. Language in Society, Vol 21, pp 207‐230. Enriquez, J.G. (2013) Being (t)here>: mobilising 'mediaspaces' of learning. Learning Media and Technology, Vol 38, No. 3, pp 319-336. Frosh, P. (2011) Phatic morality: Television and proper distance. International Journal of Cultural Studies, Vol 14, No. 4, pp 383‐400. Gannes, L. (2010) “The Short and Illustrious History of Twitter #Hashtags” [online], http://gigaom.com/2010/04/30/the‐ short‐and‐illustrious‐history‐of‐twitter‐hashtags/. Java, A., Song, X., Finin, T., & Tseng, B. (2007) Why we twitter: understanding microblogging usage and communities. In Proceedings of the 9th WebKDD and 1st SNA‐KDD 2007 workshop on Web mining and social network analysis (pp 56‐ 65). ACM. Kwak, H., Lee, C., Park, H., & Moon, S. (2010) What is Twitter, a social network or a news media?. In Proceedings of the 19th International Conference on World Wide Web (pp 591‐600). ACM. Malinowski, B. (1923) The problem of meaning in primitive languages. In: Ogden, CK and Richards, IA (eds) The Meaning of Meaning. London: Routledge and Kegan Paul, pp. 296‐355. Marwick, A. E. (2011) I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New Media & Society, Vol 13, No. 1, pp 114‐133. McFedries, P. (2013) Hashtag, you're it [Technically Speaking]. Spectrum, IEEE, Vol 50, No. 4, pp 24‐24. Postill, J. and Pink, S. (2012) Social media ethnography: The digital researcher in a messy web [online]. Media International Australia, Incorporating Culture & Policy, No. 145, pp 123‐134. http://search.informit.com.au/documentSummary;dn=992474822836114;res=IELLCC. Rogers, R. (2013) “Debanalizing Twitter: The Transformation of an Object of Study” [online], University of Amsterdam, http://www.govcom.org/publications/full_list/rogers_debanalisingTwitter_websci13.pdf. York, E. and Brewster, C. (2013) The representation of Food in Social Media Forum Twitter. Paper presented at the EFITA‐ WCCA‐CIGR Conference “Sustainable Agriculture through ICT Innovation”, Turin, Italy, 24‐27 June 2013.

49

Developing a Learning Management Platform: A Bulgarian Case Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev Sofia University, Bulgaria [email protected] Abstract: The paper shows an integrated approach for building environment supporting administrative, educational and research processes at University, and facilitating knowledge sharing and collaboration within internal and external communities (of students, researchers, industry, alumnus) using advanced Information and Communication Technologies (ICT). Its goal is to present an approach in developing an advanced learning management (LM) environment, which integrates the e‐learning and the knowledge management (KM) concepts. While the paper presents a concrete case, it adds also value to theory with the bridge made between KM and LM, the former normally discussed in enterprises, while the latter ‐ usually linked to educational environment. Another feature of the paper is the integrated approach for using technologies, on the one side, for (explicit) knowledge acquisition, organization and transfer, and on the other, for support of (tacit) knowledge sharing and access to internal and external expertise by university staff. Initially, the paper makes a brief literature review of KM and LM concepts and systems. Second, the paper presents the methodology for developing a knowledge management system (KMS), and the main results of a survey of the state‐of‐the‐art and the working processes at Sofia University (SU). The paper further presents the concept of the KMS, based on the requirements identified at the previous stage, and the feasibility to leverage the existing SU infrastructure. Finally, the results of a pilot implementation are outlined, and the feedback obtained by the main KMS users. Keywords: learning management, knowledge management, knowledge management platform

1. Introduction Universities are a key actor in the triangle of knowledge – involved in generation (research), dissemination (education) and application of knowledge (innovation). Metaxiotis and Psarras (2003) outline three major missions of universities: ‘Teaching ‐ to prepare students to become successful lifelong learners, Research ‐ to expand the frontiers of human knowledge and to promote creativity; Service ‐ to serve communities, and to participate in outreach activities that serve the local, national, and international communities.’ Having the main features of a learning organization, universities need to take measures to ensure appropriate support of all knowledge processes – creation/gathering, storage, usage and transfer of knowledge (Gourova et al, 2012). As Rowley (2000) points out, universities should consciously manage the processes related to the creation of new knowledge and its transfer to interested stakeholders in order to fulfil much better their mission in the triangle of knowledge. The Department of Software Engineering (DSE) at the Faculty of Mathematics and Informatics (FMI) of Sofia University (SU) acknowledges the present‐day challenges for universities as learning organizations and aims to facilitate knowledge and learning processes. A team of researchers launched deep changes in DSE web portal after analyzed the available web resources and electronic systems at SU and FMI, as well as the theory and practice in the field of Knowledge Management (KM), Learning Management (LM) and mobile learning. On this base a concept for a Knowledge Management System (KMS) was developed and partly implemented. The main goal of the paper is to present the concept for a KMS, and the results of its implementation. The paper considers, first, the research discussions on KMS and Learning Management Systems (LMS). Second, it briefly presents the state‐of‐the‐art at DSE and the methodology followed by the team for developing a KMS. Finally, the results achieved and the ways ahead are discussed.

2. Linking knowledge and learning management In the knowledge‐based economy knowledge has become a key resource for competitiveness and growth of enterprises world‐wide. Therefore, several researchers have studied how to facilitate knowledge processes in the organisation and how to efficiently use the available knowledge resources (Gourova et al, 2012). As essential factors for KM success are generally acknowledged leadership, organisational culture and technologies (Gourova et al, 2012). Learning is rarely explicitly stressed in research and practice. However, in a learning organisation like university the KMS should be a base for organisational knowledge enhancement, and should foster learning and sharing of knowledge among employees, researchers, students, and other stakeholders. Therefore, many researchers believe that e‐learning is a key technology supporting KM as it

50

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev facilitates the transformation of tacit into explicit knowledge (Liu et al, 2009). Other key technologies for KM are business intelligence, knowledge bases, groupware, content management, knowledge portals, customer relationship management, data mining, searching and workflow management (Luan et al, 2002). The core business processes of any university comprise teaching and learning. Therefore, the KM framework proposed in (Hoq et al, 2012) shows the relationships among several factors important for university functions (Figure 1). It emphasis the need for personal KM, and the management of the main knowledge processes: knowledge generation, acquisition, storage, and dissemination. The respective knowledge infrastructure should represent a platform with advanced knowledge services, integrated on the basis of a shared ontology and personalized for the networked participants (Maier et al, 2004). Thus, some of the main technologies for building organizational memory within a KM initiative include: corporate learning portals, e‐learning suites, integrated curriculum management systems, learning environment, LMS, etc.

Figure 1: KM framework for university (Hoq et al, 2012) Mohayidin et al (2003) consider that the KMS should comprise three layers – infrastructure, info‐structure and info‐culture. The authors stress that any of these layers could influence the success or failure of a KM project, as well as may determine the overall performance of the university in fulfilling its functions as knowledge provider. More specifically, the success of KM at universities requires some basic pillars such as leadership, organization, technology and learning (Stankosky 2005). The design of a KMS requires considering appropriate KM tools and organizational design (e.g. knowledge tasks and processes, roles and responsibilities), deciding upon knowledge topics and structures (e.g. type of knowledge, taxonomies, ontologies and meta‐data), as well as information and communication technologies (ICT) in support of KM (processes, structure and interaction of knowledge infrastructures). A special attention should be paid to people and their equipment with appropriate skills and competencies, as well as ensuring support to their communication and collaboration in various networks and communities, both internal and external. Knowledge management and learning management have followed different developments, but recent research outlines many similarities between them. As factors driving their convergence could be outlined (Zhang et al, 2003):

focus on knowledge acquisition and sharing by using equivalent technologies;

need of organizing and maintaining knowledge for higher efficiency;

support of users to access and generate new knowledge.

When designing a KMS in university environment, it is essential to take into account the specific features of LMS. According to (Avgeriou et al, 2003), LMS are specialised educational systems using advanced internet and web technologies and should ensure four clusters of tasks:

administrative tasks – control of access, registration and authentication, personalisation, help desk;

course management – course creation/ launching/ customisation, announcing, storing and maintenance;

students management – management of students’ groups, monitoring, tasks management, evaluation;

51


resources and learning management – access to various resources, learning materials, e‐books, glossary of terminology, search and retrieval, collaborative learning, information dissemination.

The main LMS users comprise (Avgeriou et al, 2003): learners – using the LMS within the educational process; instructors – using the LMS for teaching, mentoring, support of learners, assessment and control, and administrators – ensuring proper LMS functioning and access according to users’ roles. In case of developing a complete KM solution for university, in addition to LMS users should be considered also researchers, knowledge workers, administration, and external stakeholders. The latter should be added in case that the KMS aims also at knowledge transfer and serves as a platform for collaboration with external users interested in university knowledge assets.

3. Developing a concept for KMS 3.1 Methodology The methodology for developing a KMS included some steps typical for software systems design:

Problem identification – The team analyzed the available technologies, and the business processes in place in order to identify the existing gaps. On this base was defined, first, the main problem, then, assessed the opportunities for solving it with the limited financial and human resources, and last, the KMS requirements were defined.

Concept development – The team brainstorming provided some insights on the possible solutions, and the realistic options for KMS development. In addition, a survey was made on technology trends, research and practical experience for learning management and KMS in university environment. The combination of team ideas, and the knowledge gathered from the scientific community facilitated the KMS concept development.

Validation of the concept – A special session at a conference in June 2013 was devoted to the KMS concept presentation and a discussion of its practical implementation. A special emphasis was made on assessing what is realistic and achievable and how to revamp the present web site and to facilitate the access by mobile devices (e.g. tablets and smart phones).

Development of KMS prototype – The main objective was to prepare a first KMS prototype for the winter semester starting in October. Therefore, the team designed some basic functionalities and provided single sign‐in access to all learning resources.

Testing the prototype – It was decided after officially launching the new web platform to gather feedback from its users, as well as to collect comments from staff members on the available functionalities. Therefore, a special questionnaire was designed targeted at students and teachers as main respondents. In addition, a functionalities check list was prepared for project team members in order to validate the KMS in place.

Finalization of the KMS and entering into exploitation – The testing and validation phase will be the base for the next phase of the platform design and the implementation of all KMS functionalities according to the concept. A special training for teaching staff is foreseen in order to facilitate the exploitation, and the knowledge gathering and sharing processes.

3.2 Analysis of the state‐of‐the‐art DSE provides on its web site public access to its educational resources at Bachelor and Master levels, and information for its activities and educational requirements. A system for Thesis management is accessible from a closed area of the site, as well as internal resources of the staff. In addition, DSE staff uses the common resources of SU and FMI, including access to international research databases provided by SU library, and different web‐based systems:

University system for education management (“SUSI”) – contains information about SU courses and staff – their learning status, notes on exams, courses offered and chosen by students;

Several Moodle platforms maintained centrally for SU or only by its departments;

52


centralized system for project management;

research publications management (named “The Authors”) – provides information on publications of SU researchers from FMI.

As shown on Figure 2, different databases store the information for students and their education. In fact, SU has different databases with administrative and educational information, while the scientific results are stored in separate databases. Similar to other FMI departments, DSE does not maintain a centralised database with educational materials.

Figure 2: SU working processes linked to students Some of the working processes at SU linked to its lecturers are shown on Figure 3. SU lecturers are employed after a job competition and their main duties are teaching. Their career development, however, is based not only on their educational activities, but also on research achievements, including scientific publications, projects and mobility exchange. The regular staff assessment requires collecting structured information for their activities, which is stored in special SU databases for research papers and projects. In addition, FMI has a separate database with information on research achievements of its staff (the Authors). Access to scientific databases (except to IEEE и ACM) is ensured by the Ministry of Education and Science for all Bulgarian universities.

3.3 KMS goals and features When developing the KMS concept it was taken into account the need to clarify the scope of the KM initiative, and its main objectives. As the practice shows, it is always better to start with a pilot project, and then extend it to comprise the whole organization (Gourova et al, 2013b). Thus, the KMS is considered as a pilot project with the objective to be multiplied by other units as well. The main goal of the KMS is to ensure efficient access to SU, FMI and DSE knowledge resources and to facilitate realizing SU mission as educational and research organization. Additional objectives are:

to ensure appropriate knowledge needed for all university working processes;

to ensure access to individual knowledge resources;

to facilitate collaboration and knowledge transfer among internal and external stakeholders.

53


Figure 3: SU working processes linked to lecturers As main users of the KMS are considered not only DSE students and lecturers, but also SU administrative staff, as well as external stakeholders and DSE alumnus and partners. The KMS should be integrated in SU technology environment and should facilitate:

Educational activities – programmes and courses, lecturers, schedules, students’ management, students’ evaluation, thesis preparation, mobility exchanges;

Research activities – existing scientific infrastructure (labs and equipment, scientific databases, information flows), project management, intellectual products management, knowledge and technology transfer, collaboration with industry and other stakeholders, scientific supervision and mentoring;

Human resources management – recruitment, monitoring and control, assessment, career development, awarding, mobility;

Administrative management – planning and reporting, accreditation, resources management.

The team took into account that present day students belong to a new generation which grows with digital devices and gain ICT users’ skills at very early age, thus they use all forms of distance education, despite of the devices. In addition, mobile learning offers new opportunities for distance education due to the rapid development of wireless communications and the availability of various portable devices, which mass usage is a reason for the growing popularity of m‐learning world‐wide (Gourova et al, 2013a). However, the specificity of mobile devices and the difficulties for using the materials designed for e‐learning on such devices were reasons for developing new methods and standards for design of Mobile Learning Systems (MLS). The research (Gourova et al, 2013a) shows that the MLS design should take into account the device performance capability

54

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev (memory, processing power), to ensure user‐friendliness (input/output, navigation) and interoperability (different e‐learning content and applications). Very useful are the guidelines for m‐learning courses design suggested by Ebner et al (2009): focusing on layout design (flexible, simple, narrow), content quantity (less text), usage of graphics (small), the site structure (flat), and usage of animations (to be avoided). An essential characteristic of MLS is the use of device‐independent applications, small learning objects, and the modular structure of courses, flexibility and reuse (Gourova et al, 2013a). Therefore, in the KMS concept was integrated a concept for mobile version of DSE web site, which is based on well‐known requirements for software systems design with mobile access (Ardito et al, 2006; Magal‐Royo et al, 2007). The mobile version preserves the former content and the resources organisation, however, following some basic principles:

Site structure improvement with the aim of more effective, easy and fast access;

Updating and enhancing content with the aim of its higher informativeness;

Optimization of the visual users interface with the objective of easy access via different devices;

Effective web site internationalization in order easy to add new languages.

Some of the requirements of the KMS, critical for its success, are targeted at scalability, knowledge reuse, efficient searching and retrieval (Zhang et al, 2003). In addition, it is essential to provide single entry point to all knowledge and administrative resources of the university, and an intuitive interface. In fact, the present situation does not allow building a common DSE knowledge base due to the usage of distributed databases of SU and FMI with different access options. The only possible solution is to facilitate the access of DSE stakeholders to these knowledge assets, and start building additional own systems for gathering the knowledge resources of DSE. Developing Yellow pages (who is who database) could facilitate searching of internal and external expertise (interested stakeholders and DSE partners). It is essential as well as to take advantage of Web 2.0 technologies and to facilitate the collaboration using social networks (e.g. for DSE alumnus), blogs or wiki (Gourova et al, 2013c).

4. KMS implementation 4.1 Main KMS functionalities The DSE KMS concept follows the requirements for platform independent storage of digital content, accessible by various devices (incl. mobile phones), and available in different forms – text, audio and multimedia. Its functionalities guarantee easy‐of‐use intuitive interface. The overall concept for DSE KMS and its links to SU and FMI systems includes 4 layers:

The access and authentication layer has mainly information security functions, and provides access to all available resources to DSE students, lecturers and administrative staff. The access to the available SU and FMI systems is ensured by a link provided on the portal. External users have access after registration on the portal to some of DSE resources, incl. Thesis database. As LMS is used a Moodle platform, whereas lecturers are responsible for courses administration.

The collaborative intelligence and filtering layer facilitates the retrieval of information and ensures personalised services to the users providing some KM services. This faces developers with challenges for knowledge organisation and indexing, as well as databases integration. The layer also facilitates the generation of different reports on bases of the available knowledge resources.

The applications layer ensures access to different applications – Moodle platform, SU education management system, DSE research, yellow pages of the personnel, etc.

The data repositories layer includes DSE databases – education, research and information resources, combining both structured and non‐structured data.

On bases of the design principles adopted in the concept, the web platform was designed with a three‐layer interface including: navigation menu, tabular area and links area (Figure 4). The tabular area provides quick access to all research topics of interest for DSE staff. Thus, the web site visitors can navigate easily to different research activities and projects. The links area includes a set of hyperlinks to the common online resources

55

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev used by lecturers and students (e.g. e‐learning system, students’ administration system, research database, SU staff research and publications data). They are divided in three main groups: institutions, systems and libraries. As the web site concept envisages effective access by both, mobile devices (phones and tablets) and desktop computers, the DSE team designed one responsive web site that scales across a wide range of screen sizes, built on top of jQuery library, HTML5 (HyperText Markup Language) and CSS3 (Cascading Style Sheets). It was taken also into account that web sites for large screens typically use left or top menus, which are not visualized properly on smaller screens. Therefore, using the jQuery library, a dynamic web site menu was implemented which provides quick access to the content. When the browser window is narrow, the navigation in the top pane of the web site converts from a regular three level menu into a dropdown menu. Thus, the users can access easily the menu elements without a need to scroll the web site. Similar transformations are provided for the other web site elements located in the tabular area and the links area and, as shown on Figure 5, all web site elements like images, titles, texts, etc. are dynamically rearranged when the width of the browser window is reduced (Petrova‐Antonova et al, 2013).

Figure 4: Web site map (Petrova‐Antonova‐et al, 2013)

Figure 5: Mobile web site menu (Petrova‐ Antonova‐et al, 2013)

The team decided to test m‐learning within one Master course. The questionnaire sent to the students aimed to obtain feedback on the content, format, and m‐learning as a new distance learning opportunity. It is interesting to note that the respondents showed a clear preference to Android (47%) as an operating system of the mobile device, followed by Windows (33%), iOS (5%) and Linux (15%). Students like the opportunity for using mobile devices in distant education, however, they are still not ready to carry out all self‐learning activities and the interactive communications just in MLS environment. According to students, the key factors for m‐learning success are the learners’ attitude, the learning process organisation, and the technical opportunities in place. The students feedback shows that m‐learning provides a fully new educational design opportunity and environment for: educational materials storage (23%), distance learning (52%), interactivity

56

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev on study topics (28%), active students communications (60%), interactive seminar classes (43%), joint projects work (72%), and online access to educational materials (93%) (Gourova et al, 2014).

4.2 University knowledge resources Greengard (1998) considers that ‘an organisation aiming to develop a knowledge base first needs to identify the sources of knowledge available, and then to capture and manage these resources properly’. It should be taken into account that the organizational knowledge comprises individual and group knowledge assets, integrated in various knowledge objects. The first step by developing an organizational knowledge base is to identify the existing knowledge, both explicit and tacit, and make it easily accessible. In fact, researchers at universities generate various explicit knowledge resources (e.g. research papers, books, training materials, project results, etc.) which just need to be collected, organized and made available in the knowledge base. In addition, it should contain also all organizational knowledge assets (e.g. internal regulations, strategic documents, etc.). Subsequently, specific tools should be considered for knowledge transfer and use in the working processes of the organization (Gourova et al, 2012). The migration to the new KMS required initially identification of DSE knowledge resources, their classification, organization and integration in the new repositories. The next step is ensuring suitable access to these resources and active involvement of all users in their actualization and maintenance. The analysis of DSE working processes (Figure 2 and 3) facilitated the databases and applications design. While education and students management systems are in place, some new systems were designed, facilitating the access to DSE explicit knowledge – general information, publications and learning materials. More difficult is to capture the tacit knowledge available at universities. Normally, as a first step the sources of such knowledge in the community should be identified. Yellow pages or social network sites could help to visualise the existing competencies and individual or group expertise. Alumni networks could also provide access to external university partners and stakeholders, thus, facilitating collaboration and finding external expertise. This was taken into account by the team, and it was decided as an initial step to localize the available knowledge and expertise by developing yellow pages for the personnel, integrating social networks sites (e.g. Facebook and LinkedIn), and launching DSE alumnus network.

5. Conclusion Building a KMS will facilitate the access to the knowledge assets at DSE. The choice of appropriate KM technologies and the good structure and organization of all available knowledge resources, as well as adding tags and meta‐data, is expected to support the users to fast and easy search, find and retrieve the information and knowledge they need. This will be ensured also by the intuitive and user‐friendly interfaces provided, as well as the single entry concept. While the KMS concept was validated and partly implemented, some additional challenges remain. First, it is well‐known that the success of the KMS depends on respecting the real needs of the users, and facilitating their working processes. Thus, there is a need to test users appreciation of the KMS, and obtain feedback on the newly designed functionalities before finalizing it. In addition, the success of the KMS depends on the proper training of its users, and ensuring committed leaders to drive the change. While DSE management is deeply involved in the implementation, there is still a need to overcome the resistance of other lecturers and the administrative staff for using and maintaining the KMS. A special training was already conducted on preparing video lectures. Training for using other KMS functionalities is also envisaged, and awareness raising initiatives.

Acknowledgements The authors gratefully acknowledge the efforts of their colleagues and the support under grant agreement BG051PO001‐3.3.06.‐0052 of the European structural fund in Bulgaria.

References Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V. (2006) An approach to usability evaluation of e‐learning applications, In: Univ Access Inf Soc 4, 270–283. Avgeriou, P., Papasalouros, A., Retalis, S., Skordalakis, M. (2003) Towards a Pattern Language for Learning Management Systems. Educational Technology & Society, 6(2), pp. 11‐24.

57

Elissaveta Gourova, Mila Dragomirova and Pavlin Dulev Ebner, M., Stickel, C., Scerbakov, N., Holzinger, A. (2009) A Study on the Compatibility of Ubiquitous Learning (u‐Learning) Systems at University Level, In: Springer Lecture Notes in Computer Science, 34‐43 Gourova, E., Antonova, A., Nikolov, R. (eds.) (2012) Knowledge Management, Bulvest 2000, Sofia (in Bulgarian). Gourova, E., Asenova, A., Dulev, P. (2013a) Integrated platform for mobile learning, In: D. Sapmson, P. Isaias, J. M.Spector, D.Ifenthaler (eds.), Ubiquitous and Mobile Learning in the Digital Age, Springer Science+Business Media New York 2013, 67‐92. Gourova, E., Todorova, Y., Dragomirova, M. (2013b) Knowledge Management Strategy for SME, 18th EuroPLoP 2013, Irsee, Germany, (to be published). Gourova, E., Zografova, L. (2013c) Knowledge Management System at DSE, Proc. Challenges in Higher Education and st Research in the 21 Century, Sozopol, Bulgaria, (in press). Gourova, E., Petrova‐Antonova, D., Dulev, P., Bontchev, B. (2014) M‐learning pilot at Sofia University, IADIS M‐Learning 2014, 28 February ‐ 2 March, 2014, Madrid, Spain, pp. 323‐ 326. Greengard, S., 1998. Storing, shaping and sharing collective wisdom. Workforce, 77(10), pp. 82‐8. Hoq, K. M. G., Akter, R. (2012) Knowledge Management in Universities: Role of Knowledge Workers, Bangladesh Journal of Library and Information Science, Vol. 2, No.1, pp. 92‐102. Liu, Y., Wang, H. (2009) A Comparative Study on E‐learning Technologies and Products: from the East to the West, Systems Research and Behavioral Science, 26, pp. 191‐209. Luan J, Serban A.M. (2002) Technologies, products, and models supporting knowledge management. New Directions for Institutional Research, 113, pp. 85‐104. Magal‐Royo, Т., Peris‐Fajarnes, G., Montañana, I., Defez Garcia, B. (2007) Evaluation Methods on Usability of m‐Learning Environments, In: IEEE Multidisciplinary Eng, Educ. Magazine, Vol. 2, No. 2, June 2007, 34‐37. Maier, R. (2004) Designing Knowledge Infrastructures, Organizations and Society in Information Systems, OASIS workshop, accessed 03.2013: http://www1.dsi.uminho.pt Metaxiotis, K., Psarras, J. (2003) Applying Knowledge Management in Higher Education: The Creation of a Learning Organisation, Journal of Information and Knowledge Management, 2( 4), pp 1‐7. Mohayidin, M. G., Azirawani, N., Kamaruddin, M. N., Margono, M. I. (2007) The Application of Knowledge Management in Enhancing the Performance of Malaysian Universities, The Electronic Journal of Knowledge Management, Vol. 5 (3), pp 301 – 312. Petrova‐Antonova, D., Gourova, E. (2013) Web site development and m‐Learning: the case of Software Engineering Department in Sofia University, In: CHER 2013, Sozopol, Bulgaria, 46‐48. Rowley, J. (2000) Is Higher Education ready for Knowledge Management? The International Journal of Educational Management, 14(7), pp. 325‐333. Stankosky, M. (2005) Advances in Knowledge Management: University Research Toward an Academic Discipline in Creating the Discipline of Knowledge Management: The Latest in University Research, Amsterdam: Elsevier Butterworth‐ Heinemann. Zhang D., Nunamaker, J.F. (2003) Power e‐learning in the new millennium: an overview of e‐learning and enabling technology. Information System Frontiers, 5, pp. 207–218.

58

Extending the Design of Games for e‐Learning to Include Affective Computing Elements Olaf Hallan Graven1, Lachlan Mackinnon1, 2 and Liz Bacon2 1 Buskerud and Vestfall University Colleges, Norway 2 University of Greenwich, Old Royal Naval College, London, UK [email protected] [email protected] [email protected] [email protected] Abstract: The use of human emotion in learning situations has been a feature of learning and training since earliest times, since it has been seen to improve the speed of uptake and the personalisation of the learning process. In recent years, research in affective computing has focused on providing realistic and appropriate representations of human emotions both in computer software and the variety of digital environments with which we engage, and, as a result, using these to create more realistic emotional interactions between humans and computers. A strong driver in this activity has been the games industry, seeking to make games more realistic and engaging, both for entertainment and, more recently, through the growth of serious games offering visualisation and simulation of real‐world scenarios using games technologies. The authors have been involved for some time in research on the use of games for e‐learning, and this paper considers the use of affective computing techniques within such games. The benefits are seen as greater engagement and immersion of students, faster achievement of suspension of disbelief, willingness to reuse and revisit learning materials, and the potential to introduce realistic emotions and stresses into the learning situation. Two similar but contrasting approaches, from the Pandora and Maritime City projects, are also discussed. The Maritime City project has developed a training scenario for social workers, based on a well‐known child abuse case, using an avatar‐based, first party game environment. The scenario is used as a training aid during interactive sessions, with the trainees critiquing the approach taken, analysing the issues presented, and role‐playing alternative approaches. The Pandora project has developed an augmented reality training environment for Gold Commander crisis management, modelling and presenting real‐world crisis situations as a timeline‐based event network. The trainees are presented with a range of media inputs relating to the events as they unfold in the scenario, and are expected to collaboratively make decisions to deploy manpower and resources to address these events, in a timely fashion. Both projects seek to engender the stresses and emotional affects of their scenarios within the trainees, using accurate representation of the events, time constraints, and decision‐making pressures, alongside the immersive effects of the technologies used. The conclusion from both projects, and from our other research, is that affective computing techniques can significantly enhance games for e‐learning, but only if skilfully applied and with strong production values, as poor application of affective techniques can render a game unusable and laughable. Keywords: e‐learning; games; affective computing Introduction

1. Introduction The existing research on computer games for e‐learning has predominantly focused on gaining the benefits of player immersion in games for learning environments. The rationale has been that developing games for learning, or introducing learning materials into existing games, will transfer the impact of the gaming environment to the learning situation. Considerable research, carried out by the authors (Graven et al, 2005, 2008, 2009) has demonstrated that, while the transfer is achievable, considerable skill and effort is required to effectively marry game‐play with learning in an appropriate and successful balance. This paper describes some of the background research carried out by the authors, and its outcomes, before going on to consider the use of affective computing techniques in games for e‐learning. General principles on socialisation and affective computing are discussed, and then contextualised in relation to games, and games for learning in particular. Two similar, but contrasting, projects utilising affective computing techniques for serious games are then described and discussed. Finally, the paper concludes on the value of affective computing techniques for e‐ learning games, highlighting some of the benefits and identifying some of the pitfalls and dangers.

1.1 Designing for active participation in learning activities The theoretical argument for constructivism is that deep knowledge and long‐lasting knowledge is more likely to arise from constructivist learning environments. The perceived benefits of a constructivist learning environment include absorption and synthesis of facts, linking the knowledge of facts with understanding of other knowledge domains, the enhancement of collaborative/cooperative skills and time. One of the key

59

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon points of constructivism is to get the students involved in the process, and give the tutor the role of facilitating and supporting learning. Experience has shown that most students will, after some initial misgivings, elect to follow the active learning route and actively be involved in the learning process. The rationale for this is that it is simply more enjoyable. Students learn more, and enjoy themselves more when they are actively involved, rather than just passive listeners. The presentation of material in books, papers, multimedia presentation, traditional lectures or web can all encourage the student to take an active role, use the lab for assignments or even just imagine the activities. However there are few forms of learning that, like games, entice, encourage and even force the student into activity. Computer games are built to activate and keep a player active with problems and tasks, through the whole period of play. An important aim here is that, in addition to keeping the player active, they also ensure the tasks consistently offer a challenging degree of difficulty. Csikszentmihalyi introduced the concept of flow, through a study of people involved in activities such as rock climbing, chess and dance (Csikszentmihalyi , 1975). He describes flow as a state of complete absorption or engagement in an activity and refers to the optimal experience. According to flow theory, flow can occur when an activity challenges an individual enough to encourage playful, exploratory behaviours, without the activity being beyond the individual’s reach. Flow has been shown to have a positive impact on learning, see Galarneau and Kiili (Galarneau, 2004; Kiili, 2005). Many people are advocating taking the hard work and discipline out of learning. However, we would argue that this is not the answer. Rather, what we should be doing is finding tasks that will harness the passion of the student to the hard work needed to master difficult material. Therefore, one main aim for us, as designers of learning material, is to design content in such a way as to allow different students to obtain the state of flow, irrespective of their different knowledge and abilities.

1.2 Social and emotional connections Social relations between students during learning are an integral part of the authors’ philosophy for learning. Manninen has shown that the communicative aspect of current multiplayer games is enabled by a relatively limited set of interaction forms (Manninen, 2003). The communication between players in the prototypes described in this paper is mainly via IRC. But the avatars do also have a limited set of emotions they can express within the environment. Studies by Kolo and Baur have shown that many players not only connect to a online game in order to play but also to stay in contact with the fellow players, many players also connect to fellow players via messaging systems during game play (Kolo and Baur, 2004). They engage via their characters in various social interactions from trading or fighting to entertaining other characters. Many players regularly meet the same characters online and address a relatively fixed group of playing partners. Kolo and Baur have also shown that knowing and meeting people in an online environment or game triggers frequent playing and not the other way round. In terms of affective computing, the work of Picard is seminal, and she focused primarily on the ability of computers to accurately represent emotional states, and engage appropriately in human computer interactions within an emotional context (Picard, 2000). Considerable work in the games community has focused on the first of these goals, namely the accurate representation of human emotions on computers, in particular on avatars representing human characters in games. Although this work has also focused on dialogues and interactions between human players and these emotionally affective avatars, limited consideration has been given to the use of affective techniques to impact player learning. In the serious games community, however, projects such as Pandora (Bacon et al, 2012) and Maritime City (Flynn et al, 2010), which are discussed later in this paper, have used affective techniques to enhance training scenarios, by directly impacting the emotions of the trainees. The clear benefits of this approach are in increasing player engagement and immersion through emotional interaction more rapidly reaching suspension of disbelief, and impacting the effectiveness of learning achieved through greater realism and stress in the learning situation (Manninen, 2003). In terms of the concepts of flow, it is only when fully immersed in the game, with disbelief suspended and concentration entirely engaged with the gameplay, that flow can be achieved.

60


2. Computer games in learning 2.1 Learning from computer games designers One of the current challenges facing educators today is the perceived lack of motivation and engagement in learning situations by out students, especially in topics that are, by the students, perceived as difficult of challenging. This forces us as educators to make choices on how to address this, two frequently selected major choices are either to try to force the students through by setting huge amount of coursework and supervision, or if the resources are not available lower the requirements for a pass mark. However, in the gaming industry neither of these options are viable, people cannot be forced to buy and play computer games, and, in general, players do not want the short and easy option. For educators, this raises an interesting question: “How do game designers manage to get new players to learn their games which are often long, complex and difficult, and even pay for the privilege?” An answer provided by Gee is: “The answer, I believe, is this: the designers of many good games have hit on profoundly good methods of getting people to learn and to enjoy learning.” He goes on to claim that “Under the right conditions, learning, like sex, is biologically motivating and pleasurable for humans” (Gee, 2005). The notion of getting students hooked on learning, rather than seeing it as something they have to force themselves through, is every educators dream. The question that needs answering is:” How do we do it?” From the many studies of learning that have been done, from Rousseau to Piaget, we know the benefits of revisiting the same learning material on a number of occasions, reinforcement, and drill and practice models. The effect of repeated exposure to the same information again and again greatly increases the likelihood that the information is retained and aids in understanding and learning. So it can be argued that in designing games for learning a major concern is to ensure that the game is used and reused by the participants. This is an aim we have in common with the commercial game producers. Deep learning requires the commitment of time and effort. The gaming industry have invested heavily into getting players to commit the necessary time and effort to master their games, the challenge for educationalists is to tap into the same feelings, getting students to devote the same time and energy to learning as they do to playing games in the evenings.

3. Embedding learning in games The authors have previously complete a set of experiments to determine that a games‐based environment is a viable way to present learning material, and what level of embedding of that material is necessary within the game. Particularly we investigated what effect different levels of embedding of the learning material within the game context had on the players perceptions of game‐play and learning. The design for the virtual learning environment was based on the utilisation of a multi‐level, multi‐player games based model and its inherent support for constructivist learning in a higher education environment. The decision to adopt a game‐based approach to the development of e‐learning follows on from a survey of current state‐of‐the art support for lifelong learning that was carried out by the authors (Graven and MacKInnon, 2005), a key element of that investigation being the different techniques to improve learning and retention through engagement of the learner.

3.1 Embedding vs layering vs bespoke development The authors have considered placing learning material within games, so that the games environment becomes the vehicle for delivering the learning material. We have investigated several ways to create games for learning: layering on top (Graven and MacKinnon, 2008); embedding (Graven and MacKinnon, 2009a); creating the game from scratch with just the learning material and no other game content (Graven et al, 2009b); and developing a game where the learning material is abstracted from the original context and the game is based on the abstracted elements (Graven et al, 2009c). The findings from this experiment indicate that the students liked the game, the game context, and found it a useful way to present learning material. The students claimed they found the game engaging, and we did observe a small amount of reuse, not as much as we would like, but some students did return to the game to play on. This is encouraging but demonstrates that there is a need to extend the game further to determine

61

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon whether or not embedding really can generate the level of reuse we require to demonstrate the effectiveness of the approach. We need to determine whether the game was not engaging enough, or if there was not enough content to keep the students returning to the game. In our previous experiment we looked at layering and found that with layering there was practically no reuse. We have looked into designing games specifically for purpose and found some reuse in those, but the problem with that approach is both cost and the requirement for expert knowledge, and we believe these eliminate this approach from consideration. The evidence we have from this experiment is that embedding learning content into a predesigned game does achieve both the engagement we desire as well as some elements of reuse.

3.2 Emotional connection to a computer game We know, from the enormous body of research that has investigated it, from Rousseau through Piaget to the current day, that learning through play is one of the strongest forces in human development, socialisation and the development of personality. Play, by its very nature, engages us at an emotional level in the current game or scenario, real or imagined, with which we are engaged. However, the formalisation of educational processes and methods, as we progress through education, tends to remove any aspect of play from learning. It can be argued that this is to the detriment of the learning process, and in recent times the development of games‐ based learning is an attempt to rekindle the relationship between play and learning. There are those who argue that this trivialises education, and that students do not give full weight to the ideas and concepts learned through games. However, even the simplest of children's games have strong and important conceptual and contextual learning associated with them, they are often quite serious in context, and although fun are not regarded as trivial by the players. A number of studies carried out in social sciences have shown that a significant proportion of the basic knowledge by which we live our lives is derived from playground activities as children (Strom and Strom, 2011). Our research on the use of computer games for education has highlighted the balance that needs to be struck between gameplay and educational content, to ensure that the game remains fun, enjoyable and immersive, while also meeting the learning needs of the students. Gameplay is one of the intangible qualities of a game, and despite many attempts to classify and quantify it (Oxland, 2004), clarity remains elusive. However, it is clear from the work that has been done that it is strongly tied to causality in the actions, decisions and progress of the game, and to the engagement of the player. Such engagement tends to have an emotional component, so different players prefer different types of game, and there are distinct differences on the basis of age, gender and culture in the types of games played. Since this is true for games that have been developed purely for entertainment, there is no reason to believe that such differences do not apply in e‐learning games. However, another dynamic at play in e‐learning games is the serious nature of the game, and how that impacts the view of the players. Our experience to date has demonstrated that students are willing to use games for e‐learning at the behest of their lecturers, and will play the game and give positive feedback on the learning experience, but return and reuse of the game, the key goal that we aim to achieve, is fundamentally founded in gameplay. Therefore, considering the emotional engagement of the players in game design and building in affective capabilities/components, if well done and appropriately contextualised within the game narrative, can positively impact gameplay and thus improve the prospects of the game being successfully received by players.

4. The use of emotion for engagement We can demonstrate the use of affective computing techniques in terms of two similar, but contrasting, serious games projects: Maritime City (Flynn et al, 2010) is an immersive, game‐based, educational tool, currently designed to train Social Workers, although the approach has wider applicability in training other Healthcare professionals. In the Social Work field in the UK, there is a set of standards for practice that students must meet before they can become a Social Worker. These are contained within the National Occupational Standards for Social Work, and fall into six “Key Role” areas (Department of Health, 2002). Of the six, four directly relate to dealing with people face to face. However, providing trainee Social Workers with “real world” training scenarios is challenging. All students on this type of course should have a placement where they would hope to gain relevant experience, but this is not always feasible and not all placements can expose a student to the full range of experiences they need. Current solutions to filling in the experiential and training gaps revolve around

62

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon setting up role‐play activities. However, it is well known that the quality of these can vary depending on the enthusiasm, commitment, and indeed the acting abilities of the students. A more consistent approach for all students would be desirable. Due to the engaging nature of games, it was decided to develop a training environment using games technology that aims to help solve some of these issues and can be used alongside traditional teaching such as role‐play, or potentially as a replacement for it. The environment is a virtual city initially developed using the Source game engine created by Valve Corporation (Valve Corp., 2007). It will eventually hold a number of different training scenarios that are interlinked and run alongside each other, however it currently has just one scenario focused around the Baby Peter Connelly (Baby P) case. This is an infamous case in the UK, which has brought to light a number of issues in Social Work practice, and the communication challenges when dealing with a child at risk. In this particular case, the child was deemed at risk but was not removed from the situation for a number of reasons (CommunityCare, 2010), resulting in continued abuse leading to the eventual death of the child. The game engine used was chosen as it provided excellent facial animation features, which gave developers the tools to create emotionally engaging characters that can convey emotional states through their facial and body language. This is particularly important as Social Workers must develop skills to interpret both verbal and non‐verbal forms of communication. The facial characteristics used to display emotion in the avatars, coupled with the audio input provided by professional actors, are the key mechanisms by which emotions are conveyed to the trainee. This was based on a number of studies that have been undertaken in this area, such as that by (Mosera et al, 2007) which demonstrated that avatars can elicit a response similar to that provided by a real human face and concluded that they could be used in training programmes however, there are still noticeable differences in various types of brain responses when comparing real and simulated faces. In Maritime City, trainees interact dynamically within the scenario basing their actions and responses on the information they gain from the virtual clients, both in terms of factual information and the emotional representation. It is fundamental to social work training that trainees should be able to develop a non‐ judgemental, objective view of client emotions, in order to assess their coherence with the situation being presented and the factual information available. The Maritime City simulation and visualisation provides an opportunity for such training, with the added advantage that scenarios can be halted and analysed to highlight key points of learning or key failings, which is not possible to the same extent in interactive role‐play. However, the key advantage in using computer based visualisation and simulation with the concomitant representation of emotional state, is in the consistency of representation of the scenario to all trainees, across numerous different training sessions, and in the ability to consult and re‐run on the basis of mutually devised strategies. Pandora is an EU FP7 Project (Bacon et al, 2012) designed to use emotional affect in order to impact decision‐ making and enhance learning in crisis management training. The application under development is designed specifically to enhance and expand training exercises for Gold Commanders in crisis management. Individuals who carry executive responsibility for the services and facilities identified as strategically critical within these situations e.g. Police, Fire, Ambulance Service, Local Authorities, Health Service, are called Gold Commanders (in the UK), and are expected to work together to provide strategic solutions to a crisis. These crisis situations could be anything from a terrorism event, to problems caused by extreme weather, plane crashes, health crises (e.g. pandemics), or a combination of multiple types of event. Gold Commanders are in overall control of the crisis, however they will not generally be at the site of the emergency, but typically co‐located in a control room. They will set the direction and propose solutions for the tactical (Silver) commanders to implement, who give direction to operational commanders (Bronze), who are responsible for organising resources on the ground. Their objectives are to: save and protect life; relieve suffering; contain the emergency; provide the public with information; protect the health and safety of staff; safeguard the environment; protect property; maintain / restore critical services; maintain normal services appropriately; promote and facilitate self‐help; facilitate the investigation / inquiry; facilitate community recovery; and to evaluate and identify lessons learned. When a crisis occurs, human behaviour and preparedness is critical to the delivery of an effective solution and therefore the training of Gold Commanders needs to be as realistic as possible. It is therefore important to be able to simulate the information overload and related stress, together with the pressure in making decisions

63

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon during a training session. Ideally the trainees should experience emotions of a similar nature and intensity to those that they might experience when dealing with a real emergency. To this end, within Pandora, trainee behaviour and emotional state are:

Captured through the use of a pre‐event questionnaire

Monitored during training through direct biometric sensor inputs, self‐reporting by trainees, trainer inputs, and using affective media effects and information manipulation techniques to induce changes to those behavioural and emotional states

Reviewed and evaluated post‐hoc through feedback and review

The key argument for the use of affect in the Pandora scenario is in the creation of affective ambience, by which we mean the use of multimedia assets and information management and manipulation to create emotional impact and engender requisite levels of stress in the trainees, at decision‐making points. Although there is some consideration of the use of avatar representations of emotion in the distributed mode of the Pandora Advanced Training Environment (PATE), the key affective representation is associated with the creation of ambience or atmosphere. A number of techniques from film and TV production have been combined with a number of known issues in information management and manipulation, to create a set of rules by which multimedia mashups can be created and made available to the trainees across a set of configured media channels. This provides the potential to induce stress in the trainees through the use of vocative inflections, video and image representations of crisis situations, and textual updates from those situations. However, the system can also induce stress through the corruption of media channels, incomplete information, missing information, noisy channels, and a variety of other techniques to diminish the certainty of information available to trainees at those points in the scenario where they are required to make decisions. Using this kind of induced stress the system can impact on trainees in terms of their self‐efficacy, leadership capability, and existing personality traits, to determine their capability to make effective and appropriate decisions under stress. The Pandora system also offers the opportunity to analyse trainee performance on the basis of a prior behaviour analysis based on questionnaire input, real‐time and review feedback from within the scenario, and post‐hoc analysis of behavioural and decision making performance. The benefits of such an approach are clear, in the ability of the system to introduce additional stress as required to impact the decision making of the trainees, and thereby more effectively replicate real‐world conditions and determine trainee capability under stress and uncertainty. A comparison between the two systems highlights both the similarities and the differences: In terms of common features, they are both computer based simulations of real world scenarios delivered in an interactive environment, and both are designed to create an affective impact in a highly pressurised, stressful, decision making training simulation. However, they differ quite considerably in their implementation and engagement with the trainee, for example:

Maritime City uses a virtual immersive technology to engage the trainee whereas Pandora primarily provides an augmented reality, rich multimedia environment (however, there is also a virtual distributed mode of the PATE).

The training takes place within a group dynamic in both cases. However, Pandora is focused on a collaborative dynamic, whereas Maritime City is based on an individual interaction, observation and feedback dynamic.

Maritime City is based on an entirely immersive dynamic, so the trainees will be engaged within the scenario continuously, and any analytic or discursive activity within the training group will require suspension of the scenario. By contrast, Pandora provides an augmented reality scenario for the trainees, which is designed to encourage group interaction and discussion, so there will be periods of activity between trainees outside of the Pandora system per se, so engagement is not intended to be continuous, although it is time‐controlled.

64


Maritime City uses the affective computing technique of avatars that display emotion. According to the research (Mosera et al, 2007) this approach will exercise stress on a trainee, although that perception may vary depending on how immersed a trainee feels in the game. The other areas which reinforce / accentuate the emotion represented by the avatar are the use of voice, and a limited effect from the use of body postures e.g. adopting a threatening pose, and some environmental information in scene setting. In Pandora we are explicitly manipulating the atmosphere / environment. Where we have avatar based characters then we seek to employ visual representation of emotion coupled with the use of vocative inflections. However, this is in addition to mashups of auditory, visual and textual cues at different levels of emotional affect, and information manipulation techniques in the form of missing, incomplete and noisy information, in order to create ambience and induce stress where it has an affective impact on the trainees and in particular on their decision making.

The script for Maritime City is deterministic as the interaction is pre‐canned and all of the decision points are fully populated. In Pandora, if the trainer so desires, the PATE can be non‐deterministic, which is a requirement given the nature of the people being trained, who are required as part of their role to be creative in generating solutions to problems. Although a scenario in Pandora is pre‐defined and a large number of the decision points are pre‐populated, the trainer has the facility to dynamically interject events and/or take control of one of the virtual characters in the scenario in order to respond to the queries or proposed solutions of the group. This is important in order to facilitate the realism of the interaction and retain engagement with the training environment.

5. Conclusions The use of affective computing techniques in the design and development of computer games has been a growing and evolving feature as games narratives and interactions have become more complex and realistic. The impact of this on serious games, such as those used for e‐learning, can be profound and, as has been demonstrated in this paper, can encompass a wide range of outputs to the player. As described earlier, the successful use of affective techniques in games for e‐learning can lead to greater engagement, immersion and reuse, which in turn can have significant benefits in the development of well constructed learning for the players. However, what is also clear from our research in this area is the need to ensure that any components or elements introduced into a game are properly contextualised, and appropriately utilised, relative to the narrative and gameplay. This requires considerable skills on the part of the developer, and the production values applied must be of sufficient quality, particularly in representing emotions through avatars, to achieve realistic or lifelike effects. Failure to achieve this will, at best, damage the gameplay and discourage replay and, at worst, may make the game unusable and laughable.

References Bacon, L., MacKinnon, L.M., Cesta, A. and Cortellessa, G. (2012) "Developing a Smart Environment for Crisis Management Training," Special edition of the Journal of Ambient Intelligence and Humanized Computing, entitled Smart Environments and Collective Computational Intelligence for Disaster Management, vol. 3. CommunityCare.co.uk (2010) Baby Peter case in Haringey. HTTP://WWW.COMMUNITYCARE.CO.UK/ARTICLES/2010/12/09/109961/BABY‐PETER‐CASE‐IN‐HARINGEY.HTM Csikszentmihalyi, M. and Csikszentmihalyi, I. (1975) Beyond boredom and anxiety. San Francisco, Jossey‐Bass. Department of Health, UK (2002) Requirements for Social Work Training. HTTP://WWW.DH.GOV.UK/PROD_CONSUM_DH/GROUPS/DH_DIGITALASSETS/@DH/@EN/DOCUMENTS/DIGITALASSET/DH_4060262.PDF Flynn, R., Bacon, E. and Dastbaz, M. (2010) "Impact of Learning Paradigms in Games Design: How the Theory of Learning has Influenced the Design of Video Games," Proceedings Global Learn Asia Pacific 2010, Penang, Malaysia. Gee, J.P. (2005) Why video games are good for your soul, Common ground Publishing. Galarneau, L. (2004) "The e‐learning Edge: Leveraging Interactive Technologies in the Design of Engaging, Effective Learning Experiences," Proceedings of e‐Fest 2004, Wellington, New Zealand. Graven, O.H. and MacKinnon, L.M. (2005) "A survey of current state of the art support for lifelong learning," Proceedings of ITHET 2005, Juan Dolio, Dominican Republic. Graven, O.H. and MacKinnon, L.M. (2008) "Prototyping a Games‐Based Environment for Learning," Proceedings of eLearn 2008, Las Vegas, US. Graven, O.H. and MacKinnon, L.M. (2009) "Prototyping Games‐Based Environments for learning C++ programming," Proceedings of HCI Educators 2009, Dundee, UK. Graven, O.H., Samuelsen, D. and MacKinnon, L.M. (2009) "Computer‐based Role Playing Game Environment for analogue electronics," International Journal of Online Engineering, vol. 5, pp. 27‐33.

65

Olaf Hallan Graven, Lachlan Mackinnon and Liz Bacon Graven, O.H., Hansen, H.A. and MacKinnon, L.M. (2009) "A Blended Learning Exercise using a Computer Game based on Abstract Learning Materials," Proceedings of ICL 2009, Villach, Austria. Kiili, K. (2005) "Digital game‐based learning: Towards an experiential gaming model.pdf," Internet and Higher Education, vol. 8, pp. 13‐24. Kolo, C. and Baur, T. (2004) "Living a Virtual Life: Social Dynamics of Online Gaming," Game studies, vol. 4, 2004. Manninen, T. (2003) "Interaction Forms and Communicative Actions in Multiplayer Games " Game studies, vol. 3, 2003. Mosera, E., Derntla, B., Robinson, S., Finkd, B., Gurb, R.C. and Grammere, K. (2007) "Amygdala activation at 3T in response to human and avatar facial expressions of emotions," Journal of Neuroscience Methods, vol. 161, pp. 126‐133. Oxland, K. (2004) Gameplay and Design, Pearson. Picard, R.W. (2000) "Towards computers that recognize and respond to user emotion," IBM Systems Journal, vol. 39, pp. 705‐719. Strom, P.S. and Strom, R.D. (2011) Adult Learning and Relationships: Information Age Publishing Inc. Valve Corporation (2007) SOURCE. HTTP://SOURCE.VALVESOFTWARE.COM/

66

Communities of Practice: The Impact of Web 2.0 Technologies Paul Griffiths1, 2 and Teresita Arenas2 1 Birchman Group, Chile and Henley Business School, UK 2 Universidad Santa Maria (USM), Chile [email protected] [email protected] Abstract: This study re‐visits an organisation that defined its knowledge‐management strategy in 2008‐9 applying an established strategy‐intellectual capital alignment framework. It addresses questions “How has knowledge management evolved at ENTEL, and what lessons can be learnt? Does the strategy‐knowledge management alignment framework applied at ENTEL in 2008‐9 still hold today?” The enquiry applies qualitative research in the form of a single case study, applying semi‐structured interviews and analysing the evidence through coding at a phrase level. It arrives at some interesting findings, such as that leadership of communities of practice (COPs) is critical to their success, at least in the early stages of their implementation. Also that the incorporation of generation Y (GY) into the workforce is changing the culture and openness to sharing knowledge, and thus accelerating the adoption of social networking (SN) tools, but the barriers to full deployment are still embedded in the older generation of senior and middle managers. Finally, it also emerges from the study that the paradigm by which organisations needed to chose between people‐driven and technology‐driven networks may no longer be valid: due to changes in culture, to the need to speed up knowledge transfer, and the advent of low‐cost and low‐complexity SN technologies, organisations can make the most of both. Keywords: e‐learning, Web 2.0, communities of practice, knowledge management, intellectual capital

1. Introduction A study performed with knowledge‐intensive organisations in 2008 arrived at a framework that links the type of knowledge networks (i.e., technology‐driven, people‐driven) that an organisation should prioritise according to its value discipline. An interesting finding of that work was that organisations selected either technology‐ driven or people‐driven networks, not both simultaneously (Griffiths & Remenyi, 2008). In 2008‐9 one of the authors was involved with a project to strengthen knowledge management at ENTEL, a telecommunications company in Chile. The framework applied in that project was precisely that described in Griffiths & Remenyi (2008). In the present paper the authors go back to ENTEL to review their knowledge management maturity nearly five years hence, with an aim at addressing the questions: How has knowledge management evolved at ENTEL, and what lessons can be learnt? Does the strategy‐knowledge management alignment framework applied at ENTEL in 2008‐9 still hold today? The following section gives an overview of the framework; section 3 describes the methodology followed in this inquiry; section 4 gives background on ENTEL and the knowledge management project developed in 2008‐ 9; section 5 does a case study of the present status of knowledge management at ENTEL; sections 6 presents a discussion on the findings, and section 7 gives the conclusions and suggestions for future research.

2. The framework Researching with organisations in knowledge intensive industries two clear trends emerged: While some organizations rely on a knowledge codification strategy that seeks to make knowledge independent of individuals and store it in repositories for users to access through information and communications technology (ICT) tools (Davenport & Hansen, 2002), others rely on a personalization strategy that emphasizes the channeling of individual expertise to the right place at the required time through person‐to‐person interaction (Bartlett, 2000; Griffiths & Remenyi, 2007). The issue of how outward looking the organization has to be in its knowledge management also emerged as a core concept. Organisations need to find the right balance, the most effective blend, between internal and external content, and avoid the trappings of being too introverted, too satisfied with their own view of the world. Their internal networks need to link up with external ones in their areas of expertise (Bartlett, 2000; Collins, 2002,1998; Ezingeard et al., 2002)

67


IV

III

I

II

L

Technology--Based Network Technology

H -> Extramural

This can be represented in a two‐dimensional space as shown in Figure 1. One axis represents the degree of development of knowledge management founded on Technology‐based networks, from Low to High. The other axis represents the degree at which the organization has developed its person‐to‐person knowledge sharing capabilities (People‐based networks.) In both cases the “High” development indicates a robust integration with external knowledge networks.

L

Personal Network

Figure 1: Knowledge management founded on technology‐based networks vs. personal networks (Griffiths & Remenyi, 2007) By placing the organisations on this plane according to their approach to knowledge management, and then anlysing their business strategy expressed by the value proposition they make to their clients, a clear pattern emerged linking knowledge‐management‐approach to business‐strategy. Organisations that build their competitive positioning on delivering tailor‐made, one‐off services to their Clients, place themselves in quadrant II. That is they approach knowledge management by developing strong people‐based networks(Bartlett, 2000; Griffiths & Remenyi, 2007) Organisations that compete by building scale and efficiency and found their value proposition on replicating proven solutions, place themselves in quadrant IV. Their knowledge management initiative is top‐down, with standards and rigid guidelines created at the centre. They make significant investments in ICT networks aimed at producing large document repositories and powerful search engines (Davenport & Hansen, 2000; Griffiths & Remenyi, 2007; Haas & Hansen, 2005). The prior study arrived at that start‐up and boutique consulting firms are forced into quadrant I. The study also discovered that the hybrid approach to knowledge management of quadrant III was probably a utopia, and utopias are dangerous: they mobilize in the short but paralyze in the long term (Romano de Sant’ Ana, 2006). In summary, the framework states that on the one hand an approach to knowledge management driven by people‐based networks is bottom‐up, requires low cash investment but significant personal time; it relies heavily on tacit knowledge and creates an environment for moving knowledge from tacit to explicit. On the other, an approach to knowledge management driven by technology‐based networks is top‐down; requires high investment and relies mainly on explicit knowledge. The study did not find any leading organisation that was strong on both people‐based and technology‐based networks, so a significant premise was that attempting that was a utopia and revealed a lack of commitment in terms of the value proposition of the organisation.

3. Methodology This inquiry is embedded in the interpretive tradition. It does a single, in‐depth case‐study on the status of knowledge management in an organisation, by means of qualitative methods. The unit of analysis is the whole new‐co ENTEL (result of the recent merger of old‐co ENTEL and ENTEL PCS.) The main sources of evidence are semi‐structured interviews to six members of the organisation’s management team performed in January

68

Paul Griffiths and Teresita Arenas 2014, and field‐notes and reflection‐notes from the prior intervention in 2008‐9. Rich notes were taken at the interviews, and detailed interview transcripts were written up from them (Yin, 1994). The semi‐structured interview guide was informed by the framework described in section 2. It has seven sections, namely: (1) Background, that aims at establishing the position and involvement in knowledge management of the informant; (2) The Knowledge Management Network Model, that presents the characteristics knowledge networks and their application at ENTEL; (3) Old Strategy‐KM Alignment Framework, presents the alignment framework to the interviewee and seeks feedback on its application at ENTEL; (4) A Shift in Paradigm inquires on the interviewees vision on why the prior framework may be falling out of sync with the knowledge economy; (5) Our Hypothesis for the Causes of the Shift exposes our ideas of change and seeks feedback from the informant; (6) ENTEL: KM and Innovation explores the relationship between KM and innovation at ENTEL; and (7) Open Discussion creates a space for the informant to expand on any aspect of Strategy‐KM alignment not addressed in the other sections. The interview guide can be obtained from the authors. Data analysis is done via coding of the interview transcripts: Open‐coding at a phrase level, followed by Axial‐ coding to identify emergent concepts, and finally bringing it altogether in a narrative (Strauss & Cobin, 1990). As is expected in this approach, the objective is particularisation and not statistical generalisation.

4. ENTEL case study 4.1 The company ENTEL is one of the largest telecom companies in Chile. The country fully privatised and opened up its telecommunications services in the late 1980s and early 1990s, in what is generally known as one of the most successful cases of its kind. Success here is used in the sense that it resulted in a much improved quality of service and reduced cost of service for the population, and established the telecommunications service as part of the platform that enabled the radical transformation to efficiency and growth of the Chilean economy. Chile has for many years had one of the most competitive telecom sectors in the world. ENTEL, previously a state‐ owned land‐line telecom service provider, has specialized in serving the corporate sector in its voice and data communications needs. ENTEL also set up a successful mobile‐telephony service that until recently was managed through a separate company called ENTEL PCS. ENTEL had a high degree of penetration in the corporate market, where it reached close to 60 percent market share in its traditional telecom products. Its leadership team thus realized that in order to grow it would need to have new products to offer its client‐base and decided to incorporate IT Outsourcing Services to its product offering. Responding to this strategic change, in 2004 ENTEL entered the IT Outsourcing Services market and won several projects, some of them large and highly visible. It created its Corporation Services Division (CSD) with its sales and delivery capabilities by recruiting individuals with experience in the service, and by setting up ad‐hoc partnerships to tackle each opportunity, but without making the structural and organizational changes that entering this new product market required. In the 2007‐2009 period it decided to engage consultants to design the required changes. The CSD was aligned along four industry groups (i.e., Financial Services, Retail & CPG, Natural Resources and Government) with a specific technology offering for each one. In the diagnostic stage it quickly emerged that one area that needed attention was knowledge management and the leadership of the CSD agreed to apply the strategy‐knowledge management alignment framework given above to do so. One of the authors of this paper was invited by ENTEL CSD to enable its application. The initiative was kicked off in June 2008.

4.2 Application of the framework The leadership team at CSD arrived at a consensus to adopt a value discipline of “customer intimacy”, and thus make a value proposition to Clients based on “best total solution.” By doing this it was thought that CSD would position itself to compete with large international IT Outsourcing providers operating in Chile such as IBM, EDS, TCS (that come to the market with global, standard, ‘leading practice’ solutions), by leveraging its better knowledge of the local market and its clients, to offer tailor‐made solutions.

69

Paul Griffiths and Teresita Arenas Discussion then moved on to how CSD should approach knowledge‐management and, in accordance with the framework, it was agreed that they should favour a people‐driven knowledge networks approach, rather than technology‐based ones. It was decided to set up six Communities of Practice (CoP) on key knowledge areas for the company, one of which would be taken as a pilot, and the other five would follow incorporating lessons learned from the pilot. It was decided to give the pilot CoP a 5 month period for design and implementation, and that the 6 CoPs would be operating in a year. It was also decided that all 140 professionals in CSD would be encouraged to belong to at least one CoP, and that membership of a CoP would be voluntary. The six CoPs were defined in the following subject areas: COP1: Technology Convergence and New Developments COP2: Technology solutions for Financial Services COP3: Technology solutions for Government COP4: Technology solutions for Retail and CPG COP5: Technology solutions for Natural Resources (Mining) COP6: Information and Communications technology services th COP1 was defined as the pilot COP and was launched on July 30 2008.

4.3 The outcome The plan was to set up the pilot CoP and run it for three months. After that we would reflect on improvements to the design, and launch COP2 and COP3 in October 2008, and COP4, COP5 and COP6 in January 2009. The first job was to appoint the COP1 Leader for which an ideal profile was designed and the selection was done with the VP for CSD and the heads of the industry vertical groups. The COP1 Leader was appointed in their first meeting. In parallel with the appointment of the COP1 Leader, a call for volunteer members to the six COPs was done with a very good response, as 30 people were accepted for the first intake of the COP1, and similar numbers for the other COPs. It was defined that initially the pilot COP would meet in face‐to‐face meetings that needed to be moderated by the leader. The leader was coached on different techniques for moderating the meetings such as “Knowledge Cafe”, “Socratic Dialogue”, “Research & Present”...The first meeting took the form of a Knowledge Cafe to define the Vision and Mission of COP1, and to agree on what would be the channels for communication until there were a technological infrastructure to facilitate this. It was agreed that a mailing list would be set up, and that the COP1 Leader would arrange to set up a repository to store the IP produced in the meetings so that all members could have access to it. It was also agreed to create a COP e‐bulletin that would summarise issues discussed and comment on other issues of interest to its members. As planned, after three months a balance was drawn and the design of the COP was reviewed on the basis of experience. After that and on the basis of the updated design, COP2 and COP3 were launched; and subsequently COP4, COP5 and COP6 too. As would be expected, the development of the six COPs took different rhythms and differing degrees of achievement – details cannot be included here due to limitations of space, but they can be made available by the authors on request. In July 2009 the global credit crunch hit Chile severely, which led ENTEL to review downwards its budget and cut expenses on many fronts. One of the measures was that the COPs would now have to operate without the assistance of an external coach. Clearly by that time the functioning of the six COPs had made progress but was still not mature, and the technological platform had been designed and was in the process of being developed using MS Sharepoint, but was still far from being deployed. From observing the functioning of the six COPs it became evident that the characteristics of the COP Leader are crucial to its success, at least in these early stages, as can be seen from the field‐notes.

70


5. Knowledge management at ENTEL 5.1 Knowledge management at ENTEL is essential Turnover and headcount growth are extremely high at ENTEL. The present headcount is approximately 6,000 people and some key areas of the company have turnover rates of 50%. The manifestation of this is not significant at senior management level, but very much so for middle managers and supervisors down. On average people stay with ENTEL for 18 months, which means they leave just as they are becoming productive. As a result of this, the operational units are doing their own induction programmes, by‐passing those of the HR department because they find them too superficial. They need to do in‐depth and department‐specific training, and with the sort of numbers they are recruiting it takes no time to fill a classroom of twenty. ENTEL is now working on employer‐branding as a means to improve attraction and retention of staff, but with the turnover rates they suffer, knowledge management (KM) is essential

5.2 Strategy and KM alignment At the time of the initial intervention of this research, ENTEL managed its traditional services and its mobile services as completely independent companies. The former was known as ENTEL and the latter as ENTEL PCS. Due to market rationale it has merged the two companies and divided its structure into three large market segments: Corporate, Enterprise and Retail. As the names of the new divisions imply, Corporate looks after large corporations, Enterprise after SMEs, and Retail after individuals. From our interaction with company leaders it is quite clear that this new structure is not yet fully consolidated. As in the early intervention, Treacy & Wiersema’s (1995) value discipline model was applied to understand the new organisation’s business strategy. From the interviews it becomes clear that the organisation is still not fully aligned on a value discipline. Although there are informants – particularly in the Corporate market division ‐ who stated that the priority is Customer Intimacy, the issues of focus on cost and of highly rigid processes was continually lingering in our conversations. We detected expressions such as that “there is much talk of pushing for customer satisfaction but reality is that we are strongly driven by financial results”. Differences across the market segment divisions also emerged with, naturally, the Corporate market division putting more emphasis on the customer but the other divisions stating that they pursue Operational Excellence (OE). The merger of the two companies is still a highly relevant issue in people’s day‐to‐day activities. The models and culture of the two companies were significantly distant from each other and that impacted their approach to client service and KM. On the one hand, at old‐co ENTEL’s processes and procedures were highly formalised and rigid, leading to clients being relegated in the staff’s priorities (e.g., clients had to fill detailed and unfriendly forms, and there were no real procedures to keep their information updated). On the other hand, at ENTEL PCS the agents would concede to every client demand with total disregard for procedures, which later translated into great difficulties to keep track and deliver, which also translated into poor client service standards.

5.3 Knowledge networks One of the issues that became evident early on in the merger of old‐co ENTEL and ENTEL PCS was that the people coming from each part of the old organisation had very little knowledge of the technologies applied and managed by the other part. In order to overcome this problem, the Innovation division launched two initiatives. The first was a “Technology Bulletin” aimed at breaking down technology knowledge‐silos across divisions. It contained articles written by in‐house experts that circulated monthly to the 800 senior and key people in the new organisation. The other initiative was what they called “Technology Teams.” Under this initiative task forces were put together to research new or emerging technologies and their application to ENTEL services. Once the research had concluded the findings were presented to the ENTEL community in periodic meetings called “Team Day.” Introduced to the knowledge‐networks two‐dimensional framework, informants tended to agree with it. Their view on whether ENTEL followed a People‐driven or a Technology‐driven networks approach was somehow blurred. Several informants mentioned Technology‐driven ones but when they were asked to go further in

71

Paul Griffiths and Teresita Arenas detail it became apparent that they were referring not to Technology‐driven ones in the sense of the framework (i.e., repositories) but in the sense of People‐driven networks supported by some kind of technology. An interesting concept that emerged from the interviews is that knowledge networking in ENTEL is being driven by the grass‐roots rather the company. While the old stock are reluctant to get involved, the new young people are pushing the agenda by bringing in their own devices and being permanently connected. This has led to the realisation that the company has very little control over even the most sensitive information (e.g., commercial proposals) which has triggered great concern particularly for the Information Security Officer. This situation is leveraged by the fact that the average age of ENTEL’s staff is quite young (under 38 years) and, according to the informants, these people have a far more collaborative mindset which, together with connectedness, strengthens the people‐driven knowledge networks. In line with the above, the prevailing opinion is that knowledge repositories and other static document management tools are not enough. It is not enough to store the deliverables of projects, but there is a need to keep record of the history of project and conversations and other sorts of communications between project members. There is also a need to have Facebook‐type tools to find the experts and connect with them, and even to ‘follow’ them. There is a need for effective social networking tools that enable people to store and share their information and knowledge. If these are not provided, people will store this information in their own hard discs and share through their Drop‐box. In terms of social networks (SN) platforms, the previous initiative based on Sharepoint was not successful. It was adopted because ENTEL had corporate licenses, but in the end failed because of its complexity, its rigidity and the need to have a systems administrator which kills its spontaneity and agility. ENTEL has embarked on a pilot SN project based on Jive that has the advantage of not needing a centralised administration. They currently have between 300 and 500 people connected on the pilot, who are using the SN to go beyond sharing documents and are using it as a communicating platform. People‐driven networks exist in ENTEL but they are small, which is an indication of immaturity. An example of a group that works is the remnants of the Government Community of Practice (CoP). They have a core team of 5 or 6 people that call themselves “Good Ideas Community” (Comunidad de Güenas Ideas, CGI). Up to last year they used to meet weekly, but have now moved to meeting on demand. They analysed specific cases or best practices that they then share with their industry group, and are connected through a WhatsApp group. They find that knowledge sharing is happening and is growing within this community. They have noticed that SN tools have helped increase collaboration. Our informants assign to this effective KM initiative the fact that the Government and Mining Industry groups, which have been recently merged, have by far the highest rating on the “Best Place to Work” survey. This industry group scores amongst the top 30 companies in Chile, while ENTEL as a whole is well below the Chilean average.

5.4 The human factor and its impact on KM Senior management’s view on social networks is paradoxical. On the one hand this group understands the value and benefits of SN, and have hard evidence that they are here to stay, as the company’s revenue from SMS traffic has stagnated or even dropped due to the widespread adoption of WhatsApp amongst its clients. On the other hand their adoption rate of SN is low and senior management does not use SN to mobilise their people. At the middle management level, particularly amongst the older population, resistance is even greater, which is a significant barrier to implementation. The present pilot with Jive is working with a relative small sample of young enthusiasts, but how to penetrate this contingent will be a challenge, particularly if senior management does not support it explicitly. Hurdles to implementation are not only entrenched in the type of people as mentioned above, but also in their incentives. The present incentives system is based on short term financial results and strongly geared towards sales‐related compensation. It is thus not aligned with the more mid‐term objectives of incorporating SN to improve work. Success in this is therefore conditioned to a revision in key performance indicators.

72

Paul Griffiths and Teresita Arenas ENTEL does not have a position of Knowledge Manager but from the discussions it emerged that it should have. The role entails having competencies for innovation and for communications. The individual must generate trust as quite often in organisations knowledge is seen as power, so the positions of knowledge manager needs to fall upon an individual who conveys clarity and transparency. It is not strictly necessary that the individual has a strong technical background – it would help, but it is not strictly necessary.

6. Discussion Reflecting on the effects of the KM project at the CSD of ENTEL in 2008‐9, one concludes that it had a positive short‐term effect on the organisation which was getting people to know each other and extend their in‐house networks. However, the sustainability of the COPs that were the visible effect of the project, had difficulties. Of the six COPs that were set up, only one that appears to engulf two of the original COPs is still operating: the one in the subject of Technology Solutions for Government that later absorbed the one corresponding to Mining. It has to be admitted that this does not come entirely as a surprise. There are field‐notes where it was anticipated in March 2009 that in COP1 and COP2 the leaders were not taking ownership. In the case of COP3 (Government) it already looked promising and the leader has stayed in charge since then and the COP is still operating under the name CGI. COP4 and COP5 were working well, but the two leaders subsequently left the organisation. And in the case of COP6 it was anticipated that the COP leader was methodical and thorough, but his leadership style was controversial and people were already “voting with their feet” on the survival of the community. The state of the COPs is aligned with predictions made in March 2009 that the model was still not mature. There were cultural issues deriving from ENTEL being a very hierarchical organisation and not accustomed to the idea of relationships across hierarchical barriers. It was also said that in its early stages the survival of the COPs depended much on the performance of its leader, and this seems to be corroborated by the comments in the previous paragraph. ENTEL’s high staff turnover and aggressive recruiting rate has caused an increasing number of GY amongst its headcount. This alone is causing cultural changes with the new generation being far more prone to connectedness and having a more natural tendency to share through social networks. This has helped trigger the ubiquity of connection devises and leverage the network effect. The question that remains to be answered is whether this connectedness and participation in social networks has translated into business benefits for the organisation. Closely connected to the previous point is the issue of the causal relationship between the change in culture that promotes people‐driven networks, and the facility for implementing social networking. Has the fact that the organisation is making several attempts to implement social networking tools had a positive impact on ENTEL’s people increased openness to knowledge sharing? Or is it that the change in culture in terms of propensity to share is enabling the implementation of social networking tools? From analysing the evidence it is concluded that the latter is the case. Social networking tools are being implemented with relative ease in the pilot phase when it is reaching people who are avid to using this technology. As seen in section 5, when social networking initiatives reach the upper echelons of the organisation and the older mid‐managers (both groups normally baby‐boomers) the initiatives run into trouble. This corroborates prior experiences that technology adoption demands cultural change, and not that technology adoption leads to cultural change. In that sense, social networking technologies are no different from other ICTs. With regard to technological tools, it has emerged that social networking tools need to be user‐friendly if they are to be adopted and flourish. They need to have i‐Pad type touch‐screens and be self‐managed, such as Jive or Yammer. Sharepoint has proven to be problematic mainly due to poor interface, to difficulties in configuration and, above all, the rigidities imposed by the need to be centrally managed. ENTEL is tackling its high HR turnover by decentralising induction training (which consumes many resources and retards new entrants’ understanding of the corporate culture) and entering in employer‐branding campaigns. Considering that there is evidence from the Government vertical in the Corporate market that good KM increases job satisfaction and thus retention, would it not make more sense to invest those funds in strengthening the organisation’s KM?

73

Paul Griffiths and Teresita Arenas It is clear that all our informants understand and basically agree with our two‐dimensional knowledge networking framework and the model that connects it to the value discipline of the organisation. From our conversations it also emerged that they can see why this paradigm would have changed in recent years. On the one hand, knowledge repository technology has become far cheaper, so technology‐driven networking tools do not require the large investments of the paradigm described in section 2. On the other hand, the incorporation of GY with their greater openness to sharing and with them the SN tools, has increased the facilities for extending peoples’ people‐network. The fact that SN tools have incorporated knowledge repositories into a single technological platform, again undermines the need for the “or” and promotes the “and” decision. The prior trend is stregthened by changes in our industrial organisation and the reinforcement of the knowledge economy. As indicated by Viedma & Cabrita (2012, pp. 258‐81) cited in section 2, on the one hand the growing importance of Relational Capital with respect to Human Capital and Structural Capital is also undermining the “or” decision and leading to people‐driven knowledge networks being a “must”. On the other, in the present context business models need to evolve permanently so organisations cannot rely on simply replicating solutions found in a knowledge repository (the traditional model of OE) so re‐utilising old solutions needs to be complimented by creative enhancements achieved by working with experts. The external forces of the economy are thus pressing organisations to develop both technology‐driven and people‐ driven networks. ENTEL has still not matured along this line which explains why its innovation programmes (e.g., Open Innovation, Centre of Excellence) have not been successful. Finally, in retrospect it now looks surprising that the framework does not include any construct related to culture. With the evident role that culture and leadership play in the functioning of communities of practice, it is clear that future models will have to consider this.

7. Conclusions The ENTEL case is an interesting one to study due to that its KM strategy was designed in 2008‐9 applying the framework described in section 2. The lessons that do emerge clearly from this case:

Leadership of COPs is critical to their success, at least in the early stages of their implementation.

The incorporation of GY is changing the culture and openness to sharing knowledge, and accelerating the adoption of SN tools, but the barriers to full deployment are still embedded in the older generation of senior and middle managers.

KM at ENTEL seems to be moving in the right direction, but still needs a long way to go to be mature.

Moving further down this path seems to be a pre‐requisite for ENTEL to be able to materialise the potential of innovation programmes.

This paper appears to support that the paradigm of strategy‐KM alignment has given way to a new state of affairs where organisations need to pursue both people‐driven and technology‐driven knowledge networks. The impact of this is that our strategy‐KM alignment model would no longer be valid, and we are enticed into searching for a new framework that looks beyond knowledge networks. Reflection on the ENTEL case leads us to believe that such a model will need to incorporate “culture” as a significant construct. Clearly a single case is not enough to arrive at generalisations or even final conclusions on the falsification of the fraemwork and other cases will have to be analysed. And finally, back to the title, the impact factor is not Web2.0 but GY entering the workforce.

References AIIM (2013) What is Enterprise 2.0 http://www.aiim.org/What‐is‐Enterprise‐20‐E20 Bartlett, C.A. (2000) McKinsey & Company: Managing Knowledge and Learning, Harvard Business School Case Study, No. 9‐ 396‐357 Collins, R. (2002 [1998]) The Sociology of Philosophies: A Global Theory of Intellectual Change, Belknap‐Harvard University Press, ISBN 0‐674‐00187‐7 Davenport, T.H. & Hansen, M.T. (2002) Knowledge Management at Andersen Consulting, Harvard Business School Case Study 9‐499‐032 Ezingeard, J‐N., Leigh, S. & Chandler‐Wilde, R. (2002) Knowledge management at Ernst & Young: Getting value through knowledge flows, ICIS Conference Proceedings, 2002

74

Paul Griffiths and Teresita Arenas Griffiths, P.D.R. & Remenyi, D. (2007) Using Knowledge for Competitive Advantage in Professional Services: A Case Study, Proceedings of the 4th International Conference on Intellectual Capital, Knowledge Management & Organisational Learning, University of Stellenbosch Business School, South Africa, 15‐16 October, Edited by Dan Remenyi, pp.169‐ 178. Griffiths, P.D.R. & Remenyi, D. (2008) Aligning Knowledge Management with Competitive Strategy: A Framework, Electronic Journal of Knowledge Management, Special Edition, Edited by Prof Rembrandt Klopper, Vol. 6, Issue 2, pp.125‐134 http://www.ejkm.com/volume‐6/v6‐2/v6‐i2‐art5.htm Haas, M.R. & Hansen, M.T. (2005) When Using Knowledge can Hurt Performance: The Value of Organisation Capabilities in a Management Consulting Company, Strategic Management Journal, Vol. 26, pp. 1‐24 Sarvary, M. (1999) Knowledge Management and Competition in the Consulting Industry, California Management Review, Vol.41, No.2, Winter, pp.95‐107 Strauss, A., & Cobin, J. (1990) Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage Treacy, M. & Wiersema, F. (1995) The Discipline of Market Leaders, Reading, Mass.: Addison‐Wesley Work and Family Researchers Network (2013) Definitions of Aging Workforce, https//workfamily.sas.upenn.edu/glossary/a/aging‐workforce‐definitions downloaded on 13JUN13. Yin, R. (1994), Case Study Research: Design and Methods, Sage Publications, Thousand Oaks, CA.

75

Negotiating the Triple Helix: Harnessing Technology for Transformation Maggie Hutchings1, Anne Quinney1 and Kate Galvin2 School of Health & Social Care, Bournemouth University, Bournemouth, UK Faculty of Health & Social Care, University of Hull, Hull, UK [email protected] [email protected] [email protected] Abstract: Universities are embracing digital technologies to enhance learning and teaching while endeavouring to maximise the student experience, minimise risks, and manage complex, sometimes competing and contradictory agendas. Government policies are focused on placing students at the heart of the process, but with the propensity for student identities to shift from partners in learning to consumers of education. Higher education institutions (HEIs) are grappling with the potential of technology‐enabled solutions to enhance education provision in cost‐effective ways without placing the student experience at risk. These pressures impact on academics and educational institutions requiring responses to the pace of change, role transitions, and pedagogical imperatives for student‐centred learning. The paper explores strategies for effective change management which acknowledge but minimise risks in technology‐enabled approaches for transformative learning. The analysis is informed by the development of a collaborative lifeworld‐led, transprofessional curriculum for health and social work disciplines, which harnesses technology to connect learners to humanising practices and evidence based approaches. Rich data from student questionnaires and staff focus groups is drawn on to highlight individual and organisational benefits and barriers, including cultural resistance recognised in staff scepticism and uncertainty, and organisational resistance, recognised in lack of timely and responsive provision of technical infrastructure and support. Intersections between research orientations, education strategies and technology affordances will be explored as triggers for transformation in a ‘triple helix’ model of change, through examining their capacity for initiating ‘optimum disruption’ to facilitate student‐centred learning, role transitions, and organisational change. We share the findings of ‘our story’ of change to harness the positive utility of these triggers for transformation through deploying strategies for negotiating complexity, including the requirement for a shared vision, a robust team approach, the need for ongoing horizon scanning and application of soft skills (e.g. active listening, timely communication) necessary in order to build student confidence, academic partnerships, and facilitate organisational dexterity in the face of barriers to change. Keywords: transformative learning; change management; technology‐enabled learning; role transitions; organizational change

1. Introduction The challenges for universities to survive and prosper in the early 21st century are highlighted by Shore’s argument (2010, p.15) that ‘a new set of discourses has emerged around universities and their role that draws together different, often contradictory, agendas’ heralding ‘a shift towards a new, multi‐layered conception in which universities are expected to fulfil a plethora of different functions’. Government policy in the UK and other countries exerts its influence, promoting the central role of the university in contributing to the knowledge economy while also seeking greater accountability and performativity evidenced in a multitude of measured outputs, performance indicators, quality assurance measures and audits (Olssen & Peters 2005; Shore 2010). In order to survive and prosper within this complex and competitive climate, HEIs must engage in strategies to advance research, education and professional practice while seeking to attract students. These demands impact on individuals and organisations, requiring the university to negotiate and respond to the pace of change, role transitions, and pedagogical imperatives for more student‐centred learning which is risk‐ free for the student experience. Universities are embracing technological advances to facilitate teaching and learning, simultaneous with the growing use of mobile and digital technologies in students’ everyday lives. Lea and Jones (2011, p.378) suggest ‘the potential of social networking, digital and mobile technologies are permeating the academy, not only through student practice but in terms of dominant institutional drivers and government‐led funding to harness technologies and applications for supporting teaching and learning’. Technology‐enabled approaches offer potential for enhancing student learning within the complexity and demands of HE provision by crossing boundaries between research and practice, creating opportunities for co‐construction of knowledge, and releasing academic staff potential to engage with a rebalanced workload in research, education, and professional practice. While change in higher education is endemic, technology‐enabled initiatives can

76

Maggie Hutchings, Anne Quinney and Kate Galvin contribute to the complexity and pace of these changes. These developments require detailed organisational planning, co‐ordination and resourcing (Breen et al 2001) to assure effective change management and minimise risks. This is not to suggest that change can be managed scientifically in a rational, ordered and linear fashion with appropriate planning tools and resources in place. The reality of change may be experienced by different stakeholders as an amalgam of more disjointed and disruptive processes. Initiators of technology enabled approaches, offering new mixes of tutor‐facilitated and student‐managed learning, can encounter resistance to change manifested at individual and organizational levels where these developments challenge deeply held beliefs and pedagogic practices (Greener 2009, 2010a. p.188).

2. Theoretical framework Research tends to concentrate on benefits and outcomes rather than examining evidence of processes and people at work in the disjuncture, flux and movement within education initiatives. Pennington (2003, p.4) highlights the tensions between outcomes and process orientations: Structures, procedures, attitudes and behaviours underpinning the status quo have often taken years to lay down and are not susceptible to overnight transformations. For this reason the introduction and management of change should be conceived as a rolling process requiring subtle and persistent choreography rather than a defined event occurring at a particular moment. We believe the notion of the positive utility of resistance to change should not be overlooked and can be explored and better understood in order to implement change successfully. Understanding transformative learning at individual and organizational levels, and acknowledging and working with resistance, reluctance and pedagogic diversity is at the heart of negotiating change creatively and sensitively. This position acknowledges the importance of context and situated learning (Argyris & Schön 1978, Lave & Wenger 1991) and builds on constructivist (Mayes & Freitas 2007) and experiential learning theory (Dewey 1933, 1938). The utility of generating purposeful disruptions as tensions and challenges to stimulate transformative learning has been considered elsewhere (Hutchings, Scammell & Quinney 2013). While recognising the value of education initiatives as levers for transformation and organisational change, we also recognise the challenges for change agents in attempting to achieve ‘optimum disruption’ where initiatives are experienced as too uncomfortable, too difficult or simply too unwelcome and therefore resisted or rejected (Hutchings, Quinney & Scammell 2010a). This paper shares ‘our story’ of negotiating change in the development of a collaborative lifeworld‐led transprofessional curriculum for health and social work disciplines. Our purpose is to explore the intersections between three strands, (1) research orientations, (2) education strategies, and (3) technology enabled learning, described as the ‘triple helix’, through their capacity for initiating ‘optimum disruption’ towards both transforming the student learning experience and academic and organisational cultures (See Figure 1). We will examine strategies deployed for negotiating complexity, including the requirement for a shared vision, a robust team approach, the need for ongoing horizon scanning and application of soft skills (e.g. active listening, timely communication) necessary in order to build student confidence, academic partnerships, and facilitate organisational dexterity in the face of barriers to change.

3. Case study Exploring Evidence to Guide Practice (EE2GP) is an undergraduate intermediate (Level I, Year 2) unit/module designed for large student cohorts. Technology is used to connect learners to humanising practices and evidence based approaches. Students are encouraged to integrate different kinds of complex knowledge and consider how their practice could be guided in such a way as to achieve transformative learning. Key drivers for this major development were informed by the University’s and School’s strategic priorities to:

Expose undergraduate students to research undertaken in the School of Health and Social Care (HSC) and bringing research and teaching cultures closer together;

Pioneer the application of innovative teaching, learning and assessment strategies

Increase usage of technology enabled learning;

Release staff potential; by achieving economies of scale through replacing face‐to‐face teaching across all the professional programmes with one common blended learning module.

77

Maggie Hutchings, Anne Quinney and Kate Galvin The scale of transformational change effected by this initiative was considerable with anticipated and unanticipated outcomes. Key challenges associated with changing cultures, managing the scale of technology infrastructure, support required, and raised expectations for learning technology provision, were highlighted at individual, School and organization levels. Since the curriculum was introduced in 2010, 11 professional groups have been involved with over 600 undergraduate students each year from nursing (adult, child health, learning disabilities and mental health), midwifery, occupational therapy, physiotherapy, operating department practice, paramedic science, community development and social work. The initiative impacted directly on working practices, within HSC and the wider University. Key stakeholders included, academics, programme leaders, the School’s management team, and professional staff in advisory and support roles, based in the School and centrally, including a web developer/educational technologist, learning technologist, IT project manager, academic staff developer, quality and enhancement officer, and examinations coordinator. Approximately 40 academic staff have contributed to the module as developers, champions and facilitators. Introducing this module as a large change management initiative necessitated the negotiation of barriers and risks associated with resistance to change and some scepticism in our School, not dissimilar to the ‘resentment and ambiguity’ identified by Browne (2005, p.57).

Figure 1: Triple helix model of change: research process and findings

4. Methodology ‘Our story’ of negotiating change is told through the voices of those experiencing it, to enable us to explore strategies for effective change management through three major levers for change (the ‘triple helix’, of research, education and technology) towards transformations recognised in impacts on student learning, academic roles and organisational development. The methodological approach adopted was to build a multi‐ authored narrative for our story of negotiating change (See Figure 1). This approach is informed by the concept of ‘organizational becoming’ (Thomas et al 2011, p.22), where organizations are recognised ‘not as fixed entities, but as unfolding enactments’ in flux and ‘constituted by and shaped from micro‐interactions among actors, situated in their every‐day work’. Our own position is that levers for optimum disruption towards achieving transformative learning can be recognised in action at individual and organizational levels through student and staff descriptions of their experiences. We draw on qualitative data collected in 2010‐11 and 2011‐12, captured through questionnaires and focus groups, to highlight individual and organisational benefits and barriers in deploying the triple helix (See Table 1).

78

Maggie Hutchings, Anne Quinney and Kate Galvin Table 1: Demographic profile of participants Data collection method Students Online evaluation questionnaire 15 item statements 5‐point Likert scale (Strongly agree to strongly disagree) 5 Open response questions What enhanced learning

2011.1 Block 1 n = 306 n = 301

2011.2 Block 2 n = 260 n = 243

2012.1 Block 1 n =219 n = 188

2012.2 Block 2 n = 302 n = 283

Student 2

What challenged learning

Student 4, 6

What you enjoyed most

Student 3, 7, 14 Student 12, 15, 17 98% n = 12

Student 1, 9, 10 Student 8, 13, 18, 20

Student 5, 19

Student 11

Student 16

94%

86%

94%

What you enjoyed least Response rate Staff Focus group (SFG) Horizon scanning tool

The student experience was monitored and evaluated through weekly deployment of questions using the ARS (audience response voting system) and an end of module online evaluation completed following their online exam. A staff focus group (n=12) was conducted with academic champions and developers interested in contributing to the module, using an horizon scanning tool to stimulate discussions. Further staff comments were captured during a launch event and through ongoing feedback from programme teams. The core project team also shared their experiences of developing the module, considering what it meant for them, and how they engaged with the ups and downs of the process. The analysis of this data informs this paper. Ethical processes were followed to ensure informed consent and data confidentiality in compliance with institutional protocols for undertaking educational research with students and staff.

5. Analysis of findings The intersections between research, education strategies and technology with their capacity for initiating ‘optimum disruption’ are examined in relation to achieving student‐centred learning, academic role transitions, and organisational change. These complex interrelationships forced us to consider issues of interdependency, tensions and, at times, conflicting agendas in respect of changing cultures, organizational priorities and our core team goals.

5.1 Research for practice A key driver for the initiative was to expose undergraduate students to research undertaken in the School by drawing on research expertise informed by a lifeworld‐led philosophy (Galvin & Todres 2013). 17 web‐based case studies were developed to provide diverse evidence of people’s experiences of specific illnesses and conditions, such as stroke, dementia, or social isolation. Students were facilitated to explore a range of evidence from the arts and humanities including narratives and poems, informed by citizen and service user perspectives, in association with qualitative and quantitative research papers, and policy documents, to guide practice for humanly sensitive care (Pulman et al 2012). 5.1.1 Student experiences Students appreciated the relevance of different kinds of research evidence for their practice and the value of engaging with service user and carers’ stories: Watching the clips relating to my case study, discovering what people went through and it having an impact on my way of thinking and how I can use this within my practice. (Student 1) The qualitative evidence stood out for me as I began to empathise with the patients. I was able to understand their thoughts and feelings, and began thinking of how this can be applied to practice. (Student 2)

79

Maggie Hutchings, Anne Quinney and Kate Galvin Students demonstrated developing awareness and confidence to assess different kinds of research evidence and apply critical judgement in professional practice: It made me realise that not all evidence is reliable and encouraged me to make my own decision about what evidence to take into account and how to apply it into practice. (Student 3) However some students experienced difficulties understanding research methodologies and terminology and seeing the application to professional practice. They described varying degrees of disruption from feeling challenged to experiencing the tipping point beyond optimum disruption: Getting to understand all of the research terms that I had never heard of before and relating these to practice challenged my learning and has given me a deeper interest into the subject of using evidence to guide practice. (Student 4) The amount of reading you were expected to do, and learning all the research processes was incredibly difficult as this topic was totally alien to me. (Student 5) 5.1.2 Staff experiences This initiative offered opportunities to develop greater integration between research interests and teaching in the School. Academic staff feedback welcomed the integration of lifeworld‐led theoretical perspectives for guiding practice: I just really loved the idea that the evidence comes from the arts and humanities as well as the sort of traditional research evidence. (SFG) Academic staff also appreciated how the module demonstrated success in bringing the research and teaching cultures closer together: Students have often said to me they think that research is done by those people who are very academic and very senior, so I think it’s really good that those people are actually teaching at an undergraduate level and making it applicable to practice in a really exciting way. (SFG) However staff also realised the disruptive nature of this approach for their own roles. The model of the unit challenges the traditional way in which we have viewed how we carve out our time as academics and teachers. I’d quite like to develop a case study so how does that fit with my role in the rest of the world of my work, it’s not a case of contact hours, but it’s about what role do I play? (SFG)

5.2 Education strategies for change Another driver was to pioneer the application of innovative teaching, learning and assessment strategies to alter the typical mix of face‐to‐face lectures and seminars linked to essays or group presentations and encourage more active co‐construction of knowledge over information transmission (Hutchings 2008). The student learning experience was facilitated over five weeks with one day contact and one day student managed guided learning each week. Students were allocated an online case study on a particular condition or situation relevant to their professional practice, research process information, podcasts, keynote lectures, and group work activities shared through group blogs. The development of new assessment strategies, including assessment of group blogs, and delivery of a multiple choice computer assisted assessment, has been discussed elsewhere (Hutchings et al 2013). 5.2.1 Student experiences Students had to read, prepare weekly blogs and work in groups to produce their group coursework summary. They recognised the student centred learning approach as different and some enjoyed the active and collaborative learning opportunities: It is the first time that we have really had to manage our own learning rather being 'fed' the information in a lecture. (Student 6) I enjoyed working in my group to produce the final blog. We worked well together and were able to bounce ideas off each other. (Student 7)

80

Maggie Hutchings, Anne Quinney and Kate Galvin The degree of scaffolding necessary to support student learning varied with different students. The challenges were viewed positively by some and they were able to learn progressively using the online resources and guidance: Having to read, understand and submit a blog weekly challenged me and was good for me to take in what I had learned and read and think about it. (Student 8) The case studies and podcasts have been a new way of learning for me and it has encouraged me to do work on a weekly basis. This is something I usually struggle to do, but knowing that a weekly piece of work needs to be submitted has aided my learning. (Student 9) Each week it became easier to understand what was required of us to do. It all came together like a jigsaw bit by bit. I felt at the end I had learnt a lot more than I had thought. (Student 10) Other students were more reliant on face‐to‐face contact with a tutor and peers: I would have preferred normal group seminars where we are being taught information and we can freely ask questions. (Student 11) The normal pattern of engagement in lectures and seminars appeared to be disrupted by this more independent student managed learning approach, which relied on student engagement and learning with the online materials and guidance provided. The tipping point in optimum disruption, the transition to a more independent learning approach, proved too much for some students. Not being told the information that was necessary to pass the unit, having lecturers believe that "it's on MyBU" or "listen to the podcast" is a suitable response or solution to a student’s enquiry. (Student 12) 5.2.2 Staff experiences Implications for academic staff were also manifested through the introduction of these different education strategies. Staff who assisted in the development of the web‐based case studies vocalised how it can change how they interact with students. One described how she felt distanced from the body of knowledge she had created and concerned the facilitators would do justice to her work: I feel slightly detached now which has been quite difficult. It’s like giving birth...! Well there you go and look after it and make sure that you get across what I want you to get across. (SFG)

5.3 Technology and logistical impacts The University and School’s strategic priorities included increasing usage of technology mediated learning to enable the student learning experience to be enhanced and provide opportunities for academic staff to engage more fully in learning technology enhancement through championing, developing and facilitating curricular initiatives. It was anticipated the introduction of the module would bring economies of scale in staff facilitation realized through the changing balance between face‐to‐face teaching and online learning. The large cohorts of students each year were facilitated in two blocks of 300 students using a blend of learning technologies which included online case study resources, group blogs, computer assisted assessment, online frequently asked questions and a fast feedback forum supplemented by face‐to‐face lectures, student drop‐in sessions, and group work. It also required use of a 300 capacity lecture theatre complex for student contact days, booking of computer labs for online assessment, technical support for facilitation of the ARS voting pads, and provision of a robust and secure online assessment platform for delivery of the online exam. 5.3.1 Student experiences Students recognised the technology mediated approaches adopted as distinct from previous learning experiences and they welcomed the flexibility they afforded: It was so different from any other module we had done before and was highly computer based. (Student 13) I enjoyed the self‐managed learning days as I was able to complete the required work in my own time and at my own pace. (Student 14) While some students struggled initially they managed the optimum disruption initiated by these approaches and their readiness for engaging with them improved:

81

Maggie Hutchings, Anne Quinney and Kate Galvin I found blogging very difficult as I’m not very brilliant on the computer but that in itself was a learning process! (Student 15) I think if I was asked to do blogs now I would feel more comfortable with them. (Student 16) 5.3.2 Staff experiences The introduction of technology mediated learning within the module affected the roles of academics as developers, champions, and facilitators. It demonstrated role transitions, from module teachers and research staff to resource developers, from uni‐professional programme leads to transprofessional champions, and from research‐focused professoriate to module facilitators. Academic staff identified how the technology could impact on their working practices and changing roles: It does radically change how I interact with the students …… the technology is starting to take us into new areas and there is an element of being de‐skilled and wondering how I am going to cope in this brave new world. (SFG)

6. Discussion We have shared our story and outcomes of working with the complexities of change at individual, professional and organisational levels and identified the connectivity and flux between these levers in securing effective change management. While individual narratives may have focused on the nature of the technology or the education strategies adopted, or the ways the module engaged with research, these findings demonstrate the complexity and intersections of factors at work in successfully managing a major curriculum innovation. They highlight the importance of deploying strategies for change management that can negotiate through the ‘wicked’ problems (Rittel & Webber 1973), not only logistical but also significantly cultural, and seemingly intractable, which underpin this initiative. Our analysis has revealed factors identified are as much cultural as logistical. For example, one student says: I would have preferred more lectures and less ‘computerised’ study as I don’t feel this aided my learning at all. (Student 17) How are we to interpret this comment? Is this about the use of technology mediated approaches per se or could it be more deeply embedded in the degree of disruption caused by the move away from the normality of educational strategies established in the first year of the programmes and focused on the familiar structure of lecture and seminars? Could the innovation, facilitated through technology enabled learning, have strayed too far from the established culture and personalisation enabled in small face‐to‐face groups within uni‐ professional programmes? I feel this unit has used far too much ICT. I agree it is important in our future disciplines, however, this unit has been completely impersonal. (Student 18) Studying in such a large group. It lost the personal touch. (Student 19) On the one hand, organisational level logistical problems in managing complexity, dealing with risks, and achieving integration could be presented as resolvable with careful planning: There’s a lot of quite complex background issues to get resolved and sorted to be able to deliver something that’s slick and successful because it requires pulling together an awful lot of different teams. (SFG) The core planning team acknowledged the need ‘to have confidence the technology works’ with ‘Plans and processes for systems failure and managing organisational pitfalls’. (SFG) But on the other hand, there was a lot of change impacting on stakeholders at individual and organisational levels. Some staff felt an ‘element of being deskilled’ with: So many different techniques and technologies for people who have maybe not engaged in it before. (SFG) Some students felt overwhelmed by the amount of disruption generated by this initiative: Having it on a different campus, was all out of our comfort zone, different lecturers, different style of learning, different online style of accessing information. (Student 20)

82

Maggie Hutchings, Anne Quinney and Kate Galvin

6.1 Strategies for effective change management Strategies for success are needed to manage ownership and transferability of the processes and present the key challenge in assuring sustainability of the initiative so that:‘It doesn’t sit outside, it sits within’… (SFG) the programmes, framework teams, School, and University. 6.1.1 1 Creating a shared vision through a holistic model of change The success of this initiative is based on a ‘triple helix’ model of change with three major and interconnected strands, informing, grounding and aligning the processes of change management.

Strand 1: Research orientation ‐ Embedding a lifeworld‐led theoretical perspective as a model of transprofessional learning

The theory of lifeworld led care and education, bringing art and science together, is underpinned and informed by research expertise in the School. Learners are connected to humanising evidence based on the head, heart and hand for guiding and developing professional practice for critical judgement and ethical sensitivity.

Strand 2: Education strategies ‐ Realising a social‐constructivist pedagogy for informing student‐centred collaborative learning

Student effort is rewarded through reading, imagining and integrating evidence, capitalising on the significance of others through innovative arts and humanities materials as well as traditional research evidence, peer group learning, and tutors. The key message is that research is embedded in practice and not a technical toolkit. Learning is assessed formatively by means of weekly individual blogs and summatively through group coursework blog summaries and an online multiple choice exam.

Strand 3: Technology affordances ‐ Harnessing the potential of a range of technologies to enhance student learning

The learning processes are mediated by a virtual learning environment with rich multimedia web‐based case studies and collaborative groupwork facilitated through blogs, online assessment and ARS. The technologies can also deliver cost‐effective solutions for managing large student numbers and releasing staff time. This model of change is dynamic, interactive, and integrative. It has enabled us to forge ahead with managing the complexities and uncertainties wrought by change, working with systemic challenges beyond our control but not beyond the powers of a cohesive and committed team to negotiate and influence. 6.1.2 Building a robust and dedicated core team for managing change We have described our approach to change management as ‘middle‐grounded’ to signify the benefits of actively promoting and building on open, flexible, morphing teams, grounded in a humanising philosophy and a shared vision and values for developing innovative pedagogical practices endorsed by our School (Hutchings et al 2011). The shared vision, commitment and complementary team roles helped manage the integration and risks associated with changing cultures, and negotiating institutional processes, technology infrastructure, and raised expectations. Team members drew on the experience, enthusiasm and commitment of colleagues to deal effectively with challenges, constraints and uncertainties associated with the development of this complex project. 6.1.3 Managing organisational challenges through partnership, listening and regular communication Harnessing technology for enhancing student learning highlighted organisational and individual challenges in managing the changes associated with the scale of technology infrastructure, support required, and emergent expectations for learning technology provision for all. Smith (2012) emphasises the considerations underpinning the diffusion of innovative learning and teaching practices, requiring senior management support, recognition of the time resources needed to change existing practices, supportive networks and institutional infrastructure. Organisational challenges included managing timetabling logistics, organising groupwork rooms and computer labs for online exams, and overcoming systems failures. Working in close collaboration with committed and responsive IT champions helped manage organizational resistance. Pennington (2003, p.5) recognised that: “Organisational politics are heightened and amplified during a change process as individuals and groups perceive shifts in power, authority, influence and territory. For this reason successful change requires not just technical competence from ‘managers’, but also sensitivity to political and

83

Maggie Hutchings, Anne Quinney and Kate Galvin human dimensions of organisational life.” The core team experienced cultural resistance communicated in staff scepticism and uncertainty expressed by professional programme colleagues. These challenges highlighted the importance of promoting ownership and transferability through developing creative and collaborative partnerships working in flexible and supportive multi‐disciplinary/professional teams where roles merge and coalesce. The team’s efforts to consider the pedagogic and structural challenges (Browne 2005) in an integrated way were evident in the collaborative team approach, with role transitions experienced by staff being not dissimilar to those identified by Anderson (2009). While it was important to recognize the behaviours, motives and beliefs of staff who may resist change (Outram 2004), the commitment to fostering an effective collaborative team, both within and across discipline areas, assisted in the process of achieving the strategic goals of the university and realizing the vision of the team designing and delivering this module. This approach was intended to avoid what Ward et al (2010 p.40) describe as situations where ‘IT‐driven decisions and project management principles overrode the pedagogical considerations and autonomy of academic decisions making processes’. 6.1.4 Capitalising on networking opportunities and forming alliances for horizon scanning Opportunities to network with and learn from colleagues with expertise in different disciplines and other HEIs facilitated through the UK HEA Enhancement Academy (Hutchings et al 2011) provided a powerful and influential resource to inform and support the project. Links established with the University of Oxford Medical Sciences Division proved invaluable for informing the computer assisted assessment. The contribution of a Leadership Foundation for Higher Education (LFHE) ‘critical friend’ was pivotal in providing focused advice and support and instrumental in ‘winning hearts and minds’. The generosity of these colleagues sharing their expertise was highly valued and brought added caché and gravitas to the initiative.

7. Conclusions We successfully introduced a generic structure and processes through the design and development of this module. As a result, we hope the path for future developments will be made easier for other enthusiasts to follow. In placing pedagogy, informed by a lifeworld‐led philosophy and supported by a range of technologies, at the centre of the rationale for change this collaborative and creative project challenged and moulded existing organisational and individual practices (Browne 2005). Our views resonate with those of Greener (2010) that a more detailed understanding is needed of beliefs and behaviours of students and staff and environments in which these operate when introducing and adopting technology enabled learning practices. This incorporates consideration of personal and institutional pedagogies, digital skills and self‐efficacy in technology usage. Achieving ‘optimum disruption’ (Hutchings et al 2010a) requires institutions and individuals to accept the normality of what Ashcraft and Trethewey (2004 p81) refer to as the ‘dualities, contradictions and paradoxes’ embedded in day to day practices. This can lead to practices that foster the innovation, creativity and change (Barge et al 2008) at the heart of our ‘triple helix’ model of change.

Acknowledgements Development and evaluation was supported by the HEA Discipline‐focused Learning Technology Enhancement Academy (Hutchings et al 2011) and project funding from JISC/SEDA Embedding Work‐with‐IT (Hutchings et al. 2010b).

References Argyris, C. & Schön, D. (1978) Organizational learning: a theory of action perspective. Reading, Massachusetts: Addison‐ Wesley. Anderson JK. (2009) The work‐role transition of expert clinician to novice academic educator. Journal of Nursing Education, 48(4) pp 203‐208. Ashcraft KL and Trethewey A. (2004) Developing tension: an agenda for applied research on the organisation of irrationality. Journal of Applied Communication Research, 32, pp 302‐332. Barge J K, Le M, Maddux K, Noabring R and Townsend B. (2008) Managing dualities in planned change initiatives. Journal of Applied Communication Research, 36 (4) pp 364‐390. Breen R. Lindsay R, Jenkins A and Smith P. (2001) The role of information and communication technologies in a university learning environment. Studies in Higher Education, 26 (1) pp 95‐114. Browne E. (2005) Structural and pedagogic change in further and higher education: a case study approach. Journal of Further and Higher Education, 29 (1) pp 49‐60. Dewey, J. (1933) How we think. Boston: D.C.Heath. Dewey, J. (1938) Experience and education. New York: Simon & Schuster.

84

Maggie Hutchings, Anne Quinney and Kate Galvin Galvin, K. and Todres, T. (2013) Caring and well‐being: a lifeworld approach. Abingdon: Routledge. Greener S. (2009) Talking online: reflecting on online communication tools. Campus‐Wide Information Systems, 26 (93) pp178‐190. th Greener, S. (2010a) Staff who say no to Technology Enhanced Learning. In: Ismail I (ed). Proceedings of the 5 International Conference on E‐Learning, Penang, Malaysia. pp 134‐139. Greener S. (2010b) Plasticity: the online learning environment’s potential to support varied learning styles and approaches. Campus‐Wide Information Systems. 27 (4) pp 254‐262. Hutchings, M. (2008) Quick‐fix learning: challenging the concept of learning for learning’s sake. In Rust, C. (ed.) Improving Student Learning: For What? Oxford Centre for Staff Development. pp 143‐57. Hutchings, M., Quinney, A. and Scammell, J. (2010a) The Utility of Disruptive Technologies in Interprofessional Education: Negotiating the Substance and Spaces of Blended Learning, In Bromage, A. et al. (Eds) Interprofessional eLearning and Collaborative Work: Practices and Technologies. Hershey, PA: IGI. pp 190‐203. Hutchings, M., Galvin, K., Todres, L., Quinney, A., Pulman, A., Atkins, P. and Gentle, P. (2010b) Transformational Change through Lifeworld‐led Multimedia VLE engagement, Embedding Work‐with‐IT Final Report. Bournemouth University and the Leadership Foundation for Higher Education for JISC/SEDA. Hutchings, M., Galvin, K., Pulman, A., Todres, L., Quinney, A., Clark, V. and Atkins, P. (2011) Framing Lifeworld‐led Evidence to Shape Practice: Facilitating a Collaborative Transprofessional Curriculum for Health and Social Work Disciplines. Final Report for Higher Education Academy Discipline‐focused Learning Technology Enhancement Academy. Bournemouth University. Hutchings, M., Quinney, A., Galvin, K. and Clark, V. (2013a) The yin/yang of innovative technology enhanced assessment for promoting student learning. In Greener, S. (ed.) Case studies in e‐learning research for researchers, teachers, and students. Reading: Academic Publishing International. pp 62‐79. Hutchings, M., Scammell, J. & Quinney, A. (2013b) Praxis and reflexivity for interprofessional education: towards an inclusive theoretical framework for learning. Journal of Interprofessional Care 27 (5) pp 358‐366. Lave, J. & Wenger, E. (1991) Situated learning: legitimate peripheral participation. Cambridge: Cambridge University Press. Lea M R and Jones S. (2011) Digital literacies in higher education: exploring textual and technological practice. Studies in Higher Education, 36 (94) pp 377‐393. Olssen, M. & Peters, M.A. (2005) Neoliberalism, higher education and the knowledge economy: from the free market to knowledge capitalism. Journal of Education Policy 20 (3) pp.315‐345. Outram S. (2004) 53 interesting ways in which colleagues resist change. Educational Developments, 5 (2) pp 1‐4. Pennington, G. (2003) Guidelines for promoting & facilitating change. Higher Education Academy: Learning and Teaching Support Network Generic Centre. Pulman, A.J., Galvin, K., Hutchings, M., Todres, L., Quinney, A., Ellis‐Hill, C. and Atkins, P. (2012) Empathy and Dignity through Technology: using Lifeworld‐led Multimedia to Enhance Learning about the Head, Heart and Hand. Electronic Journal of E‐Learning, 10 (3) pp.320‐330. Rittel, H.W.J. and Webber, M.M. (1973) Dilemmas in a general theory of planning. Policy Sciences 4 pp 155‐169. Shore, C. (2010) Beyond the multiversity: neoliberalism and the rise of the schizophrenic university: Special issue on anthropologies of university reform. Social Anthropology 18 (1) 15‐29. Smith K. (2012) Lessons learnt from literature on the diffusion of innovative learning and teaching practices in higher education. Innovations in Education and Teaching International, 49 (2) pp 173‐182. Thomas, R. Sargent, L.D. and Hardy, C. (2011). Managing organizational change: negotiating meaning and power‐resistance relations. Organization Science 22 (1) pp.22‐41. Ward M‐H, West S, Peat M and Atkinson S. (2010) Making it real: project managing strategic e‐learning development processes in a large, campus‐based university. Journal of Distance Education, 24 (1) pp 21‐42.

85

An Empirical Survey on the Adoption of e‐Learning in Corporate South Africa: Service Providers’ Perspective Eunice Ivala Cape Peninsula University of Technology, South Africa [email protected] Abstract: In the knowledge economy, workplaces need their employees to be lifelong learners and adaptable in order to cope with changes which arise. Corporate South Africa has invested in e‐learning in order to build capacity of employees and meet strategic priorities and programmes of action. This study investigates how corporate South Africa is adopting e‐ learning in training practices and factors affecting implementation of e‐learning. It hopes to make a practical contribution to training practices in corporate South Africa by pointing out factors that affect implementation of e‐learning in this environment. Insights and ideas which emerge will contribute to knowledge in this field and be valuable to companies looking into implementing e‐learning in their training practices. A qualitative research approach was employed to investigate adoption of e‐learning in corporate South Africa. Data were gathered using in‐depth interviews and document analysis. Participants were purposely selected because of their involvement in providing or facilitating e‐learning in corporate South Africa. Domestication theory was used as a theoretical and data analysis framework as it explains the process of technology [e‐learning] adoption from acquisition to the point when the technology is fully integrated into the life of an individual or institution to improve productivity. Interviews were tape‐recorded with the consent of participants, then transcribed verbatim, and data were analysed following a deductive approach using the constructs of domestication theory. Results show that e‐learning in corporate South Africa is in its infancy, and blended learning is used in training. Most companies outsource e‐learning services, and there are a number of challenges that impede implementation of e‐ learning in corporate training. Keywords: corporate South Africa, domestication theory, e‐learning, adoption, deductive approach

1. Introduction The global knowledge economy, rapid development of information and communication technologies (ICTs), increasing internationalization and globalization, as well as changes in occupational structures (content and organization) mean that not only educational institutions but also work organizations need to develop new ways of ensuring that the workforce is competent enough to meet these challenges (Tynjälä 2008). To ensure success organizations have to attract, engage, and retain talent, but also need to help personnel perform at the top of their game once hired (Harun 2002; Tynjälä 2008; Serrat 2010). Workplaces also need their employees to be lifelong learners and adaptable because knowledge is the most critical organizational resource; making sure workers have both the capability and maximum opportunities to release their potential is a key objective of strategic learning management (Serrate 2010). E‐learning has been identified as the enabler for people and organizations to keep up with changes in the global knowledge economy, and corporate South Africa has invested in e‐learning to build capacities of employees in order to meet organizational strategic priorities and programmes of action. Underpinned by domestication theory, this paper presents service providers’ perspectives on how corporate South Africa is adopting e‐learning in their training practices and the factors affecting implementation of e‐ learning. The study was guided by the following questions:

How is corporate South Africa employing e‐learning in its training practices?; and

What factors affect implementation of e‐learning in corporate training practices?

A qualitative research approach was utilized to investigate the adoption of e‐learning in corporate South Africa and factors that affect its implementation. Data were gathered using in‐depth interviews. Participants in the study were purposely selected because of their involvement in providing or facilitating e‐learning in corporate South Africa. Domestication theory was used as a theoretical and data analysis framework as it explains the process of technology[e‐learning] adoption from acquisition to the point when the technology gets fully integrated into the life of an individual or institution to improve productivity (Habib & Sønneland 2010).The interviews were tape‐recorded with the consent of participants, then transcribed verbatim. Data were analysed following a deductive approach using the constructs of domestication theory.

86

Eunice Ivala The study hopes to make a practical contribution to knowledge in this field, as research shows that there is a gap in the literature in South Africa as to which online tools and learning strategies are suitable for human resource development in corporate training, and very little research focusing on use of e‐learning in the corporate environment in this country (Dagada &Jakovljevic 2004).

2. Literature review 2.1 Learning in the workplace According to Tynjälä (2008), learning in the workplace occurs at different levels; learners may be individuals, groups, whole organizations, inter‐organizational networks or even geographical regions. The nature of learning also varies. Learning has been described by Anna Sfard (1998) in three metaphors: as the acquisition of knowledge and skills (which sees learning as a process of knowledge acquisition); as participation in communities of practice (which stresses that learning takes place by participating in the practices of social communities); and as knowledge creation (creation of new knowledge and learning being considered a social process aimed at developing new practices). While the knowledge acquisition perspective is that which has been mostly practised in formal education, the participation and knowledge creation metaphor better describes workplace learning. People learn at work by participating in various working practices, collaborating with colleagues and clients and meeting new challenges. Learning at work as well as school has both formal and informal aspects (Tynjälä 2008), although these are weighted differently. Workplace learning is not a single unified phenomenon, and has often been described using such labels as informal and implicit; both workplace and school learning can take different forms depending on individuals’ positions and many contextual factors. At least three basic modes of workplace learning can be distinguished: incidental and informal learning, which takes place as a side‐effect of work (Eraut 2004); intentional, but non‐formal learning activities related to work (mentoring, intentional practicing of certain skills or tool use, for instance); and formal on‐the‐job and off‐the‐job training). Slotte, Tynjälä and Hytönen (2004) acknowledge informal and formal learning as equally important elements of learning at work, but emphasize that they entail different processes and outcomes. While informal learning occurs as part of everyday work processes and activities and produces mainly implicit or tacit knowledge, formal learning takes place in the context of organized training and learning activities and is meant to generate explicit, formal knowledge and skills. Slotte, Tynjälä and Hytönen (2004) argue that informal learning alone is not enough. Firstly, because informal learning often takes place without conscious effort and yields mainly tacit knowledge, it may result in non‐desirable outcomes. Secondly, in working life today new knowledge is produced at such a rapid rate that informal learning alone cannot ensure that the knowledge and skills of organizations and people will keep pace. Thirdly, formal education and planned learning situations make it possible to exploit informal learning effectively, turning tacit knowledge into explicit knowledge and integrating conceptual knowledge and practical experience, which is the foundation for development of expertise (Slotte, Tynjälä and Hytönen 2004; Tynjälä et al. 2006). For these reasons, Slotte, Tynjälä and Hytönen (2004) suggests that the different modes of workplace learning should be combined so that formal training utilizes informal learning. This has also importance for e‐learning solutions aimed at promoting organizational development (Tynjälä 2008). In organizations classroom training is no longer the most efficient training method, as personnel must be brought out of their offices at a high direct and indirect cost. The pace of change at the workplace and the constraints of having employees leave the workplace for training for prolonged periods have resulted in greater emphasis on reducing cycle time for learning and implementation of knowledge, giving rise to the emergence of “just‐in‐time” rather than “just‐in‐case” learning (Harun 2002).Organizations require training solutions that are less costly than standard classroom training, and many look towards e‐learning as an alternative training solution (Moolman & Blignaut 2008). Sambrook (2006) summarized factors influencing work‐related learning by categorizing them as organizational factors, functional factors and individual factors. Organizational factors pertain to the organizational culture, senior management support, organization of work, work pressures, tasks, and task vs learning orientation. Functional factors are related to how the role of human resources development is defined and general characteristics of the organization such as number of staff, expertise, amount of information, and use of ICTs.

87

Eunice Ivala One of the most important functional factors is the need to achieve a pace of new learning, knowledge acquisition and knowledge application that is both timely and relevant to the strategic purposes of the business (Dealtry 2002). This has to do with how organizational learning strategy needs are incorporated into the business strategy. Individual factors include managers’ and employees’ responsibility for learning, motivation to learn, time, IT skills and confidence. How these different factors are organized influences the potential for learning in an organization. Also, there are many barriers which delay or hinder people from learning. These include lack of motivation, work deadlines always taking precedence; lack of time and space for in‐house training; too few employees to cover for off‐the‐job learning; low levels of management input into development of a learning culture; lack of incentives and employer support; inappropriate timing and inaccessible courses; and geographical barriers (Harun 2002; Stephenson 2003). When it comes to use of new technology, the initial cost of networks and terminals can be a problem. In South Africa the Employment Equity Act 55 advocates for continuous training and development of employees to expand the skills base of the workforce. The Skills Development Levies Act 9 of 1999 requires employers to pay a 1% levy for training and development of their employees, and the ICT Empowerment Charter pleads for an “entrepreneurial focus” to develop a society that can adapt to modern demands and create jobs and opportunities (ICT Charter 2004).Despite these legislations, South African organizations trail in employee development compared to other developing countries, as only 10% of organizations develop their employees effectively (Moolman & Blignaut 2008). Most learning in organizations in South Africa, as elsewhere, includes simple steps such as observing and learning from peers or superiors, on‐the‐job training, and formal learning resulting in certification. To build capacities of employees to meet organizational strategic priorities and programmes of action, corporate South Africa has invested in e‐learning, which has been identified as the enabler for people and organizations to keep up with changes in the global knowledge economy.

2.2 E‐learning in the workplace Constant changes in all work environments brought about by the knowledge economy and rapid changes in technology mean there is a constant need to rapidly train and retrain people in new technologies, products and services found within the environment (Harun 2002). E‐learning is one way of training which can assist organizations in achieving the above goal and securing overall competitive advantage (Chen & Hsiang 2007). E‐ learning can raise individuals’ employability and corporate effectiveness by increasing their skills and knowledge base, and it can improve just‐in‐time training and employees’ control over their learning. E‐learning is taken to mean all forms of electronically supported teaching and learning (Serrat 2010). Industry training globally has been more ready to adopt the so‐called ‘blended learning’ approaches, where online and use of ICTs is combined with face‐to‐face learning (Bielawski and Metcalf 2003), perhaps reflecting the diversity of work‐based skills (De Freitas 2007).E‐learning is applicable across areas of workforce training including career development training; new employees orientation; new service or product information; or just updating and upgrading work knowledge, competencies and skills (Harun 2002).Analysis of learning experiences suggests that online programmes in the workplace are likely to be successful when: they are flexible in pace and duration; sensitive to participants’ unexpected pressures on time and resources; and fully explained in advance in terms of practical requirements, mode of study and expected outcomes; when individual employees manage their own work and learning priorities; when they learn through the process of work itself, supported by proactive, attentive and sensitive online supervisors; when the courses are open to ‘successful withdrawal’ when judged appropriate by participants; negotiated between the supplier and the client ‐ personalized to customer circumstances; and when they improve employees’ skills and personal development and are supported in‐house through normal working relationships (Stephenson 2003). E‐learning facilitated by the Internet expands the realm of how, where, and when learners can engage (just‐in‐ time learning) (Chen & Hsiang 2007); is a cheaper and more effective (just‐in‐time) way of providing private and public sector organizations with everyday learning opportunities they need to improve organizational outcomes. E‐learning is also convenient, allowing people to work at their own pace, with fast access to information; and allows learning to be tailored to an individual’s need. Upfront costs may seem exorbitant, but in the long run huge savings are made as access to training courses and materials only incur a fraction of the traditional classroom training costs; e‐learning eliminates travel costs and time away from the workplace and

88

Eunice Ivala job, and offers just enough learning ‐ employees can find and use exactly the piece of instruction or information support they need at the moment for the task at hand. E‐learning is not without challenges to implementation and continuing use: the cost of developing (or purchasing) software applications at the onset, compounded by learning costs once e‐learning interventions are running (Harun 2002); lack of social presence usually associated with a physical classroom, as learners miss real‐life interactions with their colleagues and instructor as e‐learning is self‐directed; perceived lack of time to devote to workplace learning and to formulate and maintain e‐learning solutions; and content issues, as quality content is not available or is not suited or relevant and must therefore be developed (Serrat 2010; Unwin 2008).

3. Domestication theory Domestication theory has its roots in studies of the sociology of technology, which used it to describe and analyse the processes of technology’s acceptance, rejection and use (Haddon 2006; Silverstone & Haddon 1996).It was developed from the social shaping of technology perspective, where use is perceived as having a dominant role in defining the nature, scope and functions of technology (Mackenzie & Wajcman 1999).Thus, domestication of technology refers to the process whereby users bring an artifact from the public realm to the private and tame, gain control, shape or ascribe meaning to the artifact in their lives (Silverstone & Haddon 1996). The concept of domestication was originally adapted from other disciplines such as anthropology and consumption studies, as well as from media studies, and used to investigate the context in which ICTs were experienced by the people using them (Haddon 2006). According to Haddon (2006) the framework looks beyond the adoption, use and benefits of ICTs to ask what the technologies and services mean to people, how they experience them, and the roles that they can come to play in their lives. Domestication theory has three dimensions: commodification, appropriation and conversion (Frissen 2000), considered as analytical tools for facilitating making sense of the complex interrelationships of cultures and technologies. The dimensions could be used as phases through which domestication of technology [e‐learning] is manifested in the individual or institution:

Commodification is the technology acquisition stage. The potential user (individual or institution) acquires technology and works towards integrating it into everyday life, adapting it to daily practices. The product is evaluated on how well it would fulfill the consumers’ perceived needs (Habib 2004). Once the technology is bought, it goes through a process of appropriation.

Appropriation is the actual integration of the technological artifact into the users’ lives and routines (Silverstone, Hirsch & Morley 1992), which involves all transactions in the passage of artifacts from the market to users’ lives, and motives for approaching the product. Habib and Sønneland (2010) divide the appropriation dimension into two stages:

Objectification: This is the physical integration of the artifact into the users’ environment, expressed in usage but also in physical dispositions of object in the spatial environment (Silverstone et al. 1992). The user decides what role the technology should play in his/her life and where it is placed and displayed. Objectification does not necessarily mean the product is accepted by the potential adopters; Haddon (2006) argues that sometimes technologies do not comply with users’ intentions, and do not fit into the routines of their everyday lives.

Incorporation: This is “integration of new activities and social practices with and around the new artifact into the temporal dimension of life” (Habib & Sønneland 2010; Silverstone et al. 1992). For this to happen the technology has to be actively used by the intended user, giving the tool a place in the individual’s everyday life.

Conversion is the process by which the product reaches ‘taken for granted’ status, becoming part of the user’s life (Haddon 2006). Technologies are bought for certain features, but may become functional in ways somewhat different to the intentions of designers or marketers. Some functions may change or disappear (Lee, Smith‐Jackson & Kwon 2009). This dimension is relevant to unintended uses of technology, adaptations made by users, or features that users may desire in future.

Although domestication theory was originally used to understand the uptake of ICTs (Silverstone & Haddon 1996; Silverstone & Hirsch 1992), a decade later the theory was found useful in studies of technologies in the

89

Eunice Ivala workplace (Pierson, 2006) and of wireless technology in education at university level (Vuojärvi, Isomäki & Hynes 2010). It is vital to note that the first two dimensions of domestication are equivalent to what is normally referred to as ‘adoption’ in most adoption frameworks (Pedersen & Ling 2003). It is noted that the domestication framework allows for investigating the processes beyond acquisition of the technology. The three dimensions of domestication theory will inform this study, because it is felt that it can explain the process of e‐learning adoption by corporate South Africa, from acquisition to the point where e‐learning is fully integrated into the life of an individual or institution to improve productivity. Hence, in this paper the three dimensions of the domestication framework were used as the lens to understand:

How corporate South Africa is employing e‐learning in their training practices; and

Factors affecting implementation of e‐learning in corporate South Africa’s training practices.

4. Methodology 4.1 Research approach A qualitative approach was used to understand how corporate South Africa was adopting e‐learning in their training practices and factors that affected implementation of e‐learning; this was thought to be useful because of its strength in investigating experiences as they are ‘lived’, ‘felt’ or ‘undergone’ by participants (Sherman & Webb 1988).

4.2 Context and participants Participants in this study were two corporate e‐learning service providers and an e‐learning trainer from a pharmaceutical distributor. The service providers provided e‐learning hosting and training services to the banking, telecommunications, transport and mining sectors. The e‐learning service providers were from Delloite and Touche and Epi Use, while the trainer was from UTiPharma. Purposive sampling was used (Neuman 1997), and participants were chosen because they had rich information gained through practice (Patton 1990), and were thought likely to reflect on the complexity of adoption of e‐learning in corporate South Africa’s training practices and factors affecting implementation.

4.3 Data collection method Data gathering methods included in‐depth interviews with the two service providers and the trainer, analysis of conference presentations, legislation and policy documents (document analysis).

4.4 Data analysis Data were analysed using the three dimensions of domestication theory (commodification, appropriation and conversion), and done deductively where relevant data from interviews and document analysis were mapped to the dimensions. The researcher does not claim that findings of this study are generalizable, but that they offer valuable insights to companies looking to implement e‐learning in their training practices. Consent to participate in the study was sought and the purpose of the study was explained to the participants; anonymity and confidentiality were adhered to.

5. Findings and discussion The findings and discussion are presented under the following themes derived from the three dimensions of domestication theory: Acquisition/adoption of e‐learning; Factors affecting appropriation of e‐learning; and Conversion of e‐learning.

5.1 Acquisition/adoption of e‐learning The results show that corporate South Africa is slowly adopting e‐learning and that e‐learning is in its infancy. All three participants echoed the following statement:

90

Eunice Ivala Because we are an outsourced partner we are not in a company, I haven’t seen that high implementation of e‐learning as in the higher education sector. It is not at that level … the adoption is quite slow, and I think it will still be [in future]. Adoption of e‐learning is said to vary from company to company, with some early adopters and many not eager to invest in e‐learning. Blended learning (Dagada & Jakovljevic 2004; Serrat 2010) is used in corporate training practices, as the following quote highlights: We are still doing that [face‐to‐face training], it does not fall away …our business processes, whenever there are changes and you need to intervene quickly and you need to get the people up‐skilled on a certain level, basic classroom is still an offering. (Trainer) Most companies in corporate South Africa do not acquire the technological platforms used for offering e‐ learning, but outsource e‐learning services (which includes the platforms and content developers), with some leaving the whole process of e‐learning to service providers and others facilitating e‐learning once systems and materials have been developed by the outsourced service provider: … We are a company that creates e‐learning objects for other companies. So we don’t manage it for the company, we develop material for other corporate sectors. We have done work … in the banking sector and, for example, for Telkom, Eskom – those kind of companies we’ve developed training material for. They manage it but we developed it. (Provider 1). … there is increased appetite for online learning where the whole thing is hosted by an external party. We produce the content and train their trainers or facilitators. They then manage their own training, but we typically have several level agreements with them in terms of which we would not only make sure that the programmes technically keep running, but that we can also update the material as and when required … (Provider 2) Companies prefer outsourcing e‐learning services as it is cost‐effective: they do not have to incur the day‐to‐ day running costs of the e‐learning platforms. Some providers work with section managers from the company they are designing materials for, to ensure relevance to their needs. Other companies leave the service provider to design the learning materials: We developed it in conjunction with the section managers from the company. … because we are not the experts in terms of mining we would sit with that expert and take their content and work that into an e‐format and also verify that … to see that the whole training material is still applicable. (Provider 2) The trainer from UTiPharma indicated that the company outsourced the e‐learning platform and bought existing e‐learning materials, and that facilitation of e‐learning and production of job‐related materials was done in‐house. In corporate South Africa, e‐learning is adopted to support the training needs of the company, in accordance with strategic plans and human resource development policy. E‐learning is seen as a training option which minimizes removal of staff from their jobs to attend training: … back in 2002 when I developed the course I realized that I would have to have these guys in the classroom for basically 5 full days to cover everything … I was told by the section manager but that’s not really possible … And then I realized but we should start thinking in terms of e‐Learning … that’s basically one of the reasons one started doing that. (Trainer). The technological tools used for delivery of e‐learning are learning objects and systems linked to databases which are developed by the service providers. These tools are used to push content to the learners with very little interaction: … e‐Learning in the corporate sector its very much learning objects. They create a learning object which is placed on a system and there’s very little communication. The social interaction is generally very limited in this kind of e‐Learning implementation. (Provider 1) … so the learning within communities is not yet as established as one would like … (Provider 2) It could be argued that the tools serve the role of managing content. Commodification of e‐learning was achieved through the consultative process with management when negotiating to outsource e‐learning services, and through the service providers’ and trainers’ introduction of e‐learning in the corporate training practices.

91

Eunice Ivala

5.2 Factors affecting appropriation of e‐learning Because most companies outsourced the e‐learning services, employees are expected to take it up as part of their training options. According to the participants, this is happening at a slow pace, and adoption of e‐ learning is enhanced if employees possess technological skills and have access to technology. Offering e‐ learning content related to employees’ jobs and linking it to their career development was also reported to encourage employees to take up e‐learning. … I slowly introduced them to more related training …to update, just to enhance their computer skills … and we added that to their basic job profiles or their career advancement … if an employee rarely gets an opportunity to work on Microsoft Word, but if they wanted to grow within their career and let’s say one of their aspirations was to work towards an administrative position, so for them it’s an advance in their career and they will feel that they have progressed somewhere. (Trainer) According to Geisman (2001), employees’ knowledge and skills in IT affect their ability to engage in e‐learning. The process of introducing e‐learning in companies should start with building the technological skills of the employees, and introducing the concept of e‐learning carefully, in a non‐threatening way. The integration of technology [e‐learning] as an empowering agent into the work environment must value the culture of the workforce, workers’ anxieties, intrinsic fears, hopes, frustration and training preferences (Moolman & Blignaut 2008).This would help in changing the culture of learning and getting buy‐in from employees. Giving rewards (certificates) and praising employees on their achievement also increased their acceptance of e‐learning: …giving them certificates when they finish and really praising them and commenting on how well they did … really pushed up the entire movement towards this e‐learning. At one stage round about in 2005/6 I had more than 600 online learners across the country in a company where we have only 900 people. So it really took off big time and that basically gave me the gap to start developing business‐related computer‐based training … and added that also onto the learning management system for them to do. (Trainer) Other factors given for successful implementation of e‐learning in corporate South Africa were regular communication with the learners, and the company’s commitment to training and management support: ... there should be a commitment to training ... there needs to be a very high level of commitment from top management to design e‐learning” (Provider 1) However, in most of corporate South Africa implementation of e‐learning is met with a number of challenges. Some employees resist e‐learning because they believe that learning happens in the classroom, and they do not see the value of e‐learning: … the first challenge we obviously had was the buy‐in from some people … many people believe that only if they see somebody sitting in a classroom then that person has learnt. (Trainer) Furthermore, learning or training is seen as a ‘day out of the office’; the culture of e‐learning is not there and its value to the employees is not known: …people in the workplace see learning or training as a day out of the office. They like going out and to sit and doing it in their own time is just not an option. … and people just don’t see the worth of it [e‐learning] (Provider 1) On the employers’ side, some companies do not see the value of e‐learning, and as a result are reluctant to allow employees to attend training. Others are focused on what they pay for e‐learning services instead of the quality of the e‐learning services they outsource, as one provider indicated: … the fact that business is not always eager to allow people the time to learn, and they would not free their people up… you’ve got 20 spaces in your computer lab and you’ve got 20 guys booked. Now business develops an emergency and he does not send his people. So now you’ve got only 10 people instead of 20… So it is getting business to see the value of the e‐learning that sometimes results in all kinds of practical hindrances to get people there. (Provider 2) This lack of awareness of the benefits of e‐learning acts as a barrier to corporate e‐learning, as the perceived usefulness of e‐learning is questioned, reducing the intention to use it (Davis 1989). The researcher suggests that making employees aware of what e‐learning is and its usefulness can go a long way to improve the uptake of e‐learning.

92

Eunice Ivala Another factor that hinders adoption of e‐learning in corporate South Africa, according to the participants in this study, is the delivery of instruction – where tools are used to push content without any interaction/communication: The social interaction is generally very limited … and I think that’s also a factor that hinders the implementation. (Provider 1) Employees resist e‐learning partly because they are used to interaction in their face‐to‐ face training. Lack of technological skills was also voiced as a factor which affects the adoption of e‐learning in corporate South Africa, since e‐learning is highly dependent on the workers’ ability to interact with computers, the Internet and multimedia.

5.3 Conversion of e‐learning Findings show that in corporate South Africa domestication of e‐learning is in its early stages, and that the e‐ learning platforms are predominantly used for pushing content to employees. However, the service providers suggest that in future they would like to see e‐learning platforms being used for more than just providing content (i.e. to include interaction). Some employees continue to resist e‐learning due to lack of awareness of the value of e‐learning and their love for face‐to‐face learning, even though blended learning is used. Of note is the UTiPharma trainer’s comment that employees who have undergone e‐learning have been promoted to higher positions, and that their attitudes to work and to bettering themselves have improved: …some of the students that have completed have already been moved from the warehouse to other positions because of the courses that they have done. … But for people that have done that [e‐learning],… attitudes with regard to the work and the drive to better themselves is one of the things that comes to the fore in many interviews. I know that many of the people who were in the warehouse have been promoted to administrative positions and …some even to managerial level. (Trainer)

6. Conclusion The findings of this study showed that domestication theory can be a useful theoretical and analytical framework for describing and understanding the adoption of e‐learning in corporate South Africa and the factors that affect its implementation. Although these findings are gathered from a limited number of providers and trainers, the lessons learnt could be useful in other organizations and contexts in which online learning is being used for human resource development. Findings show that e‐learning in corporate South Africa is in its early stages. Some of the enabling factors for implementation of e‐learning are presence of IT skills and access to technology; offering career‐related e‐ learning courses that are linked to employees’ career advancement; companies’ commitment to training; and top management support. Impeding factors include lack of awareness of the benefits of e‐learning by some of the employers and employees, and resistance to e‐learning.

References Bielawski, L. and Metcalf, D. (2003). “Blended learning: integrated knowledge, performance support and online learning,” Amherst, MA: HRD press. Chen, R. and Hsiang, C. (2007).“A study on the critical success factors for corporations embarking on knowledge community‐based e‐learning”, Information Sciences, Vol. 77, 570‐586. Dagada, R. and Jakovljevic, M. (2004). “Where have you all the trainers Gone? E‐learning strategies and tools in the corporate training environment”, Proceedings of SAICSIT, pp.194‐203. Davis, F.D. (1989). “Perceived usefulness, perceived ease of use, and user acceptance of information technology”, MIS Quarterly, September: 318‐340. Dealtry, R. (2002). “Managing the issue of learning relevance in the formulation of learning strategies”, Journal of Workplace Learning, 14(5), 205‐214. Frissen, V.A.J. (2000). “ICTs in the rush hour of life”, The Information Society, Vol. 16, 65‐75. Habib, L. (2004).Domesticating learning technology in a higher education institutions: A state of two virtual learning environments., Oslo: Oslo University College Centre for Educational Research and Development. Habib, L. and Sønneland, A. (2010). “From Alien to Domestic? Virtual learning environment use from a domestication perspective”, MERLOT Journal of Online Learning and Teaching, Vol. 6, No. 2, page numbers?.

93

Eunice Ivala Haddon, L. (2006). “The contribution of domestication research to in‐home computing and media consumption”, The Information Society, Vol. 22, 195‐203. Harun, M.H. (2002). “Integrating e‐Learning into the workplace”, Internet and Higher Education, Vol.4, 301‐310. ICT Charter (2005). “Empowerment Charter”, Final version May, [Online] http://www.ictcharter.org.za/content/ICTbeecharter04may2005‐Minister.pdf. Accessed 15 January 2014. Lee, Y. S., Smith‐Jackson, T. L., and Kwon, G. H. (2009). “Domestication of technology theory: conceptual framework of user experience”, [Online] http://goodgestreet.com/CHI09/submissions/YS_Lee.pdf. Accessed 14 January 2014. Mackenzie, D. and Wajcman, J. (1999). The social shaping of technology. Buckingham, UK: Open University Press. Moolman, H. B. and Blignaut, S. (2008). “Get set! E‐Ready, …e‐Learn! The e‐Readiness of warehouse workers”, Educational Technology and Society, Vol. 11, No. 1, 168‐182. Neuman, W. L. (1997). Social research methods: qualitative and quantitative approaches. Needham Heights, MA: Allyn and Bacon. Pedersen, P. and Ling, R. (2003). “Modifying adoption research for mobile internet service adoption: Cross disciplinary interactions”, Proceedings of the 36th Hawaii International Conference on System Science Patton, M. Q. (1990). Qualitative evaluation and research methods, 2rd edn, Newbury Park, California: Sage. Pierson, J. (2006). “Domestication at work in small businesses”, in: T. Berker, M. Hartmann, Y. Punie and K.J. Ward (Eds.), Domestication of media and technology (pp.205‐226). Maidenhead, UK: Open University Press. Sambrook, S. (2006). “Factors influencing the context and process of work‐related learning: synthesizing findings from two research projects”, Human Resource Development International, Vol. 8, No. 1, 101‐16. Serrat, O. (2010). “E‐learning and the workplace”, Knowledge Solutions. Philippines: Asian Development Bank. Sfard, A. (1998). “On two metaphors for learning and dangers of choosing just one”, Educational Researcher, 27 (2), 4‐13. Sherman, R.R. and Webb, R.B. (1998). Qualitative research in education. UK, Taylor and Francis. Silverstone, R. and Haddon, L. (1996). “Design and the domestication of information and communication technologies: Technical change and everyday life”, in: R. Silverstone & R. Mansell (Eds.), Communication by design: The politics of information and communication technologies (pp.44‐74). Oxford University Press. Silverstone, R. and Hirsch, E. (1992). “Introduction”, in: R. Silverstone, & E. Hirsch (Eds.), Consuming technologies: Media and information in domestic spaces (pp. 1‐15). London: Routledge. Silverstone, R., Hirsch, E., & Morley, D. (1992). “Information and communication technologies and the moral economy of the household”, in: R. Silverstone & E. Hirsch (Eds.), Consuming technologies; Media and information in domestic spaces London, UK: Routledge. Slotte, V., Tynjalä, P., & Hytõnen, T. (2004). How do HRD practitioners describe learning at work? ¨ Human Resource Development International, 7(4), 481‐499. Stephenson, J. (2003). A review of Research and practice in e‐learning in the work‐place and proposal for its effective use. Paper for ERA, [Online] http://www.iclml.com/ . Accessed 20 January 2014. Tynjälä, P. (2008). “Perspectives into learning at the workplace”, Educational Research Review, Vol. 3, Tynjälä, P., Slotte, V., Nieminen, J., Lonka, K., & Olkinuora, E. (2006). “From university to working life: Graduates’ workplace skills in practice”, in: P. Tynjaläa, J. Välimaa, & G. Boulton‐Lewis (Eds.), Higher education and working life: Collaborations, confrontations and challenges (pp.73‐88). Amsterdam: Elsevier. Unwin, T. (2008). “Survey of e‐Learning in Africa: based on a questionnaire survey of people on the e‐Learning Africa Database in 2007”, Twinning networks and university networks/UNESCO chairs, Royal Holloway University of London. Vuojärvi, H., Isomäki, H. and Hynes, D. (2010).“Domestication of a laptop on a wireless university campus: A case study”, Australasian Journal of Educational Technology, Vol. 26, No. 2, 250‐267.

94

An Approach in the Design of Virtual Environment for e‐Learning Based on Usability Studies Borka Jerman Blažič1, 2, Tanja Arh2 and Andrej Jerman Blažič2 1 University of Ljubljana, Faculty of Economics, Ljubljana, Slovenia 2 Jožef Stefan Institute, Ljubljana, Slovenia [email protected]‐lj.si [email protected] [email protected] Abstract: Usability evaluation is a core component of user‐centred design (UCD) approach and aims primarily to evaluate effectiveness, efficiency and satisfaction when users interact with a product/service in order to influence their decision for adoption and use (i.e., user acceptance). The focus of this research work was to assess the usability of Virtual Environment (VE) presented by the Virtual Conference Centre that was incorporated in the eCAMPUS® learning management system (LMS) with traditional usability method (usability testing) in order to get good assessment of its applicability for collaboration and friendly use. The study was a part of the GLOBAL project from FP7 with an aim of building an e‐ infrastructure for the world‐wide research community that enable easy and fruitful collaboration on distance. The overall results of the study were positive and promising for the developed platform designed to build up a social network of collaborating e‐learning community. Keywords: usability evaluation, task scenarios, e‐learning, collaborative environment, virtual environment (VE), virtual conference centre (VCC)

1. Introduction During the last years, web sites like Google, Flickr, Youtube, LinkedIn, Facebook, Myspace, Skype and many more have developed extremely successful mass services based on the paradigm known as Web 2.0 and the emerging successors. Web 2.0 was more an evolution in service design than a revolution. It proposed the use of a set of basic principles composed in a platform that position the user in the centre (rather than the technology). It enabled extensive development of the e‐learning virtual environments. These are based on simple user interfaces easy for use, on aggregate knowledge and wisdom of crowds by exploitation of so called folksonomies approach, then the social software and other advanced tools applications and on an extensive use of the URIs indicators, P2P communication protocols and some applications from the Grid networks. On the other hand, the use of videoconferencing and similar collaboration tools in the Internet was slowly taking up due to the non‐availability of more bandwidth all over the world and to the non‐existence of fully integrated and usable tools for collaboration. These tools usually did not inter‐worked in most cases and were used mainly for realising relatively simple tasks like working meetings, substituting the popular conference calls, for connect remote speakers on co‐located educational workshops, conferences or meetings, etc. The integration of videoconferencing and collaborative educational tools in advanced user friendly platform is a path that is expected to lead to more sophisticated collaborative and educational e‐infrastructure. One of th them was developed within the GLOBAL project from EU 7 Framework programme (GLOBAL project 2012) and was a subject of the usability study described in the paper. The study enabled re‐design of the platform to meet to the educational and research needs of several communities spread up all over the world. The growth use of collaboration tools being part of the Virtual Environment is heralding also a shift in the nature of Human Computer Interaction (Preece et al. 1994; Lewis 1995). Use of media technologies such as videoconferencing systems is becoming a part of everyday communication and entertainment amongst individuals. However, they are not default part of the collaborating and e‐learning environment. In that context, the user satisfaction with such technologies related to distance and collaborative applications are an integral part of the area of ICT usability studies, which are defined as the extent to which people can use the product quickly and easily to accomplish their tasks (Dumas & Redish 1999). Usability testing is performed to make sure that users can find and work with the functions of the product to meet their needs. One of the most important outcomes of usability test is a list of problems, which entail changes and thus improvement of the product. To ensure the reliability of the outcome of the usability study, several points need to be considered: representativeness of test participants for real users of the product, realness of testing tasks as compared to actual tasks performed by real users, accuracy of observations of test participants’ behaviours, and sensitivity

95

Borka Jerman Blažič, Tanja Arh and Andrej Jerman Blažič of measuring tools (Brinck et al. 2002). These points need to be addressed in the most of the usability study in the design of an ICT product. The current research publications in this filed has pointed to numerous usability requirements for collaborative environments (Patrick 2006) and are suggesting to be used in practice. However, this is still recognized as “severe usability problems that are present and common and that should be properly approached”. Usability evaluation of videoconferencing and collaboration tools is traditionally conducted by means of task performance measures and subjective measures such as questionnaires, interviews, etc. These traditional usability testing methods provide evaluators with all the information needed for an in‐depth usability research analysis of the system. This paper presents and discusses the results obtained from a usability evaluation of the platform known as Virtual Conference Centre (VCC) that employed a traditional usability method (Dumas & Redish 1999). Usability Tests (UT) were conducted on the (VCC) interface with real users being part of the e‐learning communities that were spread up in Europe, Africa and Latin America. The results of the usability studies contributed to the re‐design of the platform and enabled much more usable service to be provided. The paper is focus on the usability studies of the VCC which was later incorporated in the eCAMPUS® learning management system to provide real eLearning Virtual environment.

2. Background and product description The Virtual Conference Centre (VCC) was designed following the Model‐View‐Controller Model together with agile software design methods that allows the logic and the models of the various functionalities to be developed independently of the designers views. The software design frameworks Ruby on Rails and RESTful were chosen, as they are the most efficient rapid prototyping tools for website design.

Figure 1: Event space in the VCC The VCC was designed based on the following main features:

A single unified point of access to the virtual auditorium features enabling access to the different functionalities through the available tabs such as: Home, Events, Posts, People, and Spaces. It also contains a specific button for direct access to the GLOBAL project (www.global‐project.eu), another for description of the project partners, a Login and a Registration button.

Spaces are means that enable the organisation of different learning activities or projects. Each space has a repository with public and private documentation and events such as workshops classes or similar. The space’s public area can be customised to be the public face of any educational event or project meeting.

In the spirit of openness, during the design any user was allowed to register to the VCC. The registration procedure includes spam protection. The registration allows the user to access publicly available spaces, documents, event announcements, etc. More importantly, it allows granting access to areas of the VCC that are marked private by a particular group of people e.g. selected classes or courses.

A profile display for every registered user such that he/she can update the information stored, such as password, address, email contact, project interests, etc. Profiles will facilitate user networking and partnership building.

The People tab displays the registered users for quickly finding specific contacts and can only be seen by registered users.

96


The Spaces tab shows the available VCC spaces. Each Space has a space administrator. The space administrator can add users to or delete them from a specific space, allow an “open to all” registration policy, creates user groups under that space, and even deletes the complete space.

The Events tab is the event scheduler and the main part of the VCC. It allows the registered user to create events and organise event documentation e.g. learning material in a clear accessible manner assigning access privileges to each item. Any VCC visitor, even unregistered user, is allowed to see the event calendar with all the public events, incentivising new registrations of the community members. Only registered users can access private events of the spaces.

The Posts tab gives access to the repository of shared documents and a blog‐like or forum‐like upload system based on posts to fill it. These posts may have multiple attachments.

Virtual Conference Centre has been developed by the GLOBAL project by reusing results and developments from other projects which have transformed the ISABEL (GLOBAL project 2012) application into a cloud‐ computing service.

3. The evaluation methodology The standard user test procedures were adopted (Nielsen 1996) for the usability study. Jožef Stefan Institute (JSI) had the role of General Usability Testing (UT) Coordinator with responsibility to coordinate the overall study of the UT, the compilation and analysis of the data, and the production of the usability report that was passed to the development team at UPM (The Universidad Politécnica de Madrid). The key role of Local UT Coordinator was to coordinate the conduction of the usability tests in his or her site and to ensure the collection of the required data that were later delivered to the General UT Coordinator. To ensure the uniformity of the testing procedures and thus the validity and reliability of the testing results, a document entitled “GLOBAL Usability Test – Questionnaires & Task Scenarios” describing in detail the goals, instruments, participant requirements, procedures, data handling, etc. was produced. Local UT Coordinators were supposed to follow the Guidelines with minimum deviations. The language of communication between General UT Coordinator, Local UT Coordinators and Local Testers was written English. Hence, it was expected that all of them possess certain level of English proficiency. In addition to the classical UT methods based on tasks and scenarios the user satisfaction additionally evaluated by the System Usability Scale (SUS) method which is a simple ten‐item scale evaluation that gives global view of subjective assessments of the usability and the user satisfaction. The SUS results provide an implicit evidence of reliability and suggest comparativeness and complementarity to the subject summative usability studies. It is a Likert scale where a statement is made and the respondent indicates the agree of agreement or disagreement with the statement on a 5 point scale with the left anchor indicating the lowest level of satisfaction and the right anchor indicating the highest. Additional testing of the effectiveness of use was done after any of the educational events that were run over the three continents with the community that took part either as an event participant or as a partner of the project. In total 23 events were organized and 215 responders provided their feedback by describing their experience with the VCC as participants of the event. The events were taking part over the network among the three communities coming from the three continents with just one central location provided by the event organizer.

4. Participants 4.1 General characterization of the real user population Companies, researching and administrative staff of research institutes and higher education institutes, including director, professor, lecturer, tutor, researcher and undergraduate and postgraduate students, who have certain level of experiences and knowledge with regard to ICT were considered as users of the VCC and were invited to take part as test participants in the usability studies. Characteristics which were common to all test participants were:

Possessing experiences of using software applications;

Possessing some basic knowledge of ICT and about videoconferencing systems, etc.;

Possessing good English proficiency, at least a high level of reading comprehension.

97


4.2 Characteristics of testing sites English language version of the VCC has been tested in UbuntuNet in Malawi (Africa), Zentrum für Soziale Innovation in Austria, Jožef Stefan Institute in Slovenia, U. Politécnica de Madrid (UPM) in Spain, University College London in United Kingdom (Europe) and CLARAnet – the Association of the academic networks of South America in Peru (Latin America).

4.3 Characteristics of test participants 29 participants (8 female and 21 male) were involved in the usability tests. All of them had the educational level of at least the first university degree. Their participations were voluntary. Prior to working out the task scenarios with the VCC, the participants were required to complete a Pre‐test Questionnaire on demographic data (gender, age, job title). This questionnaire also reflects the average level of the user competence declared by the test participants in ICT (M = 3.80,) and the average level of competences in videoconference systems (M=3.09). The nominal to interval scale, with left anchor “1” indicating the lowest level and right anchor “5” the highest level of the attribute in question. None of the participants had interacted with the VCC before they took part in the usability tests. These demographic data were relevant for interpreting the results of the usability tests.

4.4 Profile of local testers Ideally, Local Testers should be usability specialists or highly experienced in conducting usability tests. However, it may not be easy to get them, especially when the resource is tight. For that reason in selecting the Local Testers the following criteria were considered:

the person must be experienced in conducting experiments with human participants,

has some knowledge in Human Computer Interaction (HCI),

must be fluent in the native language and English,

has to be motivated to do research,

is good at observation,

is able to manage several tasks at one time.

It was recommended to employ the same Local Tester to conduct all the testing sessions to ensure the consistency of data recording and interpretation. That means that the testing was performed before and after

5. Task scenarios A set of five tasks covering the core functionalities of the VCC was employed. The tasks were presented in English. All test participants possess a reasonable level of English reading ability. Below is the list of the tasks:

(T1) Obtaining a user account for a Virtual Conference Centre (VCC)

(T2) Creation of new space in VCC and joining an existing one

(T3) Creation of new event

(T4) Modifying the event in event manager

(T5) Sending private message

Each of the above five tasks was translated into task scenarios, which render the test more realistic and problem oriented (e.g. you are organising an educational a workshop or other distributed event like a meeting. Therefore, you need to create an event in the space you created in task 2. Log yourself into the VCC and create a new event. The event must be marked as Isabel event). In addition, for each of the task scenarios, quantitative usability goals in terms of task completion time and number of errors were set, which were benchmarked by an experienced user of the VCC. They were used to serve as references or baselines for data analysis. System Usability Scale (SUS) (Brook 1996) and Feedback Questionnaire (FQ) (After‐Scenario Questionnaire) (Lewis 1991) were also employed after the testing.

98


5.1 Procedure Test participants were welcomed and briefed about the goal and procedure of the usability tests, which was followed by an explanation of the equipment to be used. Participants were asked to perform a set of selected task scenarios that cover most frequent as well as critical functionalities of the VCC. After each task, participants were asked to complete the After‐task questionnaire, consisting of four questions (Q1‐Q4), which were derived from the literature on usability research (Lewis 1991; Sauro & Lewis 2009). Specifically, Q.1 and Q.2 evaluated the extent to which the participants were satisfied with the ease and amount of time required to complete the task, respectively. A 7‐point Likert scale was employed with left anchor indicating lowest level of satisfaction and right anchor the highest. Q.3 and Q.4 evaluated the same two variables, but are phrased differently (i.e. how hard and how time‐consuming) and gauged with an online tool designed to measure subjective mental effort (Usable Surveys 2009). After completing all the five tasks, participants were asked to complete the Post‐test questionnaire entitled System Usability Scale (SUS) (Brooke 1996), which consists of 10 questions and has psychometric properties.

6. Analysis and results The collected data were categorized in two dimensions: (i) qualitative vs. quantitative and (ii) objective vs. subjective (Table 1). Some analysis results of these data types are presented in subsequent sections. Table 1: Two dimensions of data types Objective

Subjective

Qualitative A list of usability problems (UPs) derived from the participants’ notes Participants’ comments in SUS

Quantitative Time‐on‐task Effectiveness % Efficiency Responses to the questionnaires’: Pre‐test Q Feedback Q Follow‐up Q (SUS)

The data present either the descriptive statistics – mean and range (comprehends six quantitative objective and subjective usability measures: duration (min), perceived ease of use, perceived efficiency, perceived difficulty, perceived time‐consuming and task completion rate (%).

6.1 Quantitative data These data are based on quantitative measures that were received such as time‐on‐task; proposed time for task competition, mean time and standard deviation. The two usability metrics – effectiveness and efficiency – were derived from these statistics (Table 2). Each participant was required to perform five tasks. Based on the data provided from the 29 participants, the value (average time) of this variable was found to be 30.65 minutes, with the range from 15.00 (JSI‐P5) to 81.00 (UBN‐P6). Altogether, 145 tasks were performed and 145 (100 %) were successfully completed. As shown in Table 2, among the 5 tasks, Task 4 (Modifying the event in event manager) was found to be most problematic. Indeed, the average time‐on‐task for Task 4 was 9.48 minutes, exceeding the benchmarked upper bound (i.e., 6.00 minutes) by 63.2 %. The range of time‐on‐ task for Task 4 is large, spanning from 4.00 min (UPM‐P2) to 20.00 min (UBN‐P1 and UBN‐P6). As a matter of fact, Task 3 (Creation of new event) is similar to Task 4 (Modifying the event in event manager) except the required additional scheduling. As evidenced by the data, the average time‐on‐task for Task 3 was less than that for Task 4. All participants performed Task 3 much faster. However, in some other cases, the reverse could be observed (e.g., UCL‐P2). This may be attributed to the fact that this user found the task very confusing and without logic. Based on the quantitative data the effectiveness and efficiency per task was computed. Effectiveness denotes the rate that a task is completed successfully without assistance from any help desk – and is known as unassisted completion rate. Efficiency is calculated through dividing an unassisted completion rate by its corresponding unassisted mean time‐on‐task. In the Table 2, for the sake of comparison, both unassisted and assisted completion rates together with their corresponding mean time‐on‐task are displayed.

99

Borka Jerman Blažič, Tanja Arh and Andrej Jerman Blažič Table 2: Effectiveness and efficiencies per task Task 1 2 3 4 5

Total Completion (with or without assist) Rate (%) Mean Time 100 4.72 100 4.79 100 6.20 100 9.48 100 5.44

Effectiveness (%) (tasks without assist) Rate (%) Mean Time 100.00 3.00 100.00 4.00 85.71 5.00 57.14 8.00 100.00 4.00

Efficiency (%/min)

Total no. Assists

33.33 25.00 17.14 7.14 25.00

0 0 1 3 0

Note. Effectiveness = Unassisted Completion Rate; Efficiency = Unassisted Completion Rate/ Mean time‐on‐ task.

Figure 2: Effectiveness per task The average effectiveness over five tasks was measured with the required time for completion and that found to be 88.57 %. All participants were capable to complete all the given tasks, with or without assistance from the Local Tester. The average efficiency over five tasks was 21.52 %/minute, ranging from 7.14 %/min (Task 4) to 33.33 %/min (Task 1). In fact, Task 4 (Modifying the event in event manager) was proved to be problematic.

6.2 Qualitative data One of the most important results from the Usability Tests is the list of Usability Problems (UP) that are identified after the participants interacted with the system (Figure 3). The six testing sites have collected different sets of UPs, with a number of them being overlapped and the others being unique. We compiled and integrated the six lists of the usability problems into a complete list (see Table 3). The implications of individual columns are:

Usability problem (UP): It is the identifier of individual UP.

Task ID: It denotes in which task the UP was identified. For instance, Task ID 2(5) means that this UP was identified in Task 2 (Creation of new space in VCC and joining an existing one) and number 5 in brackets indicating how many of the test participants found the problem.

Descriptions of Usability Problem: Detailed explanations what the UP was and how the UP was identified.

Severity: There were three levels:

Severe usability problems are those that prevent the user from completing a task or result in catastrophic loss of data or time.

Moderate usability problems are those that significantly hinder task completion but for which the user can find a work‐around.

Minor usability problems are those that are irritating to the user but do not significantly hinder task completion.

As shown in Table 3, there were identified altogether 21 important usability problems (UP). Some of the UPs had frequency only once but the highest frequency found was 12 for UP13 (Task 3: Create a new event and invite people to this event) ‐ users had difficulty in inviting people to the event, as well as the option to invite people during the event creation.

100


Figure 3: List of the most important usability problems identified in the VCC

6.3 Feedback questionnaire (FQ) In a scenario‐based usability study, participants use a computer application to perform a series of realistic tasks and the FQ used in this study was a 3‐item questionnaire that assessed the participant satisfaction after the completion of each scenario (Lewis 1991). The items sought addressed three important aspects of user satisfaction with system usability: ease of task completion, time to complete a task, and adequacy of support information (online help, messages, and documentation). Each item was rated with a 7‐point Likert scale, with 1 being “Strongly disagree” and 7 “Strongly agree”. The items are phrased in a positive manner. Hence, the higher the score, the more the user is satisfied with the system. The questionnaire takes very little time for participants to complete. Table 3 shows the results of FQ of the five tasks. Q1.1 addresses the ease of task completion for Task 1 as perceived by a user; Q1.2 addresses the degree to which the user is satisfied with the time to complete Task 1. Q1.3 addresses the adequacy of support information for Task 1 as perceived by a user. The same sequence is for Task 2 to Task 5. Table 3: Results of FQ of five tasks Qn

Q1.1 Q1.2 Q1.3 Q2.1 Q2.2 Q2.3 Q3.1 Q3.2 Q3.3 Q4.1 Q4.2 Q4.3 Q5.1 Q5.2 Q5.3

Mean

5,41

5,66

5,07

5,90

5,90

5,59

5,21

5,38

4,72

4,38

4,48

3,76

4,48

4,79

4,07

St.Dev

1,72

1,72

1,44

1,08

1,35

1,38

1,29

1,40

1,36

1,68

1,60

1,62

2,03

2,38

1,96

Min

2,00

2,00

2,00

3,00

2,00

1,00

2,00

2,00

2,00

1,00

2,00

1,00

1,00

1,00

1,00

Max

7,00

7,00

7,00

7,00

7,00

7,00

7,00

7,00

7,00

7,00

7,00

6,00

7,00

7,00

7,00

For Task 2 (Creation of a new space in the VCC), the ease of completion was rated as 5.90, the degree of satisfaction with the completion time was 5.90 and the adequacy of support information was 5.59. These ratings imply that the users generally were satisfied with this particular Task. Task 1 (Obtaining a user account in the VCC) has similar ratings, but of lesser degree. The results from Task 4 (Modify and event in event

101

Borka Jerman Blažič, Tanja Arh and Andrej Jerman Blažič manager) and task 5 (Send private message) imply that the users generally were not so satisfied with this particular tasks.

6.4 Discussion and the re‐design of the system The obtained results of the first VCC release usability study have pointed out to 21 usability problems (UPs). Some of the UPs have single frequency but the highest found frequency was 12 for the usability problem within the Task 4: Modify the event in the event manager. Users have found difficulties in inviting people to the organized event during run time of the event as well as in the option to invite people during the event creation. These problems required strong repair of the system. The system was repaired and the required functionality was improved towards friendliness and easiness of use. The redesigned system and the first version of the system for comparison are presented on Figure 4. The differences are visible in the appearance of the front page but the majority of the improvements were in the functionality of the services provided by the system. The navigation in the portal was changed dramatically in light of the usability requirements collected. The second version of the portal enabled users to create an educational event on distance more easily as this function was simplified and the setting of the distributed auditorium(s) needed two steps only compared to four before the applied change, the setting up and the running of the user community within the allocated space(s) was much more friendly enabling easy evidence of the created space and the member’s identity. The exchange of messages between the participating communities was designed to be as much as possible user friendly and the information space about on‐going event was much more survey‐able. The use of repository was also made friendlier as most of the actions were performed by clicking on the selected item and the search algorithms for specific items were improved as well. The identified and reported usability problems in the second evaluation after the redesign were of minor and of moderate nature, the later had dealt with breaking the audio signal sometimes but this is happening on other audio systems as well. The cause of this problem may lay in the particular session signal transmission problems. The total number of reported problems in the second evaluation was only 5 compared to 21 in the first evaluation process. The evaluation carried out with questionnaire delivered after each organized event addressed properties relevant for system usability, such as need for support and training for participating in such event. The answers of the audiences that participated in 23 organized distributed events over the network were collected immediately after the event and then used for assessment of the system acceptance by the users. The audience was asked to answer after they have used the VCC during the events with an intention of avoiding thinking about the system. The results were used for the re‐design of the system and were considered as satisfactory, with a score average of 78.3 %. These results suggest that the acceptance of the tool by the user communities will be as good as expected. More insight in the study and the results are possible from the Global Plaza Deliverable.

Figure 4: The system before and after the re‐design based on the usability study

7. Conclusion User acceptance assessment based on the user experience is a necessary part of every service/system‐design process. However, in contrast to the standard usability studies, the evaluation of the user experience is a

102

Borka Jerman Blažič, Tanja Arh and Andrej Jerman Blažič relatively new field within the larger scope of HCI (Isleifsdottir & Larusdottir 2008). The study presented addressed both methods for assessment of the usability of the system and the potential acceptance of the service. Application of both type of studies enable more holistic view of the user experience then the classical evaluation of usability which focus on efficiency and effectiveness only. In the VCC studies within the Global Plaza project this aspect was very strongly considered. For that reason the usability study based on the task scenarios supplemented with SUS questionnaire, parameters measuring participation and user activity, then evaluation based on the user feedback after the organized events and educational workshops have shown the user satisfaction with the system efficiency and acceptance. We may claim that objective of this study was achieved, as the exhaustive results collected through observations that lasted three years with participating communities from three continent have shown that this type of collaboration over the WEB is acceptable, useful and appreciated by many virtual communities, especially in view of the offered different possible communication accesses to the platform and the numbers of available protocol gateways enabling access, education and collaboration on distance. This conclusion has led to later incorporation of the VCC with the eCAMPUS LMS and the building up of an efficient Virtual Learning Environment.

Acknowledgements This work has been performed in the framework of the EU funded FP7 GLOBAL project.

References Brinck, T., Gergle, D., & Wood, S. D. (2002) Usability for the Web: Designing Web Sites that Work, San Francisco, USA: Morgan Kaufmann. Brooke, J. (1996) SUS: A ‘quick and dirty’ usability scale. In W. Jordan, B. Thomas, B. A. Weerdmeester, I. L. McCleland (Ed.), Usability evaluation in industry (21, pp. 189‐192). London, UK: Taylor & Francis. Dumas, J. S. and Redish, J. C. (1999) A practical guide to usability testing (Rev. ed.). Exeter: Intellect. GLOBAL project. (n.d). Retrieved January 26, 2012, from http://www.global‐project.eu. Isleifsdottir, J. and Larusdottir, M. (2008) Measuring the user experience of a task oriented software. In L. C. Law, E. et al. (Eds), Proceedings of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (pp. 97‐101). Lewis, J. R. (1991) Psychometric evaluation of an after‐scenario questionnaire for computer usability studies: the ASQ. SIGCHI Bulletin, 23(1), 78‐81. Lewis, J. R. (1995) IBM Computer Usability Satisfaction Questionnaires: Psychometric evaluation and instructions for use, International Journal of Human‐Computer Interaction, 7(1), 57‐78. Nielsen, J. (1996) International usability testing. In E. del Caldo and J. Nielsen (Eds.), International user interface. New York: John Wiley & Sons. Patrick, A. S. (2006). The Human Factors of Mbone Videoconferences: Recommendations for Improving Sessions and Software, Ottawa, Communications Research Centre. Preece., J., Rogers, Y., Sharp, H. and Benyan, D. (1994) Human Computer Interaction. Addison‐Wesley, Essex, U.K. Sauro, J. and Lewis, J. R. (2009) Correlations among prototypical usability metrics: evidence for the construct of usability. In: Proceedings of ACM CHI 2009 Conference on Human Factors in Computing Systems, 2009, pp. 1609‐1618. Usable Surveys (2009). Retrieved April, 27, from http://www.usablesurveys.com/index.php.

103

Engaging Local and Global Learners in an e‐Learning Environment ‐ University of Toronto Experiences Nohjin Kee, Dani Sarkis and Chris Perumalla Department of Physiology, University of Toronto, Toronto, Canada [email protected] [email protected] [email protected] Abstract: Technological advances have made e‐Learning possible, delivering education over the internet thus revolutionizing learning for local and global learners (Tapscott, 2009). This offers a great deal of flexibility and independence in a semi‐structured environment for individuals with established careers or family commitments. Online physiology courses developed at the University of Toronto (UofT) serve both global and local (in‐class) learners. The Distant Online Course (DOC) caters to students who are taking this course as a prerequisite for professional programs to follow an alternative career path or out of interest. In‐class Online Courses (OOCs), on the other hand, cater to students in various health science programs such as medical radiation science, physician assistant program, and pharmacy as part of their program requirements. DOC learners watch lecture videos online and have 24/7 access to a discussion board forum on the course website where questions can be asked and answered and monitored by the teaching assistant and the course lecturers. In addition, there are weekly virtual office hours with the TA on the course website. There is no in‐person interaction between the learners and the teaching staff, and correspondence is via discussion board, virtual office hours and email only. OOC learners have access to the same tools online; however they have in‐person office hours with the TA and are able to speak to the lecturers in person, as well as in‐person lectures at the beginning of the course. Data from student surveys have been collected for both course types since 2009. We will compare student experiences between the two types of online courses in order to determine whether one method of delivery is more effective than the other. Furthermore, data from these courses will be compared with a virtually identical in‐class course, which lacks any online components. This will allow us to assess the validity of our online courses as learning tools compared to the conventional classroom setting. We expect to see similar learning patterns in DOC and OOC. Additionally, we expect these results to be comparable to those obtained from the equivalent in‐class course (Moore, 1993). Keywords: e‐learning, distant, online, in‐class

1. Introduction In today's technologically advanced world, academic and professional environments are highly dynamic and require individuals to quickly adapt. This means that accreditation or training must be obtained without compromising professional or personal responsibilities. At the University of Toronto, this is made possible for students by offering online courses. The Department of Physiology offers online physiology courses which enable students to learn at their own pace and place by offering a great deal of flexibility in a semi‐structured environment (Perumalla, 2011). Furthermore, our courses allow students to achieve a healthy and reasonable balance between their home and academic lives. These courses are offered in two delivery method: Distant Online Course (DOC) and In‐class Online Course (OOC). In DOC, students enrolled need not be physically present on campus for any of the course components. Students watch lecture videos and write several online quizzes on the course website. Even the final exam (in‐person, written exam) is administered in a proctored examination facility near the student. Students from China, Sweden, England, Tanzania, Australia and other countries have taken this course offered by a Canadian university over the internet. Students taking this course generally have little or outdated scientific background who either need a human physiology credit as a prerequisite for a professional program or want to learn more about the human body to satisfy their personal interests. OOC students are different in that they are taking their course as a program requirement and have previous taken life science courses. They also need to be physically present on campus for some components of the course, although the lecture videos are virtually identical to those presented to DOC students. In a previous study, we compared student performances between the two course types and found no significant differences, thus indicating the validity of these courses as learning tools. This result is strengthened when we compared results from online courses to a virtually identical in‐class course with no online components (Sarkis, 2013). In this study, we aim to assess the learning experience for students enrolled in online courses. Namely, we will examine the sense of community, added value, clarity, and foundational knowledge offered through the

104

Nohjin Kee, Dani Sarkis and Chris Perumalla course discussion board, tutorials, lecture videos, and depth of information, respectively. We will also compare the results from our online courses to a virtually identical in‐class course with no online components.

2. Methods The makeup of the courses has been described previously (Sarkis, 2013). Both DOC and OOC courses rely on the course website on Blackboard platform and have up to 51 video lecture hours which cover human physiology in 10 modules. The lecturer faces the camera directly to establish a connection with the students watching and the corresponding lecture slides appear next to the lecturer’s image. Three to four lecture videos are made available on the course website for a period of 7 days, during which students can watch each lecture over the internet. To ensure discipline and compliance, these videos are taken down from the course website and new lectures are made available the following week. Students enrolled in DOC have open access to the course website where the course discussion board, lecture videos, and virtual office hours are available. The course discussion board is a part of the course website and serves as a forum where students can post their questions and offer insights and answers to questions posted by their peers and start discussions on any topic they are interested in, regardless of whether it is covered in the course or not. Discussion board is available 24/7 with and is constantly monitored by the teaching assistant and the lecturers to ensure the accuracy of the content and information posted by students. In addition, the teaching assistant holds two weekly virtual office hours where students can log on to the course website and enter a Java based chat room. During this time, students present questions and problems to the teaching assistant and to fellow students, where it is up to the teaching assistant’s discretion to either lead a discussion or provide a direct answer to the proposed questions. In the virtual classroom, the teaching assistant types his/her responses to the questions posted by students, and these responses are visible to all students who are logged on. Also, the teaching assistant has a white board tool to allow drawing and illustrating concepts and solutions to problems. These sessions are recorded so students who missed the virtual office hour or cannot attend can review the transcript at a later time. Students also have the option of contacting the teaching assistant and the lecturers directly via email, although communication through the course website is encouraged to enhance the learning experience and involve the entire class in the discussion. For more information on our virtual distant online physiology course at the University of Toronto, one can go to the course website http://onlinephysiologycourse.med.utoronto.ca The University of Toronto, through its Physiology department, offers 3 in‐class online courses (OOC) that have small differences in their structure. OOCs cover the same 10 modules as DOC and use the same lecture video structure, but the number of lecture hours is different based on the course and the audience. Discussion board is available for students in OOCs, as well as the option to contact the teaching assistant and lecturers via email. Instead of holding virtual office hours, the teaching assistant holds in‐person office hours where students can ask questions and participate in discussions. Students’ performance is assessed based on online quizzes, written midterm tests, and a written final exam, which are proctored and held on campus in the presence of the teaching assistant and the course coordinator.

3. Results and discussion Students were asked to complete a course evaluation survey and to provide their responses on a scale from 1 to 7, where 1 is the lowest rating and 7 is the highest. Four criteria were assessed in these surveys: value of discussion board (community), value of lectures (clarity), value of tutorials and office hours (added value; virtual and face‐to‐face), and depth of learning (foundational knowledge). Two‐way analysis of variance (ANOVA) was performed to compare groups, with Tukey’s honest significance test performed to correct for multiple comparisons.

3.1 Discussion board When asked about the value of the discussion board, there were no significant differences between responses from students enrolled in DOC (SCS2159), OOC (MRS162, PAP113, PSL205), or the virtuallyidenticalin‐class course (PSL201) that were used as a reference (Figure 1). The discussion board provides a necessary platform in which the students are able to interact with the faculty and with each other. Itenables the students to get the necessary feedback and plays a crucial role in allowing students to develop a sense of community (Sher, 2009).DOC students rated the value of discussion board the highest among the three types of courses for 3

105

Nohjin Kee, Dani Sarkis and Chris Perumalla years running. Analysis of student responses shows that in 2010, there was a statistically significant difference between DOC and in‐class evaluation of the discussion board (P=0.0378) and that in 2012, DOC there were statistically significant differences between the groups (DOC vs. OOC, P=0.364; DOC vs. in‐class, P=0.0016). This suggests that DOC students find the most value in that tool, most likely due to lack of other options to establish a sense of community, as these students are not geographically close to one another. In OOC or the in‐class courses, students can form study groups and meet in person on campus or off campus, and therefore they do not need to rely on the course website to achieve this sense of community.

**

20 12

*

20 11

20 10

*

Figure 1: Student evaluation of the value of the course discussion board

3.2 Lectures Students were also asked about the value of lectures provided in their respective courses. Our video‐based lectures provide the students with an effective means of delivering the courses online (Choi, 2005) and we have compared the student responses with that of our in‐class course.In 2010, DOC students gave a higher rating for the value of lectures than their OOC counterparts (P=0.0004), as did the in‐class students (P=0.0006). In 2012, DOC gave a higher rating for the value of lectures than their OOC counterparts (P=0.0009).While there are some statistically significant differences between the samples, there are no consistent trends or patterns. This indicates that there are no differences in the quality and clarity of teaching offered through the different type of courses despite the varying methods of delivery (Figure 2). This further supports our argument that online education is a valid and reliable teaching tool in today’s technologically advanced world.

Lectures ***

*** *** 6

DOC OOC In-Class

4

20 12

20 11

20 10

2

Figure 2: Student evaluation of the value of lectures

3.3 Tutorials Students were also asked about the value of tutorials held by the teaching assistant during the course. Similar to the previous two criteria we considered, no consistent trend in terms of which group of students gave

106

Nohjin Kee, Dani Sarkis and Chris Perumalla

20 12

20 11

20 10

higher or lower rating of the added value to the course was observed (Figure 3). Statistical analysis revealed no significant differences between any of the groups tested. This indicates consistency in the quality and perceived value of tutorials in all course types.

Figure 3: Student evaluation of the value of tutorials

3.4 Depth of Information Finally, students were asked about the depth of learning experienced in their respective courses. In 2010, DOC students found more value in the depth of information offered through the course than OOC students (P=0.0048), as did in‐class students (P=0.168). In 2011 and 2012, there were no statistically significant differences in opinions regarding the foundational knowledge offered by the courses, which indicates the lack of bias towards students of a certain course type (Figure 4). Taken together, these results indicate no differences in the students’ learning experiences in differing delivery methods of the courses. This also supports our previous study that encourages online education as a valid teaching tool.

Depth of Information **

*

DOC OOC In-Class

6

4

20 12

20 11

20 10

2

Figure 4: Student evaluation of the depth of learning

4. Conclusion The three different types of physiology courses – DOC, OOC and In‐Class offered by the Department of Physiology – provide invaluable data on the impact on learners on these three types of learning modules. In this study, we assessed the importance of clarity, community, added value, and foundational knowledge offered through our virtual distant online course and blended online courses. Although there were statistically significant differences in some of the data subsets examined, we found that there were no specific trends in the opinions of students enrolled in DOC, OOC, or classical in‐class courses delivering identical content. Our results indicate that the method of information delivery does not significantly affect students’ performance or learning experience. Thus, with the evolution of university courses as technology is harnessed in either

107

Nohjin Kee, Dani Sarkis and Chris Perumalla incorporating some aspects of online learning in on‐campus courses in the blended format or online distance education courses do not significantly impact learning, performance or sense of community among students.

.

References

Choi H and Johnson SD. (2005). The Effect of Context‐Based Video Instruction on Learning and Motivation in Online Courses. American Journal of Distance Education. 19:4, 215‐227 Moore, M. (1993). Three Types of Interaction. Distance Education: New Perspectives, eds. K. Harry , M. Hohn and D. Keegan. London: Routledge. Perumalla C, Mak J, Kee N and Matthews S. (2011) Integrating Web Applications to Provide an Effective Distance Online Learning Environment for Students. Procedia Computer Science. Volume 3, 2011, Pages 770‐784. Sarkis D, Kee N, Perumalla C (2013) Fully Online or Blended Courses: Does It Make a Difference for the Learner. International Journal of Information Technology & Computer Science ( IJITCS ). Volume 12, Issue 4, Pages 1‐6. Sher D. (2009). Assessing the relationship of student‐instructor and student‐student interaction to student learning and satisfaction in Web‐based Online Learning Environment. Journal of Interactive Online Learning. Volume 8, Number 2. Tapscott, D. (2009). Grown up Digital. McGraw Hill, New York, Toronto

108

Virtual Collaboration Competence Requirements for Entrepreneurship Education in Sparsely Populated Areas Päivi Lohikoski, Matti Muhos and Janne Härkönen Department of Industrial Engineering and Management, University of Oulu, Finland Oulu Southern Institute, Micro‐entrepreneurship Research Group, University of Oulu, Finland [email protected] [email protected] [email protected] Abstract: This study aims to define preconditions for organizing entrepreneurship e‐learning in northern sparsely populated areas (SPAs). The study is based on a literature review that combines theories of e‐learning and virtual collaboration competences in industrial settings. As a result, we propose careful planning and using constructive methods such as problem‐based learning in entrepreneurship education. A certain level of training is required for participants to ensure that they understand the special characteristics of virtual work. Furthermore, an opportunity for face‐to‐face interaction at the beginning of the project is highly recommended. These actions can help to overcome any ICT and synchronization issue‐related barriers and deficient relationship building. These actions also enhance development of trust and knowledge sharing among individuals. Lastly, lack of time and resources are typical challenges for students in SPAs, and therefore content and schedules require careful planning. Keywords: sparsely populated areas, entrepreneurship education, virtual collaboration competence, e‐learning

1. Introduction From the perspective of global markets, the sparsely populated areas (SPA) in the north of Sweden and Finland are all peripheral; furthermore, they are heavily reliant on export for gaining viable economic returns. The lack of an entrepreneurial tradition and poor availability of business support services have also inhibited the start‐ up of new enterprises in these areas (Lundmark 2006). Encouraging entrepreneurship through virtual education is a key to increasing the competitive advantage in northern sparsely populated areas (NSPAs) (See Gløersen et al. 2006). Virtual education may function as a concrete example of decreased location‐dependency for the participants and their current/future businesses. The purpose of this study is to clarify preconditions for successful virtual collaboration in organizing e‐learning in NSPAs. The business environment in NSPAs differs from urban areas, potentially influencing the entrepreneurial process and the entrepreneurial outcome. In virtual work settings, it is essential to acknowledge that successful work requires specific virtual collaboration competences as people must cope with a variety of issues, including differences in synchronization of communication, diverse backgrounds and geographical distances. These aspects, among other things, can lead to misunderstandings in communication, reduced information flow and misinterpretations in exchanging information (Drouin et al. 2010; Lohikoski & Haapasalo 2013). It is possible to overcome these challenges on both the personal and organizational level with virtual collaboration competences (VCC). As a theoretical framework, VCC is interesting and relevant; previous researchers, including Wang and Haggerty (2009) and Holton (2013), have compared virtual projects against learning processes. In this article, e‐learning theory is enriched with VCC theory, which is typical for industrial settings. In addition, best practices for successful e‐learning projects are identified; i.e. those that form the cornerstones for an effective platform for entrepreneurship education in SPAs. Figure 1 outlines the theoretical framework of this study. The research was realized by conducting a literature review on e‐learning and virtual organization theory. The particular focus was on VCC on the organizational and personal levels in order to compare best practices in virtual projects to ways of organizing e‐learning projects. The research questions were as follows:

What are the preconditions for successful virtual collaboration?

What are the preconditions for organizing e‐learning in NSPAs?

109


Motivation

Northern sparsely populated areas Need for entrepreneurship education

Literature review

Synthesis

Virtual Competences

Preliminary framework for entrepreneurship education platform for entrepreneurship education in sparsely populated areas

E‐learning

Implications Theoretical implications

Managerial implications

Policy implications

Figure 1: Theoretical framework

2. Methodology The methodology utilized in this study is a literature review, which consisted of identifying relevant data sources and publication databases and then searching these sources using chosen key words and phrases associated with virtual organizations, e‐learning and entrepreneurship education. Although the methodology has been influenced by that of content analysis (e.g. Li and Cavusgil 1995), there has been no attempt to define categories for the articles or categorize them. Nevertheless, the procedure can be considered to be a systematic, explicit and reproducible method for identifying, evaluating and analysing the existing body of recorded work (e.g. Fink 2004). The literature review was conducted in order to reveal potential preconditions for organizing e‐learning on entrepreneurship in NSPAs. The purpose was to gather the relevant previous research findings on the topic. The existing literature was reviewed and analysed to answer the research questions. A stringent quality criterion was used in terms of the relevance of the content of the found articles; hence, this paper does not contain all the work published. The databases searched include Google Scholar, Emerald, Web of Science and Scopus. These databases provide adequate coverage of peer‐reviewed literature. Some complementary Internet searches were also conducted to ensure that the coverage was sufficient. The searches provided access to a variety of sources, including journals, conference proceedings, books and business periodicals. The sources were selected based on their relevance to the studied topic and the research questions. Journal articles were preferred when possible; however, other relevant publications were utilized as well. Keywords that were associated with the topic included, but were not limited to, virtual collaboration, virtual collaboration competences, virtual organization, virtual organization theory, virtual environment, e‐learning, entrepreneurship, entrepreneurship education, sparsely populated areas and periphery. The key words were also used in multiple combinations to find relevant articles. The set of keywords were expanded and refined as articles were discovered. Both transatlantic forms of the keywords were used when different spellings exist. Only articles published in English are included in this study. The keywords were expected to appear in the articles within the actual discussion. Once relevant articles were identified, their references were reviewed to aid in locating additional papers, resulting in some beneficial findings. The articles which were found were read in order to understand their content and analyse their contribution to the research questions and aims. The findings were narrowed down to 45 documents which were found to be the most relevant to the research focus. The reference list for this article contains all of these findings. Although there is always a possibility that some important work has been missed, the reviewed articles constitute a reasonable collection of relevant research work on the focus of this article. The key findings are synthesized, and their implications will be discussed in a future.

110


3. Theoretical framework 3.1 Northern sparsely populated regions The northern parts of Finland, Sweden, Norway and the majority of Iceland can be defined as sparsely populated areas (SPAs). These areas are especially affected by globalization, energy supply, climate and demographic change. Peripherality, cold climate, low population densities and dispersed settlement patterns are characteristics common in NSPAs (Gløersen et al. 2006). In the European Union, SPAs are defined as regions with a population density below a certain threshold; a region is defined as sparsely populated if it has a population density of fewer than 12.5 inhabitants per km² (Eurostat 2012). The business environment in NSPAs differs from urban areas, potentially altering the entrepreneurial processes and the entrepreneurial outcomes (Gløersen et al. 2006). On one hand, micro‐sized businesses in NSPAs have difficulties in reaching sources of scientific knowledge, skilled employees, suppliers, customers and sufficient finance. Remoteness also brings about transportation challenges, increases manufacturing costs and may require the workforce to commute long distances. Moreover, a lack of educated entrepreneurs characterizes these areas. Education service organizations in NSPAs face similar context‐specific challenges as they seek to provide services for their customers. On the other hand, the remote location of a business may result in low‐priced business premises (Gløersen et al. 2006).

3.2 Entrepreneurship and entrepreneurship education The definition of entrepreneurship varies in the literature, which often leads to confusion among scholars (see e.g. Venkataraman 1996; 1997). By definition, entrepreneurship as a societal phenomenon leads to improved use of resources in the economic system as a whole (Davidsson 2003). This perspective emphasizes the positive outcome. Entrepreneurship as a scholarly domain aims at better understanding the societal phenomenon of entrepreneurship (Davidsson 2003), and it is this perspective which was chosen for this study. More specifically, the following definition is used: Entrepreneurship research studies all causes, activities and consequences that happen before, during, after and around creation of new product and venture under great uncertainty in any organizational context (Davidsson 2003; Landström 2005; 2010). The definition of micro‐entrepreneurship can thus be modified accordingly: Micro‐entrepreneurship research studies all causes, activities and consequences that happen before, during, after and around creation of new product and micro‐sized business under great uncertainty in any organizational context (Davidsson 2003; Landström 2005; 2010). Over the decades, researchers have been using a variety of theoretical bases and definitions for entrepreneurship, but as a whole the discipline has become quite clear. At the same time, academic institutions have increasingly taken part in entrepreneurship research and education (e.g. Johansson et al. 2012). According to Acs and Audretsch (2010) and Johansson et al. (2012), there are six key elements which form a useful cross‐section to classify the entrepreneurship research field. The following six elements can also be considered as constituting the vital elements for entrepreneurship education:

Opportunity

Emergence of new ventures

Market context

Social context

Global context

Entrepreneurial society

The majority of the businesses in the NSPAs are micro‐sized. In fact, more than 9 out of 10 (92.4 %) of all enterprises in the EU‐27’s non‐financial business economy are micro enterprises (employing fewer than 10 persons); their relative share of the non‐financial business economy workforce and value added was considerably lower at 29.9 % and 21.2 %, respectively (Eurostat 2013). However, the contribution of micro‐

111

Päivi Lohikoski, Matti Muhos and Janne Härkönen sized firms to new job creation in NSPAs is considerable. For example, according to Statistics Finland (2013), between 2011 and 2011, micro‐sized businesses generated 42 % of all new jobs, while large enterprises represent only 13 % of such job growth. Eurostat (The Statistical Office of the European Communities) classifies firms based on size as follows: micro‐ enterprises with less than 10 employees; small enterprises with 10–49 employees; medium‐sized enterprises with 50–249 persons employed; small‐ and medium‐sized enterprises (SMEs) with 1–249 employees and large enterprises with 250 or more employees. In addition, according to Eurostat, SMEs have an annual turnover of less than EUR 50 million or a balance sheet total of no more than EUR 43 million. Statistics of Finland (2013) defines turnover of micro‐enterprises to be under EUR 2 million. Independency also influences the classification, meaning that larger companies cannot hold more than 25 % of the company’s shares or voting rights. These classifications are used in this study. The entrepreneurs in micro‐sized businesses face the following challenges in terms of additional education:

Lack of time

Lack of resources

Poor synchronization of productive work and education offered

Geographical distance

Low education level

Heterogeneous study groups (students are from different fields)

According to Bennet and Bennet (2008), e‐learning offers possibilities for informal lifelong learning and formal educational structures for both businesses and academic communities. E‐learning provides flexibility, variety of choice and access to remote learners (Bell 2007; Bennet & Bennet 2008). There is also evidence that e‐ learning becomes a part of everyday life for individuals by enabling them to develop an emotional connection to learning. Nevertheless, e‐learning requires special competencies in order for a person to be able to deal with the nuances, theory, uncertainties and robustness of the learning systems (Bennet & Bennet 2008). Kotey and Anderson (2006) studied distant learners and concluded that remote students with work experience usually want to pursue their educational goals while working simultaneously. However, they tend to minimize all the extra pressure from their work in order to have enough time for education. Savery and Duffy (2001) suggest that by using problem‐based learning (Pbl), the content of their work and studies can be successfully integrated. By providing a learner with ownership of the process, authentic tasks and the possibility to develop solutions to problems, learning can be enhanced and the outcomes of the work improved. By using smart pedagogical models, such as Pbl‐learning, the outcomes of the practical work and motivation for learning can be enhanced simultaneously.

3.3 Virtual collaboration competence (VCC) requirements VCC refers to individuals’ and organizations’ skills for operating in virtual settings that enhance the positive outcomes of the projects (Lohikoski & Haapasalo, 2013). According to Kankanhalli et al. (2007), special characteristics of virtual work must be first identified and then made known to team members through training. In this way, conflicts can be avoided and the project can be made more effective. Bennet and Bennet (2008) suggest the same for the e‐learning projects. In some cases, a virtual environment can cause feelings of isolation; therefore, internal motivation for virtual work is needed (Ettinger at al. 2009). In virtual settings, it is common that there are high levels of autonomy rather than direct control, which means that competence for self‐regulating learning is needed. Malhotra et al. (2007), Dennis et al. (2013) and Snowden and Boone (2007) emphasize communication and language skills to ensure efficiency in virtual settings. One of the most critical issues of virtual collaboration is the delay in communication. However, this can also be seen as a strength of virtual collaboration as the delay allows flexibility in organizing work tasks and personal life. The level of synchronization and human and ICT‐related delays need to be planned and taken into account when planning a virtual project (Faraj et al. 2011; Li 2010; Zigurs 2003). According to Zigurs (2003), there are four important issues in leading virtual organizations: roles, leader presence, team process and technology. In virtual organizations, the manager’s responsibility is to create the basis for successful work; therefore, tools, definitions and structure are needed. Snowden and Boone (2007)

112

Päivi Lohikoski, Matti Muhos and Janne Härkönen suggest that managing complex contexts could be done by opening up discussion forums for interactive communication. Setting up barriers is also beneficial, because then the system can self‐regulate and work becomes effective. Diversity should be encouraged and pedagogical models should be used to enhance conversations. Virtual organizations have been criticized for having a limited capacity for transferring rich information and knowledge; however, based on Wang and Haggerty (2009), those limitations can be addressed with the following three competencies:

Virtual self‐efficacy (future‐oriented belief about one’s technical abilities to work in virtual settings)

Virtual media skills (using the full potential of ICT to enhance communication)

Virtual social skills (recognizing the difference between communication in a regular work environment and virtual settings)

Anantatmula and Kanungo (2010) stress the point that virtual teams and organizations need highly skilled individuals who are willing to extensively participate in conversations. Therefore, good communication skills are needed and trustworthy behaviour and credibility are also crucial in teamwork during group assignments.

4. Preconditions for learning in virtual settings According to Bennet and Bennet (2008), a learner does not truly learn before creating her own understanding and interpretation of the issue being studied. Those understandings are always a mixture of personal history, current beliefs and assumptions. Challenges in e‐learning systems are often related to technology, but also to the platform’s capability to support sharing tacit knowledge through understanding and nature of the situation. Learners need communication skills, problem‐solving skills and the ability to take actions to demonstrate their knowledge (Bennet & Bennet, 2008). Relationship building, especially at the beginning of the project, enhances knowledge transfer and learning (Holton 2013; Wang & Haggerty 2009; Faraj et al. 2011; Malhotra et al. 2007; Senge et al. 2007; Zigurs 2003). According to Wang and Haggerty (2009), the following four factors have an effect on the quality of virtual collaboration:

Early face‐to‐face meetings and training to overcome any problems in technology

Increasing technology skills and general familiarity with ICT tools.

Assimilating other employees’ backgrounds

Interpersonal relations

Ubachs (2009) suggests that a clear understanding of goals and expectations is needed in order for virtual learning to be successful. Also, a handbook of technical support and tutoring can prove useful. This is especially true if students are aware of their personal learning styles and are capable of acquiring the learning styles required by the e‐learning environment. Pedagogical support and guidance provided by the education organizer, library, information services or academic and administrative divisions may also be needed. Wang et al. (2009) suggest that in e‐learning, considering the pedagogical, social, psychological and technological aspects of learning can be beneficial. The social design of the learning environment can enhance communication between students, while psychological aspects acknowledge students’ backgrounds and preferences in order to meet the individual needs. Technological aspects take into account the tools and course structure, whilst pedagogy in general enables students to experience learning and higher achievement. Based on research conducted by Ettinger et al. (2006), the following issues should be considered in the context of e‐learning:

Enough time for planning

Individual motivation for learning

Overcoming potential resistance to e‐learning

Quality of content

User‐friendly technology

New pedagogical models

The availability of adequate technologies is crucial for working and communicating in virtual projects (Luther & Bruckman 2011). According to Malhotra et al. (2007), there are several ways to share knowledge virtually. Most companies utilize different kinds of cloud services, i.e. virtual databases that can be far more than storage places for documents. Virtual team rooms (e.g. LotusNotes) can be used in sharing knowledge;

113

Päivi Lohikoski, Matti Muhos and Janne Härkönen company‐specific virtual meeting tools are available as well. In e‐learning, popular software packages include, for example, Sakai, Blackboard and Knowledge Forum (Bennet & Bennet 2008). In the University of Oulu, the tools for e‐learning include Moodle and Discendum Optima, which enable document and video sharing, virtual collaboration and discussions. In February 2014, the new Intranet Notio was launched. Notio offers virtual workspaces for projects organized by the university. The more familiar the technology is in terms of one’s own personal use of devices, the faster the technology is adopted (Johns & Gratton 2013). Technology can offer connectivity to the world and support self‐driven and experimental learning. Collaborating in groups and in one‐on‐one situations enables different viewpoints and can also be very exciting and help accelerate learning (Bennet & Bennet 2008). With regards to e‐learning, Nummenmaa and Virtanen (2002) suggest that the modern curriculum has its basis in interaction, with a particular emphasis on psychological, technological, cultural and pragmatic factors. The learning environment is seen as reflecting the ideas regarding the learning and teaching process, including the roles of teachers and students in the process (Nummenmaa & Virtanen 2002). Table 1 summarizes the identified preconditions for a successful e‐learning project. Table 1: Preconditions for a successful e‐learning project Preconditions

References Planning

Training for virtual collaboration Shared goals, roles and objectives

Handbook of responsibilities, technical support and tutoring Technology‐structure fit

Face‐to‐face meetings at the beginning Mastery in using ICT

Diversity is appreciated

Feedback procedures and rewarding

Problem‐based learning (Pbl) Quality content

Ettinger et al. 2006; Bennet & Bennet 2008; Ubachs 2009. Zigurs 2003; Kankanhalli et al. 2007; Ubachs, 2009. Gatlin‐Watts et al. 2007; Faraj et al. 2011; Malhotra et al. 2007; Dennis et al. 2007; Li 2010; Snowden & Boone 2007; Zigurs 2003; Kankanhalli et al. 2007; Foss et al. 2009. Ubachs 2009. Balsmeyer et al. 2008; Dennis et al. 2007; Zigurs 2003; Luther & Bruckman 2011; Mitchells & Zigurs 2009; Foss et al. 2009; Ettinger et al. 2006. Holton 2013; Wang & Haggerty 2008; Dennis et al. 2007; Zigurs 2003. Wang & Haggerty 2009; Faraj et al. 2011; Dennis et al. 2007; Luther & Bruckman 2011; Kankanhalli et al. 2007. Malhotra et al. 2007; Dennis et al. 2007; Li 2010; Snowden & Boone 2007; Maude 2011; Kankanhalli et al. 2007. Malhotra et al. 2007; Dennis et al. 2007; Snowden & Boone 2007; Zigurs 2003; Kankanhalli et al. 2007; Cooper et al. 2004. Savery & Duffy 2001; Savery 2006. Ettinger et al. 2006.

Based on previous research by Lohikoski and Haapasalo (2013), the most crucial issues in organizing virtual collaboration are ICT tools that support collaborative work, communication skills that include an ability to write clear and structured messages, fact‐based communication, active presence of a leader in the feedback process and follow‐up procedures. Ubachs (2009) notes that courses should be designed to foster active learning, facilitate individual study skills, support interaction in a virtual community and place the student in control of the time, pace and place of learning. An emphasis on evaluation, feedback and reflection on the learning process are also significant. Course design, development and evaluation should involve both academic and technical aspects of the process. Carliner (2002) suggests that there are various ways of designing the content using blended learning or an entirely electronic format when planning course material. Instructions can be

114

Päivi Lohikoski, Matti Muhos and Janne Härkönen provided synchronously or asynchronously by offering videos, sound, texts or a combination thereof. Resources offered online can include textbooks, journal articles, enrichment exercises, worksheets or stimulating links. Savery (2006) suggests the importance of real‐life problems for problem‐based learning, a multidisciplinary approach, collaboration between students and students taking responsibility for their own learning outcomes. In order to avoid feelings of isolation, a certain amount of interaction is needed to ensure commitment and to test knowledge, understanding and skills. The central findings are illustrated in Figure 3.

Figure 3: Preconditions for organizing successful e‐learning projects. The above framework functions as a reference framework for this study and future projects on organizing education in SPAs. Based on our literature review, it is suggested that interactive problem based group (Pbl) assignments are needed for the entrepreneurship students. By combining practice and theory in collaboration with other entrepreneurs, along with teachers’ advice, both business and learning outcomes can be enhanced. e‐learning can offer more flexibility, choice and connectivity to distant learners who may experience similar problems (Bennet & Bennet 2008). By connecting learning activities to a larger problem or task and promoting ownership of their own learning, students’ motivation can be enhanced and learning outcomes improved. Therefore, new knowledge can be created through social negotiation and evaluation of shared understanding (Savery & Duffy 2001). At the same time, it might be possible for the entrepreneurs to solve problems that relate to their practical work. Figure 2 synthesizes challenges and solutions relevant to entrepreneurship education in SPAs.

Figure 4: Major challenges and solutions for students in entrepreneurship education in sparsely populated areas

115


5. Discussion This study aimed to define the cornerstones for organizing entrepreneurship e‐learning for SPAs. Entrepreneurship e‐learning opportunities in SPAs have special significance globally in terms of creating new job opportunities and the emergence of new ventures, markets and social contexts. Promoting an entrepreneurial society and best practices that create new job opportunities in SPAs has global significance that extends well beyond Finland. The literature review revealed the main challenges that students in SPAs are likely to meet. Most challenges are related to lack of time and resources; inadequate virtual collaboration competences might be common as well. Geographical dispersion presents challenges in arranging face‐to‐face classes, which can significantly limit the course design. E‐learning can be as effective as traditional classroom learning, but teaching arrangements need to be designed differently. It is suggested in this literature review that the framework for a successful e‐learning project can be enriched from practices typically used in industrial settings in global companies. Before starting an e‐learning project, several issues need to be acknowledged. Face‐to‐face communication is typically recommended for the beginning of the project; it is also strongly recommended for e‐learning in order to overcome the limitations caused by a virtual environment. The planning phase is significant in defining adequate ICT and the content of the class, and the tools and content need to enhance the learning of practice‐ oriented students. For a successful e‐learning project, proper tools need to be provided, together with pedagogical and instructional models (e.g. problem‐based learning). Clear and motivating learning goals along with the teacher’s skills in enhancing relationship building and trust among students are both crucial. The best way to start a project is to hold a first meeting and orientation face‐to‐face in order to clarify the goals, rules, norms and objectives. Training for virtual collaboration is usually needed for the students in SPAs at the beginning of the project in order to enhance trust and introduce the students to the content of the learning material, tools, goals and guidelines. Tutoring, mentoring and technical support are also recommended.

References Abetti, P. (2001) Accelerated growth: helping companies get and stay on the fast track. International journal of manufacturing technology and management, Vol. 3, No. 1, 15‐30. Acs, Zoltan J., and David B. Audretsch. (2010) Handbook of entrepreneurship research: an interdisciplinary survey and introduction, 2nd ed. New York: Springer. Anantatmula, Vittal S., Kanungo, Shivraj (2010) ‘Modeling enablers for successful KM implementation. Journal of knowledge management, Vol. 14 Iss: 1, 100‐113. Bennett and Bennett (2008) e‐learning as energetic learning. The journal of information and knowledge management systems, Vol. 38. No. 2, 206‐220. Bell, Jane (2007) e‐learning: your flexible development friend? Development and learning in organizations, Vol. 21 No. 6, 7‐ 9 Carliner, Saul (2002) Designing e‐learning. Alexandria, US: ASTD. Churchill, N. C. and Lewis, V. L. (1983) The five stages of small business growth. Harvard business review, Vol. 61, No 3, 30‐ 50. Coad, A. (2007) Firm Growth: A Survey. Max Planck Institute Papers on Economics and Evolution, 1‐72. Economics of internal organization. New York: Free Press. Yin, R. (1989) Case study research: design and methods. Beverly Hills: Sage Publications. Cooper, Robert G., Edgett, Scott J. & Kleinscmidt, Elko J. (2004). Bencmarking Best NPD Practises –I. Research Technology Management; Jan/Feb 2004; 47, 1; ABI/INFORM Global 31‐43. Davidsson, Per. (2003) The domain of entrepreneurship research: some suggestions. In Cognitive approaches to entrepreneurship research (Advances in Entrepreneurship, Firm Emergence and Growth, Volume 6), eds. Jerome A. Katz, and Dean A. Shepherd, 315‐372. Emerald Group Publishing Limited. Dennis, Donna J., Meola, Deborah, Hall, MJ. (2013) Effective leadership in a virtual workforce. American society of training and development. T+D Feb. 2013, 46‐51 Drouin, Nathalie, Bourgault, Mario and Gervais, Caroline (2010) Effects of organizational support on components of virtual project teams. International journal of managing projects in business, Vol. 3 No. 4, 625‐641. Ettinger, Andrew, Holton, Vicky & Blass, Eddie (2009) E‐learner experiences: key questions to ask when considering implementing e‐learning. Industrial and commercial training, Vol. 38, No. 3. 144‐147. Faraj, Samer, Jarvenpaa, Sirkka L., Majchrzak, Ann (2011) Knowledge collaboration in online communities. Organization science, Vol. 22 No. 5, 1124‐1239. Fink, A. (2004) Conducting research literature reviews: from the Internet to paper, 2nd edition, Sage Publications, Thousand Oaks, California, USA.

116

Päivi Lohikoski, Matti Muhos and Janne Härkönen Foss J. Nicolaj et al. (2009) Encouraging knowledge sharing among employees: how job design matters. Human resource management, Nov‐Dec, Vol. 48, No. 6, 871‐893. Foss, Nicolai J. and Robertson, Paul L. (2000) Resources, technology and strategy .Explorations in the resource‐based perspective, Routledge: London. Gatlin‐Watts, Rebecca et al. (2007) A guide to global virtual teaming. Team performance management, Vol. 13 Iss: 1, 47‐52. Im, Hyun‐Gyung, Yates, JoAnne & Orlikowski, Wanda (2005) Temporal coordination through communication: using genres in a virtual start‐up organization. Information technology and people, Vol. 18 Iss: 2, 89‐119. Johns, Tammy & Gratton, Lynda (2013) Spotlight on the future of knowledge work. The third wave of virtual work. Harvard business review, Jan/Feb., 66‐73 Johansson, H., Muhos, M., Niinikoski, E., Kess, P. Entrepreneurship research in Nordic countries and the EU Baltic Sea Region. The Association of Private Enterprise Education Annual Conference, Maui, Hawaii, April 14 ‐ 16, 2013. Kankanhalli, Atreyi, Tan, Bernard C. Y., Wei, Kwok‐Kee (2007) Conflict and performance in global virtual teams. Journal of management information systems, Winter 2006‐7, Vol. 23, No. 3, 237‐274. Kotey, Bernice & Anderson, Phil (2006) Performance of distance learning students in a small business management course. Education + training. Vol. 48 No. 8/9. Landström, Hans. (2005) Pioneers in entrepreneurship and small business research (international studies in entrepreneurship). New York: Springer. Li, T., and Cavusgil, S.T. (1995) A Classification and assessment of research streams in international marketing, International business review, Vol. 4, No. 3, 251‐277. Li, Wei (2010) Virtual knowledge sharing in a cross‐cultural context. Journal of knowledge management, Vol. 14 Iss: 1, 38‐ 50. Lohikoski, Päivi & Haapasalo, Harri (2013) Virtual competencies and knowledge transfer in global NPD. International journal of management knowledge and learning. Forthcoming. Lundmark, L. (2006) Restructuring and employment change in sparsely populated areas: examples from Northern Sweden and Finland. Akademisk avhandling Kulturgeografiska institutionen Umeå universitet GERUM 2006:2. Malhotra, Arvind, Majchrzak, Ann & Benson, Rosen (2007) Leading virtual teams. Academy of management perspectives. (Feb), 60‐70. Maude, Barry (2011) Managing cross‐cultural communication. Principles and practise. PalGrave MacMillan: London. Mitchell, Alanah, Zigurs, Ilse (2009) Trust in virtual teams: solved or still a mystery? The database for advances in information systems, Vol. 40, No 3, 61‐83. Paghaheh, Jamali Morteza, Shafiezadeh, Ehsan, Mohammadi, Mohsen (2011) Information technology and its deficiencies in sharing organizational knowledge. International journal of business and social science, Vol. 2 No. 8, 192‐198. Savery, John R. (2006) Overview of problem‐based learning. Definitions and distinctions. Interdisciplinary journal of problem‐based learning, Vol. 1. Iss. 1. Savery, John R. & Duffy, Thomas M. (2001) Problem‐based learning: an instructional model and its constructivist framework. Center for Research on Learning and Technology Technical report No. 16‐01. Indiana University. Senge, Peter M., Lichtenstein, Benyamin B. Kaeufer, Katrin, Bradbury, Hilary, Carroll, John S. (2007) Collaborating for systemic change. MIT Sloan management review, Vol. 48. No. 2., 44‐53. Snowden, David J. and Boone, Mary E. (2007) A leader´s framework for decision making. Harvard business review. Nov. 2007, 1‐9. Venkataraman, Sankaran. (1996) Some methodological challenges for entrepreneurial process research. Symposium paper presented at the Academy of Management meeting, Cincinnati, August 1996. Venkataraman, Sankaran. (1997) “The distinctive domain of entrepreneurship research: An editor's perspective.” In Advances in entrepreneurship, firm emergence, and growth, eds. Katz, Jerome A., and Robert H. Brockhaus, 119‐138. Greenwich, CT: JAI Press. Wang, Yinglei, Haggerty, Nicole (2009) Knowledge transfer in virtual settings: the role of individual virtual competency. Info systems journal, 19, 571‐593. Wang, Qiyun, Zhu, Zhiting, Chen, Li & Yan, Hanbing (2009). e‐learning in China. Campus‐wide information ystems, Vol. 26 No. 2. 77‐81 Zigurs, Ilse (2003). Leadership in virtual teams: oxymoron or opportunity? Organizational dynamics, Vol. 31, No. 4, 339‐ 351. Zhen, Lu, Jiang, Zuhua & Song, Hai‐Tao (2011) Distributed knowledge sharing for collaborative product development. International journal of production research. Vol. 49, No. 10, 2959‐2976.

117

E‐Learning’s Indispensability in Crisis Management Education Maria Nathan School of Business and Economics, Lynchburg College, Lynchburg, USA [email protected] Abstract: Consider that there are certain challenging contexts that don’t merely benefit from e‐learning. It may be argued that learning in such contexts cannot be as effective without e‐learning. Organizational crisis is just one such challenging context for which the diverse offerings made available through e‐learning aren’t just useful and engaging, but essential. This paper will explore crises and their key properties that make meaningful learning about them so difficult. E‐learning tools can be found uniquely capable of permitting both educator and learner to tackle these challenges in a way that virtually no other learning tool can. Furthermore, e‐learning becomes an essential in the organizational crisis context itself – as managers prepare for, manage, and, alas, learn from crises that may occur nonetheless. Keywords: e‐learning, crisis learning, crisis management education

1. Introduction The term crisis is generally used to refer to any unstable situation that poses grave danger or challenge, regardless of likelihood of its occurrence (Comer, 2010). A crisis is a low probability, high impact event that threatens the viability of an organization. It is also a “perception of an unpredictable event that threatens important expectancies of stakeholders… although identities of all relevant stakeholders to the crisis may not be readily obvious” (Coombs, 2007:2‐3; Pearson and Clair, 1998). Crises include weather and natural disasters, terrorism; human induced crises such as corporate scandals, accidents, mismanagement such as risk denial and epistemic blind spots; white‐collar crime, workplace misbehavior such as deviance, aggression, antisocial behavior, violence, and “packages of disasters” (Clair, Maclean, and Greenberg, 2002). Organizational crisis preparedness is optimal. However, crises may occur nonetheless. It is essential that crises be managed effectively from their earliest moments; if mismanaged, a single crisis can set off a chain reaction of other crises (Pearson and Mitroff, 1993). Just as a core goal of management educators is to “teach… students how to learn to manage” (Payette, 1993:452), a core goal of crisis management educators is to teach students how to learn to manage crises. Before exploring unique opportunities for crisis education made possible through e‐learning, staples of the crisis classroom are explored.

2. Traditional business classroom tools used in crisis management education Little has thus far been published that can help us to teach our students about crises (Comer, 2010). Traditional tools of business education may be adapted: (1) Class Discussion underscores the wide experience base available among students who share relevant stories that can deepen their own and others’ insights. A challenge here is for students to use specific crisis course concepts in their analysis; (2) Case Analysis: Whether oral or written, brief or extended, the case is useful in focusing students upon tools through which the facts of the case flow; (3) Film is powerful in its ability to evoke viewer emotion. There are many crisis‐centric films such as Thirteen Days in October (about the Cuban Missile Crisis), Identity Thief (about identity theft), Inside Job and Margin Call (about the 2008 economic crisis)(Clemens and Hamakawa, 2010); (4) Reflection and questioning of reasons and practice permit learners to be informed, accountable, absorbent of new learning, and critical of prevailing perspectives and propositions, (Antonacopoulou, 2010; (5) Journaling: “Go through the motions and you’ll get the emotions” is the conventional wisdom of journaling and suggests one reason why it is a powerful tool. In the learning journal students are instructed to record written reflections over a period of time (e.g., a full semester) (Hubbs and Brand, 2005; Moon, 1999). The professor focuses the exercise through specific assignments corresponding with crisis themes. Students may also be challenged to engage in self‐experiments in which they seek to audit a particular behavior for several days or longer; (6) Behavior Modeling refers to a process in which the learner attends to a model, retains the memory of the observed model, and then seeks to reproduce the behavior (Nathan and Kovoor‐Misra, 2002). Modeling the behaviors of an experienced communicator, for example, can permit the learner to experiment with distinct expert behaviors; (7) Role‐play has become increasingly popular as a learning tool (Druckman and Ebner, 2008; McSharry and Jones, 2000). Getting involved in a role ‐play is “creating experiences” (Petersson, 2006:101).

118

Maria Nathan The spontaneous nature of role‐play challenges students to understand what to do and how to effectively translate crisis concepts into practice. Alternatively, interactive drama features scenes from crises being performed by trained actors before a student audience (Boggs, Mickel, and Holtom, 2007); (8) Scenario Planning allows crisis response students to evaluate multiple potential outcomes of volatile situations to underscore the fact that the future is unpredictable, unstable, and even unknowable (Moats, Chermack, and Dooley, 2008). Note that each of the above can be readily adapted for use in complement to existing management courses. Students are taught how to prepare for and manage crises through using their own experiences, through experimentation, and while deepening their appreciation of what they may have learned vicariously through the crisis experiences of others (Nathan and Kovoor‐Misra, 2002). To conclude, the crisis course can use experientially rich student opportunities to work with complex concepts for phenomena that are low probability, unpredictable, and even unknowable. It can be argued that there are yet additional pedagogical supports needed for the crisis context – ones that are more riveting, real world, technologically savvy, and challenging to the learner. The unique support requirements for crisis education are next explored.

3. Crisis management education’s support requirements Crisis management educators seek to compel students to accept the real potential for devastation that may be wrought by crises. As students study crises it is possible that they will not have been personally affected nor moved to become more crisis conscious unless they become engaged experientially. The general challenges that e‐learning promises to tackle are: (1) Ease students from the safe harbor of the classroom to the journey outside of comfort zones as means to reveal, confront, and test self‐limitations in thoughts and beliefs;(2) Engage students in an absorbing and emotionally‐gripping exercise within a simulated organization so that they are more powerfully compelled to stop all else to learn; (3) Prompt students to more consciously begin their lifelong crisis learning process through self‐initiation of individualized learning; (4) Prepare students to be technologically competent in the face of organizational crisis; (5) Assist students to manage their crisis course workload with technologies such as podcasts that allow them to “learn on the move,” (Cebeci and Tekdal, 2006); (6) Prompt students to explore how social media, social networks, and communities affect organizational crisis vulnerabilities. While traditional classroom methods can underscore key concepts and relationships, models and theories, it can be difficult for students to fully appreciate the drama of crisis, the urgency of time in the event of crisis, and the necessity to be prepared (both internally and externally) in advance of crises that may happen. e‐ Learning supplies the exoteric and infinitely accessible classroom of the Internet.

4. Utilizing e‐learning’s unique potential in crisis management education Web enhanced screen‐casting, videos, webcams, webinars, virtual classrooms, podcasts, e‐portfolios, simulations, blogs, wikis, discussion boards, smart boards, web‐supported texts, audio/video courses, social networking, e‐Twinning, Skype, and more—the list of useful e‐learning technologies grows longer with each passing month. In addition, academic journals such as Learning and Instruction, Learning Media & Technology, Computer‐ Assisted Learning, and Interactive Learning; conferences such as the International Conference on e‐ Learning, the e‐Learning Guild, and the Learning Solutions Conference, are developing as means to help educators utilize e‐learning’s promise to enrich crisis education: 

Bring a real world application into the classroom (e.g., conference calls or Skype with crisis management experts; explore the Congressional Records in search of a particular type of crisis as experienced in U.S. history such as high school shootings; MOOCs). A relatively recent e‐learning entrepreneurial venture has been the MOOC (Massive on‐line open courses) providers such as Coursera, Udacity, and edX . These MOOCs open higher education and even course learning materials to the masses for free.



Make the experience of crises more alive, more visceral, more fast‐paced, and more current for students (through blogs, You Tube clips and films, Hulu films, TV reality shows such as Kitchen Nightmares or Bar Rescue, as well as current news.



Allow students to learn about crises’ emergent and urgent properties in a flexible and individually accommodating format (such as Udacity in its flexible start and end dates and completely self‐paced).

119

Maria Nathan Some MOOCs have no time constraints and learners can freely go back to previous materials including quizzes they have already taken. Both Coursera and edX permit the learner to use a flexible self‐study approach. edX has a foreign language option for those who could learn more efficiently with their first (non‐English) language. 

Enable students to become more knowledgeable about current technologies that are also important in organizational crisis management. The Internet has actually served to create a more crisis prone environment for organizations due to information symmetry – the public now knows more about an organization and its operations. Dangerously, students may not appreciate that the world of the Internet and social media has changed everything (Crandall, Parnell, and Spillan, 2010). Organizations need to monitor the Internet environment for what is being said about them by customers, consumer activists, competitors, suppliers, industry experts, and even potential extortionists or terrorists. Search engines such as Google and Yahoo can be scanned regularly for discussion forums, YouTube blogs, Twitter, and hate sites.



Provide a more interactive environment and more hands‐on inquiry‐based approaches to learning about crisis management (e.g., simulations, virtual mock disaster drills, and Webcasts). Organizational simulations (tabletop exercises, drills, functional exercises, and full scale simulations) improve organizational coordination, reveal gaps in resources, clarify roles and responsibilities, engage the participation of multiple external stakeholders, and train personnel (Lagadec, 2013).



Permit easier virtual collaboration with students and experts in other schools, agencies, networks, and nations… For example, learning communities can be developed among those who take MOOCs (massive online open courses). edX and Udacity MOOCs also offer course forums that allows enrollee interactions.

More and more an organization’s crisis management team (CMT) will include some members who are participating in a virtual format. Part of the CMT may meet in the organization’s command center while other members of the group are located across the globe in various other locations (Hiltz, 2006). Members are linked through software, intra‐net, Internet, cell phones, conference calls, and faxes. (7) Promote less helplessness and more control over students’ own learning experiences (as in asynchronous learning) (Armstrong, 2011) and outcomes. e‐Learning is learner‐centered with a system design that is interactive, self‐paced, repetitive, and customizable (Twigg, 2002). Abundant information can be transferred anywhere and anytime. The MOOC Coursera permits students to make their experience count for more in the job market as they build their resumes with such devices as proctored exams, certifications, a career matching service, and in the not to distant future – university course credit. The podcast is another individualized, asynchronous student learning device that offers mobile formats that are accessible anytime. Topical lectures and panels can be sent to iPods, iPads, and other electronic devices that students regularly carry around campus or in transit to home, off‐campus, and other venues. For example, the American Management Association offers its Podcast Solutions for crisis‐related talks such as Laurence Miller on Handling Difficult to Disturbed Employees and Jane Jordan’s Managing Media Thru a Crisis.

5. E‐learning limitations in crisis management education E‐Learning is here proposed as a complement to traditional classroom education. It is offered as a large, significant component within a blended learning environment because e‐learning itself is not without its limitations, including the possibility that students can’t afford or do not have access to a current technology. Possibly students may lack computer literacy. Inherent limitations of technology such as computer crashes and failed Internet connections may also exist. Additionally, not all students may possess the needed level of maturity and independence as learners. For example, the loose structure of many MOOCs has resulted in a relatively low student course completion rate. Because e‐learning has traditionally been considered a low inter‐personal interaction resource, the crisis educator must consider how opportunities for meaningful interaction with the professor and among students is necessary for more soft‐skill learning (Brown, et al. 2013). Furthermore, teachers may not be sufficiently trained in the use of online formats. Finally, there is not sufficient research on the actual effects of crisis e‐learning on student learning, nor is there sufficient research on the means by which crisis e‐ learning can be used by students for more effective

120

Maria Nathan organizational crisis management. More attention to the theory behind the technology is needed. For example, a theory that studies factors that influence usage, a Transtheoretical Model of change (Brown and Charlier, 2013), focuses upon pre‐training intervention that can boost e‐learning use. Students must not feel false confidence or over‐confidence having mastered e‐learning‐based activities, but can feel more confident that they will be able to contribute capably in the possible event of a crisis.

6. Conclusion Wrote one Chronicle of Higher Education contributor, “online education won’t replace college –‐ yet” (Youngberg, 2012). Still it is not difficult to imagine how e‐learning progress may accelerate in the not so distant future (Allen and Seaman, 2003). In fact in meta‐analytic studies of e‐learning distance education compared to classroom education, researchers have found that learning outcome differences are inconsistent or minimal if potential confounds are controlled (Allen, et al., 2006). More and more, business school key constituents are expecting managerial relevance and a critical impact on management practice. The experientially rich exercises here made possible through the Internet might be referred to as psychological mountain climbing that engages students in intellectually and emotionally gripping work. Crisis e‐learning can make a critical contribution in both classroom and ‘real world’ through its highly relevant, action‐oriented, technologically advanced, and student‐centric properties. “I have no idea where I am and I have no idea who can help me” wrote Karl Weick (1993:634‐635) of the feelings sometimes expressed by managers in the midst of crisis. e‐Learning has a grounding influence through its podcasts, reality programs, community forums, and more. The e‐learning educated manager is supplied an anchor through both intellectually and viscerally gripping content. Crises could be prepared for with e‐learning in a way that might not be accomplished without e‐learning’s indispensable, highly varied, and ever‐innovative Internet‐based resources.

References Allen, M., Bourhis, J., Mabry, E., Burrell, N. A., & Timmerman, C. E. (2006) “Comparing ¡distance education to face‐to‐face methods of education”, In B. M. Gayle, R. W. Preiss, N. Allen, I E. and J. Seaman, (2003) “Sizing the opportunity: The quality and extent of online education in the United States, 2002 and 2003”, The Sloan Consortium, Boston, MA. Antonacopoulou, E. P. (2010) “Making the business school more “critical”: Reflexive critique based on phronesis as a fondant for impact”, British Journal of Management, 21, S6‐S25. Armstrong, J. S. (2011) “Natural learning in higher education,” Encyclopedia of the Sciences of Learning. Heidelberg: Springer. Boggs, J., Mickel, and Holtom (2007) “Experiential learning through interactive drama: An alternative to student role plays”, Journal of Management Education, 31, 6, 832‐858. Borodzicz , E. and van Haperen, K. (2002) “Individual and group learning in crisis simulations”, Journal of Contingencies and Crisis Management, 10, 3, 139‐147. Brown, K. G. and Charlier, S. D. (2013) “An integrative model of e‐learning use: Leveraging theory to understand and increase usage”, Human Resource Management Review, 23, 37‐49. Burke, S. C. and S. L. Snyder (2008) “YouTube: An innovative learning resource for college health education courses”, International Electronic Journal of Health Education 11, 39‐46. Cebeci, Z. & Tekdal, M. (2006) “Using podcasts as audio learning objects”, Interdisciplinary Journal of Knowledge and Learning Objects, 2, 7‐57. Clair, J. A., Maclean, T. L. and Greenberg, D. N. (2002) “Teaching through traumatic events: Uncovering the choices of management educators as they responded to September 11”, Academy of Management Learning and Education, 1, 1, 38‐54. Clemens, B. and Hamakawa, C. (2010). “Classroom as cinema: Using film to teach sustainability”, Academy of Management Learning and Education, 9, 3, 561‐563. Comer, D. R. (2010) “Special issue: Crisis management education”, Journal of Management Education, 34, 5, 782‐784. Coombs, W. (2007) Ongoing Crisis Communication: Planning, Managing, and Responding (2nd edition). Thousand Oaks, CA.: Sage. Crandall, W., Parnell, J.A., and Spillan, J. E. (2010). Crisis Management in the New Strategy Landscape. San Francisco: Sage Publications. Druckman, D. and Ebner, N. (2008). “Onstage or behind the scenes? Relative learning benefits of simulation role‐lay and design”, Simulation and Gaming, December, 39, 4, 465‐497. Glenn, J. (2000) “Teaching the net generation”, Business Education Forum, 54(3), 6‐14. Hiltz, S. (2006) “Partially distributed teams: A tutorial, hands‐on experience, and discussion of their use in emergency response”, Paper presented at ISCRAM‐TIEMS, Summer Conference, Tilburg, the Netherlands.

121

Maria Nathan Hubbs, D. L. and Brand, C. F. (2005). “The paper mirror: Understanding reflective journaling”, Journal of Experiential Education, 28, 1, 60‐71. Lagadec, P. (2013) Simulations/Exercises, Tsunami Research Program, nctr.pmel.noaa.gov/…W3_Tsunami_Eva… Accessed December 13, 2013). McSharry, G. and Jones, S. 2000 “Role‐play in science teaching and learning”, School Science Review, 82: 298. Moats, J. B., Chermack, T. J. and Dooley, L. M. (2008) “Using scenarios to develop crisis managers: Applications of scenario planning and scenario‐based training”, Advances in Developing Human Resources, 10, 3, June, 397‐424. Moon, J. (1999) Learning Journals: A Handbook for Academics, Students, and Professional Development. London: Kogan Press. Nathan, M. L. and Kovoor‐Misra, S. (2002) “No Pain, Yet Gain”, The Journal of Applied Behavioral Science, 38, 2, 245‐266. Payette, A. (1993) “To teach experience!” Journal of Management Education, 17, 440‐453. Pearson, C. M. and Clair, J. A. (1998) “Reframing crisis management”, The Academy of Management Review, 23, 1, 59‐76. Pearson, C. M. and Mitroff, I. I. (1993) “From crisis prone to crisis prepared: A framework for crisis management”, Academy of Management Executive, 7, 1, 48‐59. Petterson, J., (2006) “The art of experience”, In T.Fritzon & T. Wrigstad, eds. Role, Play, Art: Collected Experiences of Role‐ Playing. Stockholm: Föreningen Knutpunkt. Ch. 10. [Online]. Available at: http://jeepen.org/kpbook/kp‐book‐ 2006.pdf [accessed 19 September 2011). Twigg, C. (2002) “Quality, cost and access: The case for redesign”, In The Wired Tower. Pittinsky MS (ED.), Prentice‐Hall: New Jersey, pp. 111‐143. Weick, K. (1993) “The collapse of sense‐making in organizations: The Mann Gulch disaster”, Administrative Science Quarterly, 38, 628‐652. Youngberg, D. (2012) “Why online education won’t replace college—yet, The Chronicle of Higher Education, Commentary, August 13.

122

Crowdsourcing College Examinations Using Technology to Improve Assessment and Higher Learning by Students Timothy Olson University of Minnesota, Carlson School of Management, Information Decision Science Department, Minneapolis, Minnesota, USA [email protected] Abstract: A highly ranked Management Information Systems Department at a leading american university has implemented crowdsourcing, where students generate the questions in an introductory course. This paper describes the process and results after two semesters and participation of approximately 1000 students. The course blends students attending scheduled classroom lectures and significant use of an open source Learning Management Systems, (LMS). The LMS, accessible through the university’s website, contains all course material including readings, team exercises, videos, related websites, grades and online exams. At the beginning of the semester, students are assigned to a team of five. The total number of teams in all sections per semester is approximately eighty. Each team is responsible for preparing four questions, generating a collective pool of over 300 questions for the exam across multiple course sections. After editing and removal of duplicate questions by teaching assistants a study guide of all the questions without the answers is provided to the students in advance through LMS. An on‐going issue in higher education is academic dishonesty. Crowdsourcing properly used in conjunction with an online LMS can significantly reduce the risk of students cheating on exams by increasing the complexity of sharing answers in the test setting among students. Students and teams benefit from performing higher on learning activities by engaging more thoroughly with the subject matter when comprising complex questions. Using the philosophy “to learn a subject is to teach it”, or in this case, by writing exam questions more actively engaging in personal learning whilst dictating what fellow students learn. Teams are not given credit for questions that are too simplistic, where explicit guidelines and prompts of how to write good multiple choice questions are provided. Students are given access to all the exam questions, but not the answers, so they have a comprehensive study guide before the exam. The course uses technology to improve the quality of education. Crowdsourcing is a compelling tool used in education to increase student collaboration and higher learning. Students see the input side of preparing examinations and the output in two different forms: a study guide and individually LMS generated exam. Academic dishonesty is significantly minimized by the use of the LMS’s student logs, documenting: date, time, content, duration, facilities and monitoring student IP addresses location. Keywords crowdsourcing, learning management system, digital classroom, blended courses, test banks, academic dishonesty, higher learning

1. Introduction Digital classrooms and online teaching have become pervasive throughout all levels of education. This has led to the need to develop new methods for online testing of students’ knowledge. Online testing provides numerous benefits both for the student and instructor, but also presents new challenges with technology and the administration of exams. This paper describes how crowdsourcing examination questions have been successfully utilized at a major business school with large classes and the lessons learned over two semesters. The digital classroom is not going away. When utilized correctly, the digital classroom can effectively improve student engagement. Educators must be prepared to both understand and use classroom technologies to the fullest. “We must make sure that teachers who have the responsibility of training our children to be productive members of this society are consciously aware of various technologies as they emerge and are also able to demonstrate their different uses to their students” (Okojie and Olinzock 2006). It is essential that educators recognize and understand the benefits and challenges created by technology in the classroom and how best to utilize these tools. Online education in the classroom has changed the way teachers and students transfer, apply, and demonstrate knowledge. South Korea’s Ministry of Science and Technology recently announced that “its entire curriculum will be available on computers, smartphones, and tablets by 2015”, said an Education Ministry official. “That’s why Korean students, who are already fully prepared for digital society, need a paradigm shift in education” (Janey 2013). North American, European, and African educators are not far behind. Amazon is working with public school districts to eliminate physical textbooks, saving school districts millions of dollars. In our department’s introduction to Management Information Systems course we annually save students in excess of $200,000.00 annually by using online course material. “Electronic textbooks, classroom wikis, and Learning Management Software (LMS) are just a

123

Timothy Olson few examples of the changing teaching environment rapidly becoming mainstream in all levels of education from primary school through universities all over the world” (Van Kamp 2011). To say that technology is an important topic in higher education is an understatement. Technology is complicated, so educators need to be flexible but also thoughtful as they tackle new initiatives and draft new policies. Educators are challenged by staying current with technology, while at the same time making prudent investments in infrastructure and software that needs to last for years. Jason Vaden addresses that challenge and highlights the importance of technology in his research paper “A Model Assessment Tool for Classroom Technology Infrastructure in Higher Education” (Daly 2013). The use of technology in the classroom however is only a tool; how we apply it will determine the future of education. “Educators need to complement these tools with innovation in how we use the physical space of learning environments, and how we engage with the people in them” (Rotman 2013). Utilizing technology in the classroom offers a learning experience that can boosts engagement, track performance, and provide up‐ to‐date content with the potential of significant cost savings. However recent examples of students accessing digital textbook test banks have raised questions about the security of such widely available banks. “A disturbing aspect of contemporary pedagogy is the reliance and overuse of publisher provided test banks for exam questions” (Ray 2010). Instead of using test banks for examination questions this paper asserts that crowdsourcing of questions provides an opportunity to engage students using technology with several benefits. Benjamin Franklin said “Tell me and I forget, teach me and I may remember, involve me and I learn” (Franklin, circa 18th century). This old adage can be applied to the blended digital classroom by involving the students using the new social networking technique of crowdsourcing. Crowdsourcing of student written examination questions is an excellent technique in a blended teaching environment. From our research and application of crowdsourcing there are several benefits to be gained. Students overwhelmingly support the idea, instructors are able to spend time editing question content instead of generating the exam, students were provided with hundreds, but all, the potential questions as a study guide, and cheating on exams becomes extremely difficult because of the sheer number of questions and that the questions are unique for each individual exam. Professors often rely exclusively on test banks from textbook publishers for their own exams. These test banks are widely circulated and easily obtainable by students. A recent example at the University of Central Florida describes the potential problem. Administrators at the university say one or more students in a senior‐level business class purchased a test bank for the textbook "Strategic Management: A Dynamic Perspective Concept and Cases." Then it was shared with some 200 classmates in the strategic management class. The course instructor used that same 300‐question test bank for several years which became common knowledge to the school’s students. Donald McCabe, a business professor at Rutgers University, who studies cheating on college campuses, said the “UCF case is one of the biggest in recent years”. McCabe said “his surveys on college cheating suggest the problem is serious and perhaps getting worse. About 21 percent of students admit to serious test cheating, with more issues of cheating on large campuses with large lecture classes” (Postal 2010). Test banks comprised of old exams and practice tests have been mainstays on campuses for years, pulled together by fraternities, study groups, or sold and promoted by publishing companies. In our own department, three students in a class of forty were found to have gained access to the instructor edition of a programming textbook and were accessing the test bank during a final exam. One student had to be told three times not to reference the test bank during the exam. The student was reported to the university’s office for academic integrity which resulted in the student failing the course. These two examples point out that instructor cannot merely rely on textbook test banks to prepare exam. Kendall Washington White, Associate Dean of Students at the University of Arizona, says from his experience, “you can't just use questions from a test bank, especially if they are accessible and something that can be purchased by anyone,...I think that's not the best teaching" (Postal 2010).

1.1 Crowdsourcing An innovative and far more productive alternative to the use of publisher’s textbook test banks is to source exam questions and answers from the students. Keith Sawyer, author of the book Group Genius wrote, "All inventions emerge from a long sequence of small sparks...collaboration brings small sparks together to generate breakthrough innovation" (Sawyer 2012). Not unlike Threadless, the T‐shirt company that has

124

Timothy Olson customers submit designs and vote on the style of the week, students are very capable of generating exam content. Our department has been using crowdsourcing on half the exams the past two semesters and found it to be a very useful and a very effective teaching technique complementing the digitally blended classroom. What is encouraging about this approach is that the resulting pool of questions reflects a strong comprehension of the topics studied. Another instructor who uses this technique recently wrote, “…when the students collaborate in this way, they both have to do the reflective, synthesizing work of question‐writing (which is better than cramming) and to come to an implicit agreement about what our course was about” (Okojie 2011). As compared to the publisher test banks, which necessitate being protected and kept from the student, crowdsourcing does the opposite. We are able to provide the questions to students in advance as a study guide. An insightful comment recently published reinforces this benefit, “you can publicize it so people realize it’s an open process and they can see what other people are thinking. Ultimately, it’s a very explicit form of participation. It’s truly an exercise in public democracy” (Bensen 2013). Crowdsourcing exam questions takes advantage of many of the same technological features that characterize “social media,” through which users can interact with those of similar interests. The following definition of crowdsourcing is useful for this paper. Crowdsourcing is a sourcing model in which organizations use predominantly advanced Internet technologies to harness the efforts of a crowd to perform specific organizational tasks. (Saxton 2010) We have found students are already familiar with the crowd sourcing technique, whether they know it or not, from their interaction on the web on numerous sites which solicit their opinions and voting for products, music, articles, etc. Social media sites place emphasis on the “social” aspect of community crowdsourcing which involves the management of a community via collaborative technologies to elicit the community’s knowledge and/or skill sets and thus fulfill a pre‐identified goal. Examples of how this is done in a blended classroom are described later in this paper.

2. Academic environment Our university and school of management are leaders in the field of Information Technology and actively experiment in incorporating digital technology into the classroom. The university is one the largest and oldest users of a popular open source Learning Management System utilized to deliver content and test of students’ knowledge. The school of management facility is equipped with wireless connectivity easily accessible for all students. The LMS has exam features that allow for generating questions in random order for each student by pulling a predetermined number of questions from a category (topic) covered on an exam. The school and LMS support the idea of bring your own device (BYOD) to school. Students are able to access the LMS with any manufacturer of laptop computer, tablet, or smartphone. Our department has utilized a blended classroom to teach large introductory sections of management information systems (MIS) with all content provided via the LMS lectures slides, lectures, videos, quizzes, team exercises, and grade book. The school has a set of required courses, including MIS, which require teaching approximately one thousand students each year. Given the topic of introduction to MIS, the course covers a variety of topics fundamental to MIS, plus current events. With the variety of topics covered, frequent exams have proven to be good strategy for assessing student performance. The course is designed to have an exam every two weeks for a total of seven exams. In addition to the fourteen topics, four IT news articles are discussed every week, also included on the exams. Generating and administering seven exams a semester with potentially 500 students per semester can be challenging for the instructor. We have experimented with students generating exams (crowdsourcing) as group exercises and have found the results to be extremely positive. Students are placed in teams of five to collaborate on exercises and group exams. Teams are given exercises to write multiple choice questions and answers for assigned topics within guidelines associated with a grading rubric.

2.1 Process The process of using crowdsourcing exams has three major phases,

Students generate questions and answers

Review and editing of student submissions

Execution and analysis of results

125

Timothy Olson This case study is based upon the following Introduction to MIS course. Phase One ‐ Every semester the course has between four and six sections of approximately 100 students. The process starts with students being assigned to teams of five at the beginning of the semester. A given instructor could have between 80 to 100 groups each semester. Group exercises are scheduled one week in advance of the next exam (exhibit A). Each exam covers two topics, IT News articles and guest speaker presentations; all material is available on the course LMS. Groups are assigned to a particular topic (exhibit B). Students are provided instructions on how to write multiple choice questions and instructions on how to submit their questions using Google Docs. (Exhibits C & D). Groups have a deadline for submitting their assignment. One group member submits the questions and Google Doc replies with an email confirmation. The Google Doc provides the following data used to provide content for exam questions and answers, evaluate group performance, and create a study guide: (exhibit E)

date/time stamp when submitted (grading criteria)

course section number

group number

category by (topic of IT News)

question

answers

email of submitter

score column (entered by review)

Phase Two ‐ The course has two student Peer Assistants (PA’s) assigned to each section along with one Administrative PA for the course. For every crowd sourced exam, PA’s are paired up and assigned a topic or IT News to review and edit. (Exhibit B) The course Administrative PA distributes the Google Doc Spreadsheet to the PA’s shortly after 5:00pm on Thursday. PA’s are responsible to review assigned group submissions and remove duplicate questions, edit for clarity, grammar and accuracy of questions and answers. At the same time, the Administrative PA creates a study guide in a Word Doc of all the several hundred submitted but unedited questions, no answers. Thursday evening, the new study guide is available on the course website. By midnight Saturday PA’s return to the instructor and Administrative PA the reviewed, edited questions and answers. The Administrative PA enters the questions and answers by topic/category into the LMS. The instructor does a final proof read of the exam Sunday and then makes any late changes Monday morning in the LMS. Phase Three – The final stage starts with configuring the exam. There is a total of twenty‐five multiple choice questions. Each topic and IT News is defined as one of four categories. From each category a specific number of questions is randomly pulled for each student. Generally there are over 300 questions submitted and after editing and review the number has been reduced to approximately 125. The exam is configured to be open only during a specific time period within each classroom section. The exam includes a password that is only given out in class and a question #26, which is written on the board. This is done to ensure students are actually in the classroom. A further check is made by reviewing the IP Address Log and identifying any suspicious IP Addresses. We can identify students trying to access the exam ahead of time and make them aware they have been identified and are being monitored. On the rare case when a student has the wrong answer for #26 it is easy to tell from the IP Log if they were accessing the exam from the classroom. Students are given thirty minutes and only one chance to take the exam. The LMS provides students with their score immediately after clicking “submit”.

2.2 Results Individual scores were compared in two different ways

within the same semester with the same students

four different semesters final exams, two with student sourced questions and two with instructor prepared exams

126

Timothy Olson What was discovered is there was no statistical difference between students preparing the questions as opposed to the instructor either within the same semester or compared to different semester with the same exam and material. (exhibit E) One of our initial concerns was the students would do noticeably better with the crowd sourced exam since they had access to all the questions, but the statistical difference was very narrow. Four hundred students were surveyed about which type of exam they would prefer. A. An exam with questions crowd sourced by students and study guide made available in advance or B. An exam prepared by the instructor with no study guide. The results, two thirds of the students wanted to crowd source the final. (Exhibit F) We believe that academic dishonesty is significantly reduced in the blended classroom using a LMS and crowd sourced exams in the following ways:

The test bank is new, large, and available only for less than a week

Questions are randomly generated for each student

Students only have thirty minutes and one opportunity to take the exam

Attempting to bring in answers to several hundred questions is extremely difficult while trying to keep this concealed in the classroom

Tracking of IP Addresses to location where the test was taken and user

Exam passwords and the question with the answer written on board are only given out in the classroom

Students are not provided the answers, however they can be reviewed with PA or instructor

The study guide’s usefulness expires after the last section takes the exam

No additional expenses were incurred to use crowdsourcing of questions. The University has an existing agreement with Google for the Google Doc application. The University supports the open source LMS web based software with testing functionality. The school provides the wireless classrooms with sufficient bandwidth. The department funds the course PA’s. Students already have access to a mobile device with internet access.

3. Issues There are several issues or challenges when crowdsourcing of exams as described in this case study. With exams every two weeks there is very little flexibility in the schedule. Because of this, a maximum of four exams are crowd sourced during the semester and the other three are prepared by the instructor. A valid criticism heard from some students is the quality of the questions. A reasonable percentage of questions are ambiguous, poorly worded, grammatically incorrect, or not applicable. Mitigating this issue involves the TA’s being diligent in their task of reviewing and editing. Too often teaching assistants are concerned about eliminating duplicate questions and following the rubric and can miss or overlook the quality of the questions. Another area of concern are the answers. It is simple to provide the correct answer, the hard part is often writing “good” wrong answers. Students are very good at eliminating wrong answers in multiple choice questions. We strive to have multiple choice questions that require higher levels of understanding of the material. The instructor spends a great deal of time rewriting answers to ensure students have to think about the questions and answers and not simply use the process of elimination in taking the exam. Another issue is what version of the study guide to provide:

“unedited” / early (Thursday evening)

“edited” / final list of questions (Sunday afternoon).

Our concern with the unedited version is the quality of and duplicates questions. The unedited format has over 300 questions. The edited / final version has approximately 125 questions but is not available until Sunday afternoon. When the exam is given to some sections on Monday this is a very narrow window for students to use the edited/final version. In using both unedited and edited versions students have suggested and we agree on using the unedited approach. In talking to students before and after exams it is amazing how many students actual go through the entire 300+ questions and answer each of them.

127

Timothy Olson Technical issues are always a potential risk, especially supporting students bringing their own device to class. The University provides good LMS software for course content and student assessments. The wireless infrastructure is dependable and has the capability to handle the number of users. Alan Greenberg, senior analyst and partner at Wainhouse Research, says that “colleges should closely examine their network infrastructure before moving forward with a web‐based approach. IT managers need to think about the new web‐based technology in terms of their overall BYOD policy.” Greenberg says, “If students will be using more devices in the classroom that means the college needs to upgrade its wireless access points and overall bandwidth.” (Zurier 2013)

3.1 Observations From our experience a few observations come to mind.

The University, school and department must support the blended classroom with a solid infrastructure and proven network accessible to a hundreds of students in the classroom.

A flexible learning management system with features that support large questions test banks, are available to students only on days and times of the test. The ability to randomize questions within predetermined parameters, and detailed student logs is essential.

Make it the student’s responsibility to bring any device capable of accessing the internet / LMS.

Courses utilizing the described process in this paper engage the student in several interesting ways. The students work collaboratively in crowdsourcing the questions, while creating questions from the assigned material and developing appropriate answers. As in Benjamin Franklin’s quote, “involve me I learn”. A crowd sourced study guide available for a specific exam but is worthless to keep for future use.

Very detailed instructions for both students and reviewers are necessary to minimize poor results in questions and answers. Adherence to the schedule and clearly defined guidelines are essential to manage the number and quality of the questions and allow for sufficient time to review and edit the content. Reviewers (TA’s) must provide quality review and edits of question and answer content. If the question and answers content is poor, students recognize this immediately and are dissatisfied with the quality of the examination.

Appendix 1 Exhibit A ‐Schedule

128

Timothy Olson Exhibit B ‐ Group & Peer Assistant Crowdsourcing Assignment Topic or IT News Groups Peer Asst. a. Topic 13 1 – 5 Kim & Tim b. Topic 14 6 – 10 Don & Dave c. IT News 13 11 – 14 Sally & Sue d. IT New 14 15 – 18 Jane & John Exhibit C – Scoring Guidelines Scoring Group Questions – 5 Points

Submitted four questions, on time, assigned category – 1 point

No definition questions – 1 point (example: What is ERP?)

No “Not” questions – 1 point

Topic questions ‐ Two questions that start with, “From the article or “In the article … ‐ 1 point

IT News questions ‐ Submit questions from at least two different news items, from PowerPoint slides – 1 point

Provide at least four answers for each question ‐ 1 point

Exhibit D ‐ How to Submit Questions Guide to Completing Group Exercise to Collaborate on Course Examinations Step 1: Completing Exercise (Crowdsourcing)

Identify assigned topic or IT News – see course wiki site

Group discuss material and agrees on four questions

Group discuss potential answers

One group member documents questions and answers

Suggestions: How to write Multiple Choice Questions, http://theelearningcoach.com/elearning_design/rules‐ for‐multiple‐choice‐questions/ Step 2: Submit Exercise

Google Doc URL on course wiki site

First screen enter:

Section (course)

Group Number

Continue

Second screen enter:

Category – assigned

Question text

Answers

Correct answer

Submit

Repeat for questions 2‐4

Step 3: Feedback ‐ Moodle will send a confirmation email to the group member who submitted the questions.

129

Timothy Olson Exhibit F –Statistics same semester & students, different semesters same exam Crowd Sourcing vs Instructor.

Exhibit G ‐ Poll taken of 440 students in four different sections of the Spring Semester 2013, 335 (76%) voted for crowdsourcing the final, 105 voted for the instructor preparing the final exam.

References Benson, J. (2013) http://voxxi.com/2013/08/19/crowdsourcing‐education‐latino‐voices/ Daly, J. (2013) (Without Infrastructure, the Digital Classroom Is Just a Dream, http://www.edtechmagazine.com/higher/article/2013/04/without‐infrastructure‐digital‐classroom‐just‐dream‐ infographic Franklin, B. http://www.goodreads.com/author/quotes/289513 Janey B. (2013) http://noname6335.blogspot.com/2013/07/proposal‐for‐etextbook‐in‐china‐by.html Jones, J. (2010) The Chronicle of Higher Education http://chronicle.com/blogs/profhacker/exam‐questions‐outsourcing‐vs‐ crowdsourcing/28911 Okojie, M. (2011) http://scholar.lib.vt.edu/ejournals/JOTS/v32/v32n2/okojie.html) Leslie Postal, (2010) Orlando Sentinel, http://articles.orlandosentinel.com/2010‐11‐21/news/os‐ucf‐cheating‐online‐ 20101121_1_studies‐cheating‐large‐lecture‐classes‐test‐banks Ray, B. (2010) http://www.edutopia.org/blog/edchat‐education‐reform‐rob‐jacobs Rotman, S. (2013) http://readwrite.com/2013/02/19/whats‐missing‐from‐the‐new‐digital‐ classroom#awesm=~osmS8w7M7OGfE0 Sawyer, K. (2012) http://ascc.artsci.wustl.edu/~ksawyer/groupgenius/ Saxton, D. and Gregory L. (2010) Rules of Crowdsourcing: Models, Issues, and Systems of Control http://www.acsu.buffalo.edu/~rkishore/papers/Saxton‐et‐al‐Crowdsourcing‐ISM‐Forthcoming.pdf Van Kamp, J.(2011) http://www.digitaltrends.com/mobile/south‐korean‐school‐textbooks‐will‐be‐all‐digital‐by‐2015/ Zurier, S. (2013) Colleges Weigh Options on Student Response Systems, http://www.edtechmagazine.com/higher/article/2013/03/colleges‐weigh‐options‐student‐response‐systems

130

Comparing the Student Experience of Resident and Distance Learning Students David Owen and Elly Philpott University of Bedfordshire, Luton, UK [email protected] [email protected] Abstract: The aim of this paper is to compare how the learning experience of Distance Learning (DL), part time students may be different to those experienced by Resident fulltime students. Using post experience surveys as a basis for comparison, the effect of different delivery methods on course delivery, assignment instructions, feedback and group work are evaluated and conclusions drawn. The unit of analysis is Resident MBA students based in the UK and MBA students based in Oman who are mainly supported via DL but who also have some face to face teaching by local Omani tutors at four 2‐day residential schools. It considers a single unit ‐ Operations Management. This unit was delivered to the two sets of students by the same UK tutor. The authors conclude that while the student experience is different for DL students it can be maintained by the use of appropriate technology especially videoconferencing. They also found that the DL students were more dependent on online delivery methods even though they had some access to face to face delivery methods. Additionally, the more interactive the media used for delivery, the more effective it was perceived by the DL students. Most of the literature around DL and online learning consider the advantages and disadvantages of the approach or offers guidance on good practice. However, there is limited literature on the differences in the student experience between Resident and DL modes of delivery. Therefore this paper is of value to other educators who wish to consider DL and adds to the extant body of knowledge as an empirical case. Keywords: distance learning, face to face learning, MBA, student experience, online, videoconferencing

1. Introduction White et al (2010) describe Distance Learning as, “Any course, at any Higher Education (HE) academic level, delivered to students at a distance from the host institution, which has a significant component delivered to students online”. This definition is used in this paper whose aim is to compare how the learning experience of Distance Learning (DL) students at the University of Bedfordshire (UoB) may be different to those experienced by its Resident fulltime students whose main mode of delivery is face to face teaching. The paper is relevant in an era where DL (via online education) allows a university to reach a greater number of students who otherwise might not be able to attend classes or work toward a degree and also allows the pursuit of globalisation opportunities. The International Council for Open and Distance Education (ICDE, n.d) notes that the number of distance and online students is increasing every year and in Europe alone, there are currently, some three million students studying at a distance or on line. Chickering and Ehrmann (1996) proffered 7 principles of good practice in undergraduate support and this was further developed by Bailey and Card (2009), and Hoskins (2010). These guidelines are supplemented by JISC (2012) who offer guidance on good practice in VLEs and Mayes et al (2011) who noted that the 6 main themes to consider for effective online instruction include:

Learners and Instructors: How do learner and instructor characteristics influence online learning?

Medium: What cognitive and technological tools effectively support online learning?

Community and Discourse: What social and interactive tools effectively support online learning?

Pedagogy: What instructional strategies effectively support online learning?

Assessment: How do formative and summative assessment practices influence online learning?

Content: What unique opportunities and challenges arise in content‐specific learning?

Boling et al (2011) have shown that an emphasis on traditional text‐based content and lecture can lead to disconnect between online students, teachers, and course content and goals. They noted the need for highly interactive approaches. This is supported by the work of Pardasani et al (2012) who surveyed students and confirmed that online students prefer live video instruction. The need for interactive media is confirmed by the work of Borup et al. (2012) who noted that even asynchronous video‐based communication made students

131

David Owen and Elly Philpott feel that their instructors were more real, present, and familiar, and that the instruction was similar to face‐to‐ face instruction. The literature suggests that it is more difficult to address Mayes et al’s (2011) Community and Discourse theme via DL. Smith et al (2011) note that DL (online) students were less satisfied with group work than resident (face to face) students and Overbaugh and Nickel (2011) noted that students on blended and online sections of a university‐level educational foundations course, while generally satisfied with the courses, were only neutral with respect to the value of collaborating with other students or feeling part of the student group. Pardasani et al (2012) note that even with the availability of synchronous video media it was difficult to prevent some disconnect with respect to other students and group work. Comparisons of student preferences for DL and resident (face to face) instruction are inconclusive. Farley et al (2011) and Schwartz (2012) report that students either prefer resident instruction, or secure better results with resident instruction, in accounting classes. However, Robertson (2005) notes that students perceive the quality of their online learning in graduate education courses as similar or even superior to face to face resident campus courses. Paechter and Maier (2010) reported that while students generally appreciated online learning, they preferred face‐to‐face learning for group work. Finally, Dziuban and Moskal (2011) in a comparison of online, blended and face‐to‐face university courses concluded that course modality does not impact the dimensionality by which students evaluate their course experiences. Otter et al (2013) compared the perceptions of students and tutors involved in DL and face to face courses but while addressing many of Mayes et al’s (2013) themes for effective online instruction, did not cover the “medium” theme. Bentley et al (2010) compared student learning tool (medium) preferences on the University of Bedfordshire’s MBA DL programme, but did not compare DL and resident student preferences. This study aims to provide empirical evidence to fill the gap in information related to DL student learning tool (medium) preferences and how this differs from resident student preferences whose main mode of delivery is face to face. It builds on the earlier study by Bentley et al (2010) and addresses the following research questions:

Which online tools are most helpful to distance learning students?

How do the preferences for learning tools differ between distance learning and resident students?

How do learning tool preferences relate to Mayes et al’s proposed themes for effective online teaching with respect to Community and Discourse and Assessment?

Is there a difference in the overall unit satisfaction between distance learning and resident students on the same course?

2. Background on the UoB’s MBA In the UoB’s DL MBA, students are recruited by a local overseas centre partner and then receive the bulk of their learning experience in an online DL mode from the UoB. However, in each of the 2 years that comprise the MBA, students attend 4 Weekend Business Schools at the local centre where they are given face‐to‐face classes from local centre tutors, and in the first of these Business Schools, students are also given introductory unit classes by UoB staff that fly out to Oman. The UK UoB lecturers are in charge of delivering the online course materials via a VLE called Breo which is powered by Blackboard. They are also responsible for designing and marking the assignments for each module. Assignments are submitted via a Turnitin facility on BREO which also aims to check student work for improper citation and potential plagiarism (Turnitin, n.d). In the resident, full time version of the MBA, the course content and assessment is the same but the delivery is mainly via face to face teaching. However, on the Operations Management unit considered in this study, resident students also had access to the same VLE tools and materials as did the DL students. Ayala (2009, p. 277) defined blended learning as ‘the purposeful integration of traditional (i.e. face‐to‐face) and online learning in order to provide educational opportunities that maximize the benefits of each platform and thus more effectively facilitate student learning’. Under this definition, both the resident and DL student groups considered in this study were blended learners although with different proportions of face to face learning.

3. Methodology The research addresses the research questions described earlier. The unit of analysis is MBA students who took a single unit, Operations Management, that was delivered by the same UK tutor to resident students

132

David Owen and Elly Philpott based in the UK and DL students based in Oman. An important aspect of the unit was that students were evaluated based on the quality of their contributions to their Breo Group Discussion Boards and File Exchange facilities. This “incentivised” students to make use of these group collaboration enabling facilities. The primary research method used was a questionnaire survey. This was initially based on an earlier validated survey by Bentley et al (2010) but was further developed to reflect current context for example to include Facebook. It was also extended to consider Mayes et al’s (2011) proposed themes for effective online teaching with respect to Community and Discourse and Assessment. There were initially 41 survey questions addressing the research questions and these are shown in Table 4 where the relevant Mayes et al (2011) good practice theme that each question addresses is also shown. Responses to the questions linked to helpfulness (questions 1‐38) were recorded using a 4‐point scale, where 1 = “Very Helpful”, 2 = “only a little helpful”, 3 = “not helpful” and 4 = “I did not use this.” For the statements linked to overall unit satisfaction, (questions 39‐41), a 5‐point scale was used where 1=”Definitely Agree”, 2 = “Mostly Agree”, 3 = “Neither Agree or Disagree”, 4 = “Mostly Disagree”, 5 = “Definitely Disagree”. Table 1 describes the DL and resident students that completed the survey, their status with regard to the module and the delivery methods used. A total of 82 students completed the questionnaire from a possible 166 with response rates of 48% for the DL students and 52% for the resident students. A snowball method was used to promote the online questionnaire to resident MBA students via Facebook as these students had completed their course in November 2010. Table 1: Description of survey respondents Sample size and student type 31 Omani (DL) students, (N=59) 29 Omani (DL) students, (N=65) 22 UK (Resident) students (N=42)

Questionnaire delivery method Hard copy Online Online

Survey completion date and student status with respect to the module September 2013 (post unit completion) February 2014 (post unit completion) February 2014 (course completion in 2010)

Questions 15 and 38 in Table 4 were discarded because they were not included in the initial hardcopy survey of Omani DL students. In order to establish whether the two Omani groups in Table 1 were effectively the same group, a t‐test was carried out. Question 30 “How helpful was a Videoconference facility like Skype in helping you with your group work?” was discarded as it was not relevant to the two Omani groups because videoconferencing media such as Skype are blocked in Oman. The t‐test showed that Question 35 was answered significantly differently by the two Omani groups and therefore it was also discarded from further analysis.

4. Results 4.1 Evaluation of the respondents The origins of the 22 resident students who completed the questionnaire are summarised in Figure 1. Notable is the fact that the resident students were all students for whom English was a second language similar to the Omani DL students.

Figure 1: Country of origin of resident MBA student respondents Table 2 shows that the two student groups had similar technical capabilities while Table 3 shows that although all the resident students were aged between 26‐35 years, 43% of the Omani DL students were older than this.

133

David Owen and Elly Philpott Additionally, while all the resident students rated their English language competency as A or B grade, some 25% of DL Omani students rated their competency as weaker than this level. Table 2: Comparison of the technical capabilities of the resident and DL student groups

Able to access and download material from the VLE 18 82% 58 97%

Luton Resident Oman DL

Able to make entries to a VLE discussion board 14 64% 48 80%

Took part in a course videoconference 12 37

55% 62%

Able to download and install Internet programmes 14 64% 42 70%

Table 3: Comparison of age and english language competency of resident and DL respondents Total Luton Resident Oman DL

22 60

Female 5 16

23% 27%

Age (years) English Competency 26‐35 36‐45 >45 A/B Grade C/D Grade 22 100% 0 0 22 100% 0 0% 34 57% 22 4 45 75% 15 25%

4.2 Main results Table 4: Comparison summary of resident and DL students’ responses No

Question

How helpful were the following in helping you to understand the course topics?

Mayes et al (2011) Luton good practice themes Sample Med

C&D

A

M SD

Oman Sample M

SD

Difference in Sample means D

p

1

Powerpoint slides on Breo

√

1.20 0.66 1.30 0.57

‐0.10

n.s

2

Other written material on Breo*

√

1.10 0.29 1.60 0.70

‐0.50

0.7). In Table 4, the means, standard deviations and Cronbach’s alpha values are presented. Table 4: Mean, standard deviation and Cronbach’s alpha Variable

Item

M (N=34)

SD

Collaboration

C1 C2 C3 C4 M (N=4) L1 L2 L3 M (N=3) R1 R2 M (N=2)

3.59 3.79 3.53 4.09 3.75 3.82 3.59 3.53 3.65 3.29 3.38 3.34

0.99 1.01 0.96 0.9 1.14 1.1 1.11 0.97 1.07

Communication

Resource sharing

Cronbach’s alpha 0.83 0.87 0.78

Based on analysis, it can be postulated that collaboration can be deemed as the principal antecedent for the educational use of Facebook, followed by communication and resource sharing. Even with the relatively small sample size of this study, this finding is corroborated by the findings of Sanchez et al. (2013) and Mazman and Usluel (2010) who identified collaboration as the principal antecedent for the education use of Facebook. In addition, a high correlation (r=0.83) is observed between collaboration and resource sharing, with a lower correlation (r=0.74) between communication and collaboration. Thematic data analysis of open‐ended questions revealed that participants had a positive experience with the use of Facebook. “It helped a lot to have such interactions with lecturers since we they [sic] always busy sometimes.” (participant SI:10000202) “It was ok because the lectures updated the information we need about the subjects and the communication was also good.” (participant SI:10003627)

245

Ricardo da Rocha, Pieter Conradie and Antoinette Lombard “It was very nice to communicate throughout the year, I get more information through Facebook.” (participant SI:10005883) The theme of communication support by Facebook was highlighted by several learners, demonstrating the thesis of the active use of Facebook by learners.

5. Conclusion It is irrefutable that formal education has moved into a period of creative destruction (Conradie 2013b). Multiple cloud‐based applications are and will be used to improve learner collaboration, communication and interaction. The Net Generation is, and will be, expecting instructors to create challenging and relevant educational content and environments with more autonomy. Online web‐based technologies, especially those that are actively used by learners, must form part of the pedagogical toolset of instructors. However, this move to “open” web‐based learning will result in possible divergence with traditional instructors, which value F2F instruction more. Exposure of instructors to the pedagogical benefits of online environment such as Facebook, including collaboration, engagement, interaction, communication and resource sharing, is thus evident. If not, learners familiar with the opportunities provided by the Internet will find opportunities elsewhere. Important to note is that whatever social networking tool is employed, it must be appropriate for educational purposes. This implies that it must be accessible and support teaching and learning activities and outcomes. Facebook, although not specifically designed for this role, served an effective pedagogical tool, based on the results obtained. This is further supported by the fact that the majority of students access Facebook, and the internet as a whole, through their mobile devices due to low market penetration of fixed broadband services in South Africa. This also echoes with the findings of both Foko (2009) and Stork et al. (2013). It can be postulated that Facebook has become an integral part of learner life, being accessed multiple times through the day utilizing predominantly mobile phones (Sanchez et al. 2013; Mazman & Usluel 2010; Ellison, Steinfeld & Lampe 2007). Most participants also reported that they do not consider the educational use of Facebook to be an invasion of privacy and learners welcome their instructor’s participation in SNSs. This correlates with the findings of Mason (2006), which report that Facebook’s educational properties, such as numerous interaction tools and proximal feedback, are embraced by learners. Limitations of this study relate to a relatively small sample, and that only self‐reported data was used. It has become essential, especially related to e‐learning, to utilize system‐generated data. Future research will thus extend the sample population, as well as utilizing more system‐generated data such as those provided by Facebook Analytics. Facebook has been found to be an educational tool used by learners supporting collaboration, communication and resource sharing. It is, however, this seamless support of interaction between learners that can be considered the most important value of Facebook, enabling the pedagogical learning model of social constructivism. Based on this, we advocate for the inclusion of Facebook in the educational context when appropriate.

References Ajjan, H. & Hartshorne, R. (2008) “Investigating faculty decisions to adopt Web 2.0 technologies: theory and empirical tests”, The Internet and Higher Education, Vol. 11, No. 2, pp 71‐80. Bartlett‐Bragg, A. (2006). Reflections on pedagogy: reframing practice to foster informal learning with social software. [online], http://www.dream.sdu.dk/uploads/files/Anne%20Bartlett‐Bragg.pdf. Baltatzis, G., Ormrod, D. and Grainger, N. (2008). Social networking tools for internal communication in large organizations: Benefits and barriers,’ ACIS 2008 Proceedings, p 86. Beebe, R., Vonderwell, L. and Boboc, M. (2010) “Emerging patterns in transferring assessment practices from F2F to online environments”, Electronic Journal of e‐Learning, No. 8, Vol. 1, pp 1‐12. Boyd, M. and Ellison, N. (2007) “Social network sites: definition, history, and scholarship”, Journal of Computer‐Mediated Communication, Vol. 13, No. 1, pp 210‐230. Braun, V. and Clarke, V. (2006) “Using thematic analysis in psychology”, Qualitative Research in Psychology, Vol. 3, pp 77‐ 101. Cheung, C., Chiu, P. and Lee, M. (2011) “Online social networks; why do students use Facebook?” Computers in Human Behavior, Vol. 27, No. 4, pp 1337‐1343. Cloete, S., de Villiers, C. and Roodt, S. (2009) Facebook as an academic tool for ICT lecturers. In Proceedings of the 2009 annual conference of the Southern African computer lecturer’s association pp 16‐22. Conradie, P. (2013a) Applying System Theory to develop a mobile learning pedagogical framework, In Proceedings of the 2013 Annual International Conference on e‐Learning, pp 82‐90.

246

Ricardo da Rocha, Pieter Conradie and Antoinette Lombard Conradie, P. (2013b) Connectivism: A learning pedagogy for the Net Generation, In Proceedings of the 2013 SATNAC, pp 211‐215. Creswell, J. (2004) Educational research: Planning, conducting, and evaluating quantitative and qualitative research, Prentice‐Hall, London. da Rocha, R.D and Lombard, A. (2013) Lecturer Perceptions on the use of Social Networking Services in Education. In Proceedings of the 2013 Annual International Conference on e‐Learning, pp 473‐478. Dabner, N. (2012) “Breaking Ground” in the use of social media: a case study of a university earthquake response to inform educational design with Facebook, Internet and Higher Education, Vol. 15, No. 1, pp 69‐78. DMR (2014) Facebook Statistics, [online], http://expandedramblings.com/index.php/digital‐social‐media‐ directory/8/#facebook. Ellison, N., Steinfeld, C. and Lampe, C. (2007) “The benefits of Facebook “Friends”: social capital and college students; use of online social network sites”, Journal of Computer‐Mediated Communication, Vol. 12, No. 4, pp 1143‐1168. Foko, T. (2009) “The use of mobile technologies in enhancing learning in South Africa and the challenges of increasing digital divide”. In Siemens, G. & Fulford, C. eds. Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications. pp 2535‐2540. Jacob, S. and Isaac, B. (2008) “The Mobile Devices and its Mobile Learning Usage Analysis”, Proceedings of the International Multi‐conference of Engineers and Computer Scientists, pp 19‐21. Junco, R. (2011) “The relationship between frequency of Facebook use, participation in Facebook activities, and student engagement”, Computers & Education, Vol. 58, pp 162‐171. Le Q.V, Raven, P.V. and Chen, S. (2013) “International Service Learning and Short‐Term Business Study Abroad Programs: A Case Study”, Journal of Education for Business, Vol. 88 no. 5, pp 301‐306. Lincoln, Y. and Guba, E. (2000) Naturalistic inquiry. SAGE Publications, London. Madge, C., Meek, J., Wellens, J. and Hooley, T. (2009) “Facebook, social integration and informal learning at university: it is more for socializing and talking to friends about work than for actually doing work”, Learning, Media and Technology, Vol. 34, No. 2, pp 141‐155. Mason, R. (2006) “Learning technologies for adult continuing education”, Studies in Continuing Education, Vol. 28, No. 2, pp 121‐133. Mazer, J., Murphy, R. and Simonds, C. (2007) “I’ll see you on “Facebook”: the effects of computer‐mediated teacher self‐ disclosure on student motivation, affective learning, and classroom climate”, Communication Education, Vol. 56, No. 1, pp 1‐17. Mazman, S. and Usluel, Y. (2010) “Modeling educational use of Facebook”, Computers & Education, Vol. 55, pp 444‐453. McLoughlin, C. and Lee, M. (2007) “Social software and participatory learning: pedagogical choices with technology affordances in the Web 2.0 era” Paper presented at the ASCILITE, Singapore. Moran, M., Seaman, J. and Tinti‐kane, H. (2011) Teaching, learning, and sharing: How today’s higher education faculty use social media, [online], The Babson Survery Research Group, and Converseon. http://www3.babson.edu/ESHIP/research‐publications/upload/Teaching_Learning_and_Sharing.pdf. Oblinger, D. and Oblinger, J. (2005) “Is it age or IT: First steps toward understanding the Net Generation”. Educating the Net Generation. Boulder, Colorado: EDUCAUSE, pp 1‐20. Ong, C., Lai, J. and Wang, Y. (2004) “Factor affecting engineer’s acceptance of asynchronous e‐learning systems in high‐tech companies”, Informatics & Management, Vol. 41, No. 6, pp 795‐408. Ramirez, A. and Wang, Z. (2008) “When online meets offline: An expectancy violations theory perspective on modality switching”, Journal of Communications, Vol. 58, No. 1, pp 20‐39. Roblyer, M. McDaniel, M., Webb, M., Herman, J. and Witty, J. (2010) “Findings on Facebook in higher education: a comparison of college faculty and student uses and perceptions of social networking sites”, The Internet and Higher Education, Vol. 13, no. 3, pp 134‐140. Sandars, J. (2005) “Work based learning: a social network perspective”, Work Based Learning in Primary Care, Vol. 3, No. 1, pp 4‐12. Sánchez, R., Cortijo, V. and Javed, U. (2014) “Students’ perceptions of Facebook for academic purposes”, Computers & Education, Vol. 70, pp 138‐149. Selwyn, N. (2009) “Faceworking: exploring students’ education‐related use of Facebook”, Learning, Media and Technology, Vol. 34, No. 2, pp 157‐174. Stork, C, Calandro, E. and Gillwald, A. (2013) "Internet going mobile: internet access and use in 11 African countries", info, Vol. 15 Iss: 5, pp.34 – 51. Towner, T. and Vanhorn, A. (2007) Facebook: Classroom tool for a classroom community? [online], Paper presented at the annual meeting of the Midwest Political Science Association, Chicago, http://www.allacademic.com/meta/p197133_index.html. Wang, L. and Kirkpatrick, A. (2013) “Trilingual education in Hong Kong primary schools: a case study”, International Journal of Bilingual Education and Bilingualism, Vol. 16 no.1, pp. 100‐116. Yu, A., Tian, S., Vogel, D. and Kwok, R. (2010) “Can learning be virtually boosted? An investigation of online social networking impacts”, Computers & Education, Vol. 55, No. 4, pp 1494‐1503.

247

Evaluation of Accessibility and Usability Applied to e‐Learning Platforms in Health Nuno Dominguinhos Faculty of Sciences, University of Lisbon, Lisbon, Portugal [email protected] Abstract: In a growing global market, competitive health organizations that take advantage of information technology may benefit and promote their human capital. For these institutions, e‐learning platforms are seen as valuable and competitive tools since they provide an educational environment that stimulates users to think critically and reflectively, therefore meeting the challenges posed by any time any place availability. Given the facts and stated these potentialities, it is important to check with those responsible for training, the definition, investigation and correction of current strategies applied to e‐Learning platforms. On the other hand, administrators, instructors and students may establish quality criteria for the interaction with the information and technology. This study focuses on the user point of view in a comprehensive manner, and takes into account the evaluation of concepts like accessibility, information architecture and usability in order to verify its effectiveness in real learning context. It aims to function as a guideline that points new directions in the design and implementation realms of health e‐learning platforms. It leads to a new paradigm in health, targets continuing education taking into account the evaluation of concepts already referred, from the perspective of its stakeholders. This approach is a teaching driving force that, if well conducted, may result in a higher yield and knowledge to their users. Thus, at an early stage, a survey of those institutions learning portals was conducted by a heuristic evaluator. In the near future, a lab test with health professionals from a different set of institutions will be carried out and, finally with the end‐users from the institutions where the platforms had been already implemented. The study is particularly relevant because it zooms in the validity of results obtained by a heuristic evaluator compared with the tests of users. The main contribution is to provide an integrated approach based on these methods, which will allow presenting a set of best practices for the implementation of e‐Learning platforms and, subsequently, improve efficiency in learning processes. In addition to this, it will ensure greater interaction among health professionals with the information and e‐Learning platforms. Some goals have already been achieved and implemented, as a result from the analysis made. Keywords: accessibility, education, e‐learning, health and usability

1. Introduction 1.1 Background and study justification The implementation of e‐services in health, through the e‐Learning technology, allows equipping all involved actors with new competences and enables them to constant changes in a knowledge‐based society. “Organizations and Educational institutions have been investing in information technologies to improve education and training at an increasing rate during the last two decades” (Nokelainen, 2006). The healthcare sector is a fast‐changing environment and it is clear that its performance depends on the ability of their staff to take up knowledge fast and acquire the competencies which are necessary for the adaptation to a constantly changing environment (http://www.eurodl.org/materials/contrib/2005/Sarantos_Psycharis.htm). Nevertheless, there is still a long way to walk. According to a previous survey conducted, we may conclude that, from a sample of 806 health institutions in Portugal, only 22% answered to have adopted e‐Learning platforms as a motivation tool for themselves. Although health organizations are aware of the outcome, they still face significant challenges when promoting and improving the continual learning in order to facilitate the procedure of e‐Learning adoption and increase result possibilities. In order to minimize these possible constraints, institutions should ensure the platform's interfaces to be enriched with good accessibility, information architecture (IA) and usability; otherwise it will prevent learners from using this kind of technology. The interface plays a crucial role in any application, but in e‐Learning, it is even more indispensable. Due to the importance of the interaction between the end user and the computer, it should provide a way to achieve specific goals. The main contribution of this study is to recommend an integrated approach based on these methods and show best practices for the implementation of e‐Learning platforms in Escola Superior de Tecnologia da Saúde de Lisboa (ESTeSL), Hospital de Faro and the Hospital de Santa Maria.

248

Nuno Dominguinhos

1.2 Objectives The main ambition of this case study is to examine e‐Learning platforms, according to the following points:

Evaluate the accessibility, IA and the usability of Moodle and Dokeos;

Observe and evaluate forty users (half belonging to the above institutions) through users tasks and with the think aloud protocol;

Validate both evaluations and suggest new measures to improve user’s interaction with the platforms.

The objectives are interlinked with each other. They aim to improve the qualities of utility, learnability, memorability and efficient interface of the analysed platforms. In the end, the evaluator should be able to propose a new process to improve the interface of the e‐Learning platforms so the health organizations may enhance it. “A formative product should be interactive and provide feedback, have specific goals, motivate, communicating a continuous sense of challenge, provide suitable tools and avoid distractions and factors of nuisance interrupting the learning stream” (Norman, 1993)

2. Methodology Through this study, it will be possible to understand the user’s appreciation level of the platforms implementation and obtain more knowledge from their perspective. There are numerous tools and features at the disposal of students and instructors, and it is important for the e‐Learning community to examine both preferences and usage of these features (http://ijklo.org/Volume4/IJELLOv4p113‐135Buzzetto413.pdf). To understand evaluator and user’s concerns simultaneously, the following methods will be applied. The accessibility is based on the Web Content Accessibility Guidelines 2.0 with the tool AccessMonitor provided by UMIC ‐ Knowledge Society Agency. Since the power of the Web is in its universality, the access by everyone regardless of disability is an essential aspect (http://www.w3.org/WAI/). This topic is important in the e‐ Learning context as it implies the requirement for access to the information by users with different abilities, background and preferences, in varied contexts of use. The IA is seen as a method that provides a good structure of website information and should be considered when adapting strategies to implement and adjust the information space of the e‐Learning platforms once it facilitates intuitive access to their content and task completion. Their analysis will be applied according to Rosenfeld heuristics, which correspond to five areas (main page, search interface, search results, site‐wide navigation and contextual navigation) that a user is most likely to interact with (http://louisrosenfeld.com/home/bloug_archive/000286.html). The usability is a quality attribute related to the ease of use of something (Nielsen e Loranger, 2006). Its evaluation is based on the Nielsen 113 heuristics, according to the existing techniques and methods in the literature (Nielsen & Tahir, 2002). The evaluator searches for usability problems in the platform's interface and makes suggestions intended to speed user’s interaction so they can learn to use it in an efficient way and remember it with less effort. The think aloud protocol and questionnaires are also used and lets the evaluator discover what users really think which usually turn into actionable redesign recommendations (http://www.nngroup.com/articles/thinking‐aloud‐the‐1‐usability‐tool). After analysing the results from all the above methods and techniques, a set of best practices will be made to help the health organizations improve their e‐Learning platforms.

3. Conducted reviews 3.1 Accessibility The following information helped the evaluator analyse the users’ view with respect to whether it is accommodative of users with disability and the appropriateness of language used for communication.

249

Nuno Dominguinhos Table 1: Barriers found in ESTeSL versus Hospital Faro (columns A, AA and AAA represents incremental level of accessibility)

From this evaluation we can focus on specific points that help creating a plan of adjustment, and determine whether the accessibility barrier is reflected across the platform or is isolated. Information Architecture It is expected that the users will be able to better interact with the architecture and to identify activities that can help managers and trainers of the platforms in the choice and implementation of these definitions. Table 2: Top 10 issues found in ESTeSL versus Hospital Faro

250

Nuno Dominguinhos Usability As there is only one evaluator, the evaluation consists of two steps In the first step, the evaluator accesses the platforms with administrator credentials, navigates and signs up for some courses to get the idea of how the platform works. In the second step, the evaluator tries to engage in the greatest number of tasks possible and validates the implementation of the 113 heuristics. When a violation of usability is discovered, the evaluator writes a brief explanation where it occurred and annotates in a guide for the purpose with critical comments and explanatory notes. In this way, organizations may read the guide for future implementations. Table 3: Heuristics issues found in ESTeSL versus Hospital Faro

251

Nuno Dominguinhos

4. Results achieved As a result of the applied concepts, recommendations were made, with the objective of achieving user satisfaction. The idea is that users understand platform's navigation as they perform tasks intuitively. In Figure 1, we have the initial page of the previous version Hospital Faro. It was suggested that the marked white space may have a better use. When migrating to a later version, the suggested action had an effect, as can be seen in Figure 2.

Figure 1: Version 1.8

Figure 2: Version 2.0 This change allowed for a space to inform users about updates/news. Figure 3: Administrator email Also in version 2.0 the existence of a Microsoft Outlook window that appeared with an empty address in the place of the email address marked (Figure 3) was rectified.

252

Nuno Dominguinhos

Figure 4: Executable file It was detected that the system allowed upload of executable files but it was rectified (Figure 4). The ESTeSL already made the following improvements:

Figure 5: “Nome do utilizador” (username) As can be seen in Figure 5 the label "Nome de utilizador" was corrected on the screen, to allow better reading.

Figure 6: “Alterar a sua senha” (change your password) Since both connections "alterar a sua senha” and "Continue" lead the user to the same page, it was proposed to omit the first (Figure 6).

253

Nuno Dominguinhos

Figure 7: Logos As Nielsen says, the homepage logos should not be clickable. Thus, it was rectified by the organization in conformity with Nielsen heuristic number 5 (Figure 7). With such ontologies, it was possible to give a valuable contribute to the organizations involved.

5. Conclusion To sum up, this paper states the first results of research aimed at defining best practices in the implementation of e‐Learning platforms. In the current phase, results have already been achieved, focusing on the evaluation of the three methods referred. Despite having already helped the organizations on how to improve their platforms’ interaction, further objectives/steps mentioned include conducting user evaluation to enhance/refine the preliminary set of guidelines and possibly define others. In addition, these methods have the potential to enrich the outcomes. They not only improve the interaction among all involved actors but they also make the learners feel more excited about the courses. The evaluation of educational software must consider its usability and more in general its accessibility, as well as its didactic effectiveness (Ardito et. al., 2006). As a matter of fact, we may conclude that a successful platform implementation is closely related to the methods and procedures taken. The last, but by no means the least, and as a challenge, health organizations should implement these principles and methods since they can offer considerable benefits as productivity growth, cognitive support and organizational efficiencies.

References Ardito, C., Costabile, M. F., Marsico, M. D., Lanzilotti, R., Levialdi, S., Roselli, T., & Rossano, V. (2006). An approach to usability evaluation of e‐learning applications. Universal Access in the Information Society 4(3), 270‐283, New York. Nielsen, J. e Tahir, M. (2002). Homepage Usability: 50 Websites Deconstructed. New Riders, Indianapolis. Nielsen, J.; Loranger, H. (2006) Prioritizing Web Usability. New Riders, Berkely. Nokelainen, P., 2006, An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students, Educational technology and society 9(2), 178‐197, Hämeenlinna. Norman, D. (1993) Things That Make us Smart: Defending Human Attributes in the Age of the Machine. Perseus Publishing, Cambridge:MA.

254

Work in Progress Paper

255

256

SCORM Packaging of 8085 Microprocessor Simulator and its Evaluation on Moodle Kazuhiro Muramatsu College of Science and Technology, Royal University of Bhutan, Phuentsholing, Bhutan [email protected] Abstract: Practices and experiments have important positions at scientific and technological institutions of higher education, because students promote a better understanding of lectures and master practical skills through them. The College of Science and Technology (CST), Royal University of Bhutan (RUB) is the only technical college in Bhutan, and a variety of practices are done at the college. However, the maintenance costs of practical devices or experimental equipment are expensive, and it is difficult to repair them when they have gone wrong. These problems are common in developing countries. Also, these devices or equipment are usable only for practice or experiment hours. Namely, it is impossible for students to use them to prepare or review practices. In order to solve these problems, thus, we have started on the development of simulators which simulate the above‐mentioned devices or equipment, and they are implemented on a Learning Management System (LMS). By replacing the devices or equipment to simulators, their maintenance costs will decrease. Also, this makes it possible for students to prepare or review practice by the use of the simulator on the LMS. Here, a variety of equipment is used for practice or experiment. In this paper, we focus on the practice of an 8085 microprocessor. The 8085 microprocessor has a simple architecture and an adequate instruction set. Therefore, the 8085 microprocessor is an excellent teaching material to teach basic concepts and programming of microprocessors, and it is one of the most widely used microprocessor in the subjects of microprocessor in higher education. Really 8085 microprocessor trainers have been being used for the practices of the microprocessor‐related subjects at our college. Firstly, we develop the 8085 microprocessor simulator by PHP: Hypertext Pre‐processor, because the Moodle‐based LMS of our college has been built on PHP. Then, the simulator is implemented on the LMS as an activity module. However, the implementation as an activity module is a lack of portability. When the Moodle‐based LMS is updated, for example, it is necessary to modify several forms in the activity module according to the upgraded version of Moodle. Thus, in order to improve the portability, we convert the 8085 microprocessor simulator into a Sharable Content Object Reference Model (SCORM) package by the use of JavaScript. The SCORM package is compared with the above‐mentioned activity module. The result shows that the execution speed of the activity module is faster than that of the SCORM package. At present, the cause why the activity module is faster than the SCORM package is under investigation. Keywords: 8085 microprocessor, practice, experiment, virtual learning environments (VLE), learning management systems (LMS), Moodle, SCORM package, PHP, JavaScript

1. Introduction At present, a Learning Management System (LMS) is becoming popular in institutions of higher education. For example, Moodle has been used on over 64,000 registered sites and the number of registered users is about 66 million (Cole and Foster, 2007) (Moodle, 2014). Also, Sakai CLE is used by over 350 organizations (Berg and Korcuska, 2009) (Sakai Project, 2014). In Bhutan, Royal University of Bhutan (RUB) and its ten constituent colleges launched a Moodle‐based e‐Learning system or Virtual Learning Environment (VLE) in 2011 (Royal University of Bhutan, 2012). This e‐Learning project was carried out with funding from World Bank and expertise from the University of Colombo School of Computing (UCSC) (Muramatsu and Wangchuk, 2013). Generally these e‐Learning systems are used for lectures, tutorials, and assignments. The purpose of this paper is to extend the function of the e‐Leaning system to practice or experiment by the use of a simulator. However, there is a variety of equipment for practice or experiment. In this paper, we focus on the practice of an 8085 microprocessor. Firstly, the implementation of the 8085 microprocessor simulator as an activity module on Moodle is described. Next, we describe SCORM packaging of the simulator. Finally, the SCORM package is compared with the above‐ mentioned activity module on Moodle.

2. Development of the 8085 microprocessor simulator by PHP We develop the 8085 microprocessor simulator by PHP, because the Moodle‐based LMS of our college has been built on PHP (Muramatsu, 2012). The microprocessor simulator is composed of three modules: Graphical User Interface (GUI) module, Main module, and Execution module. Figure 1 shows data flow between each

257

Kazuhiro Muramatsu module. The GUI module runs on a client, and the Main and Execution modules on a server. The details of each module are described in the following sub‐sections.

Figure 1: Modules and data flow in the 8085 microprocessor simulator by PHP

2.1 GUI module A user inputs the assembly program by the GUI module on the client, and the input program is sent to the Main module on the server. Also, the GUI module displays the results of a Hex machine program transferred from the Execution module.

2.2 Main module The Main module translates an assembly program sent from GUI module to a Hex machine program on the server. The Hex machine program is sent to the Execution module.

2.3 Execution module The Execution module executes the Hex machine program received from the Main module, and transfers its results to the GUI module. Here, the results mean the contents of registers and memory locations on the 8085 microprocessor after the execution.

3. Implementation of the PHP‐based 8085 microprocessor simulator on Moodle 1.9 After the development of the microprocessor simulator by PHP, it is implemented on the Moodle1.9‐based LMS of our college as an activity module (Muramatsu, 2012). Generally, a new activity module of Moodle 1.9 is created by modifying the NEWMODULE template which can be downloaded from the Moodle website (Moodle, 2011). The NEWMODULE template is composed of four folders and fifteen files. Main modifications from the template are described as follows (Moore and Churchward, 2010).

3.1 Input form The file ‘mod_form.php’ provides the input form when a teacher registers the simulator as an activity module on its course. Here, the file has been modified to be necessary to input the simulator name and its instructions. Also the teacher can define a period that the simulator is available, and can set the group mode.

3.2 View form The simulator is available for students after the registration by the teacher. The file ‘view.php’ is responsible for displaying the simulator as an activity module when students access it. We have modified it to display the sentence ‘Click here to run 8085 Simulator’ on the window. If the sentence is clicked, the 8085 simulator starts on a pop‐up window.

3.3 Local language form The ‘lang/en_utf8/simulator.php’ file is called a local language file, and customizes localization strings with a hash array called $string. We have defined several values for the $string hash.

258

Kazuhiro Muramatsu

4. SCORM packaging of the 8085 microprocessor simulator In the previous section, we implemented the PHP‐based 8085 microprocessor simulator as an activity module on Moodle 1.9. However, several modifications of the input, view, and local language forms are needed when the LMS is upgraded to Moodle 2.4. Also, more modifications will be necessary when the microprocessor simulator is implemented on another e‐Learning system such as Sakai CLE. Namely, implementation as an activity module on Moodle is lack of portability. In order to overcome this demerit, we adopt Sharable Content Object Reference Model (SCORM). The SCORM is a model that references and integrates a set of interrelated technical standards (Jones, 2002). If contents are converted into a SCORM package, the packaged contents can be implemented on different LMSs easily. Namely, the SCORM packaging provides portability among different LMSs if the LMSs support the SCORM (Advanced Distributed Learning, 2002). Generally a SCORM content package is composed of two principal parts: an XML manifest file and Sharable Content Objects (SCOs). These files are archived as a zip file. The details of the manifest file and the SCO are described as follows.

4.1 Manifest file We adopt the SCORM 1.2 version. The manifest file of the SCORM 1.2 contains metadata about the package, organization structures that describe the structure of the content, and a list of references to the resources in the package. Metadata is additional informative data about the package. For example, ‘Title’, ‘Language’, and ‘Keyword’ are included in metadata. Metadata is optional, and here we do not add metadata in the manifest file of the 8085 microprocessor simulator. The manifest file for the SCORM package of the 8085 microprocessor simulator is shown as follows. Organization Run 8085 Microprocessor Simulator

4.2 SCO The 8085 microprocessor simulator in Section 2 is just correspondent to the SCO. As shown in Figure 1, the simulator transfers data between the server and the client, and processes them on the server side. In the case of SCORM, it is possible to transfers data between the server and the client by Application Programming Interface (API) functions LMSGetValue and LMSSetValue. However, it is impossible to process data on the server side. Thus, we modify the microprocessor simulator by JavaScript to process data only on the client side. Data are transferred through a cookie between the GUI module and the Process module which combines Main and Execution modules on the client side, so as to modify it easily. Figure 2 shows data flow between each module in the SCORM package by JavaScript.

259

Kazuhiro Muramatsu

Figure 2: Modules and data flow in SCORM package of the 8085 microprocessor simulator

5. Comparison between activity module and SCORM package We register the SCORM package of the 8085 microprocessor simulator on the Moodle‐based LMS of our college, and compare its execution speed with that of the activity module in Section 3. Twenty nine students of Electronics and Communication Engineering Course, College of Science and Technology (CST) evaluated the execution speeds of the activity module and the SCORM package. The results are described in Table 1. Table 1: Comparison about the execution speed between the activity module and the SCORM package Activity module SCORM package Very fast 14 0 Fast 15 3 Average 0 11 Slow 0 8 Very slow 0 7

Table 1 shows that the SCORM package is slower than the activity module.

6. Conclusions In this paper, firstly we developed the 8085 microprocessor simulator by PHP and implemented it as an activity module on the Moodle‐based LMS of our college. Next, the 8085 microprocessor simulator was modified to the SCORM package by PHP, and it was registered on the LMS. Finally, the execution speed of the activity module was compared with that of the SCORM package. The result shows that the SCORM package is slower than that of the activity module. At present, the reason why the activity module is faster than the SCORM package is under investigation.

Acknowledgements I would like to thank Mr. Yeshi Wangchuk for the implementation of the 8085 microprocessor simulator as an activity module on the Moodle‐based LMS of our college. I also thank third‐year students of Electronics and Communication Engineering Course, CST for the evaluation of execution speeds of the activity module and the SCORM package.

References Advanced Distributed Learning (2002) “SCORM 1.2 Specification”, [online], ADLNet, www.adlnet.gov/resources/SCORM‐1‐ 2‐Specification?type=technical_documentation. Berg, A. and Korcuska. M. (2009) Sakai Courseware Management: The Official Guide, Packt Publishing, Birminham. Cole, J. and Foster, H. (2007) Using Moodle: Teaching with the Popular Open Source Course Management System, O'Reilly Media, California. TM Jones, E. R. (2002) “Implications of SCORM and Emerging E‐learning Standards On Engineering Education”, Proceedings of the 2002 ASEE Gulf‐Southwest Annual Conference, [online], CiteSeerX, citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.8896&rep=rep1&type=pdf. Moodle (2011) “NEWMODULE Documentation – MoodleDocs”, [online], Moodle.org, docs.moodle.org/dev/NEWMODULE_Documentation. Moodle (2014) “Moodle Statistics”, [online], Moodle.org, moodle.org/stats. Moore, J. and Churchward, M. (2010) Moodle 1.9 Extension Development, Packt Publishing, Birminham. Muramatsu, K. (2012) “Development of Web‐based 8085 Microprocessor Simulator and Its Implementation on LMS”, International Journal of Applied Engineering Research, Vol.7, No.11, pp 1209‐1213.

260

Kazuhiro Muramatsu Muramatsu, K. and Wangchuk, S. (2013) “Current Status of e‐Learning and Its Challenges at Institutions of Higher Education in Bhutan”, Information Processing Society of Japan SIG Technical Report, Vol.2013‐CLE‐9, No.13. Royal University of Bhutan (2012) “Annual Report 2011”, [online], Royal University of Bhutan, www.rub.edu.bt/images/key‐documents/annual‐reports/annual_report_2011.pdf. Sakai Project (2014) “Sakai Project | collaboration and learning ‐ for educations by educators”, [online], Sakai Project, sakaiproject.org.

261

262

Late Submissions

263

264

The Role of Open Access and Open Educational Resources: A Distance Learning Perspective Stylianos Hatzipanagos1 and Jon Gregson2 1 Kings College London, UK 2 Institute of Development Studies, UK [email protected] [email protected] Abstract: The paper explores the role of Open Access (licensing and publishing) and Open Educational Resources within Distance Education, focusing on the context of the University of London International Programmes. We report on a case study where data were gathered from librarians (including information specialists) and the University of London International Programmes directors relating to existing practice around Open Access and where the major constraints in using Open Educational Resources and the main resource implications, when adopting Open Educational Resources, were also investigated. Our aim was to (a) raise awareness and understanding of what is possible in higher education (b) identify next steps and actions that could be taken to improve institutional use of Open Access materials, including Open Educational Resources, (c) examine the implications of such actions for open and distance learning and generally the HE sector. Keywords: open access, open educational resources, open and distance learning, open licensed materials, open access publishing

1. Introducing Open Access and our investigation The movement of Open Access seems to have significant implications for reaching a global audience, on campus and open and distance learning (ODL) students. Open Access is free, immediate, permanent online access to the full text of research articles and data for anyone, webwide. There are also rights and equity issues that particularly apply to the context of open and distance learning. The paper will report on a case study where data were gathered from librarians (including information specialists) and the University of London International Programmes (UoLIP) programme directors relating to existing practice around Open Access and Open Educational Resources (OERs). Our investigation explored (a) the use of Open Access materials (i.e. journals), (b) OERs (b) awareness of creative commons licences, (c) perceptions of quality and usefulness of open licensed materials, (d) collaborative schemes for drawing together on Open Access repositories across institutions. We also investigated what were considered to be the major constraints in using OERs and the main resource implications, when adopting OERs. The purpose of the investigation was to understand how open licensed approaches are used and any policies that are being applied within the Colleges of the University of London. The objective was to acquire an understanding of the current situation; in addition, the intention was to share and discuss the results and recommendations at the workshop that took place two months after the dissemination of the survey. We hoped that this would also lead to some interesting recommendations on how the International Programmes and the sector could benefit and engage from the Open Access movement

2. The Open access spectrum In Open Access of digital artefacts, there are two broad areas where access is considered: (1) artefacts are freely accessed, with no financial costs to the person that accesses them; in addition (2) an area, which is of great interest, is how resources that are freely accessed can also be reused, with or without modification. This usually includes the conditions under which reuse and modification could be legitimate. Creative Commons (2014) is the major influential framework that has attempted to regulate access and reuse. Established areas of Open Access include:

‘green’ Open Publishing repositories, that means that, for the most part, they contain either summary data about publications (rather than full text) or final drafts of publications which their authors have posted before submitting the text to a journal. These limitations are increasingly making the combination

265

Stylianos Hatzipanagos and Jon Gregson of green repositories and subscription publishing an unsatisfactory compromise to the Open Access movement.

‘gold’ Open Publishing Repositories, in which publication costs are paid before publication, allowing the publisher to permit wider distribution without damaging loss of revenue (Swan, 2010; Swan and Houghton, 2012). But the term can be misleading, because it has been used to embrace rather different approaches. In some cases, gold means up‐front payment for limited distribution rights; a paper may be distributed but not reused in any way, including text or data mining, without further charges.

Open Data – this is a broad area that allows reuse, revising, remixing and redistributing of data. These data are data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and share alike. As Rushby (2013) pointed out data sharing is a natural extension of open access.

Open Educational Resources are teaching, learning or research materials that are in the public domain or released with an intellectual property license that allows for free use, adaptation, and distribution. (UNESCO, 2009). They are freely available online for everyone to use, whether you are an instructor, student or self‐learner. Examples of OERs include: full courses, course modules, syllabi, lectures, homework assignments, quizzes, lab and classroom activities, pedagogical materials, games, simulations, and many more resources contained in digital media collections from around the world (JISC 2013).

Open Development is about making information and data freely available and searchable, encouraging feedback, information sharing, and accountability (Smith, M. L. and Reilly, K. M. A. (2014). The World Bank is one of the organisations that have launched such open development initiatives.

Open Licensing: Creative Commons Licensing (ibid.) ‐ i.e. a creative commons attribution licence (CC‐BY) is now the de facto standard for Open Access licensing (free to copy, distribute, display, perform, make derivative works, and make commercial use, however an overall rule is that the original author must be given credit).

3. Examples There are a growing number of examples of all these forms of open access. The Institute of Development Studies is one of many institutions that now make Open Access materials available through an institutional repository Open Docs (2014), based upon the DSpace platform (an open sources repository application). Many libraries are now building digital repositories to develop capacity to make Open Access research available in this way. Initiatives are emerging to create consortia to run federated repositories with capabilities for supporting discovery of content from across repositories by use of powerful search tools. The concept of knowledge hubs that provide access to open data sets is also emerging strongly, with examples such as the FAO (Food and Agriculture Organisation of the United Nations) supported CIARD (Coherence in Information for Agricultural Research for Development network (2014), a movement dedicated to Open Access knowledge related to agriculture. The Institute of Development Studies is launching the ORIEL Open Knowledge Hub, which makes Open Access content related to development research widely available, and encourages contribution, use and innovative reuse by interested partners. These examples are indicative of a growing range of initiatives that are likely over time to transform the knowledge sharing landscape and the way research is created and made available and accessible. This transformation is also likely to have a big effect on how knowledge can be reused, for example in the design of ODL learning materials, and in the way students can access resources.

4. Open Access funding Open Access publishing is based on a new funding model, as the assumption related to gold standard published is that the author must pay to cover the loss of journal subscription fees. These are referred to as the Author Processing Costs (APC). Incentivising Open Access publishing has resulted in major policy changes and recommendations in the UK for public funding to cover APCs, but in exchange for a requirement to publish public funded research and knowledge creation. The UK Finch report (ref) produced a commitment for £30 million per annum to be allocated to supporting Open Access publishing, and RCUK and the EU are also now funding APC costs. The

266

Stylianos Hatzipanagos and Jon Gregson research Councils UK (RCUK) is doing this by providing block grants to HEFCE institutions to cover APCs for gold standard Open Access publications. Such newly introduced requirements to publish as Open Access are not optional but conflict with some of the ways in which researchers are currently incentivised, recognised and rewarded. Both their intellectual property rights and desire to publish in the ‘top’ journals are affected, so compliance is an issue. The Welcome trust responds to this by withholding 10% of grant fees if the author does not comply, and the Department for International Development policy requires that researchers must comply within six months of finishing their work. It has also recently become a requirement under the Research Evaluation Framework (REF) that certain forms of publications for academics entered in the REF must be Open Access (HEFCE 2014).

5. The Open Access debate The Open Access agenda itself raises some important global debates, as it is changing the model of access (including business model) for research and education, and whilst promoting access and affecting the forms in which knowledge is made available, it is also affecting the incentives surrounding knowledge creation. In this regard we should not immediately assume that by its very nature, Open Access wholly responds to values that promote access to public funded knowledge as a human right. Open Access brings changes to which some researchers are resistant and others question whether the APC model that underpins Open Access publishing is another form of exploitation, which may make it relatively harder for authors in developing countries or unestablished authors to publish their materials. A related issue surrounds use of the ‘ISI Impact factor’, i.e. the most commonly used metric, a measure reflecting the average number of citations to recent articles published in a particular journal publication, which is the prevalent measure used to rate the quality of research, but which gives far greater recognition to articles published in peer reviewed journals as opposed for example to academic work published on repositories. This is seen as fundamentally working against recognition and incentivisation of researchers that are not based in linguistically and culturally dominant countries and new forms of publishing (Gray et al 2013). Many are now advocating for alternative forms of metrics that relate substantively to use and the value of published research, irrespective of the form of publication (e.g. Altmetric, 2014) Altmetric collects article level metrics and the online conversations around research on behalf of publishers, institutions and funders, combining a selection of online indicators (both scholarly and non‐scholarly) to give a measurement of digital impact and reach. We do this by tracking, collecting and measuring large amounts of data collected from all of the places where scientists, patient advocates, journalists, nurses, engineers and members of the public talk about science online ‐ for example, blogs, Twitter, Facebook, Google+, message boards and mainstream newspapers and magazines.

6. Open Educational Resources There are perceived advantages and disadvantages in the use of OERs (D’Antoni, S. 2007; Hatzipanagos, 2012; 2013). They are seen to be displaced from proprietary ‘silos’, i.e. the institutional Virtual Learning Environments (VLEs). They are also copyright ‘free’, as contributions to collective knowledge. However, most often come against recent improvements in creation of technology enhanced learning content, as they are frequently didactic in nature. They are also often elliptical shells to fill in with context and meaning. Context and wrap around activities are missing as interactive aspects and their learning design are separated from content and are often implicit rather than explicit (ibid.). Previous research on engagement with OERs identified some trends:

There is a preference for ‘useful (of clear purpose), specific (contextualised) and practical (of an obvious purpose) OERs’.

“Context often missing”, preference for reusable rather than repurposable.

Main perceived potential benefit of OERs: ‘improved learning’ and less ‘saving on academic time to develop appropriate material/content’.

To get a better understanding about these areas, especially in the context of ODL, we conducted an investigation.

267


7. Methodology Our investigation comprised:

an online survey (of a quantitative and qualitative nature) was distributed to the UOLIP. We addressed a broad target audience of librarians and Programme Managers/Course leaders. The two versions of the survey were broadly similar and were adapted to the nature of the participants’ work and the context in which they operated.

a workshop/focus group during the Research in Distance Education (RIDE) 2013 conference, where we invited participants of the original survey and other experts in the field.

The survey yielded 21 returns; of these, 12 came from librarians and 9 came from Programme Managers/Course leaders. The questionnaire responses were analysed and quantitative and qualitative data collected from the questionnaires were analysed to determine common issues which were considered as fundamental by the respondents. The purpose of the focus group (in which there were 30 participants) was two‐fold. Firstly we aimed to present key findings of the analysis to the participants to gauge their perspectives in a relatively unstructured fashion. Secondly, we aimed to expand on key issues emerging from the data analysis through semi‐structured questions reprising some of the themes that emerged from the survey data analysis.

8. Findings: Open access 8.1 Institutional policies and Open Access repositories 45% of the participants in this study responded that their institutional libraries had no policies related to Open Access subscriptions, while 55% indicated that their institutions were in the process of developing a collection of recommended Open Access materials. For those institutions that had an in‐house open licensed digital repository (73% of the respondents indicated that such an digital repository existed in their institution), librarians were asked how they were promoting these Open Access collections.

8.2 Marketing and creating awareness Responses indicated that there were local marketing initiatives that created awareness of these Open Access collections. They included using social media, emails and RSS (Rich Site Summary) feeds, mailing list and blogs and creating awareness via presence at conferences, and through workshops, school committees and departmental meetings. The respondents also mentioned some other promotion methods, namely via informal academic networks inside the institution and working closely with IT services and departmental administrators.

8.3 Open Access journals Respondents were also asked whether their institutions produced any Open Access journals. The responses were again mixed with a 45% indicating that they produced and promoted open journals. These were often made available through the library by creating a catalogue record and links to full next, or added to the institutional repositories or on Open Journals System platforms and archived in the ePrints (2014) repository.

8.4 Training and support Another important dissemination and awareness avenue was via staff and student development activities, including training in information skills, publishing guides and creating sets of online resources that would signpost Open Access initiatives. An area of inquiry was whether libraries provided training or support to staff/students on how to find open licensed materials and assess their quality. 27% responded that their libraries provided support for both staff and students. Training and support on how to produce open licensed materials seemed to be staff rather than student oriented with only 18% of the respondents indicating that they provided training or support for students through the library on how to produce open licensed materials.

268

Stylianos Hatzipanagos and Jon Gregson Respondents of the survey were also asked about modes and channels of delivery of their Open Access materials. 73% responded that they had plans to make Open Access materials available via mobile technologies and tablet PCs. In addition, programme directors were asked on their opinion on quality and usefulness of open licensed materials. Overall they were positive about the existence of “many good resources”, however they also referred to the challenges of evaluating the quality of such resources. Representative comments were: “All books should be available online. I am a strong supporter of Google's scanning program.” “Variable (quality). Until there is a way of screening /rating that is robust, difficult to recommend.” “I think that it is useful to use open licensed materials where possible but aware of them being carefully used in context.” When asked whether a collaborative scheme for drawing together an Open Access repository across the colleges involved in the UoLIP would be useful, the responses were mixed. Overall, there was an optimistic attitude about the future of open access, when participants were asked how significant Open Access materials were likely to become in the next 5‐10 years. Respondents agreed that OA publishing would become part of the default “I think there will be increasing pressure to make research findings available”. The reasons that were given were financial (“shrinking budgets”), and “growing awareness of Open Access & research council funders mandates on open access, growth although unevenly across disciplines”. There were implications for student learning because “if academics are changing their practice, so too must students be prepared to learn in this new research environment”. Arguments were supported by statistics on current Open Access uptake and projections on increase. As someone commented: “I believe that OA content will grow both in importance and size. At the moment stats show that 10‐15% of articles are published with gold OA, I believe that number will reach the 40‐50% in the next 10 years”. This was linked according to someone else to a “political shift in having more scholarly research publically available”.

9. Findings: Open Educational Resources Both librarians and Programme directors were asked in the survey questions about OERs. A relatively sizeable percentage (64%) they did not make use of Open Access materials (i.e. Open Access journals, including OERs. They were also not familiar with different types of creative commons licences. The practitioners’ attitudes towards Open Educational Resources and the ‘volatile’, almost ‘touch and go’, ever changing nature of the key OER repositories made them question whether they represented a reliable source of teaching materials. As someone commented: “…the world of OER is not static enough to make it meaningful other than a snapshot of that day…” The overlap between institutional endeavours was for others an advantage for such a collaborative initiative. A respondent commented: “I expect there may be some overlap between subject materials of interest to our students and students of computational courses provided by …. and …. (other institutions of the University of London)”

10. Constraints in OER use The survey and discussions in the workshop investigated the constraints in using OERs. The main constraint according to the respondents of the survey was a limited understanding of their value. They also alluded to a cultural resistance whose facets were – a “new thing”, “not developed here”, “can we trust it?” etc. However, the main, often repeated perceived constraint was a lack of staff development to familiarise academics with the nature and opportunities of OERs and time to search and explore repositories of OERs for suitable learning and teaching resources.

269

Stylianos Hatzipanagos and Jon Gregson Respondents saw advantages in OER use as well. They thought that the almost ‘self‐nurturing’ nature OERs might require “less effort to maintain than institutional resources”. They also saw another long‐term benefit that of investing in OER development that could yield gains in the future. As someone commented: “like most ‘e’ advancements, the development of OER e‐assessment materials is time and resource intensive in the short term, but should result in cost/time savings in the medium/long term.” There was also a reputation benefit both for the institution and staff involved. Comments of this type were: “Raise the profile for the institution/materials authors/instructional designer” “As a marketing device, enhancing the institution’s reputation for quality educational materials”

11. Discussion: The emergent landscape in Open Access and OER use in UOLIP Our findings indicated that, regarding open access:

We need to recognise, support and reward Open Access initiatives and systems.

There are some non‐obvious linkages between digital repositories, standards, and quality on which librarians, IT departments, course leaders and researchers need to work closely together.

We need to build awareness of students, and make resources available as open access, e.g. data sets for ODL students to work on.

We need to author open licensed ODL materials with references to open licensed research.

There seemed to be clear advantages of Open Access that included:

Many students (inc. in developing countries) becoming more digitally literate, and libraries ‘serving’ people who are not physically present;

Promoting access, availability and usage;

Gaining more feedback and engagement with readers, who can collaborate on ongoing development of ideas and resources;

Establishing and recognising new ways to measure impact;

Supporting more effective exploration of resources and data –(purchase not needed);

Enabling data mining by allowing simultaneous access to many articles/digital resources.

Based on the survey responses and workshop discussions, ‘searchability’ and ‘discoverability’ of OERs seemed to be an overall issue. Programme directors referred to a limited number of OERs in certain disciplines; however the data we collected did not provide sufficient information on the relationship between disciplinarity and OERs, which seemed to be a rather complex landscape. Another dominant trait of responses was that practitioners were commonly not familiar with OERs and Open Access initiatives; therefore there should be a strong academic development aspect of any engagement activity.

12. Conclusion There seems to be a current debate and on‐going initiatives about reaching a global audience in HE; this affects staff that are directly or indirectly linked to learning and teaching. It also seems to affect increasingly both in the context of global education on and off campus and ODL students. Other current initiatives are helping to create awareness of Open Access issues. Multiple Open Online Courses (MOOCs) have become a driver for many higher education institutions. MOOCs and engaging with MOOCs seems to be a major key themes in current debates as many institutions have responded to the MOOCs call, embracing some of the principles of the Open Access and open educational resources movement, at least the ones that help to institute policies and regulate practice. Our investigation highlighted some opportunities that resulted into some clear recommendations that emerged from our investigation both for practitioners and for students in this area. There also seems to be a synergy between Open access and OERs as both have to address issues of ease of access, quality and visibility in order to become accepted in higher education.

270


References Altmetric (2014), [online] Available at: http://www.altmetric.com/whatwedo.php. CIARD (2014), [online] Available at: http://www.ciard.net/>. [20 March 2014] Creative Commons (2014), [online] Available at: