Technology Integration in Context-Aware Learning

Context-aware learning spaces (CALSs) are mobile-based learning environments which utilise contextual resources, such as real world objects, in the learning process. This dissertation presents the development of two technical platforms on which ten CALSs were created in 2007-2011. Based on the development experiences, a model and an evaluation tool for technology integration in CALSs are proposed. These results, both practical and theoretical, can be utilised by developers to create CALSs in which technologies have been integrated effectively so that they do not disturb the learner.

Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences ISBN 978-952-61-0621-2 ISSN 1798-5668

dissertations | No 59 | Teemu H. Laine | Technology Integration in Context-Aware Learning Spaces

Teemu H. Laine Technology Integration in Context-Aware Learning Spaces

Teemu H. Laine

Technology Integration in Context-Aware Learning Spaces

Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences

TEEMU H. LAINE


Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences No 59

Academic Dissertation To be presented by permission of the Faculty of Science and Forestry for public examination in the Louhela Auditorium in Joensuu Science Park at the University of Eastern Finland, Joensuu, on December, 20, 2011, at 12 o’clock noon. School of Computing

Kopijyvä Oy Joensuu, 2011 Editors: Prof. Pertti Pasanen, Prof. Pekka Kilpeläinen, Prof. Kai-Erik Peiponen, Prof. Matti Vornanen

Distribution: University of Eastern Finland Library / Sales of publications P.O. Box 107, FI-80101 Joensuu, Finland tel. +358-50-3058396 http://www.uef.fi/kirjasto

ISBN: 978-952-61-0621-2 (printed) ISSNL: 1798-5668 ISSN: 1798-5668 ISBN: 978-952-61-0622-9 (pdf) ISSNL: 1798-5668 ISSN: 1798-5676

Author’s address:

University of Eastern Finland School of Computing, Joensuu Campus P.O.Box 111 80101 Joensuu FINLAND email: [email protected]

Supervisors:

Professor Erkki Sutinen, Ph.D. University of Eastern Finland School of Computing, Joensuu Campus P.O.Box 111 80101 Joensuu FINLAND email: [email protected] Associate Professor Mike Joy, Ph.D. University of Warwick Department of Computer Science Coventry CV4 7AL UNITED KINGDOM email: [email protected]

Reviewers:

Professor Mike Sharples, Ph.D. The Open University Institute of Educational Technology Walton Hall Milton Keynes MK7 6AA UNITED KINGDOM email: [email protected] Associate Professor Hiroaki Ogata, Ph.D. Tokushima University Department of Information Science and Intelligent Systems 2-1, Minamijosanjima Tokushima 770-8506 JAPAN email: [email protected]

Opponent:

Professor Alfredo Terzoli, Ph.D. Rhodes University Department of Computer Science P.O.Box 94 Grahamstown 6140 SOUTH AFRICA email: [email protected]

ABSTRACT Context-aware learning spaces (CALSs) are mobile-based learning environments which utilise contextual resources, such as objects in the physical environment, in the learning process. The awareness of the surrounding context enables a CALS to take advantage of rich contextual resources in informal learning settings. Technology integration refers to the process by which a technology is introduced to a classroom so that the teacher and the students can use it e↵ectively for pedagogical purposes. While technology integration has been typically researched in the context of formal classroom-based education, it has not received similar attention in informal learning settings, particularly in the case of CALSs, even though a large part of learning occurs in informal contexts as part of everyday life. Research and development work was conducted in Finland, South Korea, South Africa and Mozambique in 2007-2011. During those four years, two CALS platforms and ten CALSs based on the platforms were developed using an exploratory software development method. The platforms were based on the client-server architectural approach and utilised technologies such as mobile devices, sensors, smart tags and wireless networking. The created CALSs were games for various purposes and contexts including, but not limited to, mathematics in classrooms, environmental awareness in a forest, history in a technology museum, and science in a science festival. The analyses conducted on the games, together with literature analyses, informed the creation process of a technology integration model which describes the requirements and the critical factors that should be taken into account in the design phase of a CALS. Furthermore, based on the model and a literature analysis, a tool was created to facilitate the evaluation of technology integration in CALSs. An evaluation conducted with the tool indicated that it provides more accurate results with a smaller data set than an evaluation without the tool. This research represents both practical and theoretical perspec-

tives, thus proposing an holistic approach to technology integration in CALSs. The outcomes fulfill the overall objective which was to provide the developers with the tools for construction and evaluation of CALSs in which technologies have been integrated e↵ectively so that they do not disturb the learner. Universal Decimal Classification: 004.9, 37.041, 37.091.33, 37.091.64 AMS Subject Classification: 68U35, 97D40 Library of Congress Subject Headings: Context-aware computing; Mobile computing; Non-formal education; Learning; Educational technology; Instructional systems; Educational games; Evaluation; Mathematics; Environmental education; History; Museums; Science – Study and teaching; Computer software – development; Software engineering Yleinen suomalainen asiasanasto: opetusteknologia; tietokoneavusteinen opetus; konteksti; kontekstuaalisuus; integraatio; arviointi; oppiminen; oppimispelit; tietokonepelit; mobiilipelit; mobiililaitteet; matematiikka; ymp¨ arist¨ otietoisuus; historia; museot; tiedekasvatus; ohjelmistokehitys Keywords: context-awareness; informal learning; mobile-based learning; context-aware learning space; system architecture; technology integration; technology integration evaluation

Preface I am deeply grateful to my supervisors Professor Erkki Sutinen and Associate Professor Mike Joy who ensured that this research did not sidetrack too much. They shared their wisdom over numerous conversations which helped me not only to become a proficient author but also a critical thinker. While long talks with Erkki inspired me to release my creativity in ways which I thought were not possible, discussions with Mike helped me to scrutinise and rationalise my research so that it made sense. I owe my appreciation to Professor Mike Sharples and Associate Professor Hiroaki Ogata for performing amazing work while preexamining this dissertation, which now meets the high standards of academic quality. I would also like to express my gratitude to the defense opponent Professor Alfredo Terzoli for challenging me so as to ensure the appropriateness of this research. Carolina Islas Sedano and Mikko Vinni are the two most important colleagues with whom I was honoured to work during the past five years. Without this e↵ective collaboration I probably would have never ended up doing a PhD. It was Carolina who had a magnificent idea of creating a pervasive game for the SciFest 2007. She invited Mikko and me to join the team, and the rest is history. Together we have gone through ups and downs of academic work, lending a hand to each other in developing games, evaluating, paper authoring and overcoming researcher’s blocks. The year that I spent in South Korea (2007-2008) would not have been possible without the great hospitality of Professor Chaewoo Lee from Ajou University who invited me to work in his Multimedia Networking Laboratory (MNLab). I am grateful to the students of the MNLab who taught me the true meaning of hard work and dedication to research. Specifically, I am thankful to Jinchul Choi and Kitak Yong who contributed to the creation of the Heroes of Koskenniska

game. Other individuals to whom I am grateful for assisting in this research are: Eeva Nygren (née Turtiainen) for inventing the UFractions game and evaluating it together with me in Africa and in Finland; Anna Gimbitskaya and Ewa Kowalik for contributing to the Heroes of Koskenniska game; Professor Seugnet Blignaut and Andrew Smith for collaborating in research related to UFractions; Leenu Juurela, Riina Linna and Marianna Karttunen from the Museum of Technology in Helsinki for participating in the creation of TekMyst, TekGame and TekGuide; Liisa Eskelinen and Marketta Haavila from the Pielinen Museum for participating in the creation of LieksaMyst; and any other individuals whom I have forgotten to mention but who would deserve to be mentioned here for contributing to this research in any way. I sincerely thank you all! The School of Computing at the University of Eastern Finland (previously: the Department of Computer Science at the University of Joensuu) supported this research by granting research and travel scholarships as well as by providing research facilities and services. The East Finland Graduate School in Computer Science and Engineering (ECSE) granted me a scholarship during which a large part of the results of this research emerged. I am grateful to University of Eastern Finland for granting me several opportunities to conduct research abroad – these experiences will be extremely valuable in the future. I am thankful to our kind and knowledgeable secretaries Eeva Saukkonen and Tarja Karhu who accepted endless requests of information with smiles. Without their hard work and professionalism I would still be navigating through the red tape. The very foundations of my education can be attributed to my parents Kaisu and Pertti who raised me with love and care. They never doubted my choice of career and always encouraged me to become the first doctor in the extended family. So here I am, extremely grateful for all your support and hopefully making you proud! My wife Monika deserves the most of my gratitude. Without her unconditional love, support and encouragement I would have never

reached this point. Since 2002, she has witnessed my long ascent from a computer support guy to a university student and then to a PhD candidate. I have been very lucky to have her to stand by my side while conducting research in amazing places around Asia and Africa. She never questioned my progress even during the times when my research was nearly in standstill. Whenever I needed to go beyond the ordinary working hours (do such exist for a PhD candidate!?), Monika was very understanding and supported me in every possible way. Finally, I dedicate this work to my late grandmother Laina Laine. I was inspired by her willpower and sisu which helped me to go through the hard times.

In Helsinki, November 30, 2011

Teemu H. Laine

LIST OF PUBLICATIONS This dissertation presents the outcomes of the author’s research on context-aware learning spaces in the field of educational technology. The following publications have been selected to be part of the dissertation: I Laine T.H. & Joy, M. (2009). Survey on Context-Aware Pervasive Learning Environments, International Journal of Interactive Mobile Learning (I-JIM), vol. 3, no. 1, pp. 70-76. II Laine, T.H., Islas Sedano, C., Vinni, M. & Joy, M. (2009). Characteristics of pervasive learning environments in museum contexts, Proceedings of the MLEARN 2009 Conference, Orlando, Florida, pp. 26 - 34. III Laine, T.H., Vinni, M., Islas Sedano, C. & Joy, M. (2010). On designing a pervasive mobile learning platform, ALT-J Research in Learning Technology Journal, vol 18, no 1, pp. 3 - 17. IV Laine, T.H., Islas Sedano, C., Sutinen, E. & Joy, M. (2010). Viable and portable architecture for pervasive learning spaces, Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia, Limassol, Cyprus, pp. 1 - 10. V Laine, T.H, Islas Sedano, C., Joy, M. & Sutinen, E. (2010). Critical factors for technology integration in game-based pervasive learning spaces, IEEE Transactions on Learning Technologies, vol 3, no 4, pp. 294 - 306. VI Laine, T.H., Sutinen, E., Joy, M.S. and Nygren, E. (2011). Active and passive technology integration in context-aware learning spaces. In Proceedings of the AACE E-Learn 2011 Conference, Honolulu, Hawaii, pp. 719 - 728. VII Laine, T.H., Sutinen, E., Joy, M.S. and Nygren, E. (2011). Rapid improvement of technology integration in context-aware learning spaces. In Proceedings of the IEEE Africon 2011 Conference, Livingstone, Zambia, pp. 1 - 6.

These publications will be referred to by Roman numerals throughout the dissertation. Chapter 2 links the publications to research questions.

AUTHOR’S CONTRIBUTION The publications selected to be part of this dissertation are original research papers on context-aware learning spaces (CALSs) and technology integration in them. The author was the primary contributor to the ideas and the manuscripts. The ideas for papers IV-VII emerged through discussions with Professor Sutinen. The CALSs which are described or referred to in papers II-V have been created in collaboration with Carolina Islas Sedano and Mikko Vinni who have also provided comments for those papers and co-created all evaluation designs for these CALSs. Islas Sedano’s role has been the game and user interface designer while Vinni and the author have created the technical foundations for the CALSs. Specifically, Vinni and the author have created the Myst platform. The author created the first version of the HoK platform in 2009 with a group of computer science students and Vinni joined the author to develop the platform further in 2010. The idea and the designs for the UFractions game presented and evaluated in papers VI-VII were originated by Eeva Nygren (née Turtiainen) and the author was responsible for the technical development of the game. Furthermore, Nygren and the author jointly designed and conducted evaluations for papers VI-VII. Dr Joy and Professor Sutinen contributed to papers in which they appear as co-authors by commenting paper drafts and giving ideas for improvements.

LIST OF TERMS AND ABBREVIATIONS 2D bar code Two dimensional bar code – an optical smart tag solution A↵ordance An enabling feature of an object or an environment that allows an individual to perform an action Android A mobile platform developed by Google Backend Part of a software that is hidden from the user and typically contains the business logic CALS Context-Aware Learning Space Client-server architecture An approach to system architecture where multiple clients connect to a centralised server component Constraint A limiting force that sets restrictions to the use of technology in pedagogy Context A collection of interrelated contextual entities Context-awareness Ability of a system to recognise and act upon changes in consecutive situations (i.e. temporal snapshots) in a context Context-aware learning A form of learning which utilises the resources of the surrounding context Context-free resource Resouce that is not dependent on a given context Context-sensitiveness Ability of a system to adapt to the user’s situation Contextual entity A property which can be used to describe a certain aspect of a context, e.g. time, weather, user’s location or an object residing within the context

Contextual resource A contextual entity that can be observed by a given set of context-aware technologies Data source A system component that provides data to the system, e.g. a sensor Disturbance factor An element in a CALS that has a negative e↵ect on the learners Frontend User interface of a system GPS Global Positioning System iOS A mobile platform developed by Apple J2ME Java 2 Mobile Edition – a Java platform for mobile phones Java A programming language originally developed by Sun Microsystems Learning space A combination of physical and virtual realities in which the learning takes place MeeGo A mobile platform originally developed by Nokia and Intel Middleware A software component that mediates data communication between data sources and a server backend Mobile learning A form of learning in which the learners have time and location independent access to learning resources via a mobile device Model-View-Controller A software architecture pattern where software comprises a model (data), a view (representation of the data) and a controller (methods to interact with the software) MUPE Multi-User Publishing Environment – a mobile publishing platform developed by Nokia NFC Near Field Communication – a smart tag solution based on RFID

Pervasive learning A special case of context-aware learning where the learning is connected to a specific context PLS Pervasive Learning Space RFID Radio Frequency IDentification – a short-range wireless communication technology comprising RFID reader devices capable of reading RFID tags Sensor A hardware component that is capable of detecting changes in a contextual entity Situation [in a context] A snapshot of a context at a given moment of time Smart tag A tagging technology that can be used to make objects detectable by a context-aware system Technological Pedagogical Content Knowledge (TPCK) A technology integration framework suggesting that a competent teacher should master knowledge of technology, pedagogy and content matter Technology integration A process by which a technology is introduced to a pedagogical setting Ubiquitous learning A special case of context-aware learning where the learning spans across contexts ULS Ubiquitous Learning Space XML eXtensible Markup Language

LIST OF FIGURES 2.1 2.2

Research objectives and outcomes . . . . . . . . . . . CALSs and platforms created during this research . .

3.1

3.3 3.4 3.5

Examples of changes in the contextual entities between two consecutive situations . . . . . . . . . . . . . . . . Detection of changes between situations in a contextaware system . . . . . . . . . . . . . . . . . . . . . . . Comparison of two context-aware system frameworks . Types of learning . . . . . . . . . . . . . . . . . . . . . Tool Integration Process model . . . . . . . . . . . . .

19 20 22 26

4.1 4.2 4.3 4.4 4.5 4.6 4.7

Working principle of MUPE . . . . . . . . . . . . . . . The Myst platform architecture . . . . . . . . . . . . . SciMyst player recording a 2D bar code . . . . . . . . Excerpts from the story of Jussi the forest worker . . . An example of a mathematical challenge in UFractions The HoK platform architecture . . . . . . . . . . . . . Two players solving a challenge in Heroes of Koskenniska

30 32 34 36 37 39 40

5.1 5.2 5.3

Technology integration (TI) process in a CALS . . . . Technology integration model for CALSs . . . . . . . . Passive and active integration of technology in CALSs

44 45 47

3.2

7 9

17

LIST OF TABLES 2.1

Connections between research questions, papers and research methods . . . . . . . . . . . . . . . . . . . . .

6

3.1 3.2

Categories of context-aware technologies . . . . . . . . Examples of context-aware learning spaces . . . . . . .

18 23

4.1

CALSs based on the Myst platform . . . . . . . . . . .

31

A.1 Characteristics of CALSs . . . . . . . . . . . . . . . .

73

B.1 Learner’s role in the evaluation tool . . . . . . . . . . B.2 Educator’s role in the evaluation tool . . . . . . . . . . B.3 Context’s role in the evaluation tool . . . . . . . . . .

75 76 77

C.1 Disturbance UFractions C.2 Disturbance UFractions

factors found in the first evaluation . . . . . . . . . . . . . . . . . . . . . . factors found in the second evaluation . . . . . . . . . . . . . . . . . . . . . .

of . . of . .

80 81

Contents 1 INTRODUCTION

1

2 RESEARCH QUESTIONS AND DESIGN 2.1 Q1: What features characterise context-aware learning spaces (CALSs) within the domain of mobile-based learning tools? . . . . . . . . . . . . . . . . . . . . . . 2.2 Q2: How can a CALS platform be constructed? . . . . . . . . . . . . . . . . . . . . . . . 2.3 Q3: How can technology integration be taken into account in the design phase of CALSs? . . . . . . . . . . 2.4 Q4: How can technology integration in CALSs be evaluated? . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

3 BACKGROUND 3.1 Informal learning . . . . . . . . . . . . . 3.2 Context-awareness . . . . . . . . . . . . 3.3 Context-aware learning spaces (CALSs) 3.4 Technology integration in education . . 3.5 Summary . . . . . . . . . . . . . . . . . 4 PLATFORMS FOR CALS 4.1 The Myst platform . . . . . . . . . 4.1.1 SciMyst 2007-2009 . . . . . 4.1.2 ADEMyst . . . . . . . . . . 4.1.3 TekMyst . . . . . . . . . . . 4.1.4 EdTechMyst . . . . . . . . 4.1.5 LieksaMyst . . . . . . . . . 4.1.6 UFractions . . . . . . . . . 4.2 The HoK platform . . . . . . . . . 4.2.1 Heroes of Koskenniska . . . 4.2.2 TekGuide and TekGame . . 4.3 Characteristics of a reusable CALS

. . . . .

. . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . platform

. . . . .

. . . . . . . . . . .

. . . . .

. . . . . . . . . . .

. . . . .

. . . . . . . . . . .

. . . . .

. . . . . . . . . . .

. . . . .

. . . . . . . . . . .

6 8 10 12

. . . . .

15 15 16 21 24 27

. . . . . . . . . . .

29 29 33 33 34 35 35 36 38 39 40 42

5 TECHNOLOGY INTEGRATION IN CALS 43 5.1 Technology integration model . . . . . . . . . . . . . . 44 5.2 Technology integration evaluation tool . . . . . . . . . 48 6 DISCUSSION

51

7 CONCLUSION

57

BIBLIOGRAPHY

60

A CHARACTERISTICS OF CALSs

73

B EVALUATION TOOL FOR TECHNOLOGY INTEGRATION IN CALSs 75 C DISTURBANCE FACTORS

79

D EVALUATION INSTRUMENTS

83

ORIGINAL PUBLICATIONS

91

1 Introduction Context-aware learning spaces (CALSs) are mobile-based learning environments, typically deployed to informal pedagogical settings, which utilise surrounding contextual elements (e.g. museum exhibits) in the learning process. The term learning space refers to a combination of physical and virtual realities in which the learning takes place. A typical CALS comprises a number of mobile devices (clients), wireless connectivity, a server, a set of context-aware technologies and a collection of context-sensitive learning content and activities. By being aware of the context, CALSs are able to connect the learning content to the surrounding context so as to motivate the learner to explore the physical environment in a way that has not been possible in traditional mobile learning [21]. This dissertation lays the foundations for CALS development from four perspectives. First, the CALS concept is defined and its characteristics are described so as to position this research in the domain of mobile-based learning tools. Secondly, the process of creating the technical infrastructure for a CALS is reviewed through the development history of two CALS platforms on which ten game-based CALSs were constructed. These CALSs were created for various purposes in diverse contexts including, but not limited to, raising environmental awareness in a Finnish forest (paper IV), taking a visitor back in time to meet people from the past in an open air museum [40], providing a science festival visitor an alternative way of exploring exhibitions and workshops [38] and teaching mathematics to South African children with a story-based game [87]. Thirdly, the role of technology integration in the design process of a CALS is analysed and a model for technology integration is proposed to assist the CALSs developers to deploy appropriate technologies to meet various requirements of a CALS. Fourthly, a technology integration evaluation tool is proposed to support iterative improvement of CALSs. CALSs have a degree of context-awareness through which they can detect changes in the surrounding context and act upon those

Dissertations in Forestry and Natural Sciences No 59

1

Teemu H. Laine: Technology Integration in Context-Aware Learning Spaces

changes. This means that a CALS delivers content to the learner based on the learner’s context, for example where they are, what time it is, what they are doing, who else are with them, how are they feeling and what is the state of the surrounding environment. The CALS uses context-aware technologies such as sensor devices [59], smart tags [2] and positioning [37] to detect these versatile contextual resources. Utilisation of the contextual resources to facilitate learning processes in a CALS depends on pedagogical and design objectives as well as availability of resources (e.g. time, money, knowhow). In an optimal case a CALS provides highly personalised and contextually relevant learning content to the learners based on the situations in which they are embedded. For example, changes in the environmental parameters such as temperature, humidity and illumination between di↵erent physical locations in a forest can be used to inform the learner of the features of di↵erent microclimates in these locations [50]. During four years of research and development, two CALS platforms and ten CALSs based on the platforms were developed. The incremental nature of the development was based on the ideas and the solutions of the previous stages. Various technologies were utilised in the process including, but not limited to, mobile devices, wireless networks, sensor devices and smart tags. The method of acquiring context-awareness was mostly based on detecting the learner’s position and the presence of physical objects within the context. In one case an environmental sensor network was implemented so as to acquire deeper information on the context (paper IV). Due to the presence of technology in CALSs, an investigation into the concept of technology integration was deemed to be necessary. Technology integration refers to the process by which a technology is introduced into a classroom so that the teacher and the students can use it e↵ectively for pedagogical purposes [20]. Poor technology integration may lead to disruptions in teaching and learning or to wasted technology resources. While technology integration has been typically researched in the context of formal classroom-based education, it is clear that the same challenge is present in informal learning

2


Introduction

contexts as well. This is particularly true in the case of CALSs due to the technology’s central role in them. For example, disregarding the influence of the technology on the learner and on the context in a museum-based CALS could result in poor learning experiences and even annoyance. Because the concept of a CALS is novel, there has not yet been research aimed at establishing the foundations of technology integration in CALSs. This dissertation summarises the results of seven original research papers (I-VII) under seven chapters as follows. First the research questions and design are described, including the methods which were applied to answer the questions. Then follows the background in Chapter 3 where the research is positioned in the field of mobile-based learning and central concepts are defined. In Chapter 4, overviews are presented of the two CALS platforms and the ten CALSs created on the platforms. Technology integration in CALSs is covered in Chapter 5. Specifically, the technology integration model is introduced in Section 5.1 and a tool for evaluating technology integration in CALSs is described in Section 5.2. Finally, the implications of the results are described and analysed in Chapter 6 before concluding the work in Chapter 7 with suggestions for future research activities.


3


4


2 Research questions and design There are four objectives for this dissertation: (i) to position the concept of a CALS within the field of mobile-based learning tools; (ii) to design and implement a reusable platform for CALSs; (iii) to explore the role of technology integration in CALS development and to find a model to facilitate it; and (iv) to evaluate technology integration in CALSs. While these goals represent both practical and theoretical perspectives of CALS development, they share the same overall objective: to provide the developers with the tools for construction and evaluation of CALSs in which technologies have been integrated e↵ectively so that they do not disturb the learner. The four goals are answered by four research questions which use various methods. The research questions are summarised in Table 2.1 together with references to the relevant papers, research methods and chapters in which the questions are answered. More detailed descriptions of each research question, its purpose and its methods are presented in the following sections. The research work presented in this dissertation was conducted between 2007 and 2011 in Finland, South Korea, South Africa and Mozambique. It was an exploratory journey during which ten gamebased CALSs, two CALS platforms, a technology integration model and a technology integration evaluation tool were created. Each game, except the first one, built on the foundations and experiences of the previous games. The platform development paths followed closely the development sequences of the games. New game designs uncovered new requirements that had to be dealt with by platform development activities. Through the experiences acquired by the research and development activities, a thought emerged that it would be beneficial to know how to integrate various technologies with CALSs in an e↵ective manner so that the end result would provide good learning experiences. Then started the theoretical work towards a technology integration model which was further extended with an evaluation


5


Table 2.1: Connections between research questions, papers and research methods Papers Methods

Ch

Q1 What features characterise contextaware learning spaces (CALSs) within the domain of mobile-based learning tools? Q2 How can a CALS platform be constructed?

Research question

I, II

Literature analysis

3

III, IV

4

Q3 How can technology integration be taken into account in the design phase of CALSs? Q4 How can technology integration in CALSs be evaluated?

II, V, VI

Exploratory software development, mixed method evaluation Literature analysis, artefact analysis Literature analysis, mixed method evaluation

VII

5

5

tool. An illustration of the objectives and outcomes of this research as well as relations between the components is presented in Figure 2.1. All evaluations performed in this research were carefully designed and necessary precautions were taken to ensure the anonymity of the participants. The participants or their guardians also signed research consent forms before partaking in any activities which involved research data collection. 2.1

Q1: WHAT FEATURES CHARACTERISE CONTEXTAWARE LEARNING SPACES (CALSs) WITHIN THE DOMAIN OF MOBILE-BASED LEARNING TOOLS?

By answering this research question I aim at defining the concept of a context-aware learning space and position it in the domain of mobile-based learning tools. This is necessary so as to give the reader a perspective on the topic of the dissertation. Chapter 3 answers this research question.

6


Research questions and design

Figure 2.1: Research objectives and outcomes

The research method for answering Q1 is literature analysis. Literature was acquired by systematically querying popular scientific search engines such as Google Scholar, ACM Digital Library and IEEE Xplore, and then following relevant references of the discovered articles. Additionally, the following conferences and workshops were used as data sources for paper I: • IEEE International Conference on Pervasive Services (20052007)


7


• IEEE International Conference on Pervasive Computing and Communications (2003-2007) • European Conference on Ambient Intelligence (2007) • International Conference on Mobile and Ubiquitous Systems (2004-2007) • IEEE International Workshop on Wireless and Mobile Technologies in Education (2002-2007) • Pervasive E-Learning Workshop (2004-2007) • Pervasive Computing Education Workshop (2004-2007) • Pervasive Learning Workshop (2007) In paper I the literature analysis focused on state-of-the-art systems, the technologies used, the roles of mobile devices and the applied learning models. In paper II the focus was on pedagogical approaches for informal learning settings, including situated learning, authentic learning, contextual learning, group-based learning, exploratory learning, problem-based learning and museum learning. Finally, part of the literature analysis of paper II establishes a division between various mobile-based learning approaches which form the basis for the concept of a CALS. 2.2

Q2: HOW CAN A CALS PLATFORM BE CONSTRUCTED?

The aim of this research question is to identify the properties of a reusable CALS platform and to describe the process of building it. Results are useful for researchers who wish to establish their own reusable CALS platforms. Chapter 4 answers this research question. Two platforms and ten game-based CALSs were developed during this research. From the perspective of software development, the Exploratory Software Development (ESD) [86] method with an iterative structure was used in the development. Trenouth [86] suggests that

8



the ESD method is suitable in situations where a client is unclear about the requirements or if a specification is unavailable because the domain of application is poorly understood. In the case of this research it was the latter that required the use of an exploratory method of software development. The process was iterative in that the development of the CALSs were based on previous CALSs, except for SciMyst 2007 which was the first one. When a new CALS was created, the platform was adapted to the new requirements while retaining the flexibility. At some point the requirements set by a CALS concept required creation of a completely new platform, hence this research proposes two platforms. The overall evolution of the platforms and the CALSs is presented in Figure 2.2.

Figure 2.2: CALSs and platforms created during this research

From the game design perspective all CALSs except UFractions were created by using the Hypercontextualised Game (HCG) design approach [40]. In the HCG, the game is deeply rooted in the same context in which the player is embedded. The process emphasises


9


creativity, innovation, self-expression and the knowledge of the stakeholders who are involved in the design process from the beginning. The design work is based on workshops which are orchestrated by an HCG expert with an aim to identify and utilise meaningful resources from the context in the game play. UFractions was designed by using the principles of the contextual design method [64] whereby the designer aggregates data from the target context and then designs a product based on the data. The designer of UFractions (Eeva Nygren) spent three months in South Africa to collect materials and to design the concepts and the contents for UFractions together with local experts on pedagogy and culture. A mixed method approach [10] combining quantitative and qualitative strategies was used to evaluate all CALSs except ADEMyst and EdTechMyst, but not all evaluations have been published. The evaluations aimed at gathering learners’ background information, opinions, motivation, suggestions for improvements and a varying number of other parameters. Data collection methods were primarily pre-test and post-test questionnaires supplemented in some cases with qualitative interviews and observations. Questionnaires had both quantitative multiple choice statements and qualitative open questions. Approaches for data analysis were study-specific, but typically average and standard deviation calculations were applied to quantitative data while supporting evidence was extracted from qualitative data. 2.3

Q3: HOW CAN TECHNOLOGY INTEGRATION BE TAKEN INTO ACCOUNT IN THE DESIGN PHASE OF CALSs?

Through this research question I aim at establishing a model of technology integration which facilitates the CALS development process. The model can be used by CALS developers to plan technology integration so that the outcome will not hinder or distract from the learning process. This research question is answered in Chapter 5 under Section 5.1. To answer the third research question, two methods were applied:

10



literature analysis and artefact analysis. The theoretical foundations of the technology integration model were based on two literature analyses which are described in Section 2.1. Additionally, several smaller scale literature analyses were conducted which investigated for example the aspects of learning in museums, CALSs for specific themes such as environmental education, the use of wireless sensor technologies, and the requirements of technology integration in educational settings. These secondary, but equally important, pieces of information also contributed to the creation process of the technology integration model. Artefact analysis is a research method which has been typically used in fields such as archaeology, history and arts to research on human-made objects. The goal of an artefact analysis is to reach a deeper understanding about an artefact and its usage than what would be possible by mere direct observation. The artefacts (i.e. CALSs) were analysed from two perspectives [69]: 1. artefacts as designed – looking at the ways in which the explicit and implicit knowledge of the designer are exposed in artefacts 2. artefacts as used – looking at the way on which people have appropriated, annotated and located artefacts in their work environment The as designed artefact analysis was used on developed CALSs in order to determine the key requirements which were met by the use of technology during the development processes. Specifically, the decisions regarding inclusions or exclusions of technologies and the methods of implementing them were scrutinised. In the as used artefact analysis, usability, users’ perceptions and experiences were evaluated for several CALSs. A mixed-method approach was used in this evaluation as explained in Section 2.2. The aim was to see how the design decisions, i.e. designers’ ideas on how technology would be integrated, were reflected in the real use scenarios. As an example, in the case of the LieksaMyst game (Section 4.1.5), the as designed analysis revealed that simple wooden tags were used instead


11


of state-of-the-art object tagging technology because the context’s culture sought to preserve authenticity (paper V). From the users’ perspective the wooden tags worked without problems but there were some other usability issues which indicated problems with technology integration. 2.4

Q4: HOW CAN TECHNOLOGY INTEGRATION IN CALSs BE EVALUATED?

To answer this research question a tool was created for evaluating technology integration. The tool is based on the technology integration model (paper V, Q3) and it can be used by CALS developers and stakeholders to evaluate how well the technology has been integrated into a CALS. Chapter 5 answers this research question. Before establishing the evaluation tool, the need for technology integration was determined by evaluating the UFractions game with a data set that was originally meant for measuring the e↵ect of reverse transfer of a learning technology from a technology-alien context (South Africa) to a technology-familiar context (Finland) [51]. A mixed-method approach was used for the evaluation as described in Section 2.2. Additionally, a qualitative data categorisation and analysis method was used, which was also used in the evaluation tool and is described in detail below. The results of the evaluation indicated the need for technology integration (see Section 5.1) but since the data set was not designed for evaluating technology integration, the work began towards creation of the evaluation tool. A literature analysis and the foundations of the technology integration model were used in the derivation of the evaluation tool for technology integration. Literature on technology integration in education was acquired and analysed, and the results combined with the technology integration model. Data were primarily collected by searching for articles related to technology integration in formal pedagogical settings. These findings were later complemented by analysing articles on technology appropriation and acceptance in the context of mobile learning.

12



Once the evaluation tool was constructed, it was used to evaluate UFractions, one of the CALSs developed during this research. The aim of the evaluation was not only to evaluate technology integration in UFractions but also to evaluate the feasibility of the technology integration evaluation tool. The evaluation utilised a mixed-method approach combining both qualitative and quantitative strategies. A descriptive analysis of the quantitative data was performed with mean and standard deviation calculations as well as Pearson correlation metrics. The findings were then supported by qualitative data. A mixed-method approach was chosen to get not only meaningful statistical results but also deeper complementary insights on the participants’ views on the technology integration. Specifically, qualitative data were used to support and explain the quantitative conclusions that were drawn from the statistical analysis of the data set. Furthermore, qualitative data categorisation and analysis were the key methods for identifying disturbance factors which a↵ect negatively the users of UFractions. First, a set of indicators were established and based on them the factors from the open questions and interviews were identified. The categorisation was coded according to the types of negative responses that the participants gave in open questions and interviews. The data collection techniques used in the evaluation included interviews, questionnaires, recording of application usage statistics, and observations. Interviews were based on a set of prepared questions with an option to apply clarifying questions. Questionnaires had both closed and open-ended questions, and observation remarks were done by hand by a researcher during the experiments.


13


14


3 Background 3.1

INFORMAL LEARNING

Informal learning occurs outside formal learning settings and it complements formal educational systems. It has been estimated that a good majority of learning takes place in informal contexts [54,85]. In the past, before the establishment of the institution of formal education, informal learning was the prevalent way of education. People would learn and share knowledge by visiting a neighbouring village to exchange news or developing a new handicraft skill as an apprentice guided by a master. This learning from experience is a key characteristic of the informal end of the continuum of formality in learning [17]. Other characteristics include, but are not limited to, implicit, unintended, opportunistic and unstructured ways of learning and the absence of a teacher [19], as well as contextual (organisational) embeddedness, action orientation, non-routine conditions, tacit dimensions, and a requirement for critical reflectivity and creativity [89]. In informal learning, the context in which the learning takes place is not solely dedicated to the purpose of learning, but learning just happens to take place there as a secondary function. This aspect is di↵erent from formal classroom-based education where the primary function of the context is to foster learning and teaching. The richness of and the interest raised by the surrounding context may increase the intrinsic motivation of the learner [13], which in turn may lead to flow, a state of mind in which the learner is completely immersed in the learning process [14]. The involvement of the context also works as a catalyst for educators to implement alternative learning activities which are connected to objects and phenomena within the context.


15


3.2

CONTEXT-AWARENESS

In this research, context is understood as a collection of interrelated contextual entities. A situation is defined as a snapshot of a context at a given moment of time. Contextual entities may be identified, for example, by knowing where the users are, what they are doing, how they are feeling, who else is with them, what resources are nearby, what time it is and what the parameters of the physical environment are. Examples of contextual entities include the current time, the current weather, and the physical location of the user (e.g. geographical coordinates). Zimmermann et al. define a contextual entity as an element which can be described by using categories of individuality, activity, location, time and relations [93]. From this follows that Zimmerman et al.’s contextual entity can be observed from many different observation points whereas in my definition a contextual entity corresponds to a single observation point. This research concentrates on real world contexts, i.e. contexts which are situated in physical spaces. A context can be divided further into subcontexts. For example, a museum context comprises the physical context (museum building, rooms, objects), the sociocultural context (of sta↵, visitors), the temporal context (time of the day/week/month/year), the political context (museum policies), the pedagogical context (learning material and objectives), and the personal contexts (previous experiences, skills, preferences) of the visitors. Figure 3.1 illustrates how some of the contextual entities of the Heroes of Koskenniska game (see Section 4.2.1) change between two situations which are separated by time. These contextual entities are not comprehensive – others could be for example user’s rate and direction of movement, user’s posture, user’s spatial relationship to nearby users, relevant people available online, and so forth. From the definition of context follows context-awareness, which is defined as a property of a system to recognise and act upon changes in consecutive situations (i.e. temporal snapshots) in a context. In order for a system to be context-aware it needs to utilise context-aware technologies. Based on my understanding of the field, technologies

16


Background

Figure 3.1: Examples of changes in the contextual entities between two consecutive situations

used in context-aware systems can be roughly categorised into front ends, wireless networking, input technologies, output technologies, smart tags, sensor devices, middleware and back ends. Table 3.1 presents these categories together with explanations and examples. With regard to the example in Figure 3.1, various sensors could be used to detect the status of the environment as well as of the users. The current version of Heroes of Koskenniska utilises temperature, illumination and humidity sensors. The user’s location is detected by solving riddles at specific locations (Magic Spots). Figure 3.2 illustrates how a context-aware system detects changes between two consecutive situations in a context. Specifically, the system detects changes ( 1 and 2 ) in contextual resources (Entity1 and Entity2 ) which form a subset of contextual entities that can be observed by a given set of context-aware technologies and then utilised by the system. In contrast, Entity3 is not a contextual resource because it is not detectable by the context-aware technologies. Context-free resources are not dependent on a given context (e.g. a


17


Table 3.1: Categories of context-aware technologies

Technology

Explanation

Examples

Front end (mobile device) Wireless networking

Connects user to the back end and to other users Relays data between wireless entities

Input vices

Receive data and commands from the user

Use mobile device to send and receive data to/from the contextaware system Collect data from a wireless sensor network and relay them to the middleware Use movement-based sensors (e.g. Microsoft Kinect) to allow natural interaction with the system Use an augmented reality software module to view 3D objects through the mobile device’s camera viewfinder Attach two dimensional bar codes or RFID tags to museum objects in order to make them detectable Adapt learning activities according to learner’s location (GPS) and stress level (skin conductance sensor) Wireless sensor network sends collected readings to the middleware which sends it to the back end and monitors data integrity at the same time A context-aware game engine which presents challenges to the user based on context information acquired through middleware

de-

Output devices

Present content to the user

Smart tags

Link physical objects or places to the context-aware system Detect parameters of various contexts (e.g. physical, personal)

Sensor vices

de-

Middleware

Back end

18

Relay contextual data between the data providers (sensors, input devices, smart tags) and the back end Contains business logic of the context-aware system


Background

theory or general knowledge of a topic). By being aware of the contextual resources and changes in them, a system can adapt both the contextual and context-free resources to fit the user’s current situation. As a result, the system provides the user with context-sensitive materials and activities with high relevance to the user’s situation. In the case of Heroes of Koskenniska, contextual resources form a subset of all possible contextual entities. These contextual resources include timestamps, the user’s location (coarse and fine), temperature, humidity, illumination, nearby users and previously visited spots.

Figure 3.2: Detection of changes between situations in a context-aware system which produces context-sensitive materials as output

The amount of context-awareness required is specific to the application – in some cases knowing the user’s location within a geographical area is enough [4] whereas in other applications it may be necessary to detect the parameters of the surrounding environment [50, 59]. Sometimes it may even be necessary to detect changes in the user’s body [3]. As creating a highly context-aware system requires money and time, trade-o↵s are necessary in the design process. Furthermore, state-of-the-art technology can also become a burden if it is not integrated properly, resulting in a situation where the technology is disrupting the user experience [44] or the system simply does not work because of a lack of technical maintenance skills. There are other works which discuss context-aware systems and the role of context in them (e.g. [5,18,55]). My definition of a context-


19


aware system in Figure 3.2 relates closely to Lonsdale et al.’s framework which is based on content recommendations [55]. In this framework a context subsystem uses metadata from the context, including the user, and the content to provide recommendations as to what type of content would be appropriate for the users in their current situations. The context is defined hierarchically so that the overall context comprises context states, context substates and context features. Figure 3.3 compares the Lonsdale et al.’s context-aware system framework to mine (see Figure 3.2). The main di↵erence between the two frameworks is that my framework also accounts for such contextual entities that are not currently detectable by the system (i.e. Contextual Entity 3 in Figure 3.3) but could be included in a later stage of development. Context Substate does not correspond to a set of Contextual Entities because the former leaves out some of the potential Context Features.

Figure 3.3: Comparison of two context-aware system frameworks: Lonsdale et al. [55] (left) and mine (right)

20


Background

3.3

CONTEXT-AWARE LEARNING SPACES (CALSs)

Context-aware learning is a fairly new concept in the domain of educational technology and it utilises resources of the surrounding context. Context-aware learning typically takes place in informal learning settings where the context is rich in terms of learning possibilities. It builds on the foundations of mobile learning (m-learning) in which the learners with mobile devices have time and location independent access to learning resources (see e.g. [21, 62, 77]). A major challenge in m-learning is that the richness of the surrounding context is not considered. This means that the same learning material can be studied at home, at school, in a bus or in a park, thus the surrounding context is disregarded and the learner’s attention can be too much focused on the mobile device’s screen [23]. As a remedy for ignoring the contextual relevance in m-learning, context-aware learning (also built on portable handsets) integrates the surrounding contextual resources into the virtual learning content. A mobile handset delivers context-sensitive instructions and learning tasks to the learner and provides relevant feedback upon the learner’s actions. Because of the context-sensitiveness of the learning content, the learner is encouraged to make observations of and to interact with surrounding objects and phenomena. Context-sensitiveness is achieved by context-aware technologies as described in Section 3.2. A learning environment which makes use of a context-aware system is referred to as context-aware learning space (CALS). The term “learning space” in this case refers to a combination of physical and virtual realities in which the learning takes place. A typical CALS comprises a number of mobile devices (clients), wireless connectivity, a server, a set of context-aware technologies and a collection of context-sensitive learning content and activities. CALSs can be further divided into pervasive learning spaces (PLSs) and ubiquitous learning spaces (ULSs). Although the terms “pervasive learning” and “ubiquitous learning” are sometimes used as synonyms, this research distinguishes the two at the level of the learner’s mobility in respect to the context. A PLS is built for a specific context (e.g. a mu-


21


seum) whereas a ULS spans across several contexts (e.g. multiple locations in a city). Figure 3.4 illustrates the di↵erences between pervasive learning, ubiquitous learning, mobile learning and desktop computer based learning in the domains of learner’s mobility and context-awareness. This division was adapted from the idea of Lyytinen and Yoo [57] and it has also been elaborated by Ogata and Yano [66].

Figure 3.4: Types of learning according to context-awareness and learner mobility

Many research projects have been initiated to build learning environments which are context-aware at various levels. A selection of these CALSs are presented in Table 3.2 which also indicates whether the systems are ULSs or PLSs, i.e. do they span across di↵erent contexts or not. Information on context-awareness in the systems is also provided. Context-awareness is most commonly achieved by establishing position of the users but these examples show that also more advances technologies have been used, such as sensors.

22


Background

Table 3.2: Examples of context-aware learning spaces CALS

Purpose

PLS/ ULS

Context-awareness

Support contextualised scientific inquiry during school field trips in forests Augmented Facilitate children’s playing Knight’s Cas- and learning by technologitle [32] cally augmented toys GreenSweeper [36] Increase awareness of green areas in an urban environment through a game similar to Mine Sweeper ContextLearn second language vosensitive micabulary through interaction crolearning with everyday objects environment [6] Cyberguide [1] Provide a context-sensitive tour to the visitors of a research laboratory Environmental Support learning of enviDetectives [47] ronmental science through a multiplayer real-world simulation game JAMIOLAS [34] Learn Japanese mimicry and onomatopoeia through context-aware learning activities LORAMS [65] Learn and share everyday tasks through context-aware videos Nottingham Receive context-sensitive inCastle Museum formation on paintings in a gallery (MO- gallery BILearn) [56] REXplorer [4] Support touristic contextaware exploration of Regensburg city through a game Via MinerExplore a mineral collecalia [31] tion through a treasure hunt game

PLS

Light and humidity sensors, GPS for user positioning RFID for object detection

Ambient Wood [71]

ULS

ULS

PLS

PLS

Manually inserted used location, captured images for detecting greenness Object/furniture usage sensors, water flow sensors, accelerometers, RFID Infrared positioning

ULS

GPS for user positioning

ULS

Wearable and wireless sensors, RFID for user positioning

ULS

RFID for object detection

PLS

Ultrasound tracking for user positioning

PLS

Camera-based motion detection, GPS for user positioning RFID for object detection

PLS


23


To establish characteristics of a CALS, two literature analyses were conducted. In the first literature analysis (paper I) it was discovered that mobile devices can be used in a CALS for data collection, content representation, communication, navigation and notifications. The same analysis suggested that client-server architectures were prevalent in the systems. The popularity of the client-server approach can be explained by the fact that the processing power of mobile devices may be too limited for many use cases. Additionally, it makes content maintenance easier as well as enabling multiuser features. Finally, the first literature analysis indicated the lack of a pedagogical model for CALSs. In the second literature analysis (paper II), inspired by the lack of a pedagogical model, a set of pedagogical characteristics for pervasive learning spaces was derived (PLSs, a subset of CALS). The literature focused on constructivist approaches to situated learning, authentic learning, contextual learning, group-based learning, exploratory learning, problem-based learning and museum learning. I combined appropriate characteristics of these approaches as none them alone was suited for a PLS. As a result, 15 characteristics were identified which were later extended to 18 (paper V). The characteristics, presented in Table A.1, are categorised into five groups: (i) user profiles and perspectives; (ii) interaction and collaboration; (iii) ownership; (iv) authenticity and relevance; and (v) support and assessment. These characteristics can be used to evaluate the potential of a PLS as a learning tool. The results can be applied also to ULSs and therefore to all CALSs because a ULS can be thought of as a collection of interrelated PLSs [51]. 3.4

TECHNOLOGY INTEGRATION IN EDUCATION

The term technology integration refers to the process by which a technology (typically digital) is introduced into a classroom so that the teacher and the students can use it e↵ectively for pedagogical purposes. Technology integration in formal education has been researched extensively [7, 20, 53, 60, 81, 82].

24


Background

Koehler and Mishra [48] have introduced the TPCK (Technological Pedagogical Content Knowledge) framework for technology integration in formal education. The TPCK framework is based on Shulman’s PCK (Pedagogical Content Knowledge) model which suggests that a competent educator should master both knowledge of pedagogy and knowledge of content [79]. Koehler and Mishra complemented the PCK model by adding the component of technology to it so as to meet the requirements of the current trends in pedagogy where various educational technologies are being increasingly combined. Koehler and Mishra are not the first authors attempting to attach a technology component to the PCK model of Shulman. Earlier related works such as Keating and Evans [45], Pierson [67] and Niess [63] also used the PCK model when addressing the challenge of technology integration in classrooms. This research uses Koehler and Mishra’s framework as a reference due to its comprehensive and clear representation. Furthermore, the distinction and analysis of di↵erent combinations of the core components can be attributed to them. Koehler and Mishra utilise the concepts of a↵ordances and constraints in the technology integration process [48]. A↵ordances are enabling features of an object or an environment that allow an individual to perform an action1 . Constraints are a limiting force which sets restrictions to the use of technology in regard to pedagogy. For example, teachers can use traditional chalkboards to present text and hand-drawn graphics (a↵ordances) but they are not able create multimedia animations with them (constraint). Together a↵ordances and constraints define how a technology can and cannot be used for educational purposes in a given context. Although traditional instruments such as chalkboards, overhead projectors and posters can be considered as technologies for education, this research focuses more on digital technologies. The aforementioned studies on technology integration concentrated on formal classroom settings. There are also research which 1

http://en.wikipedia.org/wiki/Affordance (Retrieved: 2011)


November 23,

25


focus on technology appropriation and acceptance in the context of mobile learning. Waycott [90] proposes the Tool Integration Process (TIP) model for analysing how the possibilities and constraints of a new mobile tool can mediate or change activities that the tool was built to support. This way of adopting and shaping a technology for new purposes while it is being used is referred to as appropriation. The TIP model, as illustrated in Figure 3.5, uses the foundations of activity theory, thus integrating the concepts of actions, operations, contradictions and breakdowns into the flow of events. Being user- and task-centric, the TIP model does not take into account the e↵ects of the tool on the surrounding pedagogical setting (e.g. instructor and other learners) or to the surrounding context. However, it is useful for exploring how the learners adapt new technologies for new purposes that the designers of the technology did not consider.

Figure 3.5: Tool Integration Process (TIP) model for mobile learning (adapted from [90])

Jones and Issro↵ [43] analysed the motivational issues of mobile learning from two perspectives: technological appropriation ( [11,90]) and Järvelä et al.’s model of coping strategies [41]. Through case studies they conclude that Waycott’s TIP model [90] is useful for understanding the larger contextual aspects of the use of mobile technologies, whereas the model of coping strategies is more applicable to small incidences of learning. As a third example of research on technology integration in mobile

26


Background

learning, Huang et al. complement the Technology Acceptance Model (TAM) [16] with two factors that they identified to cause individual di↵erences in mobile learning: perceived enjoyment and perceived mobility value [35]. Huang et al. show that these two factors can be used to predict user acceptance of mobile learning. This result may be useful for predicting the success of technology integration in mobile learning environment but may not be applicable to context-aware learning spaces which typically comprise a versatile set technologies in addition to a mobile device. 3.5

SUMMARY

This chapter summarised the central concepts of this dissertation and while doing answered the research question Q1: What features characterise context-aware learning spaces (CALSs) within the domain of mobile-based learning tools? Specifically, the chapter presented the interlinked concepts of informal learning, context-awareness, contextaware learning space and technology integration. A CALS is based on various context-aware technologies which, in turn, aid the CALS to connect the resources of the physical context to virtual learning content, thus transforming the physical context into a learning space. The ability of CALSs to utilise resources of the context (e.g. surrounding objects) in the learning process makes them particularly useful for informal learning activities in contexts such as museums, science centres, urban areas, national parks and fairs. In principle, CALSs can be applied to any context having rich learning contents so as to facilitate context-aware learning activities and therefore release the hidden pedagogical potential within the context. The majority of technology integration research e↵orts have concentrated on formal classroom-based education. However, it is equally important, if not more so, to consider the role and the e↵ects of technology integration in informal learning settings. The importance of proper technology integration is particularly high in CALSs where the technology plays a significant role in the learning process. Since the concept of a CALS is novel, technology integration research has


27


not yet been applied to it. This research is motivated because without a proper integration of technology a CALS may distract and annoy the learners instead of leading them to the flow [14].

28


4 Platforms for CALS Four years of research and development activities culminated in the creation of two CALS platforms: the Myst platform and the HoK platform. Both platforms were created for game-based CALSs but they can be used for other types of learning activities as well. The following sections present these platforms together with short descriptions of the CALSs that were created on the platforms. Finally, this chapter ends with a description of the characteristics of a reusable CALS platform. The aim is to answer the research question Q2: How can a CALS platform be constructed? 4.1

THE MYST PLATFORM

Similar to many other CALSs (see Section 3.3), the Myst platform (not to be confused with the commercial Myst game series) is based on a client-server architecture through Nokia’s Multi-User Publishing Environment (MUPE) [83]. The working principle of MUPE is presented in Figure 4.1 in which the Java-based MUPE server pushes requested content to the J2ME-based MUPE client in XML (eXtensible Markup Language) format, and the client renders the XML to display the corresponding user interface screen on the mobile device. The MUPE client has a plugin development interface which can be used to extend the client’s functionalities. The current set of plugins include for example support for GPS, NFC (Near Field Communication) and 2D bar codes. MUPE was chosen as the basis for the Myst platform because of its portability and the ease of deployment and maintenance as most operations are performed on the server side. The Myst platform o↵ers various game-like features to be used in CALSs. The most central features are enigmas which are a collection of challenges that the learner must solve. Enigmas are sensitive to the context in which the learner is, thus the learner must pay attention to the surrounding context in order to solve them. Enigmas come in many flavours, ranging from text-based queries to take-a-picture


29


Figure 4.1: Working principle of MUPE

tasks in which the user must locate an object based on a description and read a smart tag attached to it with a mobile device (i.e. treasure hunt). In addition to CALSs based on query-driven activities, the Myst platform also supports CALSs with a story-based structure, thus all Myst-based CALSs can be divided into story-telling games and treasure hunt/adventure games. Another feature provided by the Myst platform is evidence recording in which the player takes pictures of some details of the physical context and attaches comments on the pictures. These recordings, appended with meta information such as player’s nickname, location and time stamp, are stored on the server and can be presented on the game website. The website is a feature of the Myst platform which must be customised for each game instance. Dynamic content on the website can include basic information of the game, instructions, game results (points), and galleries of recordings and collaborative battle (in which all players’ points are accumulated and compared against those of a common enemy). The Myst platform has been used in eight game-based CALSs in various contexts. Table 4.1 shows these CALSs with descriptions and basic elements (adapted from paper III which describes the Myst platform in detail). The three SciMyst games only di↵er by content, hence they are grouped under one entry in the table. As shown in Figure 4.2, the high-level architecture of the Myst platform can be divided into four distinct interconnected parts: the server, the clients, the physical environment, and the o↵-site exten-

30


Platforms for CALS

Table 4.1: CALSs based on the Myst platform

Game

Purpose and location

Elements

SciMyst (3 games)

Scientific treasure hunt for exploration of the SciFest science festival in Joensuu, Finland Promotional treasure hunt for ADE Oy’s products and services in Helsinki, Finland Treasure hunt on simple machines for the Museum of Technology in Helsinki, Finland

Multiple choice questions, find-an-object tasks, record impressions, collaborative battle, website Multiple choice questions, find-an-object tasks, record impressions

AdeMyst

TekMyst

LieksaMyst

Storytelling time-travel game for learning how people used to live in the past at the open air Pielinen Museum, Lieksa, Finland

EdTechMyst Treasure hunt to demonstrate the Educational Technology research group at the University of Eastern Finland, Finland UFractions Storytelling adventure game for learning fractions with two leopards who are interacting with the learner. Developed for South African context but also deployed in Finland and in Mozambique.

Multiple choice questions, multiple skill levels, find-anobject tasks, record impressions, collaborative battle, website Interaction with past characters through a story, multiple choice questions, findan-object tasks, alternative story branches, guest book, website Multiple choice questions, find-an-object tasks, record impressions, collaborative battle, web site Interaction with leopards through a story, usage of fraction rods, multiple choice questions, openended questions, record evidence of fractions, collaborative battle, alternative story branches, guest book, website


31


sion. These components are described in detail in paper III. Contextawareness in Myst-based CALSs is primarily established through objects tagged either with two dimensional bar codes or short alphanumeric codes. The use of NFC tags is also supported but none of the Myst-based games uses them due to lack of support in client devices at the time of creating the games. Through the smart tags the server is aware of the user’s presence next to the objects and can also establish a rough estimate of the immediate social contexts of the users.

Figure 4.2: The Myst platform architecture

32


Platforms for CALS

4.1.1

SciMyst 2007-2009

SciMyst is a CALS which takes the form of a treasure hunt game played in 2007-2009 at the annual SciFest science festival in Joensuu, Finland. Players of SciMyst used mobile phones to explore the festival arena and to solve enigmas related to the exhibitions and workshops. Each version of SciMyst has a special theme, and before the game starts the player is familiarised with the theme. The player can then choose to play alone or team up with friends or family members for collaborative exploration. All three versions of SciMyst use multiple-choice challenges and take-a-picture tasks in which the player must locate a specific object based on a given description and take a picture of a 2D bar code tag attached to it (see Figure 4.3). The player’s location is established by 2D bar codes as well. All correctly solved enigmas yield points for the player. At the end of the game the player has to complete the last challenge where the acquired knowledge is tested by repeating some of the game’s enigmas in a limited time. The game area is divided into coloured sections and the player is provided with a map of the area, including markers on the locations of workshops and area tags. If the player needs help with solving an enigma, they can use context help to receive a hint, contact other players through the multiplayer help feature of the game, or interact directly with the festival exhibitors. SciMyst also utilises the impression recording feature of the Myst platform to allow the player to capture memories of the festival. The game has also a website which presents all player-generated content. The concept and details of SciMyst are described in [38]. 4.1.2

ADEMyst

ADEMyst is a treasure hunt game for promoting products and services of ADE Oy, a Finnish company specialising in 3D animations and visual design. After having seen SciMyst in action, the company representatives wanted to build a game on the Myst platform for promoting their business in a public relations event. A new game was build rapidly and showcased successfully during a one day event


33


Figure 4.3: SciMyst (2007) player recording a 2D bar code

in Helsinki in 2007. The game’s features were identical to those of SciMyst 2007 except the website which was omitted. This positive experience was the first indication that the platform could work in di↵erent contexts and for di↵erent purposes. 4.1.3

TekMyst

TekMyst is a game recontextualised from SciMyst for the Museum of Technology in Helsinki, Finland. One of the main motivations to create TekMyst was to test whether the SciMyst concept and technology could easily be ported to a di↵erent context, a space filled with machines and technological innovations. TekMyst is based on the SciMyst code but some game rules were changed, a mechanism for multiple game levels was added, and the user interface was customised. The TekMyst theme featured a magical kingdom of knowledge-sharing ants and their battle against ignorance and laziness which threatened the kingdom. The game was deployed and tested with several school groups during one week in August 2008.

34


Platforms for CALS

More information about TekMyst, including its design process, is available in [39]. 4.1.4

EdTechMyst

EdTechMyst is an application of the SciMyst concept to the Educational Technology research group’s (EdTech) laboratory at the University of Joensuu (now the Joensuu Campus of the University of Eastern Finland). The objective of the game is to familiarise a visitor with the EdTech group’s research activities and demonstrate the capabilities of the Myst platform at the same time. As with ADEMyst, EdTechMyst was created within a very short period of time and it has been demonstrated to various visitors to the research group since its emergence in 2008. 4.1.5

LieksaMyst

LieksaMyst is a CALS created for the open air Pielinen Museum in Lieksa, Finland. The concept of LieksaMyst di↵ers from the aforementioned games because it is a suite of applications including a story-based game, a database discovery tool, an NFC-based knowledge retrieval tool (prototype) and a story editor. The story-based game is the most complex feature of the system and its concept also di↵ers from earlier treasure hunt games. Whereas the aforementioned games are based on competitive quizzes, LieksaMyst o↵ers a relaxed (no time limits, no competition) way to make a journey back in time to visit fictitious characters from the past who live in the museum buildings (see Figure 4.4). The characters tell the player how life is like in their respective periods of time, and ask for assistance in performing various daily activities such as weaving carpets or churning butter. Relevant sound e↵ects are used to create an authentic atmosphere. Mobile phones are used for interacting with the character who ask the player various questions and ask them to locate specific museum objects. By embedding these objects into the story, the game teaches the player the usage of and the connections between the objects. The technology used in LieksaMyst is based on


35


the technology of previous games but modifications were needed in order to accommodate the story-based game structure and changes in the rules. However, these modifications were made while retaining the platform’s flexibility for the future game releases.

Figure 4.4: Excerpts from the story of Jussi the forest worker

Currently, LieksaMyst has two stories in two locations: a story of Anna, a warm-hearted 40 year-old lady of the Virsuvaara house (the largest building in the museum) in 1895, and Jussi, a 30 yearold unmarried forest worker who lives in a forest camp in the 1930s and has manners comparable to lumberjacks of that time. More information about the LieksaMyst CALS and its design process is presented in [40]. 4.1.6

UFractions

The last Myst-based game is UFractions (Ubiquitous Fractions) which was developed for students in South African rural middle schools. The development process of UFractions was financially and temporally constrained, and a decision was made to base it on the story-

36


Platforms for CALS

telling concept of LieksaMyst and mix it with competitive features of SciMyst. The game features a story of two leopards on a mobile phone and a set of colourful fraction rods (Cuisenaire rods) which are used to solve the challenges presented on the phone (see Figure 4.5). In the story the player’s task is to help a mother leopard and her cub through mathematical problem solving. For each correctly solved fraction challenge the player is rewarded points. The game has an introduction part, followed by three levels of varying difficulty of which the player can choose one or play all of them. In addition to the story, the game has a feature which allows the player to use the phone’s camera to record evidence of fractions from the real world and share this evidence with a comment on the game’s website. The game’s website also contains statistics related to players’ performance individually and collaboratively, and guest book entries that the players can submit at the end of the game play from the phones. The design process and the features of UFractions are described in detail in [87].

Figure 4.5: An example of a mathematical challenge in UFractions

UFractions was tested in South Africa (2009), in Finland (2010) and in Mozambique (2011). Consequently, the content has been made available in English, in Finnish and in Portuguese. A development


37


activity has also started towards the creation of intelligent fraction rods which are monitored in real time by the game so as to analyse the player’s actions and act upon them (e.g. instant feedback) [80]. This technology, however, has not yet reached its maturity and thus has not been evaluated with learners.

4.2

THE HOK PLATFORM

When the development of the Heroes of Koskenniska CALS commenced (see Section 4.2.1), the game concept was envisioned to be di↵erent to the Myst-based games in terms of context-awareness and content structure. In the Myst-based games, context-awareness is mostly based on detecting the locations of the users and the objects, and content is arranged according to pre-defined screen types within enigmas. In addition to detecting locations, Heroes of Koskenniska required deeper contextual information from the surrounding environment. Additionally, a more flexible content structure was required which would enable the designer to easily utilise existing screen types or create new ones. Figure 4.6 presents the architecture of the HoK platform which is also based on the MUPE software. The content structure follows the Model-View-Controller architecture in which the data model (learning content) is separated from the view (representation of the content) and control (user input mechanisms). This way the same data can have alternative representations which, in turn, can have alternative input mechanisms. The HoK platform has also the ESN (Environmental Sensor Network) Manager (middleware) for handling incoming data from sensors deployed in the environment. The support for environmental sensor data adds another layer of context-awareness to the platform. More details about the HoK platform is available in paper IV which discusses the creation of the platform and the Heroes of Koskenniska game.

38


Platforms for CALS

Figure 4.6: The HoK platform architecture

4.2.1

Heroes of Koskenniska

Heroes of Koskenniska is a CALS combining mobile and sensor technologies in a natural context to provide the means to raise environmental awareness among visitors of the Koskenniska Mill and Inn Museum area in North Karelian Biosphere Region1 in Eastern Finland. The area consists of four museum buildings, a sauna, a forest, a river and a lake. Readings from an Environmental Sensor Network [59] provide background data for the game where the player traverses the forest and the museum area while performing various learning activities. Figure 4.7 shows a team of two players in the forest area of Koskenniska. The player’s location is established by using riddles which are connected to specific locations and which must be solved before the game can be continued. The story of the game has references to the Finnish epic story Kalevala and it interweaves concepts such as the beginning of life, the afterlife, the meaning of time, 1

Part of UNESCO’s Man and the Biosphere Programme


39


energy and animals. The game has three levels ordered by increasing difficulty. Each level has three Magic Spots specific to physical locations and themes. The player can freely choose the traversal order of the Magic Spots within a level. Each Magic Spot has a number of challenges which can be text-based multiple choice questions (with one or more correct answers), image-based tasks where the player must pick a correct image from several possibilities, or special spot activities in which the player must perform hands-on activities such as building a bark boat and taking a picture of it. Each challenge can have any number of screens which introduce the player to the challenge before the actual challenge screen. A more detailed account on the Heroes of Koskenniska game is available in paper IV and [50] describes the environmental sensor network implementation.

Figure 4.7: Two players solving a challenge in Heroes of Koskenniska

4.2.2

TekGuide and TekGame

TekGuide and TekGame are two applications that together form a CALS based in the Museum of Technology in Helsinki. The HoK

40


Platforms for CALS

platform was chosen for creating these applications due to the higher degree of flexibility of content structure than that enabled by the Myst platform. The ESN Manager of the HoK platform is not used, but the platform was modified to introduce some new challenge types and rules that were not present in Heroes of Koskenniska. Both TekGuide and TekGame are operated with a mobile device which the visitor carries in the exhibition hall. TekGuide takes the visitor to a tour through the four thematic areas of the museum by using text, images and sound e↵ects as well as audio narration. The visitor may choose their pace and go back and forth between the screens of information. Long texts can be scrolled and images zoomed to fill the entire screen. Sounds and narrations are played automatically, and the visitor can stop or repeat the audio at will.

TekGame is a more complex and interactive application than TekGuide. It is based on various objects embedded in the four thematic areas of the museum but the traversal order of these objects can be decided by the player. Thus, there is no forced chronological structure in TekGame as content created for each object forms an independent entity. For the time being there are 13 objects included in the game and for each object there are two challenges: one for knowledge and one for creativity. Compared to Heroes of Koskenniska, TekGame has a few new challenge types such as an ordering challenge in which the player sorts a list of items to some order. Additionally, there are Super Challenges with a higher degree of difficulty that are shown after every three passed objects. The selection of objects is done by recording 2D bar codes with the mobile phone’s camera. As the platform is the same for TekGuide and TekGame, the latter also utilises text, images and audio content. However, interaction is of higher degree in TekGame due to increased possibilities for user input. Unlike other CALSs presented in this chapter, TekGuide and TekGame have not yet been evaluated nor described in previous publications.


41


4.3

CHARACTERISTICS OF A REUSABLE CALS PLATFORM

Because CALSs can be created for many di↵erent contexts and purposes, there is a need for a flexible and reusable system architecture. Such an architecture can be measured through the aspects of viability and portability which are defined as follows (paper IV). • Viability: the extent to which a given CALS can be adapted to the requirements of new stakeholders or a subject matter • Portability: the extent to which a given CALS can be transferred to a new physical context without adjusting the technical implementation If the architecture of a CALS allows flexible creation of new types of applications in the same physical context with minimal development e↵orts then the viability is high, whereas if a CALS is suited only for a single purpose the viability is low. A CALS has low portability if deployment to another location requires significant changes to the underlying system (e.g. rules, concepts, equipment). High portability allows a CALS to be transferred between physical contexts with minimal changes to the original implementation. Portability is high in both platforms as neither of them have components fixed or dependent on the physical context. Viability of the Myst platform is not at a high level because of its limitations in the content structure representation and lack of a sensor data middleware component. In contrast, the HoK platform’s viability is higher as it remedies these shortcomings of the Myst platform. Furthermore, the successful creation process of the TekMyst 2 game has informed us that the HoK platform can be successfully applied to di↵erent purposes and contexts (i.e. from environmental education in a forest to history education in a museum).

42


5 Technology integration in CALS As has been previously defined in Section 3.4, technology integration in a formal pedagogical context refers to the process by which a technology is introduced into a classroom to facilitate teaching and learning. In context-aware learning, technology integration can be seen as a challenge for CALS design, implementation and deployment. This is because CALS creators may not have the needed technical and conceptual knowledge and skills to choose and integrate technologies into the learning space. Without this know-how technology integration in a CALS will result in disturbed learning experiences. For example, a badly integrated technology may cause the learners to concentrate on playing with the properties of the technology instead of performing learning activities. To my knowledge technology integration in CALSs has not yet been researched. Furthermore, research on technology integration in informal learning contexts has been overshadowed by the research on technology integration in formal classroom settings. The importance of proper technology integration is particularly high in CALSs where the technology plays a big role in contextualising the learning content. The iterative process of technology integration in a CALS is illustrated in Figure 5.1 (paper VII). The idea is that the first version of a CALS is placed under evaluation of technology integration. The results of the evaluation are utilised in the revaluation process which diminishes the problems discovered in the evaluation, thus increasing the pedagogical and motivational value of the CALS. Devaluation may take place when when a technology breaks or becomes obsolete. In this case revamping the CALS with a new technology is needed. After revamping a new evaluation should be performed to ensure successful integration of the new technology. Waycott’s Tool Integration Process (TIP) model [90] was previously illustrated in Figure 3.5. We can compare the TIP model to the technology integration process shown in Figure 5.1. Waycott’s


43


Figure 5.1: Technology integration (TI) process in a CALS

model is user- and task-centric, and it aims at identifying new ways of using a mobile technology, such that were unforeseen by the designers. This information can be used to o↵er new opportunities for further technology integration and development. In contrast, the CALS technology integration process covers the entire cycle of technology integration from emergence to evaluation and from improvement to degeneration. The TIP model could be used to complement the CALS technology integration process by identifying new ways of using the CALS. However, since the TIP model was developed for the purpose of mobile learning, it may need to be adjusted to match the setting of context-aware learning. Such development is out of the scope of this research. To tackle the challenge of technology integration in CALSs, the following sections present a model to describe requirements of technology integration and a tool for evaluating technology integration in CALSs. 5.1

TECHNOLOGY INTEGRATION MODEL

The proposed technology integration model, presented in paper V and illustrated in Figure 5.2, concerns various requirements which

44


Technology integration in CALS

should be met by the integrated technology. Each main requirements category (tips of the triangle) is further divided into more specific requirement categories. The context requirements cover the requirements and restrictions set by the context’s available resources, culture, technology, environment, social aspects and scheduling. The pedagogical requirements in this case are formed by a set of characteristics for pervasive learning spaces (a subset of CALS, see Section 3.3). These requirements cover categories of user profiles and perspectives, interaction and collaboration, ownership, authenticity and relevance, and support and assessment (paper II). Finally, the design requirements include subcategories of context-awareness, game dynamics, interaction, and content design. Each requirement category has a critical factor and they are: unobtrusive technology for the pedagogical requirements; available resources for the context requirements; and context-awareness for the design requirements.

Figure 5.2: Technology integration model for CALSs (critical factors marked with asterisks)

Paper VI continued the work on the technology integration model by dividing technology integration into passive and active components. This division was done according to the roles of technology in the integration process as follows.


45


1. Passive integration: technology must be integrated into the CALS so that it becomes subtle and unobtrusive to the learner and to the context. In other words, technology is the object of integration. 2. Active integration: technology must integrate the contextual resources and context-free resources into the CALS and make the system adaptive to the changing situations of the context, including users within. In other words, technology is the subject of integration. This division is necessary in order to manage the technology’s direct and indirect influence on the learners. Both active and passive integration are driven (or restrained) by available resources (Figure 5.3). Passive integration aims at achieving unobtrusiveness of the technology from the learners’ and the context’s perspectives so that the learning process and the context will not be disturbed by the technology. The integrated unobtrusive technology is used to provide context-awareness to the active integration process via contextual resource detection. The goal of active integration is to establish context adaptation [51] within the system. This means that when the situation in the target context changes, the technology automatically adapts the contextual and context-free resources to the new situation. In addition to passive and active integration, the requirements defined by the model and the critical factors can be used by a CALS designer to prepare and assess a plan for integrating technologies into a CALS. Once design and implementation have been completed, it is necessary to evaluate the outcome of the technology integration process. To verify the need for technology integration evaluation, the aspects of active and passive integration were applied to the evaluation of the UFractions game (see Section 4.1.6) in South Africa and in Finland. The tests were conducted with 105 and 104 eighth grade pupils in South Africa and in Finland respectively. The data set was designed for investigating the reverse transfer process of UFractions [51] where a technology, which was designed and developed in

46



Figure 5.3: Passive and active integration of technology in CALSs

and for a technology-alien context (South Africa), was transferred to a technology-familiar context (Finland). Relevant metrics of the data set were used so as to evaluate the direct and indirect influence of the technology on the pupils. The results, presented in paper VI, suggest that particularly active integration failed in the Finnish context as the technology indirectly influenced the pupils by not providing contextualisation of the game content. Passive integration was fairly successful as there were no technical issues and most pupils received the technology well, but there were some individual pupils whose learning processes were disturbed by the technology. During the evaluation it was observed that the lack of active or passive integration may cause various disturbances to the learning process. These disturbances do not necessarily a↵ect the majority of the test participants, hence a qualitative approach was required. By analysing the qualitative data from questionnaires, interviews and observations, sixteen disturbance factors were discovered which relate either to active (9) or to passive (7) integration of technology. Ta-


47


ble C.1 describes the disturbance factors with indications that map the relevant evidence to the factors. Column I indicates whether the factor relates to active (A) or passive (P) integration. Identified disturbance factors are grouped by the learner’s areas of experience which are a↵ected by the disturbance factors. The disturbance factors “Below ZPD” and “Beyond ZPD” refer to Vygotsky’s Zone of Proximal Development [88]. 5.2

TECHNOLOGY INTEGRATION EVALUATION TOOL

The technology integration evaluation shown in the previous section yielded interesting results. Particularly the identified disturbance factors are useful for guiding the improvement process of UFractions. However, as described above, the data set used was not designed for measuring technology integration in CALSs, hence calling for a dedicated tool for a deeper evaluation. For this purpose a technology integration evaluation tool was created (paper VII) by grounding it on the technology integration model and on the TPCK framework for classroom-based technology integration [48] (see Section 3.4). The evaluation tool uses the viewpoints of the learner, the educator and the context to measure the critical factors (unobtrusiveness of technology, availability of resources and context-awareness) as well as affordances and constraints (from the TPCK framework) in the target CALS. These aspects formed the basis of the evaluative questions for the viewpoints of the learner, (Table B.1), the educator (Table B.2) and the context (Table B.3). These questions are to be used as a starting point for creating data collection instruments. For example, the question “How do the learners perceive the technology?” can be answered by asking the learners’ opinions on and experiences with the technology (e.g. mobile devices) as a part of the CALS. The evaluation tool also measures general perceptions of the CALS. These data include likes, dislikes, suggestions for improvements, motivation and applicability to other contexts. These aspects can be used to evaluate the attractiveness of the CALS as a learning tool both from the learner’s and the educator’s perspectives.

48



After creating the evaluation tool, an evaluation of UFractions was performed in the Mozambican context, and was targeted at 70 pupils at two schools in Maputo. Only the learner’s viewpoint was considered in this evaluation which is presented in detail in paper VII. Applied evaluation instruments are presented in Appendix D. As a result, 22 disturbance factors were identified which are described in Table C.2. The disturbance factors were derived using the same method as the evaluation described in Section 5.1. All but one of the previously identified disturbance factors (16) can be found within the 22 disturbance factors discovered with the evaluation tool. Furthermore, the size of the data sets used in this evaluation (70) was significantly smaller than the combined data set used in the previous study in South Africa and Finland (209). The participants in Mozambique were of a wider age range (10-32, average 13) than in South Africa and Finland where UFractions was tested on eighth graders. Additionally, in Mozambique the participants were of 23 di↵erent nationalities, thus making the data heterogeneous. There was also a high number (75) of significant correlations (equal to or above 0.5) between quantitative statements of the questionnaire in Mozambique. This informs us of the good quality (triangulation) and the depth of the data. In contrast, in the South African data set the number of significant correlations was 8 and in the Finnish data set it was 29. These observations on the results indicate that the evaluation tool outperformed the previous evaluation as it produced more accurate results with a smaller data set. This suggests that the evaluation tool is likely to speed up the technology integration process of a CALS through a rapid evaluation procedure.


49


50


6 Discussion By identifying the concept and the characteristics of CALSs, I have established the foundations on which researchers and educators can base their future e↵orts on learning environment developments towards richer and contextually relevant learning experiences. The definitions used in this dissertation aim at creating a common vocabulary for future work on CALSs. For example, previously the terms pervasive learning and ubiquitous learning have been used in some cases as synonyms but now they have well-defined meanings that di↵er from each other. Furthermore, my definition of contextawareness is generic enough to be used for other purposes apart from CALSs. The iterative development processes of the Myst and the HoK platforms show the elements that a viable and portable platform for a CALS should contain. Successful design and implementation of ten game-based CALSs on the two platforms indicate that the proposed platforms can be used for many di↵erent contexts and purposes. A question remains whether the platforms would be suitable for a CALS without game-like features. To my understanding they would, as both platforms support flexible use of text, graphics and sounds to present the learning content. For example, instead of a quiz-like structure of SciMyst at the SciFest festival, the Myst platform could be used to create an interactive tour guide for the festival. However, I argue, with support from Malone [58], that it is through games that the learners, particularly children and young adults, can better immerse themselves in the flow [14] through increased intrinsic motivation. Hence, although the Myst platform and the HoK platform can be used for CALSs without game-like features, the gaming approach is recommended especially for young learners. Technology integration has received much attention in the context of traditional classroom-based learning but in the domain of informal learning, particularly in context-aware learning, the issue has not received similar attention. While a CALS at its best can provide


51


highly interactive and engaging learning experiences, the technical complexity might be high, thus leading to issues of badly integrated technology. The technology integration model for CALS was created to assist CALS designers to choose and apply technologies based on various requirements set by the context, the pedagogy and the design. Technology integration was further divided into active and passive integration. Both integration types are important to consider, as was suggested by the evaluations of the UFractions CALS in three di↵erent contexts (papers VI and VII). Therefore, an important implication is that in order for a CALS to be pedagogically and motivationally e↵ective, its technology must be unobtrusive and subtle to the learner while adapting contextual resources to match the learner’s profile and the context’s requirements. The concepts of active and passive integration are generic enough to be applicable to other contexts as well. In a formal classroom-based pedagogical setting technology integration is usually performed in a passive manner, i.e. technology is introduced to the context with appropriate training for teachers and students. However, active integration could also be applied so as to make the technology in the classroom more responsive to the learners’ and the teacher’s preferences and background knowledge. For example, a new technology could provide a novice teacher with an extensive usage tutorial whereas a more technology-savvy teacher would be given access to advanced features of the technology. In a similar fashion, the active integration process could ensure that the students would receive learning materials in preferred formats and compatible with the students’ previous knowledge. This could be done for example by existing technologies used in intelligent tutoring systems [68] and content adaptation systems [52]. Based on the model of technology integration in CALSs, a tool was proposed for evaluating technology integration. Both the model and the evaluation tool are novel approaches to scrutinise technology integration in CALSs from an holistic perspective. The proposed evaluation tool connects to the TCPK model by Koehler and Mishra [48], thus building on firm theoretical foundations of technol-

52


Discussion

ogy integration in classroom contexts. The evaluation tool was used to assess the UFractions game in the Mozambican context in order to measure the tool’s suitability for technology integration evaluation in CALSs. The evaluation conducted with the tool yielded deeper results with a significantly smaller data set than a previous evaluation without the tool (paper VI), hence indicating the efficiency of the tool. The proposed technology integration evaluation tool is not perfect. It performed well with UFractions but its limits and opportunities with other CALSs are yet to be explored. Furthermore, the evaluation presented in paper VII did not consider the viewpoints of the educator and the context. Although the learner’s viewpoint is the most critical, it is necessary to consider these other viewpoints as well if comprehension of the big picture of technology integration in a CALS is desired. The current version of the tool aims at delivering an overall view of technology integration but in the future there may be a need for specialised evaluations. For this purpose, I suggest that a future version of the tool should have a suite of test instruments which are organised by areas of experiences (e.g. user experience, learning experience, social experience). The evaluations of technology integration on the UFractions game (both with and without the evaluation tool) revealed disturbance factors which guide the improvement process of a CALS. The identified disturbance factors may indicate pitfalls in the design and implementation of future CALSs. This information is useful for CALS designers who can now plan the use of technology so that the goals of active and passive technology integration are met. Furthermore, the areas of experience, which were used to group the disturbance factors, are useful for the CALS designers for ensuring that a variety of di↵erent experiences are supported in a CALS. It is clear that there are more disturbance factors and areas of experience to be discovered by future studies. These results can be used as a starting point towards a complete taxonomy of experience areas and related disturbance factors. Additionally, generalisability of the factors and their experience areas to other learning environments apart from CALSs


53


should be investigated. The results of a technology integration evaluation can be used to inform the revaluation process in which the identified problems and disturbances are diminished (Section 5). This important step towards improving technology integration in a CALS is left for future research. Assumptions can already be made on some aspects about the revaluation process. First, a taxonomy of potential solutions will be available with information on how each solution can help to diminish the disturbances. Secondly, there will be separate revaluation threads for active and passive integration because they have inherently di↵erent goals. Improving passive integration should be completed first because it may a↵ect active integration through context-aware technologies. Thirdly, in the passive integration user experience [26] guidelines or similar tools can be used so as to decrease the obtrusiveness of the technology. Fourthly, in the case of active integration, content adaptation [52] could be a viable technology for ensuring the content’s suitability for a learner in a given situation. Common sense dictates that by diminishing disturbance factors that were identified for UFractions, the game would better facilitate learning because the learner would be less distracted. However, determining the pedagogical e↵ectiveness of a CALS goes beyond common sense and also beyond this dissertation. The results of this research can merely be used to develop CALSs and to evaluate technology integration in them. To e↵ectively evaluate the learning experience in a CALS, the 18 pedagogical characteristics for CALSs (Appendix A) could be used as the basis for evaluation instruments. Additionally, it is necessary to use appropriate pedagogical assessment techniques for a given subject matter in a long term exposure. These results must then be compared to the results of a control group in order to determine how much the learners who use the CALS actually benefit from it. Another aspect that should be a subject to evaluation in addition to acquired knowledge and skills is motivation as it contributes to the overall learning experience and attitude towards the content matter. Elimination of all problems and disturbances in a CALS is very

54


Discussion

difficult despite of the use of state-of-the-art context-aware technologies. This is because the more technology is used, the harder it becomes to manage the technical complexity and to make it unobtrusive to the learner. Furthermore, price of technology is an e↵ective limitation in many projects and even if there would be infinite financial resources, it would not be possible for a system to be aware of everything within a context. For example, detecting the most finegrained nuances of a human’s body and mind is currently not possible, and the physical environment with all its dimensions is a very complex structure to monitor in an holistic manner. The good news is that as the development of context-aware technologies advances and they become more a↵ordable, it will be possible to create highly context-aware systems in financially constrained areas. These systems will converge towards Weiser’s vision on ubiquitous computing where technologies “weave themselves into the fabric of everyday life until they are indistinguishable from it” [92] while providing access to the richness of hidden resources of the surrounding contexts.


55


56


7 Conclusion During 2007-2011, I was involved in the creation of ten context-aware learning spaces in versatile contexts and for di↵erent purposes, including but not limited to mathematics in South African and Mozambican schools, environmental awareness in a Finnish forest, history in an open air museum and a technology museum in Finland, and science at a science festival in Finland. Additionally, I spent one year in South Korea to become familiar with the state-of-the-art technologies, particularly sensors and wireless communications. The richness of contexts provided the research with opportunities that would not have been possible in a single context. My role as a technical developer granted me a unique view over the challenges and the opportunities posed by the target contexts from a technical point of view. Each iteration of the exploratory software development process provided me with new ideas to narrow the focus of this research, which eventually led to the emergence of the concept and the characteristics of CALSs, a technology integration model and an evaluation tool for technology integration. These results fulfill the overall objective of this research which was to provide the developers with the tools for construction and evaluation of CALSs in which technologies have been integrated e↵ectively so that they do not disturb the learner. Research question Q1, “what features characterise context-aware learning spaces (CALSs) within the domain of mobile-based learning tools?”, was answered by a literature analysis on existing CALSs, state-of-the-art context-aware technologies and a number of pedagogical approaches for informal settings. In the process I defined the concept of CALS together with other interrelated concepts and derived a set of characteristics which can be used in a CALS design process as a checklist to increase the pedagogical value of the CALS. A challenge with this research question is that, due to the rapid development of technology, it is very laboursome to keep up-to-date with the latest


57


technologies. Consequently, the current state-of-the-art technologies should be analysed for each CALS development project. In research question Q2, “How can a CALS platform be constructed?”, the exploratory software development method was followed by which ten CALSs and two CALS platforms were created. Most of the CALSs were also evaluated. Without this practical work the other research questions would have never been formulated the way they are, thus this is the core part of this dissertation. The process started in the beginning of this research and is still ongoing. Based on the developing experiences, two concepts emerged which can be used to analyse CALS platform architectures: viability and portability. The descriptions of the CALSs, the platforms and the concepts, are useful for developers who want to establish their own flexible and reusable CALS platforms. A challenge with this research question relates to the challenge of Q1 – the rapid development of technology makes most platforms obsolete within a few years of its launch. However, high level architectural features (e.g. modular structure, Model-View-Controller pattern) can be applied across technologies. The results of research questions Q1 and Q2 informed the research question Q3 (“How can technology integration be taken into account in the design phase of CALSs?”). As a result, I established a technology integration model which is partly based on the characteristics of CALSs and partly on an artefact analysis conducted on the CALSs which were created. The model suggests various requirements and three critical factors to be considered in order to facilitate smooth technology integration. The concept of technology integration was further divided into active and passive integration according to the role of technology in the process, and UFractions was evaluated from these perspectives. The established model is useful for CALS designers in order to ensure appropriate integration of technology. The final research question Q4 was “How can technology integration in CALSs be evaluated?”. To answer this question, I created an evaluation tool based on the technology integration model (Q3) and a literature analysis on technology integration in formal educa-

58


Conclusion

tion. The evaluation tool uses the three critical factors of the technology integration model as well as a↵ordances and constraints to measure technology integration in a CALS from the perspectives of the learner, the educator and the context. Comparison of two evaluations indicated that the evaluation tool produces more accurate results with a smaller dataset than an evaluation conducted without the tool. Thus, the evaluation tool can be useful for CALS designers and developers who want to know how well technology is integrated into their systems. There are several limitations to this research which could be addressed by future studies. First, although I established the characteristics for CALS, there is still no dedicated pedagogical model that could be used to facilitate the pedagogical appropriateness of a CALS. Future research could use the CALS characteristics as a starting point towards establishing such a model. Secondly, while both Myst and HoK platforms have certain degrees of viability and portability, their underlying technology is becoming rapidly outdated. This applies particularly to client devices: the current J2ME-based client implementation is no match to Android, MeeGo or iOS platforms in terms of features and programming capabilities. Thus, an important technical research and development project would be to add a multiclient support to the platforms or rebuild the platforms to support the latest mobile software platforms. Thirdly, the concepts of active and passive technology integration have only been discussed within the domain of CALS. An interesting future study would therefore be to find out how well these concepts could apply to other informal and formal learning environments. Fourthly, when the proposed technology integration tool was used on UFractions, only the learner’s role was considered. It is, therefore, important to apply the evaluation tool for the educator’s and the context’s roles too, as well as for other CALSs. Only this way can the generalisability of the evaluation tool be determined. Fifthly, this research proposed tools only for CALS creation and evaluation. The process of technology integration revaluation, i.e. diminishing disturbances and problems discovered in the evaluation, was only speculated upon in this dissertation, hence it


59


requires more work. Once a method or a tool for the revaluation process has been established, the overall iterative process of technology integration in CALS is completed. Finally, this research has set the foundations for the CALS implementation and technology integration processes. However, both of these processes can and should be explored further as I have merely performed an initial survey and much of this territory remains uncharted. In the future we will see context-aware technologies as a more integral part of our lives. Already now, location-based services are popular and it is only a matter of time before these services will expand towards wider utilisation of contextual resources such as sensor data. Initiatives have already been launched to gather and manage global sensor data (e.g. SensorPlanet [8], SensorBase [12]) which can be used by researchers and developers to invent new ways of utilising the data in context-aware applications. This also applies to the field of education and I envision CALSs to be in the forefront of educational technology development as the importance of informal learning will be more acknowledged. In addition to sensing the surrounding context, future CALSs will also be able to observe the bodily functions of the learner and thereby sense the optimal moment to deliver educational materials, for example. Technologies such as natural user interfaces, 3D screens, micro projectors, and even cybernetic implants can be used to take the user experience to the next level. This all raises ethical concerns that must be investigated in future research.

60


Bibliography [1] G. Abowd, C. Atkeson, J. Hong, S. Long, R. Kooper, and M. Pinkerton. Cyberguide: A mobile contextaware tour guide. Wireless Networks, 3(5):421–433, 1997. [2] E. Arendarenko and T. H. Laine. Smart tagging technologies in pervasive learning environments. In Proceedings of the IADIS E-learning 2011 Conference, pages 235–242, 2011. [3] O. Aziz, B. Lo, G.-Z. Yang, R. King, and A. Darzi. Pervasive body sensor network: An approach to monitoring the postoperative surgical patient. In Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks, pages 13–18, 2006. [4] R. Ballagas, S. Kratz, J. Borchers, E. Yu, S. Walz, C. Fuhr, L. Hovestadt, and M. Tann. REXplorer: A mobile, pervasive spell-casting game for tourists. In CHI ’07 extended abstracts on Human factors in computing systems (CHI EA ’07), pages 1929–1934, New York, NY, 2004. [5] J. E. Bardram. The Java Context Awareness Framework (JCAF) – A service infrastructure and programming framework for context-aware applications. In Proceedings of the 3rd International Conference on Pervasive Computing, pages 98–115, 2005. [6] J. S. Beaudin, S. S. Intille, E. M. Tapia, R. Rockinson, and M. E. Morris. Context-sensitive microlearning of foreign language vocabulary on a mobile device. In Proceedings of the 2007 European conference on Ambient intelligence, AmI’07, pages 55–72, Berlin, Heidelberg, 2007. Springer-Verlag. [7] H. Becker. How exemplary computer-using teachers di↵er from other teachers: Implications for realizing the potential of computers in schools. Journal of Research on Computing in Education, 26(3):291–321, 1994.


61


[8] P. P. Boda. SensorPlanet: An open global research framework for mobile centric Wireless Sensor Networks. In Proceedings of the NSF Workshop on Data Management for Mobile Sensor Networks, Pittsburgh, 2007. [9] M. Boekaerts. Self-regulated learning: Where we are today. International Journal of Educational Research, 31(6):445–457, 1999. [10] V. Caracelli and J. Greene. Crafting mixed-method evaluation design. In J. Greene and V. Caracelli, editors, Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Program Evaluation, pages 19–32. Jossey-Bass, San Francisco, CA, 1997. [11] J. Carroll, S. Howard, F. Vetere, J. Peck, and J. Murphy. Just what do the youth of today want? technology appropriation by young people. In Proceedings of the 35th Hawaii International Conference on System Sciences (HICSS-35), pages 1777–1785, Maui, HI, 2002. [12] K. Chang, N. Yau, M. Hansen, and D. Estrin. SensorBase.org – a centralized repository to slog sensor network data. In Proceedings of the Euro-American Workshop on Middleware for Sensor Networks (EAWMS - DCOSS), San Francisco, CA, 2006. [13] D. I. Cordova and M. R. Lepper. Intrinsic motivation and the process of learning: Beneficial e↵ects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4):715–730, 1996. [14] M. Csikszentmihalyi. Finding Flow: The Psychology of Engagement with Everyday Life. Basic Books, 1998. [15] M. Csikszentmihalyi and K. Hermanson. Intrinsic motivation in museums: Why does one want to learn? In E. HooperGreenhill, editor, The Educational Role of the Museum, pages 146–160. Routledge, London, UK, 1995.

62


Bibliography

[16] F. Davis. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3):319– 340, 1989. [17] J. Dewey. Experience and education. Collier Books, New York, 1938. [18] P. Dourish. What we talk about when we talk about context. Personal and Ubiquitous Computing, 8(1):19–30, 2004. [19] M. Eraut. Informal learning in the workplace. Studies in Continuing Education, 26(2):247–273, 2004. [20] P. Ertmer. Addressing first- and second-order barriers to change: strategies for technology integration. Educational Technology Research and Development, 47(4):47–61, 1999. [21] B. Eschenbrenner and F.-H. Nah. Mobile technology in education: Uses and benefits. IJMLO – International Journal of Mobile Learning and Organisation, 1(2):159–183, 2007. [22] J. Falk and L. Dierking. Learning from museums: Visitor experiences and the making of meaning. AltaMira, Walnut Creek, CA, 2000. [23] C. Goth, D. Frohberg, and G. Schwabe. The focus problem in mobile learning. In Proceedings of the Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education, pages 153–160, 2006. [24] S. Grabinger, J. Dunlap, and J. Duffield. Rich environments for active learning in action: Problem-based learning. Association for Learning Technology Journal, 5(2):5–17, 1997. [25] T. Hall and L. Bannon. Designing ubiquitous computing to enhance children’s interaction in museums. In Proceedings of the 2005 conference on Interaction design and children, pages 62–69, 2006.


63


[26] M. Hassenzahl and N. Tractinsky. User experience – a research agenda. Behaviour & Information Technology, 25(2):91– 97, 2006. [27] R. Hawkey. Science beyond school: Representation or representation? In A. Loveless and V. Ellis, editors, ICT, Pedagogy and the Curriculum: Subject to Change, pages 104–128. Routledge/Falmer, London, 2001. [28] H. Hein. The Exploratorium: The museum as laboratory. Smithsonian Institution Press, Washington, DC, 1990. [29] J. Herrington and R. Oliver. An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3):23–48, 2000. [30] J. Herrington, R. Oliver, and T. Reeves. Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology, 19(1):59–71, 2003. [31] G. Heumer, F. Gommlich, B. Jung, and A. M¨ uller. Via Mineralia – a pervasive museum exploration game. In Proceedings of the 4th International Symposium on Pervasive Gaming Applications (PerGames 2007), pages 159–160, Salzburg, Austria, 2007. [32] S. Hinske and M. Langheinrich. An infrastructure for interactive and playful learning in augmented toy environments. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications, pages 1–6, Galveston, TX, 2009. [33] E. Hooper-Greenhill, J. Dodd, T. Moussouri, C. Jones, C. Pickford, C. Herman, M. Morrison, J. Vincent, and R. Toon. Measuring the outcomes and impact of learning in museums, archives and libraries. Technical report, Research Centre for Museums and Galleries: Learning Impact Research Project, Leicester, UK, 2003. [34] B. Hou, H. Ogata, M. Miyata, L. Mengmeng, Y. Liu, and Y. Yoneo. JAMIOLAS 3.0: Supporting Japanese mimicry and

64


Bibliography

onomatopoeia learning using sensor data. IJMBL – International Journal of Mobile and Blended Learning, 2:40–54, 2010. [35] J.-H. Huang, Y.-R. Lin, and S.-T. Chuang. Elucidating user behavior of mobile learning – a perspective of the extended technology acceptance model. The Electronic Library, 25(5):585–598, 2007. [36] L. Hui-chun, K. Liu, and N. Sambasivan. GreenSweeper: A persuasive mobile game for environmental awareness. In Proceedings of the Ubicomp 2008 Workshop on Ubiquitous Sustainability: Citizen Science and Activism, Seoul, South Korea, 2008. [37] Hui Liu, H. Darabi, P. Banerjee, and Jing Liu. Survey of wireless indoor positioning techniques and systems. IEEE Transactions on Systems, MAN, and Cybernetics, Part C: Applications and Reviews, 37(6):1067–1080, 2007. [38] C. Islas Sedano, T. H. Laine, M. Vinni, and E. Sutinen. Where is the answer?: The importance of curiosity in pervasive mobile games. In Proceedings of the 2007 conference on Future Play, Future Play ’07, pages 46–53, New York, NY, USA, 2007. ACM. [39] C. Islas Sedano, J. Pawlowski, M. Sutinen, M. Vinni, and T. H. Laine. From global games to re-contextualized games: The design process of TekMyst. In M. Ma, A. Oikonomou, and L. Jain, editors, Serious Games and Edutainment Applications. SpringerVerlag, UK, Accepted in 2011. [40] C. Islas Sedano, E. Sutinen, M. Vinni, and T. H. Laine. Designing hypercontextualized games: A case study with LieksaMyst. Accepted to Educational Technology & Society Journal, 2011. [41] S. Järvelä, E. Lehtinen, and P. Salonen. Socioemotional orientation as a mediating variable in teaching learning interaction: Implications for instructional design. Scandinavian Journal of Educational Research, 44(3):296–306, 2000.


65


[42] C. Johnson and M. Quin. Learning in science and discovery centres – appendix. In I. Chen and T. Kidd, editors, Science Center Impact Study. ASTC, Washington, DC, 2004. [43] A. Jones and K. Issro↵. Motivation and mobile devices: Exploring the role of appropriation and coping strategies. ALT-J: Research in Learning Technology, 15(3):247–258, 2007. [44] E. Kaasinen. User needs for location-aware mobile services. Personal and Ubiquitous Computing, 7(1):70–79, 2002. [45] T. M. Keating and E. Evans. Three computers in the back of the classroom: Pre-service teachers’ conceptions of technology integration. In Proceedings of Society for Information Technology & Teacher Education International Conference, pages 1671–1676, Chesapeake, VA, 2001. [46] L. Kelly. What is learning...and why do museums need to do something about it? In Paper presented at Why Learning? Seminar at Australian Museum/University of Technology, 2002. [47] E. Klopfer and K. Squire. Environmental Detectives – the development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56:203–228, 2007. [48] M. Koehler and P. Mishra. Introducing Technological Pedagogical Content Knowledge. The Handbook of Technological Pedagogical Content Knowledge for Educators. Routledge, 2008. [49] A. Kolb and D. Kolb. Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning & Education, 4(2):193–212, 2005. [50] T. H. Laine, A. Gimbitskaya, E. Sutinen, J. Choi, K. Yong, and C. Lee. Environmental sensor network for a pervasive learning space in a Finnish Biosphere Reserve. In Proceedings of the 5th

66


Bibliography

International Conference on Ubiquitous Information Management and Communication, ICUIMC ’11, pages 88:1–88:6, New York, NY, USA, 2011. ACM. [51] T. H. Laine, E. Nygren, E. Sutinen, C. Islas Sedano, M. Joy, and S. Blignaut. Ubiquitous mathematics from South Africa to Finland: Does reverse transfer work? In I. Chen and T. Kidd, editors, Ubiquitous Learning: Strategies for Pedagogy, Course Design and Technology, pages 249–282. Information Age Publishing, Charlotte, NC., 2011. [52] T. Lemlouma and N. Layaida. Context-aware adaptation for mobile devices. In Proceedings of the IEEE International Conference on Mobile Data Management, pages 106–111, 2004. [53] T. Levine and R. Wadmany. Teachers’ views on factors a↵ecting e↵ective integration of information technology in the classroom: Developmental scenery. Journal of Technology and Teacher Education, 16(2):233–263, 2008. [54] D. Livingstone. Exploring the icebergs of adult learning: Findings of the first canadian survey of informal learning practices. Canadian Journal for the Study of Adult Education, 13(2):49–72, 1999. [55] P. Lonsdale, C. Baber, and M. Sharples. A context awareness architecture for facilitating mobile learning. In J. Attewell and C. Savill-Smith, editors, Learning with Mobile Devices: Research and Development, pages 79–85. Learning and Skills Development Agency, London, 2004. [56] P. Lonsdale, R. Beale, and W. Byrne. Using context awareness to enhance visitor engagement in a gallery space. In Proceedings of the HCI05 Conference on People and Computers XIX, pages 101–112, Galveston, TX, 2005. [57] K. Lyytinen and Y. Yoo. Introduction. Communications of ACM, 45(12):62–65, 2002.


67


[58] T. Malone. Toward a theory of intrinsically motivating instruction. Cognitive Science, 5(4):333–369, 1981. [59] K. Martinez, J. Hart, and R. Ong. Environmental sensor networks. Computer, 37(8):50–56, 2004. [60] B. Means, J. Blando, K. Olson, T. Middleton, C. Morocco, A. R. Remz, and J. Zorfass. Using technology to support education reform. Technical report, U.S. Government Printing Office, Washington, DC, 1993. [61] C. Mims. Authentic learning: A practical introduction & guide for implementation. Meridian: A Middle School Computer Technologies Journal, 6(1), 2003. [62] L. Naismith, P. Lonsdale, G. Vavoula, and M. Sharples. Literature review in mobile technologies and learning. Technical report, NESTA Futurelab Series, 2005. [63] M. Niess. Preparing teachers to teach science and mathematics with technology: A focus on pedagogical content knowledge. Teaching and Teacher Education, 21(5):509–523, 2005. [64] M. Notess. Applying contextual design to educational software development. In A. Armstrong, editor, Instructional design in the real world: A view from the trenches, pages 19–32. Idea Group Publishers, Hershey, PA, 2004. [65] H. Ogata, Y. Matsuka, M. El-bishouty, and Y. Yano. LORAMS: Linking physical objects and videos for capturing and sharing learning experiences towards ubiquitous learning. International Journal of Mobile Learning and Organisation, 3(4):337– 350, 2009. [66] H. Ogata and Y. Yano. Context-aware support for computersupported ubiquitous learning. In Proceedings of the 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education, pages 27–34, 2004.

68


Bibliography

[67] M. Pierson. Technology practice as a function of pedagogical expertise. Journal of Research on Computing on Education, 33(4):413–430, 2001. [68] M. C. Polson and J. J. Richardson. Intelligent tutoring systems. Lawrence Erlbaum Associates, Hillsdale, NJ, 1988. [69] D. Ramduny-Ellis, A. Dix, P. Rayson, V. Onditi, I. Sommerville, and J. Ransom. Artefacts as designed, Artefacts as used: Resources for uncovering activity dynamics. Cognition, Technology and Work, 7(2):76–87, 2005. [70] L. Rennie and T. McCla↵erty. Using visits to interactive science and technology centers, museums, aquaria, and zoos to promote science learning. Journal of Science Teacher Education, 4(4):175–185, 1995. [71] Y. Rogers, S. Price, G. Fitzpatrick, R. Fleck, E. Harris, H. Smith, C. Randell, H. Muller, C. O’Malley, D. Stanton, M. Thompson, and M. Weal. Ambient Wood: Designing new forms of digital augmentation for learning outdoors. In Proceedings of the Third International Conference for Interaction Design and Children (IDC 2004), pages 3–10, College Park, MD, USA, 2004. ACM Press. [72] J. Savery and T. Du↵y. Problem based learning: An instructional model and its constructivist framework. In B. Wilson, editor, Constructivist learning environments: Case studies in instructional design, pages 135–148. Educational Technology Publications, Englewood Cli↵s, NJ, 1994. [73] E. Scanlon, A. Jones, and J. Waycott. Mobile technologies: Prospects for their use in learning in informal science settings. Journal of Interactive Media in Education, 25:1–17, 2003. [74] H. Schmidt. Problem-based learning: Rationale and description. Medical Education, 17(1):11–16, 1983.


69


[75] M. Sharples. The design of personal mobile technologies for lifelong learning. Computers & Education, 34(3-4):177–193, 2000. [76] M. Sharples. Disruptive devices: Mobile technology for conversational learning. International Journal of Continuing Engineering Education and Lifelong Learning, 12(5-6):504–520, 2003. [77] M. Sharples, M. Milrad, I. Arnedillo Snchez, and G. Vavoula. Mobile learning: Small devices, big issues. In N. Balache↵, S. Ludvigsen, T. de Jong, A. Lazonder, and S. Barnes, editors, Technology Enhanced Learning: Principles and Products, pages 233–249. Springer, Heidelberg, 2009. [78] B. Sheppard. Museums, libraries and the 21st century learner. Technical report, The Institute of Museum and Library Services, Washington, DC, 2001. [79] L. S. Shulman. Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1):1–12, 1987. [80] A. C. Smith and T. H. Laine. An intelligent fractions learning system: Implementation. In Proceedings of the IST Africa 2011 Conference, Gaborone, Botswana, 2011. [81] A. Staples, M. Pugach, and D. Himes. Rethinking the technology integration challenge: Cases from three urban elementary schools. Journal of Research on Technology in Education, 37(3):285–311, 2005. [82] N. Strudler and K. Wetzel. Lessons from exemplary colleges of education: Factors a↵ecting technology integration in preservice programs. Educational Technology Research and Development, 47(4):63–81, 1999. [83] R. Suomela, E. Räsänen, A. Koivisto, and J. Mattila. Opensource game development with the Multi-user Publishing Environment (MUPE) application platform. In M. Rauterberg, editor, Entertainment Computing – ICEC 2004, volume 3166

70


Bibliography

of Lecture Notes in Computer Science, pages 308–320. Springer Berlin, Heidelberg, 2004. [84] S. Thomas. Pervasive, persuasive elearning: Modeling the pervasive learning space. In Proceedings of the Third IEEE International Conference on Pervasive Computing and Communications Workshops, PERCOMW ’05, pages 332–336, Washington, DC, USA, 2005. IEEE Computer Society. [85] A. Tough. Major learning e↵orts: Recent research and future directions. Adult Education Quarterly, 28(4):250–263, 1978. [86] J. Trenouth. A survey of exploratory software development. The Computer Journal, 34(2):153–163, April 1991. [87] E. Turtiainen, S. Blignaut, C. Els, T. H. Laine, and E. Sutinen. Story-based UFractions mobile game in South Africa: Contextualization process and multidimensional playing experiences. In Proceedings of the Second Workshop of Story Telling and Educational Games, 2009. [88] L. Vygotsky. Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, 1978. [89] K. E. Watkins and V. J. Marsick. Towards a theory of informal and incidental learning in organizations. International Journal of Lifelong Education, 11(4):287–300, 1992. [90] J. Waycott. The Appropriation of PDAs as Learning and Workplace Tools: An Activity Theory Perspective. PhD thesis, The Institute of Educational Technology, The Open University, Milton Keynes, UK, 2004. [91] M. Weiser. Some computer science issues in ubiquitous computing. Commun. ACM, 36(7):75–84, July 1993. [92] M. Weiser. The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev., 3(3):3–11, July 1999.


71


[93] A. Zimmermann, A. Lorenz, and R. Oppermann. An operational definition of context. In Proceedings of the Sixth International and Interdisciplinary Conference on Modeling and Using Context, pages 558–571, 2007.

72


A Characteristics of CALSs Table A.1 describes the characteristics of CALSs that were identified based on a literature analysis on pedagogical approaches such as situated learning, authentic learning, contextual learning, group-based learning, exploratory learning, problem-based learning and museum learning.

Table A.1: Characteristics of CALSs Characteristic Multiple roles, perspectives and skill levels Consideration of background, prior knowledge and experiences Consideration of learning styles

Social negotiation and collaboration Multimodal exploration of the environment and objects Ownership of the learning process and outcome Ownership of the technology

Literature Rationale User profile and perspectives [15, 22, 25, In order to support learners of various backgrounds, 29, 30, 42, skills and interests, the CALS should provide access 46, 84] to various roles, perspectives and skill levels in an adaptive manner. [22, 46, 73, Prior knowledge and experiences should be taken 74, 76] into account in learning activities. For example, a first-time learner has di↵erent needs to a regular learner. [9, 49]

Di↵erent learners prefer to learn in di↵erent ways. The CALS should support the variety of learning styles by o↵ering alternative content and activities via multimodal learner interfaces. Interaction and collaboration [15, 22, 25, Sharing experiences and facing challenges together 28–30, 46, facilitate e↵ective learning. 61, 72, 73, 76, 84] [15, 42, 61] By exploring the environment through various senses the learner becomes more attached to it. This relates to the characteristic “Consideration of learning styles”. Ownership [15, 22, 27, Ownership a↵ects directly to motivation to learn. 28, 30, 42, Furthermore, having control over one’s own learning 46, 61, 72, process is necessary for e↵ective learning. 76, 84] [73, 75, 84] In addition to increased motivation, owning the technology has direct consequences on the ability to use the technology e↵ectively. Continues on next page


73

Teemu H. Laine: Technology Integration in Context-Aware Learning Spaces Characteristic Authentic context Authentic activities that have relevance to the real world Compelling narrative to facilitate immersion Gained experiences integrated and applied across di↵erent subject areas Personal relevance Unobtrusive technology

Sca↵olding techniques Support for just-in-time reflection Support for post-reflection

Immediate feedback

Integrated, authentic assessment

74

Literature Rationale Authenticity and relevance [22, 24, 29, Solving real world challenges cannot be taught e↵ec74] tively in an unauthentic setting. Authentic context is also important for deep immersion of the learner. [29, 30, 46, Connecting learning activities to the real world is an 61, 72, 73] important part of making the meaning of concepts. Without real world relevance, the concepts remain abstract. [22, 25]

[30, 33, 61, 78]

The CALS should employ a compelling narrative that helps the learner to immerse quickly in the authentic context. Knowledge can and should be transferred across disciplines. The CALS should allow generalisation and linkage of the knowledge to other contexts and subject areas.

[22, 24, 61, 73, 84]

Learning activities in the CALS should have personal relevance, so the learner is able to construct a personal meaning of a concept. [76, 91] Technology should not distract the learner in the learning process. In the best case the learner does not even notice the existence of the technology and therefore can become fully immersed in the context. Support and assessment [15, 25, 29, Support should be available when the learner needs 46, 61, 72] it the most, and it should be faded out when the learner can face the challenges themselves. [22, 29, 30, The CALS should o↵er possibilities for reflection 42, 46, 70, while performing learning activities. During reflec72] tion new knowledge is linked to existing mental models and prepared for future linkages. [70, 72] The CALS should also support reflection after the learning activities have ended. This can be done for example with an interactive website through which the learner can retrace the learning process. [15] The learner should be provided with immediate, choice-dependent feedback after each activity. This helps to maintain (intrinsic) motivation and orientation in the learning process. [29] [30] Even though CALS are often deployed in informal learning contexts, sometimes assessment is necessary. In such cases the CALS should o↵er a possibility to perform assessment as part of the learning process.


B Evaluation Tool for Technology Integration in CALSs The following tables present the evaluative questions for the technology integration evaluation tool. The questions are categorised by the viewpoints of the learner, (Table B.1), the educator (Table B.2) and the context (Table B.3). Table B.1: Learner’s role in the evaluation tool

Unobtrusiveness of technology

Availability of resources

Contextawareness

A↵ordances Constraints

Learner How good is the user experience of the CALS? Does any of the used technologies distract the learner? How do the learners perceive the technology? (or do they perceive it at all?) Do the learners a↵ord using the system (if not free)? How does the CALS take into account the learner’s available time resources? Are the learners able to use the technology efficiently? What kind of connections can the CALS create between the learning content and previous experiences of the learners? How does the CALS take into account the learner’s personal context (e.g. location in a room, previous knowledge, preferences)? How does the CALS take into account the social context of the user (e.g. other learners)? How does context-awareness take into account the learner’s cultural background? How do the features of the CALS facilitate learning? How do the features of the CALS restrict/prevent learning?


75


Table B.2: Educator’s role in the evaluation tool



Contextawareness

A↵ordances Constraints

76

Educator How does technology related to the CALS a↵ect the educator’s normal work? How good is the user experience of the CALS operating interface and maintenance tools? Are the educator’s technical skills adequate for operating and maintaining the CALS? How does the educator’s content knowledge compare with the content in the CALS? How do the educator’s time resources match with required time for operating and maintaining the CALS? How is the maintenance of the CALS organised? How does context-awareness support pedagogical goals set by the educator? How well does the CALS take into account learners’ backgrounds and prior experiences? How do the features of the CALS facilitate the educator’s work? How do the features of the CALS restrict/complicate the educator’s work?


Evaluation Tool for Technology Integration in CALSs

Table B.3: Context’s role in the evaluation tool



Contextawareness A↵ordances Constraints

Context How does the technology integration process consider the authenticity of the context? How does the technology a↵ect the maintenance activities in the context? How is the CALS integrated into the context as part of the educators’ work description and as a permanent service (rather than a prototype)? How do the available financial resources compare with the requirements of the CALS development and maintenance? How sufficient is the quality/quantity of available information/learning content to support learning with the CALS? How well does the physical infrastructure support the CALS? To what extent is the physical context detected by the CALS? How do the features of the CALS benefit the context’s operations? How do the features of the CALS restrict/complicate the context’s operations?


77


78


C Disturbance Factors Table C.1 and Table C.2 present the disturbance factors identified in two evaluations of the UFractions game: one without the evaluation tool and one with the evaluation tool. Further evidence of these evaluations can be found in Papers VI and VII, respectively.


79


Table C.1: Disturbance factors found Area of Disturbance factor experience Social experi- Harassment ence Learning ex- Below ZPD perience Beyond ZPD Wrong age group Behavioural experience Emotional experience Immersion experience

Wasting time by pleasing others Disturbing content Too much story Monotony

Cognitive experience

Inappropriate graphics Inappropriate sounds Lack of animation

Contextual experience User experience

Inconvenient interaction with rods Unclear instructions Inconvenient interaction with phone Small screen Technical faults

80

in the first evaluation of UFractions Indication

I

Group members disturbed game play References to easiness of challenges

A

References to difficulty of challenges Suggestion to use the game for younger players Avoidance of answering wrong despite the lack of challenge References to shocking or disturbing events in the content References to too long story or too much reading References to repetition or monotony of the content References to poor graphics or suggestions to improve them References to poor sounds or suggestion to improve them References of lack of animation or suggestions to add them References to negative experience of using the rods References to unclear tasks or difficulty of understanding them References to negative experience of physical handling of or properties of the phone References to small screen size or difficulty to see the content References to technical problems during playing

A

A

A A A A A P P P P A P

P P


Disturbance Factors

Table C.2: Disturbance factors found Area of Disturbance factor experience Temporal Too long game experience Too short game Learning experience

Beyond ZPD Below ZPD Wrong age group Lack of sca↵olding Conflicting content

Immersion experience

Too much story Monotony Too educational

Social experience Emotional experience

Harassment Lack of peer support Disturbing content Punishment


Lack of animation Inappropriate graphics Inappropriate sounds


Inconvenient interaction with rods Inconvenient interaction with phone Technical faults Small screen Unclear instructions

in the second evaluation of UFractions Indication

I

References to a long game or a suggestion to make it shorter References to a short game or a suggestion to make it longer References to difficulty of challenges

A

References to easiness of challenges Suggestion to use the game for younger players References to getting stuck Conflict between own idea and game’s idea References to too long story or too much reading References to repetition or monotony of the content References to the game being too pedagogical Group members disturbed game play

A A

References to lack of support from peers References to shocking or disturbing events in the content References to dislike on getting questions wrong References of lack of animation or suggestions to add them References to poor graphics or suggestions to improve them References to poor sounds or suggestions to improve them References to negative experience of using the rods References to negative experience of physical handling of or properties of the phone References to technical problems during playing References to small screen size or difficulty to see the content References to unclear tasks or difficulty of understanding them

A A


81

A A

A A A A A A

A P P P P P

P P A


82


D Evaluation Instruments The following pages describe the evaluation instruments that are based on the technology integration evaluation tool and that were applied in paper VII to evaluate UFractions game in Mozambique. The evaluation was conducted on the role of the learner. First the pre- and post-test questionnaires are presented after which follows the interview questions.


83

Questionnaire for UFractions (May 2011) University of Eastern Finland researchers have been developing a mobile game UFractions for learning fractions in a fun way. To see how well UFractions suits for you we have prepared this questionnaire. We kindly ask you to answer the questions as honestly as possible and enjoy your time with the game. All answers will be handled anonymously and identity information will be removed from published results. Please answer the BEFORE PLAYING part before you start playing, and the AFTER PLAYING part after you have finished and returned the phone to us. Thank you for your time and don't hesitate to ask if you have any questions about this questionnaire or the game. BEFORE PLAYING 1. Demographics Your name: Date: School: Ethnic group and language: Gender: Male Female Age: Occupation: Team name (in the game): 2. Background – mobile device usage a) Do you have a mobile phone?

No

Yes (if No, go to section 3)

b) Estimate how much you use the following functions of your phone (mark with X): Several times a day

Once a day

Once a week

Once a month or less

i) SMS ii) Talking iii) Taking photos iv) Multimedia message (MMS) v) Playing music vi) Playing games vii) Chatting (e.g MSN) viii) Social media (e.g. Facebook) ix) Browsing internet x) Calendar/alarm clock xi) Other (what):

3. Background – games and math a) What kind of games do you usually play and what do you like about them?

Never

b) How do you feel about math (what emotions do you feel)?

c) Do you find fractions difficult? Why/why not?

Now go to play and have some fun!

AFTER PLAYING 4. Game experience a) What did you like or enjoy about the game?

b) What did you dislike or find difficult in the game?

c) Did you find out anything surprising when you were playing the game? What was it?

d) How would you suggest to improve the game?

e) Rate the following features of the game:

a) Fractions theory b) Playing with fraction rods c) Solving questions with rods d) Controlling the pace of the game e) Story of leopards f) Taking pictures g) Writing to guest book

Very boring 1 1 1 1 1 1 1

Boring

Neither boring nor interesting 3 3 3 3 3 3 3

2 2 2 2 2 2 2

Interesting 4 4 4 4 4 4 4

Very interesting 5 5 5 5 5 5 5

f) Which of these game activities did you like?

a) I liked playing with the fraction rods b) I liked answering the questions c) I liked interacting with leopards d) I liked reading the story e) I liked using the mobile phone

Strongly disagree 1 1 1 1 1

Disagree

No opinion

Agree

2 2 2 2 2

3 3 3 3 3

4 4 4 4 4

Strongly agree 5 5 5 5 5

5. Motivation While playing the game, what was the reason for you to keep on playing (what motivated you)?

a) I wanted to know what will happen next b) I wanted to know what will happen in the story c) I wanted to save the leopards d) I was curious to see what I can learn about mathematics e) I wanted to solve all questions correctly f) I wanted to play more with fraction rods g) I wanted to learn more about leopards h) I wanted to learn more about fractions

Strongly disagree 1 1 1 1

Disagree

Agree

2 2 2 2

No opinion 3 3 3 3

4 4 4 4

Strongly agree 5 5 5 5

1 1 1 1

2 2 2 2

3 3 3 3

4 4 4 4

5 5 5 5

6. Usability What are your opinions about the following statements?

a) Using phone was familiar to me b) Mobile device disturbed my playing c) Networking errors disturbed my playing d) Fraction rods disturbed my playing e) The screen was too full f) It was easy to use the phone as a tool for playing g) Game helped me when I got stuck h) Language was easy to understand i) Questions were easy to understand j) Story was easy to understand

Strongly disagree 1 1

Disagree

Agree

2 2

No opinion 3 3

4 4

Strongly agree 5 5

1

2

3

4

5

1 1 1

2 2 2

3 3 3

4 4 4

5 5 5

1 1 1 1

2 2 2 2

3 3 3 3

4 4 4 4

5 5 5 5

7. Clarity of the screen Rate the following aspects of the screen from “very unclear” to “very clear”.

a) Screen layout b) Text c) Graphics d) Sounds e) Navigation f) Codes (e.g. “W” or “BL”) on fraction rods

Very unclear 1 1 1 1 1 1

Unclear 2 2 2 2 2 2

Neither unclear nor clear 3 3 3 3 3 3

Clear

Very clear

4 4 4 4 4 4

5 5 5 5 5 5

8. Context-awareness

a) Story was suitable for Mozambique b) Story was useful to me

Strongly disagree 1 1

Disagree 2 2

No opinion 3 3

Agree 4 4

Strongly agree 5 5

c) Questions were suitable for my skill level d) Story was suitable to me e) I enjoyed playing together with my friends f) Finding correct rods was difficult

1

2

3

4

5

1 1

2 2

3 3

4 4

5 5

1

2

3

4

5

9. Available resources a) How much time do you think is enough to play this game? _____________________________________ b) How much would you be willing to pay for playing this game? _________________________ c) Please answer the following statements:

a) It was easy to play from the beginning b) I was able to relate the game events to my own previous experiences c) Game was too long d) Game was too short

Strongly disagree 1

Disagree

Agree

2

No opinion 3

4

Strongly agree 5

1

2

3

4

5

1 1

2 2

3 3

4 4

5 5

Strongly disagree 1

Disagree

Agree

2

No opinion 3

4

Strongly agree 5

1

2

3

4

5

1

2

3

4

5

1 1

2 2

3 3

4 4

5 5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

10. Overall experience

a) Compared to a math class this was exciting b) The game helped me to learn many new things c) The game made learning fractions difficult d) The game disturbed my learning e) After this day I find fractions more interesting than before f) I will think of leopards from now on whenever I do fractions g) I would also like to meet other animals in the game and help them h) I felt important as I was saving the leopards i) It was fun to play with the phone 11. Final comments

Please write here your last comments of the game or send greetings to leopards:

Thank you so much for your help!

APPENDIX B – Interview questions for end users 1. Demographics Name: School: How long have you known each other? 2. Questions Affordances and constraints 1. Did you learn something new in the game? What was it? 2. What surprised you in the game? 3. Were there any part in the game that you found interesting/boring? Why?

4. What are the advantages/disadvantages of using the game instead of a normal math class? 5. What do you think are the benefits of this game? 6. Which part of the story did you (dis)like? Why? 7. Which challenge/task did you (dis)like? Why? Critical factors 8. What do you think about using a mobile game for learning fractions at school or at home? 9. What do you think about using fraction sticks as part of the game? 10. Which technology caught your attention and why? 11. Did you have any problems with the mobile device? If yes, what kind of problems did you experience? - Basic usage? - Navigation? - Graphics? - Layout? - (Communication) Errors in the game? 12. How did the tasks and the story suit to each location where you were playing? How can this aspect be improved? 13. Did the game events reminded you about something that you have experienced in the past? What was it? Suggestions for improvements 14. How would you change the game to make it more fun/interesting? 15. Imagine yourself in the time 20 years from now. How do you think this game is now compared to what it was 20 years ago (i.e. today)?


90


Original Publications

Original Publications


91

Paper I

Laine T.H. & Joy, M. (2009). Survey on Context-Aware Pervasive Learning Environments, International Journal of Interactive Mobile Learning (I-JIM), vol. 3, no. 1, pp. 70-76. Reprinted with permission (Creative Commons Attribution Licence), Copyright 2009 The International Association of Online Engineering.

SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS

Survey on Context-Aware Pervasive Learning Environments doi:10.3991/ijim.v3i1.680

T.H. Laine1 and M. Joy2 1

University of Joensuu, Joensuu, Finland 2 Warwick University, Coventry, UK

Abstract—Context-aware pervasive learning environments consist of interconnected, embedded computing devices such as portable computers, wireless sensors, auxiliary input/output devices and servers. Until this study there has been no survey that has evaluated and presented information regarding these environments. In this paper, we conducted a survey to identify the commonly used technologies, methods and models behind these systems, and evaluated the role of mobile devices in the reviewed papers. As a result, we made five observations: (i) RFID was the most common sensor technology; (ii) several learning models were suggested, but none was validated properly; (iii) client-server architectures are prevalent in the systems and mobile devices were used most commonly to represent information; (iv) most of the systems supported multiple simultaneous users, but few facilitated virtual communication; and (v) possible roles for physical environments in pervasive learning systems are: contexts for learning, content for learning, and system resources. Evidence indicates that suitable learning models have yet to be validated, and that more roles of mobile devices could be emphasised. Index Terms—context-aware, literature survey, mobile learning, pervasive learning environment.

I. INTRODUCTION Mobile learning, or m-learning, has become popular and is currently being intensively researched. In this paper we consider m-learning to refer specifically to learning facilitated by mobile devices such as PDAs and mobile phones. The primary aim of m-learning is to provide the users with a learning environment which is not restricted to a specific location or time. Compared to a traditional classroom setting, m-learning increases the mobility of a learner, allowing him/her to learn while sitting in a bus, for example. Furthermore, networked mobile devices allow learners to perform co-operative learning tasks in a group. Pervasive learning is the latest trend in harnessing the technology to support learning. In this form of learning, the mediator is a pervasive computing environment which consists of interconnected, embedded computing devices such as portable computers, wireless sensors, auxiliary input/output devices and servers. One could therefore consider pervasive learning as an extension to m-learning where the roles of the intelligent environment and of the context are emphasised. The physical environment is central as it provides salient resources for learning. According to [15], a pervasive learning environment is a

70

setting in which students can become totally immersed in the learning process. They further note that pervasive computing is an immersive experience which mediates between the learner's mental (e.g. needs, preferences, prior knowledge), physical (e.g. objects, other learners) and virtual (e.g. content accessible with mobile devices, artefacts) context. The intersection of these contexts is referred to as pervasive learning environment ([15]). Reference [14] regard a pervasive learning environment as a collection of mobile users, mobile services, mobile devices, contexts and policies, while [12] state that in pervasive learning, computers can obtain information about the context of learning from the learning environment in which embedded small devices, such as sensors, pads and badges, communicate together. Common to these definitions is the interplay of intelligent technology and context in which the learner is situated (i.e. context-awareness). Other terms used to describe pervasive computing include situated computing, ubiquitous computing, embedded computing, ambient intelligence, and everyware. In this paper, pervasive learning environments are based on environments with embedded intelligence in the form of sensors, tags and interaction devices. There has been research conducted on building and evaluating pervasive learning environments, however no survey has yet evaluated these environments. Such information is necessary not only for avoiding reinventing the wheel, but also for understanding the current state-ofthe-art in this area. By recognising the commonly used technologies, methods and models, we can design and build pervasive learning systems more effectively. Our intention is to provide an overview of what kind of pervasive learning environments have been developed, how they were built, what are the sensor technologies used in these systems to make them context-aware, what learning models are suggested for these environments, and what are the roles of mobile devices. By reviewing existing work, we seek to build a solid ground for further research on how different learning models can be efficiently utilised in pervasive learning environments and what are the critical features of such an environment. The role of mobile devices is an important factor from the perspective of wider work which aims to design and implement a flexible pervasive mobile learning system. This work also includes establishing and recognising the best learning models for such system. The paper is organised as follows. We first define the methodology used in the survey and continue by describing the observations resulted from the analysis of

http://www.i-jim.org

SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS the literature. Finally, we conclude by discussing implications of the results and concluding the findings. II. METHOD This section presents the research questions and designs of data collection, evaluation and analysis. A. Research Questions In this survey we focused on articles presenting research outcomes that included a design, implementation, evaluation or test of a context-aware pervasive learning environment. We established a set of questions to be answered with the information extracted from the literature. These questions and their purpose are presented in Table I. TABLE I. RESEARCH QUESTIONS AND THEIR PURPOSE Question 1. What are the currently existing context-aware pervasive learning environments and how are they built?

Purpose We seek to discover the state-of-the-art in the field of context-aware pervasive learning environments. The survey is done from a technical perspective, emphasising particularly technologies for smart environment (e.g. sensors).

2. What learning models, if any, have been established to support pervasive learning experiences in these environments?

We consider this question particularly relevant because if there are no learning models established or validated in the previous work, we will have a rationale to conduct further research on the learning models in this field. If previous work supports particular learning models for pervasive learning, those can be used together with newly established models in the future work.

3. What is the role of mobile devices in existing pervasive learning environments?

This question is intended to find out how mobile devices have been harnessed in existing pervasive learning environments. The results of this question will be used to invent and combine ways to utilise mobile devices in pervasive learning.

B. Data Collection In order to collect the data in a reliable and reproducible manner, we devised a set of rules for paper inclusion. The established inclusion rules were as follows. a) The work describes a design, implementation, analysis or test of a pervasive learning environment or system. b) The presented environment/system uses sensors or other technologies for smart environments to enable context-awareness; having people walking around with mobile devices connected to a wireless network was not enough as it is merely m-learning. c) The work was presented in one of the following forums: IEEE International Conference on Pervasive Services, IEEE International Conference on Pervasive Computing and Communications, European Conference on Ambient Intelligence, International Conference on Mobile and Ubiquitous Systems, IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE & WMUTE), Pervasive E-Learning Workshop, Pervasive Computing Education Workshop, Pervasive Learning Workshop.

iJIM – Volume 3, Issue 1, January 2009

d) Data from one work does not overlap data from another work. In the case that two or more papers present the same system, the most recent or more comprehensive one was selected. e) If the study does not present the design, implementation, evaluation or test of a pervasive learning environment, it must discuss the suitability of learning models and styles to an existing pervasive learning environment. All works that failed to meet these rules were excluded. After establishing the inclusion rules, we performed the data collection in two phases. In the first phase titles and abstracts of articles presented in the given forums were read. If the title of an article did not seem relevant (e.g. the field was completely different), the abstract was not read. If the article showed relevance based on the title and the abstract, it was selected to the second phase. In total 35 papers were evaluated suitable as a result of the first phase. In the second phase, the abstract and the introductions were read, and based on that information part of the papers were excluded as they did not meet the inclusion rules. After the second phase the number of relevant papers was decreased to 18. We recognise that this is not a comprehensive survey from the paper point of view. However, the purpose of this paper is not to be a comprehensive literature review, but rather a directed probe into pervasive computing, learning and technologies. C. Data Evaluation After the main body of the papers was collected, we proceeded to read through the remaining papers in order perform a deeper analysis of the data and extract relevant information. For this purpose, we established a set of questions to be answered with that information. The questions are based on the research questions and they are presented in table II. TABLE II. DETAILED RESEARCH QUESTIONS

Q-A0: What are system/environment?

the

Question description

and

purpose

of

the

Q-A1: Is it based on a client/server approach? If not, what is it based on? Q-A2: What is the hardware/software platform of the system? Q-A3: What is the programming language used in development? Q-A4: What kind of sensors are used and how? Q-A5: What is the role of the physical environment in the system? Q-A6: Is it a multi-user system? Q-B0: Are learning models discussed? Q-B0a: If yes, what are the suggested learning models? Q-B0b: How the suggested models have been validated? Q-B1: What learning activities does the system support? Q-C0: What is the role of mobile devices in the system?

In these questions A, B and C refer to the research questions 1, 2, and 3, respectively. The question Q-B0 has two sequential questions, namely Q-B0a and Q-B0b, which are only answered if the Q-B0 has a positive answer. We could not extract answers to all these

71

SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS questions from every paper, but majority of the papers had sufficient information available. During the evaluation process we excluded 4 papers as deeper analysis showed that they did not meet the inclusion rules, reducing the number of included papers to 14. However, as one of the papers presents 2 different systems, the total number of relevant works was 15. The observations based on the information extracted from these papers are presented in the following section. Papers that were part of the survey but are not explicitly referred to elsewhere in this article are: [1], [4], [11] and [16]. III. OBSERVATIONS After the data evaluation, we performed a deeper analysis on the extracted information. As the result, a set of observations was established. These are presented in table III and in the following sections we present each observation in detail. The questions presented in table II are linked to the observations with the question codes in parentheses. TABLE III. OBSERVATIONS ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS Observation 1 RFID (Radio Frequency IDentification) is the most prevalent sensor technology used in pervasive learning environments. (Q-A4) Observation 2 There are several learning models that are suitable for different learning activities in pervasive learning environments, but none of them was validated properly (Q-B0, Q-B0a, Q-B0b, Q-B1) Observation 3 The vast majority of the systems are based on a client-server architecture and most of them utilize mobile devices in various ways; content representation tool is the most common role for mobile devices. (QA-1, QA-2, QA-3, Q-C0) Observation 4 The majority of the pervasive learning environments support multiple simultaneous users, but only a small number support virtual communication among the users. (QA-6) Observation 5 Currently established roles for the physical environment in pervasive learning systems are: context for learning, content for learning, and system resource. (QA-5)

A. Observation 1 From the reviewed works, the most commonly used sensor technology was RFID (Radio Frequency IDentification) as 9 out of 15 works mentioned it explicitly. The second most popular sensor technology was GPS, scoring 4 hits in total. Other explicitly mentioned sensors were light sensors, moisture sensors, wired trigger sensors, water flow sensors, piezoelectric “object usage” sensors, force sensors, temperature sensors, humidity sensors, infra-red distance sensors, motion sensors, touch sensors, cameras, 3D accelerometers and microphones. Two works did not explain what kind of sensors are used as they merely presented the possibility of using sensor technologies in the respective systems. From the 13 works that mentioned some sensors being used, 7 utilized more than 1 sensor type. RFID has been successfully used for sensing nearby persons ([3]), physical resources ([3], [2]), locations of the user or objects ([12], [3], [2]), and user's actions ([2]). In addition to presenting a pervasive learning environment,

72

[13] mentioned two ambitious ongoing projects in Japan, namely food traceability and location-aware computing. The goal of the former project is to attach RFID tags onto all food products, thus increasing the visibility of the food production chains. The latter project aims to tag all places in Japan's national infrastructure, thus supporting efficient transportation, sightseeing and also pervasive learning. Most of the pervasive learning applications that utilised RFID technology used RFID reader embedded or attached (via Bluetooth or by using extension slots) to mobile devices to read the tag information. This might be an indication that RFID is likely to become the next big thing in mobile wireless near-field communication just like Bluetooth did a few years back. B. Observation 2 Out of 15 works only 7 discussed learning models and most of them did not explicitly suggest their suitability. However, we were able to extract the learning model types supported in each system by carefully analysing the descriptions of system functionalities. As a result, we devised a list of learning models that could be used in pervasive learning environments. Many systems supported more than one of these models simultaneously, e.g. a system could be both group-based and problem-based. Table IV presents the extracted learning models and examples how they were used. Reference [10] suggests the most suitable learning models for pervasive learning are on-demand learning, hands-on or minds-on learning, and authentic learning. They further divide authentic learning into action, situated, incidental and experimental learning. The authors particularly emphasize the effectiveness of authentic, contextual learning for learning a foreign language. It is clear, however, that authentic learning is suitable for any kind of learning need where environment and context are major factors. TABLE IV. LEARNING MODELS IN PERVASIVE LEARNING ENVIRONMENTS Learning model Example Group-based Reference [5] proposes a system which utilizes an learning RFID-enhanced interactive sensor board for museums. The idea is that when an object is placed on the sensor board, a projected image on the board shows more information about that object. The board is able to recognise multiple objects simultaneously, thus a group of learners can communicate and learn at the same time. Individual learning

Reference [8] presents a system in which children gather information pertaining to a range of reptiles, small mammals, insects, fish and birds both within indoor and outdoor environments. Camera and 2D bar codes are used to collect the information. The children perform the activities independently and communication between the users is not supported by the system. Naturally, ad-hoc face-to-face communication may occur, but learning is mostly individual.

Microlearning

Reference [2] constructed a pervasive environment for learning a foreign language according to the model of microlearning, in which users are continually given small chunks of knowledge. The goal of the system is to teach vocabulary through the usage of responsive everyday objects in a household. When a learner interacts with an object, the vocabulary related to it is played back as sound.


SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS

Authentic learning

Reference [10] proposes two different systems; JAPELAS for learning polite Japanese expressions through situations, and TANGO for learning vocabulary about the surrounding objects. According to the authors, both of these system are particularly well suited for authentic learning as language skills are best acquired in a real-world environment. The same authors have created the JAMIOLAS pervasive learning environment ([12]), which allows users to learn Japanese mimicry and onomatopoeic expressions through authentic situations. For example, when a user goes out and it rains, the system tells the user onomatopoeia for raining. The authors explicitly refer to this learning activity as authentic learning.

Learning by playing

The pervasive learning environment presented in [7] consists of a set of RFID- and sensor-enhanced toys and a mobile device. In the “Knight's Castle”, the toy characters respond to children's actions by, for example, telling a historical story or singing a song. This system is a good example on how a pervasive learning environment can be used in a playful manner to educate children.

On-demand learning

Reference [9] present a pervasive learning system (LORAMS) in which mobile videos and RFIDtagged objects are used to record and share learning experiences. There are two types of user role in the system: movie provider and movie watcher. In the latter role, users retrieve movies from the system according to the context, so the learning material is acquired in an on-demand basis, thus we can refer the learning activity of the second user role as ondemand learning.

Hands-on and Minds-on learning

The pervasive learning environment (LORAMS) presented in [9] supports learning by hands-on experience (see “On-demand learning”). The motivation for the system was to provide a tool to record a learner's experience and share it later with other learners. Hands-on activities are particularly useful here, as after recording, they can be easily imitated by other learners.

Problem-based learning

Reference [3] describes a pervasive learning system for a university laboratory in which learners are provided with a set of learning activities to perform. The objects in the lab are equipped with RFID tags so the system is aware what the users are doing at any given moment, and can therefore monitor the progress of the learning activities. Learning activities are represented as complex problems to be solved, hence problem-based learning.

Despite several learning models being presented in the papers, few were tested or validated. Microlearning was tested by [2] by running a non-stop scenario for several weeks. Participants in the test were optimistic about the possible use of technology and they showed increased level of knowledge of their foreign language vocabulary. However, as the scenario was executed only for two test subjects, this result does not yet validate the usage of microlearning in sensor-enhanced pervasive learning environment, but neither does it disprove the positive effect of the technology on learning. The system presented by [8] (independent learning) was validated by two test settings; an initial user study with a small group of children, and an investigation of overall performance of the system. The results of the former test suggested that the children enjoyed using the system and the overall feedback was positive. The school staff members were also supportive towards the usage of the system. The


performance test concentrated on how the process of capturing an image and awaiting a response affected the usability of the system. The time of the process varied from 6 to 26 seconds, depending on the status of the GPRS connection. The performance test did not validate the learning model directly, but it did indicate suggest that the system is usable. In the third validated system [7], the authors set up experiments in which groups of students assembled a part of a computer; one group used Google to retrieve information and the other group used the LORAMS system to watch videos previously recorded by other students who had had the same learning experience earlier. The results suggest that LORAMS helped the students of the latter group perform better than the first group. C. Observation 3 All except one of the reviewed systems use a clientserver architecture, and the exception implements a touchbased and RFID-enabled sensor board in a museum [5]. In this stand-alone system, the sensor board is directly connected to a computer which also manages the video projector used to project an image onto a board. The projected video is adapted to user actions and objects places on the board. Of the client-server based systems, two systems also allowed ad-hoc peer-to-peer communication without server intervention. Details of hardware and software were not given in many of the reviewed papers and none presented a thorough technical description. Therefore, the following information may not correspond to all the state-of-the-art technologies used in pervasive learning environments. The operating systems of the mobile devices were Windows Mobile, Windows XP and Symbian OS. On the server side, XML was used for encapsulating data and messages. Furthermore, [13] used TRON (The Realtime Operating system Nucleus) operating system on the server. In other systems the operating system was not explicitly mentioned. Communication between the server and the client was established either by GPRS or WLAN, and two papers mentioned the usage of the HTTP protocol. The programming environment on the server side was mentioned only twice (Java Servlets on Tomcat software, and ASP.Net). Information about the programming language used on the client was available for all but seven of the systems, and were: C++ (3), Java (2), Visual Basic (2), C# (1) and Flash (1). One of the systems used both Flash and C++. Mobile devices were used as learning tools in all but one of the systems. Explicitly mentioned types of mobile devices were Tablet PCs (2), PDAs (6) and mobile phones (3). Based on this information we can conclude that PDAs may be currently the most popular client type in pervasive learning environments. However, due to the recent convergence of mobile phones and PDA devices, both device types could be used for the same purpose. Tablet PCs are somewhat clumsy for pervasive learning in systems where high mobility is required. We established different roles of mobile devices based on the extracted information, and these roles are presented in table V together with their frequencies and descriptions. Frequency denotes how many times a role was present in the reviewed systems, and it is worth noticing that in one system a mobile device can have several roles, but none of the systems supported all five. One system used a mobile

73

SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS device as an auxiliary tool for reading RFID tags, but users were also able to use the system without a mobile device. TABLE V. ROLES OF MOBILE DEVICES IN PERVASIVE LEARNING ENVIRONMENTS Role of a f mobile device Data 5 collection tool

Content representation tool

Description Users collect data from the environment by using information capturing features of the device such as a camera (still and video images). Captured data can be processed further by the system or stored as a trace of learning activities, for example.

13 The high frequency indicates that this is probably the most important role of mobile devices in pervasive learning systems. In this role, mobile devices are used to view context-sensitive content provided by the system. The format of the content represented on mobile devices in the reviewed systems was text, image, audio or video.

Communicatio 4 n tool

In some of the systems mobile devices were utilised to establish communication between users of the system. The forms of communication are explained in Observation 4.

Navigation tool

2

Mobile devices were used for navigation; with the help of the device a user is able to know his/her own location or a location of a specific object within the environment. In the reviewed systems, the navigation feature was either based on GPS or RFID.

Notice receiving tool

2

In two systems, different types of announcements and notices were delivered to users' mobile devices, such as reminders and announcements submitted by the teacher.

D. Observation 4 Most of the reviewed systems were built to support multiple users. We consider two different aspects of a multi-user system: the first aspect is support for multiple simultaneous users, and the second aspect is system mediation of communication between users. In other words, a system can support multiple simultaneous users without providing methods for communication, or it can support communication among users by some means. Twelve of the fifteen reviewed systems supported multiple simultaneous users, however, this data could not be extracted from all the papers. The number of systems providing communication tools for users was only six. Communication between users was either physical (2) or virtual (4). We considered as physical communication only those cases in which the communication by conversation or other physical means was explicitly mentioned as a part of the learning experience. Some of the systems allow both virtual and physical communication if the users share the same location and time. The systems providing tools for virtual communication utilised one or several of the following methods: forum, chat, SMS, instant messaging and content sharing. It is notable that none of the systems used audio or video communication even though particularly audio communication would be natural for mobile devices.

74

E. Observation 5 The roles of the physical environment had some variation but in general three different roles were recognisable, albeit not explicitly presented. These roles and their respective frequencies were: context for learning (9), content for learning (7), and system resource (3). It is worth noticing that in one system an environment can have multiple roles. For example, there were five cases where the environment was both context and content for learning. Additionally, two of the reviewed systems, an interactive sensor board for museums [5] and an interactive toy set for children [7], did not utilise the environment, and one paper did not state the role of the environment at all. Environment is a context for learning when learning is situation-based and the system adapts according to situations and contexts in which the user is present. This is also called contextual or situational learning. The environment provides content for learning when the system utilises the information within the environment as a learning resource. Finally, environment is a system resource when some objects within the environment are triggers for system events (e.g. furniture with embedded sensors which trigger usage events [2]). Fig. 1 depicts the central role of a physical environment in pervasive learning systems.

Figure 1. Three Roles of Physical Environment in Pervasive Learning Systems

IV. DISCUSSION The evidence presented in Observation 1 suggests that RFID is the most prevalent sensor technology used in pervasive learning environments, in part due relatively cheap price of RFID tags (approx. 1€ each in the authors’ countries) and readers (150€), compared to the cost of a basic wireless sensor node of at least 300€. RFID-based readers are already available in some mobile devices as integrated chips, including models by Nokia and Samsung, and we expect that RFID will become a mainstream technology in mobile devices within 5 years. This development will enable tagging any object in a pervasive learning environment, thus making the underlying system more aware of the environment. Observation 2 identifies several suitable learning models; however these require proper validation and comparison. Many of the proposed learning models were not validated, and those that were did not provide reliable


SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS results, as the test scenarios were inadequate in terms of the numbers of test participants and repetitions. It was discouraging to discover that only a handful of papers explicitly discussed learning models, and this leads us to believe that the authors of the other papers either did not consider learning models at all or did not include that information. All the learning models followed an informal constructivist approach. Authentic learning was mentioned more than once, thus suggesting its potentiality for pervasive learning. Nevertheless, the results of the observation 2 indicate that in this field learning model validations are required before any of the models can be seriously recommended. Observation 3 concentrated on technical implementations of pervasive learning environments and roles of mobile devices in them. The use of client-server architectures in most of the systems shows that centralised control is used in preference to a distributed system. The benefits of using a centralised approach are the ease of installation and maintenance. However, a distributed system consisting of autonomous sensor nodes and one or more coordinating servers would be more fault-tolerant and load-balanced. Fault tolerance is particularly important in large systems which are running constantly and have hundreds or thousands of resources. The systems presented in the reviewed papers were quite small, thus the absence of distributed control is justified. Popularity of PDA devices (6) as clients over Tablet PCs (2) and mobile phones (3) can be explained with screen size, physical dimensions, and processing capabilities. Displays on mobile phones are often too small for viewing information other than text and low quality images/video. On the other hand, Tablet PCs have large displays, but they are more difficult to carry around due to their large physical size. PDA devices often have larger displays than mobile phones and their size is smaller than that of Tablet PCs. Moreover, PDA devices have enough processing power for handling basic media types, while the resources are often more limited on mobile phones. Despite the popularity of PDA devices, mobile phone and PDA technologies have been converging, and there is a similar trend of convergence going on between laptops and mobile phones/PDAs. These new devices are called Ultra Mobile PCs (UMPCs) and their size is smaller than Tablet PCs, but bigger than mobile phones or PDAs. In addition to being highly portable, UMPC devices are capable of running a fullscale Windows XP operating system or equivalent Linux distribution, thus making them suitable client devices for various software solutions supporting pervasive learning activities. Currently the problems of UMPCs are high price and relatively short battery life. However, we can expect these aspects to improve in the near future. According to observation 3, there were five types of roles for mobile devices in the reviewed systems: data collection tool, content representation tool, communication tool, navigation tool and notice receiving tool. Since the content representation tool was the only role having a frequency more than 10, many of the systems merely concentrated on providing contextsensitive content to the user. This indicates that there is work to be done to increase interaction between the environment and the users, as well as among the users. For example, the data gathered with a data collection tool can be saved and processed later to continue the learning


experience at another location, e.g. at home or in a classroom. As another example, communication with peers can help users to establish and strengthen social relationships. Observation 4 concluded that only a few pervasive learning environments are truly multi-user systems through supporting communication among users. The lack of voice- and video-based communication was also noted, and we suggest that a reason may be the requirement for other running applications to be closed before using mobile phones' built-in voice call capabilities. Furthermore, creating a new reliable VoIP (Voice Over IP) application is not a trivial task. Audio/video-based communication is more personal, instant and effective than forums or chats. If a pervasive learning environment is to be built on a principle of virtual collaboration, using instant communication is possibly a good way to implement it. An alternative method is to provide a meeting request tool for the users through which two or more users could meet physically after agreeing on it virtually. This kind of approach was used by [6] where two users of the system met physically after one user had sent a help request to another user. In Observation 5, we distinguished three different roles for the physical environment in pervasive learning systems: context for learning, content for learning and system resource, and the frequency figures (9, 7 and 3, respectively) indicate that context and content are used most often. Usage of the environment as a system resource would be higher if more systems would embed wireless sensor networking components for sensing different aspects of the environment. The low frequency of the system resource role is related to the lack of interaction with the environment; if the system would be able to closely observe user's behaviour and the state of the physical environment, the system would become more responsive and adaptive. This would in turn encourage users to interact more with the environment by using different objects and observing the consequences on the mobile device or in the physical environment. V. CONCLUSIONS We have reviewed 15 pervasive learning environments by concentrating on their underlying technology, suitable learning models, and roles of mobile devices. From the technological perspective, the majority of the systems used client-server architectures using mobile device clients, suggesting centralised control. The most popular client mobile devices were PDAs, and RFID was the most used sensor technology, partly due to its cheap cost compared to other sensor types. The most popular roles of the mobile devices were as a content representation tool and a data collection tool. We concluded that there is still work to be done in order to utilise the capacity of mobile devices to the full extent. Three different roles for the physical environment were identified: context for learning, content for learning, and system resource. From the point of view of learning models, the reviewed systems implicitly or explicitly suggested several constructivist models to be suitable for pervasive learning environments, in which authentic context is in a central position. However, the suggested learning models were insufficiently validated, and this is an area for future investigation.

75

SURVEY ON CONTEXT-AWARE PERVASIVE LEARNING ENVIRONMENTS As a future activity, we intend to use the results of this survey to design and build a flexible pervasive mobile learning environment that uses not only RFID, but also wireless sensor nodes, auxiliary input/output devices, mobile devices and intelligent agents. We will build this system modularly in a way that will be easy to adapt to different environments such as museums, schools, fairs, amusement parks, art houses and companies. We will use the system to investigate how different learning models can be efficiently applied in pervasive learning environments and what are the critical features of such environments.

[9]

[10]

[11]

[12]

REFERENCES [1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

76

D. Barbosa, A. Yamom, I. Augustin, L. da Silva, C. Séller, and J.Barbosa, “Learning in a large-scale pervasive environment,” Proc. of the Pervasive Computing and Communications Workshop, 2006. J.S. Beaudin, S.S. Intille, E. Munguia Tapia, R. Rockinson, and M.E. Morris, “Context-sensitive microlearning of foreign language vocabulary on a mobile device,” Proc. of the European Conference on Ambient Intelligence 2007. M. Derntl, and K.A. Hummel, “Modeling context-aware elearning scenarios,” Proc. of the Third International Workshop on Pervasive E-Learning, 2005. P. Grew, F. Giudici, and E. Pagani, “Specification of a functional architecture for e-learning supported by wireless technologies,” Proc. of the Pervasive Computing and Communications Workshop 2006. F. Kusunoki, M. Sugimoto, and H. Hashizume, “Toward an interactive museum guide system with sensing and wireless network technologies,” Proc. of the IEEE International Workshop on Wireless and Mobile Technologies in Education 2002. T.H. Laine, C. Islas Sedano, M. Vinni, and E. Sutinen, “Engagement, motivation and inspiration: The SciMyst Framework in Museums,” Proc. of the Fourth Conference on New Exploratory Technologies, 2007. M. Lampe,and S. Hinske, “Integrating interactive learning experiences into augmented toy environments,” Proc. of the Pervasive Learning Workshop at the Pervasive 2007 Conference. K. Mitchell, and N.J.P. Race, “uLearn: facilitating ubiquitous learning through camera equipped mobile phones, ” Proc. of the Third IEEE International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, 2005.

[13]

[14]

[15]

[16]

H. Ogata, Y. Matsuka, M. El-Bishouty, and Y. Yano, “LORAMS: capturing sharing and reusing experiences by linking physical objects and videos,” Proc. of the Pervasive Learning Workshop at the Pervasive 2007 Conference. H. Ogata, and Y. Yano, “Context-aware support for computersupported ubiquitous learning,” Proc. of the Second IEEE International Workshop on Wireless and Mobile Technologies in Education, 2004. H. Ogata, and Y. Yano, “Knowledge awareness map for computer-supported ubiquitous language-learning,” Proc. of the Second IEEE International Workshop on Wireless and Mobile Technologies in Education, 2004. H. Ogata, C. Yin, and Y. Yano, “JAMIOLAS: supporting Japanese mimicry and onomatopoeia learning with sensors,” Proc. of the Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, 2006. K. Sakamura, and N. Koshizuka, “Ubiquitous computing technologies for ubiquitous learning,” Proc. of the Third IEEE International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, 2005. E. Syukur, and S.W. Loke, “MHS learning services for pervasive campus environment,” Proc. of the Fifth International Workshop on Pervasive E-Learning, 2007. A. Syvänen, R. Beale, M. Sharples, M. Ahonen, and M. Lonsdale, “Supporting pervasive learning environments: adaptability and context awareness in mobile learning,” Proc. of the 2005 IEEE International Workshop on Wireless and Mobile Technologies in Education. M.J. Weal, D. Cruickshank, D.T. Michaelides, D.E. Millard, D.C. De Roure, K. Howland, and G. Fitzpatrick, “A card based metaphor for organising pervasive educational experiences,” Proc. of the Pervasive Computing and Communications Workshop 2007, pp. 165-170.

AUTHORS T.H. Laine is with the Educational Technology research group at the Department of Computer Science and Statistics at University of Joensuu, Finland (e-mail: [email protected]). M. Joy is with the University of Warwick/Computer Science, Coventry, UK (e-mail: [email protected]). Manuscript received 6 October 2008. Published as submitted by the authors.


Paper II

Laine, T.H., Islas Sedano, C., Vinni, M. & Joy, M. (2009). Characteristics of pervasive learning environments in museum contexts, Proceedings of the MLEARN 2009 Conference, Orlando, Florida, pp. 26 - 34. Reprinted with permission, Copyright 2009 The International Association for Mobile Learning.

Characteristics of Pervasive Learning Environments in Museum Contexts Teemu H. Laine, Carolina Islas Sedano, Mikko Vinni Dept of Computer Science and Statistics University of Joensuu P.O. Box 111, 80101 Joensuu, Finland [email protected]

Mike Joy Computer Science Warwick University Coventry, CV4 7AL, UK [email protected]

ABSTRACT

There is no appropriate learning model for pervasive learning environments (PLEs), and museums maintain authenticity at the cost of unmarked information. To address these problems, we present the LieksaMyst PLE developed for Pielinen Museum and we derive a set of characteristics that an effective PLE should meet and which form the basis of a new learning model currently under development. We discuss how the characteristics are addressed in LieksaMyst and present an evaluation of the game component of LieksaMyst. Results indicate that, while some usability issues remain to be resolved, the game was received well by the participants enabling them to immerse themselves in the story and to interact effectively with its virtual characters. Author Keywords

Pervasive learning environment, museum learning, learning model, pervasive game, mobile learning INTRODUCTION AND BACKGROUND

Museums are rich repositories of information to be shared with visitors. This information can often remain partially hidden despite efforts of curators in designing cues, labels and tours. This is also the case in the Pielinen Museum, which is the second largest outdoor museum in Finland. It is renowned for its authentic atmosphere, and as such most exhibits have been left intentionally without tags, labels and information boards. The challenge of conveying the hidden stories of objects and of the past lives is tackled by an innovative pervasive learning environment (PLE), LieksaMyst, which consists of a set of learning tools including, for example, an intriguing, story-based pervasive mobile game. In the process of designing and implementing the system, we derived a set of characteristics that effective PLEs for museums should conform to. In this paper we present the background to our work, the LieksaMyst PLE, the set of characteristics, and an initial evaluation of the PLE based on the feedback from a group of learners using LieksaMyst's pervasive mobile game component. Pervasive Learning Environments

Firstly, we make a distinction between the terms pervasive and ubiquitous which are often used inconsistently and interchangeably. Lyytinen and Yoo (2002) have proposed a division of different learning types along the axes of embeddedness and mobility (Figure 1), according to which, pervasive implies less mobility than ubiquitous. However, we do not see it only as a matter of place – pervasive learning also relates to time and activity, hence a pervasive learning experience is bound to vary according to place, time and a learner's activity. Despite the differences at the conceptual level, the same technologies (e.g. mobile devices, sensors, smart tags) can be applied to both ubiquitous and pervasive learning.

1

Figure 1. Types of learning according to levels of mobility and embeddedness (Lyytinen and Yoo, 2002) From another perspective, pervasive learning can be considered as an extension to m-learning with an emphasis on the roles of an intelligent environment and of the context. The physical environment is central as it provides salient resources for learning (e.g. museum objects). According to Syvänen et al. (2005), a pervasive learning environment (PLE) is a setting in which students can become totally immersed in the learning process. They further note that pervasive computing is an immersive experience which mediates between the learner's mental (e.g. needs, preferences, prior knowledge), physical (e.g. objects, other learners) and virtual (e.g. content accessible with mobile devices, artefacts) contexts, and the intersection of these contexts is the pervasive learning environment. Syukur and Loke (2007) regard a pervasive learning environment as a collection of mobile users, mobile services, mobile devices, contexts and policies, while Ogata et al. (2006) state that in pervasive learning, computers can obtain information about the context of learning from the learning environment in which embedded small devices, such as sensors, pads and badges, communicate together. Common factors in these definitions include the interplay of intelligent technology and the context in which the learner is situated (i.e. context-awareness). Currently there is a lack of a theoretical learning model on which effective pervasive learning environments can be built (Laine and Joy, 2008). In this paper we establish foundations for such a model by deriving a set of key characteristics of pervasive learning environments for museums. Learning in Museums

There are many reasons why people visit museums. For some it is a leisure activity while others may come together with a school group. Most visitors come to museums in order to learn something new or to continue refining and refreshing the knowledge gained from previous visits. Children and young adults can be an exception as their motivation for the visit can be the authority of school and not their own choice. Just as there are many reasons for visiting museums, there also exist reasons for not visiting them. One particularly strong reason is a negative attitude towards museums which only increases the unwillingness for free-choice visits (Black, 2005). Therefore it is important to work towards improving and maintaining positive attitudes of potential visitors towards museums. The type of learning that takes place in museums is a hybrid or continuum between informal and formal, and the degree of (in)formality depends on the purpose of the visit. For example, a school group may have a slightly more formal learning agenda than a senior couple. Furthermore, a school group's visit to museum is often connected to more formal learning that takes place in an ordinary classroom environment. In addition to why people visit museums, we should also consider how the visits should be organized in order for them to be effective in terms of learning. There are two basic visit types: free and guided. It has been suggested that neither of the extremes is optimal for learning but that we should choose something in between instead (Bitgood, 1989; Linn, 1980). Indeed, it has been suggested that curators and guides have a tendency to communicate one-directionally (Durbin, 2004) which reminds us of the behaviorist monologues familiar from formal classroom environments. For first-time visitors it may be beneficial to run an orientation program prior to the visit (e.g. teacher tells about the museum to children before the visit). Orientation can also be facilitated with cues which guide an (unguided) visitor (Rennie and McClafferty, 1995). Finally, visitors should not be voiceless in the museums but they should rather be encouraged to interact with the environment (Hall and Bannon, 2005) and with each other. There exists substantial evidence on positive outcomes related to museum visits. For example, Falk and Balling (1979) reported the development of positive attitudes and cognitive learning; Koran and Koran (1986) suggested that a museum visit is an interesting experience which raises curiosity, affects psychomotor development, interest, appreciation, motivation, and generalization; and Wolins et al. (1992) reported affective/emotional experiences. These outcomes relate to all three dimensions of learning in Bloom's taxonomy: cognitive (knowledge), affective (attitude), and psychomotor (skills). These are also emphasized in the evaluation of Hooper-Greenhill et al. (2003) which concluded that museums, archives and libraries impact learning in the following dimensions: (1) increase in knowledge and understanding (cognitive); (2) increase in skills (psychomotor); (3) change in attitudes and values (affective); (4) evidence of enjoyment, inspiration and creativity (motivational); and (5) evidence of activity, behavior, progression (conative). Given this support for a variety of learning types, museums are a useful context for investigating a wide range of learning activities from theory to practice. Mobile and Wireless Technologies in Museums

Using technology in museums is not a new phenomenon as various technologies have been utilized over forty years – from reel-to-reel tapes to cassettes, and from digital players to mobile devices (Proctor and Tellis, 2003). Our focus is on mobile and wireless technologies as they are key elements of pervasive learning environments. Typically mobile devices are either PDAs or mobile phones, but Ultra Mobile PCs and Tablet PCs are also used. In the future we also expect to see more wireless netbooks (small and inexpensive laptops). In addition to mobile device portability, personality, intuitiveness, and ubiquitousness, pervasive learning environments should also support location- and context-sensitiveness. Context-sensitive mobile-based systems have been used in museums and science centers before, including Xsplot (Hsi and Fait, 2005); Via Mineralia (Heumer et al., 2007); Tate

Modern's Multimedia guide (Proctor and Burton, 2003); and the MOBIlearn application in Nottingham Castle Museum gallery (Lonsdale et al., 2005)). Existing systems often utilize RFID or other smart tagging to implement contextsensitiveness. Positioning technologies (e.g. WLAN positioning, GPS) have also been used to make the system aware of the user's location (e.g. Wang et al., 2005). PERVASIVE LEARNING ENVIRONMENT FOR A LIVING MUSEUM Pielinen Museum

Pielinen Museum in Lieksa is the second largest open air museum in Finland, hosting over 70 old buildings and structures containing over 100,000 objects from different periods of time. Pielinen Museum is a living museum as it depicts how life used to be in Eastern Finland in the past. In 2007 and 2008 the museum attracted 8968 and 8692 visitors respectively. Figure 2 shows the Virsuvaara house exterior and interior, where our work has so far been concentrated.

Figure 2. House exterior (left) and interior(right) – the right half was used in the first experiments. Authenticity is one of the strengths of Pielinen Museum and in order to keep the atmosphere authentic the buildings, structures and objects have not been equipped with tags and labels. Until now, the only way to know more about the objects and buildings has been through guided tours where information has been mostly one-directional and schedules of tours have not always been convenient. We sought a solution to these challenges in user-centered design workshops with museum visitors during summer 2008 before implementing the LieksaMyst pervasive learning environment described in below. Workshop attendees wished to know more about life in specific periods of time and how particular items were used and connected to other items. Some attendees also suggested that it would be interesting to hear authentic sounds (e.g. old master's snoring). A deeper analysis of the workshop data will be presented elsewhere. LieksaMyst

LieksaMyst is the name for a pervasive learning environment (PLE) that we have developed in the Pielinen Museum together with a group of museum visitors and the curators of the museum. Rather than merely replacing the human guides, LieksaMyst offers possibilities for versatile interaction with the museum environment. LieksaMyst's core is a story-based role-playing game which takes the learner back in time to meet people who lived in the old houses and used the authentic objects for various activities. Together with these authentic albeit fictional characters, the learner experiences daily routines of the respective period of time. Interaction between the learner and the fictional character is done through the mobile device and the system supports text, images, sound and video. One game session can last from 20 minutes to several hours, depending on how much content is available and how motivated the learner is. Currently we have created a story for one character living in one of the largest buildings in the museum, Virsuvaara. The character is Anna, the 40 year-old lady of the Virsuvaara house. She lives together with her husband, children, grandparents, servants and lodgers, in total 18 persons in a single room. Among her daily activities Anna tells and shows the learner for example how butter is churned, how carpets are made, and what kind of food was eaten in her house in 1895. The learner is also presented with various challenges ranging from intriguing queries to finding a specific object needed to complete an activity. These challenges are part of the interaction with Anna – she requests the learner's help in order to complete her daily chores. Object recognition is currently performed manually by typing in a short numeric code that is carved on an authentic-looking wood piece next to the respective object. The learner is given the chance to select from various alternative story branches. At the end of the day as Anna wishes farewell, she prompts the learner to sign her guest book. This entry, together with the learning experience (story path) of the learner is stored on the server so that it can be presented later, for example on the homepage of the game for reflection purposes. In terms of social interactions in the game, learning situations often involve several learners using a single mobile device and several groups exchanging ideas with each other without explicit encouragement. In addition to the role-playing game, LieksaMyst has also other learning tools available. Currently we have implemented a database discovery tool which allows context-sensitive access to pictures and text located in the museum database. We are also currently creating a learning tool through which the learner can retrieve context-sensitive information about any object and its usage via RFID tags. Rather than mere information retrieval, this tool will also allow recording of related

evidence and posting comments in the form of text, pictures, voice and video. Additionally, in the near future we will release an easy-to-use editor tool for curators to create and edit content for LieksaMyst. CHARACTERISTICS OF PERVASIVE LEARNING ENVIRONMENTS

Currently there is no learning model established and tested to support the design and construction of PLEs (Laine and Joy, 2008). As the first step towards building such a model we have derived a set of characteristics based on inherently constructivist principles of situated learning, authentic learning, contextual learning, group-based learning, exploratory learning, problem-based learning and museum learning. We chose a hybrid approach as none of the existing models would alone suit PLEs for museums. Table 1 presents these characteristics together with references to supporting literature, rationale for inclusion, and analysis on how each characteristic has been implemented in the LieksaMyst PLE. The characteristics (15) have been organized into five categories: User profile and perspectives; Interaction and collaboration; Ownership; Authenticity and relevance; and Support and assessment.

Characteristic

Supporting literature

Rationale

In LieksaMyst

User profile and perspectives Multiple roles, Hall and Bannon, 2005; Kelly, perspectives 2002; Herrington and Oliver, 2000; and skill levels Herrington et al., 2003; Thomas, 2005; Falk and Dierking, 2000; Johnson and Quin, 2004; Csikszentmihalyi and Hermanson, 1995

In order to support visitors of various backgrounds, skills and interests, the PLE should provide access to various roles, perspectives and skill levels in an adaptive manner.

Multiple roles are provided through various fictional character and activities performed with them. As each fictional character can have several stories to share, it is possible to create a hierarchy of skill levels. Alternative learning tools offer possibilities for those who do not enjoy gaming.

Consideration Scanlon et al., 2005; Kelly, 2002; of background, Falk and Dierking, 2000; Schmidt, prior 1983; Sharples, 2003 knowledge and experiences

Prior knowledge and experiences should be taken into account in learning activities. For example, a first-time visitor has different needs to a regular customer who visits the museum frequently.

Learners' prior experiences are currently considered through free choice of characters and stories that are based on feedback from design workshops. We are also planning to include personal profiles for learners who visit the museum often.

Interaction and collaboration Social negotiation and collaboration

Scanlon et al., 2005; Hall and Sharing experiences and facing Learners have the possibility to play Bennon, 2005; Kelly, 2002; challenges together facilitate together using the same device or Csikszentmihalyi and Hermanson, effective learning. interact with each other in a shared 1995; Herrington and Oliver, 2000; physical space. More collaborative Herrington et al., 2003; Mims, 2003; activities are under development. Falk and Dierking, 2000; Savery and Duffy, 1994; Thomas, 2005; Sharples, 2003; Hein, 1990

Multimodal Mims, 2003; Johnson and Quin, By exploring the environment exploration of 2004; Csikszentmihalyi and through various senses the visitor the Hermanson, 1995 becomes more attached to it. environment and objects

The game encourages the learner to explore the environment through storytelling and various activities. Both visual and aural modalities are currently used. Some players also reported “old scent”.

Ownership Ownership of the learning process and outcome

Kelly, 2002; Csikszentmihalyi and Hermanson, 1995; Mims, 2003; Herrington et al., 2003; Falk and Dierking, 2000; Savery and Duffy, 1994; Thomas, 2005; Sharples, 2003; Johnson and Quin, 2004; Hawkey, 2001; Hein, 1990

Ownership affects directly to motivation to learn. Furthermore, having control over one's own learning process is necessary for effective learning.

The learner is in control of the story in terms of pace and choices made (story branches). Learning paths and any recorded “evidence” are owned by the learner.

Ownership of Scanlon et al., 2005; Thomas, 2005; In addition to increased motivation, the Sharples, 2000; owning the technology has direct technology consequences on the ability to use the technology effectively.

Although in the testing phase, a set of the museum's mobile devices are used. In the future the learners can use their own devices as the mobile technology is general and portable.

Authenticity and relevance Authentic context

Herrington and Oliver, 2000; Solving real world challenges cannot Schmidt, 1983; Grabinger et al., be taught effectively in an 1997; Falk and Dierking, 2000 unauthentic setting. Authentic context is also important for deep immersion of the learner.

Exhibits in Museum of Pielinen are in a very authentic state due to local preservation policy. The look and even the smell of old objects seem authentic.

Authentic activities that have relevance to the real world

Scanlon et al., 2005; Kelly, 2002; Herrington and Oliver, 2000; Herrington et al., 2003; Mims, 2003; Savery and Duffy, 1994

Activities have been designed to be authentic with curators who possess expert knowledge on local history and old traditions. Real world relevance stems from the relation of the activities to the modern life of the visitors.

Connecting learning activities to the real world is an important part of making the meaning of concepts. Without real world relevance, the concepts remain abstract.

Compelling Hall and Bennon, 2005; Falk and The PLE should employ a Stories are compelling and interaction narrative to Dierking, 2000 compelling narrative that helps the with authentic characters helps in the facilitate visitor to immerse quickly into the immersion process. immersion authentic context. Gained Herrington et al., 2003; Mims, 2003; experiences Sheppard, 2001; Hooper-Greenhill integrated et al., 2003 and applied across different subject areas Personal relevance

Knowledge can and should be transferred across disciplines. The PLE should allow generalization and linkage of the knowledge to other contexts and subject areas.

Due to the nature of the museum (living museum), the knowledge can be easily transferred to other “living contexts” to be reflected and compared. Furthermore, stories may have intersecting aspects for making connections.

Scanlon et al., 2005; Mims, 2003; Learning activities in the PLE should Grabinger et al., 1997; Thomas, have personal relevance, so the 2005; Falk and Dierking, 2000 learner is able to construct a personal meaning of a concept.

Learners can relate learning activities performed with authentic characters to equivalent activities in their own lives, hence the personal relevance.

Support and assessment Scaffolding techniques

Hall and Bannon, 2005; Kelly, 2002; Csikszentmihalyi and Hermanson, 1995; Herrington and Oliver, 2000; Mims, 2003; Savery and Duffy, 1994

Support should be available when the learner needs it the most, and it should be faded out when the learner can face the challenges themselves.

Scaffolding is something that could be improved. Currently, authentic characters and guides give hints and suggestions to the learner but the learner's skill level is not measured so as to adapt the given help.

Support for Rennie and McClafferty, 1995; reflection Kelly, 2002; Herrington and Oliver, 2000; Herrington et al., 2003; Falk and Dierking, 2000; Savery and Duffy, 1994; Johnson and Quin, 2004

The PLE should offer possibilities for reflection during and after the museum visit. During reflection new knowledge is linked to existing mental models and prepared for future linkages.

Currently post-reflection activities are not explicitly supported but in the near future a website will be launched which presents stored learning experiences for further reflection and commenting.

Immediate feedback

Csikszentmihalyi and Hermanson, Immediate feedback is necessary to The learner is provided with immediate, 1995 promote reflection and maintain choice-dependent feedback after each intrinsic motivation. activity.

Integrated, Herrington and Oliver, authentic Herrington et al., 2003 assessment if applicable

2000; Sometimes assessment is wanted. In such cases the PLE should offer a possibility to perform assessment integral to the learning process.

The application is not being used for assessment purposes as we aim to raise intrinsic motivation rather than extrinsic. However, there is a currently disabled feature that counts points for players according to their performance.

Table 1. Characteristics of pervasive learning environments for museums EVALUATION AND DISCUSSION

As evaluation we present part of the results of experiments conducted in the Pielinen Museum during NovemberDecember 2008. Participants used only the role-playing part of the game – the database discovery tool was not part of the agenda although some players voluntarily explored it. These results show the first impressions and attitudes of the players, and represent only a subset of the entire study.

Test scenario

In total 49 test participants were included in this test. Participants were of various nationalities: Finnish(31), Polish(4), Korean(4), Nepalese(2), Russian(2), German(1), Chinese(1), Pakistani(1), Latvian(1), Czech(1) and South African(1). The game had two language versions (Finnish and English). Testing was conducted in the Virsuvaara house from the 19 th century which is one of the largest buildings in the museum. For the experiment we designed a two-part questionnaire – the first part to be filled before playing (demographics, previous mobile usage, attitudes, media preferences) and the second part after playing (after-game experiences, perceptions). Observations of the participants were also recorded. We ran four test scenarios with different groups: (1) a group of local children from 7th grade; (2) a group of local senior teachers; (3) a group of foreign exchange students; and (4) museum staff and visitors from South Korea. The first three groups were selected to represent different ages and cultural backgrounds, and the last group was used as to receive the museum staff's perceptions as well as more international perspectives. Before participants were taken to the test location, they were given a short presentation of the museum and of our educational technology research in general. After the presentation, the first part of the questionnaire was filled and then followed the actual game play in Virsuvaara. The amount of time spent for playing varied from approximately 15 minutes to 45 minutes, and participants were either playing individually, in groups of two (most cases), or in groups of three. After the game play the participants were asked to fill in the second part of the questionnaire. Nokia N95 and N80 phone models were used in tests. For the first test group (local 7th graders) we (accidentally) did not remove object codes from the user interface as we used these codes for internal testing previously, hence locating objects was not a challenge for the participants. This was the major complaint heard in the questionnaires of the first group. Some players in other groups also noticed that the codes are in sequential order, thus guessing the next code was easy. Evaluation results and discussion

The average ages of males (49%) and females (51%) were 28.73 and 30.28, respectively. All test participants owned mobile devices, even the school children, thus penetration of wireless communications amongst them was very high. In the pre-test questionnaire we asked for participants' perceptions of museums in general. We compared these answers with the answers to a question which measured participants' perceptions towards the test day experience in Pielinen museum in the post-test questionnaire. In the pre-test questionnaire the statement was “In general, I think museums are:” and the answer options with respective results were: Boring (4.1%), Interesting (75.5%), Exciting (20.4%), Unexciting (2%), I have no opinion about them (6.1%), and Other (4.1%). The two participants who answered Other gave further explanations as Interesting and Something. In the post-test questionnaire equivalent statement was: “I think today's visit at Pielinen Museum was:” and the answer options with respective results were: Boring (6.1%), Interesting (89.8%), and Other (14.3%). Other answers were amended only with positive comments such as “exciting”, “quite ok”, “interesting and enjoyable”, “wonderful” and “really, really nice and interesting”. Those who answered Boring were from the first test group and it is possible that the presence of object codes in the user interface affected their experience. The sum of percentages in these answers exceeds 100% as some participants checked (against instructions) more than 1 option. We also asked in the post-test questionnaire if the participants would be happy to come back to Pielinen museum (91.8%) or if they would not be interested in coming back anymore (8.2%). From four who were not interested in coming back, three were from the first test group and one from the exchange student group. In order to discover participants' perception of LieksaMyst, we presented them two statements with scale Strongly Agree(SA) – Agree(A) - Disagree(D) - Strongly Disagree(SD) with weights 1, 2, 3 and 4, respectively. The questions and the respective answers are presented in Table 2 together with average and standard deviation values. Sum of the percentages on the first statement exceeds one hundred as one of the participants answered both D and SD. Statement

SA(1)

A(2)

D(3)

SD(4)

Avg

StDev

I think I would like to try the game again here next 44.9% summer.

44.9%

8.2%

4.1%

1.68

0.74

I think I would like to try the game also in another 40.8% museum.

53.1%

4.1%

2%

1.67

0.66

Table 2. Participants' perceptions of LieksaMyst Analysis of these tests regarding pre- and post-conceptions of museum visits and the game reveal that a good majority of the participants considered museums exciting and interesting places before trying the game. This could be due to fact that the participants came to test the application to the museum voluntarily, thus they might be representatives of the general population that prefer museums. After playing the game, the visitors' conception of the visit was very positive and a strong majority expressed their willingness to return back to the museum and try out the game again in Pielinen Museum and other locations as well. We consider this a strong indication that the game was well received by the audience. Finally, we asked the participants open questions about their likes and dislikes about the game. Table 3 presents the most common and interesting answers. Where applicable, we have included in parentheses references to the characteristics

presented in Table 1. This was an interesting result as we did not explicitly relate the questionnaire to the table of characteristics and these aspects were articulated by the participants. Usability issues were reported mostly by the group of senior teachers, thus there is a clear need to improve the game to fit all ages. These results suggest that we should improve image quality, screen size, add audio narration and provide other tools in addition to the game to explore the museum, as not everybody favors games. What did you like/enjoy about playing with Anna?

What did you dislike or find difficult about playing with Anna?

I could find out new things. Anna's intermediate comments were good, Male 13. (Immediate feedback)

Difficult to see the numbers [because of dark room], for me a little bit too much story telling, Male 26. (Multiple perspectives, skill levels)

I was feeling as if I was helping her and knowing things like making coffee, butter and know about the Finnish fire place, Male 24. (Authentic context, activities, and immersive story)

Pictures are not reproduced well enough due to small screen size of the client device, Male 55. Character keys are too small so I made errors unintentionally, Female 75.

The nice thing was that it was possible to get information Most of conversations are just text. I think sometimes it should be better through playing and action. It was nice to kind of discuss [if] the text will be explained on voice, Male 29. (Multimodal exploration) with Anna, Female 30. (Immersive story) Getting to know how Anna's typical day looked like, Female 23. (Authentic activities)

That the object numbers were already visible, Male 13.

It was a good simulation and I felt as if I was actually involved in the situation, Male 22. (Authentic context, activities and immersive story)

Before playing with Anna, it would have been better if I had an introductory guided tour about the house, Male 46. (Prior experiences)

I liked it when I had to start searching for objects, Female Anna's comments were sometimes annoying, Female 13. (Multiple 13. (Multimodal exploration) perspectives, skill levels) I could control the pace of game; I got to know how those Few questions which were quite interesting but being a non-Finnish I old objects were used in the 19th century, Female 21. found them difficult, Male 22. (Consideration of prior knowledge, (Learner's control) background)

Table 3. Participants' likes and dislikes regarding playing with Anna CONCLUSIONS

LieksaMyst PLE solves the problem of unmarked objects and information in the Pielinen Museum. At the same time, the museum visit becomes more exciting and engaging, thus having a potentially positive effect on visitors' attitudes towards museums. As the system was developed together with museum visitors and curators, the end result was highly compelling and met many visitors' expectations. Evaluation showed that the game part of LieksaMyst was well received and its story-based immersive game play would have potential in other museums as well. We concluded also that LieksaMyst met most of the characteristics of pervasive learning environments that we derived based on a large body of literature on existing constructivist learning models and theories. In the future we will refine LieksaMyst to fully conform to the characteristics of PLEs and test the technology in other locations as well, not limited to museums. We will continue further our efforts to develop a sound learning model for pervasive learning environments that will fill the existing gap and will therefore be the basis of future PLEs. ACKNOWLEDGMENTS

We would like to express our gratitude to curators of Pielinen Museum and those visitors who contributed to the design process of the LieksaMyst. Development of LieksaMyst has been partly financed by the National Board of Antiquities. REFERENCES

Black, G. The Engaging Museum - Developing Museums for Visitor Involvement. Routledge, 2005. Bitgood, S. School field trips. Visitor Behaviour, 4, 2, 1989. Csikszentmihalyi, M. and Hermanson, K. Intrinsic motivation in museums: why does one want to learn?, In Eilean Hooper-Greenhill (Ed.), The Educational Role of the Museum, 1995. Durbin, G. Learning from Amazon and eBay: user-generated material for museum web sites, in: Bearman, D and Trant, J (Eds), Museums and the Web 2004: Proceedings. Toronto: Archives & Museum Informatics, 2004. Falk, J.H. and Balling, J.D. Setting a neglected variable in science education: Investigations into outdoor field trips. Final Report SED 77-18913, Washington, DC: National Science Foundation, 1979.

Falk J.D. And Dierking, L.D. Learning from museums: Visitor experiences and the making of meaning. Walnut Creek, CA: AltaMira, 2000. Grabinger, S., Dunlap, J.C. and Duffield, J.A. Rich environments for active learning in action: Problem-based learning. ALT-J, 5(2), 1997. Hall, T. and Bannon, L. Designing ubiquitous computing to enhance children's interaction in museums. Proceedings of the 2005 conference on Interaction design and children, Boulder, Colorado, 2005, 62-69. Hawkey, R. Science beyond school: representation or re-presentation?, in: A. Loveless and V. Ellis (Eds), ICT, Pedagogy and the Curriculum: Subject to Change. London: Routledge/Falmer, 2001. Hein, H. The Exploratorium: The museum as laboratory. Washington, DC: Smithsonian Institution Press, 1990. Herrington, J. and Oliver, R. An Instructional Design Framework for Authentic Learning Environments. Educational Technology Research and Development, 48, 2000, 23-48. Herrington, J., Oliver, R. and Reeves, T. Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology, 19, 2003, 59-71. Heumer, G., Gommlich, F., Jung, B., and Müller, A. Via Mineralia - a pervasive museum exploration game. Pergames 2007, Salzburg, AT, 2007. Hooper-Greenhill, E., Dodd, J., Moussouri, T., Jones, C., Pickford, C., Herman, C,. Morrison, M., Vincent, J., and Toon, R. Measuring the outcomes and impact of learning in museums, archives and libraries. Learning Impact Research Project, Leicester: Research Centre for Museums and Galleries, https://lra.le.ac.uk/handle/2381/65, 2003. Hsi, S. and Fait, H. RFID enhances visitors' museum experience at the Exploratorium. Comm. of the ACM, 48, 9, 2005. Johnson, C. and Quin, M. Learning in science and discovery centres – appendix. In: Science Center Impact Study. Washington, DC: ASTC, 2004. Kelly, L. What is learning...and why do museums need to do something about it?, Presented at Why Learning? Seminar, Australian Museum/University of Technology, Sydney, Nov 22, 2002. Koran, J.J.Jr. and Koran, M.L. A proposed framework for exploring museum educational research. Journal of Museum Education, 11, 1986, 12-16. Laine, T.H. and Joy, M. Survey on Context-Aware Pervasive Learning Environment. International Journal of Interactive Mobile Learning (I-JIM), vol. 3, no. 1, 2008. Linn, M.C. Free-choice experiences: how do they help children learn? Journal of Research in Science Teaching, 14, 1980, 237-248. Lonsdale, P., Beale, R., and Byrne, W. Using context awareness to enhance visitor engagement in a gallery space. Proceedings of the HCI05 Conference on People and Computers XIX, 2005, 101-112. Lyytinen, K. and Yoo, Y. Issues and Challenges in Ubiquitous Computing. Communications of ACM, 45, 12, 2002. Mims, C. Authentic Learning: A Practical Introduction & Guide for Implementation. Meridian: A Middle School Computer Technologies Journal, 6(1), 2003. Ogata, H., Yin, C., and Yano, Y. JAMIOLAS: Supporting Japanese Mimicry and Onomatopoeia Learning with Sensors, Proc. of the Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, 2006. Proctor, N. and Burton, J. Tate Modern Multimedia Tour Pilots 2002-2003. Proceedings of MLearn 2003, London, 2003. Proctor, N. and Tellis, C. The state of the art in museum handhelds in 2003. In: D. Bearman and J. Trant (Editors), Museums and the Web 2003: Selected papers from an international conference. Toronto: Archives & Museum Informatics, 2003. Rennie, L.J. and McClafferty, T. Using visits to interactive science and technology centers, museums, aquaria, and zoos to promote science learning. Journal of Science Teacher Education, 4, 4, 1995, 175-185. Savery, J.R. and Duffy, T.M. Problem Based Learning: An Instructional Model and its Constructivist Framework. In Brent G. Wilson (Ed.) Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications, 1994. Scanlon, E., Jones, A. and Waycott, J. Mobile technologies: prospects for their use in learning in informal science settings. Journal of Interactive Media in Education, 25, 2005. Schmidt, H.G. Problem-based learning: rationale and description, Medical Education, 17, 1983. Sharples, M. The design of personal mobile technologies for lifelong learning. Computers & Education, 34, 2000, 177193.

Sharples, M. Disruptive devices: mobile technology for conversational learning. International Journal of Continuing Engineering Education and Lifelong Learning, 12, 5/6, 2003, 177-193. Sheppard, B. Museums, libraries and the 21 st century learner. Washington DC: Institute of Museum and Library Services, 2001. Syukur, E. and Loke, S.W. MHS Learning Services for Pervasive Campus Environment, Proc. of the Fifth International Workshop on Pervasive E-Learning, 2007. Syvänen, A., Beale, R., Sharples, M., Ahonen, M. and Lonsdale, M. Supporting Pervasive Learning Environments: Adaptability and Context Awareness in Mobile Learning, Proc. of the 2005 IEEE International Workshop on Wireless and Mobile Technologies in Education, 2005. Thomas, S. Pervasive, persuasive eLearning: modeling the pervasive learning space, Proceedings of IEEE PerCom 2005 Workshops, 2005. Wang, A.I., Sorensen, C-F., Brede, S., Servold, H., and Gimre, S. Development of Location-Aware-Applications: The Nidaros framework. Mobile Information Systems II, Springer Boston, 191, 2005, 171-185. Wolins, I.S., Jensen, N., and Ulzheimer, R. Children's memories of museum field trips: A qualitative study. Journal of Museum Education, no. 13, 1992, 533-542.

Paper III

Laine, T.H., Vinni, M., Islas Sedano, C. & Joy, M. (2010). On designing a pervasive mobile learning platform, ALT-J Research in Learning Technology Journal, vol 18, no 1, pp. 3 - 17. Reprinted with permission, Copyright 2010 Routledge.

ALT-J, Research in Learning Technology Vol. 18, No. 1, March 2010, 3–17

On designing a pervasive mobile learning platform Teemu H. Lainea*, Mikko Vinnia, Carolina Islas Sedanoa and Mike Joyb a

School of Computing, Joensuu Campus, University of Eastern Finland, Finland; bDepartment of Computer Science, University of Warwick, UK

Taylor and Francis CALT_A_466269.sgm

(Received 15 January 2009; final version received 10 December 2009) Taylor ALT-J 10.1080/09687761003657606 0968-7769 Original 102010 18 Mr [email protected] 00000March TeemuLaine & Research Article Francis (print)/1741-1629 2010 in Learning Technology (online)

This article presents the features, design and architecture of the Myst pervasive game platform that has been applied in creating pervasive mobile learning games in various contexts such as science festivals and museums in Finland. Based on our experiences with the development, we draw a set of design principles for creating successfully a pervasive game platform that can be easily ported to various contexts. These principles advocate openness, flexibility, interaction models, connections to the outside world, and participatory design of the game content. In the evaluation part we present preliminary results of tests conducted in Finland at the SciFest 2008 festival in Joensuu and at the Museum of Technology in Helsinki. The results suggest that games built with the Myst platform are particularly suitable for children and young adults, and these games motivate players to interact with the environment and help to learn by discovering new things. The Myst platform has clearly potential for similar success in other environments due to easy portability and extensibility. Keywords: pervasive learning space; mobile learning; pervasive game; design principles; participatory design

Introduction Traditionally, computing devices have been confined to a specific location inside a clearly visible physical container. Recent developments in wireless technologies and miniaturisation of components have led to the emergence of pervasive computing devices that are embedded everywhere in the environments surrounding us. Most of these devices are interconnected, thus forming a single context-aware system. Examples of pervasive computing applications in everyday lives include smart buildings, smart homes, smart clothes, smart cars and even smart dust (Warneke et al. 2001). Sensors and mobile technologies form an essential part of pervasive computing systems as they together provide high mobility for users and contextual awareness for the system. Today, pervasive computing is a hot topic in many fields across various disciplines such as mobile computing, artificial intelligence, distributed and embedded systems, agent technologies, communication technologies and human–computer interaction. Mobile learning, or m-learning, is one of the application areas where pervasive computing has become popular and is currently being intensively researched. In this paper we consider m-learning as learning facilitated by mobile devices such as PDAs *Corresponding author. Email: [email protected] ISSN 0968-7769 print/ISSN 1741-1629 online © 2010 Association for Learning Technology DOI: 10.1080/09687761003657606 http://www.informaworld.com

4

T.H. Laine et al.

and mobile phones. The primary aim of m-learning is to provide the users with access to learning environment regardless of time and location. Networked mobile devices allow learners to perform cooperative learning tasks in groups. The emergence of embedded intelligence (e.g. smart tags, sensors) has brought forth new forms of learning that have roots in m-learning, namely pervasive learning (Hundebol and Helms 2006) and ubiquitous learning (Ogata and Yano 2004). Figure 1 illustrates the different types of learning in the domains of mobility and embeddedness. In pervasive learning the roles of intelligent environment and context sensitiveness are emphasised. The physical environment has a central role as it provides context for learning, content for learning, as well as system resources (Laine and Joy 2009). In contrast to ubiquitous learning, pervasive learning is less mobile and more connected to a specific context. Compared with other learning environments such as classrooms or field trips, pervasive learning spaces (PLSs) (also known as pervasive learning environments) provide personal interaction between the learners, the environment and the relevant educational content. Furthermore, in properly designed PLSs, learning materials are delivered in a correct format at the right place in right time. PLSs can be deployed not only in traditional learning contexts but also for example in corporate training settings, museums, exhibitions and tourist attractions. The Myst platform offers a way to deploy game-based PLSs quickly in different contexts. In this paper we present the features and technical details of the Myst platform. Based on our development experiences, we draw a set of design principles that can be used in designing pervasive mobile games and platforms to be re-used in multiple contexts. Additionally, we present preliminary results from questionnaire-based evaluations of two different games in two environments, namely SciFest 2008 science festival in Joensuu and Museum of Technology in Helsinki. Figure 1.Lyytinen Source: Types and of learning. Yoo (2002).

Figure 1. Types of learning. Source: Lyytinen and Yoo (2002).

ALT-J, Research in Learning Technology

5

Related work Before getting into the details of the Myst platform, we present a representative selection of different PLSs, followed by a discussion of their shortcomings and restrictions. We have intentionally omitted pervasive game environments that do not explicitly present educational agendas. Beaudin et al. (2007) present a pervasive microlearning system for learning a foreign language in a home environment. In microlearning, a learning task is divided into a series of very quick learning activities. The system has built-in context-awareness through hundreds of wireless sensors and switches embedded in furniture, objects and appliances within the environment. Through this embedded intelligence, the system is able to sense when an object is approached, touched or used. While the system is deeply integrated with the surrounding environment, it is not very portable and does not support group learning activities. Mobile devices are used only for delivering audio information and their other communication features are disabled, thus the interaction potential of those devices is unused. LORAMS (Link of RFID and Movies System) is a personal learning assistant that supports learners to share and reuse their learning experiences by linking movies and environmental objects (Ogata et al. 2007). Users of LORAMS are equipped with PDA devices coupled to radio-frequency identification (RFID) readers and have wireless local area network (WLAN) connectivity. The key idea of the system is that users record a video of a problem-solving process related to a specific object or phenomenon, and then share that video with other users. Shared videos can be ranked according to relevance, and this ranking information is used in the video search functionality. The system currently supports only a video format that may be proprietary and therefore possibly non-functional with other types of mobile devices. Furthermore, collaboration between users is limited to asynchronous video sharing. In their work, Lampe and Hinske (2007) present the Augmented Knight’s Castle, a pervasive computing playset that enriches children’s imaginary play by using background music, sound effects and verbal commentaries on toys that react to children’s play. Toys are equipped with audio playback, RFID readers, and a set of sensors such as light meters, accelerometers, and microphones. An RFID-enabled mobile device is in auxiliary role as it can be used to provoke further interaction and to enhance the play. While the Augmented Knight’s Castle has a rich set of features and interaction possibilities, it does not provide the true mobility that is often demanded by pervasive applications. Furthermore, the system does not provide any new multi-user features apart from normal children’s play. There have been also instances of pervasive (mobile) learning environments for museums and science exhibitions (for example, Heumer, Gommlich, and Müller 2007; Hsi and Fait 2005; Thom-Santelli et al. 2005). However, most of them are designed for a single purpose and location only, thus lacking the flexibility and portability that the Myst platform offers. The closest match to our work is perhaps the IPerG project (2008), which developed among other tools the multi-user publishing environment (MUPE) for creating multi-user mobile applications. The Myst platform builds on top of MUPE, thus dramatically reducing the amount of programming effort in development. The aforementioned PLSs are designed to be used within a confined space such as a building. Examples of open-air PLSs covering larger physical areas include Ambient Wood (Rogers et al. 2004), Environmental Detectives (Klopfer and Squire 2008) and

6

T.H. Laine et al.

Insectopia (Peitz, Saarenpää, and Björk 2007). These PLSs are more flexible in terms of physical context but they still lack the flexibility over content matter as they have been designed for a specific purpose only. From our research into the development of existing pervasive systems, such as the ones described above, we have observed that the flexibility of such systems to be used anywhere, anytime, and by anyone, is still problematic. Most pervasive systems lack several design aspects that would make them more portable and more interactive. Furthermore, the conceptualisation of pervasive systems should include reflection on how to support social dynamics, which might increase collaboration among learners. In order for a PLS to be usable in wider contexts it should have a flexible design, which would allow not only the addition of new features but also easy deployment to various different environments for various purposes. Open protocols and file types increase the openness and therefore flexibility of the system. In the following sections we present the Myst platform, which solves these problems through extensibility and portability.

The Myst platform The Myst platform (not to be confused with the commercial Myst game series) offers a way to deploy pervasive m-learning games fast and easily. The following sections describe the platform’s features, architecture and examples of current implementations. The terms learner and player are used interchangeably. So far we have successfully created and tested Myst-based pervasive mobile games at SciFest science festivals (2007–2009) in Joensuu (SciMyst) (Islas Sedano et al. 2007), the Museum of Technology in Helsinki (TekMyst), the Museum of Pielinen in Lieksa (LieksaMyst) (Laine et al. 2009), and high schools in South Africa (UFractions) (Turtiainen et al. 2009). Additionally, we have organised several shortterm demonstrations with this technology, including an introductory game, to the Educational Technology research group at University of Joensuu, a marketing event of ADE Oy (Animations, Designs and Effects)1 in Helsinki, and even at a professor’s home. Table 1 illustrates the differences between each of these games so as to provide the reader with a hint on the Myst platform’s scalability to various contexts and purposes. All in all, the Myst platform is highly flexible and portable virtually to any context offering knowledge to be discovered and shared. All of these games except UFractions are based on informal learning environments outside schools. This is because we believe that pervasive learning, where the context performs a major role, has the most potential in environments such as museums where every single object or phenomenon has a story to tell. UFractions could be categorised as ubiquitous or mobile learning as it does not have deep relation to the surrounding physical environment. We do not rule out the possibility of using Myst-based games as part of the curriculum (e.g. to learn mathematics or physics in the classroom), but currently there is no evidence to support this idea other than the UFractions evaluation (Turtiainen et al. 2009).

Platform features While designing the Myst platform, our core idea was to make a system that could be used in various locations with minimal customisation effort. Therefore the platform offers various game-like features that may or may not be used in applications


7

Table 1. Myst-based pervasive mobile games. Game

Genre

Purpose

Elements

SciMyst (2007–2009)

Scientific treasure hunt

Exploration of the SciFest science festival and learning about various fields of science

–Multiple-choice questions –Find-an-object tasks –Record impressions –Collaborative battle –Web site

ADEMyst (2007)

Promotional treasure hunt

Promotion of ADE’s products and services to their customers

–Multiple choice questions –Find-an-object tasks –Record impressions

TekMyst (2008)

Adventure game on simple machines

Exploration of the Museum of Technology and learning about simple machines

–Multiple-choice questions –Multiple skill levels –Find-an-object tasks –Record impressions –Collaborative battle –Web site

LieksaMyst (2008)

Storytelling adventure on past life in authentic context

Learn how people used to live by travelling back in time to meet characters from the past

–Interaction with past characters through story –Multiple-choice questions –Find-an-object tasks –Alternative story branches –Guest book

EdTechMyst (2008)

Treasure hunt on a research group

Demonstrate educational technology research group research at University of Joensuu

–Multiple-choice questions –Find-an-object tasks –Record impressions –Collaborative battle –Web site

UFractions (2009)

Storytelling mathematics adventure

Learn fractions and save leopards by solving challenges using fraction sticks

–Interaction with leopards through story –Usage of fraction sticks –Multiple-choice questions –Open-ended questions –Record evidence on fractions –Collaborative battle –Web site

8

T.H. Laine et al.

utilising the platform. These features reach several dimensions of game play through which both players and non-players can experience the surrounding environment. Firstly, the basic game play contains context-sensitive enigmas to be solved by players. These enigmas come with many flavours, ranging from text-based queries to ‘take-a-picture’ tasks in which the player must locate a certain object or phenomenon based on given description and read a tag attached to it. Tags can be, for example, two-dimensional barcodes or RFID tags. The order of enigmas can be pre-defined, randomised or chosen by the player. Correctly solved enigmas award a number of points to the player, whereas an unsuccessful solving attempt decreases the player’s total point score. There is no time limit in solving enigmas in this game mode. If a player needs help with solving an enigma, they can either: read additional hint about related object/phenomenon on the mobile device; ask for help from other players through the multiplayer help feature in the game – the person who provides help is awarded additional points; or ask for more information from a non-player character related to the enigma. Several players can also share one mobile device and play as a team. Examples of an enigma and related additional information from the SciMyst2008 game are presented in Figures 2a and 2b, respectively. The game area can be divided into several areas and each area has a unique set of enigmas related to it. Context awareness is implemented by smart tags (e.g. 2D barcode, RFID) or automatic positioning (e.g. Global Positioning System, WLAN positioning), which can be used to determine location of the player. Secondly, the Myst platform offers a game mode through which a player can record data through the mobile device’s camera, microphone and text input mechanisms. Impressions, as we call the recorded data, are automatically marked with the player’s name, a time stamp and the location at which the data were recorded. Enigma information can be also added to the impression if necessary. Each recorded impression awards the player a number of points that are added to player’s total score. All impressions are stored in a database and published on a dedicated web site in a gallery. The players can therefore refer to their recordings after the game Figure 2.

An enigma (a) and a related additional hint (b) in SciMyst2008.

Figure 2. An enigma (a) and a related additional hint (b) in SciMyst2008.


9

play has finished and non-players can also experience how others perceived the environment. Thirdly, the Myst platform has a mode that allows creation of story-based games where enigmas can be embedded inside the storyline. These stories have one or more narrators – either fictional characters or simply the telephone – and the player can interact with these characters through enigmas and story. The story-based game mode may or may not record player’s points. Story parts can be linked together and the player can be given a choice to select from various possible branches of the story. In addition to text, stories can contain images, sounds and video. Figure 3 presents excerpts from LieksaMyst game’s story including a multiple choice enigma and an enigma where the player has to locate an object to help Anna in her daily chores. Before finishing the game play, the player may enter the final battle in which he/ she has to solve as many enigmas as possible within a limited amount of time. Additionally, wrong answers will reduce the battle score, thus giving more wrong than correct answers will reduce their total points. The enigmas presented at this stage can be from any area, thus the battle mode works as a refreshing exercise. After finishing the battle, the game may show a digital questionnaire with simple multiple-choice and open questions. This is not part of the actual game play but useful from the research point of view. Finally, the Myst platform offers synchronous integration between the players’ performance and the game’s web site. For example, in SciMyst2008 the theme of the game was battle against ignorance. The web site presented statistics of ignorance points that accumulated at a regular rate during three days of the festival. At the same time, the total scores of the players were calculated and displayed in real time against the ignorance points. As a result, the game efforts of all the players were put together to ‘fight against ignorance’ and this battle could be followed in real time on the game web site. In addition to the interactive battle against ignorance results, the game web site also presented statistics of the performance of individual players including total points, enigma points, battle points, impression points and helping points. In each Myst-based game, the collaborative battle may have a different theme but the underlying technology is the same. Figure 3.

Conversation with Anna in LieksaMyst.

Figure 3. Conversation with Anna in LieksaMyst.

10

T.H. Laine et al.

Design principles Development of the Myst platform has been an iterative process that is still ongoing. The process has taught us important lessons on how to design a PLS platform successfully. It must be noted, however, that due to the rapid development of pervasive and mobile technologies, these principles are likely to evolve in the future. One of the most important factors to consider when designing a PLS is the flexibility of design. A flexible design is needed if the system is planned to be used in various contexts with different contents. There are several dimensions of variety that can be tackled with flexible design, including: ● ● ● ●

variety of contexts (e.g. physical locations), variety of content types (e.g. different media types), variety of players (age, gender, language, preferences), and variety of interactions between players and non-player characters.

Once the platform can flexibly support these aspects, different applications can be deployed to various environments without the need to redesign the architecture. As PLSs usually comprise several physically separated components, and communication is required between these components, it is important to use open protocols and portable languages so that individual components can be easily replaced and modified if needed. Openness also allows other developers to contribute to the system by developing extensions and new features. The Myst platform, is built on an open Java/XML-based MUPE platform that utilises the TCP/IP protocol in communication. When designing the network, it is important to consider what type of network serves best the system requirements. For example, in a geographically limited urban setting, a WLAN connection is sufficient, while more ubiquitous systems require 3G connectivity to keep players mobile. There is a trade-off between WLAN and 3G in terms of range and price; whereas WLAN has shorter range and no data transfer costs, 3G has larger coverage but fairly expensive data rates in many countries. As the Myst platform is based on the TCP/IP protocol, both WLAN and 3G can be used as long as client devices support them. Interaction models in learner–learner and learner–system interactions are essential when designing PLSs. It is relevant to take into account how to support social dynamics in pervasive systems to encourage collaboration among learners. One example of the development of social dynamics towards collaboration is an online game environment such as World of Warcraft. The design of the PLS should also take into account the interaction among players (learners) and non-player characters. Hence, it is important to keep in mind the different profiles of players as well as the appropriateness of interaction among players and non-player characters. For example, some non-player characters may not want to be disturbed if they are occupied with something. On the other hand, encouraging shy players to interact with each other is another challenge that remains yet to be resolved. In order to support the experience and reflection after using the PLS, interaction with the outside world should be considered. Providing the learners with access to learning data after the learning scenario has ended will extend the learning process to locations beyond the actual learning arena. Interaction with the outside world can be implemented in diverse ways, but the easiest and probably the most accessible way is


11

to create a web site for the learning environment through which non-player characters as well as learners can access the stored data. Finally, one important thing to remember when designing a PLS is to design with the stakeholders (as opposed to designing for the stakeholders). In the case of a museum, for example, stakeholders comprises museum visitors, curators, and other content matter experts. In the design process of the Myst platform and games based on it, we have used participatory design methodology with the end-users and frequent design workshops with content matter experts to come up with a finely-tuned game experience for every each context. A word of caution is useful here though: creating motivating and effective content takes a significant amount of time. Architecture Similar to many other PLSs (Laine and Joy 2009), the Myst platform is based on a client-server architecture through Nokia’s MUPE system (Nokia Corporation 2008). The fundamental idea behind MUPE is that the server pushes content to the client in XML (eXtensible Markup Language) format, and the client renders the XML to show the corresponding user interface screen on the mobile device. The functionalities of the MUPE client can be extended easily by developing plug-ins. MUPE was chosen as the platform due to its portability and ease of deployment as most of the updates are performed on the server side. As shown in Figure 4, the high-level architecture of the Myst platform can be divided into four distinct interconnected parts – server, clients, pervasive environment, and off-site extension – connected by a wireless LAN or a 3G network, for instance. In the following sections we will describe each of these components in detail. Figure 4.

High-level architecture of the Myst platform.

Server The MUPE server is written in Java and uses an XML-based language to deliver a graphical user interface to client devices running the counterpart MUPE client. A

Figure 4. High-level architecture of the Myst platform.

12

T.H. Laine et al.

developer can extend the server by creating their own classes and XML sheets, which provide the actual game dynamics and content. We extended the MUPE server by several modules that provide the necessary data structures, rules, content and tools for pervasive Myst games. The Game Engine coordinates the various software components and also provides a set of game rules (e.g. time constraints and point management) that can be easily modified to fit a specific setting. The Content Data Module is responsible for storing enigmas, location data and other content related to the game play. All content of the game is stored in XML format, and Figure 5 presents examples of an enigma and a location. The game content can be localised to a new language just by translating content of the appropriate XML and resource files – this is a great advantage in international settings. The User Data module keeps a record of each player’s performance by recording attempted enigmas, points, impressions, questionnaire results, and any other activities that a player might do. When the game is over, the module automatically stores player’s data into a database using the Off-Site Extension component via a database connections module. Each player’s data remains active as long as the server is running and the player has not explicitly finished playing. This means that if the mobile device for some reason crashes, the game can be continued from the point at which the game was before the crash. This is an advantage as mobile devices and networks are still fairly unstable. The User Interface module consists of XML files that represent different user interface screens on mobile devices. Some of these XML files have dynamic calls to some other modules (such as User Data and Game Engine), which are executed on the client when the sheets are parsed or when a certain operation is performed by the player. Similarly, the server can dynamically change the contents of client–user interfaces. This is one of the strengths of MUPE as it allows flexible manipulation and method invocation on both sides. The basic outlook of the user interface can be changed in the Myst platform by merely creating new background graphics. Other user interface tweaking requires XML editing. The Questionnaire module is in charge of managing questionnaire questions that are presented to the players at the end of the game session. The intention of the questionnaire module is to aid researchers in the data collection process. Questions are presented in XML format and parsed into Java classes when the server is started. The module supports currently multiple-choice and open questions. Figure 5.

XML representations of an enigma (a) and a location (b).

Figure 5. XML representations of an enigma (a) and a location (b).


13

The Tools module contains classes for reading and writing files, parsing XML and managing language resources. The DB Connections module is responsible for uploading impressions, players’ points and questionnaire results to a MySQL database on the Off-site extension component. The module runs in a separate thread submitting queued jobs to the target database.

Clients During the development of the Myst platform we made some changes to the original MUPE client in order to optimise the memory consumption and modify the appearance. Furthermore, we created plug-ins for two-dimensional barcode recognition and near-field communication tag reading in order to support recognition of tagged objects. The client can in theory run on any J2ME-enabled mobile device having a decent amount of processing power and memory. However, we have tested the implemented PLS’s only on Nokia N80 and N95 devices.

Pervasive environment The game environment has several roles in pervasive games; it contains the game content, it provides a context for the game, and it is also a system resource (Laine and Joy 2009). In games developed on the Myst platform, the environment provides the information from which enigmas are built. This information can be in various formats and it can be represented by an object or a phenomenon. The Myst platform also allows players to capture the environment’s information for later reference through the impression recording feature. The game environment is tagged with barcodes or RFID tags through which the system can achieve context awareness. The flexible architecture of the system allows easy deployment of other sensor devices (such as temperature and pressure gauges, accelerometers, light and infrared meters) that can then be used to provide even richer information of the context. Finally, auxiliary devices such as microphones, speakers and (touch)screens embedded in the environment can be used to provide additional I/O capabilities to enrich the communication for the players.

Off-site extension The main aim of the Off-site extension is to provide an overview of the actual game play to people located outside the festival arena. Furthermore, it provides the actual players with a ‘rear-view mirror’ for looking back to their game results and memories (recorded impressions) after finishing the game. The extension is based on a web server supported by PHP and the MySQL database. The Off-site extension stores players’ points, impressions and questionnaires in database tables. The web site has usually several sections including, but not limited to, the introductory page, the collaborative battle page, individual scores, and the impression gallery. The collaborative battle page constantly updates and refreshes the status of game using statistical figures and a pie chart. The impression gallery is based on the Pixelpost photograph gallery software, which allows browsing, commenting and management of the images.

14

T.H. Laine et al.

Evaluation During the SciFest 2008 event (March 2008) and the first TekMyst test week (August 2008) we conducted surveys with three phases. First, players filled in a pre-game questionnaire that recorded demographic information as well as the players’ previous experience in using mobile phones and mobile games. We presented the players with a short introduction to the respective game (SciMyst2008 or TekMyst) and showed them how to read 2D barcodes with a telephone. Then we gave the players (some individually, others in groups of two or three) telephones and area maps, and told them to play freely in all areas as long as they wanted. After returning back, we asked the players to fill in a short questionnaire on the mobile device, and another paper questionnaire having questions unsuitable for answering on a mobile device. In this section, we present the results of the mobile questionnaire from SciMyst2008 and TekMyst, and related part of the paper questionnaire of TekMyst. Questionnaires were chosen as a tool because performing more qualitative data collection (e.g. interviewing) would have been too much for the available resources. Evaluation of TekMyst also involved observations done by researchers and museum curators, but this paper focuses on the questionnaire data. For SciMyst2008, anyone visiting the SciFest festival could participate in the test. In total, 50 players completed the first part of the paper questionnaire. The age range was between five and 55 years, the mean being between 16 and 20 years old, and 44% of the players were female while 56% were male. Some of the players had to finish the game early due to reasons external to the game (such as the school bus leaving), and could not therefore fill in the second part of the paper questionnaire. As a result, only 36 players answered all three questionnaires, and four answered the first and third questionnaires only. Table 2, which is based on the responses to the mobile questionnaire (n = 36), presents the statements and answer percentages for each alternative answer – strongly agree (SA), agree (A), disagree (D), strongly disagree (SD) – as well as some basic statistical information. For average and standard deviation calculations we assigned value SA as four points, A as three points, D as two points and SD as one point. Questionnaires were presented either in English or Finnish depending on the player’s preference. In the TekMyst evaluation at the Museum of Technology in Helsinki, the players were mostly pre-selected school children classes aged between 11 and 16. However, Table 2. Results of the mobile questionnaire for SciMyst2008.

Statement 1. Interaction with the environment was exciting 2. Game helped me to discover new things 3. I want to play SciMyst with my own phone 4. I want to play SciMyst at other locations 5. I liked solving enigmas 6. I liked recording impressions of the festival

Standard SA (%) A (%) D (%) SD (%) Average deviation 40.00

51.43

2.86

5.71

3.35

0.75

47.22

33.33

16.67

2.78

3.23

0.81

44.44

19.44

27.78

8.33

3.05

1.02

41.67

30.56

19.44

8.33

3.16

0.95

41.67 33.33

47.22 41.67

5.56 13.89

5.56 11.11

3.30 3.02

0.77 0.94


15

a few adults and families took also part in the evaluation during one open day. In total, the age range was from eight to 54 and average age was 15.95. Of all players, 64.8% were male and 35.2% female. The procedure was very similar to that of SciMyst2008 except for test group scheduling, which was more controlled in case of TekMyst; hence we received more complete answers. In total, 55 participants answered the mobile questionnaire thoroughly and the results are presented in Table 3. Table 4 presents results of parts of the paper questionnaire that relate to the questions of the mobile questionnaire. We received in total 111 complete answers to TekMyst’s paper questionnaire. The difference between this and the number of mobile questionnaire answers can be explained by the fact that the mobile questionnaire was answered often by a group of two or three players while the paper questionnaire was individual. In addition to the SA, A, D and SD answer options, a ‘didn’t use it’ (DUI) option was also available. DUI answers were left out from average and standard deviation calculations. These preliminary results suggest that the game was received well in both SciFest 2008 and the Museum of Technology by the players of many ages, most of whom were children or young adults. Interaction with the environment was deemed highly exciting and this shows that well-designed interaction with the environment is an essential part of pervasive learning spaces. As the responses to the second question suggest, both games helped a good majority of players to discover new things about exhibitions in both events. Although many players showed their interest in trying the games with their own telephones, a significant number disagreed. This may be because as the players were mostly children and youngsters, their personal mobile devices may have required features to play such games. However, most players were eager to try the games in other locations; this is a clear indication that the Myst platform could have potential Table 3. Results of the mobile questionnaire for TekMyst. Standard SA (%) A (%) D (%) SD (%) Average deviation

Statement 1. Interaction with the environment was exciting 2. Game helped me to discover new things 3. I want to play TekMyst with my own phone 4. I want to play TekMyst at other locations 5. Answering this questionnaire with a phone was easy

43.64

41.82

7.27

7.27

3.22

0.88

43.64

47.27

3.64

5.45

3.29

0.79

34.55

30.91

21.82

10.91

2.87

1.04

50.91

29.09

10.91

9.09

3.22

0.98

41.67

47.22

5.56

5.56

3.24

0.82

Table 4. Results of the related parts of the paper questionnaire for TekMyst.

Statement 6. I liked solving enigmas 7. I liked recording impressions of the festival 8. I liked the final battle 9. I liked helping others

Standard SA (%) A (%) D (%) SD (%) DUI (%) Average deviation 45.95 27.93

36.04 46.85

9.91 9.91

5.41 6.31

2.7 9.01

3.26 3.06

0.86 0.83

28.83 12.61

29.73 11.71 29.73 12.61

7.21 9.01

22.52 36.04

3.03 2.72

0.95 0.94

16

T.H. Laine et al.

for other arenas than museums and science exhibitions. In the fifth and sixth questions of SciMyst2008 we measured how the players perceived the two core features of the game – enigma-solving and impression-recording. Same aspects were measured for TekMyst in the paper questionnaire. Particularly, enigma-solving gained much positive attention – and the lower result in impression-recording might be only because not all players tried the feature. In the case of TekMyst, battle and peer-helping features were also somewhat appreciated although a significant part of players did not use them. Additionally, in the TekMyst evaluation we asked the players whether filling the questionnaire on the mobile phone was easy, to see whether we should keep doing it in the future. The results suggest that we indeed should. Overall results of the mobile questionnaire suggest that games built with the Myst platform are suitable for children and young adults and these games motivate players to interact with the environment and help to learn by discovering new things. The Myst platform has clearly potential for similar success in other environments due to easy portability and extensibility.

Conclusions and future work In this paper, we have presented the features, architecture and design principles of the Myst platform for pervasive mobile games. The reviewed existing systems lack some or all of the features of the Myst platform: portability, extensibility, open platform support, high interactivity, web-based access and integration with the world outside the playing environment. Based on shortcomings of prior work and our experiences with the development of the Myst platform, we drew up a set of design principles that should be considered when building flexible PLS solutions. In the evaluation of SciMyst2008 and TekMyst, two games implemented on the Myst platform, preliminary results suggest that both games suit children and young adults particularly well; however, the results do not invalidate their effectiveness for older players. Furthermore, the players found interaction with the environment and solving enigmas particularly exciting, and they expressed their interest in playing similar games in other locations as well. In future, we will continue to develop the Myst platform and apply it in new contexts such as forests and other tourist attractions, in rural and urban locations. The array of existing Myst game genres allows the flexibility to choose from various approaches (e.g. competitive versus non-competitive, story-based versus actionoriented, simulation versus adventure). Naturally we do not rule out the possibility of adopting new genres, such as strategy, in the future. Another strand of future development is to integrate the Myst platform with a mobile sensor gateway, which is currently under development. The aim of the mobile sensor gateway is to use a mobile device to gather and analyse data from a wireless sensor network, in order to increase context-sensitivity and utilisation of the properties and status of the physical environment. Yet another future goal is to develop an editor component that could be used by subject-matter experts or teachers to develop new Myst-based games and content for them. Currently we have an editor component for the LieksaMyst game, but it has been built for the use of museum curators. By adapting its core functionality, we might be able to create a general editor that could be applied to many kinds of Myst-based games. Finally, while this paper presented the technical and design aspects of the Myst platform, we are currently constructing a new model (Laine et al. 2009) for pervasive learning spaces, and in the future the PLSs implemented on the Myst platform will be evaluated against the model.


17

Notes 1. ADE Oy is a company based in Turku, Finland.

References Beaudin, J.S., S.S. Intille, Mun E. guia Tapia, R. Rockinson, and M.E. Morris. 2007. Contextsensitive microlearning of foreign language vocabulary on a mobile device. Paper presented at the European Conference on Ambient Intelligence, November, in Darmstadt, Germany. Heumer, G., F. GommLich, and A. Müller. 2007. Via Mineralia – a pervasive museum exploration game. Paper presented at the PerGames Conference, June 11–12, in Salzburg, Austria. Hsi, S., and H. Fait. 2005. RFID enhances visitors’ museum experience at the exploratorium. Communications of the ACM 46, no. 9: 60–5. Hundebol, J., and N.H. Helms. 2006. Pervasive learning environments. Paper presented at the Society for Information Technology & Teacher Education International Conference (AACE), March 20–24, in Orlando, FL, USA. IPerG Project. 2008. Project website. http://www.pervasive-gaming.org. Islas Sedano, C., T.H. Laine, M. Vinni, and E. Sutinen. 2007. Where is the answer? The importance of curiosity in pervasive mobile games. In Proceedings of the conference on Future Play, 46–53. New York, NY: ACM Press. Klopfer, E., and K. Squire. 2008. Environmental detectives – The development of an augmented reality platform for environmental simulations. Educational Technology Research and Development 25, no. 2: 203–28. Laine, T.H., and M. Joy. 2009. Survey on context-aware pervasive learning environments. International Journal of Interactive Mobile Technologies (I-JIM) 3, no. 1: 70–6. Laine, T.H., M. Joy, C. Islas Sedano, and M. Vinni. 2009. Characteristics of pervasive learning environments in museum contexts. Paper presented at the MLearn Conference, October 26–30, in Orlando, FL, USA. Lampe, M., and S. Hinske. 2007. Integrating interactive learning experiences into augmented toy environments. Paper presented at the Pervasive Learning Workshop at the Pervasive Conference, May 13–16, in Toronto, Ontario, Canada. Lyytinen, K., and Y. Yoo. 2002. Issues and challenges in ubiquitous computing. Communications of ACM 45, no. 12: 62–5. Nokia Corporation. 2008. Multi-user publishing environment (MUPE). http://www.mupe.net/. Ogata, H., Y. Matsuka, M. El-Bishouty, and Y. Yano. 2007. LORAMS: Capturing sharing and reusing experiences by linking physical objects and videos. Paper presented at the Pervasive Learning Workshop at the Pervasive Conference, May 13, in Toronto, Ontario, Canada. Ogata, H., and Y. Yano. 2004. Context-aware support for computer-supported ubiquitous learning. Paper presented at the IEEE International Workshop on Wireless and Mobile Technologies in Education, March 23–25, in Taoyuan, Taiwan. Peitz, J., H. Saarenpää, and S. Björk. 2007. Insectopia: Exploring pervasive games through technology already pervasively available. Paper presented at the International Conference on Advances in Computer Entertainment Technology, June 13–15, in Salzburg, Austria. Rogers, Y., S. Price, G. Fitzpatrick, R. Fleck, E. Harris, H. Smith, C. Randell, et al. 2004. Ambient wood: Designing new forms of digital augmentation for learning outdoors. Paper presented at the Conference on Interaction Design and Children: Building A Community, June 1–3, in College Park, MD, USA. Thom-Santelli, J., C. Toma, K. Boehner, and G. Gay. 2005. Beyond just the facts: Museum detective guides. Paper presented at the International Workshop of Re-Thinking Technology in Museums: Towards a New Understanding of People’s Experience in Museums, June 29–30, in Limerick, Ireland. Turtiainen, E., S. Blignaut, C. Els, T.H. Laine, and E. Sutinen. 2009. Story-based UFractions mobile game in South Africa: Contextualization process and multidimensional playing experiences. Paper presented at the Second Workshop of Story Telling and Educational Games (STEG 2009), August 21, in Aachen, Germany. Warneke, B., M. Last, B. Liebowitz, and K.S.J. Pister. 2001. Smart dust: Communicating with a cubic-millimeter. Computer 34, no. 1: 44–51.

Paper IV

Laine, T.H., Islas Sedano, C., Sutinen, E. & Joy, M. (2010). Viable and portable architecture for pervasive learning spaces, Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia, Limassol, Cyprus, pp. 1 - 10. Reprinted with permission, Copyright 2010 ACM Press.

Viable and Portable Architecture for Pervasive Learning Spaces Teemu H. Laine

⇤

School of Computing University of Eastern Finland P.O. Box 111 80101 Joensuu, Finland

[email protected]

Carolina Islas Sedano


[email protected] Mike Joy

Erkki Sutinen


[email protected]

University of Warwick Coventry CV4 7AL, UK

[email protected] ABSTRACT

1. INTRODUCTION

A Pervasive Learning Space (PLS) uses context-awareness to link a virtual world with real world objects. We define viability as the extent to which a given PLS can be adapted to di↵erent purposes, and portability to be the extent to which a given PLS can be transferred to a new physical context. Heroes of Koskenniska is a game-based PLS combining mobile technology and a wireless sensor network in a forest context to raise the environmental awareness in a Biosphere Reserve in Finland. The game was built upon a screen-based architecture, and our analysis shows that it has higher portability and viability than a selection of related PLSs. The screen-based architecture is highly viable and portable because it is based on the model-view-controller division. Our preliminary observations indicate that the game helps to increase visitor count of the area and to promote interaction between visitors and nature.

A Biosphere Reserve is an international conservation designation given by UNESCO under its Programme on Man and the Biosphere (MAB)1 . The North Karelian Biosphere Reserve (NKBR), located in Eastern Finland, is surrounded by coniferous forests, hills, mires and lakes. Popular recreational activities at NKBR include hiking, fishing, hunting and camping, but the region has not o↵ered interactive tools to promote interaction between nature and the people. To establish a connection between people and nature in NKBR we explored the use of Pervasive Learning Spaces (PLSs) to facilitate environmental education in a forest context. In this paper we propose a screen-based architecture for PLSs that are viable and portable (Section 2.1). The proposed architecture was used to implement a game-based PLS Heroes of Koskenniska which combines mobile and sensor technologies with environmental education. The context of the game is the Koskenniska Mill and Inn Museum area located by the Shepherd’s Trail, a 37km hiking trail in NKBR. Heroes of Koskenniska is a role-playing game that is inspired by Kalevala, the Finnish national epic. The player is guided by Ukko, the High God, in the battle against Hiisi, the malicious and horrifying spirit that seeks to destroy the environment and the people. The aim of the game is to teach the learner about the environment and cultural history in a fun and exciting way while real-time sensor readings are used to provide contextual background data of the physical context. There are several learning applications which have been developed for environmental education (e.g. [17], [6], [5]) but they lack viability and portability from both the system and content developer’s perspectives. To overcome this deficiency, we developed Heroes of Koskenniska by using a screen-based architecture where each screen has a separate data structure (model ), rendering mechanism (view ) and controller. In this paper we first show the need for a PLS at NKBR. Then we present the Heroes of Koskenniska game followed by a description of the screen-based architecture and an analysis of the viability and portability of existing PLSs. We finally analyse the architecture’s advantages and

Categories and Subject Descriptors K.3.1.b [Computers and education]: Computer Uses in Education—CAI ; K.8.0 [General]: Games; J.9.a [Mobile applications]: Location-dependent and sensitive; D.2.13 [Reusable Software]: Reuse models;

Keywords pervasive learning, wireless sensor network, environmental education, model-view-controller, viability, portability, biosphere reserve ⇤Primary and corresponding author

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MUM’10, December 1-3, 2010 Limassol, Cyprus Copyright 2010 ACM 978-1-4503-0424-5/10/12 ...$10.00.

1

http://www.unesco.org/mab

transferred to a new physical context without adjusting the technical implementation.

Figure 1: Types of learning according to levels of mobility and embeddedness [11] disadvantages based on the experiences gained during the development and testing processes.

2.

BACKGROUND

2.1 Concept definitions Pervasive learning refers to type of learning where the real and virtual worlds are bridged for learning activities in a specific context. Pervasive learning di↵ers from ubiquitous learning in terms of mobility (Figure 1), and the systems which implement the pervasive learning paradigm are referred to as pervasive learning spaces (PLS, also pervasive learning environments). This means that whereas ubiquitous learning merely refers to “everywhere” learning, a pervasive learning experience varies according to place, time and the learner’s activity. PLSs make e↵ective use of the physical environment as the context for learning, content for learning and system resources [7]. We use the term spaces rather than environments so as not to confuse the acronym PLE with Personal Learning Environments. Examples of projects that qualify as PLSs are LieksaMyst [10], Environmental Detectives [6] and Insectopia [13]. A more detailed account on the di↵erences between pervasive and ubiquitous learning is available in [8]. PLSs are tightly connected to the surroundings, and the various assets of the context are used as learning resources. PLSs use some level of context-awareness to provide contextsensitive learning materials and activities [7]. The higher the level of context-awareness, the more blurred is the boundary between the real and virtual worlds as the system becomes aware of the learner’s status and can adjust its operations accordingly. The level of context-awareness also partly dictates how immersive the learning experience can become. Because a PLS is always created for a specific context and for a specific purpose, we recognise the need of a system architecture that supports flexible creation of PLSs for various contexts and purposes. Such an architecture can be measured through the aspects of viability and portability, defined as follows. • Viability: the extent to which a given PLS can be adapted to the requirements of new stakeholders or subject matter. • Portability: the extent to which a given PLS can be

If the architecture of a PLS allows flexible creation of new types of applications in the same physical context with minimal development e↵orts, then the viability is high, whereas if the PLS is suited only for a single purpose the viability is low. A PLS has low portability if deployment to another location requires significant changes to the underlying system (e.g. rules, concepts, equipment). High portability allows the PLS to be transfered between physical contexts with minimal changes to the original implementation. For example, GreenSweeper [5] (Section 6) has low viability because it has been designed for a single purpose only. However, its portability is high as it can be deployed in any urban context.

2.2

Rationale

A Biosphere Reserve is a site recognized under UNESCO’s Man and the Biosphere Programme, which innovates and demonstrates approaches to conservation and sustainable development. In 2009 there were 553 Biosphere Reserves in 107 countries around the world. One of the most important goals of research activities done in NKBR and other Biosphere Reserves around the world is to bridge the gap between nature and people. Traditionally this challenge has been tackled by hiking trails and other outdoor activities. A major challenge remains in how to facilitate the visitors’ and NKBR inhabitants’ understanding of nature. Information boards have been used, but their space, format and possibilities for interaction are limited. To make the natural environment more responsive and visits more engaging, the idea of combining mobile technologies with sensors in the environment was developed to provide a means to create an inspiring place for people and nature to interact. The lack of visitors in the Koskenniska Mill and Inn Museum located in NKBR has been recognised as another problem. According to the official statistics, there were only 300 registered visitors to the museum during summer 2008. The small number can be partly explained by the short yearly opening period of the museum (from late June to early August). With the help Heroes of Koskenniska, we aim to increase the visitor count and extend the period during which the visitors can learn about the museum. Although the game is not designed solely for the museum, it has some content related to the mill’s history and operation, hence potentially raising learners’ interest in the museum. One of the long-term goals for NKBR is to establish meaningful connections between various Biosphere Reserves. These connections not only relate to visitors’ learning experiences but also to research and training activities. For example, researchers working at di↵erent Biosphere Reserves could share research data and develop models and procedures that would in turn support the conservation and sustainable utilisation processes of the regions. Similarly, a learner at NKBR could interact together with a learner located in a Biosphere Reserve in South Africa, for example. To support such global networking of resources and people through pervasive technologies in Biosphere Reserves, the PLS should be flexible enough for fast deployment in varied contexts. While connections to other Biosphere Reserves are not topical at the time of writing this paper, it is an important long-term objective which influenced the decision to make the architecture viable and portable.

3.

HEROES OF KOSKENNISKA

Heroes of Koskenniska is a PLS combining mobile and sensor technologies in a natural context to provide the means to raise environmental awareness among visitors of the Koskenniska Mill and Inn Museum area in NKBR. Sensor readings provide background data for the game where the player traverses the forest and museum area while performing various types of tasks. The game was developed as a joint e↵ort of foresters, wireless technology experts, local historians and educational technologists. The story of the game, inspired by the Finnish epic Kalevala, is based on the battle between Ukko and Hiisi. Ukko seeks heroes to battle against Hiisi and those who are brave enough are guided by Ukko through various challenges and tasks around various themes. The story interweaves concepts such as the beginning of life, the afterlife, the meaning of time, energy and animals. Currently the game content is in Finnish and English but adding other languages is possible due to the multi-language support mechanism. The game area is divided into two parts — forest area and open area — as depicted in Figure 2. The open area encompasses the buildings as the inn, mill, smoke sauna and storage buildings. The forest area has di↵erent properties depending on the flora and fauna residing in each area. A small river runs next to the mill and flows into a nearby lake. Because of harsh weather conditions, the game is operable from May to October.

magic spots of level 3 span both areas. Each magic spot has a number of challenges which can be text-based multiple choice questions (with one or more correct answers), imagebased tasks where the player must pick a correct image from several possibilities, or special spot activities in which the player must perform physical activities such as building a bark boat and taking a picture of it. Each challenge can have any number of screens which introduce the player to the challenge before the actual challenge screen. Sample screens of various game activities are presented in Table 1. Table 1: Examples of Heroes of Koskenniska game activities

Entering the Afterlife magic spot. The player is asked to walk quietly so as to respect the dead trees. A text “Shhh! You are not going quietly enough!” with a sound e↵ect is shown as a reminder after a while.

An example of a multiple choice challenge – the player is asked to distinguish a big dog’s footprint from that of a wolf. At this magic spot there are animal prints carved into stone.

Figure 2: Map of the game featuring magic spots, sensor locations, and WLAN access points. The content of Heroes of Koskenniska was designed to be viewed with Nokia N95 phones which were deemed to be sufficiently reliable and durable for the given conditions, but the content should appear similar on any J2ME phone with the same screen resolution. The game has three levels ordered by increasing difficulty. Each level has three magic spots, each of which has a specific physical location and a theme. For example, the level 1 magic spots have themes Beginning of Life, Details of Life, and Afterlife. The player can freely choose the traversal order of magic spots within a level. The magic spots of levels 1 and 2 are clustered in the forest and open areas while the

An example of Spot Activity – Ukko asks the player to build a boat from the forest’s ingredients. Taken pictures (with players’ comments) are stored for reflection. To change the location (magic spot), the player must complete a transfer activity which may require them to perform

Figure 3: General architecture of the Heroes of Koskenniska various tasks such as walk silently, find a specific object and take a picture of it, or reach the next spot as fast as possible (timer). When the player arrives at a new spot, they must solve a riddle which ensures the right location of the player. Riddles are presented in information boards located at magic spots. The player must find a magic word from a board and enter it into the mobile device. At the end of each level Ukko takes the player to a battle against Hiisi. In the battle the player must connect partial phrases together to form meaningful sentences. The sentences are related to information previously acquired from the individual spots, as well as to the di↵erences in sensor readings between the spots. Hence, the player is expected to pay attention to previous tasks as well as sensor readings in order to perform well. Contextual sensor readings can be checked at almost any moment via the game menu, and Ukko reminds the player to do so frequently. There are two types of points in the game: health points and wisdom points. Health points (initially 100) are reduced by 10 after an incorrect answer, except in the battle against Hiisi where the penalty is 30. Completing a magic spot and a level (i.e. battle) successfully increases the player’s health by 10 and 50, respectively. For each correctly solved challenge the player is awarded six wisdom points and three points extra for each additional answer when there are more than one correct answer options. Finishing a level gives the player 100 wisdom points. If the player’s health reaches zero they die but they are immediately resurrected by Ukko and health is then restored back to the initial value of 100. What counts at the end is the number of wisdom points as well as attempts (lives) used in the game.

4. SCREEN-BASED ARCHITECTURE Many PLS projects concentrate on creating systems for a single context and/or purpose only (e.g. [3], [6], [5], [13]) and viability and portability remain, if present at all, as secondary objectives. When we begun the work with Heroes of Koskenniska in February 2009, we had limited resources available. Nevertheless, a decision was made that, rather than creating hastily something that would work only for this context and purpose, we would reserve enough time for designing a system that would cater not only for other Biosphere Reserves around the world but also other learning contexts where story-based PLSs would have potential. The screen-based approach to the architectural design was considered to be a promising solution as it would allow developers and later content creators to easily create new screens and reuse existing templates. The overall architecture is presented in Figure 3. Before having a closer look at the details, we briefly review the underlying MUPE platform.

4.1

MUPE (Multi-User Publishing Environment) platform

To develop the system rapidly, we sought to use an existing software platform. We chose MUPE (Multi-User Publishing Environment)[15] as it has been used successfully in several other educational mobile game projects (e.g. [10], [13], [14]). MUPE, developed as part of the iPerG project2 , is a client-server platform with the advantage that the server pushes all client content as XML, hence eliminating the need to install new software when connecting to a new MUPEbased service. Furthermore, restarting the mobile device 2

http://www.pervasive-gaming.org

while running the game does not reset the game status as all data is stored on the server. Client user interfaces are defined using the MUPE XML language and the server logic is written in standard Java. MUPE supports by default text, graphics, sound, video, GPS and camera features of the mobile device. Additionally, plugins have been developed for reading smart tags such as 2D barcodes and Near Field Communication (NFC). By default MUPE does not support external databases or wireless sensor networking so those features were implemented.

4.2 Architecture description The core idea behind the screen-based architecture (Figure 3) is to represent various combinations of screen content in a way that adding new combinations is trivial. The architecture defines a screen to consist of a model (Java class) and a view (MUPE XML sheet). Controller (server callbacks) is defined using XML. This separation fits well to the Model-View-Controller (MVC) design pattern, thus enabling flexible application development and code reuse. The model implements a content update mechanism which prepares a corresponding view (MUPE XML sheet) and returns it to the caller. User actions are processed by the controller, screen models are connected to screen managers through an interface, and screen managers in turn maintain the desired screen sequence (branching). This sequence is assigned by the content creator but the game can also let the learner choose where to go next (e.g. next magic spot). Screen managers implement an interface through which their parent components (such as magic spots) can conveniently request next or previous screens. The details of other components except the Content Manager are given in Section 5. Figure 4 illustrates the flow of control between components of the game. There can be any number of levels, each having an arbitrary number of magic spots. A special story section can be embedded at the beginning or at the end of each level, and a battle section may be defined at the end of a level. Magic spots may have any number and combination of multiple choice and spot activities, each of which can in turn have any number of screens. Transfer activity can be omitted, but in the current version of the architecture riddle activity is needed as it is the method of ensuring the correct location of the player. When all levels are passed successfully the game ends.

4.3 Screen composition Each screen can have any components allowed by the MUPE XML language including — but not limited to — text, item list, image, video, sound, links and menu. Additionally, screens can include information from data fields such as player name, sensor readings, health points, wisdom points, magic spot name and character name. Special tags can be used inside the text content of the screen so as to include information such as player’s name, remaining questions or value of a specific sensor at a given location. Screens are reusable in a way that their content can be updated by a corresponding model and one model can have alternative views. Table 2 lists samples of currently implemented screens together with descriptions. By using the 14 currently implemented screen types it is possible to create a complex and versatile game structure. Adding new screens to the architecture is simple for a developer, who must (i) create a Java class (model) to implement

Table 2: Sample screens from Heroes of Koskenniska Screen type and typical use Screen shot Text-Photo Sometimes a picture can tell a thousand words. For such situations the Text-Photo screen type is optimal as it combines an explanatory excerpt with an illustrative image. Character name is used to indicate the speaker.

Text-Multichoice-Photo The Text-Multichoice-Photo screen is useful for presenting challenges where answer options are images, thus requiring the player to explore the images closely. Each image can be zoomed to full screen size and the player must select the correct answer (image) to continue.

Text-Photo-Multichoice-Text This screen presents a picture and several related choices of which one is correct. Typically this kind of challenge expects the player to find the answer (or a clue to it) from the presented image, which in turn may relate to a real world object.

Text-Multichoice-Multiselect-Text This screen looks similar to TextMultichoice-Text but the selection mechanism allows selection of multiple answers at once. This screen type can be used to make multiple choice questions more challenging than if they would have only single selection mechanism.

Battle The Battle screen, as presented at the end of each level, allows the content developer to create pairs of statements that must be matched by the player. The high penalty value (30) encourages the player to put more e↵ort on finding correct pairs.

ers can view them later and so compete with each other if they wish. Captured elements, such as pictures, comments and sounds produced by the players, are also published for later reference and use.

5.2

Figure 4: Flow of control in the Heroes of Koskenniska the interface required for the screen, (ii) create a MUPE XML sheet to represent the screen layout (view) and implement any custom callbacks (controller), and (iii) add a row to the database table that lists the di↵erent screen types. Then a dedicated content editor tool can be used to add and modify screens of the newly created screen type. Ease of adding new screens and reusing existing screens makes the architecture viable. Screens may implement other server callbacks in addition to basic next and previous screen traversal operations. For example, a Text-Multichoice-Photo screen, instead of simply requesting the next screen upon answer selection, requests the server to check whether the selected answer was correct.

5.

IMPLEMENTATION ASPECTS

5.1 Player data and website integration Data for the players are managed by the Player Manager component (Figure 3) which keeps track of the players’ progress and collected information (e.g. pictures taken). The Player Manager hosts profiles of each player so as to accommodate player-specific data and preferences. It also validates the data of newly registered players and manages all the players’ statuses. When the player finishes the game or captures data (takes a picture or records sound), the Website Integration Manager (Figure 3) uploads the new data to the game website or stores them locally on the game server if no connection is available. The game results are published so that the play-

Database and resources

A MySQL database was used to store the game content and readings from the wireless sensor network. The database table design follows closely the data structures of the game, including levels, magic spots, multiple choice questions, spot activities, transfer activities, riddle activities, start and end stories, and battles. Additionally, sensor readings are stored in a separate table. The Content DB Manager (Figure 3) handles content retrieval from the content database, and the Sensor table is used by the Sensor Data Fetcher, Data Analyser and ESN Manager (see below). We also implemented a graphical content editor tool for creating and modifying game content stored in the database. The game server also includes a data loader component which is used to load the game contents to the server at startup, in order to optimise retrieval speed during play. Finally, we implemented an environmental sensor network analyser tool which monitors sensor readings and reports anomalies to the researchers by e-mail or log file. The Resource Manager (Figure 3) takes care of providing appropriate resources for each service request, including language-dependent data such as user interface constants, images and game content. When the player connects to the service, they choose a language preference and this choice is used to deliver appropriate resources during the game play.

5.3

Wireless networking

Wireless networking used in Heroes of Koskenniska consists of two parts — an Environmental Sensor Network (ESN) and a Wireless Local Area Network (WLAN). Figure 2 illustrates the distribution of wireless sensor nodes and WLAN access points in the game area. ESN is essentially a wireless sensor network (WSN) defined as a set of spatially distributed small autonomous devices (nodes) working together to solve at least one common application [16]. Each device can observe di↵erent parameters such as temperature, humidity, vehicular movement, pressure, soil makeup, noise level [1]. WSNs are used in different application areas from military to traffic to physiological monitoring. The WSN used for environmental monitoring is ESN and its components typically have the following characteristics: small size, low cost, robustness, low power, low maintenance, non-intrusive, and low pollution [4]. The ESN in Heroes of Koskenniska consists of nine networked (ZigBee) sensor nodes deployed in trees and on the ground. One of the sensor nodes is connected to the game server and acts as a sink node (id 0) for gathered sensor readings. Each sensor node is capable of sensing temperature, humidity, illumination and infra red values from the environment. The aggregation of captured data is established by a multi-hop routing mechanism where the nodes, when deployed, automatically form a routing scheme that channels all collected data to the sink node. It is also possible to send control messages from the game server to the nodes, if necessary. Each sensor node is connected to an external battery and the entire structure is protected by a plastic and (in some cases) a wooden case. On the game server (Figure 3), the ESN Manager retrieves

data from the sensor network through the sink node. Retrieved sensor readings are stored in the sensor database. The Sensor Data Fetcher retrieves and prepares the stored sensor data for use in the game. The Data Analyser automatically monitors and detects anomalies in the database (such as missing sensor readings or abnormal values). Alarm thresholds can be set and all detected anomalies are logged. As the MUPE platform requires network connectivity between server and clients, a WLAN with powerful antennas was built to cover the entire area and was chosen in order to minimise the usage costs of the system. For smooth operation, two wireless access points were bridged and distributed optimally to cover the game area. The access points were also protected against the weather.

6.

ANALYSIS OF RELATED WORK

In this section we analyse existing PLSs and platforms that have similar elements to Heroes of Koskenniska, in terms of type (educational, game), content, technology or context. Articles related to the target applications were reviewed and available technical information extracted for analysis. Some applications were available for further testing but we did not perform thorough usage tests as this would require the same level of access for each target application. Table 3 summarises the comparison of the applications and Heroes of Koskenniska in terms of architectural approach (as per available information), sensor usage, viability, portability, mobile device usage, application type, data collection (by the learner), content, location awareness and network access. Viability and portability were evaluated according to the definitions given in Section 2.1 and based on the information available of the analysed systems. One reason why Heroes of Koskenniska appears to be more portable and viable than other analysed systems is because its screen-based architecture is based on the MVC division, thus allowing flexible definition and reuse of screen templates. Ambient wood is a PLS which facilitates children’s learning about scientific enquiry and hypothesis testing in a forest area [17]. The system utilises RFID and GPS technologies for determining when a learner enters a hotspot where they can pick up information about the flora and fauna of different habitats. The learner is equipped with a PDA and a probe device for measuring sunlight and humidity. A card-based metaphor [17] is used to present information to learners as a deck of cards, where each card presents a piece of information. The idea of the card-based metaphor is similar to the screen-based approach presented in this paper but it is designed only for presenting information, thus restricting the interaction as well as knowledge construction capabilities. In Heroes of Koskenniska the content structure is more complex than a stack of ordered cards. Moles and Mini-Moles are PC and mobile applications respectively that can be used to create and deploy mobiledriven outdoor learning activities [12]. Students can first create content on PCs with Moles which contains all needed information and knowledge questions prepared by the educator. Content is then transferred onto PDAs or mobile phones (Mini-Moles) which are subsequently taken to the field where the students answer the questions by recording observations. After returning to the classroom, recorded data is transferred to the PCs for analysis and reflection. The system is open source, but it supports only specific types of questions and adding new types may require major

changes to the source code. Prepared content cannot be dynamically modified following context changes, and content loaded on the mobile device is static. Although this is a powerful platform for implementing school field trips, the static nature of the systems renders them clumsy to be used for developing story-based PLSs such as Heroes of Koskenniska. Environmental Detectives is another PLS for outdoor environmental education [6]. Unlike Ambient Wood, Environmental Detectives is designed as a multi-player, reality simulation game. The game player assumes the role of an environmental detective whose task is to solve a contamination problem. The game uses a mixed-reality approach by mapping real world locations to virtual game content. GPS is used for positioning players during the game play. A toolkit is included for creating content for the game in order to deploy contamination problems virtually in any location. Environmental Detectives was implemented for one purpose only (environmental science) and while it comes with a powerful toolkit, it does not o↵er similar level of viability to Heroes of Koskenniska. The use of the Windows Mobile platform was also seen as a disadvantage due to the limited range of devices supported. Taiwan Weather Inquiry-Based Learning Network (TWIN) is a system in which a wireless sensor network is used for weather monitoring and inquiry-based learning [3]. There are sixty sensor nodes (stations) deployed around Taipei, Taiwan, and each node is capable of detecting temperature, humidity, air pressure, UV radiation, rainfall rate, wind direction and wind speed at regular five minute intervals. Learning activities cover weather science study and can be accessed ubiquitously via PCs. The idea of using wireless sensor networks for educational purposes overlaps with the concept of Heroes of Koskenniska, but rather than learning about weather science, players of Heroes of Koskenniska learn about environment and nature while sensor data are used for supporting learning activities. Furthermore, learning activities are not part of the TWIN system but the learners merely use the data provided by the system to complete tasks defined by the educators. The aim of the GreenSweeper pervasive, persuasive [2], game is to promote environmental awareness in an urban environment [5]. As its name suggests, GreenSweeper borrows the game idea from the popular Mine Sweeper game where the player must clean a mine field by infering the locations of the mines based on visual cues. The game informs users about the greenness of the surrounding infrastructure so as to highlight the environmental damage and impact. The greener an area, the lower is the probability of a mine residing in that area. The main purpose of GreenSweeper is to convey to the learner the presence or absence of green areas in an urban setting. GreenSweeper uses the camera and networking features of a mobile device, but not positioning technologies. The game can be deployed to any urban environment by programming the game map as a geographical map, but it lacks a deep storyline and it supports only one type of activity (clearing mines), and does not present any educational content related to the environment. Ubiquitous distribution of Bluetooth devices is harnessed in a clever way in the Insectopia [13] pervasive game where players search for virtual insects in an urban environment. Each unique Bluetooth address (i.e. every Bluetooth device that is discoverable) represents a breeding site for a type of insect which can be harvested by the player to add new

Table 3: Comparison of pervasive learning spaces

Architectural approach Application type

Heroes of Koskenniska Screenbased

Pervasive adventure game Sensors Dedicated environmental sensors Viability High Portability High Mobile de- Yes vices Data col- Yes lection Information Story, representavarious tion challenge types ContextYes awareness Network WLAN access

Ambient Wood

Moles and Mini-Moles

Environmental Detectives Map-based

TWIN

GreenSweeper

Insectopia

LieksaMyst game

Card-based

Learnerbased

Inquirybased

Mapbased

Collectionbased

Storybased

Augmented field trip

Mobile multimedia platform None

Augmented reality game None

Pervasive strategy game None

Ubiquitous treasure hunt None

Pervasive story-based game None

Low Average Yes

Average High Yes

Low High Yes

Low High Yes

Low High Yes

Average High Yes

Yes

Yes

Yes

Multimedia questionnaires No

Real time or historical weather data No

Map, players’ photos

Yes, virtual data Virtual insects

No

Information on flora and fauna, questions Yes

Yes, virtual data Story, problem solving

Information inquiry system Dedicated environmental sensors Low Average Yes, also PC No

Yes

Yes

Yes

WLAN

None

Internet access required

Yes

GPRS

WLAN

Portable probes

insects to their collection. While the game features profiles of real insects, it does not provide much educational information on the captured insects. Rather, it is likely to teach more about the high distribution of Bluetooth devices in an urban environment than details about lives of insects. The game is designed to run continuously and new players can join at any time. Ubiquitousity of Bluetooth devices and flexibility of the underlying MUPE platform make Insectopia easy to be deployed to other contexts, but viability is low because the subject matter is very specific and cannot be adjusted. LieksaMyst [10] is a PLS targeting at cultural history in the Pielinen Museum, Lieksa, Finland. LieksaMyst’s core component is a story-based role-playing game which takes the learner back in time to meet characters who lived in the old houses and used the authentic objects for various activities. Together with these authentic (albeit fictional) characters, the learner experiences daily routines of the respective period of time. LieksaMyst’s architecture was created on top of the MUPE platform to support story-based content and activities that require the player to pay attention to the surrounding objects. Originally, we considered using LieksaMyst’s architecture as the basis for Heroes of Koskenniska but it is not completely viable as it has only a limited set of available screen types and implementing new screen types is not simple. Furthermore, LieksaMyst does not support wireless sensors which are an essential part of Heroes of Koskenniska.

7.

DISCUSSION

In the Section 6 we analysed several existing pervasive learning spaces and discussed their viability, portability and overall suitability for the context of Heroes of Koskenniska. We concluded that many of these systems were designed for

Yes None

Story, various challenges

a single context and/or purpose only. The screen-based architecture was derived to meet the viability and portability requirements of the game. Other challenges were defined as bridging the gap between the nature and the people, increasing the number of visitors, and establishing meaningful connections among Biosphere Reserves. The game was designed to attract young visitors with their families to experience the natural environment through story-based game play. Knowledge about the environment is interwoven within various di↵erent tasks and an immersive storyline that places the player in the shoes of a hero. With these elements, the visitors to the Koskenniska Mill and Inn Museum area are able to learn the tacit information of the area, thus bringing them closer to the surrounding nature. Initial feedback (interviews) from multicultural adult players suggest positive experiences despite some technical problems with the network connectivity during the first tests (June 2009), as the following excerpts indicate: • [T]hese questions were really new to me, and about how to calculate age of a young tree, for example, that was very good. (Male, Finland) • I liked the last game [the battle]. (Female, Morocco) • You know, it’s really nice game. It takes your attention and takes your time so you can really spend your time...This game can also tell you some information about the place. It is not only playing just for fun but it’s also gives you some information that you are interested in as well. Another good thing is that the game can be played in teams so you can make competitions. (Male, Kazakhstan) • I didn’t know that it [usnea] is growing in places with no pollution. (Female, Namibia)

• The connection was dead quite often and the program didn’t go forward as it should have. (Male, Finland) The problems related to network connectivity occurred as the foliage in the forest interfered with wireless signals. After the first tests we improved the network by optimising the WLAN access points positions. This and other technical challenges as well as solutions to the challenges are planned to be presented in a subsequent publication. The number of visitors at the Koskenniska Mill and Inn Museum area increased significantly from year 2008 to 2009; in July 2008 there were 300 registered visitors, and in July 2009 the approximate visitor count had increased fivefold3 . While some credit of the increase can be attributed to the area’s improved facilities, advertising of the game in local media may have had an e↵ect on the increased visitor count. The long term goal of establishing connections to other Biosphere Reserves remains still to be achieved, but now we have a screen-based architecture that can be easily ported to other locations. Throughout the project we have worked in close collaboration with the representatives from the NKBR and through their connections we aim to recontextualise the game to other Biosphere Reserves for example in South Africa and South Korea. In the following sections the advantages and disadvantages of the screen-based architecture are summarised.

7.1 Advantages and possibilities The greatest advantage of the screen-based architecture is the flexible creation and reuse of screen types. We developed 14 basic screen types to meet the needs of various activities and content representation in the Heroes of Koskenniska game. One can build quite a versatile game just by using these default screen types, and for creating a new screen type only three steps are required (Section 4.3). Java mobile programming (J2ME) is not needed due to the MUPE platform. Due to the small screen size of a mobile device it is often desirable to allow navigation backwards in the story. The proposed screen-based architecture has built-in support for backwards traversal as screens are sequentially ordered by default. Branching is also enabled and is useful when the path to be followed depends on the player’s action or environmental and temporal parameters. While Heroes of Koskenniska was designed for raising environmental and cultural awareness among the visitors of the Koskenniska Mill and Inn Museum area, the same architecture can be easily adapted to other Biosphere regions by creating localised content and challenges (portability). Similarly, the architecture can be used to create games for very di↵erent purposes and contexts (viability). For example, one could write a story and challenges for a museum adventure game [10] or let the players ubiquitously explore the world of mathematics through an immersive story [8]. In fact, at the moment of writing this paper, we are in the process of creating a museum-based guide and game applications by using the screen-based architecture. Being highly viable and portable, the architecture scales well from formal education settings to informal learning arenas.

3 Personal communication with Veli Lyytik¨ ainen on August 17, 2009

7.2

Disadvantages and challenges

The underlying architecture was constructed upon the MUPE platform which has been designed to make mobile application development fast and easy. Despite its flexibility, having the architecture based on MUPE introduces restrictions and threats which are important to consider. Firstly, MUPE is not truly platform independent even though it should in theory support J2ME mobile devices with network access (GPRS or higher). Our tests have shown that the platform works best on Nokia devices but for example in some other manufacturers’ models errors may occur. We believe this to be due to non-standard implementation of the Java Virtual Machine on the respective devices. Although this is not strictly problem of MUPE, some work is needed to establish its reliable operation on multiple platforms. Secondly, MUPE is still in the beta stage and it was archived from Nokia Beta Labs in January 2009, thus development of the platform now depends on the open source community. One clear challenge for any mobile development project relates to the large number of di↵erent device configurations on the market. In particular, di↵erent screen sizes require extra e↵ort by content developers as the graphics and texts of the screens should be customised for many resolutions. Currently the Heroes of Koskenniska content is made for 240x320 resolution but with the advent of touch-screen devices with larger screens, higher resolutions and touch-based operation must be considered in the future. The content editor used for creating Heroes of Koskenniska content was built to assist researchers and is not directly suitable for educators from usability perspective. Additionally, creating new screens or modifying the models, views or controllers of the existing screens requires basic Java and XML skills. As a remedy, we are currently creating an enhanced editor tool which would allow rapid construction and utilisation of new screen types.

8. CONCLUSION Viability and portability are essential properties of a system if it is going to be deployed in various contexts for different purposes. Previous work has not been able to satisfy both of these requirements, thus a new approach was needed, and we have presented in this paper the Heroes of Koskenniska PLS which is based on a screen-based architecture. By distinguishing model, view and controller, the screen-based architecture allows flexible definition and reuse of screen templates, and we have developed 14 screen templates which were e↵ectively reused to accommodate game content and activities. In future projects these screen templates can already constitute quite versatile games and more screens can be easily created. Our previous experiences with game-based PLSs (e.g. [9], [8]) have shown that a good storyline can be an important factor for going beyond mere edutainment to keep learners motivated and immersed. The story in Heroes of Koskenniska has connections to the Finnish national epic Kalevala and is tightly woven into the surrounding nature and cultural artefacts. We chose this approach to allow the players to become immersed in the game play whilst acquiring pieces of knowledge at the same time. Post-reflection might integrate these pieces together but this remains to be verified. It was shown through discussion and observations that the game is a strong candidate to solve two of the three pre-

sented challenges of the NKBR — bridging the gap between the nature and the people, and increasing the number of visitors. The third challenge, establishing meaningful connections between Biosphere Reserves, is still work in progress. There are still a number of challenges to be resolved and development work to be done: (i) Development of an advanced editor to allow educators to easily create customised screens; (ii) Development of other PLSs with the platform to test its viability and portability; (iii) Connecting games running at di↵erent Biosphere Reserves around the world and addressing modifications needed in the game concept and architecture; (iv) making better use of the recorded sensor readings based on the analysis of historical data; and (v) pedagogical quality should be formatively evaluated. The second challenge is already being addressed as we are currently (October 2010) developing a museum-based game which will be built on the platform described in this paper.

9.

ACKNOWLEDGMENTS

We would like to thank Jinchul Choi and Kitak Yong from the MNLab at Ajou University, Korea, for their contribution to the environmental sensor network construction; Anna Gimbitskaya for programming; Ewa Kowalik for expertise in forestry; and all other individuals who helped us along the way. The development was partly supported by the European Regional Development Fund. This work was partly carried out at the University of Eastern Finland overseas campus at the Meraka Institute, South Africa.

10. REFERENCES [1] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci. Wireless sensor networks: a survey. Computer Networks, 38:393–422, 2002. [2] I. Bogost. Persuasive Games: The Expressive Power of Videogames. The MIT Press, July 2007. [3] B. Chang, H.-Y. Wang, C.-S. Chen, and J.-K. Liang. Distributed weather net: wireless sensor network supported inquiry-based learning. In CSCL’09: Proceedings of the 9th international conference on Computer supported collaborative learning, pages 365–369. International Society of the Learning Sciences, 2009. [4] J. K. Hart and K. Martinez. Environmental sensor networks: A revolution in the earth system science? Earth-Science Reviews, 78(3-4):177–191, 2006. [5] L. K. Hui-chun, L. and N. Sambasivan. Greensweeper: A persuasive mobile game for environmental awareness. In Proceedings of the Ubicomp 2008 Workshop on Ubiquitous Sustainability: Citizen Science and Activism, 2008. [6] E. Klopfer and K. Squire. Environmental detectives the development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56:203–228, 2008. [7] T. H. Laine and M. Joy. Survey on context-aware pervasive learning environments. International Journal of Interactive Mobile Technologies (iJIM), 3, 2009. [8] T. H. Laine, E. Nygren, E. Sutinen, C. I. Sedano, M. Joy, and S. Blignaut. Ubiquitous Mathematics from South Africa to Finland: Does Reverse Transfer Work? Information Age Publishing, In press.

[9] T. H. Laine, C. I. Sedano, M. Vinni, and M. Joy. Characteristics of pervasive learning environments in museum contexts. In Proceedings of the MLEARN 2009 Conference, 2009. [10] T. H. Laine, C. I. Sedano, M. Vinni, and E. Sutinen. Uncovering the richness of an authentic living museum through pervasive learning environment. Advanced Learning Technologies, IEEE International Conference on, 0:655–657, 2009. [11] K. Lyytinen and Y. Yoo. Introduction on special issue: Issues and challenges in ubiquitous computing. Commun. ACM, 45(12):62–65, 2002. [12] A. Melzer, L. Hadley, M. Glasemann, and M. Herczeg. The moles and mini moles software system: Bridging the gap between indoor and outdoor learning. In Proceedings of Mobile Learning 2006, pages 73–80. IADIS. [13] J. Peitz, H. Saarenp¨ a¨ a, and S. Bj¨ ork. Insectopia: exploring pervasive games through technology already pervasively available. In ACE ’07: Proceedings of the international conference on Advances in computer entertainment technology, pages 107–114, New York, NY, USA, 2007. ACM. [14] C. I. Sedano, T. H. Laine, M. Vinni, and E. Sutinen. Where is the answer?: the importance of curiosity in pervasive mobile games. In Future Play ’07: Proceedings of the 2007 conference on Future Play, pages 46–53, New York, NY, USA, 2007. ACM. [15] R. Suomela, E. R¨ as¨ anen, A. Koivisto, and J. Mattila. Open-source game development with the multi-user publishing environment (mupe) application platform. In Proceedings of the Third International Conference on Entertainment Computing (ICEC 2004), New York, NY, USA, 2004. Springer. [16] S. Tilak, N. B. Abu-Ghazaleh, and W. Heinzelman. A taxonomy of wireless micro-sensor network models. SIGMOBILE Mob. Comput. Commun. Rev., 6(2):28–36, April 2002. [17] M. J. Weal, D. C. Cruickshank, D. T. Michaelides, D. E. Millard, D. C. D. Roure, K. Howland, and G. Fitzpatrick. A card based metaphor for organising pervasive educational experiences. In Proceedings of the 3rd IEEE International Workshop on Pervasive Learning, pages 165–170. IEEE Computer Society, 2007.

Paper V

Laine, T.H, Islas Sedano, C., Joy, M. & Sutinen, E. (2010). Critical factors for technology integration in game-based pervasive learning spaces, IEEE Transactions on Learning Technologies vol 3, no 4, pp. 294 - 306. Reprinted with permission, Copyright 2010 IEEE Computer Society.

294

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES,

VOL. 3,

NO. 4,

OCTOBER-DECEMBER 2010

Critical Factors for Technology Integration in Game-Based Pervasive Learning Spaces Teemu H. Laine, Carolina A. Islas Sedano, Mike Joy, and Erkki Sutinen Abstract—Pervasive learning is a branch of mobile learning with an emphasis on context-awareness. Pervasive learning spaces (PLSs) create bridges from the real world to the virtual world, allowing the context-sensitive utilization of real-world objects and information in the learning process. Thus far, no model of technology integration for PLSs exists. We present a three-year process during which several game-based PLSs were developed. Based on the development experiences and a series of literature analyses, we present a technology integration model for game-based PLSs. The model meets the requirements of context, pedagogy, and game-design with technology. From these requirements, we derive three critical factors for technology integration in PLSs: 1) contextawareness, 2) available resources, and 3) unobtrusiveness of the technology. The model is discussed and evaluated through applying the model to the development process of LieksaMyst, a game-based PLS for a museum. User perceptions and usability of our games are also evaluated. The model can be utilized by PLS designers and developers for determining which requirements must be considered when integrating technology into a PLS. While the foundations of a technology integration model are now laid, work remains to be done in identifying development and evaluation methods based on the model. Index Terms—Pervasive learning, technology integration, context-awareness, pervasive learning space, game-based learning.

Ç 1

INTRODUCTION

T

ECHNOLOGY

integration is a concept often used when discussing how technology is successfully brought to school environments [1]. In the context of pervasive learning spaces (PLS), technology integration can be seen as a major challenge on the road toward successful PLS design, implementation, and deployment. Just as teachers at schools, PLSs designers may also not have the needed technical knowhow to choose and integrate correct technologies in a PLS development process. A model is needed which could be used to integrate appropriate technologies to meet a variety of requirements. To our knowledge, such a model for technology integration for PLSs does not yet exist. We invite the reader of this paper to follow a process of three years (2007-2010) during which several game-based PLSs were created. These games were created with the Hypercontextualized Game design model where the game is rooted in the same context in which the player is embedded [2]. During the process, we also derived a set of PLS characteristics and a technology integration model for game-based PLSs by literature reviews and artifact analyses on the developed games. The model comprises requirements from the context, pedagogy, and game design. Based on our experiences of a three-year-long process, we have recognized three critical factors that drive technology

. T.H. Laine, C.A. Islas Sedano, and E. Sutinen are with the School of Computing, University of Eastern Finland, Joensuu Campus, PO Box 111, 80101 Joensuu, Finland. E-mail: {teemu.laine, carolina.islas, erkki.sutinen}@uef.fi. . M. Joy is with the Department of Computer Science, University of Warwick, Room CS328, Coventry CV4 7AL, United Kingdom. E-mail: [email protected]. Manuscript received 2 Dec. 2009; revised 17 Apr. 2010; accepted 7 June 2010; published online 14 July 2010. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number TLTSI-2009-12-0180. Digital Object Identifier no. 10.1109/TLT.2010.16. 1939-1382/10/$26.00 ! 2010 IEEE

integration in PLSs: 1) context-awareness, 2) available resources, and 3) unobtrusiveness of the technology. The rest of the paper is organized as follows: We begin by defining the main concepts and the position of this research in Section 2. In Section 3, we state the methodology used after which a technology integration model is presented (Section 4). In Section 5, we present and analyze several game-based PLSs that were created over a three-year period. Then, there follows user feedback analysis in Section 6 which presents user evaluation results of the selected games. Finally, we conclude the findings in Section 7.

2

BACKGROUND

Traditional formal education [3] has been associated with locations specifically constructed for learning, such as classrooms in schools or lecture theaters in universities. Learning sessions in these environments are often teacherdriven, and student participation is limited to asking/ answering questions or short conversation sessions. School homework takes place in a home environment, and after finishing their homework a child is free to play with peers. In such a scenario, knowledge transfers between schools and homes are printed in books and written down in notebooks, but is seldom constructed ubiquitously through everyday life experiences. There are exceptions where the teacher’s role is that of a tutor, guiding the learners through the learning experience, where the learners actively construct the knowledge by themselves, but with regard to the cultural, social, and physical contexts [4], [5]. Informal learning has been complementing formal educational systems for some time now. In the past, instances of informal learning could include a visit to a neighboring village to exchange news or learn a new skill guided by a master. In informal learning, the context in which the learning takes place is not solely dedicated to the purpose of learning, but learning just happens to take place there, and Published by the IEEE CS & ES

LAINE ET AL.: CRITICAL FACTORS FOR TECHNOLOGY INTEGRATION IN GAME-BASED PERVASIVE LEARNING SPACES

Fig. 1. Types of learning according to levels of mobility and contextawareness [9].

this is perhaps the most important difference to formal classroom-based education. The potential and the interest raised by a new, stimulating context may not only increase intrinsic motivation of the learner [6], but can also act as a catalyst for educators to implement alternative learning activities which are connected with the surrounding context. Mobile learning, or m-learning, is a form of informal learning where the learner traverses a physical context or contexts carrying a personal mobile device which provides learning materials and activities. The key idea of m-learning is to enable anywhere anytime learning experiences that can be shared through ubiquitous network connectivity. M-learning has been popularized by the affordability of feature-rich handsets and the increasing availability of mobile-based learning applications. Recently, special branches of m-learning have also emerged, namely pervasive learning [7] and ubiquitous learning [8]. Fig. 1 illustrates the distinction between the basic four learning types in the domains of context-awareness and mobility. Context-awareness means the extent to which the system is context-aware through technologies such as sensors and smart tags, and mobility refers to the spatial mobility of the learner. While the terms pervasive and ubiquitous are often used inconsistently and interchangeably in computing, there exists a clear distinction between the two in terms of mobility; while ubiquitous learning refers to the “everywhere,” locationagnostic type of learning, pervasive learning concentrates on a limited geographical area but also concerns itself with the time, activities, and actors within the context. Despite the differences at the conceptual level, the same technologies (e.g., mobile devices, sensors, and smart tags) can be applied to both ubiquitous and pervasive learning. This paper concentrates on pervasive learning. As stated above, pervasive learning can be considered to be an extension of m-learning with an emphasis on the roles of an intelligent environment and of the context. The physical environment is central as it provides salient resources for learning (e.g., museum objects). According to Syva¨nen et al. [10], a pervasive learning space, also referred to as a pervasive learning environment (PLE), is a setting in which students can become totally immersed in the learning process. They further note that pervasive computing is an immersive experience which mediates between the learner’s mental (e.g., needs, preferences, and

295

prior knowledge), physical (e.g., objects and other learners), and virtual (e.g., content-accessible with mobile devices and artifacts) contexts, and the intersection of these contexts is the PLS. Syukur and Loke [11] regard a PLS as a collection of mobile users, mobile services, mobile devices, and contexts and policies, while Ogata et al. [12] state that, in pervasive learning, computers can obtain information about the context of learning from the learning environment in which embedded small devices, such as sensors, pads, and badges communicate together. Common factors in these definitions include the interplay of intelligent technology and the context in which the learner is situated. The technology facilitates context-awareness which is a prerequisite for a PLS to fully utilize the richness of the context in which the learning is situated. In other words, a PLS creates a bridge from the real world to the virtual world, allowing the context-sensitive utilization of real-world objects and information in the learning process. Games can be designed and used for a wide range of purposes, including for individual entertainment [13], as a catalyst of social interaction [14], for teaching and learning [15], as an experimental platform for new technologies and design concepts [16], and for publicity campaigns. In addition, games present a wide range of genres, independently of their digital or nondigital nature. Game-based learning, in which games are used for educational purposes, has been applied in many traditional contexts, for example, the use of nondigital games or desktop game software in language education. Most digital games with a large market share tend to have a global focus, thus disregarding the strong cultural connections that are present in many nondigital games and play (see [17] and [18]). In common with many other types of software, digital games have shifted from desktop computers to mobile devices, and the concept of pervasive gaming has emerged [19]. Pervasive games connect the physical environment to the virtual game world while retaining the elements of gameplay. At the same time, the culture of the game’s context must be taken into account in the game development process. Pervasive games with educational agendas facilitate deep immersion of the learner in the learning process, or in the flow [20]. According to Malone [21], intrinsic motivation, a component of the flow, can be facilitated in instructional environments through challenge, fantasy, and curiosity. Malone further argues that “games often provide particularly striking examples of highly motivating activities.” The potential of games for intrinsic motivation in learning is the reason why games are very suitable as the medium in educational technology research. In our personal experiences, games not only provide rewarding experiences for the players, but they are also highly motivating for us as researchers to work on.

3

METHODOLOGY

Two principal methodologies were employed for the PLS technology integration model derivation process: literature analysis (theoretical) and artifact analysis (practical). Additionally, the games we created were evaluated through various data collection techniques including pre and postquestionnaires, interviews, analyses of game data, and video recordings. The following sections describe the details of the literature review and the artifact analysis processes.

296


TABLE 1 Roles of Mobile Devices in PLSs (Adapted from [22])

VOL. 3,

NO. 4,


environments for informal learning, and museum objects, in particular, can tell many stories to the learner. Additionally, many science centers and museums offer hands-on activities for visitors where they can, for example, construct miniature models of large objects or test scientific theories. The second literature analysis is presented in more detail in Section 4. We also conducted several small-scale surveys which investigated, for example, the aspects of learning in museums, PLSs for specific themes such as environmental education, the use of wireless sensor technologies, and requirements of technology integration in educational settings. These secondary, but equally important, pieces of information also contributed to the creation process of the technology integration model as well as the individual games that were implemented during the three years.

3.2 Artifact Analysis Artifact analysis is a methodology which has been typically used in fields such as archeology, history, and arts to research on human-made objects. The goal of artifact analysis is to reach a deeper understanding about an artifact and its usage than what would be possible by mere direct observation. We analyzed the artifacts from two perspectives [23]: 1.

3.1 Literature Analysis We have performed two major literature analyses that have contributed significantly to the creation process of the technology integration model for PLSs. Papers were collected by querying popular scientific search engines such as the Google Scholar, The ACM Digital Library, and the IEEE Xplore, and then following relevant references of the papers were discovered. An initial literature analysis focused on the state-of-the-art PLSs, the technologies used, and the roles of mobile devices in the systems [22], the latter being (Table 1) of particular interest to us as a mobile device can now be found in almost every learner’s pocket. In our games, we have attempted to maximize the use of different roles for the mobile devices in order to make the interaction as rich as possible. The papers analyzed had implicit or explicit references to the following learning models/approaches that could potentially be related to pervasive learning: Group-based learning, Individual learning, Microlearning, Authentic learning, Learning by playing, On-demand learning, Hands-on/Minds-on learning, and Problem-based learning. None of the works validated the models, and therefore, we considered this evidence inadequate in terms of suitability of the models for pervasive learning. Additionally, all of models we discovered relate to the theory of constructivism, and the contexts in which the systems were built differed from traditional classroom settings. These observations support our decision to concentrate most of our efforts on informal settings that offer possibilities for rich interaction with the physical environment. Once the absence of a suitable learning model was confirmed, we started to investigate how a model of pervasive learning could be characterized. For this purpose, the second literature survey analyzed theoretical papers on constructivist learning applicable to museums. We focused on museums and science centers since they provide rich

artifacts as designed—looking at the ways in which the explicit and implicit knowledge of the designer are exposed in artifacts, and 2. artifacts as used—looking at the way in which people have appropriated, annotated, and located artifacts in their work environment. The as-designed artifact analysis was used on existing game-based PLSs to determine the key requirements which were met by the use of technology during the development processes. We considered a PLS as an artifact comprising various components such as technology, activities, pedagogy, users, content, and context, each having a unique set of requirements or restrictions. By looking at the technology integration aspects of the development processes of the artifacts, we identified categories of requirements and within them critical factors for the technology integration process. One of the games, LieksaMyst (LM), was analyzed in depth to illustrate how different requirements were met. The results of the first part of artifact analysis are presented in Section 5. In the as-used artifact analysis, we evaluated usability and users’ perceptions for several of our games. The evaluation results were then analyzed based on the critical factors found in the first artifact analysis. These findings are presented in Section 6.

4

TECHNOLOGY INTEGRATION MODEL FOR GAME-BASED PLSS

The development process of the technology integration model for PLSs started by analyzing literature on potential constructivist learning models including situated learning, authentic learning, mobile learning, contextual learning, group-based learning, exploratory learning, problem-based learning, and museum learning. The analysis was conducted by applying the principles of content analysis which makes inferences by systematically identifying specified characteristics in data (e.g., messages, articles, and logs) [24]. While


297

TABLE 2 Pedagogical Requirements (Amended from [25])

Fig. 2. Technology integration model for game-based PLSs.

content analysis was originally developed for social scientists to analyze human communication (among individuals, groups, or communities), we applied it to the analysis and derivation of categories and characteristics from established properties of various learning models and theories. The process was iterative as we started from the properties of authentic learning, eliminated those properties that were clearly inappropriate, and then amended the results with findings from other theories. Periodically, we considered each of the characteristics in the context of the existing PLSs (artifacts) to ensure their meaningfulness. In the process, we aimed to recognize champions of each learning model and used their works as the basis of the analysis. As a result of the analysis, we initially derived a set of 15 characteristics for PLSs in museums [25], which we have subsequently amended based on newly discovered evidence, thus the total amount of characteristics is 18 and they have been categorized as illustrated in Table 2. In this paper, we refer to these characteristics as pedagogical requirements. These requirements are immutable but not all them need to be met as we consider them more as a set of guidelines than must-have features [25]. This means that there is a certain degree of flexibility left for the technology integration process and for the game design process with regard to pedagogical requirements. From the perspective of learning, our experience has shown that unobtrusiveness of technology (13) is a critical factor in the technology integration process. This is because badly integrated technology may distract or even harm the user’s learning process regardless of how well the other requirements are met. For example, if a PLS shows constant erroneous behavior due to technical problems, the user may never get a chance to enter the flow of learning, hence rendering the learning experience pedagogically useless. Once pedagogical requirements were established, we performed artifact analyses on created PLSs in various contexts. By analyzing the PLSs’ technical requirements from designers’, implementors’, and users’ viewpoints, we formed

the founding blocks for a technology integration model, and considered both requirements and restrictions. The artifact analysis also helped us to confirm the appropriateness of previously discovered pedagogical requirements. Our technology integration model is illustrated in Fig. 2. The core of the model has a triangular structure in which the core is the technology (hardware and software). Each tip of the triangle represents a category of requirements that the technology can be used to meet. The requirement categories are: context requirements, pedagogical requirements, and design requirements, the latter referring to the concept design process (e.g., game concept design). The sizes of the arrowheads between the technology and the categories represent the magnitude of influence—the bigger the arrow, the bigger is the influence. For example, the technology may not have as big an influence on the set of context requirements as the context requirements have on the technology. As the diagram depicts, technology’s role is central as it attempts to resolve a versatile set of requirements and also connects the real world to the virtual world where the digital learning content is located. The connection is done by allowing the user of the system to access the virtual world by using a set of technologies (e.g., a mobile device running appropriate client software). The connection can also be made from outside the PLS context, e.g., from the home environment through a web-based interface. Context requirements cover various aspects that do not only require but may also restrict the use of technology. We have categorized context requirements under the following subcontexts: . . . . . .

Resources—financial and human [1] (e.g., size of budget and availability of required technical skills), Cultural [1], [28], [29] (e.g., prohibition of flash photography and museum curator’s attitude to technology), Technical (e.g., availability of network/electricity), Environmental [26], [28] (e.g., weather constraints on technology use and desirability of silence in the environment), Social [1], [26], [27], [28] (e.g., contextual support for collaborative learning), and Temporal [26] (e.g., limitation on usage time of the PLS and time available for implementation).

298


Context requirements are relatively static as they do not change rapidly even if the supporting technology is not available. There are cases, however, where minor changes are possible (hence the small arrow pointing at the context requirements). For example, a museum which maintains a policy of authentic environment by not using technology in its exhibitions, may allow RFID tags to be installed on the objects if the tags are not visible to the visitors. The game design process and the technology integration process may have much influence on each other—decisions made in the game design process may set requirements which specific technologies can meet, and the unavailability of a certain technology (e.g., specific sensor hardware) may restrict the game design process. As for context requirements, game design requirements can also be divided into subgroups. .

Context-awareness [31], [32] (e.g., level of contextawareness required), . Dynamics [29], [30] (e.g., use of technology to support the flow of learning experience and considered case by case), . Interaction [27], [29] (e.g., using technology to establish interactions between user, machine, and object), . Content [29], [30] (e.g., types of media used (digital and nondigital), connection of real-world objects to the digital content). Because each PLS is strongly based on a specific context, we consider context-awareness to be a critical factor of technology integration in the design process. At its simplest, context-awareness can be merely location-awareness, but the precision of context detection may be increased depending on the design requirements and the availability of technology. In addition to the context surrounding the learner, the learner’s personal context (e.g., prior experiences and preferences) is also of great importance. The categories of requirements derived in this section form the basis of a technology integration analysis method (based on artifact analysis) which is used in the next section to analyze various game-based PLSs.

5

TECHNOLOGY INTEGRATION ANALYSIS OF THE GAME-BASED PLSS

We have created game-based PLSs in various locations, including the annual SciFest science festivals [33], various kinds of museums, a South African middle school context, and a Biosphere Reserve in the Eastern corner of Finland. These games were created using the Hypercontextualized Game design model where the game is rooted in the same context in which the player is embedded [2]. A hypercontextualized game (HCG) is a novel game genre and while our PLSs are based on it, the HCG design model can also be used to create games that focus more on other cognitive processes such as creativity, innovation, or self-expression. During the process, the developed technology has matured to the point where it can be easily transferred between very different contexts and purposes. In the following account, we describe and analyze several of these games from the technology integration point of view. The games are categorized into three groups by their types. One of the games, LieksaMyst is

VOL. 3,

NO. 4,


Fig. 3. MUPE communication model.

presented in detail so as to demonstrate how our technology integration model can be used to analyze a game-based PLS. Before going deeper into the games, we first describe the core technology behind all them.

5.1 Description of the Core Technology Our game-based PLSs are built on customized game engines which we developed on top of Nokia’s MUPE (Multiuser Publishing Environment) application platform. MUPE uses a client-server approach where the server pushes game content to clients in XML format over a network connection (Fig. 3). The advantage of this content delivery model is that if changes are made to the content, the clients do not need to be upgraded. Furthermore, the same client can be used to access several MUPE-based games and the player’s status is stored on the server, i.e., the game can be resumed after a period of absence. The MUPE client is based on J2ME and should work on mobile devices that support Java MIDP 2.0. However, we have only tested it on a handful of Nokia’s S60 devices (e.g., N80, N86, and N95). In order to add new features to the client, we utilized the MUPE client plugin API to add support for 2D bar codes and Near-Field Communication (NFC), and we added support for streaming multimedia to enable playback of larger media files such as narrated audio and video clips. The scalability of the technical architecture is quite high as demonstrated by the various contexts and purposes in which different games were deployed. The MUPE server is based on several basic services, e.g., one for managing the game’s main thread, one for managing client connections, one for managing media (images, sounds, and video), and so forth. Each of these services can be placed on different physical servers and they communicate over the Internet. It is also possible to create custom services, for example, to manage information acquired from specific sensor devices. Unfortunately, we do not have benchmarking results on system performance but we have successfully had up to 15 client devices connected to a single-point server simultaneously without any observable decrease of performance. So far our greatest technical challenges have been related to WLAN network connection and the stability of client devices. These problems have been remedied by widening the network coverage and investing on client devices which have sufficient resources to run the games. 5.2 Quiz-Based Games: SciMyst and TekMyst (TM) SciMyst is a pervasive mobile adventure game played in 2007-2009 at the annual SciFest science festival in Joensuu, Finland. Players of SciMyst use mobile devices to explore the festival arena by solving intriguing enigmas related to the


299

Fig. 5. Components of SciMyst [34].

Fig. 4. SciMyst 2009 screenshots.

surrounding objects and phenomena. Each version of SciMyst has a special theme or a story, and before the game starts, the player is shown a video and/or a slide show of the story. The player can then choose to play alone or team up with friends or family members for collaborative exploration. There are several types of enigmas ranging from multiple-choice questions to take-a-picture tasks in which the player must locate a specific object based on given description and take a picture of a 2D bar code tag attached to it. An enigma relates to a specific workshop or exhibition and it is only through investigating the location that the player can find the answer. All correctly solved enigmas yield points for the player, and upon finishing the game, the points are uploaded to the game website. At the end of the game, the player has to clear the last challenge where the acquired knowledge is tested by repeating some of the game’s enigmas with a countdown timer. The player is equipped with a map of the festival area where all stands are marked and areas colour-coded. If the players need help with solving an enigma, they can use context help to receive a hint, contact other players through the multiplayer help feature of the game, or interact directly with exhibitors. SciMyst utilizes 2D bar codes for detection of objects and players’ locations. The game also has a website and a feature for sharing taken photographs and comments. Fig. 4 presents screenshots of SciMyst 2009 with a space theme. In Fig. 4a, the game asks the player to determine game location by taking a picture of a 2D bar code, Fig. 4b shows a question with expandable and scrollable fields, Fig. 4c shows the positive feedback upon correct answer, and Fig. 4d illustrates the view which is shown when the player wishes to record an impression (i.e., take a picture and write a comment) to be sent to the game website. The concept of SciMyst is located at the intersection of people, learning, technology, and playing (see Fig. 5 [34]).

The environment (the game’s context) links everything together and, in case of PLSs, it is unique for each game instance. The same basic concept was utilized in all of our subsequent game releases as per their pervasive nature. TekMyst is a game tailored from SciMyst for the Museum of Technology in Helsinki, Finland. One of the main motivations to create TekMyst was to test whether SciMyst’s concept and technology could easily be ported to a different context, a space filled with machines and technological innovations. The aspects of applicability of the SciMyst concept and technology to museums are also covered in [35] which presents SciMyst’s architecture together with a discussion of the concept’s suitability for a museum context and an analysis of a range of potential future technologies. TekMyst was based on SciMyst’s code with some small adjustments and modifications. For example, some game rules were changed, a mechanism for multiple game levels was added, and the user interface was customized. TekMyst’s story involved a magical kingdom of knowledge-sharing ants and their battle against ignorance and laziness which threatened the kingdom. Whereas in SciMyst where individual visitors or small groups came to play one by one in an unscheduled manner, in TekMyst, most players were organized in several sessions as school groups. The technology integration analysis for the SciMyst and TekMyst games revealed that the primary factors guiding the technology integration were the available budget and time, context-awareness, and game dynamics. As the resources were minimal, we used a cheap set of technologies to implement the systems quickly. Availability of resources was also guiding the game design process at some level as the design ideas were grounded in the available technology. Context-awareness was established with 2D bar codes as this approach was considered to have the best price/quality ratio. The use of bar codes also did not disturb the contexts; the Museum of Technology even encouraged bar codes due to the technological nature of the museum. The game dynamics affected mostly the design of the server software, but also contributed to the choice of specific types of mobile phones as clients. In the implementation process, we chose to create an architecture that could be reused in future games (i.e., most of our games are based on the Myst platform started with SciMyst 2007) [36].

300


5.3

Sensor-Enhanced Adventure Game: Heroes of Koskenniska Heroes of Koskenniska (2009) is a game-based PLS where mobile and sensor technologies have been combined in a natural context to provide the means to raise environmental awareness among visitors of the Koskenniska Mill and Inn Museum area at the UNESCO North Karelia Biosphere Reserve. Sensor readings of temperature, humidity, and illumination are used as background data in the game where the player traverses the forest and museum area while solving various types of tasks. The game was developed as a joint effort of forestry experts, wireless technology experts, local historians, and educational technologists. The story for the game is based on the epic battle between Ukko and Hiisi, characters from the Finnish epic story Kalevala. Ukko seeks heroes to battle against Hiisi and those who are brave enough are guided by Ukko through various challenges and tasks around different themes. The story interweaves concepts such as the beginning of life, the afterlife, the meaning of time, energy, and animals. Currently, the game content is in Finnish and English, but adding other languages is straightforward due to the multilanguage support mechanism. The game has three levels ordered by increasing difficulty. Each level has three magic spots each of which has a specific physical location and a theme, and each spot has a series of challenges for the player to solve. The challenges include text-based multiple choice questions (with one or more correct answers), imagebased tasks where the player must pick a correct image from several possibilities, and special spot activities in which the player must perform physical activities such as building a bark boat and taking a picture of it. At the end of each level, the player faces Hiisi in a special battle where they must combine the knowledge gained from the level and data from the sensors. The Heroes of Koskenniska architecture was designed from scratch as we wanted to create a more flexible design that would allow easy construction of new views and content structures—previous games had to follow a predefined structure, and content within the structure was modifiable only to a certain extent. Additionally, as the game utilizes a wireless sensor network, several new components had to be written to integrate the sensor technology into the PLS. The context and purpose of Heroes of Koskenniska are significantly different than any other game-based PLSs we have created so far. Financial support received from the European Regional Development Fund granted us the possibility to include advanced wireless sensor technology, which, in turn, provided the concept designers both challenges and opportunities. A Finnish forest environment was very challenging from the viewpoint of technology integration because of tree trunks and leaves blocking wireless signals (both sensors and WLAN), humidity, temperature fluctuation, lack of electricity, vandalism, and so on. In retrospect, it took a large amount of creativity and some luck as well to be able to integrate all the technology successfully. Available technical knowhow greatly facilitated the process. Regarding context-awareness, the game is somewhat similar to LieksaMyst, where the context is detected by user-mediated code input. Instead of a code, Heroes of Koskenniska presents a riddle to the player which must be solved correctly in order to proceed. On the other

VOL. 3,

NO. 4,


TABLE 3 Components of the LieksaMyst PLS

hand, wireless sensors provide environmental data which are not available in other games. Finally, one of the requirements was to avoid over-emphasizing the presence of technology; this was accomplished by placing the groundmounted sensor devices inside wooden boxes and attaching tree-mounted components high above the ground on branches so that they would not attract too much attention.

5.4

Story-Based Games: LieksaMyst (LM) and UFractions (UF) The Pielinen Museum in Lieksa is the second largest open air museum in Finland, hosting over 70 old buildings and structures containing over 100,000 objects from different periods of time. As a living museum, it depicts how life used to be in Eastern Finland in the past. Authenticity is one of the strengths of the museum, and in order to keep the atmosphere authentic, the buildings, structures, and objects do not have visible tags and labels other than “don’t touch” signs. In order to offer an alternative experience in addition to ordinary guided tours, we developed the LieksaMyst PLS in the Pielinen Museum together with a group of museum visitors and the curators of the museum [37]. The first public tests were run in November 2008 and the results suggested that LieksaMyst was well-received, with players being able to immerse themselves in the story [25]. The concept of LieksaMyst differs from our previous game-based PLSs because it offers not only a single game but also a suite of applications which can be used by visitors with different backgrounds, interests, and learning styles. These components, their descriptions, and target groups are presented in Table 3. The story-based roleplaying game is the most complex feature of the system and its concept also differs from that of SciMyst or TekMyst. Whereas SciMyst and TekMyst are quiz-based, competitive games, LieksaMyst offers a relaxed (no time limits and no competition) way to make a journey back in time to meet


301

Fig. 6. UFractions game screen shots. (a)-(c) Leopards let the player choose which level (time period) to play. (d)-(e) The story combines text, images, and audio. The Mother Leopard’s character is friendly and caring. (f)-(g) In addition to multiple choice questions, Leopards ask the players to give answers in numbers.

and interact with fictitious characters from the past. The characters tell the player how life was like in their respective periods of time, and ask for assistance in performing various daily activities such as churning butter or warming up the house. Relevant sound effects are used to a create more authentic atmosphere. Interaction with the characters is done through a mobile device, and in addition to answering questions presented by the characters, the player must also locate specific objects in the physical environment. By embedding these objects into the story, the game teaches the player the usage of and the connections between the objects [37]. The technology used in LieksaMyst is based on the technology of previous games but some modifications were needed in order to accommodate the story-based game structure and changes in the rules. However, these modifications were made while retaining the platform’s flexibility for the future game releases. Currently, the game has two stories in two locations: a story of Anna, a friendly 40-year-old lady of the Virsuvaara house (the largest building in the museum) in 1895, and Jussi, a 30-year-old unmarried forest worker who lives in a forest camp in the 1930s and has manners comparable to lumberjacks of that time. Both characters have very different lives and activities so as to maintain the motivation toward the gameplay. Compared to the previous quiz-like games (SciMyst and TekMyst), LieksaMyst presents deeper information about the context through a story-based approach and alternative features available to the visitors. This, together with the ability to communicate and interact with characters from the

past, as well as the connections between the objects, facilitate the immersion of the players in the authentic context. The authentic setting, coupled with an authentic (albeit fictional) story, was deemed to be one of the supporting features of the game [25]. Additionally, LieksaMyst’s development effort was supported by the grants received from the National Board of Antiquities. This enabled, for example, a WLAN network to cover the essential parts of the outdoor museum area, appropriate mobile phones that could be borrowed to visitors, and hired technical knowhow to implement parts of the system. However, other technologies could not be purchased. LieksaMyst required context-awareness without disturbing the authenticity of the context or the learning experience. This was achieved by using pieces of wood with engraved numeric codes which the players then typed in using the keypads of the mobile phones. This is an example of how a compromise is sometimes needed between the technology and the context requirements. UFractions (2009) is a story-based game built on the concept and technology of LieksaMyst. In UFractions, the player helps a mother and a cub leopard to survive by solving arithmetic fraction problems with wooden fraction sticks [38]. Screenshots of the game are presented in Fig. 6. A mobile device is used to communicate with the leopards in similar fashion in LieksaMyst, where the player communicates with the past characters. In addition to fractions, the players also learn about the lives of leopards. The major difference between UFractions and LieksaMyst is that the former is not tied to any specific context, hence contextawareness is not required from the design perspective.

302


Additionally, UFractions has a feature called “Find evidence” which aims to encourage the players to explore the physical surroundings to find real-world examples of fractions and share those examples online. Lack of funding prevented the implementation of new technologies to support it, but this did not significantly affect the game. In the following sections, we analyze LieksaMyst’s requirements (pedagogical, context, and design) and how the requirements were met according to the technology integration model (see Fig. 2). As UFractions is based on LieksaMyst, many parts of this analysis also apply to it, apart from the context requirements.

5.4.1 Meeting the Pedagogical Requirements Technology alone cannot support the pedagogical requirements as they cover the entire PLS including the concept (e.g., a game). We have earlier discussed how LieksaMyst’s features support the pedagogical requirements (i.e., PLS characteristics [25]). In the following, we present only those requirements which were directly supported by the technology integration of LieksaMyst. Consideration of learning styles. Different learning styles were considered by supporting various media types and multiple application types within the PLS. Social negotiation and collaboration. The system is based on a client-server approach and the server is constantly aware of the status of each client. Therefore, the infrastructure to allow player-to-player communication is available, but is not currently used in any of the LieksaMyst’s applications. Multimodal exploration of the environment. By using mobile technology and location-awareness it was possible to embed context-sensitive sound effects and authentic photographs in the learning experience. Ownership of the technology. LieksaMyst’s client software runs on a J2ME device with appropriate computing and memory resources, hence potentially supporting a large number of visitors using their own mobile devices. In practice, as the museum borrows Nokia N95 phones to visitors, this characteristic is not fully met. Authentic context. Technology integration respects the authentic context while attempting to increase the feeling of authenticity through various on-screen effects that could be related to the physical objects. Additionally, contextawareness support through numeric tags connects the game content to the authentic context. Unobtrusive technology. The only technology that the learner can physically see, touch, or hear while using LieksaMyst is the mobile phone. Therefore, it is imperative that the use of the phone is as smooth and error-free as possible. We chose to use the Nokia N95 as the client device as, according to our experience, it is reliable and powerful enough for running the client software. In-game media content was rendered to be as light as possible while maintaining an acceptable quality. Another technology that may have an effect on the learning experience is the wireless network, as the game system relies on continuous communication between server and clients. Network problems during the learning process may negatively affect the learning experience. To maximize reliability of the network connection, we placed the server in the museum’s premises and connected it to a WLAN network.

VOL. 3,

NO. 4,


5.4.2 Meeting the Context Requirements In the case of Pielinen Museum, the context of LieksaMyst, the context requirements and restrictions influenced heavily the technology integration process. The following account describes the technology integration analysis in each of the subcontexts defined in Section 4. Resources. Financial resources were more ample than in case of the SciMyst series. In addition to acquiring a server and phones, a wireless local area network was constructed on museum premises. This turned out to be the most costly operation as many of the old museum buildings did not have electricity outlets and they had to be installed near to WLAN access points. Human resources from the museum were excellent in terms of content matter expertise, but technical development and server maintenance were left solely to researchers. For this reason, the system was implemented with technologies that were considered stable and which supported remote maintenance. Cultural. The cultural context of the museum is characterized by the desire to maintain authenticity. For this reason, we were not allowed to use any visible stationary technologies such as touchscreens or tags, to enhance interactivity. Additionally, touching objects or photographing with flash is strictly forbidden, hence we could not use the camera feature inside the old buildings. Technical. Many of the old buildings do not have electricity, and therefore, neither artificial light nor heating systems. The museum also did not have necessary server hardware or phones, but these were later acquired. The museum is within the coverage of a 3G network, but the museum decided to build a WLAN network to keep the usage costs minimal. Environmental. Lack of artificial light makes most of the museum’s authentic rooms dark. Additionally, lack of heating renders the buildings cool especially at the beginning and at the end of the season (May-September). To conform to these environmental requirements, mobile devices were used as the primary interaction tools. Social. Museums in general are places that support social encounters and collaborative learning [26], [27]. We chose mobile devices as a technology that can easily facilitate collaboration. Additionally, a WLAN network and client-server approach were considered to be necessary to enable player-to-player interaction. The LieksaMyst game does not yet have collaborative features, but the technology is available to support it. Temporal. Outdoor exhibitions in Pielinen museum are open from May to September. In winter time, the buildings are simply too dark and cold for visitors to enjoy. The long winter break gave us enough time to integrate technologies into the design process, and after the first version of the system was finished, to develop additional features for the system as well as analyze data gathered from the usage season. 5.4.3 Meeting the Design Requirements LieksaMyst’s technology integration process and the design process supported each other, and restrictions on the use of technology set by the context requirements were also reflected in the design process. Context-awareness. From a PLS designer’s perspective, more context-awareness is better than less. In case of LieksaMyst, available resources and the museum’s culture


TABLE 4 Positive Comments on TekMyst, LieksaMyst, and UFractions

restricted the use of advanced technology to enable a high level of context-awareness. As the game design required a basic level of context-awareness, we combined authenticlooking wooden tags with manual code input on the mobile phone. By assigning a unique code to each game object, the server is able to determine the user’s location when a code is entered. In addition to location, the server is also aware of the current time and other players present in the same room. Dynamics. The availability of object-based contextawareness gave the design process an opportunity to use physical objects as part of the gameplay and to create connections between the objects. On the other hand, the design process requested the use of rich multimedia to support player immersion, hence the system was built to support flexible use of text, images, and sound. Interaction. Connectivity between the hardware components of the system was established over a WLAN connection so as to provide a means for client-to-client as well as client-to-server interactions. The game design process promoted the interaction between a player and a virtual character. This was accomplished by a dialog on a mobile device screen, which was controlled with the mobile device keys and connected to the real-world objects through wooden tags with codes (see context-awareness above). Content. One of the first requirements of the design process was that the game content should be created in such a format that new content could easily be added later. Following this requirement, LieksaMyst’s game server was designed to support the easy addition of new content in the given XML format, including various media types (text, images, and sound). Later, a graphical editor

303

TABLE 5 Negative Comments on TekMyst, LieksaMyst, and UFractions

was constructed to fulfill the requirement for easy content management. Another requirement was to support multiple languages, and currently LieksaMyst has Finnish and English content ready.

6

USER FEEDBACK ANALYSIS OF THE GAME-BASED PLSS

The game-based PLSs presented in this paper have been evaluated mostly by using a questionnaire-based evaluation on youngsters, but also some adults and senior citizens have participated in test events. All questionnaires comprised pregame and postgame parts. The pregame part was aimed to collect demographics data as well as data on attitudes and previous experiences with games and technology. The postgame part collected players’ opinions, for example, on their learning experiences, game content, usage of media, usability, the context itself (e.g., museum), and motivators. In this section, we present likes and dislikes of the players on TekMyst, LieksaMyst, and UFractions, and usability perceptions on LieksaMyst and SciMyst 2008. The numbers of test participants were 40 (SciMyst), 129 (TekMyst), 49 (LieksaMyst), and 125 (UFractions). Our view is that in technology-enhanced learning environments such as PLSs, usability is one way to measure the success of technology integration and to identify particularly obtrusive aspects of the technology. A representative collection of likes and dislikes on TekMyst, LieksaMyst, and UFractions is shown in Tables 4 and 5. The following positive aspects were reported by several players: getting information/exploring area, interaction with virtual characters, having a different experience,

304


VOL. 3,

NO. 4,


TABLE 6 Usability Suggestions and Their Relations to Critical Factors

Fig. 7. Usability of LieksaMyst and SciMyst 2008.

solving problems, entertainment, and story. On the other hand, the following negative aspects were commonly reported: small screen size (small images and text), small buttons of the device, difficult tasks, phone crashing/network problems, and the story. It was interesting to note that the story was perceived both as a positive and a negative aspect by the players. For example, some South African UFractions players did not like the cruelty of the predators against the leopards. On the other hand, the story was very much appreciated by many South African players as they seemed to be deeply immersed in the gameplay. Similarly, while many players reported “solving problems” as a positive aspect of the game, others regarded some of the tasks too difficult. This indicates the need for adjusting the content according to a player’s background and skills. Technical and usability problems with phones can be reduced by choosing a client device that has larger buttons, larger screen, and more resources (RAM and CPU), and a network that covers well the entire game area. To further measure usability of the games, we presented the players with a series of statements regarding language, use of the phone, and screen content. Results for LieksaMyst (LM) and SciMyst 2008 (SM) are shown in Fig. 7 which presents average answers on scale 5—Strongly Agree (SA), 4—Agree (A), 3—No Opinion (NO), 2—Disagree (D), and 1—Strongly Disagree (SD). Usability was generally considered good in both games, but there were some differences between the two games which we would like to point out. First, understanding the language was not easy for a few players of SciMyst (15 percent), possibly because the SciFest 2008 festival hosted a number of foreign youngsters who played the English version of the game but did not have English as their native language. Second, the phone was not deemed intuitive by a higher number of SciMyst players than in case of LieksaMyst. One reason for this could be that in the SciMyst test, the majority of phones were Nokia N80s which have arguably not as high usability and fault tolerance as N95 phones which were used more in LieksaMyst. Another possible reason could be that navigation of the game was slightly more complex in SciMyst due to the use of 2D bar codes which several players reported

having trouble with. Third, the screen was considered calm in general, but in LieksaMyst, there were slightly more people who considered the screen busy. We suspect that this could be due to a richer use of larger amounts of text, multimedia, and cartoon characters in the screen layout of LieksaMyst. Fourth, text was reported to be of sufficient size by most of the players, but in SciMyst, there were slightly more of those players (13 percent) who wished to have larger text size. Again, we believe that the cause for this could have been the use of N80 phones which have a higher screen resolution but smaller physical screen size than N95. Finally, several SciMyst players (35 percent) considered the phone a difficult tool for playing, confirming the intuitiveness result above. Based on these results, we have derived a set of suggestions for aspects that are important for the general usability of PLSs similar to our games. Table 6 describes these suggestions together with their relations to three critical factors: Context-awareness, Resources, and Unobtrusive technology.

7

CONCLUSIONS

Pervasive learning spaces are modern learning environments where mobile technology and context-awareness have been harnessed to provide context-sensitive materials and learning challenges that have deep relevance to the surrounding environment. We concluded that a model is needed which could be used to integrate appropriate technologies to meet a variety of requirements in the PLS


development process. Such a model would be useful not only in designing PLSs, but also for evaluating the user experiences within the PLS. We presented a process of three years during which various game-based PLSs were created with the Hypercontextualized Game design model. The game was chosen as the medium to support learning in informal settings. The games presented range from competitive, quiz-based treasure hunt games to more relaxed story-based roleplaying games. All games utilize the same basic technology whose flexibility makes it possible for us to develop new games with relatively few resources. From the knowledge gathered through a literature analysis on a group of existing constructivist learning models, we established a set of characteristics for PLSs, which, together with an artifact analysis of the created PLSs, contribute to the emergence of what we refer to as a technology integration model for game-based PLS. Its core consists of technology and various requirements from context, pedagogy, and game design. These requirements are divided into subcategories which, in turn, are supported by the literature. The technology integration process attempts to integrate a set of technologies to meet the various requirements and restrictions. Technology has also other purposes than satisfying the aforementioned requirements, e.g., to create a bridge between the real world and the virtual world. Based on our experiences and the supporting literature, we established three critical factors that drive the technology integration process: 1) context-awareness, 2) available resources, and 3) unobtrusiveness of the technology. Context-awareness is a key feature of PLSs and the extent to which it is supported guides the technology integration process as well as other aspects of the design process (e.g., interaction). Availability of resources in a context may set restrictions on technology integration which, in turn, may affect the PLS design process. Finally, integrated technology should be unobtrusive; this is a core element for technology integration because the learner’s immersion, the set pedagogical goals, and the motivational aspects of the game would all suffer should the technology attract too much unwanted attention. We evaluated the model by analyzing the technology integration process of LieksaMyst and discussing how the requirements were met by the use of technology (Section 5). Additionally, a subset of our game-based PLSs was evaluated by discussing user perceptions of three of the games, and usability ratings for two games. Based on these data, we suggested six aspects that are important for the general usability of game-based PLSs, and related the critical factors to these aspects. Our view is that in technology-enhanced learning environments such as PLSs, usability is one way to measure the success of technology integration and to identify particularly obtrusive aspects of the technology. The technology integration model and its critical factors were derived primarily for game-based PLSs, but they may well be applicable to other application types in informal as well as formal learning environments. This is because the model pays close attention to context and design requirements without restricting them to any specific focus area. Future user base of the model consists

305

of PLS designers and implementors who work together with other stakeholders throughout the PLS development and evaluation processes. Although three very interesting and fruitful years have passed, our work has just started. Future work involves refining the model to the point where we can establish a set of criteria for evaluating each of its components on existing game-based PLSs. Evaluations should be done on thirdparty PLSs so as to measure the generalizability of the model. Furthermore, evaluation methods for the model are to be developed. While this all relates to the theoretical part of the work, we will continue practical experiments with our existing game-based PLSs as well as looking for opportunities to create new games in new areas. During this process, we seek to develop the technology to a state where it can be conveniently ported to various contexts and for different purposes with minimal work effort.

ACKNOWLEDGMENTS The authors would like to thank all individuals involved in the design, creation, and testing processes of the PLSs presented in this paper. Special thanks go to Mikko Vinni who has had a major role in the development work and Eeva Nygren who originated the idea and content for UFractions. This research was partly financed by the Academy of Finland (grant no. 129899), the National Board of Antiquities (LieksaMyst), and the European Regional Development Fund (Heroes of Koskenniska).

REFERENCES [1]

P. Ertmer, “Addressing First- and Second-Order Barriers to Change: Strategies for Technology Integration,” Educational Technology Research and Development, vol. 47, no. 4, pp. 47-61, 1999. [2] C. Islas Sedano, E. Sutinen, J. Pawlowski, and T.H. Laine, “Games Rooted in the Player’s Context: An Introduction to Hypercontextualized Game Design,” to be published. [3] A. Kohn, The Schools Our Children Deserve: Moving Beyond Traditional Classrooms and “Tougher Standards.” Houghton Mifflin, 2000. [4] S. Papert and I. Harel, Constructionism. Ablex, 1991. [5] L.S. Vygotsky, Mind and Society: The Development of Higher Psychological Processes. Harvard Univ. Press, 1978. [6] M. Csikszentmihalyi and K. Hermanson, “Intrinsic Motivation in Museums: Why Does One Want to Learn?” Public Institutions for Personal Learning: Establishing a Research Agenda, J. Falk and L.D. Dierking, eds., pp. 67-77, Am. Assoc. of Museums, 1995. [7] J. Hundebol and N.H. Helms, “Pervasive Learning Environments,” Proc. Soc. for Information Technology and Teacher Education Int’l Conf., pp. 2226-2231, 2006. [8] H. Ogata and Y. Yano, “Context-Aware Support for ComputerSupported Ubiquitous Learning,” Proc. IEEE CS Int’l Workshop Wireless and Mobile Technologies in Education, pp. 27-34, 2004. [9] K. Lyytinen and Y. Yoo, “Issues and Challenges in Ubiquitous Computing,” Comm. ACM, vol. 45, no. 12, pp. 62-65, 2002. [10] A. Syva¨nen, R. Beale, M. Sharples, M. Ahonen, and M. Lonsdale, “Supporting Pervasive Learning Environments: Adaptability and Context Awareness in Mobile Learning,” Proc. IEEE Int’l Workshop Wireless and Mobile Technologies in Education, pp. 251-253, 2005. [11] E. Syukur and S.W. Loke, “MHS Learning Services for Pervasive Campus Environment,” Proc. Fifth Int’l Workshop Pervasive E-Learning, pp. 204-210, 2007. [12] H. Ogata, C. Yin, and Y. Yano, “JAMIOLAS: Supporting Japanese Mimicry and Onomatopoeia Learning with Sensors,” Proc. Fourth IEEE Int’l Workshop Wireless, Mobile and Ubiquitous Technologies in Education, pp. 111-115, 2006.

306


[13] P. Holleis, M. Kranz, A. Winter, and A. Schmidt, “Playing with the Real World,” Proc. Int’l Workshop Pervasive Gaming Applications (PerGames ’05), pp. 43-50, 2005. [14] V. Paelke and C. Reimann, “Vision-Based Interaction—A First Glance at Plazing MR GAmes in the Real-World Around Us,” Proc. Int’l Workshop Pervasive Gaming Applications (PerGames ’05), pp. 92-97, 2005. [15] S. Alessi and S.R. Trollip, Multimedia for Learning Methods and Development, third ed. Allyn and Bacon, 2000. [16] M. Chalmers, L. Barkhuus, M. Bell, B. Brown, M. Hall, S. Sherwood, and P. Tennent, “Gaming on the Edge: Seams in Pervasive Games,” Proc. Int’l Workshop Pervasive Gaming Applications (PerGames ’05), pp. 11-18, 2005. [17] C. Islas Sedano and C. Kampf, “Designing Games as a Vehicle for Passing on Cultural Knowledge: A Case Study of Finnish Seniors in the Digital Age,” Proc. Sixth Int’l Conf. Cultural Attitudes Towards Technology and Comm., 2008. [18] J. Huizinga, Homo Ludens: A Study of the Play-Element in Culture. Beacon Press, 1955. [19] IPerG Project Website, http://iperg.sics.se/index.php, 2010. [20] M. Csikszentmihalyi, Finding Flow: The Psychology of Engagement with Everyday Life. Basic Books, 1998. [21] T.W. Malone, “Toward a Theory of Intrinsically Motivating Instruction,” Cognitive Science, vol. 5, no. 4, pp. 333-369, 1981. [22] T.H. Laine and M. Joy, “Survey on Context-Aware Pervasive Learning Environments,” Int’l J. Interactive Mobile Learning (I-JIM), vol. 3, no. 1, 2008. [23] D. Ramduny-Ellis, A. Dix, P. Rayson, V. Onditi, I. Sommerville, and J. Ransom, “Artefacts as Designed, Artefacts as Used: Resources for Uncovering Activity Dynamics,” Cognition, Technology and Work, vol. 7, no. 2, pp. 76-87, 2005. [24] O. Holsti, Content Analysis for the Social Sciences and Humanities. Addison-Wesley, 1969. [25] T.H. Laine, C. Islas Sedano, M. Vinni, and M. Joy, “Characteristics of Pervasive Learning Environments in Museum Contexts,” Proc. World Conf. Mobile and Contextual Learning (MLearn ’09), pp. 26-34, 2009. [26] R.E. Grinter, P.M. Aoki, A. Hurst, M.H. Szymanski, J.D. Thornton, and A. Woodruff, “Revisiting the Visit: Understanding How Technology Can Shape the Museum Visit,” Proc. ACM Conf. Computer Supported Cooperative Work, pp. 146-155, 2002. [27] L. Bannon, S. Benford, J. Bowers, and C. Heath, “Hybrid Design Creates Innovative Museum Experiences,” Comm. ACM, vol. 48, no. 3, pp. 62-65, 2005. [28] J.H. Falk and L.D. Dierking, Learning from Museums: Visitor Experiences and the Making of Meaning. AltaMira, 2000. [29] L. Ciolfi and L. Bannon, “Designing Interactive Museum Exhibits: Enhancing Visitor Curiosity through Augmented Artefacts,” Proc. European Conf. Cognitive Ergonomics, pp. 62-65, 2002. [30] G. Vavoula, M. Sharples, P. Lonsdale, P. Rudman, and J. Meek, “Learning Bridges: A Role for Mobile Technologies in Education,” Educational Technology Magazine: The Magazine for Managers of Change in Education, vol. 47, no. 3, pp. 33-37, 2007. [31] A. Dix, T. Rodden, N. Davies, J. Trevor, A. Friday, and K. Palfreyman, “Exploiting Space and Location as a Design Framework for Interactive Mobile Systems,” ACM Trans. ComputerHuman Interaction, vol. 7, no. 3, pp. 285-321, 2000. [32] D. Raptis, N. Tselios, and N. Avouris, “Context-Based Design of Mobile Applications for Museums: A Survey of Existing Practices,” Proc. Seventh Int’l Conf. Human Computer Interaction with Mobile Devices and Services, pp. 153-160, 2005. [33] Univ. of Eastern Finland, “SciFest Home Page,” http://www. scifest.fi/home, 2010. [34] C. Islas Sedano, T.H. Laine, M. Vinni, and E. Sutinen, “Where Is the Answer?—The Importance of Curiosity in Pervasive Mobile Games,” Proc. FuturePlay Conf., pp. 46-53, 2007. [35] T.H. Laine, C. Islas Sedano, M. Vinni, and E. Sutinen, “Engagement, Motivation and Inspiration: The SciMyst Framework in Museums,” Proc. Fourth Int’l Conf. New Exploratory Technologies, 2007. [36] T.H. Laine, M. Vinni, C. Islas Sedano, and M. Joy, “On Designing a Pervasive Mobile Learning Platform,” ALT-J: Research in Learning Technology, vol. 18, no. 1, pp. 3-17, 2010. [37] T.H. Laine, C. Islas Sedano, M. Vinni, and E. Sutinen, “Uncovering the Richness of an Authentic Living Museum through Pervasive Learning Environment,” Proc. Ninth IEEE Intl. Conf. Advanced Learning Technologies, pp. 655-657, 2009.

VOL. 3,

NO. 4,


[38] E. Turtiainen, S. Blignaut, C. Els, T.H. Laine, and E. Sutinen, “Story-Based UFractions Mobile Game in South Africa: Contextualization Process and Multidimensional Playing Experiences,” Proc. Second Workshop Story Telling and Educational Games, 2009. [39] T.H. Laine, C. Islas Sedano, E. Sutinen, K.-T. Yong, H.-J. Woo, C.-W. Lee, and T. Jaakkola, “Heroes of Koskenniska and Self: Application of Body and Exercise Monitoring to a Pervasive Educational Game,” Proc. Third Int’l Symp. Learning and Teaching of Motor Skills, Poster Session, 2009. Teemu H. Laine received the MSc degree in software engineering in 2007 from the University of Eastern Finland, where he is currently working toward the PhD degree with an ECSE grant received from the Academy of Finland. After finishing the MSc degree, he has been focusing on pervasive and mobile learning applications, models, and platforms as part of his doctoral research. His other research interests include ubiquitous computing, software architectures, wireless sensors, digital games, and ICT for development. Carolina A. Islas Sedano received the BSc degree in electronic engineering and communication systems from the University Iberoamericana in Mexico City, and the MSc degree in communication and media engineering from the University of Applied Science in Offenburg, Germany. She is completing the PhD degree at the University of Eastern Finland. Her research interests include conceptualization, development, and implementation of games with a bottom-up approach, involving the specific environment that surrounds the gameplay to promote informal learning experiences. Mike Joy received the MA degree in mathematics from Cambridge University, the MA degree in postcompulsory education from the University of Warwick, and the PhD degree in computer science from the University of East Anglia. He is currently an associate professor at the University of Warwick. His research interests focus on educational technology and computer science education. Erkki Sutinen is a professor and the leader of the Educational Technology Research Group at the University of Eastern Finland. His research interests include using technologies for clearly defined needs, such as for complex subject domains, like programming, in developing countries, and within special education. The applied techniques cover visualization, information retrieval, data mining, robotics, and design models. He has coauthored and published more than 100 academic papers.

Paper VI

Laine, T.H., Sutinen, E., Joy, M.S. and Nygren, E. (2011). Active and passive technology integration in context-aware learning spaces, In Proceedings of the AACE E-Learn 2011 Conference, Honolulu, Hawaii, pp. 719 - 728. Reprinted with permission, Copyright 2011 The Association for the Advancement of Computing in Education

Active and Passive Technology Integration in Context-Aware Learning Spaces Teemu H. Laine, Erkki Sutinen, Eeva Nygren School of Computing, Joensuu Campus University of Eastern Finland, Finland [email protected]

Mike Joy Department of Computer Science University of Warwick, UK [email protected]

Abstract: Context-aware learning spaces (CALSs) utilise resources of the surrounding context in the learning process. UFractions is a CALS combining a storytelling game and fraction rods for mathematics education, and it was developed for the South African context and later taken to Finland. We divide technology integration into active and passive integration according to the role of technology in the process. In passive integration the technology is integrated into the CALS so that it does not disturb the learner and the context. In active integration the technology integrates resources into the CALS and makes the system adaptive to contextual changes. We analysed, by a mixed method approach, the need for active and passive integration in UFractions in Finland and South Africa. We identified sixteen disturbance factors which had negative effects on the users of UFractions. The results indicate that by improving active and passive integration in UFractions, disturbance factors can be diminished.

Introduction Technology integration describes how a technology facilitates learning and teaching in a classroom environment (Ertmer, 1999). It refers to the process by which a technology is introduced to a classroom so that the teacher and the students can use it efficiently for pedagogical purposes. Poor technology integration may lead to disruptions in teaching and learning or to wasted technology resources. It is, therefore, important to ensure educational technology's proper integration into the target context. While technology integration is often considered to be a problem of a formal classroom-based education, the same is true for informal learning contexts. If a learning technology were deployed in a museum (for example) while disregarding the influence of the technology on the learner and the context, the technology would fail to meet expectations. In this paper we focus on technology integration in context-aware learning spaces (CALSs) which are mobile-based learning environments emphasising contextuality and utilising surrounding contextual resources (e.g. museum exhibits) in the learning process. Previously, a technology integration model (Laine et al., 2010) has been developed for CALSs. The model was created to assist the CALS developers to ensure that a chosen set of technologies meets various requirements set by the context, established pedagogical guidelines and the design process. As a result, a CALS would offer an appropriate level of contextuality while being unobtrusive to the users. Elements that negatively affect the users in a CALS are referred to as disturbance factors. In 2009 we developed UFractions (Ubiquitous Fractions), a game-based CALS, to help eighth graders in rural areas of South Africa to learn fractions and become motivated towards mathematics (Turtiainen, 2009). The game combines an interactive story on a mobile phone with coloured fraction (Cuisenaire) rods used to perform fractional calculations. The colours and lengths of the rods correlate and this information is in the game (e.g. two yellow rods equal to one orange rod). The game content was originally designed in and for the South African context using the English language, but later translated into Finnish without modifying the content. In this study we established a division of technology integration for CALSs into active integration and passive integration. Most research on technology integration (e.g. Ertmer (1999), Becker (1994), Levine & Wadmany (2008), Mishra & Koehler (2006)) has concentrated on classroom based education. There has been no research on dividing the technology integration according to the roles of technology in the integration process in order to manage technology's influence on the learner. Based on the concepts of active and passive technology integration we evaluated the UFractions game in South Africa and Finland to answer the questions “To what extent technology integration is needed in UFractions?” and “What are the disturbance factors of technology integration in UFractions?”. Through the study we sought not only to propose how technology integration can be improved in the case of UFractions but also derive the principles for proper technology integration in CALS design.

!

Context-Aware Learning Mobile learning is a form of technology-enhanced learning where the learner traverses physical contexts while carrying a personal mobile device which provides learning content regardless of place and time (Eschenbrenner & Nah, 2007). Context-aware learning is a subcategory of mobile learning which emphasises context-awareness in the learning environment. Context-aware systems recognise and act upon changes in a collection of contextual entities which form a situation. We define a situation as a snapshot of a context at a given moment of time. Contextual entities define where the learners are, what they are doing, how they are feeling, who else is with them, what resources are nearby, what time it is, and what kinds of parameters the physical environment has. We define contextual resources as a subset of contextual entities which can be detected by a given set of context-aware technologies (e.g. sensors) and utilised in a CALS. We further define context-free resources as resources which are not dependent on a given context (e.g. a theory or general knowledge of a topic). By being aware of the various contextual resources a learning system can adapt both contextual and context-free resources to fit the situation in which the learner is embedded. A learning environment which makes use of a context-aware system is referred to as context-aware learning space (CALS). Figure 1 illustrates the way a CALS, by using context-aware technologies, detects changes (∆1 and ∆2) in contextual entities between two temporally consecutive situations. Entity1 and Entity2 are contextual resources because they can be detected and utilised by the CALS. By our definition Entity3 is not a contextual resource because it is out of reach of the CALS.

Figure 1. Detection of changes between situations in a CALS The level of context-awareness is specific to the application – in some cases knowing the user’s location within a geographical area is enough (Ballagas et al., 2004) whereas in other applications it may be necessary to detect the parameters of the surrounding environment (Martinez et al., 2004) or changes in the user’s body (Aziz et al., 2006). As creating a highly context-aware system requires money and time, trade-o!s are necessary. State-ofthe-art technology can also become a burden if it is not integrated properly as it may disrupt the user experience (Kaasinen, 2002) or the system simply does not work because of a lack of technical maintenance skills.

Technology Integration in CALSs The term integration refers to a process of combining various distinguishable parts to create a complex whole (Spector, 2002). Technology integration is a concept often used when discussing how a technology is brought to a complex school environment (Ertmer, 1999). It refers to the process by which a technology is introduced to a classroom so that both teacher and students can use it efficiently for pedagogical purposes. In context-aware learning, technology integration is critical because, just as teachers at schools, CALS designers may not have the needed technical and conceptual know-how to choose and integrate correct technologies in a CALS development process, without which technology integration will result in disturbed learning experiences. We have previously established a technology integration model for CALSs (Laine et al., 2010) which concerns various requirements of context, pedagogy and design which should be met by integrated technology. An analysis of the model identifies two types of technology integration that must be considered in CALSs:

• •

Passive integration: technology must be integrated into the CALS so that it becomes subtle and unobtrusive to the learner and to the context. In other words, technology is the object of integration. Active integration: technology must integrate the contextual resources and context-free resources into the CALS and make the system adaptive to the changing situations of the context, including users within. In other words, technology is the subject of integration.

From the perspective of passive integration, technology influences directly the learner for example through poor usability, technical errors or other distractions. From the perspective of active integration technology's influence is indirect – if the contextual resources are not integrated properly by technology, the learner is influenced for example through poorly contextualised learning materials, annoying presentation of content or too difficult learning challenges. Division into passive and active integration helps us to identify and manage the ways in which the technology influences the learner. Both integration types are driven (or restrained) by available resources (Figure 2). Passive integration aims at achieving unobtrusiveness of the technology from the learners' and the context's perspectives so that the learning process is not disturbed by the technology. The resulting unobtrusive technology provides contextawareness to the active integration process via contextual resource detection. When a situation in the context changes the technology automatically adapts the contextual and context-free resources. The active integration process can utilise various contextual resources such as physical objects, learner’s background and preferences, environmental parameters (e.g. weather), position of objects and learners, and available technology. It is important to note that passive integration must be completed before active integration is feasible.

Figure 2. Passive and active integration of technology in CALSs

Research design UFractions was originally designed for the South African context with local educators and cultural experts (Turtiainen, 2009), and later translated into Finnish. With these two variants we performed a comparative study which measured various aspects of the game in South African and Finnish contexts. The aim was to see how well a game developed in and for a technology-alien context could be transferred to a technology-familiar context without performing a re-contextualisation of the game. The evaluations were conducted in five South African schools and four Finnish schools during March 2009 and March 2010 respectively, using 8th grade middle school pupils (105 in South Africa and 104 in Finland). A multi method approach comprising qualitative and quantitative strategies was employed in the evaluation. Pre- and post-test questionnaires with closed and open questions were complemented by post-test interviews of a subset of pupils in each test group. Teachers were also asked to fill in a separate questionnaire and were interviewed after the game session. During the game session researchers also observed and took notes on the pupils' interactions with each other and with the game. In this paper only the students’ data are analysed. The pre-test questionnaire for the pupils queried data on demographics, technology usage, perceptions of mathematics classes and learning mathematics, and five simple fraction problems. The aim of it was to ascertain the pupils' stances towards fractions. The post-test questionnaire used open or Likert scale questions to evaluate the pupils' playing experiences from several viewpoints: motivation, likes and dislikes, game activities,

usability, contextual relevance, overall perceptions and suggestions for improvements. Additionally, there were five fraction problems at the end of the post-test questionnaire. The interview questions aimed at collecting deeper insights from pupils on motivation, learning, playing and the usage of fraction rods. In South Africa the data collection instruments were in English and in Finland the instruments were translated to Finnish. We have previously reported the aspects of pedagogy, culture and technology reception in the reverse transfer process of UFractions (Laine et al., 2011). The results suggested that while there were significant differences in the adaptation of the game to the pedagogical and cultural contexts of the test settings, the technology was well received in both cases. In the current study we analysed the same data sets from the aspect of technology integration. In particular, we wanted to answer the research questions: “To what extent is technology integration needed in UFractions?” and “What are the disturbance factors of technology integration in UFractions?”. We explored how the pupils experienced the technology and how well the resources were adapted to both contexts in order to identify disturbance factors that have negative effects. The results for research question 1 were derived from quantitative questionnaire data supported by qualitative comments from interviews and open answers of the pupils. For question 2 we first established indicators which were then used to identify the factors from qualitative data, which were then categorised according to the areas of experience.

UFractions UFractions is a game-based CALS that was developed to help the children at rural South African schools to learn fractions and become motivated towards mathematics. The game combines an interactive story on a mobile phone and fraction rods that can be used to solve fraction problems. The story, presented with text, images and audio, features two leopards, a mother and her cub, with which the player adventures through the first year of the newborn cub's life. The year is divided into three skill levels of varying difficulty. Raising the cub is filled with challenges such as finding food to eat, learning how to hunt, avoiding enemies, and finding sources of fresh water. The player assists the leopards by solving fraction problems with the help of the fraction rods. An example of UFractions tasks with an open answer is presented in Figure 3 (left).

Figure 3. Left: UFractions task with an open answer; Right: Phone is aware of the rods through colour codes. Some students employed also other tools (pen and paper, calculator). Mobile phones (Nokia N95 and N80, including client software) and coded fraction rods can be directly observed and manipulated by the players of UFractions (Figure 3, right). The hidden part of the technology is formed by the UFractions game server and a wireless network connecting the clients and the server. To the players, the game appears to be running completely on the mobile phone. The fraction rods currently used represent 10 or 12 different lengths, and each size has its respective colour. The game supports different fraction rod sets and the mapping between colours and lengths is done in a game configuration file. For each challenge the game tells the player the colours and the codes of the fraction rods that are associated with that challenge and the player, in turn, finds an answer based on the rods.

Evaluating Technology Integration in UFractions

Evaluation sessions were conducted for groups with 16 to 32 pupils. In each session, researchers first introduced the story and the basic idea of the game, including instructions for fraction rod usage. The pupils received phones and were given approximately 45 minutes to play the game in groups of two to four pupils. During game play the researchers observed participants' activities and gave guidance if required. Upon finishing the game the pupils completed the second part of the questionnaire. Afterwards, three to five pupils were randomly selected for individual interviews. Passive Integration We evaluated the passive integration by measuring how well the technological context (phones, software and rods) suited the pupils. Statements related to entertainment, user interface and ease of use, were answered using a Likert scale (1=Strongly Disagree, 5=Strongly Agree) (Figure 4).

Figure 4. Evaluation of technological context The South African pupils generally considered the user experience of the technology to be better than the Finnish pupils. An exception was the negative statement of the screen being busy which a higher number of Finns disagreed with (63%) than South Africans (42%). On the other hand, 22% of the Finnish pupils expressed no opinion on this statement compared to 9% of the South Africans (and this was one of the highest “no opinion” answers in the entire South African data set). These patterns could indicate that some pupils did not understand the meaning of the statement. The phone was considered to be an easy and fun tool for playing by both groups of pupils. All Finns owned a mobile phone while 63% of the South African pupils reported owning one. It is likely that those South African pupils who do not own a mobile phone have been using one with their friends or at home. This boy relates the ability to use the phone to fun factor and learning outcome. The actual playing it on the phone, actually, that was the fun part. … For some children it's difficult to learn while they've been telling you or reading it. But it is on the phone now, something you're good at so you can just learn quickly instead of always being stressed. (M13, ZA) Many Finnish pupils did not find rods to be fun to play with as only 64% agreed about the fun factor, which can be partly explained by the lack of interest towards the game story (see below) and partly because some of the Finnish pupils had previously been exposed to manipulatives such as fraction rods in mathematics classes (Laine et al., 2011). Many pupils commented on their rod experiences positively, for example: I enjoyed playing with rods the most. I enjoyed myself very much (F13, ZA) [I liked] calculating with the fraction rods and playing with them (F14, FI) Active integration The success of active integration was determined by analysing how well the resources are adapted to the changing contextual properties. Specifically, we measured how well the game was adapted to the target

contexts in both countries in terms of approach to and supporting attitudes towards content matter and comfort zone, using a house metaphor to justify the three perspectives. Approach to content matter shows how well the pupils entered the house. Supporting attitudes towards content matter evaluates how the pupils liked to visit the house. Finally, evaluation of the comfort zone shows if the pupils are comfortable enough to live in the house. Approach to content matter Figure 5 shows the results of the statements related to the story-based approach to content matter (the same scale as in Figure 4). As one can observe significant differences in the answers of the two data sets, it is clear that the story was not suitable for Finnish pupils while the South Africans were much immersed in it.

Figure 5. Evaluation of the approach to content matter Through the interviews and open questions we found out that there were both Finns and South Africans who disliked the story but their reasons were different. Finns considered the story to be childish for their age but South Africans were concerned of leopards' safety: [It should have] humour and shorter questions. Transform the story for 8th graders. (M14, FI) I didn't like when I heard that mother leopard is struggling to feed Senatla and keep him from a safe place. (M14, ZA) Despite of few negative comments, a good majority of South Africans and also some Finns commented positively on the story-based approach. Additionally, a few pupils in both groups suggested improving the presentation by enhancing graphics and adding narration, indicating different preferences on media modalities. Supporting attitudes towards content matter Figure 6 shows how UFractions supports the attitudes of the pupils towards mathematics. These results are similar to the results presented in the previous section – the Finnish pupils agreed much less with the statements than their South African counterparts. The number of omitted opinions among the Finnish pupils was also significant. In particular, in the last statement regarding positive attitude towards mathematics, 48% of the Finns chose not to give their opinions, leaving the proportion of positive answers to be only 19%. Overall results of the Finnish data set indicate that the game does not support well attitudes towards content matter.

Figure 6. Evaluation of the support for attitudes towards content matter The South African pupils were much more enthusiastic about the learning experience. In addition to reporting having learned fractions, several South African pupils, like this girl, reported that they learned new things about leopards although it was not the primary target of the game: I enjoyed playing with the cuisenaire rods and solving the problems. It was also interesting to see how the leopards grow and how they take care of themselves. (F13, ZA) Comfort zone By comfort zone we refer to the psychological state of the learners in which they feel comfortable, or “at home”, with the CALS. Comfort zone is the goal of technology integration – after successfully approaching the topic and forming a positive attitude towards it, the learner may enter the comfort zone where immersion in the flow can happen (Csikszentmihalyi, 1998). The statements measured the following aspects: comparison to an ordinary class, usefulness of the rods (contextual resource) and team play. The results are illustrated in Figure 7.

Figure 7. Evaluation of the comfort zone These examples show how deep the South African pupils were in the comfort zone: I have learned a lot and had fun. I realized that maths isn't boring or difficult. I won't ever daydream again in class, not even sing during math class. (F15, ZA) Would want to go on and on and just never stop like when you said that we have to stop I was so angry. (F13, ZA) Finns were not quite as much immersed but a majority of them also considered the game more exciting than an ordinary class (South Africa: 92%, Finland: 71%). Usefulness of the rods for understanding the fractions (South Africa: 96%, Finland: 74%) and team play (South Africa: 98%, Finland: 78%) were much appreciated by both groups. We observed that the group dynamics in both countries worked well except for a few cases where a pupil joined a group while she/he did not belong to the social circle formed by other members of the group. Possibly because of this, or due to personal preferences, there were pupils who would have liked to play alone.

Finally, the efficiency of the game in regard to learning experience in comparison to an ordinary class was not reported as positive by the Finnish pupils (38%) as it was reported by the South Africans (93%). It must be noted, however, that 29% of the Finnish pupils had no opinion on this statement. Table 1. Identified disturbance factors.

Disturbance Factors By analysing the qualitative data we discovered sixteen disturbance factors which relate either to active (9) or to passive (7) integration of technology. We analysed questions related to dislikes, improvement suggestions and overall experiences of the pupils. Table 1 describes the disturbance factors with indications that map the relevant evidence to the factors. Column I indicates whether the factor relates to active (A) or passive (P) integration. The identified disturbance factors are grouped by the learner's areas of experience which are

affected by the disturbance factors. ZPD refers to Vygotsky's Zone of Proximal Development (1978). Contextual experience refers to the experience related to the fraction rods because they form the context of which UFractions game is aware through interaction with the learner. Our observations supported some of the disturbance factors found from the questionnaires and interviews. Some students accidentally pressed a wrong button and exited the game software (Interaction with phone). In a few cases the game also froze and had to be restarted (Technical faults). Some of the pupils created various constructions from the rods, instead of using them for solving the problems, while other members of their group continued playing and solving challenges. This behaviour may occur for two reasons: (i) the pupils were outsiders in the peer group (team members), and (ii) the pupils considered the game uninteresting (lack of challenge, inappropriate content). Finally, some pupils requested help from their teacher or researchers to get clarification or additional hints for solving a challenging task (unclear instructions, too much challenge).

Discussion The question “To what extent technology integration is needed in UFractions?” was answered through an evaluation of active and passive technology integration. Particularly, active integration failed in Finland as technology indirectly influenced the pupils by not providing contextualisation of the content. Passive integration was fairly successful as most pupils received the technology well, although some individual pupils who were disturbed either by the rods, the phone or the game software. To answer the question “What are the disturbance factors of technology integration in UFractions?” we derived sixteen disturbance factors which followed the trend of the results of active and passive integration evaluations. The evidence suggests that some of the factors may be generic to other CALSs and even to other learning environments. For example, we can comfortably assume that the disturbance factors of “Above ZPD” and “Beyond ZPD” could relate to nearly any kind of challenge-based learning environments but without supporting evidence this remains merely a hypothesis. By diminishing the identified disturbance factors, UFractions would become more effective and context-aware as a CALS, and it would provide better experiences to the learner in all nine areas of experience. With improved experiences the learner would more likely enter the flow (Csikszentmihalyi, 1998). While these disturbance factors were derived from the evaluation of UFractions, we hypothesise that they can be used as a check list for designing, evaluating and improving other CALSs. The process of deriving the disturbance factors and the areas of experience was constrained by the fact that the data were classified by only one researcher, hence leaving a possibility for subjectivity. Additionally, it is possible that with a larger data set more disturbance factors could have been identified. Despite of these limitations, the results achieved with the available data set and with applied methods suggest that various experiences were disturbed by the lack of active and passive technology integration in UFractions. Regarding active technology integration, the results showed that the technology should integrate the learning content and its presentation format into the target context because mere translation of the content from English to Finnish was not enough to meet the requirements of the Finnish pupils in their context. Active technology integration is necessary in order to serve heterogeneous sets of users in different contexts. A flexible solution for active integration of content in the case of UFractions could be content adaptation systems (Lemlouma & Lyaida, 2004) which use models, such as a domain or user model, to provide meaningful content based on relevant parameters such as the learner's level of knowledge and learning preferences. A context model could then be used in conjunction with domain and user models to match the content with the target context and users.

Conclusion We divided technology integration for CALSs into active and passive integration. Both integration types are important to consider, as we found out while evaluating the UFractions mathematics game in two very different contexts. Therefore, one of the main implications of the results reported in this paper is that in order for a CALS to be effective, its technology must be unobtrusive and subtle to the learner while adapting contextual resources to match the learner's profile. This means that the effects of disturbance factors must be minimised. Disturbance factors established in this paper, while being specific to UFractions, may indicate pitfalls in the design and implementation of future CALSs. This information can be useful for the CALS designers to plan the use of technology so that the goals of active and passive technology integration are met. Furthermore, the eight areas of experience are also useful for the CALS designers for ensuring that a variety of different experiences

are supported in a CALS. The results yielded by this study can be used as a starting point towards a complete hierarchy of areas of experience and related disturbance factors. Additionally, generalisability of the factors and their experience areas to other learning environments apart from CALSs should be investigated. A long term goal of UFractions is to become a stable, easy-to-use platform that pupils could access at home or at school with their own handsets. Related to this goal, an implication of evaluating UFractions in controlled environments is that the pupils'/teachers' abilities to launch and troubleshoot the game were not measured. As the pupils' interaction with the technology falls into the domain of passive technology integration, it will be necessary to run another evaluation on independent use of the game in order to verify the success of passive technology integration outside the controlled environment. Finally, the concepts of passive and active integration are generic but we have only applied them to and analysed them in a CALS. In the future an important research activity is to find out how well the concepts of passive and active integration fit into classrooms or online learning environments.

References Aziz, O. Lo, B., Yang, G.-Z., King, R., & Darzi, A. (2006). Pervasive Body Sensor Network: An Approach to Monitoring the Post-operative Surgical Patient, International Workshop on Wearable and Implantable Body Sensor Networks, 13-18. Ballagas, R., Kratz, S., Borchers, J., Yu, E., Walz, S., Fuhr, C., Hovestadt, L., & Tann, M. (2004). REXplorer: a mobile, pervasive spell-casting game for tourists, CHI ’07 extended abstracts on Human factors in computing systems, 1929–1934. Becker, H. (1994). How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools. Journal of Research on Computing in Education, 26, 291–321. Csikszentmihalyi, M. (1998). Finding Flow: The Psychology of Engagement with Everyday Life. New York, NY: Basic Books. Ertmer, P. (1999). Addressing first- and second-order barriers to change: strategies for technology integration. Educational Technology Research and Development. 47(4), 47–61. Eschenbrenner, B. & Nah, F-H. (2007). Mobile technology in education: uses and benefits. IJMLO International Journal of Mobile Learning and Organisation, 1, 159–183. Kaasinen, E. (2003). User needs for location-aware mobile services, Personal and Ubiquitous Computing 7 (1), 70–79. Laine, T.H., Islas Sedano, C., M. Joy, M.S., & Sutinen, E. (2010). Critical factors for technology integration in game-based pervasive learning spaces. IEEE Transactions on Learning Technologies, 3(4), 294–306 Laine, T.H., Nygren, E., Sutinen, E., Islas Sedano, C., Joy, M.S., & Blignaut, S. (2011). Ubiquitous Mathematics from South Africa to Finland: Does Reverse Transfer Work?, in Ubiquitous Learning: A Survey of Applications, Research, and Trends. Charlotte, NC.: New Age Publishing. Lemlouma T., & N. Layaida, N. (2004). Context-aware adaptation for mobile devices. IEEE International Conference on Mobile Data Management 2004. Levine, T., & Wadmany, R. (2008). Teachers’ views on factors affecting effective integration of information technology in the classroom: Developmental scenery. J. of Technology and Teacher Education, 16, 233–263. Martinez, K., Hart, J., & Ong, R. (2004). Environmental sensor networks, Computer 37, 50–56. Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108, 1017–1054. Spector, J., & Wang, X. (2002). Integrating technology into learning and working: Promising opportunities and problematic issues. Educational Technology & Society 5(1). Turtiainen, E., Blignaut, S., Els, C., Laine, T.H., & Sutinen, E. (2009). Story-based UFractions mobile game in South Africa: Contextualization process and multidimensional playing experiences. Second Workshop of Story Telling and Educational Games, 2009. Vygotsky, L. (1978). Mind in Society: The Development of Higher Psychological Processes, M. Cole, V. JohnSteiner, S. Scribner, and E. Souberman, Eds. Cambridge, MA: Harvard University Press.

Paper VII

Laine, T.H., Sutinen, E., Joy, M.S. and Nygren, E. (2011). Rapid improvement of technology integration in context-aware learning spaces, In Proceedings of the IEEE Africon 2011 Conference, Livingstone, Zambia, pp. 1 - 6. Reprinted with permission, Copyright IEEE Computer Society.

Rapid Improvement of Technology Integration in Context-Aware Learning Spaces Teemu H. Laine, Erkki Sutinen, Eeva Nygren

Mike Joy

School of Computing, Joensuu Campus University of Eastern Finland P.O.Box 111, 80101 Joensuu, Finland Email: [email protected]

Department of Computer Science University of Warwick Coventry, CV4 7AL, UK Email: [email protected]

Abstract—Context-aware learning spaces (CALSs) utilise resources of the surrounding context in the learning process. UFractions is a CALS combining a storytelling game on a mobile phone and fraction rods for learning mathematics at middle schools. Technology integration is the process by which a technology is introduced to a pedagogical setting with an aim to use it effectively for teaching or learning. We proposed a tool for rapid improvement of technology integration in CALSs and used the tool to evaluate UFractions from the learner’s perspective in the Mozambican context. As results we identified 22 disturbance factors and made a comparison to a previous study which was conducted in South Africa and in Finland with instruments that were not meant for assessing technology integration. The results indicate that the proposed tool yields more accurate results with a significantly smaller data set than the previous study. Furthermore, the identified disturbance factors guide the improvement process of UFractions.

I. I NTRODUCTION Context-aware learning spaces (CALSs) are technologyenhanced learning environments which are typically constructed to support informal learning in contexts such as museums, parks, festivals, cities and galleries. Specifically, CALSs combine the contextual resources in the real world with the virtual world so as to motivate and engage the learners to explore the environment in an interactive way. A CALS utilises context-aware technologies (e.g. sensors, positioning) to provide the learner with content based on for example the learner’s location, time of the day, nearby people and the learner’s previous activities. Failing to integrate the technologies into a CALS leads to disruption of the learning experience by the technologies. Technology integration refers to the process by which a technology is introduced to a classroom so that the teacher and the students can use it effectively for pedagogical purposes [1]. We have previously established a model of technology integration for CALSs [2]. The technology integration model provides the CALS designers with a tool to plan technology integration based on various requirements set by the context, pedagogy and design goals. In a previous study we evaluated technology integration of UFractions, a game-based CALS for learning fractions. In UFractions the players interact with leopards through storytelling and solve fraction challenges so as to assist a mother leopard to raise her cub. The previous evaluation of UFractions

conducted in South Africa and in Finland revealed disturbance factors that guide how to improve technology integration [3]. However, the evaluation design was not created for measuring technology integration but for investigating how a reverse technology transfer works (i.e. from South Africa to Finland). Thus we hypothesise that with a proper evaluation tool the results could be deepened. In this paper we present a tool for evaluating technology integration in CALSs. The evaluation tool is grounded on the technology integration model and a literature analysis on technology integration in classroom-based education. After presenting the tool, we use it to evaluate UFractions in the Mozambican context. Then, through discussion, we analyse the results of the evaluation so as to determine the tool’s suitability for evaluating technology integration in CALSs. There are two reasons as to why the proposed tool is justified. Firstly, in the context of formal education at schools, technology integration, its evaluation, reasons for its high failure rates and how the process could be improved have been researched abundantly (e.g. [1][4][5][6][7][8]) but technology integration in informal contexts has not received similar attention. As the importance of informal learning contexts is increasing due to the latest developments of mobile and context-aware technologies, it is clear that the need for technology integration and its evaluation in these contexts is also increasing. Secondly, the previous technology evaluation of UFractions was not designed for measuring technology integration, hence calling for a dedicated tool for a deeper evaluation. II. BACKGROUND A. Context-aware learning spaces (CALSs) Context-aware learning is a fairly new concept in the domain of educational technology. It builds on the foundations of mobile learning (m-learning) in which the learners, equipped with portable handsets, have time and location independent access to learning resources [9]. One of the major limitations of traditional m-learning is that the surrounding context is not considered in the learning process, thus the learner’s attention is concentrated only on the mobile device’s screen. As soon as valuable contextual resources can enhance the learning experience, traditional m-learning becomes constrained. We

define contextual resources as context-dependent entities that can be detected by context-aware technologies. In contrast, context-free resources are not dependent on a given context (e.g. a theory or general knowledge of the topic). Context-aware learning is a subset of m-learning which integrates the contextual resources into virtual learning content. This means that the learner traverses a specific context while interacting with the surrounding environment. A mobile handset delivers context-sensitive instructions and tasks to the learner, and provides feedback based on the learner’s actions. Context-sensitiveness is achieved by context-aware technologies which include for example sensors and smart tags. A technical environment which enables and facilitates context-aware learning is called a context-aware learning space (CALS). Typically such an environment comprises a number of mobile devices (clients), wireless connectivity, a server, and a set of context-aware technologies. B. Technology integration model for CALSs The term technology integration refers to the process by which a technology is introduced to a classroom so that the teacher and the students can use it effectively for pedagogical purposes [1]. Technology integration is also important for CALSs as the designers may not have the needed technical know-how to choose and integrate technologies. The CALS development process is context-dependent and unique for each CALS instance, thus reflecting also on the technology integration process as the same set of technologies may require different approaches for different CALSs. We have previously created a technology integration model for CALSs which contains three categories of requirements that should be considered in the technology integration process [2]: context requirements, pedagogical requirements and design requirements. Each category has a critical factor which has a major impact on the category. These critical factors are availability of resources for the context requirements, unobtrusiveness of technology for the pedagogical requirements and contextawareness for the design requirements. Technology integration in CALSs can be divided into active and passive integration depending on the role of the technology in the process [3]. In active integration the technology integrates contextual and context-free resources into the CALS and makes the system adaptive to the changing context, including its users. In passive integration the technology is integrated into the CALS (and therefore into the context) so that it becomes unobtrusive to the learner and to the context. III. R ESEARCH DESIGN The research design of this study is illustrated in Figure 1. The following sections present research questions, research setting and research methods. A. Research questions The first research question (RQ1) of this study is “How can a technology integration evaluation tool for CALSs be constructed?” and it aims at creating a tool that can be

Fig. 1.

Research design

used by CALSs developers and users to evaluate the success of technology integration in a CALS. The second research question (RQ2) is “How well does the technology integration evaluation tool work?” and it aims at validating the tool for its intended use. These questions are answered by using the methods described in Section III-C. B. Research setting 1) Research platform: As the research platform in this study we use UFractions CALS which was originally developed for students on grade eight in South African rural middle schools. It features a story-based game on a mobile phone and a set of colourful fraction rods which are used to solve the challenges presented on the phone. The story is of two leopards, mother and her cub, and the player’s task is to help the leopards through solving fraction challenges. For each correctly solved challenge the player is rewarded points. The game has an introduction part, followed by three levels of varying difficulty of which the player can choose one or play all of them. In addition to the story, the game has a feature which allows the player to use the phone’s camera to record evidence of fractions from the real world and share this evidence with a comment on the game’s website. The game website also contains statistics related to players’ performance individually and collaboratively, and guest book entries that the players can submit at the end of the game. While the game can be played alone, we typically encourage students to play in teams of 2-4 in order to facilitate team work and serve more players with a limited set of resources (i.e. phones and fraction rods). Figure 2 illustrates a fraction challenge in UFractions and rods that are used for solving the challenge. A more detailed account on the design, implementation and features of UFractions is available in [10].

and ideas for future development. In addition to questionnaires and interviews, the researcher observed the participants during the game play and made notes on relevant events. C. Research methods

Fig. 2.

UFractions challenge for calculating distance to a new shelter

The technology of UFractions consists of client software installed on mobile phones (Nokia N95 and N80 in the tests) supporting J2ME MIDP 2.0, a Java-based server software on a PC, a website, fraction rods, and a wireless connectivity over a WLAN or a 3G network. The content is sent from the server to the clients in real time so it is necessary to have a reliable connection in between. During the tests we set up a WLAN access point to ensure smooth connectivity. The fraction rods are wooden and their lengths correlate with their colours. Each rod has been marked with a colour code (e.g. ’B’ for Blue) which are used in the game content. 2) Participants and the procedure: Evaluation data was collected in May 2011 at two locations in Maputo, Mozambique. First location was a Kids Club at Polana Secondary School (Polana). In a Kids Club the children are provided with an opportunity to apply and create novel information and communication technologies for learning [11]. All participants at Polana were Mozambicans thus Portuguese language was used. Three of the Kids Club instructors participated to the test as well. The second location was English speaking Maputo International School (MIS). We chose MIS so as to get a wider range of cultural backgrounds. The test participants (16 from Polana, 54 from MIS) filled in a questionnaire before and after the test. The pre-test part of the questionnaire collected demographics, mobile phone ownership and usage habits, as well as perceptions of games and mathematics. The post-test part consisted of five open questions and a series multiple choice questions with Likert scale or similar options. Open questions collected data on likes and dislikes, difficulties, surprising elements and suggestions for improvements. Multiple choice questions measured features and activities of the game, motivation, usability and clarity of the user interface, context-awareness (i.e. suitability of the CALS to the participant and to the context), availability of resources and overall experience. One to three participants from each group were interviewed by the researcher (11 in total). Interview questions gathered the participants’ opinions on game experiences, learning, advantages/disadvantages, likes and dislikes of the game’s features, applicability of the game outside the classroom, use of fraction rods, technology’s role in the game, technical problems, suitability of the game for the Mozambican context,

1) Methods for answering RQ1: The technology integration evaluation tool is based on the technology integration model which in turn was created based on a literature analysis and an artefact analysis [2]. A literature analysis was used to establish theoretical foundations for the technology integration model. In artefact analysis, with a goal of reaching a deeper understanding about an artefact and its usage than would be possible by mere direct observation, we explored several CALSs in various contexts to find out how they were designed and how they were used. Once the technology integration model was established it became clear that a technology integration evaluation tool is needed. Hence, we performed a literature analysis on technology integration in education and combined the results with the technology integration model. Data for the analysis were collected by searching articles related to technology integration in the classroom. 2) Methods for answering RQ2: The evaluation of technology integration is based on tests with the UFractions game. The tests involve participants aged 10-32 (average 13). We intentionally selected a broader range of participants than 8th graders so as to measure perceptions from different age groups. The evaluation tool utilises a mixed method approach but in this paper qualitative data categorisation and analysis were the key methods for finding disturbance factors for technology integration in UFractions. In categorisation we coded negative responses that the participants gave in open questions and interviews. Observations were also regarded. The coding was then used to identify the disturbance factors and to assign them to experience groups. IV. T ECHNOLOGY INTEGRATION EVALUATION TOOL A CALS can be seen as a constantly evolving system. The iterative process of CALS development is illustrated in Figure 3. The idea is that the first version of a CALS is placed under technology integration evaluation with the tool presented in this paper. The results of the evaluation inform the revaluation process which, by eliminating the problems discovered in the evaluation, increases the pedagogical and motivational value of the CALS. The resulting improved version of the CALS may become subject to devaluation which could happen for example when a technology breaks or becomes outdated. Devaluation is solved by revamping the CALS with a new technology, which in turn prompts a new evaluation. The technology integration evaluation tool is grounded on the critical factors of the technology integration model (see Section II-B) and it has been influenced by the TPCK (Technological Pedagogical Content Knowledge) framework [6]. The TPCK framework proposes that in a classroom context a competent teacher should have knowledge on pedagogy, content and technology. Koehler and Mishra suggest that

educator’s perspectives. V. E VALUATION

Fig. 3.

CALS development as a high-level iterative process

technologies have specific affordances and constraints in the integration process [6]. Affordances are enabling features of an object or an environment that allow an individual to perform an action [12]. Constraints, on the other hand, are a limiting force, setting restrictions to the use of technology. Critical factors of the technology integration model, affordances and constraints can be observed from the viewpoints of the learner, the educator and the context. Based on these components we have formed relevant evaluative questions of which the questions for the learner’s viewpoint are presented in Table I. We cover here only the learner’s viewpoint because the evaluation section of this paper concentrates on the learners. These questions are to be used as a starting point for creating data collection instruments. For example, the question “How do the learners perceive the technology?” could be answered by asking the learners’ opinions on and experiences with the technology (e.g. mobile devices) as part the CALS. TABLE I L EARNER ’ S ROLE IN THE EVALUATION Unobtrusiveness of technology Availability of resources

Contextawareness

Affordances Constraints

TOOL

Learner How good is the user experience of the CALS? Does any of the used technologies distract the learner? How do the learners perceive the technology? (or do they perceive it at all?) Do the learners afford using the system (if not free)? How does the CALS take into account the learner’s available time resources? Are the learners able to use the technology efficiently? What kind of connections can the CALS create between the learning content and previous experiences of the learners? How does the CALS take into account the learner’s personal context (e.g. location in a room, previous knowledge, preferences)? How does the CALS take into account the social context of the user (e.g. other learners)? How does context-awareness take into account the learner’s cultural background? How do the features of the CALS facilitate learning? How do the features of the CALS restrict/prevent learning?

The evaluation tool also measures general perceptions of the CALS. This data includes likes, dislikes, suggestions for improvements, motivation and applicability to other contexts. These aspects can be used to evaluate the attractiveness of the CALS as a learning tool both from the learner’s and the

Although the evaluation tool covers the roles of the learner, the educator and the context, in this study we only evaluate UFractions from the learner’s viewpoint. Furthermore, we report only the results regarding to disturbance factors which were acquired by analysing the qualitative data from questionnaires, interviews and observations. Table II presents the identified disturbance factors (22) with indications that map the representative evidence to the factors. The factors relate either to active (A) or passive (P) integration and they are grouped by the learner’s areas of experience which are affected by the disturbance factors. The term ZPD refers to Vygotsky’s Zone of Proximal Development [13]. VI. D ISCUSSION Technology integration has been a widely discussed topic in the domain of formal classroom-based learning but in the domain of informal learning, particularly in context-aware learning, the issue has not received similar attention. Contextaware learning spaces provide new ways of learning by combining contextual resources with learning content. While a CALS can provide highly interactive and engaging learning experiences, the technical complexity might lead to issues of badly integrated technology. To alleviate the challenges with technology integration we have previously established a technology integration model and in this study we proposed an evaluation tool based on the model. Both the model and the evaluation tool are novel approaches to scrutinise technology integration in CALSs from a holistic perspective. If we compared the results of this study (22 disturbance factors) to those of the previous evaluation (16 disturbance factors) [3], we can find all but one previously identified factors in the current results. The size of the data sets used in this study (70) was significantly smaller than the combined data set used in the previous study in South Africa and Finland (209), thus we had less qualitative data to work with. Furthermore, the participants in South Africa and in Finland were strictly 8th graders whereas in Mozambique the game was played by 6th and 8th graders as well as Kids Club members, thus making the data set more heterogeneous (in addition to a variety of nationalities). There was also a high number (75) of significant correlations (equal to or above 0.5) between quantitative statements of the questionnaire in Mozambique. This informs us of the good quality (triangulation) and the depth of the data. In contrast, in the South African data set the number of significant correlations was 8 and in the Finnish data set it was 29. These results indicate that the proposed tool yielded more accurate results with a smaller data set, thus suggesting that less time is needed for improving a CALS with the tool. The evaluation conducted in this paper considered only the learner’s role, thus leaving the educator’s and the context roles for a later study. Therefore, based on the discussion above, we can only confirm that the evaluation tool performed

TABLE II D ISTURBANCE FACTORS IDENTIFIED BY THE EVALUATION

Area of experience Temporal experience

Disturbance factor Too long game Too game

Learning experience

short

Beyond ZPD Below ZPD Wrong group

age

Lack of scaffolding

Immersion experience

Conflicting content Too much story Monotony

Social experience

Emotional experience

Too educational Harassment

Lack of peer support Disturbing content Punishment


Lack of animation Inappropriate graphics Inappropriate sounds


Inconvenient interaction with rods Inconvenient interaction with phone Technical faults Small screen Unclear instructions

TOOL

Indication

I

Evidence

References to a long game or a suggestion to make it shorter References to a short game or a suggestion to make it longer References to difficulty of challenges References to easiness of challenges Suggestion to use the game for younger players References to getting stuck

A

“The game is very big. It must have been a bit shorter” (Male, 13, Indian)

A

“I thought they could have a bit...maybe a bit longer the game.” (Male, 12, Mozambican)

A

“There were some fractions that were difficult to solve.” (Male, 13, Mozambican) “For learning purpose maybe you should make it a little harder but as a game it is ok.”, (Male, 12, Indian) “Maybe it would be better for younger kids because it’s this story of two leopards, so it would be from 8 to 11.” (Female, 13, Indian)

Conflict between own idea and game’s idea References to too long story or too much reading References to repetition or monotony of the content References to the game being too pedagogical Group members disturbed game play

A

References to lack of support from peers References to shocking or disturbing events in the content References to dislike on getting questions wrong References of lack of animation or suggestions to add them References to poor graphics or suggestions to improve them References to poor sounds or suggestions to improve them References to negative experience of using the rods References to negative experience of physical handling of or properties of the phone

A

P

“One thing that I really didn’t find that much interesting was using the phone. That wasn’t that much fun but I think that’s all really.[...] There were buttons and everything. I think it would be easier if you use something like maybe a calculator or something.” (Male, 11, Mozambican)

References to technical problems during playing References to small screen size or difficulty to see the content References to unclear tasks or difficulty of understanding them

P

“Once it turned down...it quit by itself but then we were on track again.” (Male, 11, Irish)

P

“The phone’s screen was a bit too small so I couldn’t see.” (Female, 11, Korean)

A

“I didn’t like some parts because I didn’t quite understand some questions. Like about four questions but the rest was ok.” (Male, 12, Mozambican)

A A A

A A A A

A A

“Sometimes when you were doing a question and you keep on not understanding I think there should be like where you can go to the next question if you can.” (Male, 11, Mozambican) “I was surprised because I had some answers that I was sure were correct but somehow they were wrong” (Male, 11, English) “Too much reading and after a while it gets boring” (Female, 13, Mozambican) “A part that I didn’t like was that it was always about leopards. If we had lots of settings with maybe gorilla and rhino we could all learn the lives of lots of animals which shows you lots of different fact. (Male, 11, Irish) “It was nice but the thing is like it’s not something I wanna do on a weekend or something. Maybe if you’re bored...” (Male, 12, Indian) “The thing was that two people would play it so one person would just take the phone and the other person will take it. The other person would have taken it and I couldn’t have read so that was sort of a disadvantage. (Female, 11, Korean) “Disadvantage is that maybe no one would be there to explain to you” (Female, 13, Indian) “The story of Senatla is not very good because the father of Senatla did not care for Senatla. Senatla was living with her mother...” (Female, 17, Mozambican) “[I disliked] When we got questions incorrect” (Male, 11, Mozambican)

P

“I’d just say more animations into the story, kind of hide the fact that it’s about fractions. [. . . ] (Male, 12, Indian)

P

“The screen was a bit too...all the colours around it and...it kind of...not too many colours but all the colours around it were kind of distracting. It could be one plain colour maybe.” (Male, 11, Irish”) “Make it more lively with sound” (Male, 13, Mozambican) “If you’re gonna improve it, maybe you should like...let’s say if someone has troubles reading it you should have voice over” (Male, 12, Indian) “I wouldn’t advise to use them because sometimes they make it complicated.” (Female, 15, Mozambican)

P P

adequately for evaluating technology integration from the learner’s perspective. We have prepared qualitative instruments for interviewing teachers and school representatives in order to complete the evaluation of technology integration but these will be applied in a future study. Based on this study we have useful information on how the game could be improved to meet the expectations of the learners. The next step is to perform a technology integration revaluation (see Figure 5) in which the identified disturbances will be diminished. It may be impossible to completely eliminate the disturbances because of the heterogeneity of the learner population. A set of methods for the revaluation process is yet to be established and it is out of scope of this study. After applying the evaluation tool successfully to UFractions, the big question is: how does the tool support evaluation of other CALSs in other contexts? The evaluation tool was designed to be generic in terms of viewpoints (learner, educator and context) as each informal/formal learning experience has a set of learners who are the primary users of the system, an educator who is responsible of the pedagogical goals, approaches and possibly the content, and the context itself which may have various resources to be utilised by the CALS. Furthermore, the questions presented in Table I are generic as well, thus being applicable to any learning situation. Data collection instruments of this study can be applied with minor modifications to other CALSs but the parts related to features, context and subject matter should be changed to correspond the target CALS. Finally, it is only through performing evaluations on other CALSs that we can verify the effectiveness of the evaluation tool in a larger scale. VII. C ONCLUSION We have introduced a tool for evaluating technology integration in context-aware learning spaces. The roles on which the tool is based cover viewpoints of the learners, the educators and the context. We consider the role division mandatory as influences of the technology may be perceived in different ways from different perspectives. Additionally, the role division grants us the possibility to prioritise the evaluation work and this is exactly what we have done in this paper – only the learner’s perspective was targeted in the evaluation of the UFractions game. The results of the evaluation proposed two major findings: (i) there are various disturbances, related to both active and passive technology integration, in UFractions when applied to the Mozambican context; (ii) the evaluation tool provides deeper results with a smaller data set than an evaluation done in South Africa and in Finland with other instruments. The appropriateness of the evaluation tool is based on the following facts: it considers all major roles who/which are affected by or who/which affect a CALS; it revealed more disturbance factors with a significantly smaller qualitative data set than the previous study; there is a higher number of evaluation metrics aimed at measuring technology integration than in the previous study; high number of significant correlations between the statements of quantitative data indicates interdependency links and triangulation of the data.

We have now reached one milestone in the process of creating a comprehensive set of tools for the entire technology integration process. Technology integration model, which was established before, is used in the planning, design and implementation phases of a CALS, and the evaluation tool is then applied to evaluate the effects of the technology, both in active and passive roles. The evaluation informs the CALS developer which aspects of the CALS need to be improved. The next part of our long term research agenda is to validate the evaluation tool’s generalisability against other CALSs in various contexts and then derive revaluation instruments to diminish the identified disturbances. Even before a revaluation method is established, the CALS developers can start a revaluation process by in case-by-case manner. The outcome should then be tested with another round of technology integration evaluation to ensure the desired results. ACKNOWLEDGMENT We would like to express our gratitude to pupils and teachers who kindly accepted our request to play the game. This study was partly funded by Academy of Finland. R EFERENCES [1] P. Ertmer, “Addressing first- and second-order barriers to change: strategies for technology integration,” Educational Technology Research and Development, vol. 47, no. 4, pp. 47 – 61, 1999. [2] T. H. Laine, C. Islas Sedano, M. Joy, and E. Sutinen, “Critical factors for technology integration in game-based pervasive learning spaces,” IEEE Transactions on Learning Technologies, vol. 3, no. 4, pp. 294 – 306, 2010. [3] T. Laine, E. Sutinen, M. Joy, and N. E., “Technology integration in context-aware learning spaces,” Manuscript in preparation, pp. 655 – 657, 2011. [4] T. Levine and R. Wadmany, “Teachers’ views on factors affecting effective integration of information technology in the classroom: Developmental scenery,” Journal of Technology and Teacher Education, vol. 16, pp. 233–263, 2008. [5] N. Strudler and K. Wetzel, “Lessons from exemplary colleges of education: Factors affecting technology integration in preservice programs,” Educational Technology Research and Development, vol. 47, pp. 63–81. [6] M. Koehler and P. Mishra, Introducing Technological Pedagogical Knowledge, ser. The Handbook of Technological Pedagogical Content Knowledge for Educators. Routledge, 2008. [7] H. Becker, “How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools,” Journal of Research on Computing in Education, vol. 26, pp. 291–321, 1994. [8] A. Staples, M. Pugach, and D. Himes, “Rethinking the technology integration challenge: Cases from three urban elementary schools,” Journal of Research on Technology in Education, vol. 37, pp. 285–311, 2005. [9] B. Eschenbrenner and F.-H. Nah, “Mobile technology in education: uses and benefits,” IJMLO - International Journal of Mobile Learning and Organisation, vol. 1, pp. 159 – 183, 2007. [10] E. Turtiainen, S. Blignaut, C. Els, T. H. Laine, and E. Sutinen, “Storybased ufractions mobile game in south africa: Contextualization process and multidimensional playing experiences,” in Proceedings of the Second Workshop of Story Telling and Educational Games, 2009. [11] P. J. Eronen, I. Jormanainen, E. Sutinen, and M. Virnes, “Kids’ Club Reborn: Evolution of Activities,” in Proceedings of ICALT 2005 Conference. Kaohsiung, Taiwan: IEEE Computer Society, July 2005, pp. 545–547. [12] Wikipedia, “Affordance,” http://en.wikipedia.org/wiki/Affordance. [13] L. Vygotsky, Mind in Society: The Development of Higher Psychological Processes, M. Cole, V. John-Steiner, S. Scribner, and E. Souberman, Eds. Harvard University Press, 1978.

Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences Hassinen, Minna. Thermal responses of inward and delayed rectifier potassium channels in fish cardiac myocytes, 2010. Huang, Xiang. Water quality in the Tibetan Plateau: Chemical evaluation of the headwaters of four major Asian rivers, 2010. Ukishima, Masayuki. Prediction and Evaluation of Halftone Print Quality Based on Microscopic Measurement, 2010. Tiilikainen, Raisa. The e↵ects of selective harvesting on life history traits of moose Alces alces, 2010. Lipponen, Pasi. Elasto-Plastic Modelling Approach for Paper Cockling and Curling Phenomena, 2010. Pradhan, Surendra. Yield and quality of vegetables fertilized with human urine and wood ash, 2010. Korhonen, Jenni. Antibiotic Resistance of Lactic Acid Bacteria, 2010. Poutiainen, Hannu. Tools for improved efficiency and control in wastewater treatment, 2010. Piipponen, Samuli. Kinematic analysis of mechanisms using computational algebraic geometry, 2010. Matikka, Hanna. The e↵ect of metallic implants on the RF energy absorption and temperature changes in tissues of head: A numerical study, 2010. Xiaoguang, Qi. Studies on value sharing for derivatives and di↵erence operators of meromorphic functions, 2010. Laukkanen, Janne. Fabrication of metallic micro- and nanostructures for optical solutions, 2010. Vilhunen, Sari. UVC irradiation based water treatment: A study of UV light emitting diodes, atomic layer deposited TiO2 and novel applications, 2010.

Myllymaa, Sami. Novel micro- and nanotechnological approaches for improving the performance of implantable biomedical devices, 2010. Myllymaa, Katja. Novel carbon coatings and surface texturing for improving the biological response of orthopedic implant materials, 2010. Jantunen, Paula. Role of Sorption in the Ecological Risk Assessment of Xenobiotics, 2010. Yang, Congli. Studies on warphi-Bloch functions and di↵erent type operators between Qk spaces, 2010. L¨ ahivaara, Timo. Discontinuous Galerkin method for time-domain wave problems, 2010. Matilainen, Juha. The regulation of inflammatory cytokines by vitamin D, 2010. Jetsu, Tuija. Modeling color vision, 2010. Faubert, Patrick. Responses of Non-Methane Biogenic Volatile Organic Compound Emissions to Climate Change in Boreal and Subarctic Ecosystems, 2010. Hyv¨ arinen Heikki. Studies on Partial Coherence and Electromagnetic Optics of Metamaterials, 2010. Karvinen, Petri.

Applications of resonant waveguide gratings,

2010. Liukkonen, Mika. Intelligent methods in the electronics industry – Quality analysis of automated soldering, 2010. Lehmuskero, Anni. Metallic thin film structures and polarization shaping gratings, 2010. Heikkil¨ a, Janne. Models for elastography-based monitoring of focused ultrasound surgery and diagnostics, 2011. Aula, Antti.

Computed Tomography and Ultrasound Methods

for Simultaneous Evaluation of Articular Cartilage and Subchondral Bone, 2011.

Janhunen, Matti. Phenotypic and genetic variations in reproductionassociated traits in Arctic charr, Salvelinus alpinus L, 2011. Kamppuri, Minna. Theoretical and methodological challenges of cross-cultural interaction design, 2011. Marttila-Kontio, Maija. Visual data flow programming languages: challenges and opportunities, 2011. Heikkinen, Ville. Kernel methods for estimation and classification of data from spectral imaging, 2011. Nissinen, Antti. Modeling errors in electrical impedance tomography, 2011. Vatanen, Tero. Multi-leaf Collimation of Electron Beams with Monte Carlo Modelling and Dose Calculation: Special Reference to Dosimetry and Build-up Dose, 2011. Saeidi, Rahim. Advances in front-end and back-end for speaker recognition, 2011. Nivalainen, Ville. Pre-service teachers’ objectives, challenges, and experience of practical work, 2011. Mohaibes, Mohammed. Treatment and hygiene of farm slurry and food waste, 2011. Viren, Tuomas. Arthroscopic ultrasound imaging of articular cartilage, 2011. Karjalainen, Janne. Novel Pulse-Echo Ultrasound Methods for Diagnostics of Osteoporosis, 2011. Haapio, Topi. Improving e↵ort management in software development projects, 2011. Tuomainen, Marjo. Search for molecular mechanisms related to Zn accumulation and tolerance in Thlaspi caerulescens, 2011. Gr¨ ohn, Janne. Results on complex di↵erential equations in the unit disc, 2011. Li, Xiao-Min. Uniqueness results of di↵erence operators and shifts of meromorphic functions, 2011.

Luste, Sami.

Anaerobic digestion of organic by-products from

meat-processing industry: The e↵ect of pre-treatments and co- digestion, 2011. Kuivalainen, Kalle. Glossmeters for the measurement of gloss from flat and curved objects, 2011. Kaakkunen, Jarno. Fabrication of functional surfaces using ultrashort laser pulse ablation, 2011. Niskanen, Heidi.

Modelling of fibre orientation in contracting

channel flows and in the jet-to-wire impingement, 2011. Obraztsov, Petr. Nonlinear optical phenomena in graphene based materials, 2011. Saarelainen, Markku. Teaching and learning of electric and magnetic fields at the university level, 2011. Andriyashin, Alexey. Non-negative bases in spectral image archiving, 2011. Kontturi, Ville. Optical measurements of complex liquids, 2011. Salonen, Anneli. Boreal unifloral honeys: screening of composition and properties, 2011. Karpov, Evgeny. Efficient Speaker Recognition for Mobile Devices, 2011. Tiihonen, Tuija. Information System in Context: Building a tool for Analysing the Sociotechnical Context of Organisational Information Systems, 2011. Egigu, Meseret. Secondary compounds of Yeheb (Cordeauxia edulis): production under abiotic stresses and their defensive role against phytophagous insects, 2011. Luukkonen, Jukka. Insights into Cancer-related E↵ects of Electromagnetic Fields, 2011. Roivainen, P¨ aivi. Characteristics of soil-to-plant transfer of elements relevant to radioactive waste in boreal forest, 2011. Haapala, Jaana. Mire plants and carbon dioxide dynamics un-

der increased tropospheric ozone concentration and UV-B radiation, 2011. Vartiainen, Ismo. Polarization control and coherence-polarization mixing by sub-wavelength gratings, 2011. Laine, Teemu. Technology intergration in context-aware learning spaces, 2011.

Context-aware learning spaces (CALSs) are mobile-based learning environments which utilise contextual resources, such as real world objects, in the learning process. This dissertation presents the development of two technical platforms on which ten CALSs were created in 2007-2011. Based on the development experiences, a model and an evaluation tool for technology integration in CALSs are proposed. These results, both practical and theoretical, can be utilised by developers to create CALSs in which technologies have been integrated effectively so that they do not disturb the learner.

Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences ISBN 978-952-61-0621-2 ISSN 1798-5668

dissertations | No 59 | Teemu H. Laine | Technology Integration in Context-Aware Learning Spaces

Teemu H. Laine Technology Integration in Context-Aware Learning Spaces

Teemu H. Laine


Publications of the University of Eastern Finland Dissertations in Forestry and Natural Sciences

Technology Integration in Context-Aware Learning

Technology Integration in Context-Aware Learning

Suggest Documents

A contextaware ubiquitous learning ... - Wiley Online Library

technology enhanced learning integration into ... - VDU talpykla

technology enhanced learning integration into ... - VDU talpykla

Technology Integration Assessment Rubric123 - Learning Activity Types

Scholarly Approaches to Learning Technology Integration in a ...

Integration of Computer Technology in Teaching and Learning Models ...

Technology integration applied to project-based learning in science

Scholarly Approaches to Learning Technology Integration in a ...

Barriers to Technology Integration in a Technology

System Integration in Information Technology

Technology Leadership and Technology Integration

Technology integration in education in ... - Flinders University

Technology integration in education in developing ...

Integration of Technology Enhanced Learning within Business ... - Eric

Integration of Claymation Technology as a Learning ...

the integration of technology and mathematics learning - CiteSeerX

Learning to Teach with Technology: From Integration to ... - CiteSeerX

A Simulation-Game on Technology Integration for Mathematics Learning

Technology Integration Continuum- TEACHING

Technology & Integration - DMEC

Self-Regulated Learning in Technology Enhanced Learning ...

UNDERSTANDING TECHNOLOGY INTEGRATION IN ... - UQ eSpace

System Integration in Information Technology - An Intermediation ...

Integration of technology components in cladding ...