On Gamification in Action Learning

Copyright. Citation Information: Noran, O. (2016) On Gamification in Action Learning, Proceedings of the Australasian Computer Science Week Multi-conference (ACSW 2016), Canberra, Australia, Article 15, ACM Digital Library, pp. 1-9

On Gamification in Action Learning Ovidiu Noran Griffith University Nathan Campus Nathan QLD4111 AUSTRALIA +61-7-37355382

ABSTRACT The advent of globalisation brought about by advances in information and communication technology has triggered major changes in the way people work, live and study. The modern teaching endeavour must meet the needs of a fundamentally changing learning environment and student cohorts, while preserving delivered knowledge quality in order to meet the required learning objectives. Flexible teaching (distance/online, intensive mode, afternoon classes etc.), online resources and tools, teachers with industry experience and especially new teaching models matching the new cohorts profile requirements can provide a solid platform for a new paradigm in the higher education domain. This research has investigated how a customised application of ‘gamification’ (taking game-based elements and applying them in non-gaming contexts) to higher education can improve the learning and teaching experience and student engagement and thus help towards reducing student attrition. Gamification of the educational endeavour has been studied in the past; however, in this study it is applied within a novel iterative and combined Action Research, Experiential Action Learning, Plan, Implement, Review, Improve and Plan, Do, Study, Act approach that attempts to make contributions to both learning and teaching theory and practice.

Categories and Subject Descriptors • Computing Education

Keywords Action Learning; Gamification; Action Research

1. INTRODUCTION

often must study additional degrees at a mature age in order to keep up with employer demands, re-enter the workforce or switch employment areas. These requirements are reflected in the evolution of student cohort profiles and their learning needs and habits, as people from increasingly varied backgrounds and age brackets are entering higher education. These students often have to sustain themselves and sometimes also their family, which reduces the time available for tuition, especially travel required for face-to-face interaction. As information is fast becoming ubiquitous, the capability to recall on short notice large pieces of memorised knowledge is increasingly replaced by the ability to find and filter accurate and relevant data for the task at hand. Another aspect is the reluctance (sometimes accompanied by a certain incapacity) to engage in a mainly one-sided and theoretical learning exercise by the new cohorts; nowadays, the new generation of students expect that learning should be engaging and practical with real life-type tasks and rather take the form of discussions and negotiations with the teacher, in a collaborative exercise [1]. They expect learning material to be free, available anyplace, anytime and to reflect the state-of-the-art in the domain so they are ‘job-ready’ upon graduating. Unfortunately however, new technology is also highly distracting, presents the danger of over-reliance on infrastructure (e.g. mobile networks, ‘the Cloud’) and may create a perception that hard work and commitment are no longer required to gather the required knowledge; this mindset often results in superficial learning, disappointment and high attrition rates [2, 3]. While this is particularly true of the first year of tertiary education, when students experience a significant change in learning and teaching style from high school to university, student retention is affected throughout all programs.

The current global dynamic and competitive economic environment enabled by fast evolving technologies requires companies to become agile so as to readily and continuously adapt in order to stay abreast of competition. As a result, the current and future workforce needs to acquire an increasing amount and variety of skills, in a shorter time span. Another emerging global issue is population ageing, which is putting significant pressure on healthcare and social services worldwide. Governments typically react to this challenge by extending the retirement ages; as a result, people need to work for longer and

In order to suitably prepare the future generations for the challenges that lay ahead, the teaching endeavour must match the fundamental changes experienced by the learning environment and the student cohorts, without compromising the quality and quantity of knowledge provided to the students [4]. Flexible teaching (distance/online, intensive mode, afternoon classes etc.), online resources and tools, teachers with industry experience and especially new teaching models matching the new cohorts profile requirements can provide a solid platform for a new paradigm in the higher education domain.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACE-ACSW’15, February 1–2, 2015, Canberra, ACT, Australia. Copyright 2010 ACM 1-58113-000-0/00/0010 …$15.00.

The research described in this paper has investigated the practicality of a teaching model involving the ‘gamification’ (taking game-based elements and applying them in non-gaming contexts [5]) of computing courses in tertiary education so as to improve learning and teaching experience and student engagement and thus assist in reducing attrition. Gamification of the educational endeavour is not a new proposal; however, this study applies the concept in a novel iterative and combined Action Research (AR), Experiential Action Learning (AL) [6], Plan,

DOI: 10.475/1234

Implement, Review, Improve (PIRI) and Plan, Do, Study, Act (PDSA [7]) approach (see Figure 1).

Exit the cycles Enter cycles

Identify Improvements Iterations (Act) Body of Knowledge

Action

Critical Reflection (Study)

Implement (Do), Participate

Plan (Plan): design / change approach

Figure 1. Combined AR / AL / PIRI / PDSA approach In line with the ‘action’ character present in some paradigms belonging to the set of research methods adopted, the author has been directly involved in the application of the practical research deliverables; therefore, he has sought to clearly identify his research stance and biases, including those specific to learning and teaching [8]. The research has involved two cycles, each involving a critical literature review, identification of the problems and changes necessary, making the required changes, teaching using the modified material and approach, observing the effects and gathering data and finally reflecting on the results obtained and proposing further changes. The main focus of each research cycle spanned over one semester of teaching, although elements of gamification have been present in previous offerings of the courses involved, albeit in a less systematic manner. Due to space limitations, the scope of the current paper is limited to the description of one cycle.

2. GAMES AND GAMIFICATION Games, challenges and competitions are a part of the human character and thus constitute a significant aspect of human society. Companies have long tried to leverage this fact and use games towards engaging clients or encouraging employees to work together more effectively [9]. Game concepts and mechanics (e.g. [10-12]) have been and still are being used in a variety of areas, including military, marketing, health, business and education, with varying degrees of success. The advances of Information and Communication Technologies have provided a solid platform for the development of an evolved games industry and for the encouragement of games research. As with any emerging area of research and practice, there is still debate and divergence in regards to the domain ontology- for example, the true meaning of the term ‘gamification’[13-15]. Generally, the term is perceived as describing the application of game concepts in various other contexts in order to encourage and motivate problem solving and better performance using typical human tendencies towards competition, achievement, status, altruism, self-expression, etc. [16]. Jenkins et al [17] propose a distinction between the terms ‘game’ and ‘gaming’: the first facilitates learning by playing games, while the second involves

the use of game concepts and social, technological, theatrical, virtual reality etc. knowledge. The more recent technology advances have enabled the creation of digital distribution, rights management, multiplayer, and communications platforms featuring community features (friend lists and groups, ‘Cloud’ game settings saving and sync with local devices, in-game voice and chat functionality and so on). Importantly, some such platforms (see e.g. Steam for Schools [18]) may feature ‘altered’ functionality that can be used in other domains (e.g. educational); likewise, ‘modified’ games can be used for teaching science and critical thinking, role playing etc. (see e.g. Portal 2 [19]). Gamification takes specific forms depending on the application area. For example, in the business or military areas gamification is represented by so-called ‘serious games’, which are generally used for training and as such are very specific and work-related [20, 21]. In marketing, game concepts can take the shape of rewards, loyalty points and virtual currency [22] that can be used towards future purchases - thus reinforcing the gaming behaviour. In education, ‘gamified courses’ attempt to engage students [23] e.g. by developing games in the process of learning. The concept of ‘learning games’ is used in academia to “foster habits of mind and understanding” leading to the acquisition of knowledge [20] and can be used in formal, informal or self-teaching learning environments. Gamification as a new area displays typical ‘teething’ issues; thus, there is criticism in relation to the misuse of the gamification concept resulting in a false sense of achievement [5], omitting the storytelling element, lack of game developer involvement, erroneous or narrow implementation (‘points-ification’ [24]) and flawed business research results confusing those unfamiliar with the gamification concept [25]. Such criticism also emphasizes the potential danger of gamification detracting from, or hindering the objectives rather than assisting in their achievement. Therefore, the rationale for the context and the use of gamification must be carefully considered [15] and reflected in the components of the game design ‘schema’, such as e.g. espoused by Salen and Zimmerman [26] – namely the rules, play and especially culture. Gamification is easier to implement where the users know the system or work in the environment in question (see Microsoft’s gamified environment for testing system usability, or SAP’s gamified data entry processes and online community network [27, 28]). The above-mentioned research shows that gamification is clearly not a trivial or generic ‘one size fits all’ exercise. It has to be accomplished in a tailored manner, appropriate to the specific environment (industry, education, military, etc.) and audience type (e.g. considering age, background, culture and so on). Gamification has the potential to revolutionise human competencies and human-system interfaces; however, “the hard work – strategizing – hasn’t been done yet” [29].

3. GAMIFYING LEARNING AND TEACHING AS AN INNOVATION EXERCISE As gamification of the learning and teaching environment is a relatively new and original concept, the innovation acceptance attributes identified by Rogers [30] can be used in order to determine avenues to increase the approval of this endeavour by students and higher management, resulting in broader championing and wider application of the concept.

Thus, Rogers’ first attribute is Relative Advantage, i.e. how a gamified approach is ‘better’ than previous approaches. In education, gamification may translate in an improved student learning experience and motivation reflected in better retention and student experience reports. However, there are many factors influencing retention rates, not all of which are related to the use of a specific learning and teaching approach. In addition, there is significant debate around the accuracy and indeed, the appropriateness of the assessment tools used to gather such information [31-33]. While a trend of increased engagement of students resulting in the improvement of student experience and retention may develop, generally one should not expect immediate results but rather progressive improvement. The second attribute is Compatibility, which here appears to apply to what needs to be taught, values and prior experience: i.e. gamification concepts have to be assimilated into the teaching material and students’ comprehension. In the author’s view, this aspect requires deep knowledge of the course material and gamification on the part of the teacher so that appropriate gamification concepts are matched to suitable course sections, at the appropriate time. Compatibility may also refer to the shared technology used to provide the gaming experience - in this case, competing and often incompatible technologies. The third attribute is Complexity. ‘Gamifying’ a course should not increase its complexity, but rather lower it by providing a learning curve and smaller, easier to achieve sub-goals that lead to feelings of achievement and investment resulting in students staying enrolled in the course and successfully completing it. Trialability or Replayability (fourth attribute) can be used to reduce the uncertainty surrounding the acceptance of gamification concepts and also to provide self-training opportunities. Thus, positive experiences in trials using gamified learning objects are highly likely to increase the acceptance of this strategy. Observability, the last attribute listed by Rogers, appears to relate in this context to the results of prior gamification attempts, where previous positive experiences would encourage stakeholders to embrace the concepts proposed. Feedback gathered from past ‘gamified’ offerings can be used towards this aspect. The acceptance of gamification may also be assisted by the previous exposure of the target audience to gamification concepts. For example, widely used gamification-based marketing strategies may result in students’ previous exposure to specific artefacts such as reward points, achievement badges, virtual currency etc. Technology can facilitate the introduction of gaming concepts in education; however, it demands appropriate teacher training in order to maximize the usefulness of the devices [34, 35] and poses an additional acceptance challenge for educators and educational institutions.

4. IDENTIFYING POSSIBLE IMPROVEMENT ACTIONS BASED ON GAMIFICATION CONCEPTS This section of the paper attempts to build a pool of gamificationbased potential improvements that can be applied to higher education learning and teaching. The action part of this research has applied a selection of improvements from the abovementioned collection, matching the specifics of the target courses. As earlier stated in the paper, literature shows that an important cause of poor student retention rates is the lack of engagement. This can be caused by several factors, some of which can be

addressed by gamification. For example, students that like their program, but whose motivation is low or it has withered over time could be a target for gamification-based student engagement interventions. On the other hand, students who have joined a program they were not attracted to because of poor advice (e.g. by parents or advisors), low job prospects, lack of career alternatives etc. are not likely to respond to such changes in course strategy. The following subsections identify avenues of innovative intervention leading to potential improvements in the learning and teaching effort, based on the analysis of gaming concepts identified during the critical literature review. Figure 2 summarizes the gaming concepts identified while at the same time highlighting potential dependencies, which may be useful in their deployment within learning and teaching practice.

Game State Actions, Events Storytelling

Replay / Infinite play

Investment

Engagement Rewards

Sub-goals Progression / Skill

Fulfilment

Figure 2. Relations between various gaming concepts

4.1 Good Storytelling: Rules, Actions, Events Storytelling featuring well-thought sequences of events requiring various actions [10] is an essential feature of every successful game. Similarly, an engaging course should be able to clearly articulate through its published profile (and by its teaching team) what the course is about, the knowledge it provides, its place in a program or degree and the actions and events and ‘achievement’ (knowledge) level required in order to successfully complete it. The profile is the public ‘image’ of the course presented to the potential students and the basis for their decision to select it for a program of study. Importantly, it is also the ‘contract’ between the student and the institution stipulating the obligations (‘game rules’) that the two parties must abide by. Notably, good storytelling should extend to course content and its assessment items; students should clearly understand the way a particular lecture, exercise or assessment item contributes to the overall level of and type of knowledge required in their future career. Students also must develop an understanding of what constitutes ‘quality’, and how to evaluate it [36]. Unfortunately, based on the review of a significant number of course profiles, often this is not the case. In the case studies embedded in the research cycles storytelling has been used in order to more accurately reflect course contents, learning objectives and achievement level requirements.

4.2 Student Engagement: Competition or Cooperation? Engagement through competition and cooperation (with variations such as time, space and team constraints) is a concept well established in games [5, 15, 23, 37, 38]. However, competition (involving individuals or groups of students) must be used with caution by teachers as it may bring unwanted shortcomings that outweigh the benefits; thus, it can increase attention on ‘what it takes to win’ at the expense of learning as a goal in its own right. In addition, competition in team situations may determine group

members to see each other less as peers in a learning community and more as the means to win [39]. Dialogue and reflection may be reduced or disappear altogether. There is also debate over ‘healthy vs. unhealthy’ competition [40], whereby in a healthy competition the valuable goals are the learning rather than the competitive ones and the competition is of short duration with no long term effects, with all participants understanding these features and having a fair chance of winning. The results of questionnaires administered in the research cycles have revealed a positive attitude of students towards competition. Cooperation may be a better alternative, encouraging dialogue and reflection and divergent ideas leading to novel ways to accomplish tasks. However, this approach also presents pitfalls; for example, some team members may rely on the others to do most of the work, leading to conflict, need for mediation and even expulsion of members, with disrupting effects on the team. In conclusion, both approaches are useful in the context of careful management. The onus is on the teacher to identify the best option (or a mix thereof) for each specific practical scenario.

4.3 Sub-Goals: Investment and Engagement through Fulfilment and Progression Goals and sub-goals are paramount concepts in a game. They have the role of keeping a player ‘invested’ and thus engaged in playing a game by providing a sense of gradual progression and fulfilment. In learning and teaching, such perceptions are essential to student involvement, engagement and ultimately, retention. A learning curve is also present, typically steeper in technical areas. Therefore, similar to the game concepts that include low and high stake goals, students should be provided with formative and summative assessment items featuring progressively increasing degree of difficulty. Progressive achievements will provide “the connection of a certain act with a certain situation and resultant pleasure” as espoused by Thorndike [41] based on his view of learning as the process of forming associations and bonds. Smaller, on-going assessment items are also important in order to provide a degree of self-assessment – fact acknowledged by students in the questionnaires used within the case studies with remarks like “the quizzes [on-going assessment] in this course provides a barometer of my knowledge level”. While selfassessment promotes beneficial student learning reflection [42, 43], it must be noted that summative assessment is also required because, similar to game situations, many students do not pay suitable attention to items that do not carry rewards (marks). This is particularly true of first-year and low performing students who are still transitioning from the high-school study model and / or may not have yet formed appropriate study habits. In-class small assessment items spread throughout the teaching period encourage students to focus, participate, spend time on task [44] and thus promote a consistent learning approach. Such items also provide appropriate and timely feedback on students’ knowledge level [45, 46]. Progression reflecting the skill of players is often accomplished in gaming environments using achievement badges, virtual goods that players can trade (or use toward purchasing other games), detailed statistics etc., with players typically being able to control the accessibility of such information (player profile) themselves. In the learning and teaching environment, such information is provided (e.g. through ‘running totals’, percentages etc.), albeit subject to significantly tighter privacy rules. Thus, marks and statistics for a student are typically visible just to the teacher,

admin personnel and student in question. Only overall statistics can be made public.

4.4 Replayability, Risk, Rewards, Game State Commonly, non-achievement of a sub-goal in a modern game does not translate in automatic failure; rather, it is limited to some temporary hindrance, activates an alternative scenario or allows another attempt; any of these options may also incur a penalty. By analogy, should a student should be allowed to repeatedly attempt assessment items with low or no penalty, especially when the assessment closely reflects the skills that students should have upon graduating? Many games typically provide (or game modifications are available for) penalty-free ‘training’ modes so that the player can hone their skills and achieve the desired proficiency level; however, these training modes do not apply to the entire game. In learning and teaching, this approach can be emulated by ‘risk-free’ formative partial assessment items, followed by summative over-arching assessments. In this context, an increasing trend in modern games is the replayability concept, whereby the player encounters different environments and actors in each game instance [47]; this concept can also be (and increasingly is) utilised in learning and teaching by the use of large question and scenario banks, randomly selected for each assessment item sitting. This approach can also be applied for summative assessments in order to prevent academic integrity concerns. Modern games allow saving the state of a game at critical milestones (‘checkpoints’) so that the player does not have to start from the beginning every time. This works towards maintaining the investment level high; in learning and teaching this can be achieved by recording, saving and restoring student progress e.g. in online tests used for formative assessment.

4.5 Reflection v. Observation: Depth v Agility In many multiplayer online games, latency is a crucial factor; thus, awareness of the situation enabling prompt reaction (agility) may often be the difference between a win or lose situation. However, even in such situations, the lack of extensive knowledge of the game universe of discourse (maps, strategy but also unintended game behaviour such as glitches) gained through many hours of play can make a significant difference. Thus, while low latency gives agility, thorough game knowledge gives depth of understanding; fortunately, the two aspects are not mutually exclusive but rather complementary. These concepts are highly relevant to learning and teaching. For example, using AR/AL cycles is a thorough approach involving a significant amount of time (typically several semesters or years). As a result, curriculum and teaching improvements eventuating from reflections on the results gathered during a specific iteration will only benefit subsequent offerings. On the other hand, observing class mood and participation during live lectures and workshops and while administering various assessments is a valuable rapid feedback tool at teacher’s disposal. For example, ‘secondary emotions’ analysed according to Russel’s [48] ‘circumplex model of affect’ (see Figure 3) can provide feedback that the teacher may promptly use to tune the teaching approach in real time e.g. by applying more or less emphasis on various gamification aspects. Such observations can successfully complement the AL approach: AL provides depth and method while observation provides agility.

recorded; ‘stacking’, i.e. building an overly skilled team (generally considered unethical) e.g. by using messaging and other ‘back door’ means. There are also objective factors that can distort statistics. Thus, several team members may leave in-game and be replaced with weaker or stronger players; this will reflect on the achievement of the entire team. Finally, the specific area (e.g. map) of a game and players’ knowledge of that area may also play a significant role in skewing the scores. Arousal

Valence

Figure 3. Secondary Emotions (based on [48]; arrows show desired trend)

4.6 Evaluating the Results in Gaming vs. Teaching & Learning This section aims to identify problems and potential solutions in achieving meaningful interpretations of data gathered in games, extrapolate the solutions and apply them to learning and teaching. Achievements and status are important reasons for playing games, be it single or multiplayer, off- or online. In recognition of this fact, games have always made some attempt to record player performance and typically use it to entice players to increase their investment and engagement. Thus, the competitive aspect of human nature is leveraged by providing rewards such as badges and / or statistics, skill levels etc. The sophistication of performance evaluation has steadily increased so as to match the growing games' complexity and IT infrastructure capability. Multiplayer technology and online gaming enabled by Cloudbased deployment and management platforms [49] are now affordable and accessible to any household with an average speed connection. Game state (configuration, levels played) and player information shown in statistics such as game playing achievements, level of assistance to the team, level of efficiency, preferred games and number of hours played in each, advanced skills (such as being a developer of game maps and artwork) etc. is now accessible irrespective of location and specific hardware. Players can retrieve their game state and player status and continue playing a game from where they left off. Evaluation is very important in games, even for those who do not care much for achievements levels. This is because many games and/or platforms feature match-making, where only players with comparable experience are allowed to play together. In addition, the status of a player determines the availability of bonuses (virtual cash, weapons, skills, etc.) and often, the rank and level of team acceptance within multiplayer games. One must note however that the evaluation mechanism used in games is prone to drawbacks, especially when attempting to interpret the results. For example, a player may misuse the evaluation mechanism by leaving a game running for long periods just to achieve ‘hours of play’ and a higher level; colluding with others to get specific achievements (e.g. win games, beat specific records etc.); leaving a game before losing so the loss is not

To summarise, it is quite hard to draw a conclusion in regards to the skills of a player just by looking at the statistics, as many subjective and objective factors may distort the information. What can be done to improve the situation? In order to see if a player really achieves good results one could a) get involved in games with the player (primary information) or b) seek the opinion of other players (secondary information), thus using an incremental evidence gathering method. Within the educational realm, evaluation is an important part of the PIRI (Plan, Implement, Review, Improve) cycle, the basis for many Frameworks for Quality Assurance and principles for excellence in learning and teaching (e.g. [50]). Unfortunately, the main vehicle currently used in many institutions to gather the necessary information, namely Student Experience of Course (SEC) and Teaching (SET), are a rather limited and often inaccurate quantitative reflection of the reality. Thus, SEC and SET are typically difficult to interpret [51] and present only one side of the story. Nulty [31] rightfully argues that typically, online surveys (widely used in tertiary education providers) achieve low response rates (even with various incentives and instructor encouragement); therefore, “relying heavily on student evaluations of courses and teaching is inadequate at best and misleading at worst” (ibid.). In addition, Yorke [33] points out that “[…] when students come together with differing prior experiences […], a variation is introduced into the responses that can be extremely difficult to identify”. These are important factors to consider especially for courses that must be taken (‘core courses’) by undergraduate and postgraduate students (many of which are international and from a non-English speaking background). Similar to the distortions identified in the games realm, variations due to the actual teacher (e.g. whether sessional of staff, mature or early career) and the maturity of the course offering material itself further complicate proper SEC and SET results interpretation. The open-ended sections of these reports may offer some insight as to the cause of the results distribution; however they are typically only filled in by a minority of respondents. Smith [52] proposes an SEC / SET evaluation model whose second stage supplies “an interpretive guidance system that helps lecturers (but also the executive) understand and interpret” the results; however, currently this concept does not appear to be properly implemented. One must also recognise that, similar to gaming evaluation mechanisms, SEC and SET are also prone to wilful manipulation, as scores typically important consequences (e.g. career progression) for the teachers involved. Therefore, supplementary or ‘easier’ assessment items, more lenient marking, etc. may potentially be used by teachers specifically to obtain better scores. Potential solutions resemble those investigated in the gaming realm. Thus, frequent informal feedback (verbal and written, in confidence) sought from the students before, during and after classes can help build ‘incremental’ evaluation (such as Nulty’s [32] ‘A-Z’ model), and increase course ‘agility’ by making on the fly adjustments to the teaching style. However, such feedback must be taken with a ‘grain of salt’ in the absence of anonymity

and the possibility of students ‘observing back’ and potentially attempting to please the teacher. The context of such feedback must also be carefully prepared so as to make it a non-threatening, engaging, and fun experience integrated in the learning and teaching conversation [53].

5. CASE STUDY Systems Analysis and Design (SAD) is a large course offered at three campuses of a major tertiary education provider. SAD is a second year core course that aims to impart knowledge of the various activities undertaken within each of the system development life cycles. For this reason, the course is a core (i.e. compulsory) in several IT programs and is also part of a set of ‘primers’ offered to non-IT background students undertaking IT and IT-related postgraduate programs. The course enrolment is typically large, with a mostly undergraduate cohort. The course has been offered for many years; the material is mature, the teacher highly knowledgeable and having industry experience.

5.1 Identified Areas of Improvement Reflection on the teaching practice and evaluation reports for this course has yielded several areas of improvement (the relevant gaming concepts are presented in italics): 

      

lack of early assessment and feedback for students. Students were not required to produce any assessed item until midsemester and thus did not get feedback and possible early warning in relation to their performance (sub-goals, risk); lack of continuous assessment so as to provide an on-going indication of their performance and a feeling of achievement and involvement (sub-goals, rewards, investment, fulfilment); lack of self-study, non–assessed, formative resources paced in parallel with the material (replayability, rewards, risk); insufficient practice in hands-on work and incentives to do so (sub-goals, rewards); insufficient explanation of the learning objectives and their relevance to the students’ future career (storytelling, game goals, culture); insufficient explanation of the assessment items and their relation to the learning objectives and future career (game rules, goals); insufficient explanation of the learning strategies and their relation to the learning objectives and future career (game rules, goals); insufficient interactivity (player, multiplayer, culture).

5.2 Proposed Changes In line with the gamification concepts identified as suitable as a result of the critical literature review, a range of changes were proposed as detailed below. Storytelling, Game rules, Goals: At the outset of the course, explain why the course is required – for students’ careers and also how it helps them to understand the more advanced courses. Go through the course outline together with the students and explain key areas such as Course Description, Aims, Learning Objectives, Assessment Plan and timing and how each assessment item will contribute to the Learning Objectives of the course. This helps student understanding and motivation (‘why do I have to do this course?’) and creates a perception of required engagement. The Learning Strategies are also to be explained and justified to the students in relation to the required graduate outcomes. Sub-goals, Risk: A low-points assessment item is introduced in the third week of study serving as a ‘wake-up call’ and also to

reinforce the requirement of continuous study. This is followed by a series of assessed, low-points quizzes, administered every 2-3 weeks to reinforce the need to keep studying and also to provide a feeling of achievement and encourage engagement. The risk associated with failing any individual quiz is low; however, repeated failure may lead to significant loss of points. Replayability, Rewards, Risk: Redesign the former sample assessment items so as to become self-assessment items. Thus, the solutions for these items are no longer to be released from the start; students need to solve them first. The solutions are also to be redesigned so as to contain explanations of ’why is it so and not otherwise’ (and thus promote the previously mentioned ‘dubito ergo cogito…’ approach), rather than a set of laconic statements describing the correct answers. Sub-goals, Rewards: Adequate hands-on practice (training) to be introduced via additional computer labs. They are to be spread over the period of study and cover a larger amount of topics, to the benefit of students’ practical experience. Students must attend these labs in order to progress through the course and send their work to the teacher for perusal. The above proposed changes are to be complemented by the introduction of ‘refreshers’ in the form of brief questions at the beginning of each class, relating to the material previously taught. ‘Jogging’ the students’ memory is likely to be more productive as they have to think of the answer, provide and defend it rather than passively looking on as they are shown the solutions. Game Frequent Asked Questions (FAQ): The answers to the most asked questions are also to be made available online, alleviating the requirement for the teacher to answer similar questions over and over and instead focus on material delivery quality and student engagement.

5.3 Teaching the Modified Structure The learning activities and assessment sections were revised in line with the proposed changes, using a conceptual development view of teaching [54] and keeping the emphasis on student development and making people understand [55]. As planned, storytelling was involved in order to explain students the purpose (goals) and expectations of the course (game rules). The learning and teaching strategy involved a balanced mix of the game collaboration and competition approaches in class discussions and debates, as well as in the negotiation of assessment marking criteria in order to alleviate large class alienation and loss of motivation syndromes [56]. In line with the game state perspective, in each lecture and workshop students were shown where they were in respect to the entire course. A diagram showing the areas of the main system development life cycles covered by the course was used for this purpose. This was preferred to text as due to a variety of factors such as language, culture etc., student perception and interpretation may have not reflected the intended meaning of teacher’s utterances. The lectures followed the progression of the SAD development life cycle phases, presenting and comparing the two main approaches (‘traditional’ and object-oriented) used in the industry. In line with the AL approach adopted, the students were encouraged to reflect and criticise the two approaches and ask themselves e.g. ‘what is different about them?’, ‘are they really that different?’ and ‘is any of them best, or just more suitable depending on the particular system being analysed and designed, and in view of the competencies of the people involved in the development project?’

The teacher’s experience, personal characteristics and integrity (player profile) as defined by Palmer [57] have been used to inspire students to learn as their beliefs essentially influence their development [58]. Storytelling was also used in course delivery, based on teacher’s vicarious experience of relevance [59], aiming to prepare students for real situations that sometimes involve ‘wicked problems’ requiring likewise competencies [60]. The workshops also used storytelling by following a fictitious company through its quest to (re)-develop some of its essential systems. All the concepts learned and discussed in the lectures were gradually applied to this scenario using a constructivist approach. In line with game sub-goals, fulfilment and progression aspects, students were constantly given analogies they could relate to and were encouraged to apply their previous experience(s) and newly gained knowledge to try to solve increasingly difficult problems and critically assess the solutions produced [61]. Computer labs provided functional [62] hands-on Computer Aided Software Engineering tool experience for students and (according with game sub-goals, rewards and replayability concepts) featured assessment items allowing multiple solutions which promoted creativity in model development. Large classes typically feature students with a variety of learning styles [63], moving up towards the ‘qualitative understanding’ [64] featuring Bloom’s content analysis and evaluation [65] at different paces. This situation was tackled by cycling through several teaching approaches featuring high and low complexity, practical examples and theory, moving ahead but periodically returning to test and deepen students’ understanding of the material: in essence, using appropriate storytelling to cater for a large and varied audience. Perry [66] found that students’ learning and knowledge perspective evolves from an absolutistic (objective / black and white) form to a ‘relativistic’ one where personal values and interpretations of the evidence prevail. In line with game progression and replayability notions, this perspective was taken into account by steadily increasing the complexity of the exercises (‘low-risk’ sub-goals) featuring multiple solutions and introducing debates as to why solutions are not unique and why there is no such thing as a ‘best’ solution for all scenarios. The above-mentioned low-risk assessment items were spread along the teaching period in line with the gaming sub-goals, risk and rewards and progression concepts. Sample assessment items were also created; their solutions were only disclosed after students attempted to solve them (according to the game training concept).

5.3.1 Gathering Feedback Keeping in mind the caveats of evaluation examined in the gaming analogies descried in Section 4.6, feedback has been gathered at the end of the semester but also in the middle of the teaching period using a variety of methods (survey, focus group, informal) so as to enable triangulation and at the same time to catch any early issues (see section 4.5) and correct them. SEC and SET reports have also been used, with specific focus on students’ answers to open-ended questions.

5.4 Reflection on the Feedback The feedback has reinforced issues previously identified in the literature review such as perceived advantage brought by gamification and need for compatibility with learning objectives. Many students, although previously exposed to games and gaming

concepts, were still between the persuasion and decision stage of acceptance of new paradigms and technologies [30]. The inclusion of free text fields in the questionnaire used has proven beneficial; many students wrote beyond the allocated space in an attempt to strongly make their point, thus confirming their engagement with the gamification concept. The majority of respondents believed that gamification can provide a more competitive edge to their future career and strongly endorsed the need for some competition elements in learning and teaching, as well as the need for taking controlled risks. The respondents also appeared to support the integration of gamification elements throughout the course; in addition, most declared themselves happy to participate in pilot studies. There was some inconsistency and polarisation among the opinions regarding learning objects and concepts that should not be part of a ‘wellgamified’ course; the likely underlying reason for this could be the novelty of the gamification style employed and the lack of proper reflection on the concepts proposed. Practical examples, reflecting the author’s industry experience and the state-of-the-art in SAD, appear to have kept students alert and interested; thus, student feedback has confirmed the importance of teacher’s practical experience [59] in inciting students interest and promoting deep learning [67]. Another aspect positively perceived by the students was the teacher’s willingness to stop at any time and explain more if students appeared puzzled. This approach requires the teacher to be the equivalent of ‘seasoned gamer’, so as to master the course material (game rules, maps etc.) well, be able to stop and resume at will and provide novel perspectives on the material. Should this not be the case (e.g. in the case of core courses taught by inexperienced sessional staff), the likely result would be a worsened learning and teaching experience. This issue can be addressed by proper course ‘match-making’: only sessional staff (players) with certain level of knowledge (gaming experience) may be allowed to teach courses (play games) of certain importance, size and complexity.

6. CONCLUSIONS AND FURTHER WORK This paper has presented the phases of a research cycle spanning a teaching semester, comprising the identification of potential improvements based on the learning and teaching and gamification bodies of knowledge, planning the required changes to a target course, implementation of the changes, teaching the gamified course, feedback gathering and reflection on the results. It appears that gamification does benefit students by increasing student engagement and motivation and helping them achieve the desired learning objectives. However, proper gamification requires researching, interpreting, customising and using relevant gamification principles in learning and teaching practice. Accordingly, in this research gamification has been employed in the context of a multi-pronged and iterative Action Research, Experiential Action Learning, Plan, Implement, Review, Improve and Plan, Do, Study, Act framework. An important aspect also true for gamification is that the quality of teaching can be enhanced by aligning objectives, teaching styles and milestone deliverables [68]; however, the research, implementation, action (teaching) and reflection leading to this alignment must be ongoing in order to ensure agility in the face of constant change in the industrial and academic environments. The research approach must be refined and tested in subsequent cycles in order to better assist the learning and teaching effort in its endeavour to prepare students for the global dynamic and competitive environment faced by companies and the workforce.

7. REFERENCES [1] Bovill, C., K. Morss, and C. Bulley, Should students participate in curriculum design? Discussion arising from a first year curriculum design project and a literature review. Pedagogical Research in Maximising Education, 2009. 3(2). [2] Angelino, L.M., F.K. Williams, and D. Natvig, Strategies to Engage Online Students and Reduce Attrition Rates. The Journal of Educators Online, 2007. 4(2): p. 1-14. [3] Torenbeek, M., E. Jansen, and A. Hofman, The effect of the fit between secondary and university education on first-year student achievement. Studies in Higher Education, 2010. 35(6): p. 659-675. [4] AQF Council. Australian Qualifications Framework. 2013; Available from: http://www.aqf.edu.au/Portals/0/Documents/2013%20docs/A QF%202nd%20Edition%20January%202013.pdf. [5] Deterding, S., et al., From game design elements to gamefulness: defining “gamification”, in MindTrek. 2011: Tampere, Finland. p. 9-15. [6] Kolb, D., Experiential Learning. 1984, Englewood Cliffs, N.J.: Prentice-Hall. [7] Deming, W.E., Out of the Crisis. 1986: MIT Center for Advanced Engineering Study. [8] Noran, O., 'Gamified' Action Learning Environments: Case Studies and Reflections. Master Thesis 2013. [9] DeLoitte. Gamification goes to work: Moving beyond points, badges, and leaderboards 2013 [cited 2013 Jun]; Available from: https://documents.deloitte.com/techtrends2013. [10] Björk, S. and J. Holopainen, Patterns in Game Design. 2004: Charles River Media. [11] Adams, E., Fundamentals of Game Design 2nd ed. 2009, Thousand Oaks, CA: New Riders Publishing. [12] Schell, J., The Art of Game Design: A book of lenses. 2008: CRC Press. [13] Marczewski, A., Foreword, in Gamification: A Simple Introduction & a Bit More. 2012, e-Book. p. 46. [14] Erenli, K. The impact of gamification: a recommendation of scenarios for education. in Interactive Collaborative Learning (ICL). 2012. Villach, Austria. [15] Kapp, K., The gamification of learning and instruction: game-based methods and strategies for training and education. 2012, San Francisco: John Wiley & Sons. [16] Van Grove, J. Gamification: How Competition Is Reinventing Business, Marketing & Everyday Life. 2011. [17] Jenkins, H., B. Camper, and A. Chisholm, From Serious Games to Serious Gaming, in Serious Games: Mechanisms and Effects, U. Ritterfeld, M. Cody, and P. Vorderer, Editors. 2009, Routledge: London. p. 448-468. [18] Wilde, T. Valve rolls out "Steam for Schools" to teach math and physics lessons with Portal 2 PC Gamer, 2012. [19] Valve Software. Teach with Portals. 2011 [cited 2015 August]; Available from: http://www.teachwithportals.com/. [20] Klopfer, E., S. Osterweil, and Salen, K. Moving learning games forward: The education arcade. 2009; Available from: http://education.mit.edu/papers/MovingLearningGamesForw ard_EdArcade.pdf. [21] Ritterfeld, U., M. Cody, and P. Vorderer, Serious Games: Mechanisms and Effects. 2009, London: Routledge. [22] Hamari, J. and V. Eranti, Framework for Designing and Evaluating Game Achievements, in Think Design Play: 5th International Conference of the Digital Research Association (DIGRA)2011: Utrecht, Netherlands.

[23] Muntean, C. Raising engagement in e-learning through gamification. in The 6th International Conference on Virtual Learning (ICVL 2011). 2011. Cluj-Napoca, Romania. [24] Robertson, M. Can’t play, won’t play. 2010; Available from: http://www.hideandseek.net/2010/10/06/cant-play-wontplay//. [25] Slavin, K. In a World Filled With Sloppy Thinking. 2011 [cited 2013 June]; Available from: http://slavin.tumblr.com/post/6353625142/in-a-world-filledwith-sloppy-thinking-this. [26] Salen, K. and E. Zimmerman, Rules of play - game design fundamentals, ed. T.M. PRess. 2004, Cambridge. [27] Yolton, M. Gamification: much more than just fun and games. 2013. [28] Schacht, M. and S. Schacht, Start the game: Increasing user experience of enterprise systems following a gamification mechanism, in Software for People: Fundamentals, Trends and Best Practices, Maedche et. al, Editor. 2012, Springer Verlag: Berlin. [29] Zichermann, G., My Take on Gamification, in Gamification goes to work: Moving beyond points, badges, and leaderboards 2013, DeLoitte. [30] Rogers, E.M., Diffusion of innovations. 2003, New York: Free Press. [31] Nulty, D., The adequacy of response rates to online and paper surveys: what can be done? Assessment & Evaluation in Higher Education, 2008: p. 1-13. [32] Nulty, D., Workshop 6 - Innovation and Evaluation, in 7014GIH: Curriculum Design, Innovation, Assessment and Evaluation – Innovation and Evaluation. 2011, Griffith University. [33] Yorke, M., ‘Student experience’ surveys: some methodological considerations and an empirical investigation. Assessment & Evaluation in Higher Education, 2008. 34(6): p. 721-730. [34] Glover, D. and D. Miller, Running with technology: the pedagogic impact of the large-scale introduction of interactive whiteboards in one secondary school. Journal of Information Technol for Teach Educ, 2001. 10: p. 257-278. [35] Smith, H., et al., Interactive whiteboards: boon or bandwagon? A critical review of the literature. J Comput Assist Learn, 2005. 21: p. 91-101. [36] Sadler, D.R., Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 2009. 34(2): p. 159-179. [37] Malone, T. and M. Lepper, Making learning fun: A taxonomy of intrinsic motivation for learning, in Aptitude learning, and instruction, R.E. Snow and M.J. Farr, Editors. 1987, Lawrence Erl-baum Associates: London. [38] Salen, K. and E. Zimmerman, Rules of play - game design fundamentals. 2004, Cambridge: The MIT Press. [39] Johnson, D.W. and R. Johnson, Conflict resolution, peer mediation and peacemaking, in Handbook of classroom management., C.M. Evertson and C.S. Weinstein, Editors. 2006, Lawrence Erlbaum Associates: Mahwah, NJ. p. 803832. [40] Shindler, J., Transformative Classroom Management: Positive Strategies to Engage All Students and Promote a Psychology of Success. 2009: Jossey-Bass. [41] Thorndike, E., Animal Intelligence: An Experimental Study of the Associative Processes in Animals, in Psychological Review, Monograph Supplements. 1898, MacMillan: New York.

[42] Boud, D., HERDSA Green Guide No 5. Implementing student self-assessment. 2nd ed. Vol. The Higher Education Research and Development Society of Australasia (HERDSA). 1991, Campbelltown. [43] Black, P. and D. William, Assessment and classroom learning. Assessment in Education, 1998. 5(1): p. 7-74. [44] Chickering, A.W. and Z.F. Gamson, Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, 1987: p. 3-7. [45] Newell, A. and H.A. Simon, Human Problem Solving, ed. Engelwood-Cliffs. 1972, Engelwood Cliffs, N.J.: Prentice Hall. [46] Newstead, S.E., The Use of Examinations in the Assessment of Psychology Students. Phsychology Teaching Review, 1992. 1(1): p. 22-23. [47] Booth, M. Replayable Cooperative Game Design: Left 4 Dead. 2009. [48] Russell, J.A., A circumplex model of affect. Journal of Personality and Social Psychology, 1980. 39: p. 1161-1178. [49] Breckon, N. Valve Announces Steam Cloud; Online Network to Store Saved Games Indefinitely. Shacknews, 2008. [50] Griffith University. Principles to Promote Excellence in Learning and Teaching Practices at Griffith University. 2010 [cited 2012 August]; Available from: http://www.griffith.edu.au/__data/assets/pdf_file/0006/12020 1/PrinciplesLandT.pdf. [51] Abrami, P.C., S. d'Apollonia, and P.A. Cohen, Validity of Student Ratings of Instruction: What We Know and What We Do Not. Journal of Educational Phsychology, 1990. 82(2): p. 219-231. [52] Smith, C., Building effectiveness in teaching through targeted evaluation and response: connecting evaluation to teaching improvement in higher education. Assessment & Evaluation in Higher Education, 2008. 33(5): p. 517-533. [53] Pask, G., Conversation, cognition and learning. 1975, New York: Elsevier. [54] Ramsden, P., Ways of understanding teaching (Chapter 2), in Learning and Teaching in Higher Education, P. Ramsden, Editor. 2003, Routledge Falmer: London. p. 19-38. [55] Edgerton, R., P. Hutchings, and K. Quinlan, The Teaching Portfolio: capturing the scholarship in teaching. American Association for Higher Edutation. 1991: Stylus Publishing. [56] Hejmadi, M.V., Improving the Effectiveness and Efficiency of Teaching Large Classes: Development and Evaluation of a Novel e-Resource in Cancer Biology. Bioscience Education Journal, 2007. 9. [57] Palmer, P.J., The heart of a teacher, in The courage to teach: Exploring the inner landscape of a teacher's life. 1998, Jossey Bass: San Francisco. p. 9-33. [58] Kember, A Reconceptualisation of the Research into University Academics' Conceptions of Teaching. Learning and Instruction, 1997. 7(3): p. 255-275. [59] Hodgson, V., Lectures and the experience of relevance, in The experience of learning: implications for teaching and studying in higher education, F. Marton, D. Hounsell, and N.J. Entwistle, Editors. 1997, Scottish Academic Press: Edinburgh. p. 159-171. [60] Knight, P. The assessment of 'wicked' competences. 2006; Available from: http://www.open.ac.uk/cetlworkspace/cetlcontent/documents/460d21bd645f8.pdf. [61] Paul, R.W. and L. Elder, Critical Thinking: Basic Theory and Instructional Structures Handbook. 2nd ed. 2000: Foundation for Critical Thinking.

[62] Leinhardt, G., K. McCarthy Young, and J. Merriman, Integrating professional knowledge: the theory of practice and the practice of theory. Learning and Instruction: , 1995. 5. [63] Larkin-Hein, T. and D. Budny, Research on Learning Style: Applications in the Physics and Engineering Classrooms. IEEE Transactions on Education, 2001. 44(3): p. 276-281. [64] Biggs, J. and K. Collis, Evaluating the Quality of Learning: the SOLO taxonomy. 1992, New York: Academic Press. [65] Bloom, B.S., Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. 1956, New York: David McKay Co Inc. [66] Perry, R.P., Perceived Control in College Students: Implications for Instruction in Higher Education, in Effective Teaching in Higher Education: Research and Practice, R.P. Perry and S.J. C., Editors. 1997, Agathon: New York. [67] Ramsden, P., Approaches to Learning (Chapter 4), in Learning and Teaching in Higher Education, P. Ramsden, Editor. 2003, Routledge Falmer: London. p. 39-61. [68] Biggs, J., Teaching For Quality Learning at University: What the Student Does (Society for Research into Higher Education). 2003: Open University Press.