Code Hunt – "Hunting" to Learn Programming

Code Hunt – "Hunting" to Learn Programming Sanja Maravić Čisar*, Robert Pinter*, Petar Čisar** * Subotica Tech-College of Applied Sciences, Subotica, Serbia ** Academy of Criminalistic and Police Studies, Belgrade-Zemun, Serbia [email protected], [email protected], [email protected] Abstract—Programming is a major subject in Computer Science (CS) departments. However, students often face difficulties on the basic programming courses due to several factors that cause these difficulties. The most important reason may be the lack of problem solving abilities that many students show. Learning to code can be made more effective if it is perceived as fun by the learner. Code Hunt uses puzzles that players have to explore by means of clues presented as test cases. Players iteratively modify their code to match the functional behavior of secret solutions. The aim of this paper is to introduce an educational, browser-based coding game which can help in learning coding.

I. INTRODUCTION Programming is a complex mental activity that is defined as an abstract process. Understanding and visualizing abstract processes poses a considerable problem for students when learning programming, as well as other fields with similar characteristics. Linn and Dalbey [1], [2], define an ideal chain of the learning process of learning programming and suggest it as a standard for comparing programming teaching methods. The three links of the chain are: • characteristics of the programming language – in order to log the programming solution of the problem with the given language, the student needs to understand the syntax, semantics, and expressive possibilities of the language. • the skill of forming the program is the knowledge to use a bundle of techniques which, applied and combined, are used to solve the given problem. The skill is based on the knowledge of stereotypical code samples which combine different characteristics of the language. The models implement complex functions, such as sorting, finding the lowest common denominator of two numbers, counting words in a given text, etc. Programmers design the language and model characteristics to be combined, problems to be decomposed into parts, solving every part independently, then linking the partial solutions into a unique unit – the program. At the end of the program writing process its correctness is checked by testing. • the general problem solving skills come to light during learning new formal systems and this is set as the goal to be achieved by studying programming. The same models and procedural skills are common to many, or even all formal systems. Therefore, this approach is used to learn models of logging with one formal system and the rules of transfer to the new one, the subject of the learning process, so it results in mental learning and activation of pre–existing knowledge.

Lemut et al. [3] explain the difficulty of learning programming by the need for implementing complex activities that have to be mastered even by beginners simultaneously. For example, the program is tested by executing it, using carefully selected input marginal values that will result in checking all program paths. The choice of the marginal values requires the knowledge of semantic program instructions. Opposed to this, the beginner programmer learns instructions, so they find it very difficult to choose such inputs on their own. Du Boulay [4] finds that the sources of difficulties are the following: • orientation – the general idea of students about programming and the program. • abstract engine – understanding the computing model that defines the program language. • notation – syntax and semantics of the language. • structures – knowledge of programming constructions as a composition of instructions with which certain program requirement are met. • pragmatics – skills implemented in creating the correct program (planning, decomposition, coding, testing, detecting and fixing errors). The aim of this paper is to introduce Code Hunt, an educational, browser-based coding game which can help in learning coding. The remainder of this paper is organized as follows. In Section 2 a brief introduction to term gamification is given, as well as some examples of gamification in business. Pex 4 Fun, the predecessor of Code Hunt is described in Section 3, and Section 4 gives an overview of Code Hunt. Section 5 provides some final conclusions and directions for future work. II. GAMIFICATION Learning to code can be made more effective and sustainable if it is perceived as fun by the learner. Gamification [5] is the use of game–play mechanics for non game applications. The origin of the term “gamification” is the digital media industry. The book “For the Win: How Game Thinking Can Revolutionize Your Business”, cites the authors Werbach and Hunter in the following quote: “the first use of gamification in its current sense apparently occurred in 2003, when Nick Pelling, a British game developer, established a short-lived consultancy to create game-like interfaces for electronic devices.” [6] The above–mentioned Werbach and Hunter give the following definition to gamification: “the use of game elements and game-design techniques in non-game contexts” [7]. The game elements are divided into these three groups: dynamics, mechanics and components, which are presented in Figure 1.

Figure 1.

Hierarchy of game elements [8]

Four examples are cited in the book [7] by Werbach and Hunter for the application of gamification in business: Nike+, Zombies RUN, Language Quality Games and Club Psych. The aim of Nike was to develop a device small enough to fit into a shoe and thus to track each step during the run. The device will calculate the speed and distance completed and relays it wirelessly to a Smartphone or PC. All collected data are fed to the application so as to make running more of a game-like venture. The application Nike+ includes functions such as the distance and speed that one has run, which is the longest distance of running, the top times, as well as other tracking data. One can make comparisons between the results related to previous times, and to define future goals and challenges. If the set goals are successfully completed, the user (i.e. the runner) will receive either a trophy or a medal. This way one can race their friends or be supported by them. Since the application’s launch in 2006, there are about 7 million runners using the platform. Another application also designed for running is "Zombies RUN!” According to the application, there is an army of zombies chasing the runner constantly while sitting at home and playing a game on the Smartphone, but not while running outside. The runner has the option of running from the zombies or be caught by only to have his/her brains eaten by them. Using GPS data and accelerometer information, the user (i.e. the runner) is given in-game instructions to follow so that they continue their run and pick up some virtual supplies for the use in the game. These supplies will comprise the player’s home base leading to new missions to be completed in a future running mission [9]. Game mechanics are applied by both, Nike+ and Zombies RUN! in order to create a more fun and engaging running experience. Another institution that used gamification was Microsoft, when they tested Windows 7 operation system in 36 languages. They needed to test dialogue boxes in all those languages, thus a competition was for teams made up of their employees in the various countries. They encouraged these employees to participate in the game called “Windows Language Quality Game” and make use of their mother tongue so as to detect possible errors in the system. According to the game rules, the “players”

received a dialog box in their mother tongue and they had to check if the dialogue box contained any mistakes. In case it did, the box was to be marked as bad, otherwise it was to be considered correct. In this way it was achieved that employees of Microsoft voluntarily combed through all boxes and the bugs were detected. Every team’s results were presented on a joint leader board. It was in the interest of the individual offices to show that their office was, in fact, the best at language localization, which led to a competition. This means that Microsoft managed to turn a rather boring task into a challenging game. The results achieved this way were considerable: 4500 of Microsoft’s employees were willing to spend their time trying to find bugs for free, the overall amount of checked dialog boxes was more than 500,000, with 6700 bugs reported. It is worth noting that the game did not culminate in some sort of award, instead, the game–like structure itself ensured that it was fun and enjoyable, making people willing to participate [10]. Club Psych is gamified website which was set up by USA Network, US cable channel involving one of their popular shows with the title “Pych”. The website’s chief aim was to involve viewers more in the show, therefore the site implemented a number of game elements, including rewards, challenges, badges, leader-board, points and avatars. The viewers who were involved could gain points, which eventually led them to a reward. There were badges to be earned for watching the series. This means that if the viewers wanted to be successful in this game, they had to stay connected to the show. The new gamified website acted as a sort of continuation to the series “Psych”. The results of gamification in the case of this TV show are presented below [10]: • there was an rise in overall traffic on the USA Network site by 30% • online merchandise showed a 50% rise • The page was visited by 130% more viewers • Facebook proved a very efficient sharing tool, with “Psych” content shared 300,000 times, reaching 40 million users Based on their research, the team Deterding et al. offered a definition for “gamification”: implementing game design elements in non-game contexts. They distinguish between serious games, video games designed for a purpose other than entertainment, and gamified systems, systems that use game design elements for a purpose, without the creation of fully–fledged games [11]. Gamification is defined by Zichermann and Cunningham as “The process of game-thinking and game mechanics to engage users and solve problems” [12]. Education is another field where gamification was successfully implemented. Researchers conducted a research at the University of Ulster in the United Kingdom to implement gamification mechanics with an obligatory computing course in the first year of studies. Implemented elements were, among others, a points-based system, immediate feedback and optional challenges. The rate of students passing the exam showed an increase from 82% to 95%, which was a rise of 13 percentage points. They conducted the same study again in the following semester,

but for a different academic course, and it showed a similar effect: the rate of students who failed the exam showed a decrease from 25% to less thann 10% [13]. A set of researchers at Old Dominionn University in the United States used a social networrk, with the title “socialPsych”, with all students who toook courses with the Psychology department during the sum mmer of 2010. The social network contained typical featuures, e.g. profiles, comments, walls, etc, but apart from thhis, it also included gamification mechanics. More specificaally, these features included a certification (badge) system, for students to do optional quizzes so as to be called ‘expeerts’ on a course, as well as a mentoring system, which ennabled students to complete certifications that qualified them to become mentors to their fellow students [14]. The results gained through this expperiment showed a positive tendency. A total of 385 studennts registered onto the system, out of them 113, which is 29%, completed optional tests and each student achievedd 4.8 certifications on average. In broad terms the studennts’ reactions were positive, they enjoyed the gamification mechanics, it was fun and rewarding for them. Further, reesearchers saw that students showed more engagement in i courses where participation in socialPsych was optionaal, they contributed to a greater number of discussion threaads [14]. This lent support to the assumption that gamificcation showed the highest effectiveness when it is offeredd as an option, so that the ‘players’ could choose if they wished to participate or not. OR FUN III. BACKGROUND-PEX FO Code Hunt uses puzzles that playerrs have to explore by means of clues presented as test caases. Code Hunt is based on the test/clue generation of Pexx, a white–box test generation tool that uses dynamic symboolic execution.

Pex [15] is an automatic white–boox test generation tool for .NET, based on dynamic symboolic execution. This tool is integrated into Microsoft Visual Studio in the form of an add-in. It can generate test inputs which are combined with different unit testing frameworks [16]. They have implemented Pex in classsroom teaching at various universities (for example Norrth Carolina State University, University of Illinois at Urbana-Champaign, U and University of Texas at Arlingtonn), and also in a variety of tutorials both within Microsoft (such as internal training of Microsoft developers) and outside Microsoft (such as invited tutorials at .NET userr groups). Further, they have created numerous open source research extensions upon Pex [17]. One of the most important methoodologies that Pex supports is called parameterized unnit testing, which broadens the scope of today’s industrry practice which prefers closed, traditional unit tests (i.e., unit test methods without input parameters) [16]. There are useful characteristics thhat Pex offers to support for testing. Primarily, there is the option of exploring code and suggesting the tessts that should be done. Secondly, assuming that it is a parameterized p test, Pex can determine the combination of parameters p that has to be tested so as to provide all feasiblle versions. Lastly, once Code Contracts is being usedd, Pex uses that

information to fine-tune the unnit tests that are offered or generated for the user [18]. IV.

CODE HUNTT, AN OVERVIEW

Code Hunt is an educationnal coding game that was released in May 2014 by Miccrosoft Research. The play scenario is the following: the sttudent will impersonate the code player, whose task is too detect the missing code fragments. Points are gained by b completed level, while extra points are granted for elegant solutions. The game can be implemented for eitheer C# or Java. During the game, the player will make thheir way to specific sectors and gain knowledge abouut arithmetic operators, conditional statements, loops, strings, search algorithms and more. The entire gaming exxperience has an underlying automated grading engine bassed on dynamic symbolic execution. The grading engine automatically analyzes the user code and the secret code so that the result table is generated. Most modern broowsers including Internet Explorer 9, 10, 11 and recentt versions of Chrome and Firefox are suitable for running Code Hunt [19].

Figure 2. The start scrreen of Code Hunt

Code Hunt resembles mosst games in the sense that there are sectors and levels. Code C is written in an editor window by the player, they usse either C# or Java as the programming language. The task is for the code to implement a given formula or algorithm – indicated by a top-level function, the “Puzzlle”. The function presents input parameters, then providess a result. The player needs to test if their task was succcessful, i.e. if the target algorithm was in fact appropriiately implemented. This is done by hitting the enormous “C CAPTURE CODE” button. This button will trigger a chain of events [19]:

1. the code is sent to a serveer in the cloud 2. the server compiles the code c (including an optional Java-to-C# conversion) 3. the server starts an in-deepth analysis of the code, it is compared with the goal algorrithm. 4. the player will then be presented with the results (Figure 3). The result will be either a compilation error, with information in the bottom panne, or some mismatches or agreements with the goal algoriithm. The code is presented left in Figure 3, while on the right there are the a agreements (yellow mismatches (red crosses) and checkmarks).

V.

Figure 3. The Code Hunt showing test t results

The player has completed this level successfully if the code has compiled and no mismatches were found, only agreements with the goal algorithm. Inn the game this is described as the player having “CAPTU URED!” the code, here seen in Figure 4.

Figure 4. After solving the puzzle, the plaayer gets a score

As a follow–up, the each mismatcch and agreement with the goal algorithm is returned in thhe form of a tuple (input, actual result, expected result). Inn the case when the player’s code is in agreement with the goal g algorithm, the actual and expected results are idenntical, while, if a mismatch has occurred, they are differrent. The player is given a chance to examine the mismatcches and figure out how to modify and improve the coode so a greater resemblance is achieved with the goal allgorithm [19]. Upon the completion of a given levvel, the player will be prompted to choose a somewhat more difficult level, so as to keep up the flow. Also, all sectorrs bar the first one are locked as their default setup. Only when w the player has completed a satisfactory number of leevels in a certain sector does the next sector become unloccked (Figure 5).

Figure 5. The game’s sectoor

CONC CLUSION

Games are an integral com mponent of people’s daily lives. Over $20.5 billion weree spent on video games by Americans in 2013 [20]. By 2014, 2 Gartner [25] predicts that over 70 percent of thhe Forbes Global 2,000 organizations will have att least one “gamified” application, and that “gamiffication is positioned to become a significant trend in the next five years” [24]. While the games’ primary aim is to present entertainment, given how universally applicaable they are, games have long–since outgrown their enteertainment role. Games are implemented not only to providde entertainment, but also in the fields of defense, educatiion, scientific exploration, health care, emergency mannagement, city planning, engineering, religion, and politics. Apart from entertainment games there are also the so–called serious games whose primary goal is i to train, investigate, or advertise. Games are ideal learnning environment with their built-in permission to fail, enccouragement of out-of-box thinking, and sense of control [21], but “games can make learning so much fun that theyy mask the large amount of learning required to play them successfully” s [22]. The term gamification doess not refer to the creation of a game from scratch. Insteadd, the term indicates the process of transforming educcation into a fun process, showing students it can be engaging, e yet maintaining credibility. Gamification is an aide for increasing student motivation, leading them to inncrease their coding skills, ultimately they will be stimulaated to learn. Gamification can actually have a boostingg effect leading them to study/read more [23]. Code Hunt is a highly efffective educational gaming platform that internally leveragges fitness values so as to guide the user to successsful test/clue generation furthermore, it also externally provides its users with an entertaining learning experiencces where search-based test generation is manually emulaated. Code Hunt is a new attempt to create a game using u serious search-based testing, make it available forr a huge group of users, including coders, and especiallyy learning coders [19]. The test cases are constantly changing, since they are built on w enjoy a new experience the mined data, thus players will every time. Learning to codee can become increasingly efficient and sustainable if the learner sees it as ‘fun’ as opposed to ‘learning to code’. Code Hunt was created annd presented to the wider public as recently as May 20144. While it was difficult to present any results of any experiments and research experience in the relatively shorrt life-span of this program, the authors still believe thatt this application can be successfully used for educationnal purposes. Naturally, the next steps will include thorouggh analysis and conducting experiments with students, so that the results of possible advantages and disadvantages of teaching and learning computer science via social gaaming can be analyzed. In paper [26] Pex4Fun, which iss the predecessor of Code Hunt, was experimentally usedd in learning Visual Basic. The positive results of this expeeriment are an encouraging introduction to the use of Code Hunt in learning programming, leading to futuree experiments regarding this coding game.

REFERENCES [1]

[2]

[3]

[4]

[5]

[6] [7] [8] [9] [10] [11]

[12]

[13]

[14]

M.C. Linn, J. Dalbey, “Cognitive Consequences of Programming Instruction, in Studying the Novice Programmer”, Editors Soloway E, J. C. Spohrer, Lawrence Erlbaum Associates, Publishers; Hillsdale, New Jersey; 1989. p. 57-81. M.C.Linn, J. Dalbey, “Cognitive consequences of programming instruction: Instruction, access, and ability.” Educational Psychologist 20.4,1985, pp. 191-206. E. Lemut, G. Dettori, and B. Du Boulay, “Cognitive models and intelligent environments for learning programming”, SpringerVerlag New York, Inc., 1993. B. Du Boulay, T. O’Shea, J. Monk, “The Black Box Inside the Glass Box. In Studying the Novice Programmer”, Editors Soloway E, J. C. Spohrer, Lawrence Erlbaum Associates, Publishers; Hillsdale, New Jersey; 1989. p. 431. S. Deterding, R. Khaled, L. Nacke, and D. Dixon. Gamification: Toward a Definition, CHI 2011 Gamification Workshop Proceedings , Vancouver, BC, Canada http://www.quora.com/Who-coined-the-term-gamification K. Werbach, D. Hunter. “For the win: How game thinking can revolutionize your business”. Wharton Digital Press, 2012. http://blog.panictank.net/2525-for-the-win/ http://www.gamification.co/gbase-entry/zombies-run/ http://keepexploring.hubpages.com/hub/Wharton-Gamificationexamples-in-business S. Deterding, D. Dixon, R. Khaled, L. Nacke, “From game design elements to gamefulness: defining gamification.” In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9-15. ACM, 2011. G. Zichermann, and C. Cunningham, Gamification by design: Implementing game mechanics in web and mobile apps. O'Reilly Media, Inc., 2011 T. Charles, D. Bustard, and M. Black, “Experiences of Promoting Student Engagement Through Game-Enhanced Learning”. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious Games and Edutainment Applications, 2011, pp. 425–445. R.N. Landers, and R.C. Callan, “Casual Social Games as Serious Games: The Psychology of Gamification in Undergraduate Education and Employee Training”. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious Games and Edutainment Applications, 2011, pp. 399–423.

[15] http://research.microsoft.com/projects/pex/ [16] N. Tillmann, J. de Halleux, and T. Xie, “Pex for Fun: Engineering an Automated Testing Tool for Serious Games in Computer Science”, March 2011, TechReport, MSR-TR-2011-41. http://research.microsoft.com/pubs/147143/pexforfunengineering.pdf [17] http://pexase.codeplex.com/ [18] D. Esposito, “Pex: Microsoft Research's Unit Test Generator and Evaluator”, November 26, 2012. http://www.drdobbs.com/testing/pex-microsoft-researchs-unittest-genera/240009056 [19] N. Tillmann, J. Bishop, R. Nigel Horspool, D. Perelman, and T. Xie, “Code Hunt: Searching for Secret Code for Fun”, ACM ICSE Workshop on Seach Based Software Testing 2014, http://research.microsoft.com/pubs/210651/CodeHunt%20SBST% 202014b.pdf [20] http://www.forbes.com/sites/davidewalt/2013/12/19/americanswill-spend-20-5-billion-on-video-games-in-2013/ [21] K. M. Kapp, “The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education”. John Wiley & Sons, 2012. http://books.google.ee/books?id=M2Rb9ZtFxccC&printsec=frontc over#v=onepage&q&f=false. [22] N. Whitton, Learning with Digital Games. A Practical Guide to Engaging Students in Higher Education. Routledge, 2010 [23] C. I. Muntean, “Raising engagement in e-learning through gamification”, The 6th International Conference on Virtual Learning ICVL 2011, pp. 323-329, http://icvl.eu/2011/disc/icvl/documente/pdf/met/ICVL_ModelsAn dMethodologies_paper42.pdf [24] Á. Szeghegyi, V. Szoboszlai, and J. Velencei, “Informal PostExperiential Learning”. Acta Polytechnica Hungarica, 11(4), 2014, pp. 241-252 [25] Gartner Gamification Report 2011, http://badgeville.com/wiki/Gartner_Gamification_Report_2011 [26] S. Maravic Cisar, R. Pinter, P. Cisar, & D. Radosav, Teaching Computer Science in a web-based environment. In Intelligent Systems and Informatics (SISY), 2013 IEEE 11th International Symposium on IEEE, pp. 415-418.