Training Flexible and Adaptive Arithmetic Problem Solving Skills

Training Flexible and Adaptive Arithmetic Problem Solving Skills through Exploration with Numbers. The Development of NumberNavigation Game Boglárka Brezovszky, Erno Lehtinen, Jake McMullen, Gabriela Rodriguez, Koen Veermans Centre for Learning Research, University of Turku, Finland [email protected], [email protected], [email protected], [email protected], [email protected] Abstract Traditional teaching methods often fail to develop the desired type of arithmetic problem solving expertise that is, flexibility and adaptivity in problem solving. Working with numbers, and exploring different number patterns and solution methods could represent a good basis for developing a more flexible and complex understanding of numbers and operations, which could result in the development of flexible and adaptive problem solving skills. Our aim when developing NumberNavigation (NNG) educational digital game was to provide an engaging and adaptive context for exploration with numbers. In the game, players progress by strategically selecting different sequences of number-operation combinations, using the four basic arithmetic operations within the domain of natural numbers (1-100). Players need to collect raw materials for building settlements, by navigating their ship through different maps placed over the 100 square. The game challenges players to strategically select the most optimal path and continuously adapt their choices. In this explorative case study, the goal was to test core game features and their relationship with in-game arithmetic flexibility. A trial game version was tested in two consecutive sessions with three elementary school children. Methods of data collection included video recorded observation, open-ended interview questions, and the screen capture of gameplay. Aiming to trigger more variation in players’ use of numberoperation combinations, we have tested three different game modes. In exploration mode players had to choose their moves so they bypass islands placed over the maps as obstacles. In the minimum moves mode, players had to make their journey using the least amount of moves, and in the minimum energy mode players had to reduce the magnitude of numbers used for navigating their ship. Results show that players used various number-operation combinations and they were able to adapt their ingame problem solving strategies according to the changing rules of the game. The position of harboursislands-targets proved to be a strong predictor of the amount of variation in number-operation combinations and of players’ explorative tendencies. Overall, results suggest that basic game features are promising in triggering exploration with numbers and engaged practice over an extended period. Results of the study will be used to inform the future development of NNG. Keywords: adaptive expertise, arithmetic problem solving flexibility, game design, strategy Background Developing adaptive expertise in arithmetic problem solving is one of the main objectives of elementary school mathematics education. Due to the complexity of this skill, traditional instructional methods might not be sufficient in this process. Although, there are different views regarding the meaning of adaptive expertise, it is usually agreed that the development of fluent and flexible procedural skills together with a deep conceptual understanding of numbers and operations are both necessary requirements (Baroody 2003; Schneider, Rittle-Johnson & Star 2011; Verschaffel et al. 2009). Flexibility and adaptivity in arithmetic problem solving represent strong markers of arithmetic expertise. Flexibility refers to the ability to apply various strategies during problem solving while adaptivity describes the strategic choice of the most optimal type of solution for a given problem which may depend on various factors such as problem characteristic, individual differences in problem solving skills, or the norms and rules of the context where the problem is solved (Verschaffel et al. 2009). Research shows that while placing emphasis on the development and practice of problem solving strategies can enhance flexibility, and that students of various skill levels can acquire and use different problem solving strategies, in practice these skills are rarely transferred to new problems. Despite students’ familiarity with various methods they often disregard both the characteristics of the task and their individual strategy performance when selecting problem solving strategies (Torbeyns et al. 2009). Similarly, despite understanding when and how different problem solving strategies are more adaptive than others, students often fail to apply this knowledge in practice (Blöte, Klein & Beishuizen 2000). The discrepancy between the

ability to understand different problem solving strategies and the ability to apply them adaptively in new problem situations is often explained as a shortcoming of instructional methods that place too much emphasis on teaching how to solve problems, concentrating more on the surface features of a problem and less on the underlining principles. As the development of flexible and adaptive skills in arithmetic problem solving largely depends on the ability to understand numbers and operations, instead of learning to choose from a set of different methods, the focus should rather be placed on developing well-connected knowledge of numbers and operations (Baroody 2003; Threlfall 2009). By exploring and experimenting with different number combinations, children might develop a more richly connected conceptual understanding of numbers and the relations between different numbers and operations. Thus, important features and relationships between numbers in a problem might become more visible, and so more optimal solutions could be more readily recognized (Schneider, et al. 2011). Providing children with learning environments which trigger active exploration with numbers and reflection on problem solving processes, should enhance the development of stronger connections and the understanding of underlining relationships between number and operations and as a result trigger the development of flexible and adaptive problem solving skills (Baroody 2003; Verschaffel et al. 2009). For example, in their study Canobi, Reeve & Pattison (2003) examined 5-8 year old children’s understanding of commutativity and associativity principles in addition problems and found that irrespective of age, those children who understood that numbers can be decomposed and recombined had a higher accuracy and more flexibility in their problem solving procedures. In equation solving, Rittle-Johnson & Star (2007) found that reflection in terms of comparing and contrasting different solution methods proves to be an efficient method for noticing important features of a problem leading to the development of more flexible problem solutions and exploration of different alternatives. Additionally, there is evidence that using different mental models such as the hundred square or the mental number line might be beneficial for triggering the exploration of alternative solutions and the understanding of underlining principles between numbers and operations (Beishuizen 1993; Blöte, Klein & Beishuizen 2000). Games by their nature are ideal for triggering exploration and discovery and can provide a motivating context to experiment with numbers (Devlin 2011). Additionally, educational computer games can provide novel pedagogical approaches to deal with educational content that has been challenging for traditional teaching methods (Siewiorek et al. 2012). However, up to date there is no educational digital game with the explicit aim to trigger flexibility and adaptivity in arithmetic problem solving. Examples of games which aim to develop mathematical skills, and which build on a strong theoretical background in their game design providing empirical evidence of their efficiency are rare (Young et al. 2012). One relatively well-researched domain in the field of games and mathematics is represented by games that aim to develop early numerical understanding by strengthening the mental representations of numbers. The mental number line has been successfully used in developing both the spatial representation of numbers and basic arithmetic problem solving skills of young children (Kucian et al. 2011). Other game designs with similar aims focused on strengthening the connection between different representations (e.g. symbolic and non-symbolic) of numbers (Wilson et al. 2006; Räsänen et al. 2009). Although these games are exemplary for building upon strong behavioural and neuro-cognitive evidence in their game design, and provide adequate empirical evidence regarding the learning gains associated with gameplay, they are also highly specialized environments aiming to help young children with more severe mathematical difficulties (developmental dyscalculia). Arithmetics constitutes only a limited part of these games, and includes only basic addition and subtraction within a limited number domain. Furthermore, what might be a highly efficient game design for treating developmental dyscalculia in young children might not work for more various samples and mathematical skills. Unfortunately, in the domain of more advanced arithmetic skills, the amount of empirically tested games is even smaller. What is more, even existing games are often criticized for their inability to integrate their game design features with their mathematical learning aims (Young et al. 2012). A meaningful educational game design requires embedding the educational content within the core game features (Devlin 2011) in a way that it becomes an integral part of the gameplay instead of a necessary extra placed on top of a game (Habgood 2007). Until now, there has been only one example of an educational digital game with published empirical results that manages to create a fusion between educational content and game design, as well as

target the development of complex arithmetic skills such as the discovery and understanding of number patterns (Habgood & Ainsworth 2011). Aims Our aim when developing NumberNavigation Game was to create a motivating game environment that would provide children a context to experiment with numbers and arithmetic operations. It was our goal to design a game where strategic choice in selecting the most optimal sequence of number-operation combinations would represent those core game features by which players make meaningful progress. In this study, we have tested a prototype of NNG in order to gather rich qualitative data to be used in our future game design processes. Our aim was to explore the type of emerging game patterns that could suggest the presence of in-game arithmetic problem solving flexibility and to explore those game features or combinations of game features where these patterns are more prominent. Design principles of NumberNavigation Game In NNG the player has to navigate a ship through different maps and collect raw materials for building settlements. Navigation takes place with the help of arithmetic operations that bring the ship to different locations in a map, which is based upon the hundred square. Within each map, players have to collect four types of building materials (wood, brick, stone and iron) placed in different locations and return them to the harbour. Harbours are located on a fixed number location within a map, which changes across different maps. Figure 1 illustrates a map from NNG, here the harbour is located at number 4 and the player has to pick up wood from number 94. In this example, a player might instinctively try to navigate on a straight line, adding 90 to reach the target, but this move is blocked by an island. A blocked move is signalled by a red path and a cross (see Figure 1). Islands always represent challenges the player has to consider when selecting a move. Alternatively, the player might consider 4*3 as the first correct move, which is signalled by a green path along which the ship moves to the next location, number 12. In NNG, apart from the islands, the efficiency of the sequence of moves a player uses towards the target depends on additional challenges. The game has three modes: exploration, minimum moves and minimum energy. In the exploration mode, the only challenge is to select moves so that the ship bypasses the islands. In the minimum moves mode the goal is to reach and return the target using the least amount of moves (operations). In the minimum energy mode the aim is to reduce the magnitude of numbers the player types into the command box to navigate the ship from one number to the next. For example, in case of the map presented in Figure 1, if the map is in moves mode, then continuing the journey from 12 to 48 would be a good option, reaching the target in three moves: 4*3=12; 12*4=48; 48+46=94. However, if the map is in energy mode, then this option would cost the player 53 energy units (the total value of numbers used to navigate the ship: 3, 4 and 46). If needed, this energy could be reduced by considering alternative options and using for example the following path which would only take 37 energy units: 4*11=44; 44-6=38; 38*2=76; 76+18=94. Naturally, many alternatives are available at once, and it is the task of the player to decide which option would be more efficient for reaching the target, considering the position of harbours, materials, and islands, as well as individual maps’ game modes. As the task is to reach then return each material, the player can always use the same numbers and inverse operations on the return, but also has the opportunity to re-evaluate and adapt the original strategy and chose a better path on return.

Figure 1: Feedback in case of right move and blocked move in NNG Methods Participants Game testing took place in two consecutive sessions. During the first session J. (age 11) played the game individually while during the second session T. (age 9) and E. (age 11) played the game collaboratively. All three participants were male elementary school students confident in using digital games (both educational and recreational) and had a self-reported positive attitude towards mathematics. Instruments and procedure Data was collected using the methods of video-recorded observation, interviews and recording of the gameplay. Players were recorded using a digital video camera. On-screen activity was recorded using Camtasia Studio 5.1.0 screen capture software. Video data from the two sources were synchronized and analysed simultaneously using Transana 2.42 software. Interview questions consisted of open-ended questions regarding players’ game experience. Game sessions lasted around one and a half hours, including instructions and interview questions. The game was played in three different modes introduced in the same order. First the exploration mode, followed by playing the game according to the rules of the minimum moves mode, and lastly playing the game according to the rules of the minimum energy mode. During collaborative play, participants had an extra option to decide which game mode they would like to apply for the last map (choice mode). The tested game version included three maps with 4 different target materials in each map. The tested maps and the location of harbours and target materials within each map is presented in Figure 2. Through the article, maps will be referred to according to their names used in Figure 2 (Map A, B & C).

Figure 2: Position of harbours and target materials within the three tested maps of NNG Results Table 1 presents the sequence and number of maps that players completed during the three different game modes as well as the amount of moves (arithmetic operations) made. Aside from successful moves, blocked moves were also logged and differentiated. Blocked moves include both operations that were impossible to execute due to islands blocking the path as well as operations which did not fit within the frame of natural numbers 1-100. During collaborative play, players chose to complete the last map under the minimum moves mode. Table 1: Sequence and number of maps in different game modes; total number of clear and blocked moves Sequence & Number of Maps Moves

Blocked moves

Game Mode Ind.

Coll.

Ind. Coll. Ind.

Coll.

A,B,C

A, B

92

57

18

9

Minimum moves A,B

C, A, B

52

68

9

4

Minimum energy C

C, A, B

19

67

1

8

Choice

-

C

-

16

-

4

Total

6

9

163 208

28

25

Exploration

Note: Ind. = Individual gameplay; Coll. = Collaborative gameplay As Table 1 shows, compared to the total number of moves the incidence of blocked moves was relatively low. Reasons for blocked moves were more often technical than mathematical, as from certain positions paths looked possible but were blocked by land. Total time on task (playing the game) was 1 hour for the individual game, and 1 hour 20 minutes for the collaborative game. Completing all three maps in one game mode took around 25-30 minutes irrespective of the mode. Completing one map could take 5 to 10 minutes depending on the difficulty. The time needed for retrieving a material varied from 50 seconds to 5-8 minutes, indicating that regardless of the game mode, certain quests proved to be more challenging than others. In order to identify signs of in-game arithmetic flexibility and possible effects of the different game modes on players’ strategies, the types of number-operation combinations used were explored across different game modes. Figure 3 and Figure 4 show the use of numbers and number-operation combinations in each of the three game modes for individual and collaborative gameplay. There was a clear trend in using 10 and multiples of 10 during both game sessions. This trend becomes even stronger in case of larger numbers >40 and >50 where the players almost exclusively used round numbers (numbers ending in 0). Looking at the use of

numbers over the two game sessions, five numbers seem to emerge as frequently used operands: 1, 5, 9, 10, 11, and 40. From these frequently used numbers, the use of 1 and 5 can be explained by the fact that they represent easy moves from an arithmetic perspective. The high incidence of 40 is an artefact of Map B where +40 and -40 was very frequently used for all four target materials and during both game sessions. The use of 9, 10 and 11 can be explained by the 10x10 square characteristic of the maps where using +10 and -10 means 1 move up or down, while +/- 9 or 11 represent 1 diagonal move. While using multiples of 10 to move several rows up or down was very common during the game, multiples of 9 and 11 were much less frequently used. From the common numbers, 11 was used with all four operations during both game testing sessions. It was not only used for adding and subtracting when moving diagonally, but also for replacing the addition or subtraction of large numbers by multiplying or dividing them by 11 during the minimum energy mode.

Figure 3: Number-operation combinations by different game modes during individual play From a total of 163 arithmetic operations performed during individual play, 57 where different numberoperation combinations, while from the 208 operations during collaborative play, 67 where different. As Figure 3 and 4 show, the use of multiplication and division was significantly lower than the use of addition and subtraction, with only 4 different multiplications and 3 divisions during individual play, and 5 multiplication and 5 divisions during collaborative play.

Figure 4: Number-operation combinations by different game modes during collaborative play The variety of numbers used for addition and subtraction was much larger than for multiplication and division, which is understandable given the nature of these operations. This explains why only smaller numbers were used (2, 4, 5, 8, 11, 12, and 13) for the latter. Multiplications were used for reaching the target

material and divisions almost exclusively for returning material to the harbour. During both sessions, minimum energy proved to be the only game mode that was able to trigger the use of all four arithmetic operations to the same extent. For certain materials, players almost exclusively used the same path in all game modes, while for other materials they chose alternative number-operation combinations during the different game modes. The interaction of the placement of harbours, target materials and islands had an impact on the variety of paths used by participants. For example, the last material in Map B (iron) shows high variation of moves and energy across all quests and game modes during both testing sessions, suggesting the use of various alternative number-operation combinations. To illustrate variability in this quest, Figure 5 shows a visual reconstruction of all moves players made at this map. As Figure 5 shows, reaching the target material was challenging, as islands were positioned in a way in which, apart from the usually preferred combinations of round numbers, the use of diagonal moves became desirable, which triggered addition and subtraction with multi-digit large numbers. Video data suggests that harbour-island-target combinations such as this one triggered extensive planning and strategy testing during both sessions. Visible signs of strategy testing were: clicking on those numbers with the mouse which could make part of a possible route, drawing virtual lines of the possible paths on the computer screen using the mouse or fingers, using fingers as a ruler on the screen to check for available angles. Using the game collaboratively triggered significantly more planning and strategy testing, contrasting different alternatives.

Figure 5: Visual reconstructions of players’ moves in Map B. Left: individual paly. Right: collaborative play Combinations of harbour-island-targets that were too easy resulted in automatic moves, such as +/- 40 in Map A. On the other hand, even these combinations were useful if used in a more challenging game mode. For example, during collaborative play, the younger player (T.) became passive after the minimum energy mode was introduced in Map C, perhaps due to the relative difficulty of this mode for his age group. The older player (E.) of the pair completed Map C individually, but upon revisiting Map A under the minimum energy mode, T. became active again. T. had in previous game modes developed the habit of using 4+40 as a first move, irrespective of the location of the target materials. When it was time to make the first move at Map A, now under the minimum energy mode, T. took control of the keyboard and typed in 4x11 instead of the usual 4+40. Nonverbal signs such as smiling and nodding suggest that he made a meaningful discovery at this point of the game. Discussion In this study, it was our aim to explore markers of in-game problem solving flexibility in a prototype version of NumberNavigation Game and to examine the efficiency of core game features in triggering the flexible use of number-operation combinations during gameplay.

During the total playtime of 1 hour and 1 hour 20 minutes, players carried out 163 and 208 mathematical operations. Apart from these operations, players performed extensive additional mental arithmetic for testing competing alternatives when deciding to use a given number-operation combination. During this limited playing time, three different game maps were tested, with 12 unique combinations of positioning harbours (starting number), target materials (goal number) and island (as obstacles). These 12 distinct combinations were further enriched by the 2 routes of first collecting then returning each material to the harbour, and additionally by the use of 3 different game modes in which players had to separately adapt their strategies in choosing the ideal moves. Specific combinations of these game features proved to be more efficient than others for triggering variety in player’s numbers-operation combinations. For instance, irrespective of the game mode, the interaction of positioning harbours-islands-targets was a very important factor that defined the amount of variation in players’ use of numbers. For example, having several smaller islands in a central position on the map, which allow many different but equally desirable paths, proved to be an effective design feature for triggering exploration with different alternatives. Target material placement is of major importance in future game versions, since these materials can be placed so that they are progressively more challenging within and across different maps. Additionally, it is also crucial to design the game in a way that it is clear for the player which combinations are possible and which are blocked by land. Results suggest that when available moves are unclear, players can easily become frustrated and forego exploration, moving almost automatically, using one step at a time. Using different game modes and challenging players to adapt their strategies according to the changing game rules was highly effective for triggering strategy testing, comparing and contrasting different alternatives, and planning ahead sequences of steps. Keeping the exploration mode in future game versions is unnecessary, as players quickly learnt the rules of the game and they preferred having explicit rules and challenges. Having to collect target materials while using the least amount of moves (minimum moves mode) proved to be efficient in triggering mental calculations with multi-digit large numbers and efficient in breaking the tendency of players to work with multiple of 10. The rules of the minimum energy game mode proved to be slightly more difficult to understand, especially for the young player. However, once rules were understood, the minimum energy mode managed to trigger more flexibility in player’s use of numbers and operations. It is important in future game versions that the two game modes are introduced progressively, first using the minimum moves mode, and then gradually introducing the minimum energy mode in maps with easier combinations of harbours-islands-targets. Using the game collaboratively proved to be more efficient in triggering strategy testing and exploration of alternative options. Accordingly, using the game in pairs or in groups, in a collaborative or competitive manner seems as a good model to be used in future studies with NNG. Overall, the present study managed to explore in detail how in-game markers of arithmetic problem solving flexibility and adaptivity can be better conceptualised and defined. Variety in the type of numbers used and in operations used is a good indicator of flexible game strategies, while any fixed and repeating pattern suggest a more mechanical gameplay. Strategy testing during the two different game modes, as well as contrasting different competing alternatives is also a clear indicator of more advanced strategies. However, more evidence is needed to determine if players were choosing the most effective routes, indicating more adaptive expertise with arithmetic and not simply applying a variety of non-efficient routes (Verschaffel et al., 2009). Based on the current results, future work with NNG will focus on level design and the integration of adaptive adjustment of the challenge-reward ratios. A very positive feature of NNG is the open game design, with minor changes the game can be engaging for older, more skilled players as well. Similarly, in future versions, the 10x10 square can be easily exchanged to alternative systems (e.g. base 9 or 12), creating instantly more variability in players’ number-operation combinations. Adding additional rules that would trigger the more frequent and flexible use of multiplication and division would also be desirable in future game versions. As both the sample and the tested number of game maps were limited, conclusions of this study cannot go far. Overall, the game managed to trigger player’s engagement for a relatively long period of time, there were no signs of boredom during gameplay, no random clicking or guessing, and the game design was easy and straightforward. Video data showed many identifiable signs of strategic behaviour with number combinations as well as comparing and contrasting alternative options. Even in this limited amount of time, players used a large variety of numbers and many different number-operation combinations. Additionally, players seemed to manage to adapt their strategies according to the changing rules of the game during the three different game modes. These are promising indicators that the elementary game design features of NNG are able to trigger engagement and exploration with number-operation combinations.

References Baroody, A. J. (2003) “The development of adaptive expertise and flexibility: The integration of conceptual and procedural knowledge”. In A. J. Baroody & A. Dowker (eds.), The development of arithmetic concepts and skills (pp. 1–34). Mahwah, NJ: Lawrence Erlbaum Associates. Beishuizen, M. (1993) “Mental Strategies and Materials or Models for Addition and Subtraction up to 100 in DutchSecond Grades”, Journal for Research in Mathematics Education, Vol. 24, No.4, pp 294-323. Blöte, A. W., Klein, A. S. & Beishuizen, M. (2000) “Mental computation and conceptual understanding”, Learning and Instruction, Vol. 10, No. 3, pp 221-247. Canobi, K. H., Reeve, R. A. & Pattison, P. E. (2003) “Patterns of Knowledge in Children’s Addition”, Developmental Psychology, Vol 39, No.3, pp 521-534. Devlin, K. (2011). Mathematics Education for a New Era, MA: A K Peters, Natick. Habgood, M. P. J. (2007). The effective integration of digital games and learning content, Ph.D. Thesis, University of Nottingham. Habgood, M. P. J. & Ainsworth, S. E. (2011) “Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games”, Journal of the Learning Sciences, Vol 20, No. 2, pp 169-206. Kucian, K., Grond, U., Rotzer, S., Henzi, B., Schönmann, C., Plangger, F., Gälli, M., Martin, E., von Aster, M. (2011). “Mental Number Line Training in Children with Developmental Dyscalculia”, NeuroImage, Vol. 57, No. 3, pp 782–795. Rittle-Johnson, B. & Star, J. R. (2007) “Does Comparing Solution Methods Facilitate Conceptual and Procedural Knowledge? An Experimental Study on Learning to Solve Equations”, Journal of Educational Psychology, Vol. 99, No. 3, pp 561-574. Räsänen, P., Salminen, J., Wilson, A. J., Aunio, P., Dehaene, S. (2009) “Computer-assisted intervention for children with low numeracy skills”, Cognitive Development, Vol. 24, No. 4, pp 450–472. Schneider, M., Rittle-Johnson, B. & Star, J. R. (2011) “Relations Among Conceptual Knowledge, Procedural Knowledge, and Procedural Flexibility in Two Samples Differing in Prior Knowledge”, Developmental Psychology, Vol. 47, No. 6, pp 1525-1538. Siewiorek, A., Saarinen, E., Lainema, T. & Lehtinen, E. (2012) “Learning leadership skills in a simulated business environment”, Computers & Education, Vol. 58, No. 1, pp 121–135. Threlfall, J. (2009) “Strategies and flexibility in mental calculation”, ZDM Mathematics Education, Vol. 41, No. 5, pp 541-555. Torbeyns, J., De Smedt, B., Ghesquière, P. & Verschaffel, L. (2009) “Jump or compensate? Strategy flexibility in the number domain up to 100”, ZDM Mathematics Education, Vol. 41, No. 5, pp 581-590. Verschaffel, L., Luwel, K., Torbeyns, J. & Van Dooren, W. (2009) “Conceptualizing, investigating, and enhancing adaptive expertise in elementary mathematics education”, European Journal of Psychology of Education, Vol. 24, No. 3, pp 335-359. Wilson, A. J., Revkin, S. K., Cohen, D., Cohen, L. & Dehaene, S. (2006) “An open trial assessment of "The Number Race", an adaptive computer game for remediation of dyscalculia”, Behavioral and Brain Functions, Vol. 2, No. 20. Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M., & Yukhymenko, M. (2012) “Our princess is in another castle: a review of trends in serious gaming for education”, Review of Educational Research, Vol. 82, No. 1, pp. 61-89.