CodeFruits: Teaching Computational Thinking Skills ...

6 downloads 0 Views 635KB Size Report
Roth, Wolff-Michael. "Gestures: Their role in teaching and learning." Review of educational research 71.3 (2001): 365-392. 19. Radu, Iulian, and Blair MacIntyre.
Poster Session: Purposeful Games

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

CodeFruits: Teaching Computational Thinking Skills Through Hand Gestures Sidhant Goyal Samsung R&D Institute, Bangalore, India [email protected] Simran Chopra Samsung R&D Institute, Bangalore, India [email protected] Dileep Mohanan Samsung R&D Institute, Bangalore, India [email protected]

Abstract Researchers in the past have explored several interactive tools and techniques to teach students abstract concepts of programming and computational thinking. They have explored playful and collaborative way to do so which includes text-based, visual or tangible programming toolkits. In CodeFruits, we are exploring hand gestures as an input interaction mechanism. These hand gestures represent different elements of programming languages such as functions and parameters. We have developed a gesture based interactive game which implicitly helps K12 students to learn abstract concepts of computational thinking and write a computational code in an easier, playful and engaging way.

Author Keywords Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for thirdparty components of this work must be honored. For all other uses, contact the Owner/Author. CHI PLAY'17 Extended Abstracts, October 15–18, 2017, Amsterdam, Netherlands © 2017 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-5111-9/17/10. https://doi.org/10.1145/3130859.3131335

Hand Gestures; Computational Thinking; K-12 students; Games.

ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Input devices and strategies;

Introduction The extensive research in the domain of “computational thinking (CT)” has identified itself as one of the critical skills that needs to be a part of regular K-12 curriculum.

291

Poster Session: Purposeful Games

Figure 1 A student using CodeFruits to learn computational thinking skills by performing hand gestures.

Wing has described CT as “solving problems, designing systems, and understanding human behavior, by drawing on the concepts fundamental to computer science.” [27]. She also suggested that “it is a fundamental skill of analytical thinking for everyone, not just for computer scientists”. According to Henderson, CT is the core of all modern Science, Technology, Engineering and Mathematics (STEM) disciplines, he states that it is intrinsic to all other disciplines [9]. Further in the context of K-12 education, Wing has stated that “To reading, writing, and arithmetic, we should add computational thinking to every child's analytical ability”. The prevalence of computational thinking skills in a variety of domains suggests that teaching such subjects during foundational early childhood years can be an engaging and rewarding experience for young learners as stated by Bers, 2008 [2]. Past work by researchers indicates extensive research on designing various programming languages and toolkits to teach such abstract concepts of computational thinking to K-12 students, from text based or visual programming languages such as Logo, Alice, Baltie and Scratch [15, 5, 3, 14] to tangible programming toolkits such as AlgoBlocks [23], Tern [8]. “Augmented reality (AR)” based tools such as AR Scratch [19] adds augmented reality functionality to the Scratch programming platform [14]. These interactive tools and languages aims to make computational thinking and the ability to write computational code easier, engaging and playful. These tangible programming languages use physical objects which represent various programming elements, commands, and flow-of-control structures, giving programming a physical form and making the interactions take place in the real world and more like

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

the real world interactions [12]. The use of such playful techniques has the potential to support and enhance the processes around learning. The playful state that games can engender can spark creativity, innovation and can also create a sense of fun and enjoyment. They can also remove some of the stresses and pressures that are often associated with formal education, allowing learners to engage with the game activities in a relaxed and light hearted manner [26]. Using gestures as an input interaction technique in the field of human computer interaction has been gaining popularity because they are intuitive and can be tailored to a particular individual [28], and can also make learning last [6]. Gesture use is primarily helpful in learning math and science as they “typically deal with abstract matters” [18]. They can also lighten our cognitive load and ease working memory by linking words to real objects [17]. Research [16] suggests that gestures can be used for assessing children’s knowledge and for illuminating the process of learning. They provide an alternative and embodied way of representing the problem. Learners who gesture spontaneously on a task are more likely to retain what they have learned about the task than learners who do not gesture [1, 7]. Instruction that includes gesture has been found to facilitate learning [4, 22, and 25] and children who are instructed to gesture while recalling an event report more details about the event than children who are instructed not to gesture [21]. These findings suggest that using gestures can promote child’s learning capabilities. In this project, we have explored using hand gestures as an input interaction technique as shown in Figure 1, which also represents different elements of the programming language such as functions, parameters. It makes learning basic concepts of computational thinking

292

Poster Session: Purposeful Games

more playful and engaging. We have created CodeFruits as shown in Figure 3, a low fidelity prototype of a game using Kinect, through which students can solve a problem scenario and implicitly learn computational thinking skills as well as nutritional facts about different fruits.

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

interpersonal relationship with elders which the kids can replicate in real life scenarios. Further, the emotional overtone of the game eases the actual intention of teaching computational thinking skills.

Concept and Design

Figure 2 The flow structure of the story of the game.

Playing games has been suggested as a viable addition to traditional educational curriculum [11]. CodeFruits is designed for elementary school. It is made keeping in mind the cognitive abilities of 8 - 10 year old children. The game is designed as a remediation of a deterministic board game but on a digital medium. The recent trend shows the transition of board games to videogames. Digital learning games that teach code are also prevalent. For example, Human Resource Machine which uses the concept of a corporate office worker assigned to perform tasks involving moving objects between an inbox, an outbox, and to and from storage areas as a metaphor for assembly language concepts. The player works through several puzzles in constructing a program to complete a specific task [24]. CodeFruits aims to do the same for children in a simplistic manner. It captures the attention of the child by showing a real life scenario similar to everyday urban life. The narration of the game tells the story of the player as shown in Figure 2. The child has to help the fruit truck to collect fruits from the town’s abandoned orchards which would then be delivered to the local old age home. The story provides a sense of achievement to the child at the end of a level. As the fruits are delivered to its destination, the nutritive qualities of the fruit are revealed and how it has helped the old granny in improving her health. The child is further incentivized by a short story or an anecdote by the old granny. The game also tries to develop

Figure 3: CodeFruits, a gesture based game computational thinking skills to K-12 students.

to

teach

The game is played by pre determining the number of steps and rules needed to move ahead. Research studies show that board games such as Snakes and Ladders result in children showing significant improvements in aspects of basic number skills such as counting, recognizing numbers, numerical estimation and number comprehension [20]. Playing board games has also been tied to improving children's executive functions [10]. Similar, aspect of learning is explored in CodeFruits, by exploring the use of hand gestures as input interaction method. These hand gestures also represent the different elements of programing language such as functions and parameters. A function is the basis of any code and teaches the relation between a set of inputs or variables to the output. This aspect of the game is to encourage computational thinking in children. In this game, parameters are the input integer variables that can be applied to the functions. In our game there are mainly three functions, namely Move (n),

293

Poster Session: Purposeful Games

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

RotateClockwise (n) and Loop (n), where n is no of times the function needs to be executed. Move is a function which is used to move the character in the game ‘n’ number of steps. In the game n is limited to 5 steps. Similarly, RotateClockwise rotate the character 90 degrees clockwise. Loop is used to repeat a combination of functions. For example, Loop (5) {Move (5); RotateClockwise (1);} will first move the character 5 steps and then rotate the character 90 degrees and then repeat this set five times. Rather than using such complicated syntax, children are exposed to such functions through simple hand gestures. The rules of the game are mentioned in the Table 1. Human gestures typically constitute a space of motion expressed by the body, face or/and hands, out of which hand gestures are often the most expressive Actions/ Tasks

Rules

Move the character

The character can move maximum (n=5) steps

Rotate the character

Moves the character clockwise 90 degrees, with maximum (n=4) times

Create a loop

Can be repeated max (n=5) times

Reset the code

Resets the code and starts the game from beginning at each level

Execute the code

Executes the code and shows feedback on screen after execution

Table 1. The rules of the game CodeFruits, the left side are the actions or tasks that the user does in the game, the right side are the rules the player needs to follow while playing the game.

and the most frequently used [13]. Based on hand movements, hand gestures are of two types, a posture: static finger configuration without hand movement or a Hand Gesture

How to gesture?

Mapped function

Static/ Dynamic

Moving one hand in the forward direction

Move (n), moves the character ‘n’ no of steps

Dynamic

Rotating one hand in clockwise direction

RotateClockwise (n), rotates the character 90 degree n no of times

Dynamic

Clapping once

Loop (n), repeats the set of functions ‘n’ no of times

Dynamic

Thumbs up with a single hand

Execute the written code

Static

Thumbs down with a single hand

Resets the written code

Static

Showing fingers of single hand (1, 2, 3, 4, 5)

Adding parameter value to function

Static

Table 2. Representation of different hand gestures, how to do the gesture, what it does and whether the gesture is static or dynamic.

294

Poster Session: Purposeful Games

Figure 4 Different levels in the game based on the complexity of the path to be followed and functions to be used.

gesture: dynamic hand movement, with or without finger motion. In our game, we have used both static and dynamic hand gestures to represent different movements and actions. Based on observational studies in classroom environments, we have defined the following hand gestures. We observed that these gestures as mentioned in Table 2, were the most commonly used by students to communicate with each other especially while playing in the classroom environment. The game is divided into different levels which vary on complexity as shown in Figure 4. This complexity is a combination of various functions and loops. The child can complete the task in various ways by choosing different combination of functions and loops but he/she needs to choose the most efficient way to do so. For example. In case of level 1 as shown in Figure 5, the student wants to move the truck 2 steps from starting point and rotate anti- clockwise 90 degree i.e. equal to rotate clockwise 270 degree. For writing a set of functions for this small part of the whole path, the user can write it in the following way by making use of Move and RotateClockwise functions:

Figure 5: Level 1 with least difficulty level, the player needs to use set of functions with parameters to complete the path.

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

Move (1) + Move (1) + RotateClockwise (1) + RotateClockwise (1) + RotateClockwise (1) or in this way: Move (3) + RotateClockwise (3), by making use of Move, RotateClockwise and parameters for these functions. The second way is a more efficient way of writing that code, only in case of choosing the second way, user is able to finish the task in level 1. By the end of level 1, the student is able to make use of parameters in functions. Level 2 has a path which is more complicated than level 1 shown in Figure 6.

Figure 6: Level 2 which has a set of patterns being repeated. The player needs to use loops to complete the level.

Since the students now have an understanding of using functions with parameters, they can write the whole set of functions to cross this path. Solution for this path is to write the set of functions to complete the path in level 1 i.e. Move (2) + RotateClockwise (3) + Move (4) + RotateClockwise (1) + Move (4) + RotateClockwise (1) + Move (4) + RotateClockwise (3) + Move (2) three times. Instead of writing the same set of functions three times, the students make use of the loop function to repeat a set of function multiple times. Eg. Loop (3) {Move (2); RotateClockwise (3); Move (4); RotateClockwise (1); Move (4); RotateClockwise (1); Move (4); RotateClockwise (3); Move (2);} Level 3 is more complicated than level 2, since it has 2 kinds of pattern

295

Poster Session: Purposeful Games

being repeated. The next paragraph explains the implementation part of CodeFruits.

Implementation and Testing We have built a low-fidelity prototype of the game using Kinect SDK in integration with Unity. The Kinect SDK was used to design custom hand gestures appropriate for the game which are mapped to the functions used in the game. It has been developed using Unity game engine in C# language. It is very light in terms of storage as well as graphics. Low poly models were used to ensure best performance in any computer configuration. The system is very easy to setup since Kinect is a plug and play sensor device. The initial testing of the prototype was done with five (2 boys and 3 girls) 8 – 10 year old children who were not familiar with programming elements, commands or coding in general. This initial testing was done to get some incremental qualitative feedback about the prototype to make refinements in the game design as well as development. The children were introduced to three basic gestures of Move (n), RotateClockwise (n) and Parameters value addition and asked to play the first level of the game. While testing, we observed that there is a conflict between certain gestures such as the move (n) gesture and the parameters value (5) adding gesture. The kids performed both the gestures in a similar way leading to conflicts both on the user’s side as well as from system point of view. This suggests that there should be a demarcation between different gestures in order to avoid conflicts. We would be incorporating further changes and refinements in our next robust prototype based on the children’s feedback.

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

Future Work The current prototype is a low-fidelity prototype of the game, where 3rd – 4th elementary class students can use hand gestures for solving incremental difficult scenarios. In future, we would like to create a more robust prototype which can be tested with multiple students with different age groups. This will help us in understanding different aspects of using games and gestures in learning computational thinking skills faster, efficient and in a collaborative way. We also want to explore the area of virtual reality to understand the differences in an immersive environment and how it will affect the learning of computational thinking skills. We would also want to scale the game for a variety of students with different abilities and resources.

Conclusion In CodeFruits, we have tried to ease the learning of computational thinking for K-12 students. The game engages them through a natural way of interaction by incorporating gestures from their own classroom environments. We have observed that using gestures as an input mechanism, eases the learning of abstract concepts such as computational thinking skills. The game play engages the child through gestural play and a captivating storyline. The real life setting of the game including elements and characters are very relatable to their daily life environment. It provides them a sense of belongingness in the virtual environment of the game. Further they feel accomplished because of the information imparted and the anecdote at the end of each level, making them motivated to go to the next level. Further, based on the initial testing and user feedback, the game design and implementation can be improved to provide a much richer user experience in learning abstract concepts of computational thinking.

296

Poster Session: Purposeful Games

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

References 1.

2.

3. 4.

5.

6.

7.

8.

Alibali, Martha W., and Susan GoldinMeadow. "Gesture-speech mismatch and mechanisms of learning: What the hands reveal about a child′ s state of mind." Cognitive psychology 25.4 (1993): 468-523. https://doi.org/10.1006/cogp.1993.1012 Bers, M. U. "Blocks, robots and computers: Learning about technology in early childhood." Teacher’s College Press, NY, NY (2008). Baltie, Retrieved June 28, 2017 from http://progopedia.com/language/baltie/ Church, Ruth Breckinridge, Saba Ayman-Nolley, and Shahrzad Mahootian. "The role of gesture in bilingual education: Does gesture enhance learning?." International Journal of Bilingual Education and Bilingualism 7.4 (2004): 303-319. http://dx.doi.org/10.1080/13670050408667815 Cooper, Stephen, Wanda Dann, and Randy Pausch. "Teaching objects-first in introductory computer science." ACM SIGCSE Bulletin. Vol. 35. No. 1. ACM, 2003. http://dx.doi.org/10.1145/611892.611966 Cook, Susan Wagner, Zachary Mitchell, and Susan Goldin-Meadow. "Gesturing makes learning last." Cognition 106.2 (2008): 1047-1058. https://dx.doi.org/10.1016/j.cognition.2007.04.010 Cook, Susan Wagner, and Susan Goldin-Meadow. "The role of gesture in learning: Do children use their hands to change their minds?." Journal of cognition and development 7.2 (2006): 211-232. http://dx.doi.org/10.1207/s15327647jcd0702_4 Horn, Michael S., and Robert JK Jacob. "Designing tangible programming languages for classroom use." Proceedings of the 1st international conference on Tangible and embedded interaction. ACM, 2007. http://dx.doi.org/10.1145/1226969.1227003

9.

10.

11.

12.

13.

14.

15. 16.

17.

Henderson, Peter B., Thomas J. Cortina, and Jeannette M. Wing. "Computational thinking." ACM SIGCSE Bulletin. Vol. 39. No. 1. ACM, 2007. https://dx.doi.org/10.1145/1227310.1227378 How Family Game Night Makes Kids Into Better Students. Retrieved June 28, 2017 from https://www.theatlantic.com/education/archive/201 4/07/how-family-game-night-makes-kids-intobetter-students/374525/ Harris, Christopher. "Meet the New School Board: Board Games Are Back--And They're Exactly What Your Curriculum Needs." School library journal 55.5 (2009): 24-26. Jacob, Robert JK. "What is the next generation of human-computer interaction?." CHI'06 Extended Abstracts on Human Factors in Computing Systems. ACM, 2006. http://dx.doi.org/10.1145/1125451.1125768 Karam, Maria. PhD Thesis: A framework for research and design of gesture-based human-computer interactions. Diss. University of Southampton, 2006. Maloney, John, et al. "Scratch: a sneak preview [education]." Creating, connecting and collaborating through computing, 2004. Proceedings. Second International Conference on. IEEE, 2004. http://dx.doi.org/10.1109/C5.2004.1314376 Papert, Seymour. Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc., 1980. Pine, Karen J., Nicola Lufkin, and David Messer. "More gestures than answers: children learning about balance." Developmental psychology 40.6 (2004): 1059. http://dx.doi.org/10.1037/00121649.40.6.1059 Ping, Raedy, and Susan Goldin‐Meadow. "Gesturing saves cognitive resources when talking about nonpresent objects." Cognitive Science 34.4 (2010):

297

Poster Session: Purposeful Games

18.

19.

20.

21.

22.

CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL

602-619. https://dx.doi.org/10.1111/j.15516709.2010.01102.x Roth, Wolff-Michael. "Gestures: Their role in teaching and learning." Review of educational research 71.3 (2001): 365-392. Radu, Iulian, and Blair MacIntyre. "Augmentedreality scratch: a children's authoring environment for augmented-reality experiences." Proceedings of the 8th International Conference on Interaction Design and Children. ACM, 2009. http://dx.doi.org/10.1145/1551788.1551831 Siegler, Robert S. "Playing numerical board games improves number sense in children from low-income backgrounds." BJEP Monograph Series II, Number 7Understanding number development and difficulties. Vol. 15. No. 29. British Psychological Society, 2010. 15-29. https://dx.doi.org/10.1348/97818543370009X1258 3699332375 Stevanoni, Elizabeth, and Karen Salmon. "Giving memory a hand: Instructing children to gesture enhances their event recall." Journal of Nonverbal Behavior 29.4 (2005): 217-233. https://dx.doi.org/10.1007/s10919-005-7721-y Singer, Melissa A., and Susan Goldin-Meadow. "Children learn when their teacher's gestures and

23.

24.

25.

26.

27.

28.

speech differ." Psychological Science 16.2 (2005): 85-89. Suzuki, Hideyuki, and Hiroshi Kato. "Interactionlevel support for collaborative learning: AlgoBlock— an open programming language." The first international conference on Computer support for collaborative learning. L. Erlbaum Associates Inc., 1995. http://dx.doi.org/10.3115/222020.222828 Tomorrow Corporation. 2017. Human Resource Machine. GAME [Nintendo Switch]. (16 March 2017). Tomorrow Corporation Experimental Gameplay Group Valenzeno, Laura, Martha W. Alibali, and Roberta Klatzky. "Teachers’ gestures facilitate students’ learning: A lesson in symmetry." Contemporary Educational Psychology 28.2 (2003): 187-204. https://doi.org/10.1016/S0361-476X(02)00007-3 Whitton, Nicola, and Alex Moseley, eds. using games to enhance learning and teaching: a beginner's guide. Routledge, 2012. Wing, Jeannette M. "Computational thinking." Communications of the ACM 49.3 (2006): 33-35. Wolf, Catherine G. "Can people use gesture commands?" ACM SIGCHI Bulletin 18.2 (1986): 7374. https://dx.doi.org/10.1145/15683.1044102

298