Teaching Spatial Visualization Skills Using OpenNI and the Microsoft Kinect Sensor Chih-Hsiao Tsai1,* and Jung-Chuan Yen2 1
Dept. of Information Technology, Takming University of Science and Technology, Taipei, Taiwan
[email protected] 2 Dept. of Mathematics and Information Education, National Taipei University of Education, Taipei, Taiwan
[email protected]
Abstract. Many students must learn spatial skills to improve learning achievement in science, mathematics, and engineering. An abundance of literature on the geometric learning theory is available. However, specific guidance on how students can interact with teaching materials through their body is limited. We used a group of undergraduate students as an example and argue that the Kinect sensor-assisted learning interface can provide a “learning-by-doing” framework for learning spatial skills, motivating students, and enhancing students’ effectiveness. The responses to the System Usability Scale (SUS) indicated that our system demonstrated usability and learnability. We conclude that the Kinect sensor-assisted learning system promotes the development of students’ spatial visualization skills and encourages them to become active learners. Keywords: Kinect sensor, spatial ability, geometric learning theory, system usability scale.
1
Introduction
Learning with virtual objects integrated into realistic interactive technology has produced a novel digital learning type [1] in recent years and has become a controversial discussion topic in e-learning research. However, to guide learners to learn scientific knowledge quickly and efficiently, integrating innovative technology into scientific learning activities without causing any distraction from learning is a crucial research topic. Physically interactive and augmented reality is increasingly used, and game-based learning requirements for digital learning applications have become critical. Learners can learn effectively when they use multimedia-assisted learning systems, which motivates them to focus on the information presented and effectively retain the information. Lu and Yao stated that presenting abstract material using multimedia enables learners to clearly understand the material, and audio and visual displays allow learners to interact with the material [2]. Although a teacher’s professional education seems to play a vital role in the teaching process, innovative instructional designs that *
Corresponding author.
James J. (Jong Hyuk) Park et al. (eds.), Future Information Technology, Lecture Notes in Electrical Engineering 309, DOI: 10.1007/978-3-642-55038-6_97, © Springer-Verlag Berlin Heidelberg 2014
617
618
C.-H. Tsai and J.-C. Yen
differ from traditional teaching designs must be developed. The education mode must be changed from “technology-adapted instruction” to “instruction-adapted technology” [3]. Most learners demonstrate difficulty with reasoning and the conversion of shape and space in elementary school mathematics education, causing learning achievement in this unit to be lower than that in other units (Wang & Wang, 2008). However, spatial ability plays a crucial role in learning geometry [4]. The National Council of Teachers of Mathematics [5] suggested that geometry helps students to analyze the characteristics of geometric shapes as well as to use visualization, spatial reasoning, and geometric modeling to solve problems. Van Hiele [6] described a theory of mathematical education, the development of geometric thought, to enhance the achievement of learning and to promote learner comprehension. Van Hiele suggested five teaching phases in designing teaching methods and materials step-by-step. This model of teaching phases was used as the fundamental theoretical framework for this study. Game-based learning (GBL) is an instructional method that incorporates educational content or learning principles into video games to engage learners. The use of this method in the field of natural science and technology has increasingly been an object of study in recent years. Learning through digital games not only increases motivation, active learning, and individual learning opportunities, but also reduces the learning pressure experienced by learners. The primary focus of this study was on developing and evaluating a cubic net interactive system using the Kinect sensor for enhancing learners’ spatial ability. We developed courseware for interactive learning using an interactive cubic net system and applied it in a lesson on space geometry at an elementary school. The content of the system was based on the geometric learning theory, and real-time threedimensional (3D) objects were used to provide various viewing angle controls. A qualitative analysis was also performed to evaluate the learners’ interaction with the Kinect sensor design through observation records, video recording, and structural interviews. A questionnaire, the System Usability Scale (SUS), was administered to evaluate system usability. This article is structured as follows: The foundations for the development of the research are discussed in the subsequent section; the research methodology is then presented in Section 3, with full details on the system development and the instruments and procedures used; the results are then presented, with a thorough description of the system usability, in Section 4; and finally, the results are discussed and the conclusion is presented in Section 5.
2
Related Work
2.1
Spatial Visualization Skill
Bishop [7] referred to the power of spatial ability in assisting learners in visual and figural representation and in introducing complex abstractions in mathematics. Piaget and Inhelder [8] determined that children initially recognize various objects through the sense of touch alone and then develop and use certain primitive relationships (topological space). Contrary to the historical development of geometry, which began with the treatment of straight lines, angles, distances, and plane figures, children
Teaching Spatial Visualization Skills Using OpenNI and the Microsoft Kinect Sensor
619
begin by observing the topological rather than the metric properties of objects (projective space and Euclidean space). Lohman and Kyllonen [9] determined that spatial abilities include spatial visualization, spatial perception, spatial orientation, spatial imagination, and spatial translation and transformation. We focused on the spatial visualization skill, which is the ability to mentally manipulate complex spatial twodimensional and 3D figures. 2.2
The Kinect Sensor
The Kinect sensor for Windows functions as a computer’s eyes and ears, and the sensor provide the capacity to use those functions. To access the Kinect data using nonproprietary software, the USB communication had to be reverse-engineered. The basic parts of the Kinect device are (Fig. 1) • • • •
video camera (color stream) IR cameras (depth stream) microphones (audio stream) tilt motor (direction and angle control)
The Kinect sensor receives the information stream (e.g., video, depth, or audio stream) and delivers it to the natural user interface library. We used the application interface (API) to control all of the functions of the Kinect sensor, such as hand tracking, skeleton tracking [10], speech commands, and face tracking.
Fig. 1. Kinect hardware architecture
2.3
Game-Based Learning
Game-based learning (GBL) is a specific term used in instructional strategy or activities associated with applications that have defined learning outcomes. As an emergent learning process, GBL is designed to balance subject matter with gameplay and to help
620
C.-H. Tsai and J.-C. Yen
players retain and apply the subject matter to the real world [11]. GBL typically uses video-game technology, scoring strategies, interactive interfaces, flexibility courses, and real-time feedback to engage learners in learning. GBL not only makes learning meaningful, but also creates a mental model that motivates the participants [12]. Because information and communication technology has rapidly developed, computer and simulation games have become popular leisure activities in daily life. Some scholars assert that computer games are natural and necessary elements in student learning and that they should be integrated into instructional design as well as learning environments [11]. Scholars also believe that the concept of “learning-by-playing,” which helps students overcome the boredom of classroom learning, applies to playing computer games [13]. However, weaknesses and drawbacks in the implementation of GBL remain: (a) GBL requires substantial preparation and effort, and (b) the subjects and content are usually predefined and fixed. As Clark and Mayer [14] noted in their review of e-learning, a well-designed system can help generate the content of an entertaining scenario that corresponds to the learning subject. In other words, a game-based e-learning system enables simple and efficient preparation of learning applications.
3
Method
3.1
Teaching Materials
To develop material using the Kinect sensor, various programming frameworks, such as the OpenNI architecture, which was used to control the Kinect sensor, and the
Fig. 2. System stack overview
Teaching Spatial Visualization Skills Using OpenNI and the Microsoft Kinect Sensor
621
OpenTK architecture, which was used to provide a cross-platform library, must be surveyed (Fig. 2). Using the Kinect sensor to present questions and interactive games and including a hand-mouse function enable learners to learn the geometric learning theory and relevant information (Fig. 3). We provided three types of learning models: • Shapes and space • Drawing and visualization • Skeleton interactive model
Fig. 3. Kinect-based support system process
3.2
Experiment Participants and Operation
The participants were 78 students who took information technology and science courses at the University of Technology in North Taiwan. We used the SUS [15], which is a Likert scale used to assess system usability, learnability, and users’ subjective satisfaction concerning specific aspects of the interactive interface. In addition, we included an adjective scale to rate our SUS (Fig. 4). The SUS was highly reliable (alpha = 0.91) and useful in a wide range of interface types [16]. The SUS was intended to measure only perceived ease-of-use; however, Lewis and Sauro [17] demonstrated that it provides a global measure of system satisfaction and subscales of
622
C.-H. Tsai and J.-C. Yen
usability (Items 1, 2, 3, 5, 6, 7, 8, and 9) and learnability (Items 4 and 10) (the summed score contributions of the usability and learnability items were multiplied by 3.125 and 12.5, respectively).
Fig. 4. Mean score ratings corresponding to the seven adjective ratings
Employing the Kinect sensor developed in this study to present the teaching materials required the use of computers. The learners stood in front of the Kinect sensor and provided information regarding the course (Fig. 5).
Fig. 5. Learner operation example
4
Evaluation Results
The participants operated the system and completed the SUS questionnaire. The mean score of the SUS was 71.73 (SD = 13.06). According to the adjective rating score, the assisted learning system in this study closely matched the subjective label between “good” and “excellent.” We discussed opinions on the interactive learning system with the participants after they operated the system and completed the SUS. The participants generally agreed that the interactive learning system increased their interest in the concept and made them aware of various things. Fig. 6 lists the student distribution regarding system usability, which indicates that most of the learners agreed that the Kinect-based support system is satisfactory and also suggested that the system is helpful for learning. Fig. 7 shows the student distribution regarding the two-factor orthogonal structure, usability and learnability. The scores indicate that high usability is not highly consistent with high learnability. However, usability plays a vital role in the initial adoption or rejection of a technology, and learners believed that the system can facilitate learning.
Teaching Spatial Visualization Skills Using OpenNI and the Microsoft Kinect Sensor
623
Fig. 6. Distribution of SUS scores
Fig. 7. Distribution of usability scores and distribution of learnability scores
Regarding the analysis of the scale, we further discuss the mean, usability, and learnability scores that can help improve the system. Hence, we summarize the following reasons that may affect system satisfaction: • The control and learning interfaces were not sufficiently smooth. • The materials could not be displayed properly because of the angle and distance, which may confuse learners’ operation.
5
Conclusion
In this paper, we presented the results of the usability and the learnability of a cubicnet-assisted learning system using the Kinect sensor. Regarding the SUS questionnaire, most learners highly appreciated employing the Kinect sensor to present the teaching materials and considered the material to be interesting, easy to control and play with, and simple to operate using the interactive functions. The learners also reported that teaching by employing an interactive system enhanced their motivation to learn and was helpful for learning spatial skills. Several parts of the system must still be improved. A small portion of the learners reported that the learning system performed undesirably in response to speed during interactive operation. The buttons should be clearly defined, the cubit net knowledge should be highlighted, and proper
624
C.-H. Tsai and J.-C. Yen
instructions should be provided to ensure that the learners are focused on learning the knowledge expressed in the teaching materials. Acknowledgment. The funding of this study was supported by the National Science Council of Taiwan under grant NSC 100-2511-S-152 -012 - MY3 and 101-2511-S147 -001 –.
References 1. Lin, S.C., Chen, K.H., Hsieh, W.C., Lee, W.T., Huang, K.C.: Somatosensory interaction in digital game-based learning for Taiwanese classical poems. In: Global Chinese Conference Computers in Education, pp. 187–194 (2012) 2. Lu, C.C., Yao, N.D.: The Application and Introspection of Information Education in Elementary-school Curriculum. Elementary Education 42(6), 19–24 (2002) 3. Sung, Y.T., Chang, K.E., Hou, H.T.: Technology-instruction integration: learning form America’s experience and reflecting on Taiwan’s development. Bulletin of Educational Research 51(1), 31–62 (2005) 4. Wheatley, G.H.: Spatial sense and mathematics learning. Arithmetic Teacher 37(6), 10–11 (1990) 5. National Council of Teachers of Mathematics.: Principles and standards for School mathematics. Author, Reston (2000) 6. Van Hiele, P.M.: Structure and insight: a theory of mathematics education. Academic Press, Orlando (1986) 7. Bishop, A.J.: Review of research on visualization in mathematics education. Focus on Learning Problems in Mathematics 11(1), 7–16 (1989) 8. Piaget, J., Inhelder, B.: The child’s conception of space. Norton, New York (1967) 9. Lohman, D.F., Kyllonen, P.C.: Individual differences in solution strategy on spatial and change. In: Koch, S. (ed.) Psychology: A Study of Science, vol. 3, pp. 423–475. McGraw Hill, New York (1984) 10. Kar, A.: Skeletal Tracking using Microsoft Kinect. Methodology 1, 1–11 (2010) 11. Prensky, M.: Digital game-based learning. McGraw-Hill, NY (2001) 12. Chumbley, J., Griffiths, M.: Affect and the computer game player: The effect of gender, personality, and game reinforcement structure on affective responses to computer game play. Cyber Psychology and Behavior 9(3), 308–316 (2006) 13. De Freitas, S.: Using games and simulations for supporting learning. Learning, Media and Technology 34(4), 343–358 (2006) 14. Clark, R.C., Mayer, R.E.: E-learning and the science of instruction. Pfeiffer, San Francisco (2003) 15. Brooke, J.: System usability scale (SUS): a quick-and-dirty method of system evaluation user information. Digital Equipment Co. Ltd., Reading (1986) 16. Bongor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies 4(3), 114–123 (2009) 17. Lewis, J.R., Sauro, J.: The factor structure of the System Usability Scale. In: Kurosu, M. (ed.) Human Centered Design, HCII 2009. LNCS, vol. 5619, pp. 94–103. Springer, Heidelberg (2009)
