Learning with Engaging Activities via a Mobile Python Tutor Geela Venise Firmalo Fabic, Antonija Mitrovic, Kourosh Neshatian Department of Computer Science and Software Engineering, University of Canterbury, Christchurch, New Zealand
[email protected] {tanja.mitrovic,kourosh.neshatian}@canterbury.ac.nz
Abstract. This paper presents work on a new mobile Python tutor – PyKinetic. The tutor is designed to be used by novices, as a complement to traditional labs and lectures. PyKinetic currently contains one type of activity – Parsons problems, which require learners to re-order lines of code to produce a desired output. We present results of studies conducted to evaluate the usability and effectiveness of PyKinetic for learning. The enthusiasm from the participants was encouraging. We have also evaluated menu-based self-explanation prompts in PyKinetic. Results revealed that participants significantly improved their scores from pre- to post-test. Furthermore, participants who self-explained learned more than those who did not. We aim to develop more activities for PyKinetic to support code reading and code writing skills. We also plan to improve the tutor by providing engaging features to maximise learning, and to provide adaptive pedagogical support. Evaluation studies will also be conducted for future versions of PyKinetic. Keywords: Mobile Python tutor, Parsons problems, self-explanation
1
Introduction
It takes about ten years for one to become an expert programmer [1]. Novice learners find it difficult to grasp programming concepts, which may lower their motivation to learn more. Moreover, most novice programmers of this age are millennials, who usually have short attention spans [2]. It is essential for educators to explore more effective avenues in teaching programming catered to millennial novice programmers. Python is a popular programming language, widely used nowadays to teach introductory programming, especially in the United States [3]. In New Zealand and Australia, a survey conducted in 2013 on 38 introductory programming courses revealed that majority of the courses are taught using Python [4]. This project aims to develop a mobile tutor hoping that it would appeal better to new generation of students, compared to desktop or Web-based educational tools. Apart from the booming popularity of smart phones, a mobile tutor could potentially be an effective vessel for engaging activities, which is one of the emphases of our project. The aim is not to focus on the strengths of
a mobile device, but to use it effectively to the best of its advantages for the tutor. The goals of our project are to: (R1) investigate the effectiveness of a mobile tutor with engaging activities to maximize learning, (R2) explore different activities for improving code reading, and code writing skills. Section 2 presents some background, while Section 3 describes PyKinetic and the studies we have performed.
2
Background
Parsons problems [5] are programming exercises which require a given set of randomized lines of code to be rearranged towards producing the expected output, usually by a drag and drop motion. Parsons problems are fitting for a mobile device and for novices since Lines Of Code (LOCs) only need to be rearranged to form the solution. Similarly, Ihantola et al. [7,8] perceived the same insight and have developed Parsons problems for both mobile and web interfaces. Parsons problems have many variations, such as problems with and without scaffolding (curly braces and/or indentations), with and without distractors (extra lines of code) and limited editing of lines [6-9]. Self-explanation (SE), first introduced as open-ended questions, is an activity which requires the student to reason about the problem and generate justifications which are not directly presented by the material to promote deeper learning [10]. Self-explanation has been shown to improve learning outcomes in many domains, such as in database modeling [11], data normalization [12] and electrical circuits [13]. However, some studies like that of Johnson and Mayer [13] show that open-ended SE is not always suitable. They compared open-ended SE prompts to menu-based SE prompts using a computer application teaching electric circuits in a game-like environment. Participants were randomly assigned to an open-based SE group, menu-based SE group and without SE group. Their results revealed that menu-based SE group outperformed both the open-based SE group and without SE group [13].
3
PyKinetic
We have developed a mobile Python tutor, PyKinetic [14], which is designed to be a fun way for novices to learn Python while “on the go”, and as a complement to lecture and lab-based courses. PyKinetic is developed using Android SDK and teaches Python 3.x. We have developed PyKinetic with three variants of Parsons problems: regular problems, problems with distractors, and with incomplete LOCs. The first prototype of PyKinetic contained 53 Parsons problems, with 0 up to a maximum of five distractors. The number of LOCs in problems ranges from 3 to 16. We conducted a pilot study with students enrolled in an introductory programming course in Python and tutors involved in the same course. The pilot study had two goals: to evaluate the usability and the interface of the first prototype, and identify and compare strategies used by novices and experts. As expected, experts outperformed the novices in terms of speed and problem-solving strategies. Experts demonstrated having a mental model of the solution by moving LOCs in the correct order from top to bottom. On the other hand, novices displayed strategies showing lack of knowledge, such as
trial and error, and moving lines based on indentations. Furthermore, enthusiasm from the participants was encouraging, with seven out of eight novices and two out of five experts interested to use the tutor again [15]. The current version of PyKinetic offers incomplete LOCs, and provides menu-based SE prompts after every correctly answered incomplete LOC [16]. An evaluation study was conducted in 2016. We recruited 83 volunteers: 13 high school students from Middleton Grange School and university students (47 from the University of Canterbury, and 23 from the Ateneo de Manila University). All participants were enrolled in an introductory programming course using Python and have had adequate knowledge for the study. Participants were randomly assigned into two groups, with the only difference between the control and experimental condition being that the latter received SE prompts. The study had two hypotheses: (H1) all participants will improve their Python skills by interacting with PyKinetic, and (H2) the experimental group will have higher learning gains than control group. Sessions were conducted in groups which lasted from 1.5-2 hours and had a maximum of 13 participants. The study included a pre-test and post-test completed on paper. Both tests had eight questions: six conceptual questions composed of True/False and multiple choice, and two procedural questions (an output prediction question and a Parsons problem). All actions made during the study in PyKinetic were recorded. We eliminated data collected from some participants due to unforeseen circumstances. We present results of the data collected from the remaining 76 participants. We used the Mann-Whitney U test to check for a significant difference between the prior knowledge of different populations, and between experimental and control groups. Results showed no difference on pre-test scores between populations and between groups. We used the Wilcoxon Signed Ranks test for measuring learning gains. The results revealed significant improvements for both groups from the pre- to post-test (experimental: z = -3.315, p
