A MASTERY LEARNING APPROACH: AN INTRODUCTORY PROGRAMMING COURSE DESIGN FOR DIVERSIFIED STUDENTS’ INTERESTS Suguru Arakawa1 , Akira Sasaki2 , Katunobu Itou2 1
Nautilus Technologies (JAPAN) 2 Hosei University (JAPAN)
[email protected],
[email protected],
[email protected] Abstract An introductory programming course such as CS1 is one of the most essential courses in terms that successive computer science (CS) courses heavily depend on students’ attainment level of CS1. In this study, we show our experience of our CS1 course to conform with diversified students’ interest. We introduce (1) “virtual small group classes”(VSCs) which are practical laboratory programming work from variety kinds of domain, along with (2) “mastery test” (MT) which is designed to assure fundamental knowledge and skills expected in CS1. Our VSCs employed five courses with different domains in year 2011 and students should challenge to create programs from at least one courses. MT is pass/fail test of which passing point is set at 86% and students can take till passing. One of the main objective of MT is to raise mastery levels of students especially who have difficulty in following the VSCs. Our experience of VSCs and MT of 2011 indicates that VSCs efficiently motivate students and encourage their engagement in programming on one side, with keeping their master levels by MT. We succeeded to start successive lectures with keeping students’ interest and required knowledge level. Keywords: Mastery Learning, CS1, Programming, Computer Science.
1 INTRODUCTION An introductory programming course such as CS1 is one of the most essential courses in terms that successive computer science (CS) courses heavily depend on students’ attainment level of CS1. On the other hand, the recent widening CS application areas affect the variety of students’ interest, which causes difficulty for course designers in selecting domains of programs used in class. In order to conform with diversified students’ interest, we have been improving CS1 course design by introducing “virtual small group classes”(VSC) from year 2010. VSCs are essentially practical laboratory work in which students challenge to create working programs of different domains: standard, graphics, sound synthesis, spreadsheet, web applications. Core programming knowledge and skills are provided previously by unified ”key lecture” with conventional CS1 style. Students are supposed to select at least one VSC according to their own interest. With introduction of our VSCs, we observed the cases that some kind of knowledge are difficult to be mastered sufficiently by students who select specific VSCs. From year 2011, we employ “mastery test”(MT) which is held apart from the ordinal grading test. MT is designed to evaluate only for fundamental units extracted to be acquired completely and is used for pass-fail. MT’s question are simple and easy to answer for students and its passing point is set at 86%. If a student fails the mastery test, he/she can take mastery tests till passing to a maximum of three times. In MT of year 2011, almost all of the students passed the exam. MT was held at tenth week of the fifteen-week curriculum and the students clarified their master levels and goals. Our experience of VSCs and MT of year 2011 indicates that VSCs efficiently motivate students and encourage their engagement in programming on one side, with keeping their master levels by MT. We succeeded to start successive lectures with keeping students’ interest and required knowledge level.
2 RELATED WORKS The programming practices by implementing application using character terminal and standard I/O even demotivate many students and there is a lot of work to raise students’ motivation in CS1. (e.g. [1]) The programming practices in VSCs also focus on this point in terms that students implement application with multimedia devices.
In this research, despite that most of previous work do not mention about how to apply specific method in the context of whole CS1 curriculum, we show a more total approach in terms that we provide how to effectively integrate these methods into the curriculum. We provide a custom APIs dedicated for CS education in order that students can easily manipulate various devices and functionality, including graphics, network and other devices. This approach is also taken in [2]. This book includes programming practices focusing on scientific application, and treats advanced topic beyond CS1. As far as the range of CS1 is concerned our VSCs have more variety in topics. The idea of our mastery test comes from the drivers license test in Japan. The test includes a paper examination, whose pass line is 90%, that verifies the suffice understanding of traffic rules and laws. In the mastery test, the pass line is set to 86% which issues the “license” assuring a student has premise knowledge and skills required for the following CS courses.
3 Course Design with Virtual Small Group Class 3.1 CS1 Course Design Our CS1 course focuses that the students get accustom to programming itself and acquire the basic knowledge and skills required for expressing and realizing their idea as program. The contents of the curriculum (1) is based on the learning unit (LU) of J07 which proposed by a cs/se education consortium in Japan.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Table 1 – Course Calendar Introduction, Input/Output Variable, Expressions Control (the if Statement) Subroutines (Method) Review Functions Arrays Repetition Structure Review MT #1 Interactive Programming Reference, Instance MT #2 Review Review MT #3 Final
In our curriculum, we teach “method” in early stage and use methods intensively in the later stages. This enables students to take enough time to learn the concept of procedures and functions in programming which is essential for CS and difficult to master. Each class is scheduled with a 180-minute slot, which divided into two parts. In the first part, students get the key lecture that treats new knowledge in each unit, and in the second part, students try practical laboratory work treating many domains, which we call virtual small group classes (VSC). VSCs are designed to respond to the variety of students’ interest and five kinds of domains are selected for the laboratory work (Table 2).
3.2 VSC Design The topics treated in VSCs are selected so that they motivate students and encourage their engagement in programming at ex-lecture. Students should select at least one VSC course according to their interest. In each course, both standard level and advanced level programming problems are provided. Each VSC’s programming except “plain” course employ API library dedicated to VSC for students’ easily accessing non standard functionality when programming graphics, sound, and network applications. An instance of problems given in “Canvas” course is drawing graphs of weather data stored in arrays(Figure 1). Through drawing histogram and line plot, students learn the usage of the combination of iteration (for-statement) and accessing array elements. The “Canvas” library provides creating canvas, figure drawing, plotting, simple interaction via mouse and keyboard.
Figure 1 – Graph Drawing on Canvas
Functionality of the Canvas and Spreadsheet libraries are easily understood and intuitively used for all students only by using fundamental knowledge of high school math. Sound and Web application sometimes requires concepts new to first-year students. VSC materials provide detailed explanation and no other knowledge given in the materials are required to solve the problem.
Plain Spreadsheet Web Canvas Sound
Table 2 – Topics of VSC programming exercises as in CS introductory textbooks Simple statistics, table data calculation, virtual machine Network and web application Graphics, Animation, GUI Digital signal processing, audio data (sound synthesis)
All materials are accessible via online. The main online textbook includes contents of the key lecture together with programming exercises. The VSC materials for all 5 courses are completely available online. We also provide mini exercises for pre-learning with which the students read the textbook and to prepare some of hands-on work before lecture. Students submit their assignment by learning management system. As a result of one class (49 students) out of four classes, almost all the students tried to VSC programming in more than three kinds of topics (which includes one time trial for certain topics)(Figure 2), which shows that VSCs effectively motivates students for programming. A+ students tried 1.5 times more assignments than average, and also all the A students except one challenged more assignments than average. We had once provided a CS1 class, dedicated for advanced students, besides the normal class. With VSC approach, higher grade students can challenge more assignments by themselves within a single class, which implies that the VSC can be as effective as the dedicated class approach.
Figure 2 – VSC topic
4 MASTERY TEST VSCs highly motivate students; however, it is difficult to commit this process to the whole students. The students who cannot follow the process often to fail to acquire sufficient skills for successive CS courses. Until year 2010, we had evaluated the students’ final score of CS1 by term examination and the pass line was 60%. However, pass-or-fail judgment according to term examination and/or ordinary reports does not assure students’ mastering levels: student’s test and report might be judged pass despite that he/she does not master some of fundamental knowledge or skills. In year 2011 we renounce our grading strategy of CS1 course. The final examination is divided into two kinds of tests: a mastery test and a grading test, in order to master sufficient levels. The mastery test is designed to evaluate only for fundamental units extracted to be acquired completely and is used for pass-fail. The grading test can be taken only by students passed the mastery test and is used only for grading from A+ to C. The mastery test is conducted at the tenth week. A passing point of the mastery test is set at 85%. If a student fails the mastery test, he/she can take mastery tests till passing to a maximum of three times(1).
#1 #2 #3
Table 3 – Mastery Test Result Summary examinee pass cum. examinee cum. pass 157 97 157 97 64 43 163 140 20 18 164 158
Cumulative total pass rates of the mastery test were 62% for the first time, 86% for the second time and nearly 100% for the third time. The result showed that almost all of students were achieved to master most of the fundamental units. The effects of the mastery test are as follows: 1. the test has encouraged students to recognize their weaknesses by themselves, and 2. multiple challenges of the test enable them to set their individual learning goals in a clear form. The test questions for the grading test were flexibly designed so as to measure students’ ability more precisely. 3. the test result gives an effective feedback especially for the students who have difficulty in following the lectures, which leads to their willingness to learning. The test questions for the grading test were flexibly designed so as to measure students’ ability more precisely. This can be achieved by pass-fail judgment with the mastery test. In our case, the grading test was a paper test that includes challenging, or difficult questions. Our experiments suggest that separating a pass/fail judgment from overall evaluation offers course designers flexibility with choice of a grading method.
5 EVALUATION 5.1 Relation between the final grading and the number of VSC exercises challenged
Figure 3 – Linear Regression for Modeling of Term Exam. Marks by the number of VSC topics
We model the score of term examination of a student by the number of VSC exercises by linear regression(Figure 3). The variables x and y mean the number of VSC exercises a student have challenged, and the score of the term examination of the student, respectively. One outlier is deleted from data set. The correlation coefficient is 0.64 and the regression function is described as y = 1.1x + 41 where standard error is 13.5.
5.2 The final score of years 2009-2011 and the MT score
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9
Table 4 – The distribution of the score of term examination in 2009 correct(%) correct(middle + lower)(%) method Topic 86.0 91.7 w/o Repetition 95.5 95.3 w/o Control 76.0 80.6 w/o Nested Loop 57.5 38.3 w/ Method Writing (Declaration) 50.0 43.9 w/o Complicated Repetition 54.3 48.6 w/ Program Reading 54.7 37.2 w/ Method Declaration 64.7 60.3 w/o Array Element Extraction 45.3 33.6 w/o 2D Array (advanced)
Table 4 shows the distribution of the score of term examination in 2009. One remarkable observation was that the score for questions using “methods” were much lower than we had expected, which means many students needed to relearned the topics during successive courses.
Table 5 – The distribution of the score of term examination in 2010 correct(%) method Topic Q1 93.8 w/o Expression, Variable Q2 89.1 w/o Control (Condition) Q3 91.3 w/ Method Q4 82.0 w/ Array, Repetition Q5 83.0 w/ Structure Q6 72.6 w/ Interactive Programming, Reference Q7 35.9 w/ Comprehensive Question (advanced) Table 5 shows the distribution of the score of term examination in 2010. In the term examination of the year 2010, the questions were designed as the division between mastery questions(Q1—Q5) and advanced (grading) questions(Q6, Q7). Basic questions using with “method”(Q3—Q5) were higher than the year 2009, on the other hand, advanced question (Q7) was still lower than we had expected. Because there were only two grading questions, accurate grading was impossible. Table 6 – The distribution of the score of mastery tests in 2011 correct(%) method Topic Q1 98.8 w/o Expression, Variable Q2 96.2 w/o Control (Condition) Q3 97.3 w/ Method Q4 90.7 w/ Array, Repetition Q5 90.2 w/ Structure Table 6 shows the distribution of the score of MT in 2011. The MT was designed to cover the same topics and be difficult as same as the questions Q1—Q5 of the term examination in 2010. The questions with methods Q3,Q4, and Q5 were higher correct rate than the year 2010. Even though 16% of the students marked less than 85% of Q1—Q5, they were given the credits in 2010. In 2011, however, almost every students given the credits marked more than 85% of the MT. Table 7 – Correlation among VSC exercises, MT scores, and the term examination scores in 2011 VSC MT1 MTF MT1 0.58 MTF 0.23 0.19 GR 0.64 0.67 0.06 Table 7 shows the correlation among the number of VSC exercise which students did (referred as VSC), the score of the first MT (referred as MT1), the final score of the MT (referred as MTF), and the score of the term examination (referred as GR). Even though the correlations between VSC and MT1 and between VSC and GR were high. the correlation between MTF and GR was low. The stronger positive correlation between VSC and GR suggests the VSC framework contributed that students acquired higher skills. The poor correlation between VSC and MTF suggests the MTs contributed that poor motivated students acquired the minimum required skills. The poor correlation between MTF and GR suggests the introduction of MT contributed more accurate evaluation was realized with acquisition of the minimum required skills.
6 CONCLUSION To conform with the broad variety of the students interest, the theme of programming tasks are selected from many domains, which led studens to engage in their programming mastery. The separation of pass/fail judgment from the overall evaluation achieved that: 1. the test has encouraged students to recognize their weaknesses by themselves, 2. multiple challenges of the test enable them to set their individual learning goals in a clear form, and that
3. the test questions for the grading test were flexibly designed so as to measure students’ ability more precisely
References [1] Major, L., Kyriacou, T. and Brereton, P. Simulated Robotic Agents As Tools To Teach Introductory Programming. In International Technology, Education and Development Conference (INTED 2011), Valencia, Spain, 7 - 9 March 2011. pp. 3837-3846. [2] Sedgewick, R. and Wayne, K. ntroduction to Programming in Java: An Interdisciplinary Approach. Addison-Wesley, 2008.
