Make Your Own Homework: An Alternative Approach

Make Your Own Homework: An Alternative Approach to Assignments in a Flipped Calculus Class Rahmad Dawood Telematics Research Center / Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Mohd Syaryadhi Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Muhammad Irhamsyah Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Roslidar Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Elizar Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Hubbul Walidainy Jurusan Teknik Elektro dan Komputer Universitas Syiah Kuala Banda Aceh, Indonesia [email protected]

Abstract—This paper describes an alternative approach to homework, which asked the students to create and solve a problem set. The homework was one of several activities in an Introductory Calculus course taught using the Flipped Classroom method. The homework was meant to replace readings that were usually assigned to prepare students for the next class. Results from the experiment shows that this type of homework had a large impact on increasing students’ understanding of the taught material. Students who were given this homework, on average, had a 29.98 grade points increase in posttest scores compared to students who were given the regular reading assignments. This increase can be attributed to students being better prepared for classes. By doing the homework, students were made to review the provided materials and had more problem sets to practice on. Moreover, students’ reception of this type of homework was positive. Keywords—Engineering Classroom, Homework

Education,

Calculus,

Flipped

I. INTRODUCTION Calculus is a foundational subject in any engineering program, because to comprehend advanced engineering courses will require a sound command of calculus. Thus, calculus is typically a required course in the first year of any engineering program, including in the studied Department of Electrical and Computer Engineering in Universitas Syiah Kuala (http://elektro.unsyiah.ac.id). In fact, the department’s incoming students are required to take and pass two introductory calculus courses in the first year. Because of its foundational nature, many innovations have been proposed and implemented to improve comprehension of calculus. One major innovation was employing the Flipped Classroom method [1], [2] in the teaching of calculus, where such classes are popularly known as Flipped Calculus classes [3]–[7]. Based on numerous success stories, the Introductory Calculus teaching team decided to adopt this method. This

adoption produced significant gains in student comprehension of the taught materials (compared to the traditional teaching method) and at the same time it received positive feedback from the students taking the course [8]. One intriguing piece of feedback that the teaching team kept receiving from the students of the Flipped Calculus course was questioning the need for the reading assignments handed out to prepare for the next class. Students argued that providing problem sets to practice on was more valuable for understanding than the assigned readings. Together with the suspicion that the assigned readings were rarely completed or just skimmed over, the teaching team discussed possible replacements for the reading assignments. One replacement assignment that the teaching team developed was to ask students to construct and answer their own problem set, which the team called the Make Your Own Homework assignment. The goal of this paper is to elaborate on the implementation of Make Your Own Homework in the Introductory Calculus course to test whether this type of assignment was effective in increasing students’ understanding of the taught materials.

II. MAKE YOUR OWN HOMEWORK In the Make Your Own Homework assignment, students were asked to create and answer their own problem set for the topic of the next-class. A reading list relevant to that topic was also provided as a reference. The reading list itself was identical to the list in the regular assignments. The problem set was to consist of 10 questions, with solutions, and had to be handwritten. Students were told that the problem set would be checked for similarities with other students and any duplication would result in that assignment not being graded. Prior to the start of the class, the created problem sets were handed in to the lecturer to be reviewed, commented on and graded. Fig. 1

depicts a sheet from a problem set created by a student that has been graded and commented on by a lecturer.

Fig. 1. A sheet from a problem set created by a student.

It should be noted that the Make Your Own Homework assignment was one of several activities in the Flipped Calculus class developed in the department [8]. This Flipped Calculus class has been consistently used for the past two years to teach two first year Introductory Calculus courses. However, the Make Your Own Homework assignment was an innovation implemented only in the last offering of the class.

III. RELATED WORKS The Flipped Classroom method is increasingly being adopted in the teaching of Introductory Calculus (see for example [3]–[7]). In the typical teaching method, students learn a new topic in the classroom supervised by a teacher and, to increase their understanding, are given a problem set to be worked on at home. Alternatively, in the Flipped Classroom teaching method, students first study the new topic at home through a set of materials provided by the instructor, and then in class, students are given a problem set to solve with the guidance of a teacher [1], [2].

The Introductory Calculus teaching team adopted this Flipped Classroom method and has modified so it was better suited for their environment [8]. In this adapted Flipped Calculus method, instead of videos and online lectures, students were given a list of reading materials and examples to review to prepare for the next class. At the start of every class, students were given a weekly test to determine their preparation for the class. This weekly test was also meant to incentivize students to review the given materials. After the daily pretest, a short lecture was given summarizing that day’s materials and answering any related questions from the students. The lecturer then proceeded to hand out a problem set related to that day’s material to be completed in-class by the student. The remainder of the class was the students completing this problem set aided by several student assistants and the lecturer. The class closed by informing of next week’s topics and the reference materials. Table I depicts the activities in a typical session. TABLE I.

CLASS ACTIVITIES

Activities Opening Weekly test Lecture Problem set Closing

Duration 5 minutes 15 minutes 25 minutes 100 minutes 5 minutes

Students had to correctly complete the provided problems and if any difficulties were encountered, the lecturer and assistants moved around the room to help. Students could also ask fellow classmates for help; in fact, this was encouraged since it further increased the understanding of the student being asked for help. Fig. 2 depicts a typical session in the Introductory Calculus course where lecturers and assistants circulated around the room to aid students in completing the problem set. In a Flipped Calculus class, homework was still given out, but it was not meant for students to practice or review materials that were presented in class, as would be the case in a typical class [9]. Instead, homework in a Flipped Calculus class was meant to prepare students for the next class by reviewing the given material. Practice of these materials was still required to reinforce students’ understanding, but in a Flipped Calculus class, it was conducted in class with the lecturer. If there was any confusion or questions the lecturer could immediately clarify and explain. In reality, based on experience teaching Introductory Calculus, homework in both teaching methods was problematic. In a non-Flipped Calculus class, problem sets that were assigned as homework were often not the student’s own work but copied from their classmates, thus the objective for students to practice and reinforce presented material were not met. In a Flipped Calculus class, materials to be reviewed were rarely completed and were most likely skimmed through, thus the objective to prepare the students for the next class was also not met. Moreover, based on feedback, the students saw the reading assignments as having little value and would prefer more problem sets to practice on.

attribute the problem’s properties and organize these properties to formulate a solution. The Make Your Own Homework assignment, however, falls under the Create category, where in order to create a problem set, students need to be able to plan and produce a new problem.

IV. METHODS The objective of the Make Your Own Homework assignment was to increase students’ understanding of the taught materials. In this study, increased understanding was operationalized as the difference between a student’s posttest score and pretest score. Here, the more a student understood the taught materials, the higher the posttest should be. To determine whether this type of assignment achieved the objective, an experiment was conducted that compared the difference between two randomly assigned groups of students: the treatment group (which were given the Make Your Own Homework assignment) and the control group (which were given the regular assignment). If the Make Your Own Homework assignment is effective, then the treatment group should have a significantly higher difference in scores. To control for a student’s previous mathematical knowledge, this experiment took into account the student’s mathematic score from their National Exam. In Indonesia, to graduate from high school, all students must pass a National Exam where mathematics is one of the tested subjects.

Fig. 2. Students working on problem sets under the supervision of student assistants and the lecturer.

The Make Your Own Homework assignment had several properties to attempt to resolve the above problems. First, students not only had to create a problem set, but also had to write the solution to the problem, thus providing more practice on the material to be covered. Second, since the material itself had not yet been covered, to be able to create the problem set students needed to go through and understand the given reference materials, thus preparing for the next class. Third, because it was explicitly stressed that any similarities and duplication of other students’ problem sets would result in the assignment not being graded, a copied problem set was not likely to be identical, as some sort of modification occurred (e.g. different constants). Thus, some practice and preparation were occurring, since students still needed to solve the problem set with the modifications. From the perspective of the Revised Bloom’s Taxonomy [10], both types of homework are categorized as Procedural Knowledge in the Knowledge Dimension, where students are learning a subject-specific skill/algorithm/technique/method. But in the Cognitive Process Dimension, simply giving students problem sets to complete at home falls under the Analyze category. To be able to solve the problem set, students need to be able to determine what type of problem is given,

Basically, this study was a 1x2 natural experiment that was conducted with all first year students in the Department of Electrical Engineering, Universitas Syiah Kuala who were required to take the Introductory Calculus II course. Students were randomly assigned into three classes, where one class was selected to be the treatment group and the remaining two classes were designated as the control group. There were 24 students in the treatment group and 40 students in the control group. Except for the difference in type of assignments, the remainder of the course followed the Flipped Calculus protocol that was previously developed by the teaching team [8]. The three classes were each assigned to three different lecturers, who have taught the same course for several years. All three classes were conducted on the same day, at the same time, using the same slides, adopting the same reading lists and the same in-class exercise problems. Each class had several student assistants to aid the lecturer in running the Flipped Class. The teaching team met every Wednesday morning to review materials for the next class and to discuss any course-related issues that came up. One of these meetings is depicted in Fig. 3. At the start of the semester, all students were given a pretest and a short survey, where one of the questions asked for the student’s mathematic score on the National Exam. Questions from the pretest can be seen in the Appendix of this paper. At the end of the semester, all students were given a posttest and an anonymous exit survey, where students in the treatment group were specifically asked several open-ended questions regarding the Make Your Own Homework assignment. These questions included how the assignment was completed, opinion

of the assignment and whether this type of assignment should be continued. The results from the exit survey were then coded following the standard qualitative methods for analysis [11], [12]. In addition, all students were given the regular midterm and final exams.

A linear model was developed to evaluate the Make Your Own Homework assignment. This model takes the difference between the posttest and the pretest score as a measurement of the increase in learning that students achieved in the course. The model then determines whether this difference can be explained by the group where the student was placed (the categorical variable ASSIGNMENT, where 0 indicated that the student was in the control group and 1 indicated that the student was in the treatment group) and the student’s National Exam score for mathematics (the variable MATH). The variable MATH was included to control for the student’s previous knowledge. Results from the linear model are shown in Table III. TABLE II.

Fig. 3. The teaching team in one of their weekly meeting.

This experiment, as much as possible, tried to control for factors that might influence its result, specifically factors that were related to the inside-the-class activities and interactions. But factors that were related to the outside-of-class activities and interactions were difficult to control and were generally assumed to be random events. Randomization of students into the control and the treatment groups should alleviate these outside-of-class factors.

Control 40 35 5 49.14 13.85

# Students # Male # Female Average math score Pretest score

TABLE III. Coef.

There might be a gender effect, where the treatment group had a larger percentage of female students than the control group. The treatment group had 10 female students out of 24 (41.67%), while the control group had only 5 female students out of 40 (12.5%). This potential gender effect was noted and tested in the linear model.

Treatment 24 14 10 53.26 10.83

The developed linear model was significant, having an Fscore=12.47 with a p-value=0.00, and an R2=0.29. The standard regression diagnostic was conducted to validate the model, as suggested in [14]. The diagnostic concluded that the error assumptions for the model were maintained (errors were independent with equal variance and normally distributed) and there were no outliers, high leverage or influential observations. There were also no structural problems with the model.

All data processing was conducted using the R statistical software [13].

V. RESULTS AND DISCUSSION The profile for both the control and the treatment group can be seen in Table II. In general, the random assignment of students resulted in relatively equivalent groups, where the student’s initial mathematical level of knowledge at the start of the semester was similar. This level of knowledge was measured by two metrics: the student’s National Exam score for mathematics and their pretest score. The average National Exam scores for mathematics were 49.14 for the control group and 52.26 for the treatment group; a Wilcoxon Signed-Rank Test indicated no difference between these two groups; specifically, W=498.5 with a p-value=0.80. While the average pretest scores were 13.85 for the control group and 10.83 for the treatment group, a Wilcoxon Signed-Rank Test also indicated no difference between these two groups; specifically, W=420.5 with a p-value=0.49.

CLASS PROFILE

Intercept MATH ASSIGNMENT a.

13.19 0.17 29.98

Std. Coef. 0.00 0.09 0.52

a

RESULTS SE

10.19 0.20 6.34

t-score

p-value

1.30 0.86 4.73

0.20 0.39 0.00

N = 64, F-score = 12.47 with p(F) = 0.00, R2 = 0.29 and Adj. R2 = 0.27, Residual SE = 24.14

Results from the model showed that, holding all other variables constant, students that were given the Make Your Own Homework assignment, on average, increased their posttest score by 29.98 grade points compared to students that were given the regular assignment. This result was significant with a p-value=0.00. The effect size for this linear model (Cohen’s ƒ2) was 0.42, and thus can be categorized as large. A test for the gender effect was also conducted, by creating a new model that included the student’s gender as one of its variables. To evaluate this model, an Anova test was used to compare this new gender-included model with the initially developed model. The Anova test resulted in an F-score=0.01 with a p-value=0.94, which concluded that gender was not a significant factor in the model. Results from the exit survey provided insights on how students were completing the Make Your Own Homework assignment. Students would start the assignments two to three

days before class by first trying to comprehend the materials, reviewing the provided readings and slides. Students would then go online to find further explanations to increase their understanding and to find more example problems to review. These example problems would be the basis for the student to developed their own problem sets, which were slightly modified from the original example so that the problem was easily solved without additional complexity. Modifications tended to be simply changing several numbers in the example to something like the end of their Student ID to guarantee that the created problem set would be unique to them. If students still did not understand the material, and thus could not come up with a proper problem set, classmates were asked for help and, as a last resort, the student assistants were contacted. Overall, students saw this assignment as being useful in aiding comprehension of the materials and preparing them for class. Only three students outright stated that this assignment was not useful and another three students stated that this assignment was minorly useful. However, the majority of students recommended that this assignment be continued.

VI. CONCLUSION Results from the randomized experiment implementing the Make Your Own Homework assignment were positive. This assignment had a large effect on increasing students’ understanding of the taught materials, and students viewed this assignment as valuable in aiding comprehension of the taught materials. Students who were given this type of assignment, on average, had a 29.98 higher grade point increase compared to students who were not given this assignment. To complete this assignment, students had to learn the material for themselves, not only from the provided references, but also from proactively going online to find other resources that could further increase their understanding of the assignment. These extra efforts by students validated the properties that were expected from this type of assignment. Students had more problems to practice on and were pushed to review the provided materials making them better prepared for class. Therefore, students were taking command of their own learning. An interesting question raised by the results of this study was whether the above findings were due to the readings or the additional exercises that were completed for the created problem sets. This question warrants further research, because it will determine future implication, and which activities had more of an effect on students’ understanding (reading or the exercise). This will also determine how much more reading or how many more exercises should be given out. Nevertheless, by asking students to make their own homework, it positively increased students’ comprehension of the course materials.

APPENDIX: QUESTIONS FROM THE PRETEST/POSTTEST Compute the derivative for: + 1. ! = ln(2' − 5)

& 2. ! = sin (3) + 1) $% 3. ! = # &' ) 4. ! = sin ( ' 5. ! = sinh (3* + 1)

6. ! = cos 2' Find the integral for: 7. 8.

! = # √2& '& ! = # 3%&2 + % )

!" " = 9. Solve the differential equation: !# #

10. If log 3 = a and log 2 = b, what is 2log 3?

ACKNOWLEDGMENT The authors would like to thank the following teaching assistants, who participated in this class and untiringly aided their fellow students in understanding the course materials: Alifa Annisa Amru, Ardiansyah Lubis, Chairuman, Fachrur Rozi, Hendrik Leo, M. Andry Maulana, M. Rizky Dimas Syahputra, Maidi Yari Andika, Nizam Salihin, and Teuku Raja Usman.

REFERENCES [1]

J. Bergmann and A. Sams, “Flip Your Classroom: Reach Every Student in Every Class Every Day,” Washington, D.C.: International Society for Technology in Education, 2012. [2] J. L. Bishop and M. A. Verleger, "The Flipped Classroom: A Survey of the Research," Proceeding of the 120th ASEE Annual Conference & Exposition, Atlanta, GA., 2013. [3] V. Jungiƒá, H. Kaur, J. Mulholland, and C. Xin, "On flipping the classroom in large first year calculus courses," Int. J. Math. Educ. Sci. Technol., vol. 46, no. 4, pp. 508-520, May 2015. [4] J. McGivney-Burelle and F. Xue, "Flipping Calculus," PRIMUS, vol. 23, no. 5, pp. 477-486, Apr. 2013. [5] K. Palmer, "Flipping a Calculus Class: One Instructor's Experience," PRIMUS, vol. 25, no. 9-10, pp. 886-891, Nov. 2015. [6] A. Sahin, B. Cavlazoglu, and Y. E. Zeytuncu, "Flipping a College Calculus Course: A Case Study," J. Educ. Technol. Soc., vol. 18, no. 3, pp. 142-152, 2015. [7] L. B. Ziegelmeier and C. M. Topaz, "Flipped Calculus: A Study of Student Performance and Perceptions," PRIMUS, vol. 25, no. 9-10, pp. 847-860, Nov. 2015. [8] R. Dawood, M. Syaryadhi, M. Irhamsyah, and Roslidar, “Measuring the Increase in Students’ Comprehension in a Flipped Introductory Calculus Course,” Teaching and Learning in a Digital World, 2017, pp. 202–207. [9] H. Cooper, J. C. Robinson, and E. A. Patall, "Does Homework Improve Academic Achievement? A Synthesis of Research, 1987-2003, 19872003," Rev. Educ. Res., vol. 76, no. 1, pp. 1-62, Mar. 2006. [10] L. W. Anderson et al., A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's, Abridged Edition. Essex, England: Pearson Education Limited, 2014.

[11] M. B. Miles, A. M. Hubberman, and J. Saldana, Qualitative Data Analysis: A Methods Sorucebook, 3rd Edition. Thousand Oaks, CA.: Sage, 2014. [12] C. F. Auerbach and L. B. Silverstein., Qualitative Data: An Introduction to Coding and Analysis. New York, NY.: New York University Press, 2003.

[13] R Core Team, R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2018. [14] J. J. Faraway, Linear Models with R, 2nd Edition. Boca Raton. FL.: CRC Press, 2015.