Could Alice Equalize Student Learning? - Duke Computer Science

Could Alice Equalize Student Learning? Alka Harriger Purdue University CIT Department, Knoy 243 West Lafayette, IN 47907-2021 1-765-494-2565

[email protected] ABSTRACT During the 2008-09 school year, a group of secondary school teachers in a variety of subject areas implemented classroom lessons that employed the 3D storyboarding tool, Alice. [2] Based on feedback from multiple teachers in different subject areas, implementing Alice-based lessons appears to have a positive impact on student performance across all student abilities. This paper will share the results that support this possible effect and offer additional explanations for improved student performance.

Categories and Subject Descriptors K.3.1 [Computers and Education]: Computer Uses in Education – computer-assisted instruction (CAI).

General Terms Human Factors, Experimentation, Performance, Theory

Keywords Alice, 3D Storyboarding, Student Learning

1. INTRODUCTION Surprising Possibilities Imagined and Realized through Information Technology (SPIRIT) is a three-year project, funded by the National Science Foundation, that seeks to encourage more young women to pursue collegiate study in an Information Technology (IT) field. [4] The SPIRIT approach includes a twoweek professional development program for high school teachers, a one-week professional development program for high school guidance counselors, and an educational summer camp for high school students. All three groups are introduced to Alice as part of the SPIRIT experience. Based on initial observations during the summer program, the SPIRIT approach appeared to be achieving success. [3] The external evaluator’s report confirmed that there was evidence to support that progress had been made toward achieving SPIRIT goals. [5] Another program that also used Alice to engage teachers in diverse subjects focused on middle school teachers. [7] SPIRIT’s teachers are expected to implement Alice interventions Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference’04, Month 1–2, 2004, City, State, Country. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.

in their schools during the school year and return to the Purdue campus four times to report on their progress throughout the school year. During the in-service meetings with teachers, the SPIRIT team discovered that Alice may have another unexpected, but very important benefit—equalizing student learning across all student abilities. This inference is based on multiple teacher testimonials in a variety of subject areas.

2. SPIRIT PROGRAM FOR TEACHERS A total of 24 teachers were accepted in the SPIRIT 2008 program. 18 of these teachers taught at schools with a large (at least 20%) underserved and/or underrepresented student population. The subjects taught by these teachers included science, math, computer programming, graphic arts, business, consumer and family sciences, English, Spanish, and French. Teachers accepted into the SPIRIT program are expected to spend approximately 140 hours completing SPIRIT program-related activities. During the first week of the summer program and under direct supervision of SPIRIT staff, the teachers learn Alice and spend additional time in the evenings completing Alice-based lesson plans for their subject areas. The following week, they practice using their lessons on a small group of high school students to solicit their suggestions for improvements. Based on the feedback received, they continue to make improvements and finalize the lessons by weeks’ end. Before they leave the summer program, they have developed three tested Alice-based lessons. This week also provides opportunities to participate in other hands-on activities designed to show all participants many unexpected ways that computing benefits society. During the school year, the teachers implement their Alice-based lessons into their classes. They report on their progress when they return to campus four times to participate in all-day, in-service conferences. These meetings are scheduled approximately once per nine-week grading period. Before and after these meetings, they interact with SPIRIT staff to successfully implement Alice into their classes.

2.1 SPIRIT 2009 Planned Changes Based on lessons learned from the first offering of SPIRIT, future offerings will implement a few important changes. Currently, teachers must provide a letter of support from their principals or superintendents that commit the school to installation of the Alice software for use by the teachers and their students. Because there is no installation deadline specified in the letter and many schools did the installation in mid-fall, several teachers were unable to implement their lessons as they had originally planned. For SPIRIT 2009, the school support letters must commit to

completing the installation before the school year begins. Additionally, signatures from both the principal/superintendent and the school’s technical support person will be required before a teacher will be accepted into the program. For the 2008 program, the teachers were not required to select lessons for different nine-week grading periods. This resulted in a few teachers not having implemented any lessons until the third in-service meeting. For the 2009 program, teachers will be asked to select lessons such that at least one lesson will be used in the first half of the first nine weeks, a second lesson will be used no later than the middle of the second nine weeks, and the third lesson will be used no later than the third nine weeks. This change will enable teachers to report on the impact of different Alicebased lessons at each in-service meeting.

3. TEACHER TESTIMONIALS There is evidence to support improved student learning takes place when students engage in activities in which they apply what they have learned to real-world problems. [1] Nearly all of the SPIRIT teachers used this approach for integrating Alice into their classrooms. Although several teachers had not yet used an Alicebased lesson by the first in-service meeting, for those who had, it became clear that their Alice-based classroom lessons were having a positive impact on the students. This positive feedback served to reinforce enthusiasm for Alice by the other teachers. At the second in-service meeting, one teacher made the statement that after she implemented the Alice-based lesson, 100% of her students passed the assessments for that particular activity. When asked about the range of abilities in her class, she said that the class included gifted, average, and low-performing students, and all of them passed the assessment she used. She further stated that this was the first time that such a success was realized. A few of the other teachers began to share similar experiences. By the end of the in-service, teachers of science, math, English, Spanish, French, and programming had all noticed that their implementation of Alice produced better performance by their students, regardless of learning ability. A summary of some of these testimonials follows:

3.1 Business Teacher In a business class with 25% underrepresented, 10% underserved students, the teacher commented that students who lose focus very easily were very engaged while working with Alice. More importantly, the teacher felt that some of the low ability students produced Alice Worlds that were just as good as those of higher ability students.

3.2 English Teachers In a ninth grade English class with 34% underrepresented students with a large majority in the special education program, the teacher noticed that students became so engaged that it took two to three class sessions on Storytelling Alice before they could move on to the actual English lesson. This teacher was observed by the school administrator during one of the Alice-based lessons, and he commented that the students were practicing at least five of the six levels of Bloom’s taxonomy. Students felt empowered and were excited to share their worlds with others. The students who learned to use Alice quickly taught others having problems.

An English teacher in a rural school where 17% of the students receive free or reduced lunches employed Alice in both honors and regular English classes. Her students loved experimenting with Alice, problem solving with Alice, and sharing discoveries with each other and the teacher. They loved presenting their work in new and creative ways.

3.3 Computing Teachers A computing teacher impacted directly and indirectly a small group that included special needs through high honor students. One of the special needs students has Down’s syndrome, and this teacher considers the impact on this student to be the most significant. This student is currently in a work program at the school. When he was introduced to Alice, he became excited when he saw the cow in the gallery. When he has time in between jobs, he is given the option of working with Alice, and the teacher reports that he looks forward to his time using Alice. This teacher also presented a workshop to teachers at two schools. The teachers taught grades K-12. The greatest excitement came from the elementary school teachers who believed that Alice could be used to help young students remember words/letters through animation. Another computing teacher teaches at a school where 29% of the students receive free or reduced lunches. All ability levels are present in this class. One of the interesting tasks given to the students was to develop Alice worlds for use by the MultipleHandicapped Program (MHP). The teacher reported that the MHP students love the animation and sound features in the students’ Alice worlds and ask their teacher when the students will return with more Alice worlds for them.

3.4 Math Teachers One math teacher used Alice to teach the commutative property of addition to students in grades 9, 10 and 11. This teacher reported that the students were excited to go to the computer lab to work with Alice. Another math teacher used Alice in Algebra I Honors and Algebra II Honors. Lower level math students were shown very little about the Alice program and given permission to work on their own. The students found the program interesting and wanted to learn more. The most interesting example of a math teacher’s use of Alice was share by one of the SPIRIT foreign language teachers. Apparently, a math teacher at the same school was walking by the foreign language teacher’s class the day that they happened to be using Alice. When he noticed how engaged the students were, he asked for more information. His students had been struggling with math word problems for a week or two, and he thought that Alice might be a way to help the students understand. He created a world to explain a specific math word problem and discovered that the students understood almost instantaneously.

3.5 Alice’s Impact in Spanish One Spanish teacher teaches at a school where 45% of the students receive free or reduced lunches. When the students were introduced to Alice, they enjoyed the experience and easily comprehended programming concepts from using Alice. Unfortunately, lack of memory and other issues have caused many system crashes, and this causes students to avoid using Alice. This

teacher reported a feeling that the amount of effort needed to learn Alice took too much time away from teaching the subject of Spanish. Perhaps this experience would have been different if better systems were available and Alice was used instead of Storytelling Alice. Another Spanish teacher works at a rural school where 35% of the students receive free or reduced lunches. In this class, students use other technology, such as Quia to practice their vocabulary. Quia allows educators to develop online exercises, learning games, and assessments. [6] On days when there is extra time such as after a test, students are given the option to play with Quia or Alice for the remaining time available. After one exam in particular, this option was given to three different classes. In the first class 60% of the students selected Alice, and in the second and third classes, 100% selected Alice. Another Spanish teacher has a middle school class where she used her Alice worlds. The most unexpected “positive” outcome has been that the computer teacher says that the students only want to work with Alice. Additionally, other teachers have approached the Spanish teacher to get more information about Alice. The Spanish teacher integrated audio from native speakers into her worlds to help students see the objects and the words as well as hear them in Spanish. The other SPIRIT teachers requested a demonstration of this feature to assess its application in their respective subject areas.

3.6 Alice’s Impact in Science The most impactful results have been observed by SPIRIT’s science teachers. A teacher of chemistry used Alice to create an interactive world to illustrate the Bohr Model of electrons orbiting the nucleus in real time. The example world allowed the user to use the mouse to position inside the electron to view from the inside the electrons traversing their orbitals. This concept cannot be conveyed with words, but the Alice simulation not only showed how it works, but allowed the user (student) to view it from many different perspectives. A middle school science teacher observed the greatest impact of Alice. Half of the students in the class receive free or reduced lunches, and 20% of the students are considered special needs. The subject of the lesson was to teach students Newton’s Three Laws of Motion. Before Alice, the approach used the textbook and table top labs, but only 50% of the students were able to demonstrate true understanding beyond rote memorization. Once Alice was integrated into the lesson, 100% of the students demonstrated mastery of all three laws.

4. EXPLANATIONS FOR SUCCESS In an attempt to collect quantitative data about the impact, SPIRIT teachers were asked to complete a survey where they shared their observations from teaching the selected subject before they completed the SPIRIT program and after they integrated Alice into their classes. Additionally, they were asked to provide summary performance data for both groups. Unfortunately, an insufficient number of responses were received and many of the teachers misinterpreted the request and provided data only from classes taught after completing the summer SPIRIT program and before/after integrating Alice. Collecting this data will remain an open objective for SPIRIT, and plans are being made to collect

the pre-program data from 2009 participants before the summer program. In the absence of quantitative data, the author offers a few hypotheses that may explain the improved student performance observed by the SPIRIT teachers. Because most of these teachers integrated Alice in such a way that students had to apply course concepts to the task of creating a relevant Alice world, the current body of evidence regarding student learning may provide the most likely explanation. [1] All of the teachers reported that their students were more engaged when Alice was used. Many of the teachers used the word, “addictive” to describe what they observed regarding their students’ reaction to Alice. Getting the students’ attention is sometimes the biggest challenge a teacher may face. If Alice can be used to draw student interest in a given subject area, then students may be more open to learning about the subject area. Many of the teachers reported that their students spent much more time working on fine-tuning their Alice worlds. The additional time spent on working with Alice also forced the students to spend more time on the course concept. The additional time likely helped solidify the students’ understanding better than past approaches that did not use Alice and where students spent less time. The additional interest of the students could perhaps be related to the increased enthusiasm of the teachers themselves. Alice provided a new tool to reduce the boredom that has been reported with overuse of PowerPoint slides and the like. On a related note, curiosity about the new tool motivated students to spend more time with the tool, and the subject. It will be interesting to track whether this interest remains in a few years once Alice is no longer a novel as it is today. Many of the teachers used the viral approach to spread the use of Alice across the school. As teachers wondered why students were spending so much time on the computer, many thought that they, too, could use it to draw more student interest in their subjects. In some cases, students who got to use Alice were viewed as having a perk that other students lacked. When a person has something that draws considerable attention, others want to get it for themselves, too. As of mid-spring, there have been three in-service meetings with the SPIRIT 2008 teacher participants. Most of the teachers had implemented at least one lesson by mid-spring, but a couple teachers had not. There were three explanations for this situation. One teacher lost his job and was only been able to find work as a substitute teacher. He plans to implement his Alice-based lessons as soon as he finds a permanent teaching position. One teacher faced tremendous obstacles from her school regarding installation of Alice on their computer network. Although the SPIRIT team offered to provide on-site technical support, the school refused the free support citing security concerns. When the team provided the school with a copy of their signed commitment letter in which they promised installation of Alice and the approximate investment SPIRIT had made in the teacher, they finally agreed to do the installation. A few teachers selected lessons that they typically covered during the latter part of the school year. These selections resulted in a

situation where some teachers had not implemented any Alicebased lessons until after the second in-service meeting. Although these three situations produced less than desirable results for the individual teachers, the SPIRIT team learned important lessons and was able to develop an enhanced process to ensure that the 2009 program will minimize such problems. For example, by requiring the technology support person to provide a support letter to SPIRIT, the teachers accepted into the 2009 program will already have the necessary computing facilities in place when they return to their schools so they can deliver their Alice-based lessons as planned.

5. CONCLUSION The majority of teachers that completed the 2008 summer SPIRIT program have experienced positive results from their Alice-based classroom lessons. This paper shared anecdotal results, the majority of which were overwhelmingly positive, from teachers and provided possible explanations to support those results. Although there is a lack of quantitative data to explain the amount of and reasons for these successes, the fact that there have been many successes across multiple disciplines suggests that further study regarding Alice integration in middle school and high school is needed.

6. ACKNOWLEDGMENTS This work was supported by a grant from the National Science Foundation, #DRL-0737679.

7. REFERENCES [1] Barron, B. & Darling-Hammond, L. 2008. Powerful Learning: Studies Show Deep Understanding Derives from Collaborative Methods. Edutopia. (October 2008). DOI= http://www.edutopia.org/inquiry-project-learning-research.

[2] Carnegie Mellon University. 2008-2009. Alice: An Educational Software that teaches students computer programming in a 3D environment. DOI= http://www.alice.org/. [3] Harriger, A. 2008. Finding Success through SPIRIT. Journal of STEM Education. 9, 3&4 (July-December 2008), 31-36. DOI= http://ojs.jstem.org/index.php?journal=JSTEM&page=article &op=view&path[]=1403&path[]=1229. [4] Harriger, A., Dunsmore, H., & Lutes, K. 2007. Surprising Possibilities Imagined and Realized through Information Technology (SPIRIT). NSF, DRL- 0737679. DOI= http://www.nsf.gov/awardsearch/showAward.do?AwardNum ber=0737679. [5] Munson, A.H., Moskal, B., & Harriger, A. Surprising Possibilities Imagined and Realized through Information Technology (SPIRIT): Attracting High School Students to Information Technology. In Proceedings of the 2009 American Society for Engineering Education (ASEE) Annual Conference & Exposition (Austin Convention Center, Austin, Texas, June 14-17, 2009). ASEE, Austin, TX, (inpress). [6] Quia Corporation. 2009. About Quia. DOI= http://www.quia.com/company/educators.html. [7] Rodger, S., Hayes, J., Lezin, G., Qin, H., Nelson, D., Tucker, R., Lopez, M., Cooper, S., Dann, W., & Slater, D. Engaging Middle School Teachers and Students with Alice in a Diverse Set of Subjects. In Proceedings of the Fortieth SIGCSE Technical Symposium on Computer Science Education, pages 271-275, March 2009.