Learning Science by Doing Design:

Learning Science by Doing Design: How can it Work at the Middle School Level? Yael BAMBERGER, Clara CAHILL, James HAGERTY, Harry SHORT, and Joseph KRAJCIK School of Education, University of Michigan Ann Arbor, MI 48109-1259, USA

ABSTRACT The purpose of this study was to investigate how a design-based curriculum that tied scientific concepts of energy to technological design can help students understand concepts of energy transfer and transformation, as well as increase their technological literacy. In addition, our goal was to increase our knowledge of pedagogical strategies that can help facilitate students learn the process of design in the middle-school level. Nine instructors, divided into teams, instructed the curriculum and collected data during a two-week summer science camp for 38 middle-school students from diverse economic and racial backgrounds. Data was collected through instructors’ online reflections and group discussions at the end of each day of instruction, as well as students’ pre-post tests, online reflections after each design project, and sketches of their designs during planning. Findings indicated significant improvements in students’ capability to connect the scientific content knowledge of energy to their design projects, as well as improvement of some technological literacy skills. Pedagogical strategies for implementing design-based instruction in the middle-school level are included.

Keywords: Design-Based Learning, Middle-School, Science Education, Technological Literacy, K-12 Engineering Education.

1. INTRODUCTION Science education and technology education are two separate school subjects in the National Benchmarks [1] and Standards [2]. Therefore, technological literacy is often disconnected from scientific literacy, or even neglected entirely in the school curricula [3]. Since scientists and engineers often cite the construction or dissembling of designed objects as pivotal to their identity and initial interest in science and engineering, it would seem important to embed design and technology throughout the curriculum [4]. George DeBoer [5] calls for such a reform, and exclaims that technological literacy should be one of the goals of school science teaching. He claims that science education should include investigations and discussions about the nature of technology, and the practice of skills required to technological design. Technological design is central to the work of engineers, and incorporating design at an earlier age could be a way to support awareness of engineering career [6] [7]. Design also has been discussed in the educational literature as a tool to enhance science learning. Design-based learning is a learning-by-doing strategy which help students understand scientific concepts [8] [9] [10]. The design artifact helps to focus students’ attention, and can be used as a concrete referent from everyday life to refer

to [11]. A design project consists of a problem-based challenge with an open end that leaves ample opportunity for individual creativity [12]. Design projects can increase engagement and motivation for learning by providing students active learning along with opportunities for choice [5]. Among the different design-based instructional strategies for the middle- and the high-school levels described in the educational literature, there are both correlations and different emphases. For instance, the ITEA [2], Davis, Hawley, McMullan and Spilka [13], and Burghardt and Hacker [14], emphasize the importance of generating alternative solutions and exploring a variety of possibilities during the planning step. The final communication step of the process, which includes presenting both the achievements and the process, is highlighted particularly by Kolodner and colleagues [10], Burghardt and Hacker [14], and the ITEA [2]. All of those different design-based instructional strategies come to guide and facilitate teachers the instruction of science through design. Design-based teaching may be difficult for teachers because it is extremely open-ended and student-driven. Students may come up with a wide array of design solutions that can meet the challenge, and some of them can be unexpected by the teacher [8]. Teachers concerns on and difficulties of teaching science through design in the middle-school level has not been studied yet. The goals of our efforts were to increase our knowledge of teachers concerns on teaching science through design, as well as pedagogical strategies that can help facilitate students’ understanding of the process of design. In the current study, we developed and taught a curriculum that tied scientific concepts of energy to technological design. The research questions that guided the study were: 1) What pedagogical strategies can help teachers facilitate the design process for middle-school students? 2) What concerns do teachers have about teaching science through design, in the context of teaching energy transfer and transformation?

2. RESEARCH CONTEXT A fifty-hour curriculum was developed to engage middle-school students from diverse economic and racial backgrounds in learning about forms of energy and technological design. In design projects tied to contemporary issues in sustainability, the students explored various energy forms and how energy can be transferred and transformed. Throughout the curriculum, sensemaking discussions, reflections, and practical experiences helped students develop an understanding of energy concepts and the process of design. The curriculum was piloted during a non-selective, free, two-week summer science camp. The 38 students (15 females and 23 males) were all middle-school students from a diverse Midwestern public school district in which over 50% of the students qualify for free or reduced-price lunch. Students were placed in mixed-race, mixed-gender,

single-grade teams of three, and worked in these same teams throughout the entire camp. During the camp, nine instructors taught the curriculum. These instructors included three undergraduate science students who were also engaged in a program to support K-12 classrooms, one middle-school science teacher, one district science and mathematics curriculum specialist, and four science education researchers, three of who were graduate students. Teachers were divided into three teams, and during each lesson one of the teams was responsible for the instruction, one for the data collection, and one for giving support when it was needed, by helping students in different stages of the design process. The instructional intervention included inquiry-based explorations of phenomena, laboratory experiments, small group and whole class discussion, and design projects, as well as a field trip and a presentation from an engineering team. The design projects and their related energy content are described in Table 1.

Topic Introduction to Design

Kinetic Energy Electrical Energy Solar

Table 1. Design projects Driving question Design project How can I wake up my brother staying in my bed?

Build a device to drop a Teddy bear off his bed from a distance

How can I light up my room with a bike? How can wind charge my iPod?

Build a windmill to produce electricity

How can I use the sun to bake cookies?

Build a solar cooker that can heat water

Through all the design projects, students followed a strategy of design, which we synthesized from previous scholars’ work on design-based instruction for middle-school students [8] [10] [15]. The process included the following stages: 1) Defining the problem – students identified the problem their design addressed, their goals for the design, and their criteria for success. 2) Gathering information – students identified constraints, identified the users of the design, researched previous designs, generated several alternative designs with their team, and considered the strengths and weaknesses of each possibility. 3) Planning – students selected their optimal design, and made a ‘storyboard’ panel, which included a labeled diagram and description of their design, as well as an explanation of why they believed their design would be successful. No materials were given until the plan was presented and explained to a teacher. 4) Building – students constructed a 3D prototype according to their plan. As they built, they reflected on difficulties in creating their models and on how they adapted their plan to overcome these difficulties. 5) Testing – students tested their designs publicly, using a quantifiable ‘test against nature’ [15]. 6) Evaluating – students recorded and reflected upon how their designs met all of the criteria of success, and evaluated what was successful about their designs. Students identified and explained ways they could

improve their designs, gathering additional information if necessary. 7) Redesigning – students drew a new storyboard panel to demonstrate how they were going to change their designs, and why they expected their new designs to help them better achieve their criteria for success. When their redesign gained teacher approval, they were permitted to return to the ‘building’ stage. 8) Communicating – students presented their work during the whole class session. Groups shared their design process, identifying how they chose their design, their test results, how they improved their designs after testing, what strengthens and weaknesses of their designs were, and what they had learned from the designs of other teams. Labels with the eight steps of our strategy of design were hanged on the wall, and teachers pointed on the current step through the instruction. Through their work, students could go back and forth between steps, according to the needs of their design. The three design projects - wake up your brother, windmill, and solar cooker - were embedded into the curriculum in different stages. Here is a short description of these projects. The first design project: Wake-up your brother On the first day of camp, the students were introduced to the process of design through this lesson. Students learned about defining the design problem, including the criteria for successful design, gathering information about constraints, the users and about previous designs. Students were asked to imagine they share their room with their big brother who has to go to work early in the morning. The clock alarm the brother sets doesn’t wake him up, but instead wakes up the student. The design challenge was to design a device that will allow the student to knock the older brother out of bed without getting up. By using provided materials, the device should knock the older ‘brother’ (represented by a Teddy bear), off his ‘bed’ (represented by a plastic bin), from a distance of 4 feet. Through a whole group discussion, students were led to gather information about machines that control things from a distance, such as surgeries, military equipment, and robots. A design prototype was shown in order to help students understand what they were expected to do, as was suggested by Sadler and colleagues [15]. After that, each group got a piece of butcher paper to plan their design, and a picture of the materials they were going to get, including dimensions of the materials. Once students presented a teacher with an adequate plan, they were given materials to begin building. The testing stage was done in public. Each group tested its product in front of the whole classroom as suggested by Sadler and colleagues [15]. Groups with unsuccessful designs were given the opportunity to redesign their product, and groups with successful designs were asked to improve their product to meet a new challenge: waking their ‘brother’ from a distance of 6 feet. The second design project: Build a windmill to produce electricity This lesson was embedded in the middle of the curriculum, after students were exposed to electrical and kinetic energy concepts, and conducted experiments to better understand these two types of energy. Students were asked to build technologies to harness kinetic energy from the wind and convert it to electrical energy. For gathering information, they watched a movie about different types of windmills, and discussed pros and cons of each of them. Students had a challenge to design blades for a windmill in order

to produce electricity and light up a bulb. Each group was given a graph sheet to plan their design, a list of the materials they could choose from, with prices and quantity limits for each object, and a budget sheet. Thus, students were compelled to plan carefully to stay within their budget allowance. This encouraged students to think deeply about their plan and to draw a more scientific model of their design, and added an economical perspective to the design, representing a real-world constraint. Once groups had received teacher approval for their plans and budget, they were permitted to purchase their materials in the ‘store’. Each group attached their blades to the turbine and measured the current produced when the blades spun in the ‘wind’ produced by a stationary fan. Groups who did not successfully produce measurable current were given the opportunity to redesign their product, and groups who successfully produced current were challenged to increase their current output. The third design project: Build a solar cooker After learning some aspects of solar energy, and meeting with engineers who designed a solar car of a University, students were introduced to the challenge of designing a solar cooker. They discussed constraints and criteria for success with the whole class. The class continued to gather information together, discussing the concepts they had learned about in their experiments they did in class that could help them think about their designs, and watched a short video showing many different types of solar cookers. As for the windmill design project, they received a list of materials and cost, and had to plan and ‘buy’ their materials with a limited budget. In addition, groups were asked to draw an energy flow diagram in their plan sheet that detailing the energy transfers and transformations in their design. In order to test the design product, a beaker of water with a thermometer was placed in each solar oven. The students measured the temperature of the water after every 5 minutes, for about 20 minutes. Groups evaluated their performance, and redesigned their ovens, based on how they had performed in the initial test.

3. METHODOLOGY Data collection In order to improve our understanding of several aspects of learning science by doing design, we implemented diverse research tools that aimed to collect both teachers’ and students’ perspectives. The data that was collected through the teachers’ materials enabled us to reveal both pedagogical strategies and concerns teachers have regarding teaching science through design. The data gathered through students’ materials reflected upon the ways students connected the scientific knowledge about energy to their design projects, and their understanding of the design process. Here is a description of the research tools: Teachers’ online reflection. At the end of each day, teachers gathered to reflect upon the curriculum and instruction. Written reflections were submitted online to a wiki space. The following questions guided these reflections: What went well in today’s lessons? How could today’s lessons have been improved? What are some next steps for upcoming lessons? What do we need to improve? How did you feel about the lessons today? Teachers’ group discussions. After submitting the written reflection, teachers discussed the day of teaching. Each teacher had the opportunity to share and discuss ideas, concerns, and feelings regarding his or her own teaching or peer teaching. Suggestions were made to adapt or modify subsequent lessons.

Each 60-90 minute group discussion was audio-recorded and transcribed. Students’ pre-post tests. At the beginning and the end of the camp, a pre- and post-test was conducted to assess student conceptual understanding of types of energy and energy transformations. This provided an opportunity to observe any new patterns in student thinking that emerged from the instructional intervention. This assessment focused on having student make predictions, observations, and explanations about six different devices or phenomena. For each phenomenon, students were asked to identify how energy was involved, what caused the phenomena they were observing, and to explain what evidence they had for their observations. Students were able to manipulate or closely inspect each device at each station, and facilitators at each station demonstrated phenomena related to each device. Students’ online reflection. At the end of each design project, each student had to submit an online reflection regarding the project. The questions in the self-reflection addressed how their design was related to the content of energy, and how energy was converted and transferred in their design. This tool provided information about ways students connected their design to the content knowledge of energy transfer and transformation. Students’ sketches. During the planning step of design each group of students had to draw a sketch of their design as a condition for getting materials for building the prototype. These sketches provided information about students’ understanding of the process of design, as they contained design intent and other important details. Data analysis The transcribed group discussions and online teachers’ reflections were analyzed using a grounded theory approach. Three main themes emerged from the data, reflecting the concerns and focus points of the instructors: 1) Making connections, including teachers’ ideas regarding the connections students found between the design project and the energy content. 2) Balancing between structured and mentored teaching, including teachers’ ideas regarding whether instruction should involve more lecture-based lessons or more active learning strategies, or whether the teacher should give the students information or let them explore to get the information by themselves and construct their own ideas. 3) Students' understanding of the design process, including statements regarding teachers’ concerns about specific stages of the design process, such as drawing and labeling diagrams during planning, or problems with students’ documentation of the redesign step. The analysis process was conducted by two researchers in order to establish an acceptable level of inter judgmental reliability. Themes were developed, discussed and clarified, and then each researcher coded statements independently. Ten percents of the data were scored by both researchers, with an inter-rater reliability of 0.9. All rating inconsistencies were discussed and resolved between the researchers. The students’ pre-post tests and online reflections were analyzed and scored according to ways students were identifying energy in systems and the mechanisms by which they perceived the energy was being transformed and transferred within and between objects and systems. Specifically, we evaluated students work for: 1) Accuracy: Correct association of energy types with parts of the system.

2)

General sequential flow: Correct flow of causal relationships. Each pre- and post-test and online reflection was quantitatively scored for accuracy or flow. This enabled us to identify changes in students understanding and areas of difficulty. The analysis process was conducted by two researchers, and the inter-rater reliability of 1.0 was achieved for accuracy and 0.87 for flow. All rating inconsistencies were discussed and resolved between the researchers. In addition, we analyzed students’ sketches of their design during the planning stage. Each group sketch was analyzed by the extent the sketch included the following dimensions: 1) Measurements: The amount each component of the diagram is labeled by its measurement, or by a specific object that has defined measurement. 2) Materials: The amount each component of the diagram is labeled by its material. 3) Design intent: A written explanation or justification for the design in general, or for specific components of the design. 4) Design details and clarifications: Different views of the design or zooming-in on a specific important component of the design that has unique characteristics that have to be clear. Each of those dimensions was quantitatively scored, and a total score was given to each draw that enabled us to identify changes in students’ engineering drawing skills.

4. FINDINGS Making connections One of the main teachers’ concerns reflected through the discussions and the reflection was that students were not making strong connections between the scientific concepts of energy and the design projects. Some teachers were very skeptical regarding students’ understanding of those connections. For example: ‘I don't think that any systematic content knowledge/principle(s) were learned from this (the solar cooker) activity.’ (Heather, online reflection) ‘So maybe if they think of that, and they think oh like the windmills spinning faster so it has more kinetic energy, and then it goes to the generator. And like so they just sort of thinking like oh that’s where it converts, I remember this… goes to the wire into this thing that’s giving me a number. So, so like I think at worst we’re giving them a way to use the language and see how it works.’ (Samuel, group discussion) Other teachers were less skeptical, and noticed that the ability to connect the energy content knowledge to the design is individual to each student. For example: ‘The other thing is [regarding the question of whether] students were able to use several form of energy conversion to solve technical problems, I saw a huge variety of understanding today where as um some students could not see uh how their own groups design was supposed to accomplish the goal…[and others] coming up with a brilliant answer um using different kinds of energy conversions, using like twenty steps… I think it has to do with um individual students uh motivations as well as their group dynamics as well so.’ (Jack, group discussion)

In contrast to the skeptical teachers’ voices, and aligned with the more positive voices, students’ materials showed a significant improvement in students’ understanding of energy transformation in everyday phenomena as well as in their design. The pre-post test analysis suggested that students’ ability to identify energy improved significantly, for both Accuracy (ES=2.5, p