Innovative Instructional Module Uses Evaluation to

Innovative Instructional Module Uses Evaluation to Enhance Quality Martha C. Monroe, Annie Oxarart, Tracey Ritchie and Christine Jie Li

Abstract

The instructional module, Southeastern Forests and Climate Change, is an example of innovation in sustainability education. The module was designed for high school science teachers and developed as part of a research project on southern pine productivity in a changing climate. As a result, it combines climate science with pine ecophysiology and economic productivity. It also encourages classroom debate and role playing activities to explore relevant ethical issues. It deftly brings together science education and education for sustainability. The process of developing the instructional module utilized a needs assessment, experimentation, and evaluation which improved program quality. The summative evaluation provided insights about the success of the program. This tight coupling of evaluation and program development created a high quality product that educators are requesting and using. Keywords

Climate change Curriculum development STEM Education for sustainable development Environmental education

M.C. Monroe (&) A. Oxarart T. Ritchie School of Forest Resources and Conservation, University of Florida, 110410, Gainesville, FL 32611, USA e-mail: [email protected] A. Oxarart e-mail: [email protected] T. Ritchie e-mail: [email protected] C.J. Li School of Natural Resources, University of Missouri, Columbia, MO, USA e-mail: [email protected] © Springer International Publishing AG 2018 W. Leal Filho et al. (eds.), Handbook of Sustainability and Social Science Research, World Sustainability Series, https://doi.org/10.1007/978-3-319-67122-2_23

401

402

1

M.C. Monroe et al.

Introduction

There is a striking similarity between instructional materials that help learners understand the nuances required for sustainability to become reality and environmental education (EE) materials (Disinger 2001; Eilam and Trop 2010; Jickling and Wals 2012; Kopnina 2012; McKeown and Hopkins 2003; Monroe 2012; Sauvé and Berryman 2005). While there can be differences between these two forms of education, such as education for sustainable development (ESD) activities that limit their focus to economic growth or EE programs that only teach about nature, there can also be significant overlap. In cases where the topic of study is an environmental issue and students are learning about it through various lenses, including economics and social justice, the content is clearly situated in both camps. One of the first manuals of EE activities, for example, organized exercises into five concepts that cover all sustainability components: ecosystems, population, economics and technology, environmental decisions, and environmental ethics (Stapp and Cox 1974). For education to adequately prepare learners to imagine and work toward sustainable solutions to environmental problems, it is essential that they appreciate and understand the potential conflicts and synergies within the three pillars of environment, economy, and society. Educators must aim to build students’ skills and agency in critical thinking, systems thinking, and action taking. To fulfill these goals, instructional materials should also be interdisciplinary. Interdisciplinary education has been discussed and debated with a variety of assumptions and definitions (Klein 1990). The process of integrating different subject areas can result in a multi-disciplinary approach that provides students with a series of perspectives and expects the student to link them together, an interdisciplinary program that coalesces various perspectives and may involve team teaching around a common theme, or a transdisciplinary approach to education that may restructure the curriculum into a kaleidoscope of possibilities (Klein 2006). Not only is there a good match in content between ESD, EE, and interdisciplinary education, there is also similar pedagogy. Klein (2006) suggests that teachers use innovative approaches in interdisciplinary education to “promote dialogue and community, problem-posing and problem-solving, and critical thinking” (p. 15). Echoing the emphasis Dewey and Piaget placed on projects that use real challenges, interdisciplinary teaching also uses inquiry, constructivist, and student-centered approaches (Ellis and Stuen 1998). These strategies have long been a hallmark of quality environmental education. It may seem obvious that solutions to environmental challenges are best approached from multiple perspectives, but the reality in the United States is that schools, courses, textbooks, and teachers are organized around disciplines: biology, chemistry, history, economics, etc. Even though environmental themes such as energy could be the basis for interdisciplinary instructional materials, teachers tend to select those portions that best fit their discipline-based course (Ireland and Monroe 2015) and continue their disciplinary tradition, teaching about the biology or economics of using wood for energy, for example, but rarely linking the two. To

Innovative Instructional Module Uses Evaluation …

403

create materials that are implemented in an interdisciplinary fashion might require a topic that is not easy to subdivide, individual concepts that have roots in multiple disciplines, or exercises that do not require expertise to facilitate and encourage teachers with any disciplinary background to engage. An additional challenge to interdisciplinary sustainability education in the U.S. is the current emphasis on STEM education, emphasizing science, technology, engineering, and mathematics, in part because of the lure of future jobs in a technological world. While somewhat interdisciplinary, STEM education favors themes such as robotics, genetics, and computers more often than environmental challenges such as energy, agriculture, or climate change. But solutions to these environmental issues will also involve technology and mathematics, and they are appropriate STEM topics as well (Holdren et al. 2013). Science education materials in U.S. schools often miss the opportunity to link science principles to current issues and build skills that will enhance stewardship and sustainability. In some communities current and controversial issues are avoided, even though they can motivate students to become more engaged in civic practice (Klosterman and Sadler 2010). Our background in environmental education, rather than science education, provided a framework for addressing skills that lead to both student and community outcomes. In addition, the constraints of the U. S. education system guided us to develop this material to reflect the science objectives that teachers are required to meet. The opportunity to create a novel instructional package brings a responsibility to use evaluation strategies throughout the program development process to assure that the material will meet needs and function as intended. In addition to providing an orientation to the innovations associated with this instructional module, this chapter reports the results of the evaluation process, conducted in four phases: needs assessment, quasi experiment, formative evaluation, and summative evaluation.

2

The Opportunity

In 2011 the authors joined a team to begin a six-year, grant-funded project that focused on managing pine plantations in the southeastern United States in a changing climate. The project, PINEMAP (Pine Integrative Network: Education, Mitigation, and Adaptation Project), was funded through the USDA National Institute of Food and Agriculture. As an integrated project, it included biological, ecological, and policy research; education; and outreach to stakeholders. One of the education activities was the development of an instructional module for middle and high school science teachers, Southeastern Forests and Climate Change. The instructional module was closely based on the framework and objectives that defined PINEMAP’s research activities in tree physiology, genetics, soil carbon, forest management and landowner preferences, and life cycle analysis. Those

404

M.C. Monroe et al.

objectives were derived from the overall goal of the project: to create, synthesize, and disseminate knowledge that enables southern private forest landowners to • Manage forests to increase carbon sequestration, • Increase the efficiency of nitrogen and other fertilizer inputs, and • Adapt forest management approaches and plant improved tree varieties to increase forest resilience and sustainability under variable climates. These goals specify the forest management and biology research goals, but their implementation required a number of other research activities, which are articulated in the outcomes for the project: • Increased carbon (C) sequestration from silviculture and genetic enhancement of productivity and efficiency of fertilizer use, and resilience to climate variability and disturbance; • Engaged and literate public with the capacity to make informed, practical decisions related to climate, forest ecosystems, and forest management; • Public policy that supports sustainable management of planted pine under future climate scenarios; • Enhanced capacity for regional, interdisciplinary collaboration among climate and forest scientists and Extension and education professionals; • Enhanced connections between corporate and non-corporate forest landowners and forestry and climate researchers and education and outreach professionals; and • A more robust and resilient forest-based economy in the Southeast U.S. The instructional module was developed to achieve the outcome of building an engaged and climate literate public, and does so by helping students understand carbon sequestration, genetic enhancement of trees, climate impacts on forests, forest impacts on climate, and the role of consumers in selecting products that mitigate climate change. Because our focus is on the links between forests and climate, the module does not venture into energy efficiency and reduction of fossil fuel combustion. Although most students will not become forest landowners or forest managers, they may travel through the southeastern region, appreciate forested landscapes, and purchase wood products. We used these assumptions to keep the module content relevant to learners in the southeastern U.S. In keeping with our commitment and orientation to environmental education and education for sustainability, the module goals also reference skills and attitudes that are important for the development of learners who will help move their communities toward sustainability: • Understand how climate change could impact forests in the southeastern U.S.; • Understand how forests can be managed to address changing climate conditions and to reduce greenhouse gas emissions;


405

• Enhance decision making skills to make informed choices as consumers to mitigate climate change; • Develop systems thinking skills to understand connections between climate change forests, and people; • Recognize that individual and community actions can help mitigate and adapt to climate change; and • Become part of future community conversations about climate change and potential solutions. The module scaffolded concepts so that basic information oriented learners to the principles (such as the carbon cycle) before applying forest specific information (such as measuring carbon storage in trees). Similarly, in order to understand the impact that consumers can have on carbon emissions, it was necessary to introduce life cycle analysis and externalities. Linking together carbon sequestration and product life cycles, a culminating activity explores sequestered carbon in forests and wood products, as well as carbon that is “saved” through wood substitution. Additional activities introduce students to the history of climate science, evidence of climate change, climate models, genetic variation in loblolly pine, and forest management strategies to improve forest resilience. Similar themes were organized together into sections, and section introductions provided background information for teachers that was common to each of the activities in that section (Table 1). To enhance the dissemination of the material and give it a long-term home for future adaptations, we partnered with Project Learning Tree (PLT). This U.S.-based environmental education program develops instructional materials and manages state coordinators who train workshop facilitators to deliver professional development to educators. PLT offers issue-specific modules for secondary teachers (teaching 13–18 year-old students) but did not yet have a module on climate change. PLT staff and coordinators were part of the development process from the beginning of the project and provided suggestions to improve activities and evaluations.

3

Innovatively Addressing Sustainability, Interdisciplinarity, and STEM Goals

The topic of private forest management in a changing climate sits squarely between the economic and environmental aspects of sustainability. Several of the 14 activities incorporated discussion or worksheet questions to help learners focus on these elements. In addition, some discussion questions focused on aspects of social justice. For example, the activity of measuring carbon stored in a tree is extended by comparing the carbon emitted by a state’s population to that sequestered in the state’s landscapes with discussion questions about whether other states should be responsible for sequestering “our” carbon or if cities should pay rural communities to sequester their carbon waste. In another activity, students roleplay members of a

Theme

Three activities introduce the module theme by conveying how scientists currently understand observed changes in weather and climate that are impacting forest ecosystems.

Climate changes are projected to affect surface temperature, precipitation patterns, and frequency of storm events. As scientists study how forests might change as a result, forest managers can be encouraged to alter management practices to help create resilient forests that will survive these challenges.

Sequestering carbon in trees, soil, and wood products keeps it out of the atmosphere. Scientists are exploring if we can sequester more carbon in these carbon pools.

Section

1. Climate Change and Forests

2. Forest Management and Adaptation

3. Carbon Sequestration

Table 1 Southeastern forests and climate change activities by theme Activity 1. Stepping through Climate Science—Students walk along a timeline of climate science and policy initiatives and then explore connections between forests and climate. 2. Clearing the Air—After an introduction to the evidence of climate change, students explore common confusions and role-play a community discussion with the goal to reach consensus on strategies to reduce greenhouse gas emissions. 3. Atlas of Change—Students are introduced to climate modeling to understand past changes and project future possibilities, and then use Web resources to consider how forest ecosystems might change over the next 100 years. 4. The Changing Forests—Students review how scientists are monitoring forest changes and exploring adaptive strategies to keep forests healthy. 5. Managing Forests for Change—Students develop and use a systems diagram to model a forest so they can advise a forest landowner how to manage a pine plantation in light of climate projections. 6. Mapping Seed Sources—Across the native range of loblolly pine, variations in genotype create trees that may do better under new climatic conditions. This activity helps students analyze data from three trials to determine the origin of the seeds. 7. Carbon on the Move—Students become familiar with the carbon cycle and pathways that increase and decrease atmospheric carbon. 8. Counting Carbon—Students measure trees near their schools and calculate the amount of carbon stored in individual trees. Students then compare the carbon (continued)

406 M.C. Monroe et al.

Theme

Consumer choices can play a role in reducing and preventing greenhouse gas emissions. These activities introduce the concept of externalities to consider the environmental problems that can occur from the production, shipping, and disposal of various products. Greenhouse gas emissions are one of the many criteria that students can use to assess products as they develop their own personal code for deciding what to purchase.

Three activities that help teachers summarize the concepts in this module. These can be adapted to reflect the activities that teachers selected. Students can be empowered with the knowledge and hope that all of us can help work toward healthy, sustainable forests and communities.

Section

4. Life Cycle Assessment

5. Solutions for Change

Table 1 (continued) sequestration potential for land-use types in their state, compare this to the estimated amount of carbon released by human activities, and discuss forests’ ability to sequester atmospheric carbon. 9. The Real Cost—Through a simulated shopping activity, students learn about the impact, or externalities, of consumer choices on the environment. 10. Adventures in Life Cycle Assessment—Students investigate life cycle assessment data for three types of outdoor dining furniture to determine which type would generate the lowest amount of greenhouse gases. This detailed analysis of inputs and outputs is another tool for systems thinking. 11. Life Cycle Assessment Debate—Students debate four pairs of similar products to develop their own sets of questions about product life cycles that can help guide consumer choices. 12. The Carbon Puzzle—Students use a series of facts to realize how forest plantations, wood products, and wood substitution can reduce atmospheric carbon, and then interpret a graph published by the researchers who explored this concept. 13. Future of Our Forests—Student teams review information from the module and share their knowledge with an appropriate audience. 14. Starting a Climate Service-Learning Project— Students select and complete an action project to mitigate climate change or help their communities adapt to projected changes.

Activity

Innovative Instructional Module Uses Evaluation … 407

408

M.C. Monroe et al.

community committee assigned to develop recommendations to reduce potential impacts of climate change. Roles for committee members represent a diversity of opinions about the causes and importance of climate change that reflect the range of opinions of the public. This leads students to consider strategies that seek agreement on actions rather than causes, and debate the trade-offs associated with economic and environmental benefits. A series of activities lead students to understand externalities and life-cycle assessments, and then debate the impacts of product pairs: e-book and paperback book, paper cups and drinking glasses, plastic bottles and aluminum cans, or paper and plastic bags. A written assignment asks students to develop their own criteria for making purchasing decisions and explain which would be more important to them. While the module was designed for science teachers, the activities incorporate concepts from social studies and skills from language arts and mathematics (Table 2). Recognizing that teachers may not have the background to feel comfortable with concepts outside their disciplinary training, the module and accompanying website includes: (1) significant background information for teachers, (2) discussion questions for each activity, as well as appropriate responses, (3) worksheet answer keys, (4) lists of common misconceptions and clarifying corrections about each major concept, (5) slide presentations with teacher notes to help explain the activities, (6) short videos of PINEMAP research professors and graduate students providing additional background, and (7) links and references to additional resources and materials. Climate policies are introduced in three activities, and the economics of forest management and wood products are featured prominently in four, supporting the social studies component. Students are asked to explain beliefs and assumptions, debate products, and design posters in activities that address language arts objectives. Many of the activities in this module also address the objectives of STEM education. Strong science themes, such as evidence for climate change, the carbon cycle, and genetic variation within a population, form the backbone of the module. Technology and engineering concepts are introduced through a detailed comparison of the life cycles of plastic, aluminum, and wood picnic tables and the function of Table 2 Subject correlation by activity SUBJECT

ACTIVITY 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Agriculture (including forestry) Biology Chemistry Earth Science Environmental Science Language Arts Mathematics Social Studies (including economics, government)

X X X X X X X X X X X X X X X X

X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

X

X X

X X

X X

X X

X

X X

X

X X

X


409

models to explain and predict phenomena. Technology is also alluded to in potential solutions to climate mitigation and adaptation and the management options to create more resilient forests. Mathematics skills are taught and practiced, such as creating and interpreting graphs, calculating carbon-equivalent emissions, and using trigonometry to calculate tree height (Table 3). Table 3 STEM connections to each activity Activity

STEM Connection

1

Stepping through Climate Science

2

Clearing the Air

3

Atlas of Change

4

The Changing Forests

5

Managing Forests for Change

6

Mapping Seed Sources

7

Carbon on the Move

8

Counting Carbon

9

The Real Cost

10

Adventures in Life Cycle Assessment

11

Life Cycle Assessment Debate The Carbon Puzzle

• Understand the progression of science findings over time • Create a graph of atmospheric carbon over time • Make observations about the relationship between science and policy • Explore scientific evidence of climate change • Understand the causes of climate change • Develop a chart of criteria for making an informed decision • Learn about computer models • Use a computer model to understand the impact of climate change on forests • Use data from a computer model to create a poster • Explore five scientific studies that scientists are currently doing • Use a systems diagram to convey forest ecology • Consider management strategies that can help a forest adapt to climatic changes • Analyze data and explain hypothesis about heredity • Graph data and interpret results • Explain carbon cycling and the ways in which carbon can be removed from and added to the atmosphere • Illustrate the carbon cycle, including carbon pools and fluxes • Collect data • Practice using field tools to measure trees • Compute comparisons of carbon sequestration and emissions • Apply concepts to determine whether a state could be carbon neutral • Understand how technology affects the environmental impacts caused by a product • Understand how products are engineered • Calculate the emissions of three products at each step of their life cycle • Assess environmental impacts of common products • Draw conclusions based on information assessed • Interpret a graph • Understand how carbon moves through three pools • Synthesize climate and forest science • Develop problem solving skills as they 0plan and implement a project

12 13 14

Future of Our Forests Starting a Climate Service-Learning Project

410

M.C. Monroe et al.

Finally, content is not the only avenue for addressing sustainability and STEM goals. Pedagogy can also be used that cultivates attitudes, empowers learners, reinforces skills, and builds capacity for change. Several of the most commonly used pedagogies in EE and ESD were employed in this module: experiential learning and reflection, small group discussions, jigsaw discussions, group projects, and community action projects. Other engaging techniques, such as theatre, debate, and solving a mystery were used to stimulate learning.

4

Methods

Evaluation was an integral aspect of this program’s development to answer questions about the design of the program and to assess the quality of the product (Ernst et al. 2009; Patton 1997). We used a combination of a needs assessment, a quasi-experiment, formative evaluation, and summative evaluation to collect data from secondary educators and students.

4.1 Needs Assessment Needs assessments are typically conducted at the beginning of a project to help frame the program. In our case, however, the proposal and the research activities narrowed the realm of possibilities regarding the topic, audience, and purpose. Within those limitations, we began to design the objectives of the activities, which led us to a series of questions that teachers could answer to provide guidance. Our first assessment of our audience, therefore, involved questions for programmatic guidance rather than traditional needs. To collect data for the needs assessment, we conducted an online survey of science teachers in the southeastern U.S. (Monroe et al. 2013). Survey questions were developed, reviewed by an Advisory Board of 24 educators, and pilot tested with practicing teachers. The survey contained 28 questions regarding current and future preferences for including climate change in secondary science courses, knowledge and comfort for teaching about climate change, usefulness of instructional materials, educational goals, and demographics. The survey invitation and three reminders were sent through state science coordinators, environmental education coordinators, and environmental education associations, and recipients were encouraged to share the link with colleagues. As we did not have access to the email lists, we do not know the overall population size, nor could we assess non-response bias. We assume that respondents likely represent those educators most interested in teaching about climate change in the region, and therefore most likely to use supplemental module on climate change.


411

4.2 Quasi-Experiment Reactions of conservative adults when conversations turn to climate change range from polite indifference to physical movement away from the speaker. If some students feel strongly about climate change, they are not likely to engage in learning. As we designed some of our lessons we had a choice about whether to reveal the climate connection at the end of the lesson, or at the beginning. We wondered which would lead to increased knowledge. Our second data collection opportunity was a quasi-experiment to explore this important question. We designed a quasi-experiment with two equivalent groups of youth (ages 15– 17) during a summer science camp (Monroe et al. 2016). After a pre-test of carbon knowledge, one group was introduced to the connections between carbon, climate change, and forests. This group learned that adding fossilized carbon to the atmosphere is one important cause of climate change and that trees can sequester carbon and thus be a potential solution for removing carbon from the atmosphere. The other group was introduced to carbon as a ubiquitous element and learned about the carbon cycle with trees as one carbon pool. Both groups completed an activity reinforcing the carbon cycle and measured carbon in nearby pine trees. A post-test was conducted with the second group before continuing the discussion about human-generated carbon dioxide emissions and the carbon sequestration potential of ecosystems across the state.

4.3 Formative Evaluation Additional opportunities to interact with educators and students followed the more traditional expectations for formative evaluation–gathering input on the structure of the activities, areas of confusion, practicality of the materials, time requirements, and to collect teachers’ ideas for adaptations. In particular, advisors and reviewers suggested that our draft materials were most appropriate for Environmental Science and Advanced Placement classes that are typically taken by students 16–18 years old. We were uncertain about whether the activities could be meaningful to younger students and what adaptations might be necessary for the activities to be successful. We conducted the formative evaluation during fall 2013 and spring 2014 to answer the following questions: 1. What are teachers’ perceptions of the secondary teaching module? 2. To what degree did students meet the activity objectives? 3. To what extent did these activities change students’ knowledge, skills, and attitudes? 4. How can these activities be improved? Survey items were pilot tested with students, teachers, and advisors, and refined several times. Data were collected from 28 middle school teachers who used two activities of their choice with students, ages 11–15 years, and made any adaptations

412

M.C. Monroe et al.

they wished (Li and Monroe 2015). Teachers completed an online survey that captured their perceptions about the value of the activities, whether or not the students met the activity objectives, and the ways they revised them.

4.4 Summative Evaluation Our summative evaluation assessed student learning and enabled us to discover if the core assumption of this program–that science teachers could convey information outside their discipline that relates to a current interdisciplinary issue–was met. To collect student data, 32 teachers located in 10 southeastern states used 5 module activities with their 9–12th grade students and conducted pre and post student surveys. The survey used some of the formative evaluation questions; new items were pilot tested. Items measured knowledge, attitudes, skills, and demographics. A final evaluation tool, an online survey, was sent to teachers who received the module, either through an educator workshop, the module website, or by request. The survey invitation and three reminders were emailed approximately 6 months after the person received the module and contained questions regarding if and how the activities had been used and their perceptions of student outcomes and reactions. The survey contained three tracks for different types of educators: classroom teachers, youth non-formal educators, adult non-formal educators.

5

Results

Evaluation results were key to guiding all major decisions about the development of this module. In addition to helping us design the materials, the results allowed us to • better meet the needs and expectations of the teachers most likely to use the materials, • use teachers’ needs to help market the materials, • ask and answer questions about the structure and value of the activities, • add teacher comments about the materials to the website and final printed version as testimonials and implementation tips, and • provide our funder with details on the ways the materials were being used. This section will describe the results of our evaluations by questions that were answered.

5.1 Needs Assessment: Will Teachers Use a Unit on Forests and Climate? How Should It Be Structured? The needs assessment survey was completed by 746 respondents, who were mostly female (67%), taught in public schools (87%), and located in Florida (49%), Virginia (14%), or North Carolina (10%). The results provided important insights into


413

educator preferences and priorities (Monroe et al. 2013). For example, we wondered if science teachers would use activities on product life cycle assessments and learned that they are motivated to provide strategies students can use to mitigate climate change. In that context, 85% of the respondents were willing to include information about product life cycles and make the link to carbon sequestration. We also wondered whether and how teachers currently taught about climate change and learned that teachers tended to use informal discussions in agriculture, chemistry, and physical science courses. Environmental science and ecology courses covered climate change with planned lessons for more than one week. Biology and earth science teachers tended to spend less than one week on the topic with planned lessons. Most of the respondents already covered climate in some context (77%) and 82% intended to do so in the future. We learned that the following were among the highest priority goals for these respondents: • • • • •

Connect science to students’ lives (98%) Emphasize critical thinking skills (98%) Develop data analysis skills (94%) Emphasize choices that affect sustainability (92%) Emphasize systems thinking skills (92%)

5.2 Quasi-Experiment: Will Students Learn if the Lesson Is Introduced in the Context of Climate Change? The two groups’ pre-test scores were not significantly different, but the post-test scores from the group introduced to climate change were significantly higher than the control group, suggesting they learned more about carbon (Monroe et al. 2016). Follow-up interviews with all students suggested that linking the two concepts, carbon and climate, was critical; some students remembered learning about both in school, but did not realize the two concepts were connected. Other students said the climate context made learning about trees more interesting and relevant.

5.3 Formative Evaluation: How Could Middle School Educators Use Lessons that Are Designed for Older Students? Twenty-two middle school teachers implemented activities and completed the online survey. We learned that some activities were appropriate for younger students as written, though fewer middle school teachers than high school teachers agreed that their students were able to meet the stated objectives (mean = 3.91 vs

414

M.C. Monroe et al.

4.27 on a scale of 1–5). We gained a variety of suggestions for strategies to simplify or emphasize key concepts for the more complicated exercises. For example, one of the middle school teachers suggested that rather than dividing students into small groups as instructed in one activity, teachers could keep the class together and facilitate a discussion with the entire class. In addition, the feedback from middle school teachers resulted in a new introductory activity to connect forests and climate. The formative evaluation enabled us to revise draft activities and to include a Modification section for each activity with teacher’s suggestions about alternative formats for conducting the activity.

5.4 Summative Evaluation: Do Learners Gain Interdisciplinary Knowledge and Skills After Being Exposed to These Activities? Table 4 indicates that students in both the biology and environmental science classes increased knowledge of several concepts, even those typically included in social science classes such as life cycle analysis. The environmental science

Table 4 High school students’ knowledge scores before and after instruction with five activities from summative evaluation Concept

Pre score

Post score

Difference

T value

Students from biology classes (n = 168) 2.04 2.71 0.67 4.40 Forest management (6 items)a Carbon (3 1.29 1.56 0.27 3.07 items) Climate (2 0.91 1.07 0.15 1.91 items) Life cycle (1 0.65 0.72 0.07 1.29 item) Students from environmental science classes (n = 627) Systems (7 4.01 4.45 0.44 5.45 items) Carbon (2 1.05 1.15 0.10 3.20 items) Climate (5 2.77 3.23 0.46 6.96 items) Life cycle (3 1.32 1.93 0.61 11.46 items) a refers to the number of multiple-choice items that tested this

P value

% students who answered more correctly