Developing a Research Agenda in Science Education - Springer Link

C 2005) Journal of Science Education and Technology, Vol. 14, No. 2, June 2005 ( DOI: 10.1007/s10956-005-4424-4

Developing a Research Agenda in Science Education Patricia E. Simmons,1,8 Herb Brunkhorst,2 Vincent Lunetta,3 John Penick,4 Jodi Peterson,5 Barbara Pietrucha,6 and John Staver7

The Science Summit reinforced a question upon which many of us in science education are focused: How can we, the science education community of researchers, practitioners, and consumers, lead policy? We include a brief review of the No Child Left Behind Act and its implications for teachers, and elaborate about one ongoing and growing effort to answer the concerns about the paucity of research expressed at the Summit. We describe a unique and growing collaboration across professional science education and science organizations and societies that focuses on the development of a research agenda. The term ‘consilience’ refers to the “jumping together of knowledge” that leads to scientific advancements, progressive, creative, fluid scientific research and intellectual capacity to move a research community toward an enlightened research agenda. A coherent research agenda enables us to specify what we know, what we need to know, and how research can be employed for creating and implementing policy. The use of a dynamic organizer (such as Pasteur’s Quadrant) for a research matrix of topics provides a possible structure for organizing and cataloging research questions, designs, findings from past studies, needed areas for research, and policy implications. Through this unique collaboration, the science education community can better focus on needs and priorities and ensure that teachers, policy makers, scientists, and researchers in education at local through national levels have an important stake in research priorities and actions. KEY WORDS: research agenda; science education; professional organizations.

One of the areas we will address today is the need for better research into what works for science

education. This is an arena in which some progress has been made, but for the most part, we are still blindfolded and trying to find our way through a cluttered room. Much more high quality science is needed to determine what methods, what resources, what curricula, best educates students at each grade level . . .

1 University of Missouri-St. Louis, Institute for Mathematics & Sci-

ence Education and Learning Technologies, Regional Center for Education & Work, Suite 7, One University Blvd., St. Louis, Missouri 63121. 2 California State University, Institute for Science Education, 5500 University Parkway, San Bernardino, California 92407. 3 Penn State University, 155 chambers, University Park, Pennsylvania 16802. 4 North Carolina State University, Department of Math, Science, and Technology Education, 326 Poe Hall, Box 7801, Raleigh, North Carolina 27695. 5 National Science Teachers Association, 1840 Wilson Blvd., Arlington, Virginia 22201. 6 706 Third Avenue Rear, Bradley Beach, New Jersey 07720. 7 Kansas State University, 252 Bluemont Hall, 1100 Mid-Campus Drive, Manhattan, Kansas 66506. 8 To whom correspondence should be addressed; e-mail: [email protected]

(Secretary of Education Paige, Washington, DC, March 16, 2004)

With these opening remarks, the Science Summit in Washington, DC began as 700+ attendees heard leaders in education and in the sciences and defense industry make assertions about the state of science education research. As we listened to comments made by invited panelists from a variety of fields about science education and research in science education, we were intrigued by the thoughtfulness 239 C 2005 Springer Science+Business Media, Inc. 1059-0145/05/0600-0239/0

240

Simmons, Brunkhorst, Lunetta, Penick, Peterson, Pietrucha, and Staver

of some presentations and outraged by the sweeping misrepresentations offered as expert testimony by other presenters. Since there was no opportunity for attendees to interact at any point with the invited panelists for any dialogue or follow up questions, the Summit reinforced a question upon which many of us in science education were focused: How can we, the science education community of researchers, practitioners, and consumers, lead policy? To address this question, we will briefly review the No Child Left Behind Act and its implications for teachers, and elaborate about one ongoing and growing effort to answer the concerns about the paucity of research expressed at the Summit (a special collaboration among professional organizations and societies to develop a research agenda in science education to help with policy). Excellence in science education for all is especially warranted in an age of increasingly complex science and high technology and of global interdependence and competition. International comparisons and other data suggest, however, that we are a long way from achieving the goals of excellence in scientific understanding and skills for diverse young people who will reach and contribute to the frontiers of science and technology as well as for scientific literacy for all in the society (National Research Council, 1996, 1999a, 1999b, 2001, 2002, 2003, 2004). For decades in the United States, commissions, federal agencies, and professional organizations have issued numerous reports articulating the nature of problems and needs in science education. For almost five decades the National Science Foundation and other agencies have provided a series of grants intended to develop and promote more effective science curricula, science teaching, and science teacher education. In the past 45 years we have developed an array of curriculum resources, the National Science Education Standards, high stakes testing, alternative teacher licensure, and technology-rich curricula, to mention only a few of the actions taken (Fraser, 2003; Gabel, 1994). These responses have had the potential to promote the ends we seek, but they often are applied in ways that interfere with those ends. Unfortunately, careful scholarship and student performance data have not consistently driven the policies and practices that have ensued. Thus, there is an urgent need to promote, conduct, and use more focused, scientifically-based scholarship that can inform policy and practice in science in non-formal and formal education settings.

The March 2004 Science Summit held in Washington, DC (http://www.ed.gov/rschstat/research/progs/math- science/sciencesummit04) provided the venue for a wide variety of stakeholders to learn more about science education. Officials in the US Department of Education stated that there is little research in science education to guide effective practice. As a science education community, we must respond professionally and proactively to these assertions by informing all levels of stakeholders and policy makers about high quality research studies in the literature base; just as importantly, we (the science education community and science community working through our professional societies and organizations) must work with our officials to set out an agenda that delineates areas of needed research. An extensive body of research exists in professional journals on broad, thematic issues of teaching and learning and classroom practice in traditional school disciplines—science, mathematics, social studies, and the language/communication arts. The National Research Council (NRC) published two digests of specific areas of thematic research— How People Learn: Brain, Mind, Experience, and School (Bransford et al., 1999) contains the work of sixteen scholars. The book documents the influence of cognitive science on teaching and learning in science and mathematics. The positive reception of this digest prompted the NRC to publish an expanded edition a year later—Knowing What Students Know: The Science and Design of Educational Assessment (Pellegrino et al., 2001). The NRC Committee on the Foundations of Assessment produced a digest that examines educational assessment, focused on thoroughly unpacking several pressing, complex issues related to assessment. An extensive body of scholarship also exists within science education. One of the core themes of the National Science Education Standards (National Research Council, 1996) asserts that students should experience inquiry as a core component of the content of science as well as through inquiry-based curricula and instruction. Research on inquiry-based curricula and instruction extends prior to the postSputnik era of curriculum development in school science. One example is the widely published work of Professor Karplus of the University of California at Berkeley and the Lawrence Hall of Science, who developed the Learning Cycle for the post-Sputnik K-6 Science Curriculum Improvement Study (SCIS). The Learning Cycle in its original form is a threestage, guided inquiry instructional model which

Developing a Research Agenda in Science Education was extensively investigated by Karplus and his colleagues. This line of research yielded numerous publications and a series of reports during the following two decades. Modifications of the Learning Cycle were extended from elementary classrooms into middle and high school science classrooms (through dissemination of Science Teaching and the Development of Reasoning). More recently, the Biological Sciences Curriculum Study (BSCS) adapted the Learning Cycle, renaming it the 5-E Instructional Model (Engage, Explore, Explain, Elaborate, and Evaluate) as the core of its curriculum development projects. The effectiveness of the Learning Cycle as a guided inquiry instructional framework has been well documented in the science education research literature. Clearly, the science education community must become more proactive in providing leadership at all levels of education and government. In no case is this involvement more imperative than in the area of policy.

NO CHILD LEFT BEHIND President George W. Bush signed the No Child Left Behind (NCLB) Act into law on January 8, 2002. NCLB reauthorizes the Elementary and Secondary Education Act, the principal federal law affecting K-12 public education, first passed by Congress in 1965. This sweeping overhaul of the federal law has had an affect on virtually every aspect of state, district, and local efforts to improve K-12 education, including science. NCLB was designed to close the achievement gap between high and low performing students. According to the US Department of Education, the law is built on four common sense pillars: accountability for results, an emphasis on doing what works based on scientific research, expanded parental options, and expanded local control and flexibility. The main focus of the law is to improve the academic achievement of students in low performing schools. The premise behind NCLB is that by providing information about gaps in student achievement, via the results of yearly student assessments in reading and math, the public will hold teachers, policymakers, parents and others accountable for raising the achievement of all students. The goal is that by the 2013–2014 school year every student will achieve a proficient level in reading and mathematics.

241 NCLB requires states and districts to work together in number of ways to achieve this goal. Beginning in 2005–2006, states will be required to test all students annually in grades 3–8 in mathematics and reading/language arts. States also must test students at least once annually in mathematics and reading/language arts at grade levels 10–12. Annual assessments in each state must be aligned to state standards and include the participation of all students, including most students with disabilities and limited English proficiency. Test results must include individual student scores and be reported by race, income and other categories to measure not just overall trends, but also gaps among, and progress of, various subgroups of students. Results of the math and reading assessments serve as the primary indicators of whether schools and districts have made adequate yearly progress (AYP). Schools must create and report an annual student performance report card that provides parents and the community with comparative information about performance at the school district and individual school levels, based on AYP results. Title I schools as identified as needing improvement or corrective action, or schools that do not meet AYP are subject to a set of corrective actions, which are listed below. • No AYP for two consecutive years: Parents can transfer their children to another public school or charter public school. • No AYP for three consecutive years: Parents can transfer their children or they can select a supplemental service provider for special instructional help for their children. • No AYP for four consecutive years: Parents can transfer children or get supplemental assistance. The school must begin restructuring by replacing staff and centralizing the decision-making. • No AYP for five consecutive years: The state may take the school over or turn management over to a for-profit company. For example, NCLB requires that all states must have science standards by the 2005–2006 school year. Science assessments must be developed and put into place by the 2007–2008 school year and administered not less than one time during grades 3–5, 6–9 and 10– 12. (NCLB does not require that science be included in AYP reports, however states can opt to include science in their AYP framework).

242


The challenge facing states and districts is to develop science assessments that effectively measure student understanding in science and are aligned with state standards. Ongoing work done at the national level is focused to better determine what makes an effective science assessment. Most of the discussion is focused on how to assist science educators so they can adapt their teaching to meet these new requirements and ensure that students achieve performance goals. NCLB also mandates that all teachers of core academic subjects—including science—must be highly qualified (HQ) by end of 2005–2006. A HQ teacher must be certified or licensed, hold a bachelor’s degree, and have demonstrated competencies in his or her teaching area (as determined by the state). Science educators must prove they are highly qualified by meeting their state specific HQ requirements, (which can be found on their state department of education website). In most cases, states are requiring that teachers seeking to become highly qualified must take course work, usually in a content area, and/or participate in professional development activities. To help teachers become highly qualified, Congress included a $2.9 billion Title II Teacher Quality program in NCLB. Eligibility for NCLB Title II funds means that a district must first conduct a needs assessment to determine the professional development needs of all teachers—including teachers of science. Teachers must be involved in the needs assessment process done at the district level. Prior to NCLB, the Eisenhower professional development grant program provided dedicated funding for science and math teacher professional development. This program was consolidated into the Title II grant under NCLB, and districts may now use these grants to provide teacher training and/or to hire new teachers to reduce class size. With the implementation of the NCLB legislation, educators at all levels of schooling have expressed concerns about issues ranging from credentials/certification of highly qualified teachers to classroom activities to accurate assessments of student performance. These issues have also been extensively discussed in sessions dedicated to NCLB at our professional state and national science education organizations. Following are perspectives offered by two educators—a public middle school science teacher and a university-based science teacher educator, both of whom were presidents of their respective state/national professional organizations.

No Child Left Behind—Perspective of A Middle School Educator Since the NCLB legislation was signed into law in 2002, concerns of educators in the middle school classroom vary from state to state and school district to school district (Coffman, 2004). Two main issues that impact all teachers nationwide relate to accountability: (1) certification as a “Highly Qualified Teacher” [Section 9101(23)(B)(ii) and (C)(ii)] and (2) students’ progress assessed in the school’s Adequate Yearly Progress (AYP). All teachers with primary responsibility for direct instruction in one or more of the core academic content areas must demonstrate that they satisfy the federal definition of “Highly Qualified Teacher” by the conclusion of the 2005–2006 school year. A teacher is defined as “highly qualified” if he or she has (1) a bachelor’s degree, (2) receives a passing score on a rigorous state test in the content area (Praxis II), (3) holds a graduate degree in the content area, (4) has earned at least 30 credits in the content area, or (5) meets the state’s definition of highly qualified in the content area by passing the state’s Highly Objective Uniform State Standard of Evaluation (HOUSSE). In 2002, only 58% of the science teachers in grades 7–8 held science certification. Twenty-one percent of the middle level teachers held elementary certification and 21% were not certified at all (DeBlieu, 2003). In many middle school settings, a child-centered and interdisciplinary teaching philosophy is encouraged. In these schools, the teaching format adjusts each year so that an elementary school child transitions from a self-contained elementary classroom to a 5th/6th grade program which provides the student with a Math/Science teacher and a Reading/Social Studies/Language Arts teacher. This transition continues to prepare the student for high school by providing a different teacher for every core subject by grades 7 & 8. Now the teachers of departmentalized middle grades must meet the requirements for “highly qualified teacher” according to the new law (DeBlieu, 2004). As expected, there has been a flurry of activity by the middle school teachers who are generalists with K-8 elementary certification to determine their specific degree of qualification. In the science middle school classroom, the generalist must become qualified in biology (for the life science classes), physical science (for the physical science classes), and earth sciences (for the earth science and environmental

Developing a Research Agenda in Science Education science classes). Most middle school science teachers have been trained as generalists without a concentration in one specific science area. This issue and concern is similar to the difficulty in the high school setting, where science teachers will need multiple certifications to teach more than one science course. Certain school districts, especially schools within urban areas, are dealing with the NCLB provisions by returning their school structure to a K-8 setting. In that vein, the teachers can hold K-8 elementary certification to teach a self-contained class and students can “change classes” in the upper grades for science if there is a qualified teacher. Within the urban areas, this return to smaller “neighborhood elementary schools” as opposed to the larger district-wide middle schools keeps students in their home environment longer and cuts back on the issues of gangs, drugs, and other problems that plague many of our urban and city schools. Another issue affecting the middle school science educator is accountability on state-wide assessment of the core academic content areas. School administrators are pressured to make AYP with all children in all subgroups. The interesting segment of the law is that a school can fail to make AYP if: • fewer that 95% of the students are present to take the two standardized tests (in math and reading) regardless of the scores, and • if one subgroup of students—minority students, students with limited English skills or special education students—does not meet AYP standards. School administrators pass this pressure on to classroom teachers by encouraging them to teach information that will be tested on the statewide assessment. In lower ability groupings, the teacher’s opportunity to pursue more interdisciplinary units with other core academic teachers is squelched. Fear of not covering the required material that will be assessed and reported to the state hampers the creativity of the classroom. Consequently, students miss opportunities to participate in authentic learning experiences which demonstrate real world connections rather than only learning factual concepts that may be on a test. The results in the creation of a mini-high school within the middle school environment. In the end, accountability for academic core subjects should advance science education in the middle grades. Providing students with a highly qualified science teacher will enhance the quality of science

243 within the middle school grades. It appears that we are moving back to the junior high school concept of content focus. Until appropriate certifications are in place and enough science teachers can be convinced to teach in the middle grades, a crisis will continue in middle school science programming. No Child Left Behind—Perspective of a Science Teacher Educator The impact of No Child Left Behind for science teacher education and classrooms centers around the use and definition of what is meant by “highly qualified teacher.” According to NCLB, a “highly qualified” teacher is defined as someone who holds at least a bachelor’s degree from a four-year institution; holds full state certification and demonstrates competence in their subject area either by majoring in that subject in college, showing that they know the subject they are teaching, or by passing a rigorous subject matter test or other state-mandated evaluation (Education Trust, 2003). In addition, the law requires that colleges or departments of education must to publicly report what they are doing to improve teacher quality including the identification and distribution of “highly qualified” teachers across low and high poverty schools. What the NCLB law provides is an opportunity for teacher preparation programs at institutions of higher education to reshape teacher education. The law provides an opportunity to align the standards of professional accreditation organizations, i.e., NCATE, INTASC, state professional teaching standards and university mission statements and conceptual frameworks into a coherent design that prepares highly qualified teachers and provides them with the skills and knowledge to raise the achievement of an increasingly diverse population of students. Secretary of Education Dr. Paige, in the Second Annual Report on Teacher Quality, suggested that the key to meeting the goal of “highly qualified” is found in subject matter knowledge with little emphasis on the role of pedagogical content knowledge. This view suggests that content is considerably more important than pedagogy—the “what” which is taught is much more important than “how” content is taught. Such an assertion provides an impetus to market-driven initiatives that attempt to do away with traditional teacher education requirements, and in some cases, colleges of education. This perspective encourages the idea that teaching is merely a case of knowing the subject matter content, and the

244


assumption that teacher education has a very weak knowledge base. The US Department of Education has suggested that a single, machine graded test (being developed by the American Board for Certification of Teacher Excellence) can serve as the sole proxy for teacher preparation and licensure. (Blair, 2003). The current research base supports the assertion that teachers need to know and understand both content and pedagogy (how to teach students) in order to be effective professionals. Ever increasing demands on teachers necessitates that they have the knowledge base and skill set to understand how people learn, how to manage the learning process including standards-based lessons, assessing student work, working appropriately with special-needs students and English language learners, and using appropriate technologies to prepare students for life in the 21st century. This kind of “highly qualified teacher” requires a comprehensive vision of teacher development. Over the past few years, there has been a growing body of research that indicates that the teacher is the most important school-related determinant of student achievement. (Ferguson, 1991; Sanders and Rivers, 1996; Haycock, 1998; Darling-Hammond, 2000). Closer to home, NCLB has caused confusion within teacher preparation programs because of new state mandates and requirements. Most future teachers (teacher education students) have had to deal with transition issues from old programs to new programs. Frequently students are caught in these transitions, suddenly finding that they have additional requirements to complete their credentials. Because of an increase in alternative routes for receiving a teaching credential, science departments are opting out of offering state recognized credential programs in lieu of standardized examinations. This is particularly the case with departments that do not have many teacher candidates within their pool of majors. An issue that looms larger and larger is the Teacher Preparation Examinations mandated by each state. Though there is value in the examinations, especially for NCATE-accredited institutions, the requirement to document evidence in terms of outcome assessments and the scoring and reliability/validity issues are very problematic and daunting. For example, the costs involved in scorers trained, “calibrated” and “recalibrated,” as well as, the number of scorers necessary to demonstrate legitimate reliability and validity measures is making the cost of the entire enterprise prohibitive.

A SCIENCE EDUCATION RESEARCH AGENDA We offer that one essential response by the science education community to the implementation of policies and meetings such as the Science Summit is the establishment of a national research agenda in science education. The education research community has responded to recent critiques and complex needs in science education with new calls for enhancing the validity and reliability of research in education. Expanded scientific research in education with the same high standards of expectation, design, analysis, or process as research in the natural and social sciences has the potential to become far more effective in promoting society’s goals for student learning. Teachers and policy makers must make decisions that are informed by appropriate and careful scientific research illuminating what we know and what we need to understand about learning and teaching in science. This article describes one part of a long-term plan to develop and employ a Research Agenda in Science Education (the RAISE project). The Agenda will enable policy makers, educators, and researchers to be more responsive to important needs in science education, to identify and use the best knowledge that has been accumulated, and to develop and apply relevant new knowledge and understanding in all educational settings. Over the past two years, a special collaborative working group of elected representatives from major science education and science organizations (K-12 teachers, researchers, teacher educators, scientists, and others) met at science and education professional meetings to discuss and lay the foundation for a coordinated research agenda. The basis for this effort is to build a coherent research agenda around which educational research can be structured similar to effective scientific research practices. The agenda will enable us to specify what we know, what we need to know, and how research can be employed for creating and implementing policy. The Research Agenda in Science Education (RAISE) collaboration seeks to assist science education professionals (including researchers and teachers) and support systems (including schools, universities, education agencies, and communities) to better understand and employ principles of scientific research as described by Shavelson and Towne (2002) in Scientific Research in Education and by Donovan, Wigdor, and Snow (2004) in Strategic Education Research Partnership as we pursue organized

Developing a Research Agenda in Science Education research in science education. The development of the research agenda and associated activities will inform policy and practice to support the excellence in science education that is needed and sought by society. The goals of the long-term RAISE are to: • Create and Implement a national research agenda in science education with substantive input from key professional organizations, agencies, and individuals with expertise in teaching and learning science; • Provide Mechanisms and Resources that will support applied scientific research in science education. The research should support evidence-based decision making, actions, and policies at all levels in science education. • Stimulate Research and Build Research Capacity, especially scientifically-based research in science education by individuals, groups, and associations at all levels; • Enhance and Develop the Expertise of science education researchers, teachers, and others who support that research. • Provide a Structure and Process through which key professional organizations, agencies, and individuals in science education may examine the relevant literatures related to teaching and learning to establish and subsequently continue to develop a systematic, sustainable, and focused science education research mission; and • Produce a Research Matrix that assists professional organizations, agencies, and individuals to identify, understand, and pursue research questions that are appropriate to their expertise and resources and that contribute to consensus and new knowledge about effective teaching and learning practices. For example, by convening regular meetings at our professional societies, science education specialists, K-12 educators, scientists, mathematics education specialists, engineering/technology education specialists, education researchers, and learning theorists can review the current research in science education, outline and prioritize topics for policy briefs on the research base, and make recommendations for continued or missing areas for research, policy, and practices with evidence-based research. To begin the process of assembling a research agenda, we must focus on building a strong longterm collaboration among science/science education

245 professional organizations and on delineating a research matrix. A research matrix is a structure in which research questions are identified and prioritized, testable hypotheses about learning and instruction are generated, results are incorporated into a shared knowledge base, studies are replicated in a range of school environments, and studies on innovations are scaled up.

BUILDING A COLLABORATIVE MODEL—A NETWORK OF PRACTICE Wilson (1998) applied the term ‘consilience’ to the “jumping together of knowledge” that leads to scientific advancements, progressive, creative, fluid scientific research and intellectual capacity to move a research community toward an enlightened research agenda. Our goal is to develop a coherent, wellarticulated, long-term collaboration across professional organizations and societies which enables the science education community to advance research capacity and practices by the “jumping together of knowledge.” It is only through collaboration among the research, practice, and policy communities that schools, classrooms, and teachers will be receptive to research which is useful to them (Donovan et al., 2004; Elmore, 1997; Garet et al., 2001; Hinds, 2002; Ingersoll, 2001; Lester and Ferrini-Mundy, 2004; Resnick, 1987; Wilson et al., 2001). Our unique collaboration will set an agenda that gives strong voice to research, practice, and policy communities; the research agenda will focus our efforts to synthesize and act on comprehensive research, practices, and policies. To assemble such a collaboration requires that we build a “network of practice” (Brown and Duguid, 2000) that is similar in concept to an interdisciplinary research center. This kind of network functions as a broad social system through which knowledge and information are shared in forms ranging from traditional publications to public policy initiatives to long term product development to popular media contributions (Rhoton, 2003). The chief aim of our developing collaboration (“network of practice”) is to address problems and issues in new ways, building on the talents, expertise, multiple perspectives, and connections of the members and leaders in our professional organizations. Key characteristics of individuals who work in effective interdisciplinary research centers representing various expertise areas include their: general problem solving skills to learn, un-learn, and re-learn

246


across disciplines; seeking and implementing new modes for knowledge production and use; and preference for fact-to-face and interpersonal communications across disciplines (Rhoton, 2003). Clearly, the process of effective and long-term collaboration requires us to: (1) employ conference venues to bring together various organizations and communities (including the business/industry sector); (2) develop a “network of practice” that strengthens inter- and intra-organizational connections, becoming more inclusive while developing long-term action plans; and (3) promote the development of science concepts for learners and the development of professional communities of inquiry and practices (similar to the California Science Teaching Development Project— a model of teachers, researchers, and scientists linked into classrooms). This collaborative model of professional organizations will be the means for us to organize effectively the networks of practice from the communities of practitioners, researchers, and other stakeholders. A special working group of elected representatives of the National Science Teachers Association, Association for the Education of Teachers of Science, and National Association for Research in Science Teaching (teachers, researchers, teacher educators) convened at the NSTA headquarters in September 2003. Over the course of our discussions, we expanded our partnership connections and network to include the American Association for the Advancement of Science, Sigma Xi, the Council for Scientific Society Presidents, the National Science Education Leaders Association, and other leading associations. As our preliminary partnerships were growing, we laid the foundation and sought input for a long-term plan for a research agenda (developing a prototype research matrix, gathering feedback on the matrix from professional associations). At the National Congress for Science Education (July 2004), teachers and other participants generated research questions they [practitioners] considered most pressing. The majority of practitionergenerated questions fell into the categories of (1) teacher characteristics/background/experiences, (2) impact on student achievement, and (3) impact on instruction. A call for research questions was also solicited from the 160,000+ recipients of the NSTAexpress (members and non-members). In addition to the input from the practitioner community (K-16 teachers), the research community (science education researchers) and selected science education leaders from professional organizations

contributed research questions. Responses from all groups were consolidated and categorized. Parallel questions were also sought from groups such as the National School Board Association and the National Association of Secondary School Principals.

RESEARCH MATRIX AND PASTEUR’S QUADRANT As noted earlier, the development of a useful Research Matrix (see Fig. 1) for science education can become a central element in the RAISE collaboration. The Research Matrix organizes and catalogs research questions with suggested designs, policy briefs for science education initiatives, and relevant implications for effective curriculum and teaching practice. It can guide the structuring of research topics into cells, and serve as a resource for those seeking answers to questions about teaching and learning in science. As currently envisioned, each cell contains research questions and associated ideas patterned on the Pasteur’s Quadrant (Stokes, 1997) organizer shown in Fig. 2. Stokes described Pasteur’s Quadrant as a way to better understand scientific research as a systematic and pragmatic process of inquiry. The quadrant offers a two dimensional structure for theory and practice divided into four cells. The cells force the classification of research questions and findings into pure, applied, and “use-inspired” research. Through intensive iterative cycles, the matrix cells can be revised to most accurately reflect the current research base and needed questions and areas for research. Through these processes, we will be able to substantively answer questions about `how does program/project/treatment X/Y/Z transfer to other settings and have systemic outreach and impact?’ With a rich, developed matrix and suggested needs, questions, and priorities, the community of science education researchers can employ the matrix as a guide for research. Using such a guide, researchers and graduate students seeking research ideas can build on the most promising ideas of others, expanding the richness, depth, and scale of evidencebased research. Funding agencies may also find the matrix to be useful in identifying areas that warrant special attention and support. Continual review and development of the matrix cells by the research community and other stake holders should promote critique and understanding of limitations in current research data and paradigms, and stimulate dialogue that will lead to new studies, to adjustments

Developing a Research Agenda in Science Education

247

Fig. 1. Research matrix.

in prevailing research paradigms, and occasionally to new paradigms. In time, increasing numbers of cells in the matrix will reflect scientific research in science education, and raise the diversity, quality, and

utility of research in science education. Through this process, we believe that research in science education can become more collaborative and focused, increasing the probability that research will be based

248


Fig. 2. Pasteur’s quadrant (Stockes, 1997, p. 73).

more regularly on the concepts, findings, and theories of a developing science education community in which research based theory and practice are well integrated. The power provided by a relatively elaborate matrix allows us to make teaching, curriculum, and policy decisions that are based on research rather than hunch, personal experience, or ideology alone. Such research-based decisions should promote excellence in teaching and learning of science for all. Following are two examples of cells in the research matrix drawn from topics of subject matter knowledge (physical sciences) and science teacher education. Background for Fig. 3, quadrants 1, 2, and 3: Students study concepts of motion in introductory

Fig. 3. Illustration from the research matrix cell on disciplinespecific concepts from the physical science/physics/motion at the middle school level.

Fig. 4. Illustration from the science teacher eduction research matrix cell.

physical science classes at several levels of schooling. More specifically, students are expected to understand the concepts of displacement, time, speed and their interrelationships. These concepts are fundamental to many more complex concepts that are also studied in school science involving force and motion. Background for Fig. 4, quadrant 1: While most certified science teachers complete numerous science and technical courses, these courses rarely focus on the content from the point of view of decision making in the classroom—how to teach concepts, the sequence in which concepts should be taught, or the subtleties of explaining difficult concepts. Typically it is common practice to rely on the prospective teacher to notice how decision-making occurs by watching others in schools. Unfortunately, merely participating in or observing someone teach is not an effective way to teach the complex and coordinated aspects of teaching and decision-making. More definite information is needed about both sides of the equation— how to demonstrate and model the desired teaching roles and behaviors and how to observe and participate in ways that cause student learning to take place. Background for Fig. 4, quadrant 3: In a typical classroom with few electrical outlets, seating not designed for group work, and other outmoded aspects of classrooms, teachers are expected to use the latest materials and techniques and technologies with their students. Although classroom and school building infrastructure may not be easily changed, new technological devices are continuously becoming part of the hardware. Studies about how existing classrooms and standard classroom patterns can be modified

Developing a Research Agenda in Science Education to take advantage of new technologies, materials, and ideas will be very necessary to document and prescribe innovative practices in science education. Especially important input to the research dialogue and subsequent reviews will be sought from teachers and policy makers. This input will provide regular feedback on research questions, priorities of topics, classroom and policy applications, and strategies that promote research as part of science teaching. Building on this prototype research matrix, topics and research questions of high priority (use-inspired research, pure basic research, applied research) can be identified and categorized. For example, when selected cells within the matrix are completed, they will be assessed and reviewed in several iterations. This cycle of construction, development, critique, and further elaboration of cells will enable us to establish research priorities and, with time, catalog existing research within each cell, while simultaneously ranking research as to validity, reliability, and need for replication or additional study. A subsequent step and outgrowth of the matrix is a catalog containing proposed research methodologies and designs appropriate for groups of questions. Such designs might include examining the rich contexts of classrooms, creating profiles that document the knowledge and wisdom of teachers, moving to a subsequent stage of correlational studies, on to studies that suggest causality, and then to experimental or laboratory studies with carefully selected comparison or control groups. Hopefully, the discussions will give rise to a recognition of needs for new methodologies that should be developed and to teaching practices and policy that regularly are shaped by relevant research. The matrix structure will permit the articulation of short and long term programs for research by the generators and consumers of research. These programs can lead to progressions of research studies that yield meaningful findings for practice, and establish the research infrastructure that will support and promote consensus on valid and rigorous findings (i.e., 5-year action plans, periodic syntheses and critiques of the body of research studies). As the agenda develops and support is invested in the research infrastructure, the academic community interacts and collaborates with the practitioner and policy communities—this linkage is vital if the findings of high quality research are to be successfully used by all stakeholders.

249 HOW CAN WE, THE SCIENCE EDUCATION COMMUNITY OF RESEARCHERS, PRACTITIONERS, AND CONSUMERS, LEAD POLICY? In the past 50 years, we have developed many science education resources, and we have accumulated information and ideas about how people learn and about the nature of effective science teaching. The excellent materials and strategies we have developed and what we have learned about science teaching and learning must be applied more consistently in practice and policy. We also need to identify and pursue careful, scientific research on what we still need to know about learning and teaching in science, purposefully disseminating these materials and outcomes to the public. These outcomes and products can be institutionalized through a variety of channels by incorporating research into standing committee charges and organizational priorities, establishing research circles or collaboratories (regional/local levels and at the national level, such as an AETS Collaboratory on Research in Science Teacher Education) to focus on areas of research (i.e. how to do action research) and inform others, using products to inform/revise NSTA/NCATE work and NSTA (and other) position statements, and helping the NSF/DOE leadership set recommendations for funding priorities. Most important is our communication with elected officials and key education and scientific governmental agencies. Each person needs to not only serve as an advocate for scientifically-based scholarship, but be proactive in policy initiatives at all levels. One arena in which we do not take as strong an advocacy role that we should is policy initiation and analysis. Our professional associations should be viewed as repositories and `databases’ for expertise in science education policy, especially policy analysis. To engage in policy analysis and comprehensive planning, we must become familiar with and use the principles by which analysts draft and enact policies. A policy is defined as a settled course of action to be followed by a government body or institution (used synonymously with plan or program), and policy analysis is a final product (in the form of a memorandum, issue paper, or legislation) resulting from a process that usually begins with a problem definition, includes a discussion of alternatives, focuses on a specific client, promotes a specific perspective, and has an openly political approach (Patton and Sawicki, 1993). For example, to draft policy requires

250


that participants engage in more narrative styles of reporting. The problem to be solved must be defined, followed by the assembly of evidence and the construction of alternatives and selection of criteria by which to assess the policy. Possible courses of action (supported by evidence) with trade-offs ultimately lead to stated recommendations for decisions. Among two important attributes for policy analysis is that we in the science education community must learn the language of policy planning, policy analysis, and the legislative language. Science education research can only make a strong impact on practice if it [education research as an enterprise] is supported through ongoing collaborations among researchers, practitioners and policy makers. The unique collaboration described here is a first attempt to bring together all stakeholders through professional organizations. Through this model for collaboration, representatives from professional societies and organizations will be able to: (a) formulate, evaluate, reassess, and produce an evolving and useful Research Matrix for science education (elaborating the dimension or elements of the matrix); (b) incorporate feedback and input from multiple constituencies (including teachers, policy makers, researchers, and other consumers); (c) outline long term action plans for research in science education; and (d) provide templates for successful collaborations among professional organizations and members, science teachers, researchers, and policy makers. The science education community, working in partnership with the science and policy communities, can assert a leadership role in generating templates and elaborating research studies for selected domains by systematically identifying and classifying: • extant relevant research; • important findings that emanate from the research base; • gaps in the research; • research questions, potential studies, and suggested designs and related issues of highest contemporary priority; and • general policy implications based on available knowledge/research findings.

Research in education (conducted following tenets similar to those employed in research in science) has generated significant findings that can guide and inform teaching at all levels. Science as an enterprise builds upon successive iterations of research and a firm belief that through a systematic process of observation and inquiry we can come to know, understand, and potentially explain or even predict a wide variety of natural phenomena. Characterized by professional scrutiny and criticism, science creates a community of scholars who continually report and review their own work as well as that of their peers. The systematic nature of scientific inquiry requires both individual investigators and groups to collaborate by sharing information, processes, techniques, outcomes, and predictions. Agreements and understandings in science result from logic, skepticism, persuasion, argument, based on evidence. The evaluation of ideas, causes and consequences derive their justification from the critical analysis of evidence and scientific discourse. Scientific practices have been very important in developing understandings of the natural world, and they have informed designs and practices that have been transformative in improving living conditions, health, and the quality of life, and informing problem-solving practices and policy. It is very important to apply scientific practices in the social sciences to the study of science education to enable more successful teaching and learning in science and related fields (Stokes, 1997). As Shavelson and Towne wrote in Scientific Research in Education (2002), “At its core, scientific inquiry is the same in all field . . . research . . . is a continual process of rigorous reasonings supported by a dynamic interplay among methods, theories, and findings. It builds understanding in the form of models or theories that can be tested” (p. 2). The process of creating and implementing a national research agenda, the first-ever collaboration of this kind among professional societies, is a unique model of knowledge generation and application. This novel process of collaboration will enable us to construct a matrix of needed research and to promote the development of that research. The collaboration will assist the science education community by focusing our work on national needs and priorities and by ensuring that teachers, policy makers, scientists, and researchers in education at local through national levels have an important stake in research priorities and actions. To these ends, the unique

Developing a Research Agenda in Science Education collaboration among the science education and scientific professional organizations seeking to develop a national research agenda and research matrix can assist our field (including science education support systems, schools and education agencies, teachers, and researchers) to better understand and employ principles of scientific research applied to education. ACKNOWLEDGMENTS The authors would like to thank Cherry Brewton, Carolyn Randolph, Anne Tweed, and Gerry Wheeler and our respective professional organizations for their interest and input. REFERENCES Blair, J. (2003, September 3). Essays on new teachers’ test to be graded by computers. Education Week 11. Bransford, J. D., Brown, A. L., and Cocking, R. R. (Eds.). (1999). Howpeople Learn: Brain, Mind, Experience, and School, National Academy Press, Washington, DC. Brown, J., and Duguid, P. (2000). The Social Life of Information, Harvard Business School Press, Boston. Coffman, W. (2004). The highly qualified teachers challenge [Electronic version]. RBS Currents 8(1): 1–3. Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Educational Policy Analysis Archives 8(1): 1–46. Darling-Hammond, L. (2000). Solving the Dilemmas of Teacher Supply, Demand, and Quality, National Commission on Teaching and America’s Future, New York. DeBlieu, M. O. (2003). How New Jersey teachers can show they meet ‘highly qualified’ rule. NJEA Review. DeBlieu, M. O. (2004). No child left behind: Rhetoric v. reality. NJEA Reporter 47(3): 2. Design-Based Research Collective. (2003). Design-based research: an emerging paradigm. Educational Researcher 32(1): 5–8. Donovan, M. S., Wigdor, A. K., and Snow, C. E. (2004). Strategic Education Research Partnership, National Academies Press, Washington, DC. Education Trust. (2003). Need of Improvement: Ten Ways the US Department of Education has Failed to Live up to Its Teacher Quality Commitments, Author, Washington, DC. Elmore, R. (1997). Investing in Teacher Learning: Staff Development and Instructional Improvement in Community School District #2, New York City, National Commission on Teaching and America’s Future, New York. Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money matters. Harvard Journal on Legislation 28: 465–498. Fraser, B. (2003). International Handbook of Science Education, Kluwer Academic Publications, Dordrecht. Gabel, D. (Ed.). (1994). Handbook of Research on Science Teaching and Learning, A Project of the National Science Teachers Association, Macmillan, New York. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., and Yoon, K. S. (2001). What makes professional development effective:

251 Results from a national sample of teachers. American Educational Research Journal 38(4): 915–945. Goodlad, J. (1990). Teachers for Our Nation’s Schools, San Francisco, Jossey-Bass, California. Hamilton, L., McCaffrey, D., Stecher, B., Klein, S., Robyn, A., and Bugliari, D. (2003). Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis 25: 1–29. Haycock, K. (1998). Good teaching matters . . . A lot. Thinking K16 3(2): 3–14. Hinds, M. (2002). Teaching as a Clinical Profession: A New Challenge for Education, Carnegie Corporation of New York, New York. Ingersoll, R. (2001). Teacher turnover and teacher shortages. American Educational Research Journal 38(3): 499–534. Kelly, A. E., and Lesh, R. A. (Eds.). (2000). Handbook of Research Design in Mathematics and Science Education, Mahwah, Lawrence Erlbaum Associates, NJ. Lester, F. K., Jr., and Ferrini-Mundy, J. (Eds.). (2004). Proceedings of the NCTM Research Catalyst Conference, Reston, National Council of Teachers of Mathematics, VA. National commission on teaching and America’s Future. (2002). Unraveling the “Teacher Shortage” Problem: Teacher Retention is the Key, Author, Washington, DC. National Research Council. (1996). National Science Education Standard, National Academy Press, Washington, DC. National Research Council. (1999a). How People Learn: Brain, Mind, Experience, and School, National Academy Press, Washington, DC. National Research Council. (1999b). Improving Student Learning: A Strategic Plan for Education Research and Its Utilization, National Academy Press, Washington, DC. National Research Council. (2001). Educating Teachers of Science, Mathematics, and Technology: New Practices for the New Millennium, National Academy Press, Washington, DC. National Research Council. (2001). Knowing What Students Know: The Science and Design of Educational Assessment, National Academy Press, Washington, DC. National Research Council. (2002). Investigating the Influence of Standards. A Framework for Research in Mathematics, Science, and Technology Education, National Academy Press, Washington, DC. National Research Council. (2003). Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, National Academy Press, Washington, DC. National Research Council. (2004). Engaging Schools: Fostering High School Students’ Motivation to Learn, National Academy Press, Washington, DC. National Science Teachers Association. (2003). NSTA/NCATE standards—Revised 2003, National Science Teachers Association. Patton, C., and Sawicki, D. (1993). Basic Methods of Policy Analysis and Planning (2nd ed.), Englewood Cliffs, Prentice Hall, NJ. Pellegrino, J., Chudowsky, N., and Glaser, R. (Eds.). (2001). Knowing What Students Know: The Science and Design of Education Assessment, National Research Council, National Academy Press, Washington, DC. Resnick, L. (1987). Education and Learning to Think, National Academy Press, Washington, DC. Rhoton, D. (2003). A Multi-Method Analysis of the Social and Technical Conditions for Interdisciplinary Collaboration, Final report for the National Science Foundation BCS 0129573. Sanders, W., and Rivers, J. (1996). Cumulative and Residual Effects of Teachers on Future Student Academic Achievement, Knoxville, University of Tennessee, TN.

252


Shavelson, R., and Towne, L. (2002). Scientific Research in Education, National Academy Press, Washington, DC. Stokes, D. E. (1997). Pasteur’s Quadrant: Basic Science and Technological Innovation, Brookings Institution Press, Washington, DC. U. S. Department of Education. (2003). Meeting the Highly Qualified Teachers Challenge. The Secretary’s Second Annual

Report on Teacher Quality, US Department of Education, Washington, DC. Wilson, E. O. (1998). Consilience: The Unity of Knowledge, Knopf, New York. Wilson, S. M., Floden, R. E., and Ferrini-Mundy, J. (2001). Teacher Preparation Research: Current Knowledge, Gaps, and Recommendations, Center for the Study of Teaching and Policy, Seattle, WA.