William H. Schmidt. Michigan State University ... Cogan, Barrier, Gonzalo, Moser, Shimizu, Sawada, Valverde, McKnight, Prawat, Wiley,. Raizen, Britton, Wolfe ...
Culturally Specific Patterns in the Conceptualization of the School Science Curriculum: Insights from TIMSS Leland S. Cogan HsingChi A. Wang William H. Schmidt Michigan State University
ABSTRACT This article draws upon curriculum and teacher data collected in the Third International Mathematics and Science Study to document differences in national (cultural) definitions of the science eighth grade students study in school. Differences are explored with each curriculum instantiation: the intended curriculum as evidenced in official content standards, the potentially implemented curriculum represented by textbooks, and the implemented curriculum as measured by teachers’ reports of the amount of time they taught specific topics. In addition, nations demonstrated differences in how curriculum policy, as indicated through official standards and reflected, however imperfectly, in textbooks, were related to what students were taught in their classrooms. Not only were unique portrait of science demonstrated by each curriculum instantiation within each country but the relationships among the three instantiations also demonstrated unique cultural profiles. Ideally, nations may strive for alignment across the three curricular instantiations. Nonetheless, variations in what constituted eighth grade science in any one country remained. The many differences demonstrated make it clear that there is more than one way to do eighth grade science. The authors conclude that curriculum development and reform efforts may profit from thoughtful consideration of the many different national or cultural approaches to school science.
Introduction The Third International Mathematics and Science Study (TIMSS) represents the most comprehensive study of education yet undertaken. In addition to the traditionally Page 1
expected student assessments, TIMSS also surveyed teachers and school administrators and conducted an in-depth analysis of mathematics and science curricula (National Center for Education Statistics (NCES), 1996; Schmidt, McKnight, Valverde, Houang, & Wiley, 1997; Schmidt, Raizen, Britton, Bianchi, & Wolfe, 1997). Over 40 countries participated in these various aspects of TIMSS that focused on eighth grade students.1 Results from the student assessments were disappointing for some countries and have prompted many policy makers to consider more carefully the curriculum portraits TIMSS produced – especially those for the highest achieving countries – in an effort to discern just what it might mean to have a “world class” science curriculum. Such an endeavor reflects one of the goals motivating any country’s participation in an international comparative study like TIMSS and illustrates the value a multifaceted comparative education study can provide educators and policy makers in various participating countries as they have the opportunity to examine common practices and evaluate them from a global perspective. The goal of such studies is not to hold an international Olympics in which only one country may claim the prize but, rather, to develop a better understanding of education by which all participating countries may win. The variation in key aspects of the education system – such as are reflected in their curricula – suggests that countries vary in their definitions of schooling and we submit that culture plays a significant role in these observed variations. By this hypothesis we simply mean that schooling itself is part of a country’s culture. How schooling is organized – the goals and purposes identified for each year, how decision making authority is distributed (or not) among the education system’s constituent centers,
Page 2
how the curriculum is specified and organized, what students are expected to learn and be able to do – are all reflections of that culture. More specifically, what topics constitute school science – how topics are grouped together for study at specific grade levels, the sequence of topics studied from grade to grade, and how much emphasis each receives in any one year – all appear to reflect culturally or nationally specific patterns or choices. Thus, a country’s curriculum is itself a cultural artifact. We conceive of the curriculum as having three instantiations: the intended curriculum as found in official content standards, the potentially implemented curriculum represented by textbooks, and the implemented curriculum as measured by teachers’ reports of the amount of time they taught specific topics ( Schmidt, Jorde, Cogan, Barrier, Gonzalo, Moser, Shimizu, Sawada, Valverde, McKnight, Prawat, Wiley, Raizen, Britton, Wolfe, 1996). Within a given educational system, all the stakeholders have various degrees of influence in shaping these instantiations. Thus, in order to understand a given educational system, we must study the differences in these instantiations. Considered together, these three instantiations provide a triangularized portrait of the result of any one country’s curricular decision making. TIMSS provides data for all who care about education an opportunity to study what role each instantiation plays in defining science education for each country or for groups of countries who may share a common culture of science education. Some might argue that if our hypothesis is correct, it is not useful or meaningful to conduct comparative research since each country has its own culturally embedded definitions for schooling and, therefore, we have nothing to learn from such cross-country comparisons. Each country, so the argument might go, does its “own unique thing”, that Page 3
it is entirely unproductive to compare the fruits of cultures so unlike or alien to one’s own. However, therein lies the valuable opportunity possible through an international comparative study. As the world “shrinks” through technology, most individuals will inevitably encounter products and ideas rooted in an alien culture and experience to some degree feelings of disorientation or confusion Bock(1970) has referred to as “culture shock.” Inherent in this shock is an “attempt to understand an alien way of life” (Bock, 1970, p. ix-x) which may liberate us from provincial habits of thought and can yield a new perspective on our own common practices.
Educational Cultures
Education is one of the fundamental infrastructures that supports and shapes society. As the goals and purposes for education differ from one country to another the nature and content of an education system’s curriculum would also reasonably be expected to vary. Indeed, all the aspects of culture mentioned above – yearly goals for schooling, school organization, decision making authority, etc. – all have the potential to noticeably and meaningfully affect the substance and character of subject matter curricula studied in schools. Given the array of influential factors potentially shaping curricula, the notion of country-specific expressions for commonly taught subjects seems rather reasonable. In other words, given the myriad points of potential influence on curricula that may vary from one country to another, common measurements of subject specific curricula may reasonably be expected to reflect country-specific patterns or country-bysubject interactions.
Page 4
The history of education has demonstrated how cultural, societal and vocational needs have played crucial roles in shaping a nation’s educational goals and policies. Longstreet and Shane (1993) contended that the goals of education provide conceptual guidelines for curriculum design. Consequently, educational curricula must continue to evolve as the goals of education change. However, the dynamics of these relationships between the culture, educational policy, school curricula, and what occurs in classrooms have little empirical evidence either to confirm the influence of culture on education and curriculum or to elucidate an understanding of such relationships. This article presents a first attempt to understand how the cultural context of curriculum policy relates to what students are taught in the classroom. More specifically, we examine how curriculum-asintended-learning-outcomes, as found in the form of content standards, is related to content coverage both in terms of what is included and emphasized in textbooks and in terms of what is taught and emphasized in classroom instruction. The main hypotheses explored here is that what constitutes school science varies from one culture (nation) to another and that these differences pervade all instantiations of the curriculum. In addition, the instantiations within any one culture (nation) provide unique profiles of school science. What data were gathered and analyzed? The TIMSS Science Framework provided the topic specifications used in coding the data considered here which reflect the three different curriculum instantiations: content standards, textbooks, and classroom instruction. 2 Representatives from each country coded their country’s curricular documents, i.e., content standards and textbooks, after participating in training sessions designed and conducted by the framework’s
Page 5
authors. In addition, the topics teachers responded to on the TIMSS Teacher Questionnaire represented the entire scope of the framework. Thirty-six of the forty-one countries that participated in the TIMSS grade eight student assessments had data for the three sources of data discussed in this article. The TIMSS Science Framework is organized according to three major areas of science – earth science, life science, physical science – together with several interdisciplinary areas such as science and mathematics, science and technology, science, technology, and society, history and science, and environmental issues. Particularly within the three major science areas, subtopics are arranged in a somewhat conceptual hierarchy that allows content to be specified at several different levels.3 In this article, science content is considered according to three different organizations of the TIMSS science framework topics. The TIMSS science framework contains 79 topics at the most finely defined level at which content standards and textbooks are evaluated. The TIMSS Teacher Questionnaires presented teachers with a smaller number of categories – 22 for population 2 science teachers – to be considered that were mutually exclusive and exhaustive of all 79 TIMSS science framework topics. A third organization of science content considered is represented by the 17 mutually exclusive assessment topic areas addressed by the TIMSS population 2 student assessments. A few of these areas encompass a single topic from the total list of 79 but most encompass between two and four topics. One assessment area addressed issues of procedures and processes in science that were embedded in specific science topics but did not require specific science content knowledge to complete. In all, 48 of the possible 79 topics were addressed by the student population two TIMSS science assessment.
Page 6
For each of these three sources the data can be considered as a matrix with columns representing specific curriculum topics and rows for each TIMSS country. Depending on the specific data source, matrix elements consist of either proportions or dichotomous indicators, e.g., 0 or 1. Curriculum Content Standards were characterized either as addressing a particular framework topic (1) or not addressing a particular topic (0). For those tested topics that included more than a single framework topic, the values for the appropriate framework topics were summed to yield the number of topics within the assessment area that were addressed by the standard. For textbooks, each element represents the proportion of the textbook’s blocks (small portions identified for coding according to well defined guidelines) that addressed a particular topic. Information from teachers were summarized in two ways. In one summary, each data element represents the percent of teachers within a country that addressed a specific topic in their classroom teaching.4 In the second, each data element represents the national average percent of teaching time devoted to a specific topic. Our consideration of the data follow this matrix organization and, accordingly, are examined three ways. The first is to examine differences among the row summaries (e.g., row marginals) to discover differences among countries. Second, we look among the column summaries (e.g., column marginals) to discover variation among different topics. Finally, individual elements or cells may be examined to discover interaction effects – interesting anomalies representing unique or otherwise atypical treatment of a specific topic by a particular country. Differences in row summaries document variation from one country to another and differences in column summaries document variation in how individual topics are treated. The unique flavor of science in the school within a country
Page 7
is documented by the extent that individual elements Table atypical or unique values. These atypical values provide insight into the many different meanings that studying science in school may have across the considered nations and their cultures.
What does grade eight science look like? Content Standards Table 1 lists the 17 TIMSS Population 2 science assessment areas, the 48 framework topics included in these assessment areas, and Tables the percentage of countries that intend instruction in each assessment area as well as for each of the science framework topics addressed by the assessment. The overall percentages for assessment areas indicate both the percentage that cover all assessment area subtopics and the percentage that cover at least one of the subtopics represented by that assessment area.
Page 8
Assessment Areas
Represented in Standards (N, %)
TIMSS Framework Topic
1 Earth Features
All subtopics
10
27.8
###
• Landforms
20
###
• Bodies of water
22
###
• Atmosphere
###
• Rocks, soil
2 Earth Processes
Tested Content Area
At least 1 subtopic
27
Represented in Standards (N, %) All subtopics
TIMSS Framework Topic
9 Properties & Classification of Matter
75.0
24
66.7
55.6
###
• Classification of matter
31
86.1
61.1
###
• Physical properties
30
83.3
15
41.7
###
• Chemical properties
30
21
58.3
10 Structure of Matter
14
38.9
28
77.8
###
30
• Atoms, ions, molecules
###
• Weather & climate
26
72.2
11 Energy & Physical Processes
• Physical cycles
20
55.6
###
• Energy types, sources, conversions
33
91.7
###
• Building & breaking
16
44.4
###
• Heat & temperature
29
80.6
###
• Earth's history
20
55.6
###
• Wave phenomena
20
55.6
###
• Sound & vibration
22
61.1
3 Earth in the Universe
16
44.4
20
55.6
41.7
###
• Earth in the solar system
20
55.6
###
• Light
30
83.3
###
• Beyond the solar system
16
44.4
###
• Electricity
32
88.9
###
• Magnetism
26
4 Diversity & Structure of Living Things
21
58.3
33
91.7
###
• Animals
27
75.0
12 Physical changes
###
• Other organisms
25
69.4
###
• Physical changes
24
###
• Organs, tissues
30
83.3
###
• Explanations of physical changes
19
###
• Cells
25
69.4
13 Chemical changes
5 Life Processes & Functions
29
80.6
34
94.4
###
• Chemical changes • Energy & chemical change
###
• Energy handling
33
91.7
###
###
• Sensing & responding
30
83.3
14 Forces & Motion
6 Life Cycles & Genetics
14
38.9
33
91.7
16
22 10
###
• Types of forces
24
66.7
• Life cycles
26
72.2
###
• Time, space, & motion
24
66.7
• Reproduction
29
80.6
###
• Dynamics of motion
20
55.6
###
• Variation & inheritance
19
52.8
###
• Fluid behavior
11
###
• Evolution, speculation, diversity
27
75.0
15 Science, Technology, & Society
61.1
32
88.9
###
Influence of sci, tech. on society
###
• Interdependence of life
29
80.6
16 Environmental & Resource Issues
###
• Animal behavior
25
69.4
161
8 Human Biology & Health
32
88.9
##
• Human Biology
32
88.9
###
• Nutrition
21
58.3
###
• Disease
24
66.7
32
88.9
• Pollution Conservation of land, water, & sea 162 • resources Conservation of material & energy 163 • resources 17 Scientific Processes
27
29
80.6
28
77.8
27
75.0
33
91.7
30.6 75.0
27
75.0
61.1
###
22
27
80.6
###
7 Interactions of Living Things
97.2
52.8
27.8
27
35
66.7
61.1 29
83.3
72.2 44.4
22
30
83.3
###
15
97.2
83.3 83.3
30
At least 1 subtopic
35
75.0 75.0
29
80.6
32
88.9
29
80.6
• no content
Table 1. Percent of 36 countries intending instruction in topics covered by the TIMSS population 2 science assessment.
Page 9
The two most covered topics as indicated by countries’ content standards are ‘energy handling’ (including photosynthesis and respiration) and ‘energy types, sources, and conversions’, the later a physics topic. Each of these topics was in the content standards of about 92 percent of the participating countries (all but three countries). Three other topics were in the content standards of all but four countries (89 percent). These were ‘human biology’, ‘electricity’, and ‘conservation of land, air and sea resources.’ Other topics that were included in the content standards of at least 80 percent of the countries included the biology topics of ‘organs, tissues’, ‘sensing and responding’, and ‘reproduction.’ There were three environmental science topics – ‘interdependence of life’, ‘pollution’, and ‘conservation of material and energy resources.’ There were seven topics at this level from physics and chemistry: ‘classification of matter’, ‘physical’ and ‘chemical properties of matter’, ‘atoms, ions and molecules’, ‘heat and temperature’, ‘light’, and ‘chemical changes.’ The physical science area (e.g., physics and chemistry) had the most topics represented cross-nationally at eighth grade but environmental science and biology were also well represented. The only major area not represented was earth science. Thus, in the “world according to TIMSS”, the core curriculum – as determined by 80 percent or more of countries – at least as intended and tested, was about aspects of physics, chemistry, environmental science, and some of the more advanced topics of biology such as ‘sensing and responding’ and ‘energy handling.’ Not too surprisingly, all the tested topics intended by the smallest numbers of countries all came from earth science. These included ‘atmosphere’, ‘building and breaking’, and ‘beyond the solar system.’
Page 10
A number of topics were widely intended according to countries’ standards but not included on the TIMSS Population 2 science assessment. The most widely covered of these was ‘world pollution’ which was in 81 percent of countries’ content standards. Four other topics not tested but included in over 75 percent of the countries’ content standards were ‘plants and fungi’, ‘biomes and ecosystems’, ‘food production and storage’, and ‘the nature of scientific knowledge.’ Ten of the not tested topics were in the content standards of 50 percent or less of the countries. Also shown in Table 1 is the percentages of countries that cover at least one of the topics subsumed by a given assessment area. At least 75 percent of the countries covered at least one topic in the assessment areas represented by the test items with the exception of ‘earth in the universe.’ Of the 16 science content test areas, only nine had 50 percent or more of the countries intending coverage of all topics within a tested area. Table 2 reveals the number of topics included in the content standards for each country according to the three organizations of the TIMSS science framework: the number relative to the total number of framework topics (79 topics), the number relative to the total number of TIMSS Teacher Questionnaire categories (22), and to the number of framework topics included in the assessment areas (48). The number of framework topics in content standards and, thus, presumably intended for coverage varied from as few as eight to as many as all 79 (in New Zealand, Iran, and the US). Focusing only on those framework topics relevant to one of the 17 assessment areas for the TIMSS science test (48 topics), a considerable range of topics that were intended for coverage was still evident. This ranged from as few as six (Korea) to all 48
Page 11
Country
Australia Austria Belgium (Fl) Belgium (Fr) Bulgaria Canada Colombia Cyprus Czech Republic Denmark France Germany Greece Hong Kong Hungary Iceland Iran Ireland Israel Japan Korea Latvia Netherlands New Zealand Norway Portugal Romania Russian Federation Scotland Singapore Slovak Republic Slovenia South Africa Spain Sweden Switzerland USA
Standards (79)
Textbook (79)
Teacher (22)
66 58 60 59 43 58 58 41 33 50 50 35 25 22 53 56 79 61 37 19 8 70 48 79 69 66 29 62 57 38 48 69 39 66 47 69 79
65 60 60 59 57 74 65 53 49 9 37 32 49 37 62 46 19 58 32 17 38 35 67 52 61 64 53 42 44 27 49 62 49 67 49 78 78
22 22 22 22 22 22 22 22 22 22 21 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 16 22 12 22 22 22 22 22 22 22
Tested Topics (48) Standards Textbook
43 36 36 38 28 38 43 31 22 29 30 21 22 17 36 42 48 42 25 18 6 44 32 48 41 40 19 37 42 27 29 44 26 43 34 44 48
44 39 36 38 37 46 46 39 30 8 25 22 37 26 43 30 16 40 21 15 29 24 43 37 40 39 33 28 36 20 30 41 35 41 32 48 48
Table 2. Number of science topics included in the eighth grade curriculum of each country.
(Iran, Switzerland, and the US). This, of course, does not mean that Korea did not intend coverage for closer to the total of 48 possible topics but rather that it did not intend them to be covered during the eighth grade year according to its content standards.
Page 12
Textbooks Table 2 also indicates the number of topics covered by each country’s textbooks. Over all 79 possible topics this ranges from around 10 (Denmark) to 78 (Switzerland and the US). Looking only at the 48 tested framework topics, there was a range from around 10 to all 48. For most countries, their textbooks covered from around 30 to around 40 of the 48 topics. Six countries covered less than 50 percent of the 48 tested topics (Denmark, Japan, Iran, Singapore, Israel, and Germany). Table 3 shows the range, mean, median, and standard deviations for each of the 79 TIMSS framework topics in terms of the percent of the textbook that addressed that topic. The size of the standard deviation provides one indication of the size of the differences across countries in the emphasis textbooks displayed in addressing a topic. Large standard deviations likely presage country by topic interactions. Across all 79 topics, the standard deviations range from less than 1 to over 17 indicating that for some topics very little variation existed across countries in terms of textbook emphasis but, for other topics, variation was quite large. The single topic most covered on average over all of the TIMSS countries in student textbooks was ‘electricity.’ This averaged over 11 percent of the textbook across countries. This is a strong contrast to eighth grade mathematics in which the most emphasized single topic covered almost twice as much of a textbook on average (Cogan & Schmidt, in press). This topic, ‘electricity’, also had the largest standard deviation and a range from 0 percent (that is, not in a country’s textbook) to 98 percent (essentially a science textbook in which electricity was a part of virtually all of the content).
Page 13
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
Composition Landforms Bodies of water Atmosphere Rocks, soil Ice forms Weather & climate Physical cycles Building & breaking Earth's history Earth in the solar system Planets in the solar system Beyond the solar system Evolution of the universe Plants, fungi Animals Other organisms Organs, tissues Cells Energy handling Sensing & responding Biochemical processes in cells Life cycles Reproduction variation & inheritance Evolution, speculation, diversity Biochemistry of genetics Biomes & ecosystems Habitats & niches Interdependence of life Animal behavior Human Biology Nutrition Disease Classification of matter Physical properties Chemical properties Atoms, ions, molecules Macromolecules, crystals Subatomic particles
Minimum
Mean
Maximum
SD
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1.0 1.5 1.1 0.8 2.6 0.3 3.8 0.7 2.0 1.6 1.5 0.5 0.4 0.1 2.7 4.0 0.9 6.2 1.3 3.3 2.1 0.6 2.7 3.2 0.8 1.8 0.3 1.6 1.0 2.1 1.3 9.1 1.4 2.3 4.3 4.4 4.9 3.5 0.8 0.7
9.0 10.5 6.3 7.7 13.9 3.1 29.9 4.3 14.1 6.7 14.2 4.1 2.7 0.8 13.1 32.9 10.8 17.7 6.1 14.0 8.6 6.3 24.4 26.5 8.1 18.3 3.9 19.0 5.1 20.7 9.3 40.0 20.0 15.6 16.9 15.5 23.5 18.4 6.5 4.9
1.9 2.6 1.6 1.5 3.9 0.6 7.5 1.2 3.1 2.0 2.6 1.0 0.7 0.2 4.0 6.7 2.0 5.2 1.6 3.4 2.2 1.2 4.9 5.1 1.8 3.7 0.9 3.9 1.5 3.8 2.0 10.0 3.4 3.3 4.2 4.3 5.7 3.8 1.3 1.1
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
Energy types, sources, conversions Heat & temperature Wave phenomena Sound & vibration Light Electricity Magnetism Physical changes Explanations of physical changes Kinetic theory Quantum theory & fundamental particles Chemical changes Explanations of chemical changes Rate of change & equilibria Energy & chemical change Organic & biochemical changes Nuclear chemistry Electrochemistry Types of forces Time, space, & motion Dynamics of motion Relativity theory Fluid behavior Nature or Conceptions of Technology Influence of math, tech in sci Applications of sci in math, tech Influence of sciemce, tech on society Influence of society on sci, tech History of Sicence Pollution Conservation of Land, Water, & Sea Resources Conservation of Material & Energy Resources World Pollution Food Production, Storage Effects of natural Disasters Nature of Scientific Knowledge The Scientific Enterprise Science & Mathematics Science & Other Disciplines
Minimum
Mean
Maximum
SD
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
4.9 3.3 0.6 1.0 4.2 11.1 1.4 2.4 1.1 0.3 0.2 4.5 1.3 0.6 0.5 1.3 0.3 0.7 2.0 1.7 1.3 0.2 0.6 1.4 0.2 3.0 1.4 0.3 1.5 1.1 1.2 0.9 0.8 1.2 0.4 1.2 0.4 0.6 0.6
38.4 15.8 8.7 14.7 24.3 97.9 6.6 11.6 5.8 2.4 5.6 18.4 7.0 3.7 2.9 12.4 2.3 3.6 10.2 9.3 8.1 2.6 4.4 15.9 1.5 16.1 13.0 2.5 10.1 4.5 5.4 3.5 7.9 7.9 1.8 7.3 5.2 5.5 4.3
7.0 3.8 1.6 2.5 5.4 17.1 1.9 2.9 1.3 0.6 0.9 3.6 1.6 1.0 0.7 2.7 0.5 0.9 2.3 2.7 2.0 0.5 1.2 3.4 0.4 3.6 2.6 0.6 2.2 1.1 1.3 1.0 1.5 1.6 0.6 2.0 1.1 1.1 1.0
Table 3. Distribution of percent of science textbooks’ devoted to specific topics for 37 countries.
Page 14
The second and third most emphasized topics came from biology – ‘human biology’ (nine percent) and ‘organs and tissues’ (six percent). After the two biology topics, a group of physics and chemistry topics dominated other topics with each capturing around five percent of the textbook space averaged across countries. These included ‘the classification and physical properties of matter’, ‘energy types, sources and conversions’, ‘light’, and the ‘chemical properties and changes of matter.’ The five topics receiving the most emphasis on average in textbooks include the three disciplines of biology, chemistry, and physics. The only discipline in the TIMSS framework not represented among the top five was earth science. The same general conclusion applies to the tested topics since all of the top five topics emphasized in textbooks were among those included in the TIMSS Population 2 science assessment. However, several of the tested topics did not have, on average, much textbook space associated with them. All three of the earth science test areas fall into this category since the topics defining them had only about two percent or less of textbook space. ‘Weather and climate’ was the only real exception. It had, on average, about four percent of textbook space and was one of the four topics comprising ‘earth processes’, a more global earth science topic area. In addition to most earth science topics, there were other topics from assessment areas that had only around two percent or less coverage in the textbooks. These included ‘interactions of living things’, ‘physical changes’, and ‘forces and motion.’ Table 4 similarly shows key parts of the distribution for each country for the conditional percent of textbook emphasis typically given a topic (the conditional percent
Page 15
Country
36 40 100 124 170 196 200 208 250 280 300 344 348 352 364 372 376 392 410 428 528 554 578 620 642 643 827 702 201 890 710 724 752 756 840
Australia Austria Bulgaria Canada Colombia Cyprus Czech Republic Denmark France Germany Greece Hong Kong Hungary Iceland Iran Ireland Israel Japan Korea Latvia Netherlands New Zealand Norway Portugal Romania Russian Federation Scotland Singapore Slovak Republic Slovenia South Africa Spain Sweden Switzerland USA
Minimum
Mean
Maximum
SD
0.0 0.0 0.1 0.0 0.1 0.0 0.1 0.7 0.1 0.1 0.2 0.2 0.0 0.3 0.5 0.0 0.1 0.7 0.2 0.2 0.0 0.1 0.1 0.1 0.0 0.1 0.1 0.2 0.1 0.1 0.0 0.0 0.1 0.0 0.0
2.3 1.8 3.2 2.3 3.5 2.4 2.9 15.7 3.8 3.1 2.6 3.5 2.8 4.3 5.8 2.5 5.3 8.2 3.2 3.0 1.8 3.2 2.4 2.8 2.3 4.3 4.1 4.0 2.7 2.7 2.7 2.0 2.5 1.7 2.8
11.5 12.5 26.8 12.6 17.0 38.4 18.2 97.9 14.1 29.9 21.4 19.0 12.9 20.7 14.3 12.6 26.5 26.5 22.8 24.3 21.1 15.9 40.0 25.9 25.1 32.9 25.5 18.5 35.9 20.0 24.9 6.0 20.0 20.0 16.1
2.4 2.6 5.2 2.8 3.8 5.5 3.7 31.3 4.7 5.6 3.9 3.7 2.9 4.8 4.2 3.2 7.3 8.4 4.7 5.2 3.0 3.5 5.9 4.0 5.0 6.1 5.4 5.0 5.3 3.8 4.5 1.6 3.7 3.0 2.5
Table 4. Distribution of conditional percent of science textbooks devoted to topics.
represents those statistics calculated using only the number of framework topics out of the 79 possible that the textbook actually addressed). Considering the space a textbook actually devoted to a particular topic varied across countries from a minimum of less than one percent (which accurately describes the minimum in every country) to a maximum of nearly 98 percent
Page 16
(Denmark). The range of magnitude of the standard deviations for these conditional textbook percents in each country are only somewhat smaller than the those seen in Table 3. The implication concerning the magnitude of the standard deviation is also similar: those countries with relatively smaller standard deviations would likely have more similar textbook space devoted to their topics than those countries exhibiting larger standard deviations. Indeed, the percent of textbook space devoted to topics is not characterized by great consistency across topics. The difference between the most and least emphasized topic in countries’ textbooks (excluding Denmark) ranged from as little as less than 10 percent (e.g., French-speaking Belgium and Spain) to nearly 40 percent (Cyprus and Norway). Considerable difference from one country to another is also evident in the amount of textbook space devoted to a typical topic (represented by the mean in Table 4) which ranges from as little as about two percent to around 16 percent (in Denmark). Japan also has a higher percent of textbook space averaged over topics (about eight percent). The variation in the standard deviations across countries is important because it indicates whether topics tend to receive similar amounts of space when included or whether they tend to receive an uneven distribution of textbook space. Clearly the two extremes would represent two very different approaches for or conceptions of science instruction. In the instance represented by a low standard deviation, science would seem to be treated as a systematic coverage of a number of relatively equally important topics. A relatively large standard deviation, conversely, suggests the existence of some principle to determine differential emphasis – national beliefs about curricular preparation, topic difficulty, or something else. The flat, “even emphasis” model seems to hold for most countries. The “uneven emphasis” model appears to hold for about 15 countries – based on standard deviations of around Page 17
five or more together with differences between the most and least emphasized topics of about 20 percent or more. The “flat”, or “even emphasis” (or, perhaps even, “lack of emphasis”) model was particularly characteristic of the US, Spain, Belgium (French language educational system), Australia, Austria, Hungary, and Canada. The US, Canada, and Australia demonstrated the same “flat” model for both eighth grade mathematics and science. Only Japan exhibited the “uneven emphasis” model for both science and mathematics (cf. Cogan & Schmidt, in press). As has been described, Tables 3 and 4 reveal the variability in the measures of variation for both topics across countries and for countries across topics. One way to examine these two phenomena at the same time is a median polish. A median polish examines the contents of a twoway table (e.g., data matrix) by repeatedly removing row and column effects from each element. Once the row and column summaries (or ‘marginals’) approach zero, cells that exhibit relatively larger absolute values indicate sites of interaction between the row and column factors. As employed here, for example, median polishing thus identifies unusual values (i.e., topic x country interactions) by comparing typical (median) values for a topic across all TIMSS countries and each country’s typical (median) value across all topics.5 Figure 1 is a plot of the cell values resulting from a median polish analysis of the percent of textbook space devoted to topics (37 countries by 79 topics). By definition, the vast majority of cells must have values of zero or very near zero. Yet nearly 80 country by topic interactions are evident (employing an absolute value criterion of 10 percent) which are found in over 40 percent of the science topics. Over three times as many country by topic interactions are evident if a five percent criterion is employed and these are evident in about 75 percent of the topics. Of the 48 tested science topics, 40 displayed large country by topic interactions. The tested areas
Page 18
that demonstrated no significant interactions were earth sciences (four topics) and conservation (two topics).
1600 1400
Number of Cells
1200 1000 800 600 400 200 0 -20
-10
0
10
20
30
40 50 Cell Value
60
70
80
90
100
Figure 1. Plot of cell values resulting from a median polish analysis of textbook space devoted to 79 science topics in 37 countries.
The topic areas with the most marked indication of country by topic interaction effects are the two topics most emphasized by textbooks across all countries – ‘electricity’ and ‘human biology.’ Both show especially large effects in some countries (e.g., relative to the effects of other topics within a particular country) and large effects in almost two thirds of the countries. These two topics are treated very differently in how they are emphasized in textbook space in various countries, although on average they dominate textbook coverage across all of the countries. The other topic with a relatively large number of estimated interaction effects is ‘energy types, sources and conversions.’
Page 19
Whatever the source of the differences, the data we gathered and have discussed here clearly show variations among how science topics are emphasized by textbook space in various countries and how countries differ in the topics they emphasize. They also show how there are widespread interactions among countries and topics showing different characteristic emphases in the eighth grade science curriculum as indicated by the proportion of textbook space devoted to different topics in different countries. As these descriptive analyses indicate, grade eight science textbooks present different views of science across countries. Although speculative, the different profiles of topic emphasis in textbooks across countries seem likely to reflect cultural notions of the sciences and of “school science.”
Classroom Instruction Under teacher coverage we consider primarily two aspects: the percent of a country’s teachers who teach a topic and the average percent of teacher time over the school year allocated to the topic. Table 2 shows that virtually all countries intended coverage for all 22 teacher categories. Notables exceptions were the Russian Federation and Singapore which intended coverage for about one fourth to one-half fewer topics respectively. Tables 5 and 6 show key aspects of the distributions (e.g., minimum, mean, median, maximum, and standard deviation) for each topic in terms of the percent of teachers in a country who taught it and the average percent of time spent teaching the topic over the school year. Again, the larger standard deviations indicate topics likely to exhibit country x topic interactions. Eighth grade science is very different from mathematics in this area. On average across the TIMSS countries, the eighth grade mathematics topic covered by the most teachers in a country had an average of 90 percent of teachers teaching it. That is, on average nine out of ten
Page 20
teachers spent some time teaching that topic which represents a high level of commonality. In science, the topic taught by the greatest percentage of teachers on average was ‘environmental and resource issues’ which was covered by only about 60 percent of the teachers. The next topics – ‘properties and classification of matter’ and ‘the structure of matter’ – were taught on average by about 55 to 60 percent of the teachers across counties. It would appear that 50 to 60 percent represents far less consensus and uniformity in curriculum implementation within a country than does 90 percent. However, there is a difficulty in interpreting this difference. Because of the way eighth grade science instruction was organized and how science instruction was sampled in the TIMSS countries, it is possible that the different teachers sampled may well have been teaching different courses. Many countries had eighth grade science instruction organized into separate courses – such as biology, chemistry, earth science, and physics – which likely were taught by different teachers. The TIMSS sampling rules sampled students and the teachers who taught those students. The sampling rules also specified that a maximum of only two teachers per sampled student would complete teacher surveys. Thus the sampled teacher pool within any one country may well have drawn teachers from different science courses. There may well have been virtually complete consensus among physics teachers but when this was averaged with biology teachers who didn’t teach that topic, averages of around 50 percent could occur. This could also help to explain the large standard deviations and ranges observed in the data. Caution, therefore, must be exerted in placing too much emphasis on these measures of variation for science topics. The top five topics according to the percentage of teachers teaching them on average across all countries were ‘environmental and resource issues’, ‘structure of matter’, ‘human biology & health’, ‘properties of matter’, and ‘energy processes.’ Nonetheless, it may well be Page 21
that a topic such as ‘environmental and resource issues’ was covered in different courses in countries that have multiple eighth grade science classes all for the same students.
topic
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Minimum Mean Maximum
Earth features Earth processes Earth in the universe Diversity & structure of living things Life processes & systems enabling life functions Life cycles, genetic continuity, diversity Interactions of living things Human biology & health Type & properties of matter Structure of matter Energy types, sources, conversions Energy processes Physical changes Kinetic & quantum theory General chemical changes Specialized chemical changes Forces & motion Relativity theory Science, technology, & mathematics History of science & technology Environmental & resource issues Nature of science
0.0 0.0 0.0 0.8 0.0 0.0 4.2 0.0 18.7 7.1 0.0 13.2 0.0 1.4 0.0 0.0 0.0 0.0 2.3 0.0 28.4 0.0
45.7 35.9 31.1 44.7 50.0 32.6 39.8 56.8 56.7 57.2 48.2 53.0 39.8 17.9 38.0 17.9 31.8 6.0 30.8 38.6 59.4 41.5
81.3 77.5 70.9 83.4 99.5 83.0 95.5 100.0 100.0 100.0 88.7 100.0 78.3 98.4 99.7 81.8 67.8 26.9 86.8 100.0 100.0 92.9
Std. Dev.
21.5 21.1 19.3 25.7 25.1 20.3 23.1 29.9 23.5 25.8 24.1 22.6 18.5 18.8 24.6 17.7 20.3 6.1 19.3 19.9 18.3 23.4
Table 5. Distribution of mean percent of eighth grade teachers teaching each science topic.
The topic that received the most teaching time on average across countries was ‘human biology & health’ (see Table 6). It received about 13 percent of the instructional time for eighth grade science. This value is consistent with the amount of time (also about 13 percent of instructional time) allocated to the most taught eighth grade mathematics topic – ‘equations and formulas’ (Cogan & Schmidt, in press). The problem discussed above for percentage of teachers covering a topic is not the same for the percentage of instructional time devoted to a topic. Since, in countries with multiple science courses for each student a student receives instruction in all of the science courses
Page 22
defined for eighth grade, an average over teachers of different disciplines is an appropriate estimate of the percentage of student-experienced instructional time devoted to that topic. This is true as long as it is understood that the unit referred to is the percent of instructional time is with respect to students and not teachers. One also must assume that the sampling rules were reasonably well followed in each country so that such averages actually represent the range of science courses and are not biased by over- or under-sampling some of the sciences. The behavior of the data in Table 6 appears concordant with this assumption. The only other eighth grade science topic to receive over ten percent of instructional time on average was ‘energy processes.’ This topic also had the largest variation (a standard deviation of 6.6 percent) with a range from two percent to about 30 percent.
topic
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Minimum Mean Maximum
Earth features Earth processes Earth in the universe Diversity & structure of living things Life processes & systems enabling life functions Life cycles, genetic continuity, diversity Interactions of living things Human biology & health Type & properties of matter Structure of matter Energy types, sources, conversions Energy processes Physical changes Kinetic & quantum theory General chemical changes Specialized chemical changes Forces & motion Relativity theory Science, technology, & mathematics History of science & technology Environmental & resource issues Nature of science
0.0 0.0 0.0 0.1 0.0 0.0 0.2 0.0 2.4 0.6 0.0 2.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.0
7.6 3.9 3.0 6.6 5.8 3.2 3.6 13.1 6.5 5.9 5.1 10.0 3.5 0.9 3.5 1.1 3.7 0.2 1.9 2.7 5.3 3.1
18.7 11.3 7.2 21.9 15.6 8.7 9.8 36.7 12.8 19.8 16.4 29.5 10.5 3.2 12.1 4.7 11.3 0.9 4.8 9.4 12.3 7.9
Std. Dev.
4.7 2.9 2.1 5.2 3.4 2.6 2.7 8.6 2.9 3.5 3.4 6.6 2.0 0.8 2.6 1.2 3.0 0.2 1.1 1.9 2.4 2.1
Table 6. Distribution of mean percent of eighth grade teaching time for each science topic.
Page 23
The five topics that received the most actual classroom instruction were (in order) ‘human biology’, ‘energy and physical processes’, ‘earth features’, ‘diversity and structure of living things’, and ‘types and properties of matter.’ It is worth noting that one of the top five topics implemented by teachers was an earth sciences topic despite the fact that no earth science topics were among the top topics intended as reflected in content standards. Given the sampling and estimation difficulties discussed earlier, it is encouraging that three of the topics on the “top five” lists were the same for both teaching indicators of curriculum. Table 6 indicates the standard deviations for the percent of instructional time devoted to each topic. The standard deviations range from around one percent (e.g., ‘kinetic and quantum theory’, ‘relativity theory’, ‘specialized chemical changes’) to around nine percent (‘human biology and health’). The range for different topics further illustrates the variability among different topics in different countries. ‘Human biology and health’ shows a considerable range from no time in one country to over one third of the instructional time in another. Other topics had relatively low variability (e.g., a range from zero to about three percent for ‘kinetic and quantum theory’). The variation across topics within a country also differs across countries, further suggesting an interaction of topic and country in instructional time. For some countries (Singapore and the Russian Federation) the percent of teachers who taught a topic ranged from some topics where no teachers taught it to others where all teachers taught it. For most (about 70 percent) of the countries, the lowest percent of coverage for any topic was five percent or less. Table 7 exhibits for each country the conditional distribution over topics of the percent of teachers teaching various topics given that a topic was taught by one or more teachers in that
Page 24
country. Despite the rather uniform way this data appeared in Table 2, the conditional distributions provide some interesting contrasts. For example, a typical topic in the US had a little over three-fifths of the teachers teaching it. Only Singapore had more teachers teaching a typical topic. The vast majority of countries had fewer than 50 percent of teachers teaching a typical topic. Most countries had between about 30 to 40 percent of their teachers teaching a typical topic whereas the range for mathematics was between 45 and 55 percent (Cogan & Schmidt, in press). This difference likely
Page 25
40 56 57 ## 170 ## 200 ## ## 300 ## 348 ## 364 ## 376 ## 410 ## 528 ## 578 ## 642 ## 702 ## 250 840 ## ## 752 ##
Australia Austria Belgium (Fl) Belgium (Fr) Canada Colombia Cyprus Czech Republic France Germany Greece Hong Kong Hungary Iceland Iran Ireland Israel Japan Korea Latvia Netherlands New Zealand Norway Portugal Romania Russian Federation Singapore Slovak Republic Slovenia Spain Sweden Switzerland USA
Minimum
Mean
Maximum
Std. Dev.
3.7 5.5 1.3 1.4 8.5 21.3 2.7 2.7 1.8 2.4 2.3 4.9 6.0 5.0 14.9 8.2 3.0 1.1 3.6 6.8 3.6 5.9 3.0 0.4 2.3 48.5 0.8 2.5 6.7 11.0 4.0 5.4 26.9
50.4 20.2 25.4 28.6 50.0 57.7 40.7 37.3 29.4 32.0 38.6 35.5 26.5 36.0 57.6 49.6 34.8 43.5 41.4 28.9 30.9 59.0 53.6 29.1 24.4 56.4 81.6 34.9 37.3 53.8 37.7 39.5 61.8
75.6 37.4 82.7 75.5 81.2 95.9 92.9 74.0 59.0 57.2 74.7 80.7 50.4 77.8 99.6 94.2 97.0 96.3 80.5 61.2 72.7 96.9 84.9 54.2 54.3 100.0 100.0 96.0 63.9 92.5 64.1 80.7 91.7
19.3 8.6 23.3 21.5 20.8 22.2 30.2 19.5 21.3 16.0 22.9 23.5 12.4 23.0 26.9 24.3 27.2 31.6 23.7 14.9 16.4 25.2 27.5 17.9 12.7 17.1 37.9 26.6 14.8 22.2 18.2 22.1 19.8
Table 7. Distribution of conditional percent of eighth grade science teachers teaching each topic.
reflects a fundamental difference in the organization of science versus mathematics. Whereas students typically study a single mathematics course, for science many countries had eighth grade students study multiple courses – such as biology, chemistry, physics and earth science – which may have been taught by different teachers perhaps as alternating courses in students’ schedules throughout the school year. Notably, all the countries that administered the multiple science version of the TIMSS Student Background Questionnaire are among those with the Page 26
lowest means with the exception of the Russian Federation (see Schmidt & Cogan, 1996). In each case, the percent of teachers teaching a typical topic (i.e., the mean in Table 7) if multiplied by the number of sciences taught (e.g., three or four) yields approximately 100 percent indicating a decisive pattern of science topics taught for each type of science course. (The same holds true for the Russian Federation if one assumes that each teacher teaches two of the science courses students took.) The fact that the US had so many teachers teaching a typical topic, one of the highest percentages of teachers teaching the least taught topic, together with the fact that all possible topics were taught by teachers of several different types of science provides further evidence of the “mile wide, inch deep” nature of the US curriculum first reported in A Splintered Vision and further amplified in Facing the Consequences. Table 8 provides the median polish results for the percent of teachers teaching a topic (a median polish for the percent of teaching time given each topic yield a similar pattern but is not displayed here). The median polish analyses clearly indicate the presence of sizable country x topic interactions. For every topic there are numerous countries displaying at least one interaction effect greater than or equal to a ten percent difference and every country exhibited interactions for multiple topics. (This was less true for teaching time.)
Page 27
mask: 10. Australia Austria Belgium (Fl) Belgium (Fr) Canada Colombia Cyprus Czech Republic France Germany Greece Hong Kong Hungary Iceland Iran Ireland Israel Japan Korea Latvia Netherland New Zealand Norway Portugal Romania Russian Federation Singapore Slovak Republic Slovenia Spain Sweden Switzerland USA Marginal Medians
1 + ~ : 14.6 15.1 + -15.9 + 15.9 -15.1 31.1 ~ -18.2 ~ + ~ -25.1 18.1 37.0 18.1 11.1 -21.6 27.0 -20.8 ~ + -34.6 ~ -18.1 + ~ 22.6 ~
2 + + + + ~ + -15.9 + + -13.1 29.1 -22.9 -14.2 + -11.2 -30.0 -23.1 34.1 43.0 + 18.1 -25.6 33.0 -21.8 ~ + -25.6 ~ ~ ~ ~ 29.6 :
3 + + ~ ~ ~ : -12.5 15.3 20.3 ~ + -23.5 + + -31.8 -13.6 -21.8 -12.5 -13.6 ~ + + 28.4 17.6 ~ -35.6 -22.3 + 22.3 ~ 13.0 18.0 13.4
4 ~ -11.3 20.9 42.4 ~ 22.5 -12.0 -35.3 -29.3 + : + -16.3 39.0 + + + 24.0 + ~ -18.0 11.3 28.9 -29.9 ~ ~ -33.8 -35.5 ~ + 18.5 32.5 -35.1
5 + + 43.9 16.4 ~ 24.5 + -16.3 -38.3 ~ ~ 37.0 ~ 32.0 -15.3 27.9 -12.3 37.0 27.9 -10.1 : ~ 29.9 ~ -14.6 ~ 65.3 -32.5 -24.3 + + 19.5 -34.1
6 -17.4 21.8 41.0 + -16.9 37.6 ~ -18.1 ~ + -16.9 -11.9 15.8 35.1 11.8 + 19.9 -12.9 -12.0 ~ + -26.6 : -16.8 16.5 14.1 -20.6 -20.4 11.9 36.1 ~ + -15.0
7 : ~ + 31.9 13.5 36.0 ~ -15.8 -20.8 10.3 ~ -16.5 -13.8 29.5 -21.8 + ~ ~ -14.6 + ~ + 31.4 -25.4 12.9 + 68.8 -26.0 + -11.6 21.0 23.0 -22.6
8 : + 42.4 + -23.5 26.0 25.5 -38.8 + ~ ~ 29.5 ~ -28.5 21.2 27.4 45.3 41.5 19.4 -33.6 -11.5 -22.3 18.4 ~ ~ -58.6 54.8 -42.0 -11.8 23.4 -32.0 27.0 -34.6
9 + -11.3 -19.1 ~ 14.0 ~ 40.0 ~ + ~ 16.0 ~ -20.3 : + ~ 33.8 29.0 + ~ -12.0 18.3 16.9 + ~ ~ 57.3 -14.5 -25.3 11.9 + ~ +
10 ~ -14.8 -36.6 -36.1 ~ : 21.5 + + ~ + -17.5 -22.8 -32.5 + + 37.3 29.5 11.4 ~ -22.5 + 17.4 + ~ ~ 51.8 ~ -25.8 16.4 + -31.0 +
11 ~ ~ -12.0 -22.4 24.1 -16.4 37.1 21.9 -24.1 + 17.1 13.1 ~ -16.9 17.8 13.0 ~ -27.9 -25.0 + 12.1 11.4 ~ ~ + + -37.6 47.6 11.9 + -23.4 -11.4 :
12 + ~ -13.9 -12.3 + -20.3 32.3 ~ + : + 30.3 ~ -32.8 18.9 16.1 14.0 21.3 -12.9 -12.8 + + -24.9 + + ~ 58.5 51.8 -10.0 -29.8 -18.3 -17.3 ~
13 + ~ -14.9 + + ~ 22.3 ~ + : 10.3 ~ ~ ~ + + -26.0 -14.8 11.1 + ~ 10.5 -14.9 + ~ + -31.5 26.8 -18.0 -23.8 + -16.3 10.1
14 ~ 13.7 + ~ + ~ + + ~ ~ : + 10.7 ~ -21.3 ~ + + 31.9 + : -10.8 -19.1 + + 35.9 95.3 22.5 ~ ~ ~ ~ ~
15 14.5 13.7 -12.1 -15.6 ~ ~ : 12.8 ~ ~ ~ 11.0 + ~ 26.7 + ~ 57.0 26.9 ~ ~ + ~ 16.1 + -35.1 78.3 ~ ~ 31.9 + ~ 13.9
16 ~ 33.7 + ~ ~ ~ ~ 31.8 ~ ~ -10.0 + 20.7 + 44.7 ~ + ~ ~ + : -32.8 -15.1 + 13.4 -17.1 ~ 36.5 12.8 + ~ + 14.9
17 -15.4 + + 45.6 ~ ~ ~ -10.1 -18.1 + 26.1 38.1 + ~ ~ 21.0 -14.1 -25.9 + ~ + + ~ -16.8 ~ -33.9 -19.6 16.6 + 24.1 + + :
18 -16.0 19.2 + + ~ : + ~ + + ~ + 11.2 + ~ ~ + ~ ~ + + -26.3 ~ ~ 11.9 ~ + 10.0 + ~ ~ ~ ~
19 ~ 13.7 ~ + 16.0 ~ : 12.8 -19.3 + -15.0 + + ~ -33.3 -22.1 ~ -14.0 ~ + + ~ ~ + 10.4 20.9 69.3 33.5 15.8 ~ ~ -10.5 29.9
20 ~ ~ ~ ~ 12.1 ~ -13.9 13.9 + 17.9 -23.9 -13.9 23.8 26.1 -35.2 ~ + + : + ~ ~ ~ + 16.5 62.1 -24.6 34.6 23.9 ~ 14.6 ~ +
21 -20.0 ~ + ~ 10.5 14.0 14.5 14.3 ~ + ~ -20.5 + 20.5 -31.8 -12.6 -27.8 -24.5 -17.6 + 20.5 -12.3 + -17.4 10.9 37.4 48.8 : + ~ ~ : ~
22 13.5 ~ ~ -16.6 31.0 10.5 60.0 -13.3 20.8 24.8 + ~ 14.7 24.0 -28.3 -24.1 + -14.0 ~ -15.1 : 22.3 ~ 16.1 ~ 10.9 -27.8 ~ 11.8 -13.1 11.5 -16.5 28.9
48.0
35.0
30.0
46.0
48.0
28.0
39.0
50.0
54.0
62.0
51.0
46.0
37.0
12.0
29.0
11.0
29.0
4.0
23.0
40.0
57.0
40.0
Table 8. Median polish for percent of teachers teaching each topic in 33 countries.
Page 28
Marginal Medians 55.0 21.0 19.5 24.0 52.5 51.5 27.5 42.5 30.5 26.0 36.0 42.0 32.5 24.0 29.5 60.0 50.5 28.0 31.0 38.5 29.0 30.5 59.0 51.5 34.0 22.0 48.0 1.0 53.0 37.0 35.0 68.5 38.0
Along with the number of framework topics found in content standards and textbooks, Table 2 also indicates the maximum number of topics covered by science teachers in each country (out of the 22 categories listed on the teacher questionnaire which represented the entire TIMSS Science framework). If any science teacher in a country taught a topic, that topic was considered to have been taught in that country for the purposes of Table 2. Perhaps not too surprisingly given this criterion, in virtually all countries all topics were covered by teachers. The only notable exceptions were the Russian Federation whose teachers taught about three-fourths of the possible topics (e.g., 16) and Singapore whose teachers taught little more than half of the possible topics (e.g., 12). For percent of instructional time, only seven of 22 topics failed to exhibit an indication of a strong interaction with either the five or 10 percent criteria. Of these seven, three were not tested and two others are not content belonging to a specific science (e.g., ‘science, technology and society’ and the ‘nature of science’). Large effects were noted for about a third of the topics.
Commonalties across countries: the notion of a world core Up to this point, we’ve examined for each of the three curricular instantiations some of what is typical across countries for specific topics as well as what is different. The question now arises, but what of the overlap? That is, to what degree are the three instantiations similar across all these countries? Do the three instantiations say the same thing about what science is taught at eighth grade? Is the curricular message from content
Page 29
standards and textbooks the same? How does the curricular message from these instantiations mesh with the classroom reality – what teachers report they actually taught? For policymaking, these questions ultimately must be asked at the country level. Portions of these issues have already been addressed for all TIMSS countries (see especially Table 6.1 in Many Visions, Many Aims, Vol. II, pp. 118-121) and with a particular focus on the US (see especially chapter two in Facing the Consequences). Here we examine these issues more directly for the whole TIMSS world using the 36 countries that provided data for all three curriculum instantiations.
Content Standards 1 2 3 4 5
Textbooks
Energy handling Energy types, sources, conversions Human biology Electricity Conservation of land, water, & sea resources
1 2 3 4 5
Teachers 1 2 3 4 5
Electricity Human biology Organs, tissues Energy types, sources, conversions Chemical properties Instructional Time
Environmental & resource issues Structure of matter Human biology & health Types & properties of matter Energy processes
1 2 3 4 5
Human biology & health Energy processes Earth features Diversity & structure of living things Types & properties of matter
Table 9. The five most emphasized eighth grade science topics for four curriculum indicators averaged across all countries.
Table 9 exhibits the top five topics for each of the three instantiations, including the two manifestations of teacher implementation – percent of teachers teaching and percent of teaching time. Unlike mathematics where there was a single topic that was most emphasized across all indications, in science each of the four identifies a different top topic and they are not at all similar (cf. Cogan & Schmidt, in press). All five of the
Page 30
top topics for content standards were covered by 90 percent of the countries. Remembering this, the one topic that emerges clearly near the top of all four lists is ‘human biology.’ Although the order does not agree, in addition to ‘human biology’, there is fairly good agreement across the indications for energy (‘electricity’, ‘energy types, sources and conversions’, and ‘energy processes’) and environmental resource issues (including conservation of land, water and sea resources). One anomaly is the presence of ‘earth features’ which has a large percent of instructional time although it is not widely emphasized by content standards or textbooks. These differences do not necessarily imply inconsistencies since they are all aggregations. However, it is interesting to note that in the “world according to TIMSS”, what is emphasized in eighth grade science seems to differ somewhat given the indication used to make curriculum visible. Certainly differences are apparent in what is emphasized by content standards (an expression of national intentions), textbooks, and teacher-based indicators of curricular content actually implemented. The greater consistency across indications observed for mathematics likely reflects the greater diversity in the way science is organized and delivered across the TIMSS countries, at least in comparison to mathematics (cf. Cogan & Schmidt, in press). In order to better portray what is common, it is possible to define a “world core” curriculum for eighth grade science from these data. Previously we’ve discussed this type of core curriculum based only on content standards and textbooks (see Schmidt, McKnight, & Raizen, 1997). Here we are able to add curriculum indicators from the
Page 31
teacher data as an additional criterion. Applying a 70 percent criterion, a “world core” curriculum is defined by those topics appearing in 70 percent of countries’ textbooks, content standards, and taught by at least 70 percent of teachers in at least 70 percent of countries. While the “world core” eighth grade mathematics curriculum consisted of 12 topics when defined by content standards and textbooks, it consisted of only three mathematics topics – ‘perimeter, area, and volume’, ‘2-D geometry basics’, and ‘equations and formulas’ – once the teacher criterion was added. Similarly for science, 26 topics defined the “world core” curriculum according to standards and textbooks. However, no topic met the 70 percent criterion with respect to teachers. In fact, we had to adjust this criterion to 50 percent of teachers in order to get any common topics for teachers. If the teacher criterion is adjusted so that only 50 percent of teachers in a country teach a topic, then one topic meets the 70 percent criterion: ‘environmental and resource issues related to science.’ This means that this was the only topic that was intended (by content standards), covered in textbooks, and implemented by at least 50 percent of the teachers in at least 70 percent of the TIMSS countries providing these data. If the percent of teachers were dropped to 40 percent, the ‘structure of matter’ topic would be added to this 70-percent “world core.”
Conclusions From these descriptions of the three curricular instantiations, two conclusions are warranted. The first is that it is quite clear that there are different cultural approaches, as exhibited by the differences across countries, in the way science is defined for eighth Page 32
grade students. These different cultural approaches are manifested by each of the three instantiations of curricula: content standards, textbooks, and teachers’ instruction. Obviously, there is more than one way to conceptualize and teach eighth grade science. Looking at the specific topics included and emphasized both across the three instantiations within each country and comparing these across countries, the science – or sciences – that students study in one country can look quite different from what students in another country study. These cultural differences in what constitutes eighth grade science are quite likely not inconsequential – not only for how students might perform on any given assessment but for their future learning as well. Even if all the different cultural approaches to science schooling yielded identical performances at the eighth grade level, one might still find that the particular curricular pattern observed in one country provided a stronger foundation for more advanced science study than other alternatives. Given the various performance patterns observed on the TIMSS eighth grade and end of secondary science tests, both of these ideas appear quite plausible. Currently we are pursuing more formal analyses relating the aspects of curricula presented here to various country level measures of students performance (Schmidt McKnight, Houang, Wang, Wiley, Cogan, & Wolfe, in press). The second conclusion is that even though countries may strive to have alignment across the three curricular instantiations, there remains some variation in the definition of what constitutes eighth grade science within any one country. The fact that the way in which these three instantiations vary amongst themselves from one country to the next simply reinforces the first conclusion. That is, there is no single way in which the three instantiations differ from one another from one country to the next. This fact emphasizes
Page 33
the cultural context in which these instantiations have been developed and operate. Furthermore, it accentuates the folly of adopting in a wholesale fashion the curricular patterns observed in an alien culture. Clearly we can (must?) learn from other cultures but these lessons must be thoughtfully analyzed and creatively translated into our own unique cultural context for education. Failing to recognize the cultural nature of schooling and measures of it precludes useful insights and conclusions being developed for improving educational policies and practice. On the other hand, failing to examine the sometimes shocking diversity displayed in the many ways different societies accomplish science education may doom our students to a stale and sterile educational experience for many years to come.
NOTE: This article draws upon material presented in a chapter of the forthcoming book, Why schools matter: using TIMSS to investigate curriculum and learning. All of this work was funded through a grant (Award #9551017) from the U.S. National Science Foundation and gratefully acknowledge this support. The findings, opinions, and conclusions expressed here are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Page 34
REFERENCES
Bock, P. K. (1970). Culture Shock: A Reader in Modern Cultural Anthropology. New York: Alfred A. Knopf. Cogan, L. S., & Schmidt, W. H. (in press). “Culture Shock” - Eighth grade mathematics from an international perspective. Educational Research and Evaluation: An International Journal on Theory and Practice. Emerson, J. D., & Hoaglin, D. C. (1983). Analysis of Two-Way Tables by Medians. In D. C. Hoaglin & F. Mosteller & J. W. Tukey (Eds.), Understanding Robust and Exploratory Data Analysis (pp. 166-210). New York: John Wiley & Sons, Inc. Longstreet, W. S., & Shane, H. G. (1993). Curriculum for a new millennium. Needham Heights, MA: Allyn & Bacon. National Center for Education Statistics (NCES). (1996). Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching, Learning, Curriculum, and Achievement in International Context (97-198). Washington, D.C: NCES , U.S. Government Printing Office. Robitaille, D. F., Schmidt, W. H., Raizen, S., McKnight, C., Britton, E., & Nicol, C. (1993). Curriculum Frameworks for Mathematics and Science (TIMSS Monograph No. 1). Vancouver: Pacific Educational Press. Schmidt, W. H., & Cogan, L. S. (1996). Development of the TIMSS context questionnaires. In M. O. Martin & D. L. Kelly (Eds.), Third international mathematics and science study technical report (Vol. Volume I: Design and Development, pp. 5-1-5-22). Chestnut Hill, MA: Boston College. Schmidt, W. H., Jorde, D., Cogan, L. S., Barrier, E., Gonzalo, I., Moser, U., Shimizu, K., Sawada, T., Valverde, G., McKnight, C., Prawat, R., Wiley, D. E., Raizen, S., Britton, E. D., & Wolfe, R. G. (1996). Characterizing Pedagogical Flow: An Investigation of Mathematics and Science Teaching in Six Countries. Dordrecht/Boston/London: Kluwer. Schmidt, W. H., McKnight, C., Cogan, L. S., Jakwerth, P. M., & Houang, R. T. (1999). Facing the Consequences: Using TIMSS for a Closer Look at US Mathematics and Science Education. Dordrecht/Boston/London: Kluwer. Schmidt, W. H., McKnight, C., & Raizen, S. (1997). A Splintered Vision: An Investigation of U.S. Science and Mathematics Education. Dordrecht/Boston/London: Kluwer. Schmidt, W. H., McKnight, C., Valverde, G. A., Houang, R. T., & Wiley, D. E. (1997). Many Visions, Many Aims, Volume I: A Cross-National Investigation of Curricular Intentions in School Mathematics. Dordrecht/Boston/London: Kluwer.
Page 35
Schmidt, W. H., McKnight, C. C., Houang, R. T., Wang, H. A., Wiley, D. E., Cogan, L. S., & Wolfe, R. G. (in press). Why schools matter: using TIMSS to investigate curriculum and learning. San Francisco: Jossey-Bass. Schmidt, W. H., Raizen, S. A., Britton, E. D., Bianchi, L. J., & Wolfe, R. G. (1997). Many Visions, Many Aims, Volume II: A Cross-National Investigation of Curricular Intentions in School Science. Dordrecht/Boston/London: Kluwer. Valverde, G. A., & Schmidt, W. H. (2000). Greater Expectations: Learning from other nations in the quest for ‘world-class standards’ in US school mathematics and science. Journal of Curriculum Studies, 32, 651-687.
1
The TIMSS gathered data for three student populations: the two adjacent grades containing the majority of nine-year-olds (Population 1), the two adjacent grades containing the majority of thirteen-year-olds (Population 2), and students in the final year of secondary school (Population 3). The focus in this article, as was the case in the main TIMSS study, is on the upper grade of population 2 with an eye towards developing a clear understanding of what science is intended to be taught and learned according to the curriculum. Although the title for this grade level varied from country to country, with few exceptions these students had had eight years of formal schooling hence, here, we use ‘eighth grade’ to refer to these students and their curriculum.
2
Initially, the frameworks were presented and explained only in technical reports but have been reproduced, documented and published. See Robitaille, D. F., Schmidt, W. H., Raizen, S., McKnight, C., Britton, E., & Nicol, C. (1993). Curriculum Frameworks for Mathematics and Science (TIMSS Monograph No. 1). Vancouver: Pacific Educational Press.
3
For further discussion of the TIMSS frameworks as they relate to coding and analyzing documents, see Characterizing Pedagogical Flow: An Investigation of Mathematics and Science Teaching in six Countries (Schmidt, et al, 1996) or one of the international curriculum analysis volumes: Schmidt, W. H., McKnight, C., Valverde, G. A., Houang, R. T., & Wiley, D. E. (1997). Many Visions, Many Aims, Volume I: A Cross-National Investigation of Curricular Intentions in School Mathematics. Dordrecht/Boston/London: Kluwer. Or Schmidt, W. H., Raizen, S. A., Britton, E. D., Bianchi, L. J., & Wolfe, R. G. (1997). Many Visions, Many Aims, Volume II: A Cross-National Investigation of Curricular Intentions in School Science. Dordrecht/Boston/London: Kluwer.
4
Although we refer to the ‘percent of teachers’ or the ‘percent of teaching time’ in a country as a shorthand reference to the data, such language is technically inaccurate since the TIMSS did not sample teachers but students. The sampling design first selected a stratified random sample of schools with probability proportional to the number of students in the sampled grade. Mathematics classrooms were then randomly selected from each school and these students’ teachers – both their mathematics teacher and the science teachers who taught science to the students in the selected mathematics class – responded to the appropriate TIMSS Teacher Questionnaire. Teacher questionnaire responses are then weighted by the number of students associated with that teacher. Therefore, reference to the percent of teachers in a country actually refers to the percent of students who had such teachers and reference to instructional time refers to the amount of experienced instructional time averaged across all students. In light of our goal of characterizing curriculum, the shorthand adopted is reasonable.
5
A median polish makes use of the fact that data in a two-dimensional (row by column) array has an additive structure. In our case, rows indicate countries and columns topics. Each “cell” (a specific topic for a specific country) may be thought of as the sum of numbers from four sources. First, there is an overall “median of medians” that is common to all topics and countries. It puts a specific cell’s number
Page 36
into a general range of values. Second, there is a “row effect”, that is (for us), an additional percentage common to a country (row) across all columns (topics). It represents the contribution of the specific country’s characteristic emphasis to the total emphasis of the specific topic for that country. Third, there is a “column effect”, that is (for us), an additional percentage common to a topic (column) across all rows (countries). It represents the contribution of the specific topic’s characteristic emphasis to the total emphasis of that topic for the specific country. Fourth and finally, there is an additional “cell effect” or “residual” that represents an adjustment of the combination of overall effect, row effect, and column effect to get the specific effect for that country and topic combination. It represents what is unique for a specific topic in a specific country. Median polishing is a technique that uses the iterative subtraction and accumulation of row and column medians to try to separate these four components. The result is a table with a median of medians (overall effect or “centering” of all the numbers in the table), row medians (characteristic country contributions across topics), column medians (characteristic topic contributions across countries), and the residual (what is left that is special to that topic-country combination in each cell of the array). Unusually high or low (that is, large absolute values with either positive or negative signs) residuals indicate unusually strong interaction effects of country and topic. We spot these by performing median polishes and identifying large residuals. The usual method of estimating two-way models is by row and column means. The advantage of using medians is two-fold: (i) medians are more robust than means and (ii) residuals are not assumed to be random and can estimate row by column interaction effects. For further information on the median polish procedure, see Emerson & Hoaglin’s chapter (pp. 166-210) in Hoaglin, D. C., Mosteller, F., and Tukey, J. W. (eds.). 1983. Understanding Robust and Exploratory Data Analysis. New York: John Wiley & Sons, Inc.
Page 37
