LEARNING ARGUMENTATION SKILLS THROUGH ... - Springer Link

SHU-SHENG LIN and JOEL J. MINTZES

LEARNING ARGUMENTATION SKILLS THROUGH INSTRUCTION IN SOCIOSCIENTIFIC ISSUES: THE EFFECT OF ABILITY LEVEL Received: 10 July 2008; Accepted: 22 March 2010

ABSTRACT. This article describes an effort to explore and enhance argumentation skills of Taiwanese grade 6 students through instruction in socioscientific issues. An experienced elementary school teacher was given 8 months of personalized instruction on argumentation skills and socioscientific issues, then subsequently implemented a 17-h classroom unit on the establishment of Ma-Guo National Park. His students learned to establish claims and warrants, construct counterarguments, offer supportive arguments, and provide evidence for each one. Data consisted of student responses to questionnaires and individual follow-up interviews. A multiple regression analysis revealed that success in learning argumentation skills was not substantially related to pre-instruction argumentation skills, but significantly related to the student ability levels. High-ability students were significantly better than low-ability students at generating complete arguments. Most students elaborated their arguments, and more highability students offered rebuttals after instruction. However, even these high achievers did not completely understand the meaning of evidence and often misused supplementary warrants as evidence. KEY WORDS: ability level, argumentation skills, elementary school students, science instruction, socioscientific issues

Science education has emphasized not only what students know but also how and why they know (Millar & Osborne, 1999). The underlying assumption is that scientifically literate individuals are able to conduct science inquiries, interpret and evaluate evidence, make claims and warrants or justifications, and construct deeper understandings about science. These thinking strategies are an essential component of the fundamental sense of scientific literacy, which in turn contribute to a conceptual understanding of the “big ideas” of science—the derived sense of scientific literacy (Yore, Pimm & Tuan, 2007). Familiarity with argumentation skills enables people to understand others' perspectives, evaluate the sufficiency or necessity of existing warrants, and judge the validity of common assertions. When students learn to construct valid scientific arguments and then integrate those thinking skills with knowledge about specific socioscientific issues (SSIs), they are better able to justify their positions on the issues and are ultimately better prepared to engage in debate on controversial matters. In Taiwan, the Grades 1–9 Science and Technology Curriculum Guidelines and Competence Indicators encourage teachers to develop International Journal of Science and Mathematics Education (2010) 8: 993Y1017 # National Science Council, Taiwan 2010

994

SHU-SHENG LIN AND JOEL J. MINTZES

instructional materials on socioscientific issues that enhance students' critical thinking and argumentation skills. This approach attempts to develop citizens who can make deliberate and rational decisions. However, most science teachers in Taiwan have little experience in teaching argumentation. Owing in part to the strong cultural influence of Confucian philosophy, students are taught to revere teachers, seek consensus, and avoid confrontation. As a result, science teachers rarely give students opportunities to justify their knowledge claims and students rarely rebut teachers' assertions in science classes. Most previous studies on teaching argumentation skills through SSIs have focused on secondary school or undergraduate college students. The research that explores elementary school students' argumentation skills through SSIs is still in its infancy (Anderson, Chinn, Chang, Waggoner & Yi, 1997), especially in Taiwan. There are few studies about the level of argumentation skills elementary students possess, the problems they encounter when they attempt to make arguments about a SSI, and whether factors such as ability level or gender play a significant role. The current study attempted to address this important gap in the research literature by exploring an experienced and trained elementary school science teacher's development and implementation of instructional materials for teaching argumentation skills through SSIs and to explore differences in the argumentation skills of the grade 6 students in his classes. Three research questions guided this study:

Can elementary school students of diverse ability levels learn to use argumentation skills? Is the acquisition of argumentation skills related to pre-instruction argumentation skills or ability level? What problems persist in argumentation after instruction?

THEORETICAL FRAMEWORK The basis for this study involves the importance of equipping students with argumentation skills promoted by the current international science education reforms that are fundamental components of science literacy and which lead to a fuller and more informed participation in the public debate about critical issues facing society. Furthermore, it explores the relationship between learner characteristics and argumentation skills and establishes design principles for argumentation instruction through SSIs.

ARGUMENTATION SKILLS

995

Science Education Reform Science literacy for all has been the main goal of international reforms of science education in many countries (Hand, Prain & Yore, 2001). To cultivate scientifically literate citizens who can be responsible for actively participating in public debate about SSIs or science, technology, society, and environmental (STSE) issues, deliberately making decisions, and wisely solving controversial problems has become an important task for science education (Kolsto, 2001). Students' argumentation skills on STSEs and SSIs have been identified as potentially powerful in enhancing scientific literacy (Sadler & Zeidler, 2005). Given this international context, Taiwan has been encouraged to launch new, innovative grades 1–9 science and technology curriculum guidelines and competence indicators, which put considerable emphasis on developing and fostering higher-order thinking skills, such as problem solving, critical thinking, and argumentation (Ministry of Education of ROC, 2001). This curriculum change means that improving the quality of students' arguments has priority in science education in Taiwan, which is similar to practices involving argumentation elsewhere (Duschl & Osborne, 2002). Kuhn & Udell (2003) stated that the term argument is usually used as a product, which refers to the results where an individual or a group of people are asked to justify claims or viewpoints, whereas the term argumentation refers to the process of constructing arguments. In this process, there are two or more people engaging in debate on opposing claims. Toulmin (1958) developed a model of argument involving claims, warrants, evidence, counterclaims, and rebuttals that many researchers have used to identify the characteristics of an argument. A claim is an assertion about an issue. A warrant is a reason for making a claim, which can be challenged with an alternative assertion, called a counterclaim. A rebuttal is a valid rejection of a warrant that is in support of a counterargument. Evidence, including numerical or descriptive data, concrete examples, or facts, is explicit information related to the claim or counterclaim derived from experiments, observations, or surveys (Aikenhead, 2005; Sandoval & Millwood, 2005). The evidence may support a claim, a warrant, or a rebuttal or it may tend to refute them. Learner Characteristics and Argumentation Skills Studies focusing on argumentation skills have revealed that the majority of students are inadequately prepared to analyze the knowledge claims and warrants offered by others and to construct and support their own arguments (Driver, Newton & Osborne, 2000; Jiménez-Aleixandre,

996


Rodriguez & Duschl, 2000; Naylor, Keogh & Downing, 2007). Most studies have ignored the existence of individual differences between students in argumentation skills. Therefore, it is important to explore individual differences in students when they make an argument and to consider these individual differences when designing instructional materials and adopting teaching strategies. Rivard (2004) pointed out that one learner characteristic contributing to individual differences in science learning is students' past academic achievement levels. They reflect their prior knowledge, general skills, motivation, or self-confidence. Students with different academic achievements (high, average, and low) show differences in gaining higher-order thinking skills (Dori, Tal & Tsaushu, 2003). Zohar & Dori (2003) found that middle school students had a significant improvement in higher-order thinking skills—question posing, argumentation, and system thinking—at all academic levels after the instruction. However, students with levels of high academic achievement demonstrated higher scores than students with low academic achievement, but the net gain of high achievers was significantly lower than that of low achievers in one of their four studies. Yerrick (2000) examined the effects of inquiry instruction on five low-achieving students' argumentation; he found that these students were able to actively make arguments and use evidence to support their arguments after the instruction. Cognitive ability is a crucial attribute of students' level of academic performance, which may be related to their argumentation performance. von Aufschnaiter, Erduran, Osborne & Simon (2008) explored the connection between prior achievement and argumentation and found that “when engaging in argumentation students draw on their prior experiences and knowledge; [and that] such an activity enables students to consolidate their existing knowledge and elaborate their science understanding at relatively high levels of abstraction” (p. 101). Lawson (2003) suggested that the ability to engage in argumentation could be traced to the preverbal reasoning level of young learners. Together, these studies support the view that the acquisition of argumentation skills is strongly related to the learners' declarative knowledge within related domains (mathematics, language, science, social studies) and to the general level of cognitive reasoning. Thus, ability or achievement levels might serve as predictors for exploring students' acquisition of argumentation skills (Stanovich, 1999). Argumentation Instruction through Socioscientific Issues Developing instructional materials (teaching strategies, learning activities, and assessment techniques) to improve students' argumentation skills


997

becomes an essential task for science educators. A 25-year review of language arts in science education revealed that authentic issues and illstructured problems with multiple solutions and decisions flowing from the deliberations could be the most effective means to stimulate discourse, discussion, and debate (Yore, Bisanz, & Hand, 2003). In this context, it has been suggested that SSIs offer a suitable domain within which to equip students with argumentation skills (Sadler, 2004; Simonneaux, 2008; Zeidler, Sadler, Simmons & Howes, 2005). Using SSIs motivates students through their relevance to their everyday lives and creates a meaningful space for students to engage in discussion of potentially controversial topics. SSIs such as radiation safety, the risk of genetically modified organisms, or the utilization of environmentally friendly energy sources are current, important, controversial issues involving the utilization of scientific knowledge. Authentic SSIs that are open-ended, ill-structured, and unresolved problems tend to elicit different and sometimes conflicting perspectives by various interest groups; often a consensus among interests groups is difficult to reach (Levinson, 2006b). In these circumstances, the teacher can guide students to understand multiple perspectives on the issue and to construct arguments in which students take a controversial position and make claims, assert warrants, and provide evidence to defend their stance (Oulton, Dillon & Grace, 2004). Zohar & Nemet (2002) suggested that teaching argumentation skills is best achieved when learning focuses on a real problem occurring in students' everyday lives. The result is that students become more engaged in the argumentation activities and scientific discourse. Wray & Lewis (1997) offered a writing frame as a structure and teaching strategy to help students generate a written argument. The frame provides scaffolding as students construct an argument (e.g. my idea is that …, my reasons are that …, arguments against my idea might be …). Science teachers use this frame as a model for their students to practise their written argumentation skills. This approach is consistent with the results of previous research that shows that argumentation needs to be taught explicitly (Osborne, Erduran & Simon, 2004). Role play is another effective strategy for SSI instruction (Simonneaux, 2001). Through this technique, teachers attempt to help students understand perspectives held by others by discussing the claims and evidence others might present and by justifying claims and refuting counterclaims in a debate, which creates opportunities for students to reflect on their arguments. Group or whole-class discussion is another avenue for teaching argumentation skills through SSIs (Maloney & Simon, 2006). Discussions

998


enable students to have dialogic interaction in which they listen to the arguments of others, are stimulated to elaborate their own arguments, or seek out further evidence in support of their positions or to rebut others' arguments. Moreover, in whole-class discussions, the teacher can lead students to examine, evaluate, and share arguments and to view the problems involved from different perspectives. The teacher needs to model the strategy, provide examples of the desired outcome, and allow time for students to practise and adopt the discourse strategies as their own. Effective instructional materials need to consider both teaching and assessment of an innovative idea like argumentation. Many researchers have used Toulmin's Argument Pattern (TAP) for classical and extended arguments as the basis for evaluation and scoring rubrics. Osborne et al. (2004) emphasized identifying the elements of argument and their linguistic functions in a scientific or a socioscientific context when characterizing the dialogic arguments. TAP included analyses of students' use of data, claims, warrants, and backings to support their arguments and students' engaging in claiming, elaborating, reinforcing, or opposing the arguments of each other. However, the TAP rubric does not fully assess the quality of the arguments made in complex situations. Students' argumentation skills can be assessed by evaluating the development and qualities of their arguments, counterarguments, supportive arguments, and evidence (Brem & Rips, 2000; Mason & Scirica, 2006; Sandoval & Millwood, 2005). If students can make arguments and counterarguments at the same time or if they can justify their claims with different and acceptable reasons, it means they are able to show multiperspective thinking (Simonneaux, 2008) and have better argumentation abilities (Means & Voss, 1996). Zeidler, Osborne, Erduran, Simon & Monk (2003) regarded arguments with rebuttals as a fundamental element of better quality SSI arguments. The ability to construct rebuttals requires the individual to integrate an original and alternative theory into considering whether the claim or counterclaim is more valid (Kuhn, 1991). Hence, students can use rebuttals as their supportive arguments for a counterclaim when demonstrating a higher level of argumentation skills (Sadler & Donnelly, 2006). Sandoval & Millwood (2005) argued that students have to consider the appropriateness of the evidence they generate to justify the claim besides formulating different forms of evidence (e.g. numerical data, graphs, or photos). An appropriate piece of data that is justified by theoretical backing can directly support the claim and warrants, and it can further enhance an argument to be supported or rejected.


999

METHODS A mixed-methods case study was conducted on a single teacher and his grade 6 science classes at a suburban elementary school. The case study documented the university–school collaboration on professional development, unit design, implementation, and evaluation of a unit that emphasized argumentation and utilized embedded, explicit instruction through SSIs. The mixed qualitative and quantitative approaches produced a design with complementary strengths, which reflect the development of the problem space can provide stronger evidence for a conclusion, elaborate the results from different methods, and add insights that might be missed when only a single method is used (Johnson & Christensen, 2008). Case Teacher An experienced, male elementary school teacher, Fang-Yi (pseudonym), in Chiayi County, southwestern Taiwan, was invited to participate in this study. He has a master's degree in science education and 6 years of elementary science teaching experience, but no previous experience in teaching SSIs or argumentation. He engaged in this research knowing very little about argumentation and SSIs. In order to familiarize him with research related to instruction in SSIs and argumentation, the first author, some graduate students, and Fang-Yi met once every 2 or 3 weeks and engaged in readings about and discussions of the content and strategies of the proposed project. The meetings continued for 8 months. Fang-Yi constructed professional knowledge (content pedagogical knowledge) regarding the goals of the new grades 1–9 curriculum, SSIs, argumentation, and argumentation instruction through SSIs during the meetings. The research group spent significant time discussing how to choose an appropriate SSI for students and what teaching strategies should be adopted. The new curriculum encouraged and empowered teachers to design supplementary teaching materials. Fang-Yi had demonstrated a positive disposition toward designing new teaching materials and trying new teaching strategies that went beyond simply following the science textbook. In addition, he grouped his students into cohorts of six students, used a mixture of lecturing, questioning, and group discussion, and sought to adopt teaching strategies he had not used before, such as role play and debate. He was also prepared to integrate argumentation into his teaching as a component of teaching both the content and the process after becoming familiar with argumentation and its component skills.

1000


Participants Fang-Yi randomly chose two of his three grade 6 classes as the student participants. These two mixed-ability classes of 34 students each with various abilities and achievement patterns had 18 boys and 16 girls in each class. He had taught these students 1 year previously. Prior classroom observations revealed that over half of the students in each class regularly volunteered responses when Fang-Yi asked questions. He interacted well with the students. In order to explore the effect of the students' ability and achievement levels on the qualities and difficulties of the different students making arguments, we categorized the students in each class into three ordinal groups—high, middle, and low achievers—based on their average grades in mathematics, language, natural science, and social studies courses on two midterm examinations and one final examination during the previous semester. These three locally constructed examinations had their content validity explored and confirmed by experts (one by two science education professors and two by three experienced teachers) before they were administered. Students in each class were divided into thirds using cut scores that reflected the achievement patterns in Fang-Yi's classes. High achievers (11 students in each class) ranked in the upper one third of the mixedability classes, low achievers (11 students in each class) ranked in the lowest one third, while the remaining students (12 students in each class) were classified as middle achievers. The means (with standard deviations [SD] in parentheses) for the high, middle, and low groups based on the combination of the examinations were 90.57 (3.86), 80.50 (5.82), and 67.48 (9.89), respectively. The differences of the mean between the three groups were statistically significant, F = 60.9, p G 0.001. Pair-wise comparisons resulted in the means of the high achievers being significantly higher than the middle and low achievers (p G 0.001) and the mean of the middle achievers being significantly higher than the low achievers (p G 0.001). After the instruction, the first author interviewed a conveniently sized sample of 18 students. Six students were selected from each achievement group. Three students at each achievement level were from each class. Design of the Unit The case teacher chose a SSI involving the establishment of Ma-Guo National Park as the topic for the unit. This issue is controversial in that it


1001

involves conflicting arguments related to environmental and ecological conservation, economic development, land utility, and the rights of aboriginal peoples. Fang-Yi designed a unit plan to teach this SSI and presented his plan to the research group. The group deliberations identified strengths and challenges in the proposed unit. The main suggestions provided were: 1. The activities and teaching should lead students to understand both sides of the arguments about the issue. 2. The teaching needed to adapt teaching strategies that allowed students to easily express and exchange their opinions. 3. Teaching may sometimes need to require students to take sides on an issue to experience supporting an argument in which they do not believe. 4. The activities and teaching would need to be very encouraging and supportive of students in speaking their opinions without any hesitation, risk, or pressure. Based on these suggestions, the unit plan was revised and Fang-Yi prepared for implementation of the plan during the following academic year. The final unit consisted of two parts: the fictional issues portrayed in movies and the actual issues involved in the proposed Ma-Gou National Park. The first part was composed of three activities focused on helping students to construct argumentation knowledge. Each activity began with a portion of a film clip to motivate students. The clips were chosen from movies, such as Jurassic Park I, Time Machine, and I, Robot. These DVD movies were borrowed from the school library, which had secured release for such use in classrooms. Fang-Yi then described the nature of claims, warrants, rebuttals, and evidence; provided examples and time for wholeclass discussion to guide students; discussed the question; and practised making arguments. Each activity had one guided question with different foci and functions. The guided question for the first activity was: Do you agree or disagree with the establishment of Jurassic Park? Why? Fang-Yi focused on helping students make claims and warrants. The guided question in the second activity was: Do you agree or disagree with the development of a time machine? Why? The students were asked not only to generate claims and warrants but also to seek evidence backing their warrants. The guided question in the third activity was: Do you agree or disagree with the use of robots to replace human power? Why? In this activity, students were

1002


arranged into pairs, and one student in each pair was asked to refute the arguments constructed by the other student. A learning sheet was used during this period to supplement the lecture. Finally, Fang-Yi provided an opportunity for students to make oral presentations of their answers to the class, and he led a whole-class discussion of alternative answers and solutions. Part 1 of the unit involved 9 h of class time. The second part of the unit was devoted to SSI instruction involving an actual SSI and included five activities: living organisms and the environment, human beings and biological resources, functions of a national park, problems of a national park, and debate and argumentation. The foci of these activities were to help students construct the necessary background knowledge on the issue and to develop a sound argument about the establishment of Ma-Guo National Park. Fang-Yi adopted several teaching strategies for these activities, such as questioning, lecture, whole-class discussion, group discussion, library research, role play, and debate. In one activity, he organized students into four groups for a role play, which included a local government officer, landowners, tourists, aboriginal people, ecologists, and legislators who were focused on resolving the central issue. Three groups agreed to the establishment of Ma-Guo National Park and the others did not. Subsequently, the groups debated each other. Part 2 of the unit involved 8 h of class time. Implementation of the SSI Unit The SSI unit was implemented in a regular classroom with intact classes to evaluate its effectiveness and effects on a normal range of elementary students. The unit was taught in the same classroom to both classes by Fang-Yi using the normal school schedule. The first author videotaped 17 h of Fang-Yi's teaching in each class over the 6 weeks of the unit's instruction: with 3 h in weeks1 to 5 and 2 h in week6. In order to ensure that Fang-Yi's instructional treatment was appropriate and consistent, we met weekly to watch the videotapes together, discuss and reflect on FangYi's teaching, and confirm or adjust the teaching plan for the next week. Instrument The Argumentation Skills Questionnaire (ASQ) was designed by the authors to (a) explore students' abilities to make claims and warrants, (b) construct counterarguments and supportive arguments, and (c) formulate evidence in support of a claim. The first part was a case scenario that provided information about the social, political, economic, and cultural context of the issues surrounding the establishment of Ma-Guo National


1003

Park and the opinions of the interest groups involved in this issue. For instance, based on economic perspectives, the people agreeing with the establishment of Ma-Guo National Park stated that it would foster the government in investing much money to construct the area, attract many tourists to visit the park, and thus, improve the economic development of the local area. However, the people disagreeing with the establishment of Ma-Guo National Park stated that special business groups, instead of the aboriginal people, would reap the financial benefits the tourists bring; that aboriginal people living there would not share in these benefits; and that their quality of life would not be improved significantly. The second part consisted of four open-ended questions. Each question explored the students' responses to different components of an argument for or against the establishment of the park: 1. Do you agree or disagree with the establishment of Ma-Guo National Park in Taiwan? Please write down your ideas and reasons. (Assesses students' ability to make claims and warrants.) 2. If somebody disagreed with the opinions you expressed in the first question, (s)he might have some reasons. What might his/her reasons be? (Assesses students' ability to construct counterarguments.) 3. How would you convince somebody who disagreed with you if they had given such reasons in the second question? (Assesses students' ability to generate supportive arguments, including rebuttals.) 4. If you were asked to provide evidence to support your own opinions in questions 1 or 3, what might the evidence be? (Assesses students' ability to generate evidence.) Previous research found that, if students had not experienced the explicit argumentation instruction (Osborne et al., 2004) or they were not familiar with the background knowledge about the issue for which they wanted to construct arguments (Voss & van Dyke, 2001), then most students would not be able to construct arguments. Therefore, the pretest for the ASQ included two parts—the informational scenario and the probing questions about potential arguments and counterarguments. Pretest and Posttest. The pretest consisted of both parts of the ASQ. Ten minutes were provided for the students to read the scenario and 50 min to complete the second part of the questionnaire. The ASQ posttest included only the second part containing the four open-ended questions. The students were given 50 min to complete the posttest questionnaire.

1004


Interviews. Individual interview questions were based on the students' responses to the pretest and posttest. These interviews mainly served as a cross-examination to identify the answers that students provided and any problems they had in making the argumentation. The interview protocols contained questions that were designed to provide data to elaborate on the questionnaire responses (e.g. Could you elaborate on what you wrote down in question 1? Could you explain why you still hold a positive standpoint for question 2 that forms an opposite argument to question 1? What do you think the evidence is? Why do you think the answers you had for question 4 are evidence?). Individual interviews were audiotaped and transcribed verbatim into a word-processing document. Data Collection, Scoring Rubrics, and Analyses Issues of data and data interpretation are central to argumentation and to quality research designs. Therefore, this study attempted to improve on earlier argumentation research by using a mixed method that collected data across all participants (breadth) and rich follow-up information to better understand these data from a smaller set of participants (depth). Data were collected from the ASQ (N = 68) and individual interviews (n = 18) with students. Zeidler et al. (2003) pointed out that Toulmin's model for analyzing student arguments resulted in some problems about the criteria to identify the elements of an argument, such as the distinction between data and warrants or data and backup arguments. Therefore, the analyses of the ASQ items involved only some parts of the model dealing with the definitions of a claim, counterclaim, warrant, and rebuttal. We were also concerned about the quality and content of student arguments not fully reflected in the TAP checklist. The rubric in this study (Table 1) was based on argument scoring frameworks and procedures in recent studies (Mason & Scirica, 2006; Sadler & Donnelly, 2006; Walker & Zeidler, 2007; Wu & Tsai, 2007). The rubric scored student responses for claims, warrants, counterarguments, supportive arguments, and evidences. For example, in response to the first question, one student agreed with the establishment of the national park and his warrants were: “The National Park can protect animals from being hunted by humans, and the local economic development will be improved if many tourists come.” His perspectives were based primarily on ecological conservation and economic development. Accordingly, we gave two points for his two warrants in question 1. Compared with the responses in questions 1 and 2, the supportive

Q4—Evidence

Q3—Supportive arguments

Q2—Counterarguments (compare Q1)

0 1+ One point for each warrant 0 1+ One point for each elaborated warrant 1+ One point for each supplementary warrant 2+ Two points for each rebuttal 0

0 1 One point for a claim 1+1 One point for claim plus one point for each warrant 2+1 One point for each additional warrant

Scoring and explanation

No evidence or supplementary explanation Blank or Protecting endangered species is right. 1+ Valid evidence According to the reports of the other National Park, the numbers of Formosan One point for each evidence Sambar deer increased 15% since 2000.

I agree. The National Park can protect animals from being hunted by humans and the local economic development will be improved if many tourists come. No answer or invalid warrant Blank or I think it is wrong. One or more valid warrants The rights of aboriginal people will be influenced. They cannot hunt anymore. No answer or invalid warrant Blank or I support it should be established. Elaborated and valid warrant (compare Q1) The National Park can protect the animals, especially endangered species. A supplementary and valid warrant If the National Park can hire the aboriginal (compare Q1) people as an employee, it will improve their living status. Rebuttal to counterargument (compare Q2) We can make a law to protect their rights.

I agree. The National Park can protect animals from being hunted by humans.

An acceptable claim and a valid warrant

An acceptable claim and morethan one valid warrant

Blank or I do not know. I agree.

No answer or invalid warrant Only an acceptable claim and no warrant

Q1—Claims and warrants

Example

Answer category

Question

Scoring rubric for ASQ

TABLE 1


1005

1006


arguments in question 3 were categorized into three kinds of answers: an elaborated warrant, a supplementary warrant, or a rebuttal. An elaborated warrant meant the student refined his or her answers to question 1 in a supportive way. In a supplementary warrant, another reason based on a different perspective supported the student's claim in question 1. A rebuttal was a warrant used to reject a counterargument the student had made in question 2. The use of rebuttals is a requirement for higher quality and higher level arguments. As a result, we awarded two points for each rebuttal. In this study, we scored all valid student responses and disregarded whether their warrants or evidence were strong or weak. The analysis of individual interviews mainly focused on the problems that students had in constructing arguments after the instruction. Two researchers independently analyzed and classified unresolved problems. A constant comparison approach was applied to all responses to similar questions or issues flowing from the pretest and posttest questionnaire responses. Pretest and posttest descriptive statistics were calculated, and the differences were examined using the t test statistic. Students were classified into three ability groups based on overall academic performance (i.e. lowest third, n = 22; middle third, n = 24; and highest third, n = 22). Subsequently, separate multiple linear regression analyses were run to explore the effects of pretest scores and ability levels on posttest scores. A series of one-way, univariate ANOVA were run to explore ability-level differences.

RESULTS The results are organized to reflect the order and focus of the research questions and the mixed-method design of the study. Therefore, the general changes in argumentation performance for all students are discussed first, followed by student characteristic effects on performance, and finally the qualitative insights into argumentation and students' problems in argumentation are presented. Effects of Instruction The descriptive statistics of students' argumentation indicate that their performance on the pretest was not generally good before the instruction (M = 4.89, SD = 1.57). Even after the instruction, the students did not demonstrate well-developed skills in making arguments, counterarguments, supportive arguments, and in the evidence on the posttest (M =

1007


5.21, SD = 2.12). The general changes in argumentation performance (pretest–posttest gain = 0.31) for all students were not obvious after the intervention. The simple univariate effect of the instructional treatment was explored by comparing total pretest and posttest ASQ scores for all students using a paired t test. The results of the t test (t = 1.10, df = 67) revealed no overall statistically significant (p 9 0.05) difference between the pretest and posttest scores. Initially, it seemed that the students' argumentation skills were not significantly affected by the treatment. However, this simple picture became substantially more complex after further consideration of the effects of the pretest scores and student ability levels in argumentation skills. Pretest Scores and Ability as Predictors of Argumentation Skills In order to explore the combined and relative effects of pretest scores (X1) and ability levels/achievement level (X2) on argumentation skills (Y), the pretest scores and ability levels were entered into a series of separate multiple linear regression analyses to predict posttest scores: Y ¼ Constant þ alpha X1 þ beta X2 : Five separate regression analyses were performed: one each for the four items on the ASQ and a fifth for the total scores (Table 2). The results reveal that pretest scores were significant (p G 0.01) predictors of the ability to formulate claims and warrants (question 1), but not predictive (p 9 0.05) of the ability to generate counterarguments (question 2) and supportive arguments (question 3) or to construct evidence for a

TABLE 2 Predictors of argumentation skills; results of multiple linear regression analysis Predictor

Q1

Q2

Q3

Q4

Total

Pretest (X1) Ability level (X2) Intercept (I)

[0.32] (6.52*) [0.06] (0.12 NS) 1.48

[0.11] (0.65 NS) [−0.29] (6.26*) 0.93

[0.04] (0.056 NS) [−0.45] (5.74*) 1.62

[−0.19] (1.29 NS) [−0.25] (5.92*) 0.32

[0.18] (1.25 NS) [−0.90] (5.02*) 4.34

Regression coefficients for variables are in square brackets. Multiple linear regression F values are in parentheses *p G 0.01; NS, p 9 0.05

1008


claim (question 4). Furthermore, pretest scores were not predictive (p 9 0.05) of total posttest scores. In contrast, ability levels were a significant predictor of argumentation skills (p G 0.01) on all questions (except question 1) and on total scores. These results suggest that ability level, as indicated by established achievement, was a principal determinant of learning argumentation skills in this study. Effect of Ability Level on Argumentation Skills To document the effects of ability level on argumentation skills, a series of separate, one-way univariate ANNOVAs were run, using the pretest and posttest scores as the dependent variables (Table 3). The results revealed no significant differences (p 9 0.05) among groups on the pretest, but significant differences (p G 0.01) on four of the five posttest measures. Except for question 1 on the posttest, the mean scores of the high-ability students were the highest and the mean scores of the low-ability students were the lowest. Paired Tukey–Kramer honestly significant differences post hoc tests were conducted to explore the source of the differences between student groups. In all cases where differences were found in the posttest measures, the high achievers differed significantly (p G 0.05) from low achievers; no other comparisons were significant. These results suggest that explicit instruction on SSIs and argumentation skills seems to TABLE 3 Effects of ability level on argumentation skills: means, SD, and F value Ability level

Question Q1 Q2 Q3 Q4 Total **p G 0.01

Pretest Posttest Pretest Posttest Pretest Posttest Pretest Posttest Pretest Posttest

Low (n=22)

Middle (n=24)

High (n=22)

M

SD

M

M

SD

F value

2.14 2.23 0.82 0.73 1.36 1.23 0.14 0.05 4.41 4.23

0.35 0.69 0.40 0.55 0.79 0.97 0.35 0.21 1.01 1.99

2.32 2.18 1.14 1.36 2.00 2.32 0.27 0.50 5.55 6.36

0.48 0.59 0.56 0.58 1.02 1.09 0.46 0.60 1.87 2.01

0.69 0.03 2.16 7.50** 3.61 6.75** 2.53 5.49** 3.22 6.71**

2.29 2.21 0.96 1.00 1.46 1.50 0.04 0.33 4.75 5.04

SD 0.75 0.51 0.55 0.51 0.72 0.21 0.20 0.48 1.57 1.85


1009

have a differential effect on students by ability level, with high-ability students benefiting most by the intervention. Question 1: Claims and Warrants. Students' ability to make claims and warrants results revealed that virtually all students could successfully make a claim and at least one warrant on the pretest and posttest. However, there was no statistically significant difference (p 9 0.05) between the three groups before and after the treatment. Question 2: Counterarguments. Students' ability to construct counterarguments results revealed that only high achievers could generate more than one warrant to support their counterarguments. Although no significant differences were found among the groups prior to instruction, statistically significant differences (p G 0.01) between the groups emerged on the number of warrants after the instruction. This result indicates that the gains made by the high-ability (0.22), middle-ability (0.04), and lowability (−0.09) students produced a significant main effect on the ability to generate warrants to support counterarguments. Compared with the contents of the students' arguments and counterarguments on the pretest, most students made improvements in elaborating their arguments or counterarguments after instruction. High achievers not only could refine their arguments and counterarguments but could also use more perspectives to make warrants in support of their claims or counterarguments. In contrast, middle achievers showed some improvement, but were still limited as fewer perspectives were shown, while low achievers demonstrated a decrease in argumentation skills. It seems that most high-ability students were able to generate better arguments or counterarguments even though they were not provided with the contextual information on the posttest. Question 3: Supportive Arguments/Rebuttals. The results related to students' ability to generate supportive arguments, which include rebuttals, produced a developmental distribution for ability groups on the pretest and posttest. Most low achievers tended to use one elaborated warrant or one supplementary warrant to support their claims. However, high achievers were more inclined to make one rebuttal or one rebuttal with one elaborated warrant or supplementary warrant for their claims. Some high achievers even generated two rebuttals to strengthen their claims. More high achievers used rebuttals to strengthen their claims after the instruction. Further analysis of the responses revealed that the mean of scores for constructing only rebuttals disaggregated from other elements between

1010


the three groups increased following the instruction from 0.72 to 0.82 (low achievers), from 0.90 to 0.92 (middle achievers), and from 1.10 to 2.00 (high achievers). Although the score differences between pretest and posttest results were not large, significant differences of F = 5.70, p G 0.01 were found between the groups favoring the high achievers on the posttest. These findings suggest that high-ability students made more progress in constructing rebuttals after the instruction. Question 4: Evidence. Students' ability to generate evidence was the most difficult dimension of argumentation measured. The total number of statements of evidence made on the pretest was ten, with high achievers constructing more than half of them. The total number of statements of evidence made on the posttest was 20, in which only one statement of evidence was made by a low achiever. Those who could provide evidence usually described a fact or used a concrete example on the pretest. However, several students included findings of scientific surveys found on websites to serve as evidence on the posttest. One high achiever adopted quantitative data shown on a website in reference to another national park in Taiwan to support his warrant about the proposed park in the focus SSI for this study. This evidence summarized the data on the increase in endangered animals since national parks were launched in Taiwan. In summary, high-ability students could construct more warrants, use more rebuttals, and generate more statements of evidence than low-ability students did after the instruction. These findings suggest that high-ability students benefitted the most and that low-ability students benefitted the least from the instructional treatment in this study. Persistent Problems in Argumentation An analysis of the interview responses revealed continued problems in constructing sound arguments after instruction, especially among low and middle achievers. Among these problems were misunderstanding how to construct a counterargument, failing to cite fully the content of evidentiary statements, and misusing elaborated or supplementary warrants as evidence. For instance, one low achiever, who supported the establishment of the national park, offered as his warrant, “The national park can protect animals and plants.” However, as a counterargument, he suggested, “It can protect Taiwan's red cypresses. They are a very unique species in the world.” In fact, this was an elaborated warrant backing his claim rather than a counterargument. This student told the researcher in the post-instruction interview, “I did not understand what a


1011

counterargument was. I thought it just extended what I thought in question 1.” Four low achievers (5.8% of the total) had this problem on the posttest. Eleven students (16.2% of the total) had problems identifying the source and failed to describe the content of evidentiary statements. For example, one middle achiever agreed to the establishment of Ma-Guo National Park, and her warrant was, “Many wild animals can live there without human interruptions.” Her evidence was, “I got it from national park websites.” When the researcher interviewed her after the instruction, she said, “I thought it was enough. Actually, I did not know what the evidence was, even though my teacher had taught me before I wrote my answers down.” Thirty-five students (51.5% of the total) misused elaborated or supplementary warrants as evidence. It seems that many students did not understand the meaning or function of evidence after instruction. Even the high achievers still did not completely understand. For instance, one high achiever who agreed to establish Ma-Guo National Park offered as her warrant, “The establishment of a new national park will bring some money for Taiwan. Many foreigners will come to Taiwan to visit.” She suggested as her evidence, “There are many precious and endemic species in Ma-Guo National Park. I think many foreigners would like to visit Taiwan to watch them.” Clearly, her evidence was simply an elaborated warrant that did not describe any data to support her claim. In the interview, she told the researcher, “I thought it would be a fact and it was evidence.” It appears that she did not understand that scientists make observations, conduct surveys, or perform experiments to justify their claims before a fact becomes evidence.

DISCUSSION AND IMPLICATIONS This study attempted to incorporate assessment techniques that would move beyond simply a checklist of elements in an argument toward assessing the quality of an argument and counterargument about a relevant SSI for elementary school students in Taiwan. Furthermore, this study attempted to evaluate an instructional unit and the influences of learner characteristics in the instructional context. Constructing Argumentation Skills The results of this study showed that the development of argumentation skills of elementary school students was limited if they were not explicitly

1012


taught. Although some Taiwanese elementary school students were capable of constructing arguments, counterarguments, supportive arguments, and evidence before the instruction, the quality of their arguments was found to be substantially below the standards suggested in the new grades 1–9 curriculum. Previous research has shown that middle school, high school, and undergraduate students can be taught to construct better arguments when the teacher explicitly focuses on argumentation skills and provides opportunities for practising these skills (e.g. Osborne et al., 2004; von Aufschnaiter et al., 2008). The results of the current study concurred with these studies and suggest that, if the elementary school teacher knows how to choose a suitable SSI, has mastered the basics of argumentation, and understands the appropriate background knowledge of the focus issue, it is feasible for elementary school students to learn argumentation skills through SSI instruction. Moreover, the results of the study further suggest that explicit instruction in argumentation skills could have positive effects on student learning, particularly on high-achieving elementary school students. During instruction, the high achievers appeared to learn more about both the target issue and how to make appropriate arguments; therefore, they could master more argumentation skills about the specific SSI. The low achievers lagged in their ability to master skills in constructing arguments, counterarguments, and rebuttals and in generating supportive evidence for a claim, which could be partially due to their lack of conceptual knowledge about the target issue. This illustrates that individual differences in learning argumentation skills and learning the knowledge about the issue cannot be ignored. Zohar & Dori (2003) found that students with high and low academic achievement levels improved their higher-order thinking (including argumentation skills) scores after instruction. They provided students with longer teaching duration (20– 39 h) and more chances to analyze data, practise solving problems, and posing questions through case studies, while we provided only 17 h and adopted relatively fewer teaching strategies and less practice time. The low achievers appeared to need more instruction on the relevant concepts in the SSI, scaffolding that is explicit, and practice time to learn and master argumentation skills. Meanwhile, argumentation is seen not only as a form of discourse but also as a language-based activity. In these kinds of activities, students are involved in using language to convey what they think, whether through talking or writing. Rivard & Straw (2000) suggested that low achievers are better able to learn science when they have an opportunity to engage in classroom talk, whereas high achievers benefit more from writing than


1013

talking. Given the fact that our intervention focused primarily on writing, this may explain why high achievers made greater improvement in their ability in argumentation than did low achievers. We also suggest that science teachers actively encourage low achievers to express their arguments in class using oral approaches, such as direct questioning strategies and small group discussions, to overcome their lack of confidence and to establish a foundation for argumentation. Nearly half of the students, including some high achievers, could not generate rebuttals against counterarguments after instruction. Rebuttal is a process that uses higher-order thinking skills, and it is a more difficult cognitive task for most students. The results from the current study are consistent with those of Wu & Tsai (2007) who found that only 38% of high school students could generate rebuttals. Kuhn (1991) found that students needed to take an argument and an opposing argument into consideration before constructing rebuttals to counterarguments. Therefore, it might be unreasonable to expect grade 6 students to fully develop the argumentation skills needed to consider arguments, counterarguments, supportive arguments, and the consistency between them at the same time. The cognitive demands may be beyond the capacity and experience of grade 6 students, especially the low achievers (Stanovich, 1999). Most students in this study could not generate a correct statement of evidence after instruction, especially low achievers. Sandoval & Millwood (2005) argued that to coordinate claims and evidence is not a simple cognitive skill; there is a need for students to understand the content of the evidence, the interpretation of evidence in the SSI, and its relationship with claims. Levinson (2006a) stated that most evidence for an SSI is not located within well-established theoretical frameworks in school science; students have to find the evidence from external sources. In another persistent problem, Brem & Rips (2000) found that college students often used explanations as a substitute for evidence. Similarly, in our study, we found that students constructed an elaborated or supplementary warrant instead of offering evidence. In order to improve students' understanding of evidence, to learn to obtain evidence from all kinds of information sources, and to generate exact evidence, science teachers must fully explain the definition and give more examples of evidence in SSI instruction. Possible Cultural Influences On a final note, we suggest that difficulty in learning argumentation skills may be connected to cultural factors, including several that are deeply

1014


embedded in the social, philosophical, and educational contexts of Taiwanese society. This is a research area worthy of future exploration. Teaching and learning in Taiwanese elementary schools is deeply influenced by a Confucian theoretical framework that prizes respect for teachers and what they teach. Many science teachers are not accustomed to giving students opportunities to justify their knowledge claims, and most students are not used to constructing arguments in the classroom. The classroom culture in this study is very similar to that in Hong Kong where Chan & Watkins (1994) and Thomas (2006) showed that the classroom environment is heavily dominated by the teacher and is embedded in a Confucian-heritage culture. Chinese students are instructed to be obedient to their elders because of the concept of filial piety (Gow, Balla, Kember & Hau, 1996). As a result, Chinese students show much respect for their teachers and refrain from challenging their authority. Jiménez-Aleixandre et al. (2000) indicated that developing the ability to make arguments is not a matter only related to the curriculum design or teaching strategies, but is more connected to a learning environment specially characterized by the perspective of teaching or learning science. Taiwanese teachers must create a nonthreatening environment and relinquish the teacher-centered environment in favor of one that encourages students to talk or write about their views on potentially controversial topics. As students have greater opportunities to generate, challenge, justify, and defend a claim, we can expect that they will become more aware of their own voices and listen more to those of others. ACKNOWLEDGEMENTS The authors express their thanks to the National Science Council in Taiwan for their support and grants for this study (NSC95-2522-S-415001-MY3). We also wish to thank the International Journal of Science and Mathematics Education for its support, Dr. Larry Yore for his mentoring, and Mrs. Sharyl Yore for the technical editing assistance. REFERENCES Aikenhead, G. S. (2005). Science-based occupations and the science curriculum: Concepts of evidence. Science Education, 89, 242–275. Anderson, R. C., Chinn, C., Chang, J., Waggoner, M., & Yi, H. (1997). On the logical integrity of children's arguments. Cognition and Instruction, 15, 135–167.


1015

Brem, S. K., & Rips, L. J. (2000). Explanation and evidence in informal argument. Cognitive Science, 24, 573–604. Chan, G. Y.-Y., & Watkins, D. (1994). Classroom environment and approaches to learning: An investigation of the actual and preferred perceptions of Hong Kong secondary school students. Instructional Science, 22, 233–246. Dori, Y. J., Tal, R. T., & Tsaushu, M. (2003). Teaching biotechnology through case studies—Can we improve higher order thinking skills of nonscience majors? Science Education, 87, 767–793. Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84, 287–312. Duschl, R. A., & Osborne, J. (2002). Supporting and promoting argumentation discourse in science education. Studies in Science Education, 38, 39–72. Gow, L., Balla, J., Kember, D., & Hau, K. T. (1996). The learning approaches of Chinese people: A function of socialization processes and the context of learning. In M. H. Bound (Ed.), The handbook of Chinese psychology (pp. 109–123). Hong Kong: Oxford University. Hand, B., Prain, V., & Yore, L. D. (2001). Sequential writings tasks' influence on science learning. In P. Tynjala, L. Mason, & K. Lonka (Eds.), Writing as a learning tool: Integrating theory and practice (pp. 105–129). Dordrecht, The Netherlands: Kluwer. Jiménez-Aleixandre, M. P., Rodriguez, A. B., & Duschl, R. A. (2000). ‘Doing the lesson’ or ‘doing science’: Argumentation in high school genetics. Science Education, 84, 757–792. Johnson, R. B., & Christensen, L. (2008). Educational research: Quantitative, qualitative and mixed approaches. London, UK: Sage. Kolsto, S. D. (2001). Scientific literacy for citizenship: Tools for dealing with the science dimension of controversial socioscientific issues. Science Education, 85, 291–310. Kuhn, D. (1991). The skills of argument. Cambridge, UK: Cambridge University Press. Kuhn, D., & Udell, W. (2003). The development of argument skills. Child Development, 74, 1245–1260. Lawson, A. E. (2003). The nature and development of hypothetico-predictive argumentation with implications for science teaching. International Journal of Science Education, 25, 1387–1408. Levinson, R. (2006a). Teachers' perceptions of the role of evidence in teaching controversial socioscientific issues. The Curriculum Journal, 17, 247–262. Levinson, R. (2006b). Towards a theoretical framework for teaching controversial socioscientific issues. International Journal of Science Education, 28, 1201–1224. Maloney, J., & Simon, S. (2006). Mapping children's discussions of evidence in science to assess collaboration and argumentation. International Journal of Science Education, 28, 1817–1841. Mason, L., & Scirica, F. (2006). Prediction of students' argumentation skills about controversial topics by epistemological understanding. Learning and Instruction, 16, 492–509. Means, M. L., & Voss, J. F. (1996). Who reasons well? Two studies of informal reasoning among children of different grade, ability, and knowledge levels. Cognition and Instruction, 14, 139–178. Millar, R., & Osborne, J. (1999). Beyond 2000. London, UK: King's College London. Ministry of Education of ROC (2001). The grades 1–9 science and technology curriculum guidelines and competence indicators. Retrieved from http://dl2k.dc2es.tnc.edu.tw/ capability/ [In Chinese]

1016


Naylor, S., Keogh, B., & Downing, B. (2007). Argumentation and primary science. Research in Science Education, 37, 17–39. Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41, 994–1020. Oulton, C., Dillon, F., & Grace, M. (2004). Reconceptualizing the teaching of controversial issues. International Journal of Science and Education, 26, 411–423. Rivard, L. P. (2004). Are language-based activities in science effective for all students, including low achievers? Science Education, 88, 420–442. Rivard, L. P., & Straw, S. B. (2000). The effect of talk and writing on learning science: An exploratory study. Science Education, 84, 566–593. Sadler, T. D. (2004). Informal reasoning regarding socioscientific issues: A critical review of research. Journal of Research in Science Teaching, 41, 513–536. Sadler, T. D., & Donnelly, L. A. (2006). Socioscientific argumentation: The effects of content knowledge and morality. International Journal Science Education, 28, 1463– 1488. Sadler, T. D., & Zeidler, D. L. (2005). Patterns of informal reasoning in the context of socioscientific decision making. Journal of Research in Science Teaching, 42, 112–138. Sandoval, W. A., & Millwood, K. A. (2005). The quality of students' use of evidence in written scientific explanations. Cognition and Instruction, 23, 23–55. Simonneaux, L. (2001). Role-play or debate to promote students' argumentation and justification on an issue in animal transgenesis. International Journal of Science Education, 23, 903–927. Simonneaux, L. (2008). Argumentation in socioscientific contexts. In S. Erduran & M. P. Jiménez-Aleixandre (Eds.), Argumentation in science education: Perspectives from classroom-based research (pp. 179–199). Dordrecht, The Netherlands: Springer. Stanovich, K. E. (1999). Who is rational? Studies in individual differences in reasoning. Mahwah, NJ: Lawrence Erlbaum. Thomas, G. P. (2006). An investigation of the metacognitive orientation of Confucianheritage culture and non-Confucian-heritage culture science classroom learning environments in Hong Kong. Research in Science Education, 36, 85–109. Toulmin, S. (1958). The uses of argument. Cambridge, UK: Cambridge University Press. von Aufschnaiter, Erduran, S., Osborne, J., & Simon, S. (2008). Arguing to learn and learning to argue: Case studies of how students' argumentation relates to their scientific knowledge. Journal of Research in Science Teaching, 45, 101–131 Voss, J. F., & van Dyke, J. A. (2001). Argumentation in psychology: Background comments. Discourse Processes, 32, 89–111. Walker, K. A., & Zeidler, D. L. (2007). Promoting discourse about socioscientific issues through scaffolded inquiry. International Journal of Science Education, 11, 1387–1410. Wray, D., & Lewis, M. (1997). Extending literacy: Children reading and writing nonfiction. London, UK: Routledge. Wu, Y-T., & Tsai, C-C. (2007). High school students' informal reasoning on a socioscientific issue: Qualitative and quantitative analyses. International Journal of Science Education, 29, 1163–1187. Yerrick, R. K. (2000). Lower track science students' argumentation and open inquiry instruction. Journal of Research in Science Education, 37, 807–838. Yore, L. D., Bisanz, G. L., & Hand, B. (2003). Examining the literacy component of science literacy: 25 years of language arts and science research. International Journal of Science Education, 25, 689–725.


1017

Yore, L. D., Pimm, D., & Tuan, H.-L. (2007). The literacy component of mathematical and scientific literacy. International Journal of Science and Mathematics Education, 5, 559–589. Zeidler, D. L., Osborne, J., Erduran, S., Simon, S., & Monk, M. (2003). The role of argument during discourse about socioscientific issues. In D. L. Zeidler (Ed.), The role of moral reasoning on socioscientific issues and discourse in science education (pp. 97– 116). Dordrecht, The Netherlands: Kluwer. Zeidler, D. L., Sadler, T. D., Simmons, M. L., & Howes, E. V. (2005). Beyond STS: A research-based framework for socioscientific issues education. Science Education, 89, 357–377. Zohar, A., & Dori, Y. J. (2003). Higher order thinking skills and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences, 12, 145–181. Zohar, A., & Nemet, F. (2002). Fostering students' knowledge and argumentation skills through dilemmas in human genetics. Journal of Research in Science Teaching, 39, 35– 62. Shu-Sheng Lin Graduate Institute of Science Education, National Chiayi University, Chiayi 600, Taiwan R.O.C. E-mail: [email protected] Joel J. Mintzes Biological Sciences and Science Education, College of Natural Sciences, California State University, Chico, CA 95929, USA E-mail: [email protected]