Argumentation Skills as Prerequisites for ... - Semantic Scholar

Educational Research and Evaluation Vol. 11, No. 4, August 2005, pp. 365 – 384

Argumentation Skills as Prerequisites for Collaborative Learning among Finnish, French, and English Secondary School Students Miika Marttunena*, Leena Laurinena, Lia Litosselitib, and Kristine Lundc a

Department of Education, University of Jyva¨skyla¨, Finland, bDepartment of Language and Communication Science, City University, UK, and cGRIC, CNRS, University of Lyon 2, France

(Received 4 May 2004; accepted 10 February 2005)

Argumentation skills of secondary school students were evaluated in Finland (n = 290), France (n = 54), and England (n = 41). The data were collected from 4 tasks comprising 7 variables. The results indicated that most of the students had correctly justified arguments and conclusions, and composed clear claims and relevant arguments. However, many students had difficulties in recognising the main claim and arguments for it in an expository text, and in commenting analytically on an argumentative text. Thus the students possessed the prerequisites for argumentative reasoning and writing but need further practice in analytical and critical reading.

Introduction The citizens of today’s network society are increasingly required to examine critically current societal issues from different points of view, to form reasoned opinions and to engage in public debate around them. Today’s students also need to critically analyse information they encounter in various teaching materials, textbooks, and Internet sources. One important task of secondary school education is thus to teach the argumentation and critical thinking skills students need to be able to participate *Corresponding author. University of Jyva¨skyla¨, Department of Education, Seminaarinkatu 15, P.O. Box 35, Building Educa (D), FIN-40014, University of Jyva¨skyla¨, Finland. E-mail: [email protected].fi ISSN 1380-3611 (print)/ISSN 1744-4187 (online)/05/040365–20 ª 2005 Taylor & Francis DOI: 10.1080/13803610500110588

366

M. Marttunen et al.

successfully in debates on societal questions and to critically examine different information sources. Argumentation is also an essential element in collaborative learning. The aim of collaborative discussion is not only to promote the reconstruction of participants’ own knowledge structures (Bereiter & Scardamalia, 1999) but to guide them to actively construct new knowledge structures together (Dillenbourg, 1999). During collaborative argumentation the participants are, critically but constructively, striving to get to the core of the issues in question by jointly examining different points of view and the arguments for and against each others’ positions. Unfortunately, secondary school students’ ability to engage in critical argumentative discussions is not self-evident. This study examines the level of argumentation skills among secondary school students in order to clarify how well prepared students are for active collaboration in argumentative learning groups. Collaborative Argumentation as a Means for Learning When engaging in critical argumentative discussion, one has to be able to present well-grounded arguments for one’s opinions, put forward counterarguments and refute criticism by others (Van Eemeren & Grootendorst, 1999). Successful engagement in argumentative dialogue thus necessitates the ability to evaluate, compare, and contrast the strengths and weaknesses of diverse standpoints. Recent research (Biemans, Deel, & Simons, 2001) has shown that more successful readers compared and contrasted their prior knowledge with new information encountered when studying expository texts. This led to good learning results. In contrast, less successful readers had difficulties in analysing the texts, and consequently their learning results were poorer. Thus, the teaching of cognitive skills and strategies, such as argumentation, that promote students’ ability to compare and contrast their earlier knowledge with new information in an adequate way should be incorporated into the school curriculum. There is much research to suggest that interaction and collaboration have positive effects on learning. Studies in the Piagetian research tradition find sociocognitive conflict an essential element in a person’s learning. Such conflict arises when one notices a difference between one’s prior knowledge and one’s new knowledge obtained through discussions with other people (e.g., Webb, 1995). This knowledge discrepancy usually triggers the need to solve the conflict—to find new information in order to explain the different conceptions and to restore one’s mental balance. Howe and Tolmie (1999) compared learning results in groups in which the students had either very similar or distinctly different prior knowledge of the issues handled. They found the best learning results were attained among those groups in which the students’ prior knowledge differed the most. Sociocognitive conflicts are typical in argumentative discussions. When students contrast each others’ statements, their argumentation often reveals conflicting understandings on issues. Students can solve the conflict by exploring the issue together and refining their earlier conceptions to achieve more elaborated and better

Argumentation Skills for Collaborative Learning

367

reasoned knowledge. Argumentative discussion also helps students to better understand how issues are typically supported or attacked (see Andriessen, Baker, & Suthers, 2003). Stein and Bernas (1999) found that people, regardless of age, recognise twice as many arguments in favour of their own opinion compared to arguments against it. An essential pedagogical aim of argumentation is met when students are engaged in counterargumentation, thereby familiarising themselves also with standpoints contrary to their own opinions. Studies in the Vygotskian research tradition emphasise learning as mainly a result of interactive and social processes. Mercer (2000, p. 141) emphasises the importance of collaborative interaction based on teachers’ and students’ common knowledge and aims with respect to a specific situation. This kind of interactive space he calls an intermental development zone. During interaction in the intermental development zone, both teacher and student strive towards a common goal by negotiating and putting forward relevant arguments. In the discussions the student benefits from the larger knowledge base and richer experience of the teacher. The findings by Mercer, Wegerif, and Dawes (1999) on student-student interactions support the importance of exploratory talk in learning. During exploratory talk students explain their conceptions, put forward their own arguments, and challenge and counterchallenge arguments presented by other students. Dillenbourg (1999) emphasises that collaborative interaction needs to allow for negotiation between students. To be successful in negotiation presupposes the ability to give reasons for one’s own view as well as consider the views and arguments of others. An essential feature of exploratory talk is constructive criticism based on negotiation: Although students question each other’s thoughts they also offer alternative hypotheses for further examination. Exploratory talk brings argumentation into the foreground: Students become collectively responsible for the knowledge they construct together. Argumentative Discussion Culture and Curricula in Finland, France, and England Secondary school students’ ability to engage in collaborative argumentation is dependent on the educational system, curricula, and the discussion culture prevalent in the country they live in. There are differences between Finland, France, and England in this respect. The discussion culture in Finland can be characterised as consensus oriented, while in France, as in many other European countries, arguing is a sign of closeness and agreement is deemed boring; to keep things interesting, you have to disagree—preferably with great animation (e.g., Tannen, 1998, pp. 209 – 210). For example, the French major newspapers habitually pit one side of their audience (public opinion) against the other (ruling elite) in their editorials (Le, 2004). In Finland, people prefer to obtain a diversified picture on an issue through exploring it from many different points of view. Finnish university students have been found even to avoid critical argumentative discussions (Steffensen, 1996). English and French students, by contrast, are much more used to engaging in verbal debates the aim of which is to win the dispute by good arguments and cogent criticism of the opponent’s opinions.

368

M. Marttunen et al.

Mauranen (1993) found that in a Finnish university seminar teachers concentrate particularly on providing students with feedback on their written papers, while in an English seminar the focus is on encouraging students to engage in critical discussion among themselves and with the teacher. In a Finnish seminar disagreements, if anything, are avoided since criticism can be easily experienced as insulting. The curricula in Finland, France, and England differ from each other in how argumentation is emphasised in them. In Finland, secondary schools should, along with the traditional ‘‘teaching to know’’, guide students to examine knowledge critically and to apply it in solving various problems in various situations (see National Board of Education, 1994). The need for teaching students critical thinking and argumentation skills is emphasised at many points in the curriculum, both at the national and municipal levels. Furthermore, the Finnish secondary school system is course based, including obligatory, optional, and specialisation courses. This system provides individual schools with good opportunities to allocate their teaching resources themselves and to concentrate on their own areas of specialisation. Thus the cornerstone of the Finnish secondary school curriculum is flexibility, which also allows schools to organise crossdisciplinary courses that may include studies in argumentation and critical thinking, provided that expert teachers in this specific area are available. In the Finnish language, argumentation skills are practised in obligatory courses and could be studied in specialised courses as well. Obligatory courses in Finnish language include exercises in oral debate, text analysis, and argumentative writing. The obligatory courses also include work on different aspects of the argumentative power of language in various areas of social life. In addition to Finnish language, the other subjects relevant for practising argumentation are Foreign Languages, History and Social Studies, Physics, Chemistry, Biology, Geography, Religion (especially an obligatory course on Ethics), Philosophy, and Psychology. In these subjects, argumentation can be seen as a method of learning and in some cases even an indirect object of learning. As an indirect object it is usually expressed in the curriculum by using the words ‘‘critical thinking’’ (see National Board of Education, 1994). At the level of the French high school (Lyceé), students may choose between a general technological teaching curriculum and a vocational teaching curriculum where different occupations are learned. If students choose the general teaching track, a great variety of teaching on argumentation can be found, mainly in French language, in Philosophy, and in Civic Education. The French national curricula underwent a major reform in 2000 (see http://www.education.gouv.fr/see/ default.htm). Firstly, in the French language class, the focus includes understanding the great intellectual debates, training the citizen in order to structure thinking, expressing oneself, reflecting on opinions, and developing the capacity to argue (see Ministe´re e´ducation nationale, 2000). Secondly, one of Philosophy’s stated objectives is to train students in argumentation, with the emphasis on both written and spoken argumentative discourse. Finally in Civic Education, preparation for debate, management of debate and oral or written synthesis of debate are carried out on such themes as the foundations of social life and the functioning of political institutions.


369

According to official national curricula as well as analyses of text books (Baylac, 2000; Laborde-Milaa, 2000), the following methods of study are intended to support the learning of argumentation in the French school system. In the French language class, differences between demonstration, convincing, and persuading are studied and debating exercises are organised. Forms and practices related to deliberation are explored; students are asked to compare several opinions in order to form their own, and the effects of argumentation on the hearer are discussed. In Philosophy, analyses of philosophical texts are carried out and students learn to compose the different phases of a dissertation. Teachers are also asked to organise situations in which students perform argumentative discourse. This consists of problem definition, impartial exposition of a thesis, elaboration of concepts, confrontation of theses aiming at the elaboration of a response to the problem posed, evaluation of an argument, confrontation with examples, insertion in a critical dialogue, argumentative progression and finally assessment of the results of an argumentative line of reasoning. In Civic Education, students are asked to prepare for debate by carrying out documentary research, taking notes, and performing critical analyses of the subject material. Debate is carried out by a variety of methods including varying group formation and engaging different types of moderators. Students are then asked to produce an oral or written synthesis of the debate. In general, debate and argumentation hold an important place in French society, evident in the commonly used expression ‘‘culture of debate’’. Indeed, during the last biennial on education and training organised in 2004 by the Institute of National Research on Pedagogy (see http://www.inrp.fr/biennale/Biennale.htm), one of the 15 chosen themes was ‘‘Towards a culture of debate: Citizenry and democracy’’, where close to 40 different papers were presented. Such issues are therefore important for teachers and teacher educators. In secondary schools in England, there is limited provision for argumentation and debating skills within subjects such as Mathematics, Science, History, Philosophy, Religious Education, and Special Educational Needs, while there is no explicit provision in Modern Languages and Geography. Provision for argumentation and debating skills is more evident in the teaching of English, and the range of aims includes teaching the students how to explore, hypothesise, debate, and analyse what is being said during discussions, and how to take different roles in groups, such as organising or leading the discussion, supporting others, and enabling focused talk (Qualifications and Curriculum Authority [QCA], 2005a, 2005b). In addition, some provision for argumentation and debating skills in the curriculum can be seen in the promotion of Thinking Skills and in the introduction for the first time in 2002 of the subject of Citizenship. Thinking skills are subdivided into information processing, reasoning, enquiry, creativity, and evaluation, and they emphasise qualities such as giving reasons for opinions, making informed judgements, hypothesising, developing, and applying evaluation criteria. Citizenship education aims to develop students’ understanding of their roles and responsibilities as citizens in a modern democracy, and prepare them for dealing with difficult moral and social questions and engaging with economic, social, and cultural change.

370

M. Marttunen et al.

Despite the above, in most subjects, the development of argument and debating skills in English schools is not motivated by National Curriculum requirements. It is also introduced on a fairly random basis, sometimes by individual teachers, and sometimes through extracurricular activities, such as occasional debating societies or after-school clubs. Backley, Saxton, and Sillince (1999) report that there is an assumption that responsibility for teaching argumentation lies primarily with the teachers of English, who then rely on a random set of resources for teaching and practising argumentation skills (e.g., exercises taken from English Studies textbooks, and self-prepared material such as newspaper extracts and advertising media). Review of Research Previous research has shown that the level of argumentation is associated at least with intelligence (Perkins, 1985), gift (Voss & Means, 1991), age (McCann, 1989), gender (Litosseliti, 1999), level of education (King, Wood, & Mines, 1990), level and type of facilitation by the teacher (Ravenscroft & Matheson, 2000), and characteristics of computerised tool use (Baker, Quignard, Lund, & Se´journe´, 2003). In teaching argumentation, several studies have indicated that students’ argumentation skills can be developed during short-term teaching interventions (e.g., Wesp & Montgomery, 1998). However, there are also results suggesting that development in argumentation is mainly a result of long-term engagement in an intellectual academic discussion culture (Pascarella, 1989). Sargeant (1993) examined the discourse included in secondary students’ journals and found that girls showed far more appreciation of different types of argument, and of argument as a two-way process or as a sharing experience. Boys’ accounts, in contrast, were more reminiscent of a battle, a war to be won by confidence, boldness, or expertise. These results are supported by Wood (1997), who suggests that for women communication seems to be mainly a way of establishing and maintaining social relationships, while for men communication often acts as a means of exerting control and enhancing status. Comparative cross-national studies on argumentation skills among secondary school students have not previously been done. However, the comparative study across 32 countries, Programme for International Student Assessment (PISA), organised by the Organisation for Economic Co-operation and Development (OECD), sheds some light on students’ argumentation skills as one element in literacy. In the PISA report (OECD, 2001), argumentation is included, especially in measurements of reading literacy and scientific literacy. Reading literacy is defined as the capacity to retrieve, interpret, reflect on and evaluate written texts (pp. 22, 36). The most demanding tasks at the highest level of reading proficiency required students to draw hypotheses or write grounded arguments on the basis of specialised knowledge (pp. 36 – 37). Scientific literacy, on the other hand, is defined as the ability to relate evidence or data to claims or conclusions and the ability to produce an argument based on a given situation (p. 83).


371

With regard to Finland, France, and the UK, the results of the PISA study indicate that the reading literacy of Finnish secondary school students was higher than in any of the other participating countries (546 points as against the OECD average of 500 points). The mean performance of the UK students (523) was also significantly above the OECD average whereas the mean of the French students (505) was close to the average (OECD, 2001, p. 87). Furthermore, the PISA results indicated that the mean scores for scientific literacy were similar to those of reading literacy: Finland (538), United Kingdom (532), and France (500). In this study, students’ argumentation skills were investigated not only as a component of literacy but as a phenomenon in its own right. The study focuses on two research questions: 1. What is the level of argumentation skills of secondary school students in four school classes from Finland, in two classes from France, and in one class located in England? 2. Are there differences between the samples from the three countries or between males and females in students’ argumentation skills?

Method Students The participants of this study consist of 385 secondary school students (males 49% and females 51%) from Finland (n = 290), France (n = 54), and England (n = 41). Half of the students were 16 years of age, and the age range was from 15 to 17 years. The Finnish students come from four schools located in urban (Jyva¨skyla¨), semiurban (Muurame), semi-rural (Lievestuore), and rural (Sysma¨) districts. In Finland, the secondary schools are municipal schools except for 13 teacher training schools which are a part of the Finnish university system. The funding of the secondary schools is tax based, and the Finnish school system is thus regarded as equal for all. Equality is also manifested by only minor differences in school achievements among Finnish schools compared to many other European countries (OECD, 2004). The students participating in this study represent secondary school students from different geographical areas of Finland. The French students came from two classes of a secondary school located in a suburb of a medium-sized French city, Grenoble. The school is about double the size of the average French high school. However, in terms of the type of general high school diploma offered to students, the school is quite typical—students choose an economic science, literary, or scientific diploma. In the year in question (2002) in France, approximately 1,500,000 students followed the general technological curriculum while 800,000 students followed the vocational curriculum. The students who participated in this study were representatives of the first larger group. The students from England came from one class in General Studies in a comprehensive secondary school located in London. General Studies is a subject

372

M. Marttunen et al.

typically taken, as an option or on a compulsory basis, by 16- to 17-year-old students (Sixth Form). General Studies offers useful preparation for university, in that it aims to develop students’ key skills, study skills, and knowledge of the links across curriculum subjects. The school where the study was conducted is representative of state secondary schools in England. As it follows the National Curriculum, argumentation as such is not formally taught, but there is a well-established and thriving Sixth Form Debating Society as part of the school’s extracurricular activities. This is an informal forum which students attend voluntarily and organise themselves, as happens with many after-school activities in English schools. Since the schools that participated in the study do not stand for all the secondary schools in Finland, France, and England, it is important to note that the data are limited in terms of its representativeness. Thus, one cannot generalise the research results to other secondary school students in the countries in question. However, the study provides an initial description of differences in argumentation skills among students from diverse educational and cultural backgrounds, laying a foundation for further research on the topic. Data Collection and Analyses The students’ argumentation skills were measured with four tasks: analysing an argumentative text, composing claims and arguments, commenting on an argumentative text, and judging arguments and conclusions. The tasks were originally written in Finnish and English, and then translated from English into French. In each country, the students completed the tasks in their mother tongue. For the data analyses, the French students’ answers were translated into English. Translation of methodological instruments often makes cross-cultural comparisons of research results complicated (Brislin, Lonner, & Thorndike, 1973, pp. 32 – 58). The inconsistencies concerning translations detected in this study were, however, so minor that they most likely did not lower its reliability. The data were collected during autumn 2001 and spring 2002. The students in all three countries performed the tasks during a normal school day in a controlled situation with the teacher and/or the researcher present. It took around 45 minutes to complete the test. The reliability of this method in evaluating students’ argumentation skills has been found to be good (Marttunen & Laurinen, 2001). The four tasks are described in detail below. Analysing an argumentative text (Task1). This task was based on an expository text from a published book (Broady, 1986) dealing with progressive pedagogy from the point of view of different social classes. The text touched upon the students’ real life world, making the task meaningful to them, since the topic of the text, progressive pedagogy, very well characterises teaching practices in today’s schools in Western countries. The text also had a distinct argumentative structure, including a clear main claim and supporting arguments that made it eminently suitable for a task involving the analysis of an argumentative text. The text contained 24 numbered sentences.


373

The students were asked to analyse the text by writing down the number of the sentence they thought best expressed the main claim made in the text, and the numbers of those sentences that contained the arguments in support of that claim. One sentence contained the main claim of the text (correct option), and two sentences contained only part of the content of the main claim (partially correct options). Two points were awarded for the correct option and one point for the partially correct options. The arguments the students had selected to support the claim were analysed in terms of their relevance according to the following formula: [Relevance score = R – I/2] in which R is the number of relevant arguments, and I is the number of irrelevant arguments. If the relevance score was one or less, no points were awarded, if the score was from 1.5 to 3, the student got one point, and for scores from 3.5 upwards two points. Composing claims and arguments (Task 2). In this task the students were asked to formulate an opinion on the topic ‘‘Driving in city centres’’, and to write arguments in support of their claim. The analysis focussed on the clarity of the claims (Marttunen & Laurinen, 2001), and the relevance and sufficiency of the arguments (see Bacig, Evans, Larmouth, & Risdon, 1990). A claim was qualified as clear and understandable if it did not include any unclear or bewildering statements, and if it focussed on a single and clearly expressed claim. The claims were scored from zero to two. The analyses of the claims are illustrated in examples one and two. Example 1: An unclear claim A claim: It looks a bit strange when a road goes through a market street in big cities in Finland. There have been continuous efforts to reduce driving but radical changes are unlikely to be made. But why should we reduce driving? I do not find many reasons. (student 209) Analysis of the claim (no points): Instead of putting forward a clear claim the student rather presents two separate opinions: the first relating to roads going through market streets, and the second relating to the presumed failure of efforts to reduce driving. Furthermore, the way she expresses herself – raising a new question instead of clearly expressing an opinion – leaves the reader finally unsure about the standpoint of the writer. Example 2: A clear claim A claim: Driving in the heart of cities should not be banned. (student 54) Analysis of the claim (two points): This claim is understandable and focussed on a single claim only. Thus, two points were given.

The students’ arguments were classified into three categories: (1) arguments that were mainly irrelevant and too few (no points); (2) arguments that offered only a narrow support for the claim, and that were possibly overlapping (one point); and (3) arguments relevant to the claim and offering broad support for it (two points). The analyses of the students’ arguments are illustrated in examples three and four.

374

M. Marttunen et al. Example 3: Irrelevant and insufficient argumentation A claim (C) and arguments (A1, A2, and A3): Driving private cars should be allowed in city centres only during certain times (C); Cars should not be banned in city centres since some people need a car in order to get to the city centre (A1); Cars could be allowed in the centre but I think that a timetable could be drawn up – drivers should not be punished because they cannot get to the city centre by any other means than by car (A2); Cars should not be allowed in city centres in the afternoons and in the evenings because at that time there are too many people around and there is not enough space for parking (A3). (student 507) Analysis of the arguments (no points): In the two first arguments the writer does not present any reasons for why driving in city centres should only be allowed during certain times. In the first argument, instead of presenting reasons for driving restrictions, as one would have expected, the student presents a reason why driving should not be banned. In the second argument she repeats her initial claim that a timetable for driving in centres is needed. Furthermore, she mentions implicitly the inadequacy of public transportation services. This cannot be regarded as relevant when one is supposed to give reasons for setting limits on driving. The third argument, when widely interpreted, can be regarded as supporting the claim. However, the support is indirect since, instead of putting forward reasons for allowing driving at certain times, the writer gives reasons for restricting driving in the evenings and afternoons. The writer’s arguments as a whole have been judged to be mainly irrelevant and too few. Example 4: Relevant and sufficient argumentation A claim (C) and arguments (A1, A2, A3, A4, and A5): Driving in city centres should be reduced (C); Exhaust fumes and toxins released into the air would be decreased if driving was diminished (A1); People would resort to public transport more (A2); Traffic accidents would be reduced (A3); The city centre would be cosier, and this cosiness could even be increased, for example, by plants (A4); Drunk people coming out of bars would not be able to drive so easily (A5). (student 21) Analysis of the arguments (two points): All the five arguments are relevant since each of them gives a reason for why driving should be reduced. In addition, the student’s argumentation can be regarded as sufficient since she makes the harmfulness of driving in city centres clear from many perspectives: by appealing to the environmental effects of cars (argument 1), to safety aspects (arguments 3 and 5), to the aesthetic benefits of reduced driving (argument 4), and to aspects concerning people’s traffic behaviour (argument 2).

Commenting on an argumentative text (Task 3). In this task, the students commented on an imaginary argumentative text on gender equality at school. The text consisted of a provocative claim, four extreme or false arguments provided in support of the claim, and a biased conclusion: A claim: A newspaper article claimed that girls and boys are equal at school. Arguments: First, school books are equally interesting from the point of view of girls and boys (1). Second, the division of school subjects has been planned so as not to be biased in favour of either boys or girls (2). Furthermore, there are both male and female


375

teachers in every school (3). And, finally, discipline and punishments are the same for girls and boys (4). A conclusion: On the basis of the above it can be concluded that equality between the sexes exists at school.

The students were asked to comment freely on the text. The students’ comments were assessed on the basis of how analytical they were. In an analytical comment (two points), a student responded explicitly to most of the argumentative elements of the text: to the claim or the conclusion, and to at least three arguments. The comments classified as in-between (one point) included explicit responses to two or three elements, and in a non-analytical comment (no points) only one or none of the elements was responded to. Judging arguments and conclusions (Task 4). The task on judging the correctness of given arguments and conclusions consisted of two different subtasks. The first task included a claim and five alternative arguments: A claim: It is useful to study foreign languages. Arguments: When one can speak foreign languages it is easier for him/her to communicate with foreigners (1); There are several different member countries in the European Union (2); Many employers value language skills (3); There are many different cultures in the world (4); Studies have shown that women are often linguistically talented (5).

The students were asked to evaluate the arguments and to choose the ones they thought supported the claim. One point was awarded for a correct choice (arguments 1 and 3) and one point subtracted for a logically irrelevant choice (arguments 2, 4, and 5). The range of the scoring was from zero to two points. In the second task, the students were given three arguments and three conclusions: Arguments: Remembering facts is emphasised too much in school teaching (1); Many students get poor marks because feelings of tension during the exam hamper their ability to remember things (2); Exam results do not give a good picture of students’ learning of particular subjects, but rather measure the ability to pass an exam (3). Conclusions: Teaching and examination practices in school should be improved (1); Giving marks is not a reliable way to measure learning (2); Verbal assessment is a more just way to evaluate learning than an exam (3).

The students were asked to evaluate the conclusions and to choose the one they judged to be the correct one. The correct conclusion (no. 1, two points) was supported by all three arguments, a partially correct conclusion (no. 2, one point) by two of the arguments, and a false conclusion (no. 3, no points) by none of the arguments.

376

M. Marttunen et al.

Statistical Analyses The statistical analyses investigated the associations between two independent variables (Country, Gender) and seven dependent variables (Identifying the main claim, Identifying grounds, Composing a claim, Composing grounds, Analytical approach, Judging grounds, Judging conclusions) derived from the students’ answers to the four tasks. Nonparametric techniques were used in the data analyses. First, the data proved limited in meeting the assumptions about parametric techniques (i.e., the existence of a normal distribution of the dependent variables and homogeneity of variances; Stevens, 1986, p. 199). Second, the dependent variables of the study are in the first place ordinal scale variables, which means that the data should mainly be analysed using nonparametric statistics (Siegel, 1980). The analyses were carried out by testing both the independent variables separately with all the individual dependent variables. Kruskal-Wallis variance analysis (X2) was used in testing the differences between the countries. The two-by-two comparisons between the countries and between the different sexes were tested by the Mann-Whitney test (z).

Results Analysing an Argumentative Text The text analysis task was rather difficult for the students (Table 1) as their mean scores in both tasks were less than 1 (max 2). No statistically significant differences (p values were .070 and .072) between the countries were found. However, according to Siegel (1980, pp. 8 – 9), the need to adhere rigidly to the arbitrary level of significance at the .05 level should not always be taken for granted, as findings at low probability levels (for example less than .10) may also be of interest. Thus, a more detailed examination of the differences between the countries through two-by-two comparisons was warranted. The results showed that the English students performed better than the French students in identifying the main claim, z = 7 2.19, p 5 .05, and that the Finnish students outperformed the French students in identifying arguments,

Table 1. Differences between countries in analysing an argumentative text (Kruskal-Wallis test) Identifying the main claim (df = 2; X2 = 5.32; p = .070)

Finland France England Total

Identifying arguments (df = 2; X2 = 5.28; p = .072)

N

M

SD

N

M

SD

290 54 41 385

0.96 0.83 1.05 0.95

0.47 0.42 0.50 0.47

286 50 40 376

0.45 0.22 0.45 0.43

0.63 0.42 0.68 0.62

Note: The range of the variables is from 0 to 2.


377

z = 7 2.30, p 5 .05. The differences between the Finnish and English students were statistically nonsignificant in both identifying tasks. Composing Claims and Arguments The task, in which the students were asked to compose a claim about driving in city centres and arguments in support of the claim, was not difficult for them (means 1.50 and 1.60, Table 2). This may have been due to the familiarity and clarity of the theme and the instruction to the students. When the three countries were compared, it was found that there were differences in the students’ skills in composing a claim (p 5 0.01) but not in composing arguments. The country comparisons indicated that the students in France composed their claims more clearly than the students in Finland, z = 7 2.89, p 5 .01. The claims of the Finnish and French students differed from each other particularly in terms of focus. The French students more often than the Finnish students wrote single-focussed claims (80.4% vs. 62.4%), such as I think that driving should be banned in city centres (student 475), and I think driving in city centres should be allowed only at certain times (student 507). By contrast, the claims put forward by the Finnish students were more often multifocussed (37.6% vs. 19.6%) including two or even three separate statements, like I think that driving in the nuclear centre of a city is not necessary depending, however, on the size of the city. At least a part of the city should consist of so called ‘‘walking street’’ (2 statements; student 15), and I think that private driving should be banned in city centres. Service driving, public transport and emergency vehicles should be allowed. In small cities it should be permitted to use private cars as well (3 statements, student 146). Commenting on an Argumentative Text On an average, the students’ comments were not very analytical in nature (Table 3). The Finnish and the French students performed quite poorly, obtaining means below

Table 2. Differences between countries in composing claims and arguments (Kruskal-Wallis test) Composing a claim (df = 2; X2 = 9.72; p = .008)


Composing arguments (df = 2; X2 = 4.13; p = .127)

N

M

SD

N

M

SD

286 54 41 381

1.45 1.72 1.61 1.50

0.71 0.60 0.67 0.70

281 53 40 374

1.61 1.49 1.75 1.60

0.58 0.67 0.49 0.59

Note: The range of the variables is from 0 to 2.

378

M. Marttunen et al.

Table 3. Differences between countries in commenting on a provocative and biased argumentative text (Kruskal-Wallis test) Analytical approach (df = 2; X2 = 22.48; p = .000)


N

M

SD

284 52 41 377

0.60 0.85 1.17 0.69

0.68 0.83 0.74 0.73

Note: The range of the variable is from 0 to 2.

one (0.60 and 0.85), while the average performance of the English students (1.17) can be regarded as moderate. The means between the countries differed statistically from each other (p 5 .001). The mean difference for Finland and England was 0.57, and for Finland and France 0.25. Both of these differences were statistically significant (z = 7 1.98, p 5 .05; z = 7 4.60, p 5 .001). The mean difference for France and England (0.32), although bigger than for Finland and France, was not statistically significant. The differences between the comments of the students from the different countries become particularly evident in how they responded to the main claim, gender equality exists at school, included in the given text. The data extracts in Table 4 illustrate the typical ways in which the Finnish, French, and English students responded to the main claim. The extracts are comparable in that all of them are extracts from analytical comments. Typical of both the Finnish (54.7%) and French (70.4%) students was that they expressed either direct agreement (extracts 1, 3, 4 and 6) or disagreement (extracts 2 and 5 in Table 4) with the claim. The English students, instead of clearly committing themselves for or against the claim, like the Finns and French did, mostly (52.4%) questioned the validity of the claim. They, for example, agreed with the claim only partly (extract 7), rebutted it with counterarguments (extract 8) or showed states of affairs in which the claim was not valid (extract 9). Responses of this kind were much rarer among the Finnish (33.5%) and French students (22.2%). The challenging attitude of English students towards written and spoken arguments (e.g., Mauranen, 1993) may have been one reason for the English students’ higher proportion of analytical responses to the text they were asked to comment on. Judging Arguments and Conclusions The students’ ability to judge the logic of arguments was measured by multiplechoice tasks. Students generally prefer multiple-choice tasks because they have only to recognise and indicate the correct answers and are not required to compose or produce anything. However, the easiness of multiple-choice tasks might be deceptive,


379

Table 4. Typical responses of the Finnish, French and English students to the main claim in a given text. Finnish students

French students

English students

Extract 1 (student 235, female) I think there is no reason to claim that equality would not happen.

Extract 4 (student 509, female) Extract 7 (student 443, female) Equality between the sexes is a I agree that there is a certain fact in schools. amount of equality in school between the sexes, but not enough equality to be concluded that there is total equality. Extract 2 (student 46, female) Extract 5 (student 500, female) Extract 8 (student 433, male) Total equality between girls and First, I think that the sexes are I think that boys and girls both boys does not exist at school. not equal at school. are equal at school academically, but research has told me that boys do better in mixed schools and girls do better in single-sex schools. Extract 3 (student 224, male) Extract 6 (students 515, male) Extract 9 (student 448, female) Though I agree that these days Almost everything in newspaper I think that the text above is articles is factual. right. girls and boys are more equal at school, this isn’t the case in every school or with every student or with every problem.

especially when used to measure argumentative reasoning. Nevertheless, these tasks proved to be easy for most of the students (means 1.52 in judging arguments and 1.33 in judging conclusions, Table 5). Some statistically significant differences between the students from the different countries were found in judging conclusions (p 5 .001) but not in judging arguments. Further analyses revealed that the Finnish students outperformed both the English, z = 7 2.42, p 5 .05, and French students, z = 7 3.73, p 5 .001. The slight difference between the English and French students may be attributed to random factors. Argumentation Skills by Gender The results indicated that the female students’ (n = 196) comments on an argumentative text were more analytical than those of the male students (n = 189): females more often responded explicitly to most of the argumentative elements of the task (means 0.80 and 0.59; z = 7 2.78, p 5 .01). The gender differences in other variables were not statistically significant. Discussion and Conclusion The results showed that in the multiple-choice tasks most of the students correctly recognised the relevant arguments from the alternative options and were able to

380

M. Marttunen et al.

connect the correct conclusion with the arguments given (Table 5). Furthermore, many of them scored highly in composing claims and arguments in the writing task (Table 2). These results suggest that the secondary school students who participated in this study were rather skilful in logical argumentative thinking. However, they were either unable or unwilling to apply their argumentative thinking spontaneously in the first and third tasks, where an analytical approach towards a written text was in order. In the first task it proved to be difficult for many students to recognise the main claim in the text and even more difficult to connect the correct arguments with that claim (Table1). The students did not succeed very well in the third task either, in which they were asked to comment on a short provocative argumentative text (Table 3). Thus, although these students seemed to have the prerequisites for both argumentative reasoning and composing argumentative writings on familiar topics, most of them need more practice in their analytical and critical reading skills. The English students in particular proved to be good at writing analytical comments in the commenting task. It is important to note, that the students came from a school in which extracurricular argumentative activity, through the ‘‘Debating society’’, is popular with students. Engagement in the activities of this society may have played a role in the more developed analytical abilities of the students. Further, the students’ analytical skill could also be due to the fact that analytical writing is encouraged in English schools. One of the key aims in the English curriculum is for students to develop the thinking through review, analysis, hypothesis, recollection, and summary. In the same vein, the French curriculum emphasises the development of students’ structured thinking, self-expression and reflection, and comparison of contradictory opinions as a basis for constructing their own opinions. Although parallel aims are included in the Finnish secondary school curricula, the time allocated for exercising and practising these activities is very restricted compared with England and France. The English and French students performed, however, less satisfactorily than the Finnish students in the task of judging conclusions. The poorer judging skills of the English students might be affected by the fact that in English schools the elements of argumentative skills, such as producing conclusions on the basis of given data and

Table 5. Differences between countries in judging arguments and conclusions (Kruskal-Wallis test) Judging arguments (df = 2; X2 = 3.97; p = .137)


Judging conclusions (df = 2; X2 = 17.57; p = .000)

N

M

SD

N

M

SD

289 53 37 379

1.53 1.34 1.65 1.52

0.66 0.77 0.68 0.68

285 51 37 373

1.41 1.08 1.14 1.33

0.68 0.52 0.67 0.67

Note: The range of the variables is from 0 to 2


381

providing evidence for conclusions, are introduced in the context of oral debate rather than essay writing and textual analysis. However, although French students are well versed in the written analysis of philosophical texts, the emphasis is not on ‘‘judging conclusions’’ per se, but rather on the processes of argumentation themselves, for example impartial exposition of a thesis, elaboration of a concept in an argument, and confrontation of theses. The results of this study show similarities with the results of the PISA study (OECD, 2001). The PISA research indicated that the profiles of the most proficient readers are reversed in Finland and the UK. The percentage of the Finnish students at the uppermost level of proficiency was 26% in retrieving information, 24% in interpreting texts, and only 14% in reflective and evaluative reading, when in the UK the respective percentages were 16%, 14%, and even as high as 20%. In the PISA study, the UK students also outperformed their Finnish counterparts in scientific literacy in the most difficult tasks concerning assessing evidence and writing conclusions. These findings are parallel to the results of the present study, as the English students performed better than the Finnish ones in the task in which they were asked to comment analytically on the provocative argumentative text. In this task, both evaluative reading and the skill to assess evidence were of particular importance. Furthermore, in the present study, the female students were more proficient than the male students in producing analytical comments on the argumentative text. This result refers to the better analytical reading skills of girls compared to boys in this study. The results of the PISA survey also showed that female students were superior to male students in reflective and evaluative reading in each of the 32 countries that took part in the survey. Similar results have consistently been emphasised within other literacy studies as well (e.g., Francis, 2000). In the present study, female students’ better performance in commenting on a text may also be due to the ill-structured task type used. Cohen (1994) has highlighted that for illstructured tasks, particularly, the amount and quality of writing and interaction are crucial factors. Previous studies suggest that girls are better writers than boys in terms of the clarity, focus, and organisation of the text (Knudson, 1995), compositional fluency (Berninger, Whitaker, Feng, Swanson, & Abbott, 1996), and grammatical writing (Engelhard, Walker, Gordon, & Gabrielson, 1994). The more analytical comments of the girls in this study may thus also be due to their better writing skills. It is important to note that due to the relatively small number of students participating in the study and the nonrandomised sample for data, one should not generalise the results across secondary school students in these three countries. In spite of this limitation, however, the study describes differences in the argumentation skills of students coming from various cultural backgrounds, and who also have experienced teaching-learning situations based on different curricula. The results suggest a need for further research to obtain a better understanding of the cultural dimension of argumentation and to develop suitable teaching materials and learning environments for studies of collaborative argumentation.

382

M. Marttunen et al.

Acknowledgements The authors gratefully acknowledge the teachers and students of the secondary schools in Finland, France, and England for their collaboration during the data collection and Pekka Penttinen for his valuable help in statistics.This study was carried out within the SCALE project funded by the European Community. References Andriessen, J., Baker, M., & Suthers, D. (2003). Argumentation, computer support, and the educational context of confronting cognitions. In J. Andriessen, M. Baker, & D. Suthers (Eds.), Arguing to learn. Confronting cognitions in computer-supported collaborative learning environments (pp. 1 – 25). Dordrecht, The Netherlands: Kluwer. Bacig, T. D., Evans, R. H., Larmouth, D. W., & Risdon, K. C. (1990). Beyond argumentation and comparison/contrast: Extending the Socrates CAI design principles to classroom teaching and the interpretation and production of other forms of discourse. Computers and the Humanities, 24, 15 – 41. Backley, P., Saxton, M., & Sillince, J. A. A. (1999). ADS in the curriculum: Results of interviews with secondary school teachers. Working Paper, Management School, Royal Holloway, University of London. Baker, M. J., Quignard, M., Lund, K., & Se´journe´, A. (2003). Computer-supported collaborative learning in the space of debate. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change in networked learning environments: Proceedings of the international conference on computer support for collaborative learning 2003 (pp. 11 – 20). Dordrecht, The Netherlands: Kluwer. Baylac, M.-H. (Ed.). (2000). Education civique, juridique et sociale (Seconde) [Civic, legal and social education]. Paris: Bordas. Bereiter, C., & Scardamalia, M. (1999). Intentional learning as a goal of instruction. In L.B. Resnick (Ed.), Knowing, learning and instruction: Essays in honour of Robert Glaser (pp. 361 – 392). Hillsdale, NJ: Erlbaum. Berninger, V., Whitaker, D., Feng, Y., Swanson, H. L., & Abbott, R. D. (1996). Assessment of planning, translating, and revising in junior high writers. Journal of School Psychology, 34, 23 – 52. Biemans, H. J. A., Deel, O. R., & Simons, R.-J. (2001). Differences between successful and less successful students while working with the CONTAC-2 strategy. Learning and Instruction, 11, 265 – 282. Brislin, R. W., Lonner, W. J., & Thorndike, R. M. (1973). Cross-cultural research methods. New York: John Wiley. Broady, D. (1986). Piilo-opetussuunnitelma [The hidden curriculum]. Tampere, Finland: Vastapaino. Cohen, E. G. (1994). Restructuring the classroom – Conditions for productive small groups. Review of Educational Research, 64, 1 – 35. Dillenbourg, P. (1999). Introduction: What do you mean by ‘‘collaborative learning’’? In P. Dillenbourg (Ed.), Collaborative learning. Cognitive and computational approaches (pp. 1 – 19). Amsterdam: Pergamon. Engelhard, G, Jr., Walker, E. V. S., Gordon, B., & Gabrielson, S. (1994). Writing tasks and gender: Influences on writing quality of black and white students. Journal of Educational Research, 87, 197 – 209. Francis, B. (2000). Boys, girls and achievement: Addressing the classroom issues. London: Routledge. Howe, C., & Tolmie, A. (1999). Productive interaction in the context of computer-supported collaborative learning in science. In K. Littleton & P. Light (Eds.), Learning with computers. Analysing productive interaction (pp. 24 – 45). London: Routledge.


383

King, P. M., Wood, P. K., & Mines, R. A. (1990). Critical thinking among college and graduate students. The Review of Higher Education, 13, 167 – 186. Knudson, R. E. (1995). Writing experiences, attitudes, and achievement of first to sixth graders. Journal of Educational Research, 89, 90 – 97. Laborde-Milaa, I. (Ed.). (2000). Français (Seconde et premie`re): Texts, langues ećriture. [French: Text, language and writing]. Paris: Belin. Le, E. (2004). Active participation within written argumentation: Metadiscourse and editorialist’s authority. Journal of Pragmatics, 36, 687 – 714. Litosseliti, L. (1999). Moral repertoires and gender voices in argumentation. PhD thesis, Department of Linguistics & Modern English Language, Lancaster University. Marttunen, M., & Laurinen L. (2001). Learning of argumentation skills in networked and face-toface environments. Instructional Science, 29, 127 – 153. Mauranen, A. (1993). Opiskelijan diskurssimaailmat – Vaihto-opiskelijoiden perspektiivi [The discourse worlds of students – The perspective of exchange students]. In H. Jalkanen & L. Lestinen (Eds.), Korkeakoulutuksen kriisi? [Higher Education in a crisis] (pp. 169 – 188). Jyva¨skyla¨, Finland: University of Jyva¨skyla¨, Institute for Educational Research. McCann, T. M. (1989). Student argumentative writing knowledge and ability at three grade levels. Research in the Teaching of English, 23, 62 – 76. Mercer, N. (2000). Words & Minds. How we use language to think together. London: Routledge. Mercer, N., Wegerif, R., & Dawes, L. (1999). Children’s talk and the development of reasoning in the classroom. British Educational Research Journal, 35, 95 – 111. Ministe´re e´ducation nationale. (2000). Bulletin Officiel Hors Se´rie n86, n87, n88, 2nd and 1st levels. Available at http://www.education.gouv.fr/bo/2000/ (27.1.2005). National Board of Education. (1994). Framework curriculum for the senior secondary school. Helsinki, Finland: Author. OECD. (2001). Knowledge and skills for life. First results from the OECD programme for international student assessment (PISA) 2000. Organisation for Economic Co-operation and Development. Available at http://www1.oecd.org/publications/e-book/9601141e.pdf (27.1.2005). OECD. (2004). Learning for tomorrow’s world. First results from PISA 2003. Organisation for Economic Co-operation and Development. Available at http://www.pisa.oecd.org/dataoecd/1/60/ 34002216.pdf (27.1.2005). Pascarella, E. T. (1989). The development of critical thinking: Does college make a difference? Journal of College Student Development, 39, 19 – 26. Perkins, D. N. (1985). Postprimary education has little impact on informal reasoning. Journal of Educational Psychology, 77, 562 – 571. QCA. (2005a). Qualifications and Curriculum Authority. Guarding standards. Available at http:// www.qca.org.uk/ (27.1.2005). QCA. (2005b). Qualifications and Curriculum Authority. National Curriculum online. Available at http://www.nc.uk.net/ (27.1.2005). Ravenscroft, A., & Matheson, M. P. (2000). Evaluating and investigating learning through collaborative argumentation: An empirical study. Dialogue and design for new media research group Technical report DDRG-01-01. Institute of educational technology, Milton Keynes, UK: Open University. Sargeant, J. (1993). Gender and power: The meta-ethics of teaching argument in schools. Curriculum, 14, 6 – 13. Siegel, S. (1980). Nonparametric statistics for the behavioural sciences. Tokyo: McGraw-Hill Kogakusha. Steffensen, M. S. (1996, August). How Finns and Americans persuade? Paper presented at the 11th World Congress of Applied Linguistics (AILA 96), Jyva¨skyla¨, Finland. Stein, N. L., & Bernas, R. (1999). The early emergence of argumentative knowledge and skill. In J. Andriessen & P. Coirier (Eds.), Foundations of argumentative text processing (pp. 97 – 116). Amsterdam: Amsterdam University Press.

384

M. Marttunen et al.

Stevens, J. (1986). Applied multivariate statistics for the social sciences. Hillsdale, NJ: Lawrence Erlbaum. Tannen, D. (1998). The argument culture. New York: Random House. Van Eemeren, F. H., & Grootendorst, R. (1999). Developments in argumentation theory. In R. Andriessen & P. Coirier (Eds.), Foundations of argumentative text processing (pp. 43 – 57). Amsterdam: Amsterdam University Press. Voss, J. F., & Means, M. L. (1991). Learning to reason via instruction in argumentation. Learning and Instruction, 1, 337 – 350. Webb, N. M. (1995). Group collaboration in assessment: Multiple objectives, processes, and outcomes. Educational Evaluation and Policy Analysis, 17, 239 – 261. Wesp, R., & Montgomery, K. (1998). Developing critical thinking through the study of paranormal phenomena. Teaching of Psychology, 25, 275 – 278. Wood, J. T. (1997). Gendered lives: Communication, gender, and culture (2nd ed.). Belmont: Wadsworth.