The Force Concept Inventory as a Measure of

The FCI as a Measure of Students’…

The Force Concept Inventory as a Measure of Students' Conceptual Coherence1

Antti Savinainen and Jouni Viiri

Antti Savinainen, PhD, ([email protected]) is a physics teacher at Kuopion Lyseo High School, Kuopio, Finland; he teaches in both the National Curriculum and the International Baccalaureate Program. His doctoral studies dealt with students’ conceptual coherence in the case of the force concept. He is continuing his studies and acts as a supervisor in close co-operation with Professor Jouni Viiri.

Jouni Viiri ([email protected]) is Professor of Science and Mathematics Education at the Teacher Education Department, University of Jyväskylä, Finland. He has worked as a physics teacher at different levels from secondary school to polytechnics and university. His main research interest is in physics education, especially in developing research-based teachinglearning sequences, the use of models in physics education, and the communication between teacher and students.

1

Accepted for publication in International Journal of Science and Mathematics Education.

1


The Force Concept Inventory as a Measure of Students' Conceptual Coherence

Abstract

The Force Concept Inventory (FCI) is a multiple choice test designed to monitor students’ understanding of the conceptual domain of force and related kinematics (Hestenes, Wells & Swackhamer, 1992; Halloun et al., 1995). It has gained wide popularity among both researchers and physics instructors in the USA and elsewhere. The FCI has also been criticized and its validity as a measure of the coherence of a student’s understanding of the force concept has been questioned. In this paper we provide a characterization of students’ conceptual coherence and a way to evaluate it using the FCI. We divide students’ conceptual coherence into three aspects: representational coherence (the ability to use multiple representations and move between them), contextual coherence (the ability to apply a concept accross a variety of contexts) and conceptual framework coherence (the ability to fit related concepts together, i.e. to integrate and differentiate between them). Post-instruction FCI results and interview data from two Finnish high school groups (n = 49 altogether) are discussed: the data provide evidence that the FCI can be used to evaluate students’ conceptual coherence – especially contextual coherence - of the force concept.

Key words: conceptual coherence, the Force Concept Inventory, multiple representations, teaching force, Newton’s laws

2

The FCI as a Measure of Students’… Introduction

The Force Concept Inventory (the original FCI: Hestenes, Wells & Swackhamer, 1992; the 1995 version of the FCI: Halloun et al., 1995) is a widely used multiple choice test in evaluating students’ understanding of the force concept and related kinematics. It has gone through a lengthy process of validation and its reliability has been well established (for a review, see Savinainen & Scott, 2002a). However, a concern is raised by Huffman and Heller (1995): they claim that the FCI measures not so much the coherence of student’s understanding of the force concept but only “bits and pieces of student knowledge”. They base their claim on factor analysis, which revealed few significant factors even among socalled “confirmed Newtonian thinkers” who scored over 85% on the FCI. Their criticism is answered to some extent by Hestenes and Halloun (1995a, b): basically they argue that factor analysis is not suitable for analyzing the type of data that the FCI provides. However, as far as we know no real data set has been published to show that the FCI can indeed measure the conceptual coherence of students’ understanding of the force concept.

In this paper our aims are to introduce a new way to characterize students' conceptual coherence and to show how the Force Concept Inventory (FCI) can be used to evaluate students' conceptual coherence. Post-instruction FCI and interview data from a Finnish upper secondary school are used as an example. First we consider what the notion of students' conceptual coherence might entail, based on Savinainen (2004).

3

The FCI as a Measure of Students’… Earlier Research on Students' Conceptual Coherence

There is a body of research showing that student’s views are not ‘consistent’ or ‘coherent’ after an introductory course in mechanics (see references below). Most of this research does not, however, provide a detailed definition of what is meant by consistency or coherence. Generally speaking, in the earlier research lack of coherence or consistency seems to mean that students respond differently to different types of tasks involving the same concepts.

When Finegold and Gorsky (1991) investigated consistency in students’ concept of force, they found that many students did not understand, or had great difficulty in applying, Newton's laws. They found no alternative framework which was consistently used by students: only the Newtonian framework was consistently used across different tasks by a few students. The same conclusions were drawn by Halloun and Hestenes (1985) from their study of over 4000 college students’ concept of force.

Other investigators have reported worrying findings regarding the effect of traditional teaching: for example, “they [students] rely on various special knowledge elements stored in memory, try to achieve one of these, and apply it without much subsequent reasoning” (Reif 1987). McDermott (1993) concluded that “a coherent framework is not typically an outcome of traditional instruction”. This conclusion is well supported by Hake’s large survey, which strongly suggests that traditional courses fail to convey much basic conceptual understanding of Newtonian mechanics to the average student (Hake, 1998; Hake, 2002).

It has also been reported that some students showed correct conceptual understanding in some exercises involving the concept of force but did not apply this in other contexts (Steinberg &

4

The FCI as a Measure of Students’… Sabella, 1997). These investigators argue that ‘different contexts and presentations can trigger different responses from a given student, even if the underlying physics is identical’. Even varying magnitudes of the quantities involved can trigger different responses in the same academic context (Mildenhall & Williams, 2001). Students may respond to irrelevant contextual features of the question, such as the type of object in motion or the direction of the motion (Palmer, 1994).

Conceptual Coherence of Qualitative Knowledge

In the above-mentioned review of earlier research, the terms ‘coherence’ and ‘consistency’ have been virtually used as synonyms. We propose that they are indeed related but not identical notions: ‘coherence’ refers to a student’s internal mental models which are not directly “seen”. On the other hand, ‘consistency’ refers to a student’s answers in questionnaires or interviews, i.e. to a student’s responses to various tasks which are amenable to objective analysis. Hence, we assume here that a student’s consistency (or lack of it) in his/her answers may be used to infer his/her level of conceptual coherence. We note that our notion of coherence concerns the student’s thinking; the coherence of the Newtonian system itself is discussed in detail by Hestenes (1992).

We have divided the conceptual coherence of students’ qualitative understanding into three aspects: representational, contextual and conceptual framework coherence (Figure 1). Each aspect is briefly outlined by describing the skills it entails. Naturally there is some overlap between the aspects.

5

The FCI as a Measure of Students’… CONCEPTUAL COHERENCE OF QUALITATIVE UNDERSTANDING OF A PHYSICS DOMAIN

Conceptual framework

Contextual coherence

Representational coherence

Being able to apply in

Being able to apply in

coherence

Being able to differentiate and integrate between related concepts

Familiar context

Novel context

For instance acceleration and velocity

Verbal representation

moving between Graphical representation

Figure 1.

Diagrammatic representation

Aspects of conceptual coherence of qualitative knowledge in physics.

1) Representational coherence Representational coherence means that the student is able to use multiple representations of the same situation correctly and move between them. Verbal (written and oral), diagrammatic (vectors, motion maps, path diagrams) and graphical (graphs, e.g. velocity against time) representations are efficient tools in analysing physical situations (Van Heuvelen, 1991). Mathematical representation is also naturally very important but it is not within the scope of this study.

Meltzer (2002) illustrates the significance of representation in assessing student understanding. He gives examples of test items which address Newton's Third Law in the

6

The FCI as a Measure of Students’… context of gravitational forces between the earth and other heavenly objects. Similar questions were framed in verbal and vector diagram representations: the proportion of correct responses was halved when vector diagram representation was used. Meltzer notes that the question “was measuring not only students' knowledge of Newton's Third Law of motion and law of gravitation, but also (in part) students' understanding of vector diagrams”. He also points out that it is very hard to design fully equivalent questions in multiple representations since there are some differences with respect to details of the information presented. Naturally for an expert different representations encode essentially the same physical information, i.e. in this case that forces are equal and opposite no matter what representation is used to frame the question. Hence, we interpret students’ poor performance in the vector diagram question as a lack of representational coherence in their understanding of Newton's Third Law.

2) Contextual coherence This means that the student can apply a concept (e.g. acceleration) or a physical law (e.g. Newton's laws) in a variety of familiar and novel contexts. The context here refers to the circumstantial features in which a task is posed. Contextual coherence cannot be evaluated in isolation since the student must use some representation to express his/her understanding in given situations. The effect of contextual factors can be probed if a representation is kept constant, and conversely, students’ representational coherence can be evaluated only if contextual factors are kept as constant as possible.

Even slight changes in context or contextual features can make a difference for a student who lacks contextual coherence. Schecker and Gerdes (1999), for instance, found that when certain FCI questions were posed in slightly different contexts, student responses were significantly

7

The FCI as a Measure of Students’… different when for instance, a golf ball was replaced by a soccer ball or a steel ball thrown upward was replaced by a vertical pistol shot.

3) Conceptual framework coherence This aspect addresses relations between concepts and overlaps the other aspects to some extent. In order to apply a concept in a variety of contexts, the student must relate (integrate) a concept to other concepts. The student also needs to differentiate that concept from related concepts (McDermott, 1993). To put it briefly, conceptual framework coherence is about how the relevant concepts fit together. Evaluation of this aspect is possible if the given tasks demand the use of many related concepts at the same time (in fact, it is very difficult to write diagnostic questions that probe only one single idea: usually any question requires some understanding of ideas other than the one actually being probed). For instance, answering questions on Newton's Second Law demands some conceptual framework coherence, since the usual formulation of Newton's Second Law includes the concept of acceleration. Furthermore, acceleration is underpinned by the concept of velocity. A good performance in a question on Newton’s Second Law in one representation and context implies that the student has reached at least some degree of conceptual framework coherence. However, a student may have achieved conceptual framework coherence in some representation and context and still fail in other representations and contexts. Hence, conceptual framework coherence is a necessary but not sufficient condition for representational and contextual coherence.

Figure 2 presents example questions based on Hake (2002) to illustrates how the aspects of conceptual coherence can be used in categorising a question.

8

The FCI as a Measure of Students’… A student in a lab holds a brick of weight W in her outstretched horizontal palm and lifts the brick vertically upward at a constant speed. All the following questions refer to the situation of the brick moving vertically upward at a constant speed. 1.

The magnitude of the force on the brick by the student’s hand is: A. constant in time and zero. B. constant in time, greater than zero, but less than W. C. constant in time and W. D. constant in time and greater than W. E. decreasing in time but always greater than W.

2.

Draw a free-body diagram showing all the forces acting on the brick.

3.

a) Graph velocity against time. b) Graph position against time. c) Graph acceleration against time.

Figure 2.

Conceptual questions based on Hake (2002) to illustrate analysis in terms of conceptual coherence.

All three questions in Figure 2 are framed in a single context. Question 1 involves the application of Newton’s First Law in verbal representation whereas question 2 involves moving between representations (from verbal to diagrammatic representation), as does question 3a (from verbal to graphical representation). Question 3 addresses moving within graphical representation. In addition to representational coherence, questions 3b and 3c also address conceptual framework coherence, since successful performance demands integration of position, velocity and acceleration.

Now we turn to the FCI test and its structure from the point of view of conceptual coherence.

9

The FCI as a Measure of Students’… Force Concept Inventory and Conceptual Coherence

The Force Concept Inventory (FCI) is a multiple-choice test designed to monitor students’ understanding of the conceptual field of force and related kinematics (Hestenes, Wells & Swackhamer, 1992; Halloun et al., 1995). The strengths and limitations of the FCI in assessment are thoroughly discussed in Savinainen and Scott (2002a). It would help the reader if the FCI were available in this paper, but the FCI is not presented here for copyright reasons and to protect the confidentiality of the test. A copy of the FCI can be obtained from the Modeling Project web site (see the reference Halloun et al., 1995).

The FCI addresses six conceptual dimensions within the domain of force and related kinematics. These six dimensions provide a ‘conceptual map’ of the area covered by the items in the FCI: •

Kinematics

•

Newton's First Law

•

Newton’s Second Law

•

Newton’s Third Law

•

Superposition Principle

•

Kinds of Forces: contact forces and gravitational forces (we analyse these separately here)

Hestenes and Halloun (1995a) have argued that the entire FCI test should be used for the purposes of course and teaching evaluation. They argue that ”the total FCI score is the most reliable single index of student understanding, because it measures coherence across all dimensions of the Newtonian force concept”. Single FCI items cannot be used to make reliable conclusions but several items addressing the same dimension of the force concept can

10

The FCI as a Measure of Students’… provide valuable information about specific learning difficulties that students may have (e.g., see Savinainen & Scott 2002b). It may well be the case that the total score is the best single measure of a student’s overall conceptual coherence of the force concept, but we believe that a more detailed analysis in terms of aspects of conceptual coherence is possible.

Hestenes, Wells & Swackhamer (1992) classified the FCI questions in terms of the above mentioned six dimensions of the force concept above. In addition to their classification we use the

idea of representations in classifying the FCI questions. Table 1 presents the

classification in terms of representation and the dimensions of the force concept for the 1995 version of the FCI (Halloun et al., 1995)

Table 1.

Classification of FCI questions in terms of representation and the dimensions of the force concept.

Dimension of

Kinematics

Newton’s First

Newton’s

Newton's

Law

Second Law

Third Law

force

Representation FCI question

Diagram

Verbal Diagram

12, 14, 19, 20 10, 17, 24, 25

6, 7, 8,

Kinds of Forces

Gravitation

Contact

Verbal

Verbal

Verbal

Verbal

22, 26, 27

4, 15, 16, 28

1, 2, 3, 13

5, 11, 18, 29, 30

23

Some questions were classified into two dimensions by Hestenes et al. (1992). These questions were carefully considered in order to decide the most appropriate dimension. Question 27 (about slowing down due to friction) is classified as Newton's Second Law dimension in Table 1, whereas Hestenes et al. classified it as Kinds of Forces (Solid Contact). Two Solid Contact questions (5 and 18) have a dynamic situations and hence at least

11

The FCI as a Measure of Students’… implicitly involve Newton’s Second Law. In addition, question 8 demands quite complex reasoning at first in terms of Newton's Second Law, then the vector nature of velocity and finally Newton's First Law. To answer this question correctly with correct reasoning demands conceptual framework coherence. Clearly, the classifications in Table 1 are not sharp.

The superposition question (9) and diagrammatic Newton's Second Law question (21) are not included, since one question in those domains is not enough to allow evaluation of conceptual coherence. Questions 15 and 16 (Newton's Third Law) and also questions 26 and 27 (Newton’s Second Law) have the same general context but the contextual features (states of the systems) are different. For instance in question 15 the velocity of the car pushing the truck increases, while in question 16 it is constant. This change, which is irrelevant from the point of Newton's Third Law, was crucial for many students in previous studies (Hestenes et al.1992, Savinainen & Scott, 2002b). There are several questions in the FCI which are framed in the same contexts but they fall into different categories of representation and dimension of the force concept. From the point of view of the classification used, a clear majority of the FCI questions have different contexts. Hence, the FCI results can provide information on contextual coherence within verbal and diagrammatic representations in different dimensions of the force concept. Interestingly, Huffman and Heller (1995) give support to this conclusion with their claim that determining students' understanding of the force concept and students' familiarity with the context are inextricable in the case of the FCI. We would add that students' contextual coherence implies correct responses even in novel contexts.

It should be kept in mind, however, that a few multiple choice questions are not enough for drawing definite conclusions about students' contextual coherence. Multiple sources of

12

The FCI as a Measure of Students’… information, such as student writing, oral discussions and Socratic dialogue, are required for a reliable assessment of students' knowledge (Dufresne et al., 2002). In this study data complementary to the FCI were provided by interviews.

We have now characterized the FCI in terms of conceptual coherence. Using this characterization we now turn to the following questions:

1) To what extent does a group of high school students whose post-instruction FCI average was very high (over 80%) exhibit contextual coherence? 2) In what dimensions and representations of the force concept do the students show the strongest/weakest contextual coherence?

Research Method

Teaching Approach

The teaching approach used in this study - Interactive Conceptual Instruction (ICI) - and its rationale have been documented by Savinainen and Scott (2002b). Even though the teaching approach was proven to be relatively successful, the study revealed some dimensions of the force concept (e.g. Newton’s Third Law) which were not well understood after ICI teaching. Some of these deficiencies were taken into account by focusing on forces as interactions. The overall structure of ICI teaching was not changed. The revised ICI teaching and its effect on students' understanding of Newton’s Third Law are discussed in Savinainen, Scott and Viiri (2005).

13

The FCI as a Measure of Students’… Data Collection and Method of Analysis

Two groups of Finnish students in a Finnish upper secondary school (n = 27 and n = 22) followed a 27-hour course on high school mechanics taught by author AS in fall 2001. A Finnish translation of the 1995 version of the FCI was used (Halloun et al., 1995; Koponen et al., 2000) and was administered prior to, and on completion of, the teaching programme. In this paper we concentrate on the post-test data and do not focus on evaluating changes between the pre- and post-test results.

FCI Questions and Contextual Coherence The FCI results for different dimension and representation categories (Table 1) were classified into three levels of achievement in contextual coherence: I.

'no contextual coherence': zero or one question answered correctly

II.

'partial contextual coherence': at least two correct answers and at least one incorrect answer

III.

'contextual coherence': all questions answered correctly.

The third level of achievement is taken as an indication of contextual coherence in any given dimension and representation of the force concept. For instance, there are four FCI questions on Newton’s First Law using verbal representation in various contexts. Consequently, a student exhibits contextual coherence in verbal representation in the case of Newton’s First Law if he/she answers correctly all the four questions. The second level of achievement implies partial contextual coherence (e.g., one or two mistakes in the four Newton’s First Law questions using verbal representation), and the first level no contextual coherence (e.g., more than two mistakes in the above-mentioned example). This classification resembles Thornton's

14

The FCI as a Measure of Students’… (1995) three-fold classification (Student View, Transitional State and the Physicist View) to describe conceptual dynamics, i.e. the process by which students' views are transformed during instruction.

Interview Questions and Contextual Coherence Written questions derived from Reif (1995), McDermott et al. (1998) and Brown (1989) (see Appendix) were used as interview questions. These questions were chosen for two reasons: first, they allowed probing of students’ understanding of forces in different contexts, and secondly, the original forms of the questions had already gone through a validation process. Verbal representations of Newton's Second and Third Laws were evaluated in three different contexts, whereas Newton's First Law was evaluated in two contexts. All the cases which involved a zero net force were classified under Newton's First Law. The questions had three different contexts with varying contextual features (e.g. constant velocity vs. changing velocity). The classification of the interview questions in terms of conceptual dimensions is presented in Table 2.

Table 2.

Classification of the interview questions in terms of conceptual dimensions. All the dimensions involved only verbal representation. Newton’s First Newton’s Second Newton's Third Law Law Law 2 a) (iv) 1 b) 1 a) (i), (ii), (iii) 3 a) (iii) 2 b) (iv) 2 a) (iii); 2 b) (iii); 3 b) (iii) 3 c) (iii) 3 a) (i), (ii) 3 b) (i), (ii); 3 c) (i), (ii)

All the conceptual dimensions in Table 2 involved only verbal representation. In addition, questions 2 (a) (ii) and 2 (b) (ii) involved force diagrams. However, this is not enough to allow evaluation of contextual coherence within a diagrammatic representation. 15


Newton’s Third Law was hard for students in our earlier study (Savinainen & Scott, 2002b). The teaching in this study emphasized forces as interactions in order to help the students to come to terms with the third law. Hence, it was gratifying to find out that in this study the students answered all the FCI questions regarding Newton’s Third Law correctly. Given this background it was of interest to discover whether they could apply Newton's Third Law in more complex situations as well. Consequently, the third law was addressed eleven times in the interviews. Verbal representations of Newton's First and Second Laws were addressed three times. It is regrettable that we did not have more questions on these dimensions of the force concept.

There were similarities between the interview and FCI questions. The first interview question concerns a collision between a bug and a car: this maps well with the FCI question 4 in which a truck and a small car collide with each other. The correspondence is especially good because the forces due to contact interaction are already identified for the student in both questions. The second interview question is set in the context of an elevator descending with one crate on a top of another; a similar context is addressed in the FCI question 17 about one crate ascending in an elevator. The third interview question concerns a man pushing two crates in contact with each other (a three-body problem): the FCI has a woman pushing one crate (questions 25, 26 and 27) and a car pushing a large truck (questions 15 and 16). The main difference between the last two interview questions and the corresponding FCI questions, at least from the student’s point of view, is that the interview questions involve forces acting on the sub-systems (two crates as two sub-systems) of the larger system (two crates as one system). We believe that although there is not an exact correspondence between the FCI questions and the interview questions, the interview data provide information that

16

The FCI as a Measure of Students’… supplements the FCI results. Moreover, the interviews can be used to probe the limits of student understanding in more complex situations than those addressed by the FCI.

Six students were chosen for interviews on the basis of their performance in the preinstruction FCI as representative of the top, middle and bottom performance levels in the groups. These students were interviewed at the end of the teaching to provide triangulation for students’ conceptual coherence and to check the FCI data. A semi-structured interview was used. The students were asked to answer the questions while 'thinking aloud' during the interviews. Some students spontaneously wrote down formulas or diagrams. The students were free to change their answers during the interviews if they so wished. The interviews were conducted by author AS using neutral comments and questions to clarify the students' views.

Students' responses were classified using a scheme of levels of contextual coherence corresponding to that used for the FCI data: I.

'No contextual coherence': zero, one out of three, or fewer than four out of eleven answers with explanations are correct.

II.

'Partial contextual coherence': two out of three or four to eight out of eleven answers with explanations are correct.

III.

'Contextual coherence': all answers and explanations are correct.

Analysis and Evaluation of the post-instruction FCI Data

The groups were combined (n = 49) because the post-test results of the two groups A and B were very close to each other and the students had very similar pre-course curriculum

17

The FCI as a Measure of Students’… histories. The average post-instruction FCI score was 82% (s.d. 14%), which is near the limit for Newtonian thinkers (85%) proposed by Hestenes and Halloun (1995a): in fact, 53% of the students had a score over this limit. Figure 3 presents the post-instruction FCI results in terms students’ levels of contextual coherence within the verbal and diagrammatic representation categories. As described in the Data Collection and Method of Analysis section above, the level of contextual coherence refers to students’ ability to answer all the questions probing a certain dimension of the force concept correctly across different contexts while the representation is constant.

No contextual coherence

Contextual coherence

96

100

Percent of Students

Partial contextual coherence 86

80 60

43

57

55

49

45

43 33

40 20

0

39

24

14

8

55

45

0 4

0

8

6

0

Kinematics Diagram

Newton I Diagram

Newton I Verbal

Newton II Verbal

Newton III Verbal

Gravitation Verbal

Contact Verbal

Dimension and Representation

Figure 3.

Post-instruction FCI results in terms of achievement levels of contextual coherence.

The results are most impressive for Newton's First Law (verbal representation) and Newton's Third Law (verbal representation). One cannot claim, however, that the students mastered Newton's First Law, as fewer than half showed contextual coherence in diagrammatic representation. This is a clear indication that representation as well as context is important: a student may use verbal representation coherently across different contexts but still lack contextual coherence when diagrammatic representations are used.

18


Only half of the students showed contextual coherence in questions on Newton's Second Law. This is in good accordance with the Kinematics results. As mentioned earlier, Newton's Second Law involves the concept of acceleration, which is probed by three out of four questions in the Kinematics dimension. These results and those of the earlier study (Savinainen & Scott, 2002b) suggest that the Second Law is the hardest dimension of the force concept for high school students to master. However, there were only three items which directly tested the Second Law using verbal representation. More questions involving other representations would be needed to fully evaluate students’ contextual coherence as regards the Second Law.

Analysis of the Interview Data

The FCI and interview results regarding Newton's laws are presented in Figure 4: the students (labeled from 1 to 6) are represented in the boxes. The analyzed interview questions involved Newton’s laws and used verbal representation. Analysis was carried out separately by the authors. The few discrepancies in the classification of the achievement levels were resolved through discussions. Due to timetable constraints, students 1, 2 and 3 were interviewed before the post-instruction FCI whereas other students had their interviews after the post-instruction FCI. It is reasonable to suspect that these students got an additional chance to learn Newtonian mechanics before the post-instruction FCI especially because the correct answers were briefly explained at the end of the interview to students 1 and 2 (Student 3 took so much time in answering the questions that there was no time left to explain the correct answers.) However, this effect must have been miniscule because the change in pre/post-instruction FCI

19

The FCI as a Measure of Students’… score for these students was relatively small: student 1 improved his FCI score from 90% to 93% and student 2 from 50% to 60%. Achievement Level

III

II

1 3 4 6

1 4 5 6

2 5

2 3

1 5

post-FCI N1

Interview N1

1 2 3 4 5 6

post-FCI N2

2

1 3 4 5 6

3 4

2 3 5 6

I

Figure 4.

1 4

2 6

Interview N2

post-FCI N3

Interview N3

Students judged to be at each level of contextual coherence in questions using verbal representation in different dimensions of the force concept in the postinstruction FCI and interview data.

Both the post-instruction FCI data and interview questions on Newton’s First Law indicated that students had reached a good level of contextual coherence in questions using verbal representation. This is evident also in the following interview excerpt translated from Finnish (student 1, post-instruction FCI 93%):

Interviewer:

The elevator is going downward. It says in part 2a that the elevator is moving downward at constant speed. Compare accelerations of the crates A and B.

Student 1:

They have, uh, when it is going downward at constant speed; that is they both have zero acceleration. And and the net force, well the net force that A has, of course they both are zero [net forces on the crates] when a is zero since speed is constant. No acceleration.

According to the post-instruction FCI results in Figure 4, the teaching of Newton’s Second law was relatively ineffective for the students. In general, students showed a slightly better

20

The FCI as a Measure of Students’… understanding of the second law in the interviews than in the FCI. In particular, student 5 (post-instruction FCI 67%) showed very good understanding of Newton’s Second Law in the interview whereas she did significantly less well in the corresponding post-instruction FCI questions. It strongly suggests that she had sorted out the second law after taking the postinstruction FCI. On the other hand, student 4, who exhibited perfect understanding of Newton's Second Law in the FCI (90% total score), had some difficulty with the idea of net force acting on the sub-systems of the system as the following excerpt demonstrates:

Interviewer:

What about the net forces [after the student has drawn and explained his force diagrams for both crates A and B when the elevator is going down with decreasing speed]?

Student 4:

Well, there are net forces upward acting on both crates and then the net force and its magnitude, [hesitates a bit] it depends on mass, so they must be different, the net forces I mean.

Interviewer:

Which one has the greater net force? The question says A has a greater mass than B.

Student 4:

[looks at his force diagrams; the large crate A is on top of the small crate B] Yes, but A is acting on B, That is, at least I would think that A in principle is acting on B so it [refers to crate B] has a greater net force. But if we considered them as two separate objects, of course the one with greater mass has a greater net force.

Interestingly, he provides both correct and incorrect answers; it seems that he is distracted by the situation in which crate A is on top of crate B. This is why his interview views were not classified as 'contextual coherence' in the second law. Otherwise he answered the interview questions very well. There are no questions in the FCI which address net forces acting on subsystems. This is not surprising since one test or interview cannot exhaustively assess all aspects of the force concept.

In the post-instruction FCI all the students identified the correct answers to questions involving Newton's Third Law. As discussed earlier, the first interview question, which

21

The FCI as a Measure of Students’… involved identifying the magnitudes of the interaction pair of forces in a collision, is fully analogous to FCI question 4. It was very well answered: only one student gave an incorrect answer. However, the students were not as successful in more complex interview questions. Question 2b (ii) (drawing force diagrams for both crates when the elevator is going down with decreasing speed) was especially demanding: four out of six students failed to apply the Third Law correctly. As an example, the force diagrams that student 6 (who scored 70% in the FCI) drew for question 2b are shown in Figure 5.

m1 g

Crate A

N1

Figure 5.

m1 g

Crate B

N2

m2 g

Force diagrams for interview question 2 b) (ii) drawn by student 6.

She explained her thinking while drawing the diagrams. She realized that the forces cannot now balance each other since there is acceleration but according to her diagram the crates would have increasing instead of decreasing speed. She also figured out that crate A exerts a force on crate B but incorrectly identified this force as the weight of crate A and hence missed the correct interaction force pair (that is, N1 and its reaction force acting on crate B). This suggests that student 6 lacked both contextual coherence (i.e., failed to apply Newton’s Third Law in this context) and conceptual framework coherence (i.e, did not use Newton’s Second and Third laws correctly together in this situation). It is worth noting that the FCI questions on Newton’s Third Law identify the forces due to interaction for the student, whereas the

22

The FCI as a Measure of Students’… students had to figure out the relevant forces by themselves in the interview questions 2 and 3 and then identify their relative magnitudes. This is clearly a more demanding task.

Discussion

The research questions in this study concerns the extent of conceptual coherence across different dimensions and representations of the force concept in a high school group with a high post-instruction FCI average (over 80%). The students showed very well-developed contextual coherence at the end of the teaching as regards Newton's First and Third Laws in questions using verbal representation: 86% and 96 %, respectively. The students showed somewhat weaker contextual coherence in Kinematics and Newton's Second Law. From the point of view of physics theory, it is understandable that students who have difficulties with kinematics cannot master Newton's Second Law either, since it is intimately linked with the concept of acceleration. The percentage of students reaching contextual coherence in other dimensions than Newton’s First and Third Law varied from 43% to 59%. These percentages might appear surprisingly low when compared with a high post-instruction FCI average (82%). On the other hand, the percentage of so-called ”confirmed Newtonian thinkers or experts” (53%) is in good agreement with the percentage of students judged to have reached contextual coherence. This seems to suggest that the expert limit proposed by Hestenes and Halloun (1995a) is justified from the point of view of contextual coherence. More research, however, is needed to fully support this conclusion.

The interviews conducted in this study provide some insights into the relationship between students’ performance on the FCI and in situations where they need to produce the answers themselves. On one hand, the interviews in this study suggest that if a student does well in the

23

The FCI as a Measure of Students’… FCI, he/she generally also demonstrates good command of the Newtonian view in the interview situation as well. This lends support to the validity of the FCI as a measure of students’ understanding of force. On the other hand, the interview results suggest that the identification of a correct answer in the FCI is not necessarily enough for good performance in a more open and complex situation, and this was especially evident in the case of Newton’s Third Law. This should not come as a surprise if the FCI is interpreted as a minimum competence test in the domain of force. It should be noted, however, that in this study only six students were interviewed, so caution must be exercised when interpreting the results.

Contextual coherence is the main aspect of conceptual coherence which can be probed using the FCI. Representational coherence can be addressed only for Newton’s First Law since only in this dimension are there enough FCI questions with both verbal and diagrammatic representations. Full evaluation of representational coherence is not possible even in this dimension since the FCI questions do not involve graphical representation. The third aspect of conceptual coherence, framework coherence, is explicitly addressed only in questions 8, 19 and 20, which involve differentiation between position, velocity and acceleration. One can argue, however, that many questions actually demand conceptual framework coherence, at least implicitly: for instance, Newton’s Second Law cannot be understood without kinematics.

The analysis of the FCI results in terms of conceptual coherence revealed detailed information on the strengths and weaknesses of students’ understanding of the force concept. This can be used to plan future teaching to address the weaknesses. For instance, more emphasis should be placed on teaching diagrammatic representation of Newton’s First Law, since it seems to be harder to master than verbal representation. On the other hand, mastery in verbal representation may be partially due to the teaching approach used in this study, which

24

The FCI as a Measure of Students’… employed peer discussions, as a result of which students got more practice in verbal than in diagrammatic representation.

In conclusion, we have provided evidence based on the analysis of the student data that the FCI can indeed be used, at least to certain extent, to evaluate the coherence of students' understanding of the force concept. Hence, our results provide counter-evidence to Huffman and Heller’s (1995) conclusion that the FCI just measures ”bits and pieces of student knowledge”. Their conclusion was based on factor analysis, and it would have been interesting to perform factor analysis on our data to find out whether it identified significant factors. Unfortunately, our data set is not extensive enough for this purpose. It would, however, be interesting to run various forms of factor analysis with a larger data set and compare their outcomes with results obtained using our analysis of conceptual coherence.

Acknowledgements

We wish to express our gratitude to Professor Philip Scott, whose input was crucial in defining the aspects of conceptual coherence. Author AS worked closely with him as a Marie Curie Fellow in spring 2001 at the University of Leeds, UK. We also wish to thank Professor Robin Millar and Dr. Charles Henderson for their clarifying comments.

25

The FCI as a Measure of Students’… APPENDIX. Interview questions.

1. A bug hits the windshield of a car travelling at 80 km/h along a highway. a) Is the magnitude of the force exerted on the bug by the car larger than, smaller than, or equal to the magnitude of the force on the car by the bug? Describe your reasoning in reaching your answer. Consider three instants of time: (i)

just when the collision starts

(ii)

in the middle of the collision

(iii)

just before the collision ends.

b) As a result of this collision, is the acceleration of the bug larger than, smaller than, or equal to the acceleration of the car? Describe your reasoning in reaching your answer.

2. Two crates, A and B, are in the elevator (crate A on the top of crate B; a diagram of the situation was included). The mass of crate A is greater than that of crate B. a) The elevator moves upward at constant speed. (i)

How does the acceleration of crate A compare with that that of crate B? Explain.

(ii)

Draw and label separate free-body diagrams for the crates.

(iii)

Rank the forces on the crates according to magnitude, from largest to smallest. Explain your reasoning.

(iv)

Consider the direction and magnitude of the net force acting on crate A. Consider the direction and magnitude of the net force acting on crate B. Compare the magnitudes of the net forces.

b) As the elevator approaches its destination, its speed decreases while it continues to move downward. The same questions were asked as in case (a).

26

The FCI as a Measure of Students’… 3. A man is pushing two crates in contact with each other in the World's Strongest Man competition. (A diagram of the situation was included.) The bigger crate has mass of 140 kg and the smaller crate has mass of 70 kg. The mass of crate A is greater than that of crate B. Consider the following situations. a) The crates do not move. (i)

Compare the forces that the 140 kg and the 70 kg boxes exert on each other.

(ii)

Compare the forces that the man and the 140 kg box exert on each other.

(iii)

Compare the net forces acting on the crates.

b) The crates are moving at constant velocity. (i)

Compare the forces that the 140 kg and the 70 kg boxes exert on each other.

(ii)

Compare the forces that the man and 140 kg box exert on each other.

(iii)


c) The crates are moving at constantly increasing velocity. (i)

Compare the forces that 140 kg and 70 kg boxes exert on each other.

(ii)

Compare the forces that the man and 140 kg box exert on each other.

(iii)


27

The FCI as a Measure of Students’… References

Brown, D. (1989). Students’ Concept of Force: The importance of Understanding Newton’s Third Law, Physics Education, 24, 353-358. Dufresne, R.J., Leonard, W.J. and Gerace, W.J. (2002). Making Sense of Students' Answers to Multiple-Choice Questions, Physics Teacher, 40, 174-180. Finegold, M. and Gorsky, P. (1991). Students’ Concepts of Force as Applied to Related Systems: A Search for Consistency, International Journal of Science Education, 13 (1), 97-113. Hake, R. (1998). Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, American Journal of Physics 66, 64-74. Hake, R. (2002). Lessons from the physics education reform effort, Conservation Ecology 5(2), 28. Online at . Accessed on 9.6.2007. Halloun, I. and Hestenes, D. (1985). The initial knowledge state of college physics students, American Journal of Physics, 53, 1043-1055. Halloun, I., Hake, R., Mosca, E. and Hestenes, D. (1995). Force Concept Inventory (Revised 1995). Password protected at . Accessed on 9.6.2007. Hestenes, D. (1992). Modeling Games in the Newtonian World. American Journal of Physics, 60: 732-748. Hestenes, D., Wells, M. and Swackhamer, G. (1992). Force Concept Inventory, Physics Teacher, 30, 141-158.

28

The FCI as a Measure of Students’… Hestenes, D. and Halloun, I. (1995a). Interpreting the Force Concept Inventory, Physics Teacher, 33, 502-506. Online at . Accessed on 9.6.2007. Hestenes, D. and Halloun, I. (1995b). The search for conceptual coherence in FCI data, Working paper. Online at . Accessed on 9.6.2007. Huffman, D. and Heller, P. (1995). What does the Force Concept Inventory Actually Measure? Physics Teacher, 33, 138-143. Koponen, I., Jauhiainen, J. and Lavonen, J. (2000). A Finnish translation of the 1995 version of the Force Concept Inventory, available upon request. Department of Physics, University of Helsinki. McDermott, L. (1993). Guest Comment: How we teach and how students learn – A mismatch? American Journal of Physics, 61 (4), 295-298. McDermott, L., Schaffer, P. and the Physics Education Research Group (1998). Tutorials in Introductory Physics. Homework. Preliminary Edition. Prentice Hall, USA. Meltzer, D. (2002). Student learning of physics concepts: Efficacy of verbal and written forms of expression in comparison to other representational modes. Online at . Accessed on 18.6.2007. Mildenhall, P. and Williams, J. (2001). Instability in students' use of intuitive and Newtonian models to predict motion: the critical effect of parameters involved, International Journal of Science Education, 23, 643-660. Palmer, D. (1994). The effect of direction of motion on students' conceptions of forces. Research in Science Education, 24, 253-260. Reif, F. (1987). Instructional design, cognition, and technology: Application to the teaching of scientific concepts, Journal of Research in Science Teaching, 24, 309-324.

29

The FCI as a Measure of Students’… Reif. F. (1995). Understanding Basic Mechanics. Workbook. John Wiley & Sons, USA. Savinainen, A. (2004). High school students’ conceptual coherence of qualitative knowledge in the case of the force concept. Dissertations 41, Department of Physics, University of Joensuu.

A

link

to

the

electronic

publication

is

provided

at

. Accessed on 9.6.2007. Savinainen, A. and Scott, P. (2002a). The Force Concept Inventory: a tool for monitoring student learning, Physics Education, 37, 45–52. Online at . Accessed on 9.6.2007. Savinainen, A. and Scott, P. (2002b). Using the Force Concept Inventory to monitor student learning and to plan teaching, Physics Education, 37, 53-58. Online at . Accessed on 9.6.2007. Savinainen, A., Scott, P. and Viiri, J. (2005). Using a bridging representation and social interactions to foster conceptual change: Designing and evaluating an instructional sequence for Newton’s third law. Science Education, 89, 175 -195. Preprint is online at . Accessed on 9.6.2007. Schecker, H. and Gerdes, J. (1999). Messung von Konzeptualisierungsfähigkeit in der Mechanik: Zur Aussagekraft des FCI, Zeitschrift für Didaktik der Naturwissenschaften, 5 (1), 75-89. Steinberg, R. and Sabella, M. (1997). Performance on multiple-choice diagnostics and complementary exam problems, Physics Teacher, 35, 150-155. Thornton, R. (1995). Conceptual dynamics: Changing students views of force and motion. In C. Tarsitani, C. Bernandini and M. Vincentini (Eds.). Thinking Physics for Teaching. Plenum, London, 157-183. Van Heuvelen, A. (1991). Overview, Case Study Physics, American Journal of Physics, 59 (10), 898-907.

30