Development and application of a diagnostic instrument to evaluate ...

JOURNAL OF RESEARCH IN SCIENCE TEACHING

VOL. 26, NO. 4, PP. 301-314 (1989)

DEVELOPMENT AND APPLICATION OF A DIAGNOSTIC INSTRUMENT TO EVALUATE GRADE-11 AND -12 STUDENTS’ CONCEPTS OF COVALENT BONDING AND STRUCTURE FOLLOWING A COURSE OF INSTRUCTION RAYMOND F. PETERSON AND DAVID F. TREAGUST Curtin University of Technology, Perth, Western Australia, 6102

PATRICK GARNETT Western Australian College of Advanced Education, Perth, Western Australia, 6009

Abstract This article initially outlines a procedure used to develop a written diagnostic instrument to identify grade-1 1 and -12 students’ misconceptions and misunderstandings of the chemistry topic covalent bonding and structure. The content to be taught was carefully defined through a concept map and propositional statements. Following instruction, student understanding of the topic was identified from interviews, student-drawn concept maps, and free-response questions. These data were used to produce 15 two-tier multiple-choice items where the first tier examined content knowledge and the second examined understanding of that knowledge in six conceptual areas, namely, bond polarity, molecular shape, polarity of molecules, lattices, intermolecular forces, and the octet rule. The diagnostic instrument was administered to a total of 243 grade11 and -12 chemistry students and has a Cronbach alpha reliability of 0.73. Item difficulties ranged from 0.13 to 0.60; discrimination values ranged from 0.32 to 0.65. Each item was analyzed to ascertain student understanding of and identify misconceptions related to the concepts and propositional statements underlying covalent bonding and structure.

Introduction Over the past decade there has been increased interest in students’ understanding and misunderstanding of science phenomena either prior to or following instruction. Research evidence has demonstrated that students often have views of science concepts that differ from those generally accepted by the science community. These alternative student views have been labeled, among other terms, as “children’s science” (Gilbert, Osborne, & Fensham, 1982; Osbome, Bell, & Gilbert, 1983), and “alternative frameworks’’ or “misconceptions”(Driver & Easley, 1978). In this article the term misconception refers to students’ ideas following instruction that are different from the acceptable and intended scientific viewpoint. 0 1989 by the National Association for Research in Science Teaching CCC 0022-4308/89/040301-14$04.00 Published by John Wiley & Sons, Inc.

302

PETERSON, TREAGUST, AND GARNETT

Of the many approaches used to identify students’ understanding and misconceptions of science concepts and phenomena, interview methodologies have acquired strong support as a viable approach (Osborne & Gilbert, 1980; Watts, 1981). However, although interviews with students have been successful in ascertaining students’ understanding of science phenomena, this methodology has possible limitations for use by classroom teachers. This is in part due to the teacher time required to administer individual interviews and the fact that many science teachers are not trained to conduct interviews, to record and transcribe interview data, or to interpret findings (Fensham, Garrard, & West, 1981). To assist the classroom teacher in identifying misconceptions of science phenomena, it is necessary to develop methodologies that can be readily used by teachers in their classroom environments. One method that would enable science teachers to measure students’ understanding of concepts and identify misconceptions is to have an easily administered and scored pencil-and-paper instrument based on multiple-choice items. Multiple-choice items have previously been used to measure students’ understanding of science subject matter. However, test items have generally evaluated students’ misconceptions of subject content knowledge (see for example, Doran, 1972; Duncan & Johnstone, 1973; Driscoll, 1974) and have not specifically considered students’ reasoning behind their content choices. Tamir (197 1) and more recently Treagust (1986, 1987) have recommended the use of alternative multiple-choice items to questions specifically based on student reasoning, including known misconceptions. This methodology was utilized by Peterson (1986) to examine secondary students’ understanding of covalent bonding and structure concepts.

Purpose While a large number of research investigations completed in physics have identified students’ misconceptions of particular concepts, few studies have attempted to examine students’ understanding and identify misconceptions of chemistry concepts (Pfundt & Duit, 1988). The research reported helps address the situation by (1) describing the development of a test instrument to diagnose grade-1 1 and -12 students’ understanding of the chemistry topic covalent bonding and structure following a course of instruction, and (2) highlighting common misconceptions or misunderstandings identified through the administration of the diagnostic instrument. The topic of covalent bonding and structure was selected for this study, as it had previously been identified by the authors in an investigatory study as an area of conceptual difficulty for grade-11 and -12 students.

Methods and Procedures Prior to the development of the items, the content boundaries for the topic covalent bonding and structure were defined through a concept map and a list of propositional statements. Diagnostic items were prepared on areas of conceptual difficulty and students’ misconceptions and misunderstandings initially identified in regular classroom teaching. These areas of conceptual difficulty and student misconceptions were further investigated from unstructured interviews, analyses of student-drawn concept maps, and openended pencil-and-paper test items.

DIAGNOSTIC INSTRUMENT

303

Identification of Concepts and Propositional Statements

The necessary concepts that comprised the topic of covalent bonding and structure were identified and arranged hierarchically in a concept map accordingto the procedures recommended by Novak (1980). Propositional statements were also written for the concepts relevant to the topic. Both the concept map and the propositional statements were prepared at a level of sophistication appropriate to the understanding required by grade-11 and -12 secondary students taking chemistry. The propositional statements were related to the concept map to ensure that the content was internally consistent. Six science educators (two university science lecturers, two experienced chemistry teachers, and two chemistry curriculum specialists) independently content-validated both the concept map and the list of propositional statements. Based on the reviewers’ comments a list of 33 propositional statements was prepared (see Table I). Instrument Development

Items for the diagnostic instrument were based on the two-tier multiple choice format described by Treagust (1986, 1987). The first tier of each item consists of a content question having two, three, or four choices; the second part of each item contains four possible reasons for the answer given in the first tier. Included in these reasons are the correct answer and three identified alternative reasons involving misconceptions. Items were based on areas of conceptual difficulty identified from studentdrawn concept maps and unstructured interviews. For example, the influence of nonbonding electron pairs and determining the shape of a molecule from a given formula were both cited by students as conceptual difficulties when predicting a molecule’s shape. These two areas of conceptual difficulty were used in the development of two items in the diagnostic instrument. Incorrect reasons for each item were based on students’ alternative views of concepts and propositions identified by interviews and by administering an open-response version of the diagnostic instrument where students were required to supply a reason for their content choice. Three pilot studies were conducted to develop, test, and refine items for the diagnostic instrument. The Covalent Bonding and Structure diagnostic instrument consists of 15 items, each of which refer specifically to the conceptual areas of bond polarity, molecular shape, polarity of molecules, lattices, intermolecular forces, and the octet rule. An example of an item testing an understandingof lattices and another testing an understanding of the shape of molecules are presented in Figure 1. Most items were based on problem-centered situations and tested a limited number of the previously content-validated propositional statements (Table II). Two experienced chemistry teachers and a tertiary chemistry educator reviewed the final items of the diagnostic instrument. Subjects

In South Australia, chemistry is a two-year upper-school course, though only grade 12 is assessed for entrance to tertiary institutions. Students taking chemistry in grades 1 1 and 12 do so on a selective basis and are usually the more able students of those completing grade 10. While all students study basic chemistry concepts as part

PETERSON, TREAGUST, AND GARNETT

304

TABLE I List of Propositional Statements Developed for the Topic of Covalent Bonding and Structure 1.

Covalent bonds form between atoms of non-metals.

2.

A single covalent bond forms through the sharing of an electron pair by two atoms of

non-

metals, each atom contributing one electron to the shared pair. 3.

The bond i s due to the resultant force of attraction and repulsion between the positive nuclei of each atom and the shared electron pair.

4.

The shared electron pair is located between the nuclei of the atoms involved in the bond.

5.

The position of the electron pair is dependent upon the electronegativity o f each non-metal atom in the bond.

6.

Electronegativity is the relative attracting power of a bonded atom for the pair of electrons it is sharing with another atom.

7.

Equal sharing of the bonding electron pair occurs between atpms with the same electronegatrvrty.

A.

Equal sharing results in the electron pair being centrally located between the non-metal atoms.

9.

Equal sharing of the bonding electron pair results in a non-polar covalent bond.

10.

Unequal sharing o f the bonding electron pair results in a polar covalent bond.

11.

Polar covalent bonds occur between atoms with different electronegativities.

12.

The shared electron pair in a covalent bond is located closer to the atom with the higher electronegativity.

13.

The unequal sharing results in one nuclei being slightly negatively charged. the other slightly positively charged.

14.

The polarity of a polar covalent bond is symbolised by a partial positive charge (6+) and a partial negative charge (6-).

15.

The partial negative symbol is assigned to the atom with the pnrtial negatlve charge. which is also the atom with the greater electronegativity value.

16.

Multiple covalent bonds metallic atoms.

17.

Multiple covalent bonds can be either polar or non-polar.

form when more than one pair o f electrons are shared

between

two non-

18. The octet rule is a useful guide in determining the number of covalent bonds an atom can form. 19.

The octet rule states that atoms share electrons to acquire the equivalent of eight electrons or four electron pairs in their outer valence shell.

20.

Either discrete molecules or continuous lattices can be formed from covalent bonding.

21. Diamond exists as a three dimensional continuous covalent lattice. 22.

Graphite exists as a two dimensional continuous covalent lattice.

23.

Strong bonding forces throughout the continuous covalent lattice are indicated by the high melting and boiling points of these substances.

24.

The shape of simple molecules of the form A X can be predicted and explained by the Valenc:

25.

The V.S.E.P.R. Theory states that the valence electron pairs (bonding and non-bonding) in the valence shell o f an atom are repelled as far from each other as possible.

26.

The actual shape of a molecule is dependent on the total number of bonding and non-bonding pairs of electrons surrounding the central atom of the molecule.

27.

Molecules can be either polar or non-polar.

(n < 4). containing either single or multiple bonds, Shell Electron Pair Repulsion Theory (V.S.E.P.R.).

28. Polar molecules contain polar bands in a non-symmetrical shape. 29. Non-polar molecules contain non-polar bonds

or

polar bonds in a symmetrical shape.

30.

Intermolecular forces

31.

Whether a substance composed of molecules exists as a solid, liquid or gas at room temperature will depend on the magnitude of the intermolecular forces between molecules.

12.

Stronger intermolecular forces exist between polar molecules than non-polar molecules of the same approximate size.

33.

Molecular substances with high mqltrng and boiljng points have strong intermolecular forces between molecules.

o r weak forces of attraction exist to varying degrees between molecules.

DIAGNOSTIC INSTRUMENT

305

Item Testing L a t t i c e s The c o m m e r c i a l l y a v a i l a b l e s u b s t a n c e ‘Vase1 i n e ’ has a smooth, t h i c k crearrll i k e texture. Based on t h i s . V a s e l i n e would be c l a s s i f i e d as b e i n g a

(1) (2)

c o v a l e n t molecular substance c o v a l e n t network ( c o n t i n u o u s c o v a l e n t ) substance

Reason

(A)

The s u b s t a n c e has a c o n t l n u o u s l i n e a r l a t t i c e s t r u c t u r e .

(B)

The h i g h v i s c o s i t y o f t h e substance r e s u l t s from t h e c o n t i n u o u s c o v a l e n t network.

(C)

The m o l e c u l e s i n t h e substance e x p e r i e n c e weak i n t e r m o l e c u l a r f o r c e s and e a s i l y move t o accommodate changes i n t h e shape o f t h e s o l i d .

(D)

The bonds w i t h i n t h e m o l e c u l e s o f t h e s u b s t a n c e break e a s i l y t o accommodate t h e changes i n t h e shape o f t h e s o l i d .

I t e m T e s t i n g t h e Shape pf M o l e c u l e s N i t r o g e n ( a g r o u p 5 e l e m e n t ) combines w i t h bromine ( a group 7 e l e m e n t ) t o form a molecule. T h i s m o l e c u l e i s l i k e l y t o have a shape w h i c h i s be5t d e s c r i b e d as (1)

(2) (3)

trigonal planar t r i g o n a l pyramidal tetrahedral

Reason

(A)

N i t r o g e n forms t h r e e bonds w h i r h e q u a l l y r e p e l e a r h o t h e r t r i g o n a l p l a n a r shape.

(B)

The t e t r a h e d r a l arrangement o f t h e b o n d i n g and non-bondlny t,lc,rtrori p a i r s around n i t r o g e n r e s u l t s i n t h e shape o f t h e m o l e r u l e .

(C)

The p o l a r i t y o f t h e n i t r o g e n - b r o m i n e bonds d p t e r m i n r s t h p shnpt, o f t b r v molecule.

(D)

The d i f f e r e n c e i n e l e c t r o n e g a t i v i t y v ~ ~ l u cfor ~ \ broniine dnd determinc, t h e shape o f t h e m o l e c u l e .

t o forin

t i i t

,i

roi,t’i

Fig. 1. Items testing lattices and the shape of molecules.

of general science in grades 8-10, for those students selecting to do further study in chemistry the basic concepts of covalent bonding and structure are first introduced in grade 11 and consolidated in grade 12. The 15-item diagnostic instrument Covalent Bonding and Structure was administered to 159 grade-1 1 (15-16 years old) and 84 grade-12 (16-17 years old) chemistry students in five coeducational high schools in South Australia. Both groups of students had completed six to seven months of chemistry instruction, which included the topic covalent bonding and structure. Scoring of Items

A student’s answer to an item was considered correct if the student selected both the correct content choice and the correct reason. Items of the diagnostic instrument were evaluated for both correct and incorrect response combinations selected by the

PETERSON, TREAGUST, AND GARNETT

306

TABLE I1 Propositional Statements and Topic Areas Tested by Each of the 15 Items in the Diagnostic Instrument, Covalent Bonding and Structure I t em Nuiii be r

Propositional Statements

Topic Area

1

Borid Polarity

4. 5. 6. 1 1 . 12

2

Molecul ar Shape

18. 19, 24, 25. 26

3

Bond Polarity

1 1 , 12. 13. 14, 15

4

Lattices

23

5

Molecular Shape

18. 19, 24. 25, 26

6

Polarity

1 1 , 27. 28. 29

7

Intermolecular Forces

30. 31, 33

8

Nolecul ar Shape

24, 25, 26

9

Polarity

1 1 , 27. 28

of

of

Molecules

Molecules

10

Octet Rule

18, 19

11

Latt1ces

30. 37. 33

12

Polarity o f Molecules Intermolecular Forces

1 1 , 27. 28, 29. 30. 37

13

Molecular Shape

24. 25. 26

14

Bond Polarity

10. 11. 12, 13, 14, 1 5

15

Molecular Shape Octet R u l e

18. 19. 24. 25. 26

students. For example, the response combinations selected by grade-11 and - 12 students in the sample for item 1 dealing with bond polarity are shown in Figure 2. Analysis of incorrect response combinations provided data on students’ misconceptions of concepts and propositions related to that item.

Results and Discussion A summary of the test characteristics of the Covalent Bonding and Structure instrument are presented in Figure 3. The reliability of the instrument as measured by Cronbach alpha was 0.73 when both the content and reason parts of the item were analyzed. Difficulty indices ranged from 0.13 to 0.60, providing a wide range of difficulty in the items. Discrimination indices ranged from 0.32 to 0.65 and those greater than 0.30 were considered acceptable without the need for further revision of the test items (Lien, 1971).

DIAGNOSTIC INSTRUMENT

307

To establish an overview of students' knowledge of covalent bonding and structure at the grade-1 1 and -12 level it was necessary to determine the criteria appropriate to assess students' understanding of the conceptud areas being tested. Based on Gilbert's (1977) work using multiple-choice items with four or five distractors, student understanding of the conceptual area was considered satisfactory if more than 75% of the students correctly answered the item. With less than four distractors, a higher percentage mean of correct responses would be seen as representing mastery of the item. The percentage of students in grades 11 and 12 correctly answering each item is presented in Table 3. For each item, a higher percentage of students made the correct content choice than provided the correct content with correct reason. Indeed, taking into account the guessing probability one would expect higher correct responses on the first tier with two or three distractors than for a combination of first- and second-tier responses when the second tier has four responses. When only the first part of the item, the content

Which o f t h e f o l l o w i n g b e s t r e p r e s e n t s t h e p o s i t i o n o f t h e s h a r e d e l e c t r o n p a i r i n t h e HF m o l e c u l e ?

(1)

H

(2)

: F

H

F

:

Reason

(A)

Non-bonding e l e c t r o n s i n f l u e n c e t h e p o s i t i o n o f t h e b o n d i n g o r shared electron pair.

(B)

As hydrogen and f l u o r i n e f o r m a c o v a l e n t bond t h e e l e c t r o n p a i r must be c e n t r a l l y located.

(C)

F l u o r i n e has a s t r o n g e r a t t r a c t i o n f o r t h e shared e l e c t r o n p a i r .

(D)

F l u o r i n e i s t h e l a r g e r o f t h e two atoms and hence e x e r t s g r e a t e r c o n t r o l o v e r t h e shared e l e c t r o n p a i r .

Percentage o f S t u d e n t Responses Reason Grade

11

12

*

Choice t o f i r s t part

A

B

C

D

Total

1

1.3

2.5

43.4*

5.0

52.2

2

6.3

33.3

8.2

0.0

47.8

1

0.0

3.7

61.4"

2.4

67.5

2

2.4

22.9

7.2

0.0

32.5

C o r r e c t response c o m b i n a t i o n

Fig. 2. The percentage of grade-11 (n = 159) and grade-12 (n = 84) chemistry students selecting each response combination for item 1 of the Covalent Bonding and Structure diagnostic instrument.

PETERSON, TREAGUST, AND GARNETT

308 Areas E v a l u a t e d

Bond D o l a r i t v - I t e m s 1.. 3.. 14 L a t t i k e s - I t e m s 4, 11 M o l e c u l a r shape - I t e m s 2, 5, 8. 13, 15* P o l a r i t v o f m o l e c u l e s - I t e m s 6.. 9.. 12" O c t e t r i l e - I t e m s 10, 15* I n t e r m o l e c u l a r Forces - I t e m s 7. 1236

Content based on

V a l i d a t e d p r o p o s i t i o n a l s t a t e m e n t s and c o n c e p t map

Number o f i t e m s

15

Response Format

Two t i e r m u l t i p l e c h o i c e F i r s t t i e r - c o n t e n t knowledge Second t i e r - reasons f o r c o n t e n t response

Recommended grade l e v e l :

Grade 11 and 12 c h e m i s t r y

Time t o complete t e s t

:

25 - 40 m i n u t e s

Discrimination Indices

:

Mean

Range

0.46

0.30 - 0.39 ( 4 i t e m s ) 0.40 - 0.49 ( 6 i t e m s ) 0.50 - 0.59 ( 4 i t e m s ) 0.60 - 0.70 ( 1 i t e m )

Mean

Range

0.35

0.10

D i f f i c u l t y Indices

- 0.19 ( 2 i t e m s 0.20 - 0.29 ( 2 i t e m s

0.30 - 0.39 0.40 - 0.49 0.50 - 0.59 0.60 - 0.69 Cronbach Alpha Reliability

( 6 items ( 3 items (1 item) ( 1 item)

: 0.73

Note These two i t e m s t e s t e d more t h a n one c o n c e p t u a l a r e a

Fig. 3. Summary of characteristics for the diagnostic instrument Covalent Bonding and Structure.

part, was considered, students' correct responses on the 15 items of the diagnostic instrument, as shown in Table 111, ranged from 17% to 99% for grade 11 and from 40% to 100% for grade 12. When both parts of each item were considered, the correct responses were much reduced to a range of 10% to 57% for grade 11 and 22% to 78% for grade 12. Accordingly, the results from the two-tier items in Table I11 suggest that grade11 and -12 chemistry students in this sample did not acquire a satisfactory understanding of the concepts being evaluated in the topic. A comparison of the percentage of students who correctly answered the content part of the item compared with the number of students correctly answering the complete two-tier item suggests that students may have acquired accurate content responses without an adequate understanding of the concepts involved. Grade- 12 students' responses to the diagnostic instrument indicated that they had a better understanding of the areas tested than did the grade-1 1 students.

DIAGNOSTIC INSTRUMENT

309

Such a finding was to be expected, given the manner in which topics were introduced in grade 11 and consolidated in grade 12. Student’s selection of an incorrect content choice and/or incorrect reason choice indicated the existence of misconceptions of covalent bonding and structure. For the purposes of this article, misconceptions have been listed if they existed in at least 20%of the student sample. Thirteen misconceptions, identified through analysis of the items from the diagnostic instrument and shown in Table IV, are grouped under the headings of bond polarity, molecular shape, polarity of molecules, octet rule, intermolecular forces, and lattices. Bond Polarity. Student’s ability to establish the correct polarity of a bond was unexpectedly low, as measured through items 1, 3, and 14. For instance, in item 1, 43% of grade-11 students and 61% of grade-12 students could correctly ascertain the position of the shared electron pair in the H-F bond and give the correct reason for their choice. Analysis of the alternative responses provided the three misconceptions

TABLE 111 The Percentage of Grade-1 1 (n = 159) and Grade-12 (n = 84) Chemistry Students Correctly Answering the First Part and Both Parts of the 15-Item Diagnostic Instrument, Covalent Bonding and Structure Percentage o f g r a d e 11 s t u d e n t s c o r r e c t l y answering

Item Numbered

_________________-first part

both parts

Percentage o f g r a d e 12 s t u d e n t s c o r r e c t l y answering

___-____-____--___first part

_ _ ~ _ _

both parts

____

1

52

43

68

61

2

35

13

63

38

3

53

25

79

60

4

99

26

100

49

5

55

30

68

47

6

79

28

76

49

7

87

11

98

33

8

85

40

90

42

9

55

31

60

46

10

85

57

90

14

11

56

37

79

63

12

46

27

50

42

13

82

41

94

78

14

71

23

80

58

15

17

10

40

22

310

PETERSON, TREAGUST, AND GARNElT

TABLE IV

Percentage of Grade-11 and -12 Students with Specific Misconceptions Identified from the Covalent Bonding and Structure Diagnostic Instrument Misconceptions

Percentage Grade 1 1 Grade 12 (n = 159) (n = 84)

Bond Polarity 1.

2. 3.

Equal sharing o f the electron pair occurs in all 33 covalent bonds. The polarity of a bond i s dependent on the number o f 22 valence electrons in each atom involved in the bond. Ionic charge determines the polarity of the bond. 26

23 -

-

Molecular S h a p e

1.

2. 3.

46 The shape o f a molecule is due to equal repulsion between the bonds. Bond polarity determines the shape cf a molecule. 23 The V-shape in a molecule of the type SC12 is due to repulsion between the non-bonding electron pairs.

25 27 22

Intermolecular Forces

1. ?.

3.

Intermolecular forces are the forces within a mo 1 ecu 1 e Strong intermolecular forces e x i s t in a continuous covalent solid. Covalent bonds are broken when a substance changes shape.

.

-

23

48

33

49

-

40

34

32

-

-

20

27

-

Polarity of molerules

1. 2.

Non-polar molecules form when the atoms in the molecule have similar electrone ativities. Molecules o f the type O F ? a r e pojar as the nonbonding electrons on the oxygen form a partial negative charge.

Ortet Rule

1.

Nitroqeri dtorris

ran

share 5 electron pairs in bonding

Ldttire