Please cite this article as: Levy, D. (2013). How Dynamic Visualization Technology can Support Molecular Reasoning. Journal of Science Education and Technology 22(5): 702-717. DOI 10.1007/s10956-012-9424-6 .
How Dynamic Visualization Technology can Support Molecular Reasoning Dalit Levy Kibbutzim College of Education
[email protected]
Abstract This paper reports the results of a study aimed at exploring the advantages of dynamic visualization for the development of better understanding of molecular processes. We designed a technology-enhanced curriculum module in which high school chemistry students conduct virtual experiments with dynamic molecular visualizations of solid, liquid and gas. They interact with the visualizations and carry out inquiry activities to make and refine connections between observable phenomena and atomic level processes related with phase change. The explanations proposed by 300 pairs of students in response to pre/post assessment items have been analyzed using a scale for measuring the level of molecular reasoning. Results indicate that from pre-test to post-test, students make progress in their level of molecular reasoning, and are better able to connect intermolecular forces and phase change in their explanations. The paper presents the results through the lens of improvement patterns and the metaphor of the "ladder of molecular reasoning," and discusses how this adds to our understanding of the benefits of interacting with dynamic molecular visualizations.
Keywords: High school chemistry - Simulations - Technology-enhanced learning in science
Introduction Chemistry is often referred to as the “molecular” science, in the sense that understanding chemistry requires an understanding of molecular properties and processes. As such, students of chemistry are required to imagine scientific phenomena on an atomic-scale level and to employ a molecular point of view when explaining these phenomena. Typically, textbooks and classroom instruction include static visualizations and models that help students “see” the symmetries and structural factors involved in chemistry. Students are expected to make connections between these static visualizations and the observable phenomena, as well as to understand that the static representation is just one specific frame of the ever-changing dynamic molecular world. However, research suggests that students have difficulties both in connecting the observable and the molecular points of view (Smith, Wiser, Anderson, & Krajcik, 2006) and in realizing that molecules are always in motion (Pallant & Tinker, 2004). In their report focusing on a learning environment designed to support understanding of chemical concepts, Michalchik et al. (2008) also noted that the understanding of molecular properties and processes has always been a challenge, in a large part because molecules and their properties are not available to direct perception. Even if molecular phenomena could be clearly demonstrated, research in science education informs us that observation alone might not be enough for learners to develop their scientific understanding. Crouch, Fagen, Callan, & Mazur (2004) compared students learning concepts of physics from different modes of classroom demonstrations and found that those students who passively observed demonstrations understood the underlying concepts no better than those students who did not see the demonstration at all. In his Computing Education Blog, Guzdial (2011) cites Eric Mazur in concluding that in some cases observing a demo is even worse than having no demo at all. However, Mazur and his team also showed that students who were actively engaged with a demonstration by predicting the outcome before it had been conducted were better able to recall and explain the scenario posed by that demonstration (Mazur, 2011). A similar idea is reflected by the predict-observe-explain pattern (Kali & Linn, 2008; Linn & Eylon, 2006), according to which students benefit from recording their predictions prior to the demonstration, to be better able to discuss the contradictions after observing it (Wouters, Paas, & Van Merriënboer, 2008). Today’s technology can provide even more ways for students to engage actively with scientific concepts and ideas. In their recent book, Linn & Eylon (2011) assert that "Visualizations that take advantage of advances in technology can enable learners to explore phenomena that are too small (molecules), fast (electrons), abstract (forces), or massive (the solar system) to observe directly" (p. 186).
-2-
Furthermore, in typical middle or high school classroom settings, where it is impossible to perform real experiments, computer simulations are considered an attractive low-cost alternative (Feurzeig & Roberts, 1999). In describing the simulations designed as part of the CPU project1, Goldberg (2001) also states that these can help students in collecting both phenomenological and model-based conceptual evidence, as well as in providing multiple representations of such kinds of scientific phenomena. Recent powerful visualization environments such as Molecular Workbench (MW) 2 (Xie & Tinker, 2006; Tinker & Xie, 2008; Xie & Pallant, 2011), NetLogo (Wilensky & Rand, in press; Sengupta & Wilensky, 2009), Connected Chemistry (Stieff, 2005), and PhET (Wieman, Adams, & Perkins, 2008), not only dynamically demonstrate such phenomena, but also enable learners to interact with the otherwise inaccessible world. Learning and understanding can be enhanced further by embedding such dynamic visualizations in a learning environment, such as the Web-based Inquiry Science Environment3 (WISE), in which students can make their own thinking visible and their own knowledge-integration processes assessable (Kali & Linn, 2008). Nowadays, many computer simulations can be found on the web as Open Educational Resources (OER) and can therefore, be introduced as an alternative to real experiments in science classrooms at all levels (Panoff, 2009; Finkelstein et al., 2005). Simulation-aided teaching is regarded as an important instructional technology "as it adapts to today’s students who grew up in an increasingly digital world and are more accustomed to visual learning" (Xie & Pallant, 2011, p. 122). Dynamic simulations afford learners new ways of visualizing complex domains that may lead to more effective learning (Ainsworth & Van Labeke, 2004; Gerjets et al., 2010) while raising fewer misconceptions (Vosniadou, 2010). They provide visualizations of phenomena and processes that are impossible to see in the real world, yet the experience of which will provide understanding that is it difficult to achieve without such representation (Ainsworth, 2008). In the case of molecular science, visualization technology enables learners to dynamically model scientific phenomena on the atomic- and even the nano-scale levels (Russell, Kozma, Becker, & Susskind, 2000; Stieff, 2005; Shipley & Moher, 2008; Xie & Lee, 2012). For example, high school students explorations of and interactions with atomic-scale dynamic models of solids, liquids and gas, can lead to a better understanding of the connections between atomic-scale events, such as the overcoming of an intermolecular force, and those events that they can observe at the macroscopic scale, such as a cube of ice melting (Berenfeld & Tinker, 2001; Buckley et al., 2004; Tinker, Berenfeld, & Tinker, 1999; 2000). Other concepts, such as temperature, pressure, equilibrium, and many more can only be understood in terms of dynamic processes (Xie & Pallant, 2011). At the same time, technologies that allow for manipulations of dynamic media, such as deconstructing molecular processes using 2D/3D visualizations (Rundgren & Tibell, 2010), scrutinizing visual data, pacing and segmentation of events (Wouters, Paas, & Van Merriënboer, 2008), reflecting on portions of dynamic media, and generating animations (Bailey & Worth, 2004), can support higher-level thinking and reasoning about molecular processes. The study reported here was conducted as part of the NSF funded TELS center 4 and involved Molecular Workbench (MW) dynamic visualizations. MW learning environment began ten years ago with a molecular dynamics engine that calculated in real time the interactions among and between molecules (Fredette, 2012). The software offers educational tools to visualize the collective motions of atoms and molecules based on estimations of classical dynamics and applicable forces. Each run of MW calculates Newtonian approximations of inter-molecular forces to decide how particles will move (Xie and Tinker, 2006). MW has been expanded and currently provides several additional engines for dynamic visualization of DNA transcription and translation, electron clouds, chemical reactions, and more. These computational engines facilitate the instructional design of interactive model-based learning activities for middle school to college science classes, but not only in Chemistry, as is illustrated in Figure 1 and as is stated on the MW website: "The power of MW lies in a set of underlying computational engines that support experimentation at multiple scales and across disciplines"5. However, the focus of this paper is on learning about "Phases of Matter and Phase Change" as part of the high school level Chemistry curriculum.
1
Constructing Physics Understanding in a Computer Supported Learning Environment. http://cpuproject.sdsu.edu . By the Concord Consortium. See http://mw.concord.org . 3 http://wise.berkeley.edu . 4 TELS – Technology Enhanced Learning in Science, an NSF supported center headed by Prof. Marcia Linn (UC Berkeley) and Dr. Robert Tinker (The Concord Consortium). See http://www.telscenter.org/. 5 The citation and the image in Figure 5 can be found at the "About" section of Molecular Workbench (MW) website http://mw.concord.org/modeler/moremw.html. 2
-3-
Figure 1. Molecular Workbench simulations support experimentation across disciplines The rationale for the study derived from the assumption that by manipulating MW dynamic visualizations, by collaboratively interacting with the molecular models, and by reflecting on the interactions, students have the potential to develop a deeper conceptual understanding of the underlying chemical phenomena (Kozma, 2003; Xie & Pallant, 2011). Such an understanding is hereby termed “Molecular Reasoning” (MR). MR is defined in this paper as the ability to describe and explain scientific phenomena by referring to the atomic, molecular world. In other words, MR is the ability to think, reason, and talk about familiar phenomena in a new scientific way (Scott, 1998) using terminology of the atomic scale. As this case study focuses on learning about phenomena linked with phase change, its aim has been to explore the advantages of dynamic molecular visualizations for offering 'access to new conversation' (Sutton, 1996) about phases of matter, and for the development of a better understanding of the molecular processes of phase change. The research question was therefore: How do students improve in their ability to use molecular reasoning (MR) when explaining phase change, following the interaction with dynamic molecular visualizations? In order to answer this question, an online learning module was developed, focusing on "Phases of Matter and Phase Change", while employing dynamic MW visualization tools as part of the online learning process. The implementation of the module in eight public high schools was accompanied with extensive data collection, including online pre/post assessments designed to stimulate thinking about the molecular world. Students' responses to these assessments were analyzed using a scale for measuring the MR level before and after learning with the dynamic MW visualizations. The paper is therefore organized as follows: Section 2 describes the online learning module; Section 3 presents the case study design; and Section 4 discusses the results and answers the abovementioned research question. The last section draws conclusions and presents some implications from these findings for curriculum developers, teachers, and researchers.
The "Phases of Matter and Phase Change" Module The “Phases of Matter and Phase Change” learning module was designed and implemented in high school chemistry classes as part of the TELS online free curriculum (TELS stands for Technology Enhanced Learning in Science, see for example Lee, Linn, Varma, & Liu, 2010). The instructional design followed an inquiry-based, technology-enhanced learning approach (Linn, Davis, & Bell, 2004) and the assumption that the educational value of models emerges from deep interaction with them (Tinker & Xie, 2008). Like other TELS modules, the "Phases of Matter and Phase Change" module was created and implemented using WISE, and integrated novel visualization technologies from the Concord Consortium (Clark, Varma, McElhaney, & Chiu, 2008; Levy, 2009). WISE is an open-source digital learning platform developed in the University of California, Berkeley. It offers a library of tested curricula to implement in middle school and high school classrooms, and supports technology-based inquiry pedagogy that encourages students to develop solutions to problems, generate predictions before conducting investigations, use scientific evidence to create scientific arguments, and debate contemporary science issues (Linn, Clark, & Slotta, 2003). WISE also provides researchers with functionality, such as logging student interactions with the environment, and designing pre/post and embedded assessments in order to capture student thinking prior to, during, and following the process of inquiry (Liu & Linn, 2011). As is detailed in the next section, online pre/post assessments served as the main data source for the study presented here. The "Phases of Matter and Phase Change" learning module also utilizes Molecular Workbench (MW) dynamic molecular models of solid, liquid, and gas. Students interact with these visualizations by changing variables, such as heat or pressure, as is shown in Figure 2. The models of the phases of matter
-4-
are embedded in the WISE environment in such a way that the learners’ interactions with these dynamic representations are preceded by prediction making and then followed by reflective note taking.
Figure 2. Web-based dynamic molecular model of boiling water Each TELS module is constructed from a set of web-based learning activities divided into steps. The screenshot in Figure 2 is taken from step 3 of the first activity of the module. When one clicks the play button of the model player, the molecules begin to move, and when heat is (virtually) added by clicking the temperature bar on the right, they move faster. The dynamic MW visualization in step 3 comes after recording a prediction (in step 2). It is evoked on top of a textual explanation window in which the learner can read about boiling, and is followed by a reflective note (in step 4) in which the learner can write about the interaction with the dynamic visualization of boiling. The instructional design pattern of (a) predict (and record the predictions); (b) observe (and interact with the MW models); and (c) explain (and compare the observable results with the recorded predictions), enables learners to make and refine connections between observable phenomena and atomic level processes related to phase change (Linn & Eylon, 2006). As is also summarized by Wouters, Paas, & van Merriënboer (2008), studies in science education indicate the importance of actively inciting learners to anticipate the outcomes of animated models, for example, by predicting the model's next step or by answering pre-questions. As has been noted in the introduction, the general goal of the “Phases of matter and Phase Change” module is to develop the ability of learners to employ a higher level of molecular reasoning (MR) when explaining daily phenomena involving phase changes. This goal is achieved by embedding interactive dynamic MW visualizations in a structured web-based learning sequence, such as the predict-observeexplain pattern in the first activity, which enables individual inquiry, peer discussions, and reflective note taking. The learning module comprises five activities; each planned for one day. In brief, the activities are: Activity 1. Cooking, Boiling and the Liquid Phase of Matter. Presenting the leading question “can you speed up the process of hard boiling an egg?”, introducing the Molecular Workbench (MW) software and the first model, and predicting the shape of the heating curve. Activity 2. The Molecular Point of View: Solids, Liquids, and Gases. Investigating three MW models and reflecting upon the properties of solids, liquids, and gas represented in these dynamic models. Activity 3. Energy and Phase Changes. Presenting two schemas of phase transitions, relating selected types of phase change to the schema and to the terminology of endothermic and exothermic processes. Conducting an experiment of heating a cube of ice and recording the temperature, drawing the experimental curve, and reflecting on the differences between the predicted shape of the curve and the experimental curve. Activity 4. Melting and Boiling: Where does the Energy Go? Interpreting the results of the experiment in Activity 3 both in terms of the exchange between the thermal and kinetic energy, and relating the processes of phase change to changes of the intermolecular forces. Rethinking the question presented in the beginning; “Can you speed up the process of hard boiling eggs?” and introducing the consideration of the possible effect that pressure might have on the boiling point. Activity 5. Apply your knowledge.
By the end of the module, the learners are expected to be able to: Use commonly accepted vocabulary associated with phases of matter and phase changes; Describe the role of intermolecular forces in solids, liquids, and gases and in the cases of solid-toliquid and liquid-to-gas phase changes; Describe how intermolecular forces lead to an associated potential energy and its interplay with kinetic energy; Express the equivalence of average kinetic energy and temperature; Describe the heating curve of water and discuss accurately the reasons for its structure;
-5-
Distinguish between heat and temperature in the case of the heating curve of water, and explain why there is no change in the temperature when melting and evaporation occur, even though heat is added to the system; Compare the five parts of the heating curve, construct an image of the typical molecular dynamics in each part, and explain the differences between the parts using descriptions of the atomic-scale.
The Phase Change Case Study Context The "Phases of Matter and Phase Change" module is one of more than twenty free online learning units that engage students in scientific inquiry through challenging collaborative activities that emphasize visualization, simulation, and investigation, using information technology (Clark, Varma, McElhaney, & Chiu, 2008). Typically, TELS modules involve one week of web-based activity. By accessing information through guided inquiry steps, manipulating visualizations, interacting with molecular models, and reflecting on the interactions within the integrated framework of a complete learning activity, students have the potential to develop a deeper conceptual understanding of the underlying chemical phenomena, and the opportunity to develop their molecular reasoning skills. TELS technology also enables the storing of the written reflections of the students as they work online and thus, provides teachers with an eye into their students’ ideas (Slotta & Linn, 2009). On the larger scale, by collecting and analyzing the written explanations given by learners from several classes, researchers can investigate the progress students make and answer questions like the abovementioned. Participants All TELS curricular modules are free for use after an easy registration process. Since the "Phases of Matter and Phases Change" module design was completed in the fall of 2006, more than one-hundred US teachers signed up for using it with their classes. Among these are ten pioneers from eight TELS public high schools in North Carolina, Massachusetts, Virginia, and Wisconsin, who shared their students’ data as part of this case study. The implementation of the module in these classes was accompanied with extensive in-class and online data collection, in order to create a rich documentation of the learning process, to gather a wide collection of learners’ ideas, and to track changes in those ideas. This paper is based on analyzing one portion of the data gathered in these classes, as is discussed further in section 3.3. Overall, data were collected from six hundred students in grades 9-12 that learned using the "Phases of Matter and Phases Change" module as part of their regular chemistry course. As has been often the case in TELS experimental schools, the students worked through the module in pairs, so that the data they entered online were collected and saved as some sort of a shared knowledge (Levy, 2002) rather than an individual response. As Rowell (2002) highlights, shared technological activity is a social situation in which talk not only mediates, but also in part, constitutes the nature of the action. While the talk may focus on specific features of the subject at hand, it also sustains the social relations among participants. In such interplay between subject matter and social relations, the learners in the pair bring their viewpoints into contact in what is regarded in the literature as 'collaboration for inquiry' (Linn & Eylon, 2011; Clark & Sampson, 2008). The discussion of the pros and cons of technology-enhanced collaborative inquiry learning is beyond the scope of this paper. However, it is still important to mention here that a number of metaanalyses support the claim that collaborative learning enhances academic achievement, student attitudes, and student retention (Prince, 2004), and call for more widespread implementation of small-group learning in high school, as well as at the undergraduate level (Springer, Stanne, & Donovan, 1999). "When peers indicate what they do, and do not, understand, the student who is communicating gets valuable feedback" (Linn & Eylon, 2011, p. 221). In addition, recent studies demonstrated that not only collaborative learning, but even collaborative testing may yield learning benefits, whilst having no negative effects on individual scores (Slusser & Erickson, 2006). These studies suggested that the improvement was related to cognitive processes, such as thinking aloud about the information being tested and being able to participate in discussions (Kapitanoff, 2009). In the current case study, the collaborative testing also allowed discussions among the pairs of students, and even encouraged verbal interaction throughout the pre/post assessment. The interaction among pairs while completing the module’s activities has indeed been beneficial for eliciting ideas (Sampson & Clark, 2011) as well as for developing individual Molecular Reasoning (MR) ability; however, neither the researcher nor the teachers could distinguish the level of reasoning of individual students from the collaborative written response of the pairs. Thus, although the data analysis reveals common ideas among these students, and although these ideas helped in the construction of the MR
-6-
scale, as is detailed in section 3.4, this emergent construct was not used to scale, grade, or evaluate individual knowledge, nor individual students. Data Collection The data used in this study came from online pre-test and post-test pair assessments. The assessment items were carefully designed to stimulate thinking about the molecular world and to describe daily phenomena using molecular terminology. Both assessments were administered during regular class periods, shortly before and immediately after running the "Phases of Matter and Phase Change" module, when each pair of students used one computer to enter its answers online. Figure 3 presents a part of this online test, showing three out of its ten items, which were all traditional e-assessment items, either in the form of multiplechoice (MC), constructed response (CR), or as a combination of the two.
Figure 3 Combined MC and CR items in the online pre-test The data collected from the online tests included 314 pre/post pairs; one pair of pre/post tests for one pair of students. Each pair of students was given an identifying number (for example, pair #257). The teachers were instructed to have their students take the pre-test on the first day of running the online project and to keep the same student pairing throughout the run, so that the same pairs of students would also take the post-test together. That way, the responses of pair #257 in the pre-test could be compared with the responses of the same pair in the post-test. The post-test was usually administered less than two weeks after the pre-test, shortly after the class had finished all the activities of the online module, as described in Section 2. The two tests were similar, except for a variation in one combined item (number 5 – see Appendix A, asking about a heating curve in the pre-test versus asking exactly the same about a cooling curve in the post-test). Like other TELS modules, the ten assessment items for the "Phases of Matter and Phases Change" module were developed by a team of TELS researchers and composed of some research-based and some standardized items (Lee, Linn, Varma, & Liu, 2010). Half of the items (including those in Figure 3) asked students to link and connect ideas and give explanations for their conjectures, consistent with the knowledge integration framework that guides the design of TELS curriculum materials (Linn & Chiu, 2011). By consisting of detailed explanations, as phrased by the students themselves before and after the module, the responses for these items potentially offer the greatest contribution to answering the current research question - How do students improve in their ability to use MR, following the interaction with dynamic molecular visualization? Accordingly, the current case study focuses on analyzing the data from four items6 (see Appendix A), so that the overall database included ~1200 paired pre-test/post-test written descriptions (~300 paired responses for each of the four items). The “Molecular Reasoning” Scale The analysis began by examining the different kinds of explanations students gave in response to Item 7; “What happens to water molecules when a cube of ice is taken out of the freezer and left at room temperature?” in the pre-test. In responses like “the molecules expand and melt” (pair #89), “they spread apart” 6
The responses to Item 8 in Figure 3 as well as to the MC-only items (without the students explaining why the choice was made) were not helpful in expressing levels of molecular reasoning, and therefore, were excluded from the current analysis.
-7-
(#214), and “the water molecules start to excite and move around again” (#297), it was quite obvious that students had thought about molecules and thus, revealed a certain level of molecular reasoning, while many other pretest responses did not mention the molecular point of view at all (even though the question directly requested a description of molecular behavior). As Vosniadou (2010) writes, summarizing the extensive literature on students' misconceptions, many chemistry students believe, like pair #89, that molecules expand when heated, but although based on a misconception, such a reply does at least mention molecules. However, those responses that did mention molecular behavior reveal different levels of molecular reasoning. Those describing how the water molecules “expand” or “spread apart” when a cube of ice is taken out of the freezer, seem to have a more static image of the molecular structure, compared with those who explained how the molecules start to move faster and faster as the cube of ice melts, as was the case with pair #297. The Molecular Reasoning (MR) scale has emerged naturally from the above observation. Recall that MR has been defined previously as the ability to describe and explain scientific phenomena by referring to the molecular world (Levy & Tinker, 2008). After carefully screening all the written responses to Item 7 in the pre-test, given by the 314 pairs of high school students from various TELS schools, four types of explanation were recognized with regard to MR. Explanations of one type did not refer at all to the molecular world and thus, were categorized as “non MR“. The other three types of explanation all mentioned the molecular world. The more common types included phrases and keywords that were interpreted as either expressing a basic static view of the molecular world and thus, categorized as “static MR”, or a more advanced dynamic view categorized as “dynamic MR”. Very few pre-test responses hinted at an even higher level of reasoning, referring to the intermolecular forces in explaining the phase change phenomena. These rare responses were categorized as “intermolecular MR”. Therefore, the four types of MR can be placed on an ordinal scale constructed of the zero level of no MR, and then the static, dynamic, and intermolecular levels of understanding in thinking about phases of matter and phase change (see Table 1 for example responses). Table 1. The Molecular Reasoning (MR) Scale Score
Description
Examples7
3
Intermolecular MR
"The molecules start to move faster and break the bonds holding them together" (#18) "It melts because the molecules gain energy, motion, and they can overcome cohesive forces and come apart" (#44) "The cube of ice absorbs energy from the enviroment and begins to warm up. This increases the kenetic energy and the molicules move faster and the intermolecular bond between them decreases. This makes the ice change to water" (#225) "The molcules change state because of the differences in heat. The molecules can move around easier in a liquid rather that a solid where they are close together" (#94) "The water molecules start to move faster because of the energy flowing from the surrounding to the ice cube. As the water molecules start to move faster the ice cube starts to melt" (#232) "With the room temperature added, the water molecules start to excite and move around again, so which in turns reverts the ice cube back into water form" (#297) "The ice molecules separate in the freezer because the bond weaken. So when it reaches room temp the bonds come back together again". (#268) "They become scattered because of phase change from solid to liquid" (#16) "The water molecules are tightly packed together when the ice cube is first taken out of the freezer and then they slowly become further apart as the temperature of the ice cube increased until the water molecules reach the liquid state" (#113) "The water molecules would breakdown and would not be compacted together because the ice cube is melting" (#277) "The molecules have more space" (#292) "The Cube of ice will melt, but will melt slowly depending on the room temperature" (#13) "It begins to melt from the temp. change in the room" (#289) "Its melting because the cube is at room temperature and the cube is solid at a freezing temperature. It loses energy" (#301)
Dynamic MR + including correct description of the weakening of intermolecular forces/bonds
2
Dynamic MR Referring to molecular motion (specifically mentioning the molecules moving) -ORReferring to the role of intermolecular forces in phase change (without specifically speaking about molecular motion, as in response #268)
1
Static MR Speaking about molecules, but not about their motion.
0
7
No MR
These responses were entered online by real students answering Item 7 in the pre-test, including the typos.
-8-
The development of the coding scheme using the MR scale, involved a team of TELS researchers as well as leading members of the MW development team. According to this coding scheme, explanations that do not contain any trace of thinking in the molecular level are scored “0”, and those that do are scored “1” (static), “2” (dynamic), or “3” (intermolecular). Interestingly, the scale also reflects the historical development of thinking in the atomic level. The historical point of view can be found in a story about the 1999 Nobel Prize winner in Chemistry, Prof. Zewail (from CalTech). The story goes back to 1944, when Pauling, another CalTech professor, won the Nobel Prize in Chemistry. In the reporter’s words: “Pauling worked from crystallographic data, and his bonds were static, stable, and enduring. Now, 45 years later, Zewail has set those bonds in motion, making them as alive and dynamic as chemistry itself.”8 According to the MR scale, the highest score of molecular reasoning is given to explanations that not only describe the motion of the molecules, but also refer to the role that intermolecular forces play in the phase change related phenomena. It is regarded as the highest level because without consideration of intermolecular forces, one gets stuck when trying to explain melting, for example. Even if students correctly describe the molecules moving faster and faster as heat is added, that accelerated motion alone does not explain why the solid suddenly becomes a liquid (Schmidt, Kaufmann, & Treagust, 2009). For a fuller account, one must also describe how that energy of motion is used to weaken the intermolecular forces. Thus, for example, while using the MR scale to code the explanations students gave to another pretest item (the heating curve in Item 5, see Appendix A), it was quite clear that students were capable of recognizing that the flat parts of the heating curve were related to the phase change (probably because they learned that in the past), but they failed to explain why the temperature stayed constant during these times. Therefore, in the pre-test students could answer a question like; “What happens to the temperature during a phase change?” (stays constant), but they could not answer the question; “Why does it happen?” (energy of heat is used to weaken intermolecular forces) because they lacked the knowledge about the role that intermolecular forces play in this process. In such cases, even using the 2nd dynamic level of the MR scale, describing the change in molecular motion, cannot lead to a full explanation. The next analytic step consisted of coding all the responses in the database using the MR scale. The pre-test responses for each of the four assessment items were coded first, independently from the posttest responses. The coding itself consisted of carefully reading each response in the database and scoring it according to the scale (Appendix B presents some examples). Two assessment experts, as well as one of the TELS chemistry teachers, helped in confirming the trustworthiness of the coding of partial sets of responses. As a last step, the resulting scores for each pre-test item were matched with the resulting scores for each post-test item, to construct the final dataset of MR scores. Table 2 presents a small part of this dataset. Table 2. Partial Dataset of MR Scores Pair number 4 5 7 9
5 PRE 1 0 0 0
5 POST 0 0 0 0
80 81 82 83 84 85 86 87 88 89 90 91 92 93
0 0 2 0 0 0 0 1 1 0 2 0 0 0
0 1 2 0 0 1 0 0 2 0 2 0 0 1
6 6 7 PRE POST PRE 2 2 1 0 1 1 0 2 2 0 2 2 End of class 517-19 0 2 1 1 2 0 0 2 2 1 1 2 0 0 0 1 1 1 0 2 1 0 2 0 1 3 0 1 2 1 2 2 2 0 3 0 0 1 0 0 1 0 End of class 517-2
7 POST 2 2 0 2
9 PRE 1 2 1 1
9 POST 2 2 2 0
3 0 1 0 0 2 2 1 1 0 1 0 3 2
2 1 0 0 0 1 0 0 1 0 0 1 1 0
3 1 3 3 0 1 2 2 2 3 0 0 1 0
The overall dataset is organized by classes. As can be seen in Table 2, certain pairs of students are missing from their class dataset. These pairs (for example, number 6 and 8) did not respond to both the pretest and the post-test questions and their responses could not be matched. Out of 314 pairs of students in all
8 9
Retrieved September 15, 2012 from http://www.zewail.caltech.edu/nobel/Zewail_Feature.pdf. The TELS teacher identification number is 517. 1, 2, etc., are the different chemistry classes of that teacher.
-9-
the classes participating in this study, the pre/post MR scores of 241 pairs could be matched and have been included in the final dataset. The MR scale represents an ordinal scale and therefore, the sign test was used to compute the significance of the changes in the MR score from pre- to post-test, and to answer the research question. The results are presented in the next section.
Results Improvement patterns Examination of the scores of the four items revealed significant differences in the positive direction between pre-test and post-test responses. Table 3 presents the detailed frequencies. Table 3. Frequencies of differences and p-values Negative Differences (postpre) Ties (post=pre) Total (N) Z Significant? (
