Exploring the Changes in Students’ Understanding of the Scientific Method Using Word Associations Ozcan Gulacar, Olcay Sinan, Charles R. Bowman & Yetkin Yildirim
Research in Science Education ISSN 0157-244X Res Sci Educ DOI 10.1007/s11165-014-9443-9
1 23
Your article is protected by copyright and all rights are held exclusively by Springer Science +Business Media Dordrecht. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”.
1 23
Author's personal copy Res Sci Educ DOI 10.1007/s11165-014-9443-9
Exploring the Changes in Students’ Understanding of the Scientific Method Using Word Associations Ozcan Gulacar & Olcay Sinan & Charles R. Bowman & Yetkin Yildirim
# Springer Science+Business Media Dordrecht 2014
Abstract A study is presented that explores how students’ knowledge structures, as related to the scientific method, compare at different student ages. A word association test comprised of ten total stimulus words, among them experiment, science fair, and hypothesis, is used to probe the students’ knowledge structures. Students from grades four, five, and eight, as well as firstyear college students were tested to reveal their knowledge structures relating to the scientific method. Younger students were found to have a naïve view of the science process with little understanding of how science relates to the real world. However, students’ conceptions about the scientific process appear to be malleable, with science fairs a potentially strong influencer. The strength of associations between words is observed to change from grade to grade, with younger students placing science fair near the center of their knowledge structure regarding the scientific method, whereas older students conceptualize the scientific method around experiment. Keywords Science process skills . Word association test . Knowledge structure . Science fair . Project-based learning . Scientific method
Introduction It is generally accepted that science knowledge is important, and a science, technology, engineering, and mathematics (STEM) education is important to both individual students and society at large (Toulmin and Groome 2007). Science knowledge is learned when students encounter problems in the course of conducting research (Bagci Kilic 2003). Because of this, science teachers should be more aware of the importance of how students conceptualize science and what those conceptions look like (Slotta et al. 1995). O. Gulacar (*) : C. R. Bowman Sam Houston State University, Huntsville, TX, USA e-mail:
[email protected] O. Sinan Balikesir University, Balikesir, Turkey O. Sinan : Y. Yildirim The University of Texas at Austin, Austin, TX, USA
Author's personal copy Res Sci Educ
Knowledge of scientific inquiry is seen as an important component of science literacy, and a goal for all students (Committee on Conceptual Framework for the New K-12 Science Education Standards and National Research Council 2012). Regardless of career choices, it is recommended that every person becomes familiar with this method and uses it in solving problems faced in academic environments and in everyday life (Haines 1997; McPherson 2001; McCleery and Tindal 1999). The old term, “scientific method,” sparks debate in the educational community (Watson and James 2004). Textbooks’ focus on “problem,” “research,” “hypothesis,” “experiment,” and “observation” (typical examples of the scientific method) has been criticized as not being representative of the scientific process and that the reliance on such rigid structures focused on checking off each step has stigmatized science as lacking in creativity and flexibility (Alexakos 2010; Windschitl et al. 2008). Others, however, believe that teaching the scientific method is worthwhile for younger students, as it offers necessary structure (Gerde et al. 2013; Watson and James 2004). The National Science Teachers Association (NSTA), a non-profit group in the USA, publishes science education standards that promote science inquiry, rather than the scientific method and a group from UC Berkeley has created a website to help dispel this simplistic view of science, showing how the real scientific process is an interconnected web of exploration, testing, community analysis, and real-world benefits (Harreid 2010). As a consequence, knowing how students organize their knowledge around scientific inquiry is important. The way students organize knowledge in their heads (i.e., knowledge structures) has been correlated with the ability to solve complex problems (Ausebel 1968; Bédard and Chi 1992; Caramazza et al. 1981; Chi et al. 1981; McCloskey 1983; Wandersee et al. 1994). According to Tsai (2001), students’ long-term memory is organized into a hierarchy of concepts and relations. It is thought that this knowledge is comprised of many fragmented pieces of information that are connected to each other, forming a more complex knowledge structure that is used to solve problems (diSessa 1988). As a student matures, connections between the various pieces of information are reorganized into a better, more coherent structure, reflecting a better conceptual understanding (diSessa 1983; Thagard 1992). The addition of new information, such as scientific facts or phenomena, into a knowledge structure can require major structural changes in order to create a more coherent understanding (diSessa 1983). With this in mind, the following research question was investigated: How do students’ knowledge structures, as related to the scientific method, change as students mature? The construction of knowledge structures has been a source of much debate and it had generally been assumed that either (1) knowledge is incorporated first by similarity judgments (i.e., what matches what) and later organized by a native ability to understand causality or (2) that children only use similarity to organize their knowledge and organization by causality comes at a later age (Keil et al. 1998). However, Keil et al. suggest that neither is explicitly true, and a hybrid model, where both can be true to some extent, is more accurate. In either case, visualizing students’ knowledge structures is important for understanding how they understand a given subject (Nakiboglu 2008). The question, then, is how is a student’s knowledge structure of the scientific method changes with age, and is the structure correct? This paper aims to compare students’ conceptions of concepts involved in the scientific process with Keil et al. (1998) model model. To be clear, this study is interested in the knowledge of scientific processes, which are activities related to collecting data, doing experiments, and so forth. This is distinct from the nature of science (NOS), which is generally more philosophical in nature and concerned with imagination and inquiry (Lederman et al. 2002). There are several methods that researchers can use to externalize and reveal the knowledge structure of students, such as knowledge space theory (Arasasingham et al. 2004; Villano 1992), self-organizing feature maps (Harp et al. 1995), or
Author's personal copy Res Sci Educ
knowledge integration (Lee and Liu 2010). A word association test (WAT) is one of the most common methods for mapping knowledge structures (Bahar et al. 1999; Maskill and Cachapuz 1989). A WAT generally shows conceptual learning more than a subject’s knowledge of words (i.e., vocabulary), and is very good at assessing conceptual development as students age, though it can be problematic for subjects who are doing the WAT in a language other than their first language (Cremer et al. 2011). The WAT has been used often and in varied contexts. Bahar et al. (1999) used WATs to map students’ cognitive structure by revealing the type and number of concepts in students’ minds. WATs have been frequently used to observe changes before and after instruction in various science disciplines (Nakiboglu 2008; Shavelson 1973). Hovardas and Korfiatis (2006) used this technique to demonstrate the effectiveness of a given teaching method. Proponents of word association tests note that they provide a diverse list of concepts associated with the given concept in the students’ minds (Gussarsky and Gorodetsky 1988), and that participants are left free to spontaneously produce their own conceptual field, unencumbered by any predefined frame of reference, such as one imposed by a researcher or teacher (Daskolia et al. 2012). The test is also simple to write and to administer, requiring only a few minutes of class time (Cachapuz and Maskill 1987; Hovardas and Korfiatis 2006). The analysis is, however, much longer.
Method Participants for the study were selected from fourth (53 students), fifth (67 students), and eighth graders (56 students) at a charter school in central Texas (grades kindergarten–12; K-12), whose main mission is to enhance STEM education, and from first-year undergraduates (91 students), who were science majors and registered for General Chemistry I at a central Texas university. The student population was diverse; approximately 30 % of the students were white, 40 % Hispanic, 17 % African-American, with the remainder of Asian descent. Approximately 4 % were of low English proficiency and 38 % were from low-income families (Texas Education Agency 2010). These demographics reflect the demographics of the area surrounding the charter school. The undergraduate students were selected from the authors’ institution and reflect the demographics of the surrounding region. To construct the word association test (WAT), the following ten key words or phrases acting as stimulus were given to students in the following order: research, project, problem, experiment, data collection, hypothesis, dependent variable, observation, writing report, and science fair. These words are related to the scientific method (Colley 2006; Lawless and Rock 1998) and were selected based on their importance to the charter school’s emphasis on technical and scientific instruction. At this school, each student must participate in a science fair every year, beginning in fourth grade. Likewise, science and technical instruction is required each year. The words were selected by a science educator and an engineer. They selected words they believed to be closely tied to the scientific process (Lederman et al. 2002). The students were provided with a booklet, each page containing a table with two columns (the first column repeated the stimulus word ten times, the second column was left blank). Students were asked, after reading the stimulus word on the left, to list up to ten words that they considered to be most closely associated with the stimulus word. Students were allowed 20 min to complete the whole process. The time limit and the repetition of the key phrase was adopted as a method from previous studies to prevent distraction and gather more accurate data about the level of interconnectedness between the stimulus and associated words (Bahar and Hansell 2000; Nakiboglu 2008; Shavelson 1972).
Author's personal copy Res Sci Educ
In a word association test, the most important factor to determine is the strength of connections between the stimulus words and the response words. A shorter response time is presumed to reflect a closer conceptual association (Shavelson 1972). For the analysis of WAT results, the relatedness coefficient method suggested by Garskof and Houston (1963) was utilized. According to this method, a relatedness coefficient (RC) value is calculated between pairs of stimulus words for each student (Bahar et al. 1999). Afterwards, the RC values between each pair of stimulus words were ranked from largest to smallest and the 20 highest RC values were then diagramed. The RC value accounts for both rank order and number of shared responses when comparing two stimulus words (Garskof and Houston 1963). Sample calculations can be seen on page 282 of Garskof and Houston (1963). One note about the fourth-grade cohort: 65 fourth-grade students took the survey but due to several mistakes in their booklets, 12 students’ entries were ignored. For example, some students listed the same stimulus word in every box, some students listed their teacher’s name for all the words, etc. In the end, 53 of the original 65 were determined to be acceptable. It may be that the WAT is limited in its ability to be accurately used with young students.
Results and Discussion Relationships of Stimulus Words For each cohort of students, the relatedness coefficient (RC) was calculated for each stimulus word pair using the formula developed by Garskof and Houston (1963). The stimulus word pairs were then ranked based on the strength of their association, as measured by their RC. Finally, the 20 strongest relationships were mapped for each grade; the stronger the link, the thicker the line between the two words. The results are shown in Figs. 1, 2, 3, and 4. When looking at the concept maps, it should be noted first that these maps are just partial concept maps with only the top 20 links being shown for any grade. In addition, they are representative of the “average” map for each grade, based on 50–90 individual student responses each, and thus do not constitute one student’s knowledge structure. Furthermore, each grade represents a different cohort of students and, as such, the maps do not represent the changing average of one cohort. However, a number of relevant observations about the development of students’ knowledge structures can still be made. (The stimulus words are emphasized in italics.) With the fourth graders, it appears that their knowledge structures focus around data collection and hypothesis, two steps in the scientific method. They are closely tied in a loop also containing observation, and represent elements usually emphasized when teaching the scientific method in grade school (Pflugfelder 2013; Science Buddies 2013). It is interesting that fourth graders do not associate problem with other steps in the scientific method. Instead, it is only weakly connected to observation. This is problematic as students need to understand that every step in the scientific method serves a larger purpose: solving and answering a real-world problem (Harreid 2010). It appears that fourth-grade students have a basic understanding of observation’s place in the scientific method (i.e., we make observations while collecting data to come up with or test the hypothesis). The fact that experiment is connected to science fair, project, and hypothesis, but is not linked with research, suggests that fourth-grade students do not realize that the preparation for projects and science fairs is research, or that experiments are the core of any research activity. In the fifth-grade cohort (Fig. 2), observation is connected only to science fair; no other stimulus words are directly connected. Unlike the fourth graders, the fifth graders did not associate observation with problem or hypothesis, which may indicate that their understanding of the scientific method is not yet fully (and accurately) formed. Furthermore, the terms science
Author's personal copy Res Sci Educ
Fig. 1 Concept maps for fourth-grade students showing top 20 connections between stimulus words
fair, research, writing report, project, and data collection are strongly associated with each other in the fifth graders’ network. Compared to the other grades, this appears to be an exceptionally strong network. However, this does not persist through eighth grade and beyond, which may indicate that their understanding of the scientific method is not yet fully formed.
Fig. 2 Concept maps for fifth grade students showing top 20 connections between stimulus words
Author's personal copy Res Sci Educ
Fig. 3 Concept maps for eighth grade students showing top 20 connections between stimulus words
This particular concept map may also be atypical due to an exceptional science teacher. Since the cohorts of students in each grade are different and the teaching methods of each teacher were not studied in detail, this cannot be determined conclusively. Science fair and data collection are
Fig. 4 Concept maps for first-year undergraduate students showing top 20 connections between stimulus words
Author's personal copy Res Sci Educ
found in the center of the network for the fifth-grade students. It seems that these students make sense of the scientific process through the science fair experience, which may make it an important way of teaching the scientific method. This is likely a result of the focus on science fairs at the school where the data were collected: the school where these data were collected has students participate in a science fair each year from fourth grade on. Although there are portions of the network that are not well connected to the steps in the scientific method, the general associations appear to be good. Teachers can strengthen connections by giving clear descriptions of the steps and how they work together within a scientific process. The more teachers emphasize connected concepts in context with each other, the more likely it is that the concepts will become connected in students’ minds. After multiple years of participation in science fairs, the concept is centrally connected with almost all steps of the scientific method in the eighth graders’ minds (Fig. 3). As a result, it may be surprising to see that experiment is strongly connected to science fair but not research. However, eighth-grade students have no research experience outside of science fair projects, and it is likely that they do not see science fairs as research. Connected to several components of the scientific method, the stimulus word project has three strong connections: writing report, science fair, and data collection. Although research is not associated with either experiment or science fair directly, it is connected to data collection and project. It seems that eighth-grade students conceptualize the scientific method within the framework of projects. Furthermore, the eighth graders appear to associate research with the big picture (i.e., projects and writing reports). Data collection is connected with every other stimulus word except problem, which indicates that students focus on the procedure of data collection at every step, and that data collection continues to become more centralized in students’ conceptual maps. A lack of connection between hypothesis and problem suggests that eighth graders do not know that a hypothesis is the tentative answer for a problem statement. For the eighth graders, the stimulus word problem is not connected to science fair, observation, or experiment but only writing report. Although its association has changed with each grade, problem has remained on the fringe of each conceptual network (fourth, fifth, and eighth grades). This likely means that students associate the word problem with the non-scientific meaning of “difficult” more than with the scientific method. It should be noted that this does not continue for undergraduates, who strongly associate problem with hypothesis and experiment (Fig. 4). The stimulus word experiment is in the center of undergraduates’ conception of the scientific method (Fig. 4), and is strongly connected to problem and hypothesis. Strong connections also exist between observation, data collection, and dependent variable. Observation is weakly associated with experiment and, interestingly, is not connected to problem. This may indicate that the undergraduates do not place a strong emphasis on observation in answering problems or do not make a habit of observing their environment for discovering new problems to be investigated. There is, however, a strong connection between problem and hypothesis, as well as a strong connection to experiment. This is an important connection that represents a sound understanding of the role of hypothesis in the scientific method. It is interesting, however, that research is not associated with experiment or problem. One reason could be that the undergraduates have not conducted active research (i.e., research that answers questions by way of experiments, rather than by literature search) and, therefore, have little or no grasp of its connection to experiments. Like the eighth graders, the concept map of the undergraduate students shows that data collection has stronger and more numerous connections with other keywords than any other step in the scientific method, except experiment. Experiment has one more connection, but its connections are weaker. This could mean that undergraduate students focus more on the process but worry less about the big picture, or it could reflect the focus of their high school and college research experiences to date, as most
Author's personal copy Res Sci Educ
students have yet to do experiments that contribute to a larger picture. However, their connections appear closest to the newer standards for teaching science process skills (Harreid 2010). It is also evident that the undergraduates in the sample have outgrown their science fair backgrounds. Science fair is only connected to one keyword: data collection. Unlike the fourth, fifth, and eighth graders, the undergraduates represent a much wider range of K12 educational experiences, many of which did not likely place an emphasis on science fairs.
Conclusions The word association test applied in this study proved to be helpful in understanding and analyzing the knowledge structures of the four student groups. The knowledge structure of fourth graders holds concepts in a naïve organization: students do not appear to understand that science and scientific research aim to answer larger, real-world problems. The knowledge structures of fifth and eighth graders show a rearrangement of that structure, which resulted in the gradual centralization of science fair and data collection. The changing structures show that there is potential for creating a more robust understanding of the nature of science, as the knowledge structures are still forming. Furthermore, science fairs clearly have a strong influence on students’ conceptions about the scientific process, and care must be taken that science fairs do not reinforce the rigid, formulaic view of science. Finally, there was a fairly dramatic reorganization of the knowledge structure of undergraduate students, where the structure was observed to reorganize around experiment, rather than science fair and data collection. This appears closer to the ‘ideal’ view of science as a process to investigate questions about the world (Harreid 2010). However, further study is needed to determine if this view is flexible, and if it involves the community as reviewers and stakeholders. Teachers, especially those of younger students, can use this as a guide for what is most important to emphasize in science courses. Ultimately, students need to know the central role that experimentation takes in the scientific process, and the flexible nature of the scientific process as a whole. Guided-inquiry learning may help students better understand the role of experimentation, especially if combined with lab experiments or demonstrations, and is one of the best models of how science actually works (Alexakos 2010). Students need to know that experiments are not just reserved for science fairs; they are used to solve problems and broaden understanding of the scientific world. In K-12 environments, science teachers should emphasize real-world applications and encourage their students to investigate real-world problems for their science fair projects. For undergraduates, a focus on active research would give students a chance to experience real science at work and gain a more sophisticated understanding of how scientific knowledge is constructed. Acknowledgments Research reported in this publication was partially supported by NSF DMR-PREM under award number DMR-1205670.
References Alexakos, K. (2010). Teaching the practice of science, unteaching the “scientific method”. Science Scope, 33(9), 74–79. Arasasingham, R. D., Taagepera, M., Potter, F., & Lonjers, S. (2004). Using knowledge space theory to assess student understanding of stoichiometry. Journal of Chemical Education, 81(10), 1517– 1523.
Author's personal copy Res Sci Educ Ausebel, D. P. (1968). Educational psychology: a cognitive view. New York: Holt, Rinehart and Winston, Inc. Bagci Kilic, G. (2003). Concept maps and language: a Turkish experience. International Journal of Science Education, 25(11), 1299–1311. Bahar, M., & Hansell, M. H. (2000). The relationship between some psychological factors and their effect on the performance of grid questions and word association tests. Educational Psychology, 20(3), 349–364. Bahar, M., Johnstone, A. H., & Sutcliffe, R. G. (1999). Investigation of students’ cognitive structure in elementary genetics through word association tests. Journal of Biological Education, 33(3), 134–141. Bédard, J., & Chi, M. T. H. (1992). Expertise. Current Directions in Psychological Science, 1(4), 135–139. Cachapuz, A. F. C., & Maskill, R. (1987). Detecting changes with learning in the organization of knowledge: use of word association tests to follow the learning of collision theory. International Journal of Science Education, 9(4), 491–504. Caramazza, A., McCloskey, M., & Green, B. (1981). Naive beliefs in ‘sophisticated’ subjects: misconceptions about trajectories of objects. Cognition, 9(2), 117–123. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152. Colley, K. E. (2006). Understanding ecology content knowledge and acquiring science process skills through project-based science instruction. Science Activities: Classroom Projects and Curriculum Ideas, 43(1), 26– 33. Committee on Conceptual Framework for the New K-12 Science Education Standards, & National Research Council. (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Washington, D.C.: The National Academies Press. Cremer, M., Dingshoff, D., de Beer, M., & Schoonen, R. (2011). Do word associations assess word knowledge? A comparison of L1 and L2, child and adult word associations. International Journal of Bilingualism, 15(2), 187–204. Daskolia, M., Dimos, A., & Kampylis, P. G. (2012). Secondary teachers’ conceptions of creative thinking within the context of environmental education. International Journal of Environmental and Science Education, 7(2), 269–290. diSessa, A. A. (1983). Phenomenology and the evolution of intuition. In D. Gentner & A. L. Stevens (Eds.), Mental models. Hillsdale: Lawrence Erlbaum. diSessa, A. A. (1988). Knowledge in pieces. In G. Forman, P. B. Pufall, G. Forman, & P. B. Pufall (Eds.), Constructivism in the computer age (pp. 49–70, The Jean Piaget symposium series). Hillsdale: Lawrence Erlbaum Associates, Inc. Garskof, B. E., & Houston, J. P. (1963). Measurement of verbal relatedness: an idiographic approach. Psychological Review, 70(3), 277–288. Gerde, H. K., Schachter, R. E., & Wasik, B. A. (2013). Using the scientific method to guide learning: an integrated approach to early childhood curriculum. Early Childhood Education Journal, 41(5), 315–323. Gussarsky, E., & Gorodetsky, M. (1988). On the chemical-equilibrium concept—constrained word-associations and conception. Journal of Research in Science Teaching, 25(5), 319–333. Haines, L. (1997). Consumer testing: applying the scientific method to everyday life. Science Scope, 21(4), 34– 38. Harp, S. A., Samad, T., & Villano, M. (1995). Modeling student knowledge with self-organizing feature maps. IEEE Transactions on Systems, Man and Cybernetics, 25(5), 727–737. Harreid, C. F. (2010). The scientific method ain’t what it used to be. Journal of College Science Teaching, 39(6), 68–72. Hovardas, T., & Korfiatis, K. J. (2006). Word associations as a tool for assessing conceptual change in science education. Learning and Instruction, 16(5), 416–432. Keil, F. C., Carter Smith, W., Simons, D. J., & Levin, D. T. (1998). Two dogmas of conceptual empiricism: implications for hybrid models of the structure of knowledge. Cognition, 65(2–3), 103–135. Lawless, J. G., & Rock, B. N. (1998). Student scientist partnerships and data quality. Journal of Science Education and Technology, 7(1), 5–13. Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of nature of science questionnaire: toward valid and meaningful assessment of learners’ conceptions of nature of science. Journal of Research in Science Teaching, 39(6), 497–521. Lee, H.-S., & Liu, O. L. (2010). Assessing learning progression of energy concepts across middle school grades: the knowledge integration perspective. Science Education, 94(4), 665–688.
Author's personal copy Res Sci Educ Maskill, R., & Cachapuz, A. F. C. (1989). Learning about the chemistry topic of equilibrium: the use of word association tests to detect developing conceptualizations. International Journal of Science Education, 11(1), 57–69. McCleery, J. A., & Tindal, G. A. (1999). Teaching the scientific method to at-risk students and students with learning disabilities through concept anchoring and explicit instruction. Remedial and Special Education, 20(1), 7–18. McCloskey, M. (1983). Naive theories of motion. In D. Gentner & A. L. Stevens (Eds.), Mental models. Hillsdale: Erlbaum Associates. McPherson, G. R. (2001). Teaching & learning the scientific method. The American Biology Teacher, 63(4), 242– 245. Nakiboglu, C. (2008). Using word associations for assessing non major science students’ knowledge structure before and after general chemistry instruction: the case of atomic structure. Chemistry Education Research and Practice, 9(4), 309–322. Pflugfelder, B. (2013). The scientific method. http://www.sciencebob.com/sciencefair/scientificmethod.php. Accessed 25 Mar 2013. Science Buddies. (2013). Steps of the scientific method. http://www.sciencebuddies.org/science-fair-projects/ project_scientific_method.shtml. Accessed 25 Mar 2013. Shavelson, R. J. (1972). Some aspects of the correspondence between content structure and cognitive structure in physics instruction. Journal of Educational Psychology, 63(3), 225–234. Shavelson, R. J. (1973). Learning from physics instruction. Journal of Research in Science Teaching, 10(2), 101– 111. Slotta, J. D., Chi, M. T. H., & Joram, E. (1995). Assessing students’ misclassifications of physics concepts: an ontological basis for conceptual change. Cognition and Instruction, 13(3), 373–400. Texas Education Agency. (2010). 2009–10 Academic Excellence Indicator System Campus Reports. http://ritter. tea.state.tx.us/perfreport/aeis/2010/campus.srch.html. Accessed 26 Dec 2012. Thagard, P. (1992). Conceptual revolutions. Princeton: Princeton University. Toulmin, C., & Groome, M. (2007). Building a science, technology, engineering, and math agenda. Washington, D.C.: National Governors Association (Innovation America). Tsai, C.-C. (2001). Probing students’ cognitive structures in science: the use of a flow map method coupled with a meta-listening technique. Studies in Educational Evaluation, 27(3), 257–268. Villano, M. (1992). Probabilistic student models: Bayesian belief networks and knowledge space theory. In C. Frasson, G. Gauthier, & G. McCalla (Eds.), Intelligent tutoring systems (Vol. 608, pp. 491–498, Lecture Notes in Computer Science). Berlin: Springer. Wandersee, J. H., Mintzes, J. J., & Novak, J. D. (1994). Research on alternative conceptions in science. In D. L. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 177–210). New York: Macmillan. Watson, S. B., & James, L. (2004). The scientific method: is it still useful? Science Scope, 28(3), 37–39. Windschitl, M., Thompson, J., & Braaten, M. (2008). Beyond the scientific method: model-based inquiry as a new paradigm of preference for school science investigations. Science Education, 92(5), 941–967.
