THE DEVELOPMENT AND VALIDATION OF AN ASSESSMENT THAT ...

THE DEVELOPMENT AND VALIDATION OF AN ASSESSMENT THAT MEASURES MIDDLE SCHOOL STUDENTS’ LUNAR PHASE UNDERSTANDING By Sonya E. Sherrod, B.S., M.A. A DISSERTATION IN CURRICULUM AND INSTRUCTION Presented to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Approved

Dr. Walter Smith Chairperson of the Committee

Dr. Barbara Morgan-Fleming

Dr. Carl Seaquist

Dr. Eugene Wang

Fred Hartmeister Dean of the Graduate School

August, 2009

Copyright 2009, Sonya Ellouise Sherrod

Texas Tech University, Sonya Sherrod, August 2009

ACKNOWLEDGEMENTS

This research has not only broadened my knowledge but has also revealed to me the depths of my character, both strengths and weaknesses. The prayers and support of innumerable family members, friends, and teachers carried me to the end. I want to thank my husband, Kim, for lovingly listening to me as I vocally constructed my own understanding. His constant care and encouragement sustained me through daunting days and months of data analysis and writing. The sacrifices he endured in support of my quest are testaments of his abiding devotion to me and will never be forgotten. I am grateful to my advisor, Dr. Walter Smith, who graciously gave of his time and expertise in the midst of his numerous responsibilities to shepherd me. He put my needs before his own and afforded me the same measure of respect he does his colleagues. He was patient, kind, and ever mindful of the insecurities that so often engulf a graduate student in the throws of her dissertation. I offer my sincere gratitude to the other members of my committee, Dr. Barbara Morgan-Fleming, Dr. Carl Seaquist, and Dr. Eugene Wang. They are exemplary teachers and mentors from whose deep wells of knowledge I have insatiably drunk. They have my undying respect. No man is an island. This research could not have been completed without the support of family and friends. I would like to thank my father, James Burnett, who

ii


believed in me, my children, Sabrina and Adam, who would not allow me to quit, and Dr. Susan Duncan who repeatedly dispelled my doubts. I love you all.

iii


TABLE OF CONTENTS ACKNOWLEDGEMENTS .................................................................................................. ii ABSTRACT .................................................................................................................... vii LIST OF TABLES ............................................................................................................ ix LIST OF FIGURES .......................................................................................................... xi CHAPTER I INTRODUCTION .............................................................................................................. 1 Assessment................................................................................................................. 2 Adolescents’ Misconceptions .................................................................................... 4 Learning Objectives Regarding Lunar Phases ........................................................... 6 Statement of the Problem ........................................................................................... 7 Research Questions .................................................................................................... 8 Definition of Terms.................................................................................................... 9 Summary of Introduction ......................................................................................... 10 CHAPTER II LITERATURE REVIEW.................................................................................................. 11 Methodology for the Development and Validation of Previous Instruments .......... 12 Lunar Phases Concept Inventory ............................................................................. 16 Lunar Phase Domains for Assessment Development .............................................. 19 Periodicity........................................................................................................................ 21 Periodicity misconceptions.......................................................................................... 22 Motion.............................................................................................................................. 22 Motion misconceptions................................................................................................ 23 Terminology ..................................................................................................................... 28 Phase Appearance ........................................................................................................... 29 Phase appearance misconceptions ............................................................................... 30 Cause of Lunar Phases .................................................................................................... 32 Cause of lunar phases misconceptions ........................................................................ 33 Scale................................................................................................................................. 39 Scale misconception .................................................................................................... 40 Phase Geometry ............................................................................................................... 42 Phase geometry misconception.................................................................................... 43 Phase Sequence................................................................................................................ 46 Phase sequence misconceptions .................................................................................. 47 Cardinal Direction........................................................................................................... 47 Phase Time....................................................................................................................... 48 Global Lunar Perspective ................................................................................................ 49

iv


Summary .................................................................................................................. 50 CHAPTER III METHODOLOGY ........................................................................................................... 52 Purpose and Objectives ............................................................................................ 52 Research Design....................................................................................................... 53 Groundwork for Construction of Draft Items .................................................................. 53 Item Selection................................................................................................................... 56 Administration.................................................................................................................. 56 Validation......................................................................................................................... 57 Statistical Analyses .......................................................................................................... 58

CHAPTER IV RESULTS AND DISCUSSION .......................................................................................... 62 Statistical Analyses .................................................................................................. 63 Exploratory Factor Analysis............................................................................................ 63 Confirmatory Factor Analysis ......................................................................................... 72 Identification of Factors............................................................................................... 78

Expert Review.......................................................................................................... 84 Content Validity ............................................................................................................... 85 Evaluation of Item Distractors......................................................................................... 92

CHAPTER V SUMMARY AND CONCLUSIONS .................................................................................... 93 Introduction .............................................................................................................. 93 Summary of the Research ........................................................................................ 93 Summary of the Findings ......................................................................................... 96 Exploratory Factor Analysis............................................................................................ 96 Confirmatory Factor Analysis ......................................................................................... 97 Expert Review .................................................................................................................. 97

Conclusions .............................................................................................................. 97 Limitations of the Research ................................................................................... 100 Recommendations for Future Research ................................................................. 101 REFERENCES .............................................................................................................. 103 APPENDIX A Comprehensive Moon Phases Assessment ............................................................ 111 APPENDIX B States Which Established Standards for Each Lunar Phase Concept Domain Assessed by the CMPA.......................................................................................... 120

v


APPENDIX C State Standards Addressing Lunar Phases ............................................................. 123 APPENDIX D Taxonomy of 33 Guidelines for Creating Multiple-Choice Tests that Achieved Consensus from 46 Authoritative Sources............................................................. 182 APPENDIX E Level of Abstraction of Each Item of the CMPA According to Bloom’s Taxonomy.......................................................................... 184 APPENDIX F Expert Evaluation of the Comprehensive Moon Phases Assessment (CMPA) ..... 193 APPENDIX G Expert Evaluation of Item Content Validity .......................................................... 194 APPENDIX H Expert Evaluation of Item Distractors ................................................................... 248

vi


ABSTRACT Adolescents from across the United States experience instruction targeting lunar phase concepts. However, a single, age-appropriate, multiple-choice instrument that measures understanding of these concepts could not be located. Such an instrument must measure knowledge valued by the 50 state education agencies and the knowledge of lunar phase differences due to hemisphere perspective (ushered in by Internet learning forums). The design of this much needed assessment must take into account common lunar phase misconceptions held by adolescents. This research reports the development and validation of such an instrument called the Comprehensive Moon Phases Assessment (CMPA). State standards and previous research of adolescent misconceptions pertaining to lunar phase concepts framed the CMPA design. The method used to analyze the state standards and identify lunar phase domains was the constant comparative method (Glaser, 1965; Strauss & Corbin, 1994). The taxonomy of rules for writing multiplechoice items compiled by Haladyna and Downing (1989) directed the construction of the CMPA items. The research questions that directed the validation of the CMPA were: (1)

Does the CMPA measure the lunar phase constructs found in the state standards?

(2)

Does the CMPA measure the construct of global lunar perspective?

Research questions were answered through expert review and exploratory and confirmatory factor analyses. vii


An exploratory factor analysis showed 19 items loading onto a 7-factor model, with a confirmatory factor analysis suggesting it to be a model of good fit: χ2 = (75, N = 865) = 162.179, p < 0.001, the CFI was 0.964, and the RMSEA was 0.037. While no item received strong scores of five (on a 5-point Likert scale) from all eight reviewers, seven reviewers rated 21 of the 40 CMPA items with scores of four or five. These 21 items were evaluated as valid. Two or more members of the expert review board, using a 5-point Likert scale, ranked 10 of the 19 items that loaded onto a 7-factor model with a score of 3 or lower on content validity. Future research needs to consider feedback from the expert reviewers in developing or revising additional items for the seven factors that were confirmed by the analysis. It is suggested that a revised version of the CMPA be administered to a new sample of middle school students and a factor analysis conducted on the new assessment data.

viii


LIST OF TABLES 2.1

Domains Assessed by the Lunar Phases Concept Inventory ...............

17

2.2

Lunar Phase Concept Domains Assessed by the CMPA and Number/Percentage of States that Established Standards for Each .....

20

Frequency of Motion Models by Grade (Vosniadou & Brewer, 1994) ...............................................................

27

2.4

Number/Percentage of States including Each Term ............................

29

2.5

Students’ Notions about the Phases of the Moon ................................

35

2.6

Understanding of Cosmic Body Size by Third and Sixth Grade Males and Females (Jones & Lynch, 1987) .....................

40

Lunar Phase Concept Domains and Number and Percentage of States that Established Standards for Each Domain ......

55

4.1

AIC Values for Models ........................................................................

64

4.2

Factor Pattern Coefficients for Items of the CMPA .............................

65

4.3

Factor Structure Coefficients for Items of the CMPA .........................

67

4.4

Factor Correlations and Standard Errors ..............................................

69

4.5

Factor Determinacies ...........................................................................

70

4.6

Correlation Matrix, Means, and Standard Deviation for Items of the 7-Factor Model ...........................................................

73

4.7

Standardized Loadings (Standard Errors) for 7-Factor Model ............

75

4.8

Factor Correlations and Standard Errors for 7-Factor Model ..............

76

4.9

CMPA Items Loaded on Factors ..........................................................

78

4.10

Rankings for Content Validity by Expert Reviewers ...........................

86

4.11

CMPA Items Loaded on Factors and Received High Ratings by Six of Eight Experts ...........................................................

88

2.3

3.1

ix


4.12

Rankings for the Wording Appropriateness by Expert Reviewers .................................................

90

5.1

CMPA Items Constructed for Each Lunar Phase Domain ...................

95

5.2

CMPA Items Loaded on Factors ..........................................................

96

x


LIST OF FIGURES 2.1

Mental Models of the Day/night Cycle (Vosniadou & Brewer, 1994) ...............................................................

25

2.2

Student Notions of the Moon’s Phases (Baxter, 1989) ........................

33

2.3

Models for Vision Held by Children Aged 9-10 (Selley, 1996) ..........

37

2.4

Models of the Sun-Earth-Moon System (Jones & Lynch, 1987) .........

44

4.1

The 7-factor Measurement Model ........................................................

71

4.2

Parameter Estimates for the Items of the 7-Factor Model ....................

83

5.1

Diagram of Percentage of CMPA Items Identified as Valid by Expert Reviewers and Factor Analysis .............................

98

xi


CHAPTER I INTRODUCTION The behavior of celestial bodies has long fascinated Earthly inhabitants. Copernicus, Ptolemy, and Aristarchus have taken up permanent residence in contemporary textbooks to enlighten students of just a few of the numerous achievements in astronomy. But these three were certainly not the only heavenly scholars, for peoples from diverse cultures have studied the sky since the beginning of time. Knowledge acquired through extensive investigation of the Sun, Moon, and stars has informed us of changes in weather and climate, enabled the surveying and mapping of Earth, and directed crop production. Our quest to master the mysteries of astronomy is relentless. Will the structure and order of our solar system help us better understand the structure and order of the miniscule components of nature? This human interest in space has led the National Science Teachers Association to propose that the study of aerospace (defined as the Earth’s atmosphere and the region beyond) “offers a relevant context for the learning and integration of core content knowledge, makes numerous multidisciplinary and multicultural connections, and directly addresses content standards in many subject areas (NRC 2006)” (NSTA, 2008). Furthermore, NSTA claims that critical thinking skills are cultivated in the pursuit of aerospace knowledge. Based on this claim, it follows that students who have acquired a sophisticated understanding in space science have consequently subjected their critical thinking skills to a workout. One way for instructors to determine whether their students have achieved the

1


learning objectives (in space science or any other area of study) is through assessment. Summative assessments can effectively inform educators about their success, or lack of success, as well as evaluate students’ achievements. Assessments may also be used to direct instruction. Assessments used for these purposes are often referred to as educative assessments (Wiggins, 1998) or formative assessments (Boston, 2002; Crooks, 2001).

Assessment W. James Popham (2008), a national leader in the field of educational evaluation, defined assessment as “a formal attempt to determine students’ status with respect to educational variables of interest” (p. 6). The term, educational variables, in this definition refers to a broad range of skills and knowledge. Educative assessment can also be defined as a judgment of where we are with respect to where we want to be (Wiggins, 1998). In order to come to a valid determination of a student’s status, a task or instrument is needed. Although many situations may call for an authentic task, the most objective method for assessing students’ progress is with a paper and pencil instrument. Because assessments are formulated based on instruments’ scores, the instruments themselves have come to be known as assessments. For the remainder of this manuscript, the term assessment will be used to describe the determinations based on an instrument’s scores as well as the instrument used to make those resolutions. While most people are quite familiar with what assessments are, some may be unfamiliar with their numerous purposes. Currently, the prominent purpose for assessment is to assign grades (Kirschenbaum, Simon, & Napier, 1971). Most

2


classroom assessment is conducted for this reason. Teachers gather evidence of students’ knowledge or performance, much as a defense lawyer does prior to a court case, in order to assign a grade. The outcry for accountability from parents, administrators, and bureaucrats demands that assessment be conducted for this reason (Wiggins, 1996). Another reason to assess students’ status is to identify their strengths and weaknesses (McMillan, 2004). Since time is in short supply and therefore a precious resource, wise stewards will direct their instructional time toward students’ weaknesses. This direction can only be pursued, however, once the teacher has diagnosed these weaknesses. A third purpose for assessment is to keep track of students’ progress. Research has shown that tracking a student’s progress toward achieving learning objectives is more effective than comparing students’ progress with their peers (Cameron & Pierce, 1994; Kluger & DeNisi, 1996). Effective teachers design their lessons with educational objectives in mind. Assessment will inform teachers when students have either achieved or fallen short of the targeted objective. If students have successfully mastered the targeted content or skill, then the assessment can be looked upon as a summative assessment. If, however, students have failed to meet the teachers’ academic expectations, then the assessment can be characterized as one of a formative nature. In the latter case, the assessment can serve to direct the design of future lessons. Fourth, assessment can also be used to measure a teacher’s instructional effectiveness (Cambridge, 1996). If student understanding is measured before a unit of study commences and then at its completion, then teachers can attribute some portion of student learning (or lack of learning) to their instructional

3


design and execution. Finally, assessments are used to make judgments about a school’s achievement, as well as the success of its programs (Stark & Thomas, 1994). Assessments are invaluable to teachers who hold to the constructivist theory of learning. The constructivist teacher understands that students build their own knowledge bases by synthesizing new experiences with prior knowledge. With this in mind, assessing students’ prior knowledge is crucial to designing instruction that will facilitate knowledge construction. In the instances where students’ prior knowledge is faulty, identifying misconceptions is even more critical in informing teachers’ curriculum to bolster knowledge reconstruction.

Adolescents’ Misconceptions A multitude of experts in learning theory agree that adolescents bring their own ideas to the classroom (Dai, 1990; Driver, Guesne & Tiberghien, 1985; Erickson, 1979; Stead & Osborne, 1980; van de Berg & Sandura, 1990). Some of these ideas are shared by scientists while others are not. Conceptions not shared by contemporary scientists are often referred to as misconceptions. Misconceptions may be transmitted by parents, textbooks, instructional models (Callison & Wright, 1993), or even teachers (Lawrenz, 1986; Sadler, 1987; Skamp, 2004; Trundle, Atwood, & Christopher, 2002). However, they are most likely knowledge constructions formulated in early childhood, based on personal experiences and observations, interpreted with childlike logic. “Children are not blank slates when they are first exposed to the culturally accepted, scientific views, but bring to the acquisition task some initial knowledge about the physical world that appears based on interpretations

4


of everyday life” (Vosniadou & Brewer, 1994, p. 124). Although their knowledge is founded on personal experience, their underdeveloped critical thinking abilities hinder them from using those experiences to draw scientifically sound conclusions. As a result, adolescents’ understanding of the natural world is often at odds with the knowledge of contemporary scientists. For example, children often believe that if a bowling ball and a marble were simultaneously dropped from the same height, the bowling ball would reach the ground first. In 1687, Isaac Newton postulated that gravitational force is a constant. Yet, it was not until 1798 that Henry Cavendish measured this force. Because the gravitational force exerted by the Earth on all objects is the same, a bowling ball and a marble dropped at the same moment from the same height would hit the ground at exactly the same time (if all other factors are held constant). Although children that hold to this misconception regarding gravity are in agreement with scientists up until the 17th century, educators have fervently attempted through instruction to assist them in restructuring their thinking to align it with that of present-day scientists. However, because these misconceptions are molded from personal experience, they are exceedingly resistant to modification. Keeping in mind the prominent role misconceptions play in the construction of knowledge, it follows that teachers must identify their students’ misconceptions in order to address them in classroom instruction. Contemporary constructivists believe “What children need in order to abandon their synthetic model[s]…is instruction that focuses on their underlying entrenched beliefs…Otherwise, one misconception will be followed by another and students will remain confused” (Vosniadou, 1991, p. 234).

5


An objective and expedient way to identify these misconceptions is through the administration of a suitable assessment.

Learning Objectives Regarding Lunar Phases The National Research Council (1996) and the American Association for the Advancement of Science (1993) set learning objectives and benchmarks for adolescents regarding lunar phase understanding. The National Science Education Standards states: The understanding that students gain from their observations in grades K-4 provides the motivation and the basis from which they can begin to construct a model that explains the visual and physical relationships among earth, sun, moon, and the solar system. Direct observation and satellite data allow students to conclude that earth is a moving, spherical planet, having unique features that distinguish it from other planets in the solar system. From activities with trajectories and orbits and using the earth-sun-moon system as an example, students can develop the understanding that gravity is a ubiquitous force that holds all parts of the solar system together. Energy from the sun transferred by light and other radiation is the primary energy source for processes on earth's surface and in its hydrosphere, atmosphere, and biosphere. By grades 5-8, students have a clear notion about gravity, the shape of the earth, and the relative positions of the earth, sun, and moon. Nevertheless, more than half of the students will not be able to use these models to explain the phases of the moon, and correct explanations for the seasons will be even more difficult to achieve. (NRC, 1996, p. 159) The American Association for the Advancement of Science (1993) states: By the end of the second grade, students should know that (1) the sun can be seen only in the daytime, but the moon can be seen sometimes at night and sometimes during the day, (2) the sun, moon, and stars all appear to move slowly across the sky, and (3) the moon looks a little different every day but looks the same again about every four weeks. By the end of the fifth grade, students should know that (1) the earth is one of several planets that orbit the sun, and the moon orbits around the earth, (2) the rotation of the earth on its axis every 24 hours produces the night-and-day cycle. To people on earth, this turning of the planet 6


makes it seem as though the sun, moon, planets, and stars are orbiting the earth once a day, (3) light travels and tends to maintain its direction of motion until it interacts with an object or material. Light can be absorbed, redirected, bounced back, or allowed to pass through. By the end of the eighth grade, students should know that (1) the earth is orbited by one moon, many artificial satellites, and debris, (2) the moon's orbit around the earth once in about 28 days changes what part of the moon is lighted by the sun and how much of that part can be seen from the earth- the phases of the moon. (AAAS, 1993, http://www.project2061.org/publications/bsl/online/ index.php?chapter=4#A2) Each state of the United States took these standards into consideration in establishing their individual learning objectives targeting lunar phase knowledge. An instrument based on the state standards and written in age-appropriate language that will effectively measure early adolescent students’ learning achievements and identify misconceptions in lunar-phase concepts is needed to inform and direct teaching and learning.

Statement of the Problem The lunar phases can be easily observed from almost every location around the globe and an inquiry-based study of this natural phenomenon can be conducted at little cost. The topic of lunar phases has long provided an engaging study for adolescents to learn the process of investigation and inquiry. For these and other reasons, all of the fifty states of the United States have included lunar phase objectives in their education standards. As a result, middle school science teachers throughout the United States have and will continue to devote classroom time to purposeful instruction targeting the numerous lunar phase concepts while engaging their students’ critical thinking skills and enculturating them in the scientific process. Indeed, many science teachers

7


collaborate with researchers and curriculum designers to improve curriculum and instruction pertaining to these concepts. The Lunar Phases Concept Inventory (LPCI), a multiple-choice instrument, has recently been used in research of this topic in middle school science classrooms (Lindell & Olsen, 2002). However, the LPCI was developed for college-age astronomy students. Currently, there is no single, multiple-choice test, designed specifically for middle level students. Such a test would need to measure understanding, respected and valued by science education experts. Lunar phase misconceptions commonly held by adolescents must also inform the development of this much needed instrument. Finally, this futuristic assessment should measure lunar phase understanding that has been ushered in by the use of the Internet in instruction (e.g., differences in northern and southern hemisphere perspectives of the lunar phases). In other words, an age-appropriate, standards-based instrument that measures lunar phase misconceptions and understanding in differences in global lunar perspectives is needed to effectively assess middle school students’ understanding of lunar phase concepts. This dissertation reports the development and validation of such an instrument called the Comprehensive Moon Phases Assessment (CMPA). The CMPA can be found in its entirety in Appendix A.

Research Questions The overarching question on which this study will focus is: What knowledge of lunar phase concepts does the Comprehensive Moon Phases Assessment measure in early adolescent students?

8


The specific research questions are: 1.

Does the CMPA measure the following lunar phase constructs found in the state standards: a. Periodicity b. Motion c. Lunar Phase Terminology d. Phase Appearance e. Cause of Lunar Phases f. Scale g. Phase Geometry h. Phase Sequence i. Cardinal Direction j. Phase Time

2.

Does the CMPA measure the construct of Global Lunar Perspective (GLP) in addition to those found in the state standards?

Definition of Terms The following terms are defined according to their use in this research study: •

Assessment items – items of the CMPA that assess lunar phase understanding

•

Early adolescents – children between the ages of 9 and 14 years

•

Construct – interrelated variables

9


•

Construct validity – the degree to which an instrument measures the theoretical constructs it was designed to measure

Summary of Introduction Assessments can be administered at the end of instruction to determine whether students and teachers have achieved their educational goals. Assessments can also be administered before instructional intervention to identify students’ misconceptions. Teachers can then use this information to direct curricular and instructional design to facilitate a more scientifically sound construction of knowledge. While an instrument is available to measure lunar phase understanding of college-age astronomy students, no multiple-choice instrument exists that effectively assesses knowledge of concepts related to lunar phases (standards-based, misconceptions, and global perspective) in early adolescents. This research study describes the processes of development and validation of such an instrument.

10


CHAPTER II LITERATURE REVIEW In order to provide middle school science teachers with a valid multiple-choice instrument to measure their students’ understanding of lunar phase concepts, a comprehensive list of expectations needed to be compiled. This list consists of the knowledge adolescents are expected to have previously constructed and the knowledge they are expected to construct during formal instruction. The lunar phase concept domains of the LPCI were used to synthesize this vast amount of knowledge. The development of the CMPA essentially was based on the following resources: (1) state standards targeting lunar phase understanding, (2) lunar phase concepts that address differences between northern and southern hemispheres, and (3) lunar phase misconceptions commonly held by adolescents. The literature that was reviewed to inform the development of an instrument to measure understanding of concepts related to lunar phases had four foci. First, previous research that described the development and validation of a wide range of instruments provided a broad overview of how to create a valid instrument specifically designed for use with adolescents. The Lunar Phases Concept Inventory (Lindell & Olsen, 2002) is a multiple-choice instrument that was recently developed for college astronomy students. Its strengths and weaknesses were examined and are presented next. The standards pertaining to lunar phase concepts set forth by all 50 states of the United States were harvested, analyzed, and sorted into ten domains. A summary of the states’ lunar phase standards will follow the strengths and weaknesses of the LPCI.

11


Finally, the research findings on lunar phase misconceptions held by early adolescents were summarized and categorized into the same ten domains. Lunar phase misconceptions will be presented after the summary of state standards within each domain.

Methodology for the Development and Validation of Previous Instruments The methodologies followed to develop a variety of instruments that assess knowledge, skills, behavior, or dispositions of adolescents or children were reviewed to direct the development and validation of the CMPA. A broad overview of how to create valid instruments to be used with adolescents follows. Kiss and Nikolov (2005) examined several language aptitude tests in their efforts to design a language aptitude test for 12-year-old students. Although they found existing instruments lacking in one area or another for measuring the aptitude of Hungarian adolescent learners of English, they adopted the task types used in one of the existing tests. Mattheis and Nakayama (1988) modified the process skill objectives of the Middle Grades Integrated Science Process Skills Test (Cronin & Padilla, 1986) to frame the items of their new instrument, Performance of Process Skills (POPS), which measures middle school students’ integrated science process skills. In the absence of existing tests, designers often conduct research to define constructs or dispositions they wish to identify or measure. To develop the Writing Disposition Scale (WDS), empirical and theoretical research was conducted to “identify the critical affective stances [confidence, persistence, and passion] that compose the dispositional side of writing” (Piazza & Siebert, 2008, p. 277). The EArth

12


Representation Test for cHildren, or EARTH (Straatemeier, van der Maas, & Jansen, 2008), began with the erroneous mental models children carry of the Earth reported by Vosniadou and Brewer (1992). Once the foundation is set, test items are drafted. Establishing criteria for item construction helps to maximize the development efforts. In developing the POPS, an instrument which measures middle schools students’ integrated science process skills, Mattheis and Nakayama (1988) deemed the following criteria important: (1) students must have multiple opportunities to demonstrate their competence in each skill, (2) a wide range of difficulty must be represented in the items, (3) the test must require a suitable amount of reading, (4) the instrument must constructed at the appropriate reading level, (5) the test must not require students to understand specific terms, (6) the format of the instrument must be multiple-choice with the option of four answer choices, and (7) the length must allow students to complete the test in 25 minutes. Feedback from the targeted population and experts on the topic are often used to refine the instrument. In the development of the Healthy Eating Self-monitoring Tool, or HEST (Di Noia, Contento, & Schinke, 2007), the fruits and vegetables that are commonly consumed by African American male adolescents were discovered in focus groups made up of African American male adolescents. A second focus group provided feedback on the computer screen format in which participants were to record their data. Children are sometimes randomly selected from the sample that completed the pilot version of the instrument for interviews as in the development of EARTH

13


(Straatemeier, van der Maas, & Jansen, 2008). Information acquired from student interviews are then used to revise items and improve the instrument. Expert panels also agree to validate and provide suggestions for improving instrument. To validate the performance-based and multiple-choice versions of the Test of Basic Process Skills in Science, BAPSST and BAPS respectively (Padilla, Cronin, & Twiest, 1985), the developers first sent the instruments to an expert panel for verification of content validity. The 40 items of the POPS were also sent to an expert panel of four science educators who were experienced in integrated science process skills and test construction. Their agreement on the correct answer and keying to the objective process skill for 21 of the 40 items, served as evidence of content validity. Face and content validity of the WDS items were established by a panel of five experts who used a 5-point Likert scale to rate the items on content and the extent to which they measured one of the three stances (confidence, persistence, and passion) of writing. Experts were also asked to evaluate the wording of the WDS items on their suitability for elementary and middle level students. A third sample of African American male adolescents evaluated the revised version of the HEST for criterion validity. Some instruments that are developed to assess knowledge, skills, behavior, or dispositions of adolescents or children are also evaluated on their readability. The FOG index (Gunning, 1968) was used to assess the readability of the POPS. The language level of the WDS was calculated with more than one readability index; however, they were not named.

14


After items are discarded or revised, based on sample or expert feedback, a new version of the instrument is often administered to a new sample. Kiss supervised the administration of the language aptitude test. Developers are not always able to do that, however. Data collected from the administration of the revised instrument is then subjected to analyses. In the development of the HEST, student self-reported consumption was compared to student consumption observed and reported by an outsider. Results of both were correlated to determine the validity of the HEST. Similarly, a randomly selected sample of elementary school students from the Netherlands who had completed the EARTH-2 was interviewed. Five raters scored the children’s answers to interview questions. The raters’ scores were then correlated to determine the validity of EARTH-2. In validating the BAPSST and BAPS, sample participants completed both instruments as well as two similar tests that had already been validated. High correlations between measures of the same traits of the four tests served as evidence of validity. Likewise, low correlations between measures of different traits of the four tests were also evidence of the validity of BAPS and BAPSST. Reliability, item difficulty, and item discrimination were reported to show evidence of validity and reliability of the POPS (Mattheis & Nakayama, 1988) and the language aptitude test developed by Kiss and Nikolov (2005). In the validation of the WDS, an exploratory factor analysis was conducted on a randomly selected half of the sample, while a follow-up confirmatory analysis was conducted on the other half of the sample. Redundant information was removed by identifying variables that were highly correlated. A principal component analysis helped to refine the selection of

15


variables. Principal axis factor analyses were used to identify items that adequately represented the latent factors. The maximum likelihood method estimated the fit of the data to the model. The procedures for developing and validating an instrument that measures adolescent knowledge, skills, behavior, or dispositions may vary depending upon the instrument’s objective. Understanding the rationale of the methodology followed in the development and validation a variety of instruments offered insight into the appropriate methodology for the development and validation of the CMPA.

Lunar Phases Concept Inventory The LPCI (Lindell & Olsen, 2002) is a multiple-choice inventory that was recently developed for college students studying astronomy. The LPCI informed the construction of the CMPA items in that the lunar phase concept domains young adults are expected to master will structure the organization of the lunar phase learning objectives state education agencies expect early adolescents to achieve. Both concept maps (LPCI and the state standards) will direct construct continuity. Lindell and Olsen recognized that students’ prior understandings influence the ways in which students construct their future knowledge, even when the students are of college age. They concluded that teachers need to identify both the mental models pertaining to lunar phases that reside in the schema of their students and the magnitude in which they are thoroughly entrenched. To assist university instructors in identifying their students’ understanding of lunar phases, Lindell and Olsen (2002) developed the Lunar Phases Concept Inventory (LPCI). After first conducting a qualitative

16


investigation of lunar phase mental models held by students prior to formal instruction, Lindell and Olsen designed their inventory based on the Model analysis theory which asserts that a student’s mental model and the consistency with which he or she uses that model can be revealed through a mathematical analysis of his or her responses to the various items on an instrument. This information directed the design of the LPCI, a multiple-choice instrument that assesses the lunar phase mental models of college students. The lunar phase conceptual domains assessed by the final version of the LPCI are listed in Table 2.1 shown below.

Table 2.1. Domains Assessed by the Lunar Phases Concept Inventory Lunar Phases Concept Domain Period of the Moon’s orbit around the Earth Direction of the Moon’s orbit around the Earth as viewed from a point above the north pole Period of the Moon’s cycle of phases Motion of the Moon Phase and Sun-Earth-Moon positions Phase – location in sky – time of observation relationship Cause of Lunar Phases Effect of lunar phase with change in location on Earth * Science domains identified by Lindell & Olsen (2002) 17


The instrument was field tested at five universities of different sizes. A total of 346 college students, from four of the five universities, took the LPCI as a post-test to instruction. A Cronbach’s alpha of 0.75 was reported for the post-test scores. However, since this reliability testing was conducted, seven of the 20 items on the LPCI have been revised. Content validity of the LPCI was established by expert review. For this reason, the lunar phases concept domains will serve as a foundation for the development of the Comprehensive Moon Phases Assessment. Although this instrument might be adequate for students at the university level, it has weaknesses that compromise its use with adolescents. Haladyna and Downing (1989) examined 46 authoritative sources in educational measurement. They compiled a taxonomy of rules for creating multiple-choice assessment items that is considered to be complete and authoritative. Several items in the LPCI violate some of these rules. For example, eight of the 20 LPCI items include “None of the above” as an answer choice, while 72% of the authoritative sources in educational assessment agreed that this phrase should be avoided. The last item of the LPCI items asks students: Which direction did the Moon travel around the Earth? Students are offered these three choices 1) Clockwise, 2) Counter Clockwise, and 3) Either Direction. This can only be answered if a vantage point is identified. Someone in the southern hemisphere would correctly answer this question by selecting “Clockwise”, while someone in the northern hemisphere would correctly answer this question by selecting “Counter Clockwise”. However, the item that precedes this question identifies a vantage point as “above the Earth’s North Pole.” While a university student might conclude that the

18


item was intended to be interpreted with information from the previous item, an adolescent should not be expected to make that inference. Finally, adolescents might question the plausibility of the Moon traveling around the Earth in “either direction.” The rule “Use plausible distractors” was included in 43 of the 46 authoritative sources in educational measurement. The lunar phase concept domains addressed by the LPCI served as a filter through which the state lunar phase standards were analyzed and categorized. The state lunar phase standards, the global lunar perspective goals, and the lunar phase misconceptions framed the development of the CMPA.

Lunar Phase Domains for Assessment Development The student expectations regarding lunar phase concepts vary from state to state and grade level to grade level with as much diversity as the landscapes from coast to coast. Some states communicate low-level expectations with the use of verbs like recognize, explore, and recall. Others convey higher expectations with the use of verbs like describe, explain, and model. Often, standards are framed with the verb know, yet these offer no specific way for students to demonstrate what they know. This wording may have been purposefully chosen to provide teachers freedom in choosing tasks or tools to measure their students’ knowledge. A thorough comparison of the standards by state is beyond the scope of this dissertation. However, each lunar phase concept domain that emerged in the state standards is listed in Table 2.2. The number and percentage of states that include at least one standard for each of the lunar phase concept domains can also be found in Table 2.2. Tables that indicate which

19


lunar phase concept domains were addressed in the space science education standards for each state can be found in Appendix B. Tables that include the education standards targeting concepts related to lunar phase understanding for each state are provided in Appendix C.

Table 2.2. Lunar Phase Concept Domains Assessed by the CMPA and Number/percentage of States that Established Standards for Each Lunar Phases Concept Domain

Number/Percentage of States

Periodicity

49 (98%)

Motion

46 (92%)

Terminology*

41 (82%)

Phase Appearance*

39 (78%)

Cause of Lunar Phases

30 (60%)

Scale*

22 (44%)

Phase Geometry

20 (40%)

Phase Sequence*

7 (14%)

Cardinal Direction

4 (8%)

Phase Time

3 (6%)

Global Lunar Perspective*

0 (0%)

*Domains not found in the LPCI

20


A summary of how the states communicated their learning objectives in each domain is followed by domain specific misconceptions uncovered in previous research. Periodicity The domain of Periodicity encompasses the knowledge that both the Moon’s changing appearance and day and night occur in a cyclical, predictable pattern. All of the 50 states, with the exception of Wyoming, have included space science education standards that fit within the domain of Periodicity. Thirty-two states used the word pattern in their standards that fell within the domain of Periodicity as defined above. Forty-three states value the knowledge that the cycle of day and night is completed in approximately 24 hours. Furthermore, these 43 states expect their students to know that the Earth’s rotation causes this daily solar cycle. Sixteen states want their students to know that the Moon’s appearance changes a little from day to day but does so in a predictable cycle. Some refer to this pattern as a natural cycle. Indeed, 14 states want students to understand that the duration of the lunar cycle is approximately four weeks, 28 days, or one month and the result of the Moon’s revolution about the Earth. Periodicity is also exhibited more specifically within the lunar cycle, in that the period of time between certain lunar phases (as defined by man) can be reasonably approximated. Periodicity within the lunar cycle will be addressed in the domain of Phase Time. Periodicity is intertwined with the domain of Motion, while Phase Time is bound by the domains of Motion and Phase Geometry.

21


Periodicity misconceptions Misconceptions in this domain are common. Results of a large cross-age study (Schoon, 1992) involving 1202 participants from early adolescence to college-age showed that 36% believed that the Moon completes its orbit about the Earth in one day, while 20% thought the Moon travels around the Earth in one year. Almost 20% attributed the day/night period to the Earth’s orbit about the Sun. Motion The domain of Motion incorporates the real movements of rotation and revolution, as well as the apparent movement of the Moon which appears to rise, move across the sky in a westerly direction, and then set along the western horizon, as a result of the Earth’s rotation. Understanding of concepts in this domain is highly regarded in that 46 states have addressed the Earth’s and Moon’s motions in their space science standards. While all desire that their students come to understand the real motion of the Earth and Moon, 31 communicated an expectation that students associate apparent movement of the Moon across the sky with the real movement of the Earth on it axis. This was conveyed by the use of the phrases “appear to move,” “seem to move,” or “observe and describe the motions.” It was determined that motions that could be observed and described were indeed apparent motions of the Sun, Moon, and stars across the sky. It is important for teachers to address the movement that has been or can be easily observed by their students. To target only the real movement and fail to address the apparent movement will most likely leave students in cognitive conflict.

22


Georgia included the standard, “Students will explain the motion of objects in the day/night sky in terms of relative position.” This might be interpreted as a reference to the Moon’s apparent motion; however Georgia was not included as one of the 31 states due to the ambiguity of this standard. Utah established the following learning objective for their third-grade students: Describe the movement of Earth and the moon and the apparent movement of other bodies through the sky. Use a model to demonstrate why it seems to a person on Earth that the sun, planets, and stars appear to move across the sky. The Moon’s apparent movement was omitted in two distinct places for students in grade three. However, it was addressed for students in grade six. Knowledge of motion is linked to understanding within the domains of Phase Geometry and Phase Time. It seems logical that students who fail to construct a firm foundation of knowledge in the domain of Motion will experience great difficulty understanding the concepts in Phase Geometry and Phase Time. Motion misconceptions Young children have been shown to possess alternative beliefs about the Moon’s motion. When Haupt (1950) asked a group of 20 first-grade students who were engaged in free discussion “Which way does the Moon move?” ten asserted that the Moon does not move. Three curious responses were also teased out by the question: (1) one child did not know, (2) one stated that the Moon moves around the Sun once every 24 hours, and (3) one said the Moon moves north and also moves south.

23


Alternate conceptions about the motion of the Sun, Earth, and Moon affect accurate knowledge construction of space science concepts. Participants in Schoon’s previously mentioned cross-age study (1992) provided a misconception as explanation for the day/night cycle. Almost nine percent of those surveyed believed the Sun orbits the Earth. In another study, sixty children (20 first-, 20 third-, and 20 fifth-grade students) were asked to sketch their mental models to explain the day/night cycle (Vosniadou & Brewer, 1994). It was hypothesized that elementary school children would construct one of three mental models to explain the day/night cycle. The three types of mental models they expected to find were (1) Initial models – models based on observations alone, (2) Synthetic models – models constructed in an attempt to reconcile their prior observations with the more culturally accepted (scientific) model, and (3) Scientific models – models that align with the contemporary scientific model. The eight models shown in Figure 2.1 are representative of the models sketched by the children in the study. The students who sketched initial models to explain the day/night cycle were 13 first-grade students, two third-grade students, and one fifth-grade student. A misconception demonstrated in one of the initial models is that the Sun and Moon move up and down on the ground. Some of the children that sketched Mental Model 3 in Figure 1 described the Moon’s movement in relation to the Sun’s movement as “hydraulic” indicating a belief that the Moon goes up when the Sun goes down and vice versa.

24


Figure 2.1. Mental Models of the Day/night Cycle (Vosniadou & Brewer, 1994).

25


In general, Vosniadou and Brewer found that the younger students believed that the Moon moves up and down in relation to the Earth’s horizon. Older students tended to believe that the Moon’s position is fixed while the Earth rotates on its axis. Frequencies of motion models chosen by students in the first and third grades are shown in Table 2.3.

26


Table 2.3. Frequency of Motion Models by Grade (Vosniadou & Brewer, 1994) Earth Motion 1. Rotates, revolves around sun

Moon Motion

Sun Motion

Grade Grade 1 3

Total

Rotates, revolves around earth Revolves around earth

None

0

1

1

None

1

5

6

Moves parallel to earth around sun None

None

0

1

1

None

2

2

4

5. Rotates, revolves around sun and moon

None

None

0

1

1

6. Rotates

Rotates

Rotates

1

0

1

7. Rotates

None

None

0

2

2

8. Rotates

Revolves around earth Revolves around earth

Revolves around earth Revolves around earth

0

1

1

0

2

2

Rotates, up and down Up and down

1

0

1

11. Rotates

Rotates, up and down Up and down

2

1

3

12. None

Up and down

Up and down

9

3

12

13. None

None

None

2

0

2

Rotates, up and down

1

0

1

19

19

38

2. Rotates, revolves around sun 3. Rotates, revolves around sun 4. Rotates, revolves around sun

9. None 10. Rotates

14. Rotates, Moves with revolves around sun earth around sun, up and down Total

27


These misconceptions of the movements of the Sun, Earth, and Moon can greatly hinder an accurate construction of knowledge pertaining to lunar phases. Teachers who desire to develop effective curriculum must take these misconceptions into consideration. “Presuppositions may be innate or empirically acquired constraints which are present from early infancy and which guide the way children interpret their observations and the information they receive from the culture to construct knowledge structures” (Vosniadou & Brewer, 1994, p. 124). A more recent study was conducted by Trumper (2001) with 448 adolescents in grades seven through nine who attended rural schools in Israel. Although almost half of the participants had an accurate understanding of the cause of day and night, the remainder had constructed faulty ideas such as the Earth moves around the Sun (36%) and the Sun moves around the Earth (11%). Terminology An understanding of basic terminology is needed in order to effectively communicate knowledge. Terms associated with lunar phase concepts are listed in Table 2.4 with the number and percentage of states that included each in their standards.

28


Table 2.4. Number/Percentage of States Including Each Term Number/Percentage of States

Term Solar eclipse/lunar eclipse

50 (100%)

Rotate/revolve

32 (64%)

Orbit

22 (44%)

Waxing/waning

2 (4%)

Crescent

1 (2%)

Gibbous

0 (0%)

Zenith

0 (0%)

Phase Appearance The domain of Phase Appearance embodies the visual descriptions and images corresponding to each lunar phase. While half of the Moon’s surface is always illuminated by the Sun’s rays, the Moon’s appearance changes from an Earthly perspective due to its changing position (in relation to the positions of the Earth and Sun) in its orbit about the Earth. Thirty-nine states set learning objectives for their students with respect to the Moon’s changing appearance. Most of these 39 states expect their students to either know the way in which the Moon’s appearance changes (e.g., waxing followed by waning) or identify, illustrate, or distinguish between various lunar phases. Arkansas wants their second-grade students to illustrate only four specific moon phases (i.e.,

29


new, crescent, half, and full). Six states, Georgia, Louisiana, Maryland, New York, North Carolina, and Vermont, require that their students not only observe but also record the Moon’s changing appearance in a journal. It must be noted, however, that the wording of standards from two states may foster a misconception. A Louisiana space science standard states “Students are to participate in observing and recording the changing appearances and positions of the Moon in the sky at night and determining the monthly pattern of lunar change.” A space science standard for Alabama students states “Describe the appearance and movement of Earth and its Moon, identifying the waxing and waning of the Moon in the night sky.” Both may promote the misconception that the Moon can only be seen at night. Georgia exhibited great care in wording their standards. For example, they insist that their students “Know that the Moon does not change shape, but at different times appears to change shape.” Phase appearance misconceptions Young children have interesting notions about the Moon’s appearance. For example, two first-grade students engaged in free discussion voiced their misconceptions about the Moon. One student claimed, “The sun has a job now and the moon has a job at night” (Haupt, 1950, p. 226). Many students have failed to notice the Moon’s appearance in the daylight hours. When asked if the Sun has anything to do with the Moon, another first-grade student in the study answered with “Only half the sun – the bright part. It reflects with the world as the world goes round. The other side of the sun is a piece which is the moon” (p. 226).

30


Prior to formal instruction in lunar phase concepts, early adolescents have limited experience with the various lunar phases. As a result, they tote misconceptions with them into their science classrooms. Knowledge of moon phases and predictable patterns of 48 fourth-grade students were investigated prior to instruction (Trundle, Atwood, & Christopher, 2007). Students were told they would be sketching the Moon on a daily basis over a nine-week period and then asked to draw all of the Moon’s shapes they expected to see. New moon omissions were not reported since no shape can be seen during a new moon phase and the number of full moons was excluded from the data collection because a full moon was provided as a prompt. Fewer than 50% of the students sketched a waxing crescent, first quarter, waxing gibbous, waning gibbous, or third quarter moon. Trundle et al. (2007) acknowledged that these omissions could be attributed to a memory lapse and concluded that the inclusion of shapes that cannot be observed is an indication of the presence of misconceptions. Over 56% of the combined drawings made for two different tasks illustrated non-scientific shapes. Shapes were described as nonscientific if (1) crescent phases were “over or under articulated” (p. 606), and (2) first quarter, third quarter, and gibbous phases were sketched similar to a partial lunar eclipse. Approximately 87% of the students sketched a partial lunar eclipse. The researchers interpreted these sketches to be erroneous illustrations of a gibbous moon and thus, evidence of the existence of the commonly held misconception that lunar phases are caused by the Earth’s shadow. It can be argued that students who drew a partial lunar eclipse may not have intended to illustrate a gibbous moon but simply

31


sketched an image they recalled from a newspaper photo of a partial lunar eclipse, for partial lunar eclipses, relatively rare in occurrence, draw greater attention than gibbous moons. That is, although the Moon would not be expected to take such a shape in its customary cycle, one cannot necessarily conclude from the sketch that the Earth’s shadow misconception is the culprit, at least not before formal instruction. Another alternative explanation for the production of sketches that resembled partial lunar eclipses is that some adolescents are artistically challenged. Nevertheless, misconceptions regarding the Moon’s changing appearance abound. Cause of Lunar Phases Students who are able to construct an understanding of the cause of lunar phases must synthesize their knowledge in the domains of Periodicity and Motion. They must know that the regular and predictable motions of the Sun, Earth, and Moon explain the natural phenomenon of lunar phases. Some might even argue that knowledge in the domains of Phase Geometry and Phase Appearance is also needed to fully explain the cause of the Moon’s phases. Thirty states expect their students to develop an understanding of the cause of lunar phases. However, the space science standards of Hawaii and Virginia document their expectations of knowing the cause of lunar phases without requiring their students to connect the specific phases with their corresponding images (Phase Appearance). Ohio, South Carolina, and Tennessee want their students to know the cause of lunar phases but do not expect them to relate each phase to its geometric configuration (Phase Geometry). Oklahoma, Oregon, and Texas expect their students to come to an understanding of the cause of lunar phases, yet

32


have established specific learning objectives in neither of the domains of Phase Geometry nor Phase Appearance. These state standards are evidence of the belief that an inability to connect the geometric configurations to the specific phases for each of the Moon’s relative positions to the Sun and Earth does not preclude the ability to explain the cause of the Moon’s cyclical change in appearance. Cause of lunar phases misconceptions Numerous misconceptions about what causes the Moon’s appearance to change have been unveiled in previous research. Some children believe that various parts of the Moon are hidden at times by the night, clouds, the sky, or rain. When asked what makes the Moon change its shape, two children in Haupt’s study (1950) answered this thought provoking question poetically. One stated, “The north wind is greedy and eats the moon a bit a day and the south wind blows it back.” The other commented, “A fairy washes different bits of the moon and puts them up in the sky. She’s the moon’s laundress and washes and pastes on parts of the moon” (p. 226). While these last two responses may not be widely held beliefs for the cause of lunar phases, they may provide insight as to the influence of literature on children’s knowledge construction of concepts relating to the natural world. Misconceptions regarding the cause of lunar phases may gain a foothold in the absence of proper instruction. In an effort to provide a middle school in southwest England with an astronomy curriculum, individual interviews were first conducted with 20 students to identify their misconceptions of lunar phases and how they were developed (Baxter, 1989). This information was then used to construct an instrument

33


to identify astronomy misconceptions on a much larger scale. The assessment was administered to 100 middle school students (ages 9 to 16 years) who were enrolled in the school. Baxter’s results were illustrated in the bar chart, shown below in Figure 2.2.

Figure 2.2. Student Notions of the Moon’s Phases (Baxter, 1989).

An estimation of percentages per age bracket is represented in Table 2.5 for comparison purposes. It appears that the percentage of students who adopted the faulty explanation that lunar phases are the result of the Earth’s shadow grew in each successive age bracket.

34


Table 2.5. Students’ Notions about the Phases of the Moon Explanation for the Moon’s Phases

Percentage Per Age Bracket

Clouds cover part of the Moon

Ages 9-10

Ages 11-12

10%

15%

Shadow of a planet

Ages 13-14

Ages 15-16

8%

Shadow of the Earth

50%

54%

69%

80%

Portion of illuminated side of the Moon visible from the Earth changes

40%

23%

31%

20%

Similar results were found in a cross-age study conducted by Schoon (1992). Fifth-grade students (n = 307), eighth-grade students (n = 237), eleventh-grade students (n = 340), college-age students (n = 226), and trade school students (n = 92) participated in a survey to determine common alternative conceptions in Earth and space sciences. Unfortunately, Schoon did not segregate his data according to age brackets. However, he did report that 48% of the total participants held the Earth’s shadow misconception for the cause of lunar phases. Additionally, it was reported that this alternative conception gained prominence with age. In fact, 70% of the collegeage participants had adopted this explanation. In 1991, 180 middle school students were surveyed by Bisard, Aron, Francek, & Nelson (1994). While 39% of these early adolescents indicated a correct explanation for the lunar phases, almost as many (38%) attributed the cause to the Earth’s shadow. Another 19% credited it to the varying angle of Sun’s rays reflected off the Earth. 35


Other questions included in Schoon’s (1992) multiple-choice survey exposed additional misconceptions that would most definitely hinder sophisticated knowledge construction regarding the cause of lunar phases. Over 15% thought the Moon is like a star and 10% believed the Moon radiates its own light. These two distractors were provided for the same survey question. Thus, almost 25% of Schoon’s 1202 participants had subscribed to this misconception. Children’s conceptions about light and vision may also shape their understanding of the cause of lunar phases. Selley (1996) investigated the development of children’s thinking pertaining to light and vision over a two-year period. Fourth-grade students (n = 21), who attended a middle school near London, were organized into groups of five or six and asked to represent their thoughts on how they are able to see a flower in a sketch. They were then encouraged to embellish their sketch with words or details that might help clarify the meaning of lines that had been drawn between the drawn objects (e.g., human eye, flower, Sun, or lamp). Children followed the sketching activity with conversation focused on explaining their sketches to the other children of their group. The various mental models of vision that were illustrated by the children are shown in Figure 2.3. Nineteen of the illustrations included lines drawn from the flower to the human eye. While only one student indicated a direction with the lines, the other 18 students reported that the lines were drawn on their sketches to show the line of sight. Consequently, Selley (1996) transformed their lines to rays in Figure 2.3 to represent the children’s explanation of line of sight that was communicated to the researcher

36


during group conversation. Eight of the 21 believed that the eye emits light to illuminate objects. Some of the eight believed the eye reflects sunlight. Some of the children thought that an object can only be seen when sunlight is added to the light emitted from the eye. Four children (through drawings and conversation) indicated that the eye takes a picture of an object, much like a camera.

Figure 2.3. Models for Vision Held by Children Age 9-10 (Selley, 1996).

37


The following year, the same students (now in the fifth grade) and five additional students were given a written test targeting the same ideas. Most of these children (75%) had now adopted the mental models in which rays linked the eye, light source and object. The other children held to the Sea of Light mental model. Almost one-half of the children (13/26) stated that the eye emits something (e.g., sight, a beam, light) in the act of seeing, yet eight did not believe this was so. Although the Cooperative Emission Model was not popular in the test, it gained support during the conversations. One child stated that the light bounced off of the object into the eye and back to the object. By the next year, most of the students (21/23) were convinced that the eye emits something in the act of seeing. Four students in the fifth grade had denied that the eye emits something, yet two of the four were persuaded to abandon their accurate model for an emission model in the sixth grade. In a separate study, four third-grade students who attended an elementary school in the Midwestern United States and were chosen as representatives of their class provided written surveys, two-dimensional sketches, three-dimensional modeling, and verbal responses to interview questions in an effort to communicate their explanations for lunar phases (Stahly, Krockover, & Shepardson, 1999). Two of the four students claimed one’s vantage point from the Earth is a factor in the appearance of lunar phases. However, one of the two indicated with his threedimensional model that the vantage point changes due to the Earth’s rotation while the other’s model demonstrated an understanding that one’s vantage point changes due to

38


the Moon’s revolution about the Earth. A third participant reported that the lunar phases are caused by the clouds. The fourth student offered no coherent explanation at all. Trumper’s (2001) research with middle school children from rural schools in Israel revealed that 19% of his 448 participants maintained that lunar phases are caused by the Earth’s shadow. Even more surprising was the discovery that 25% believed that the Moon’s appearance changes when it passes into the shadow of the Sun. Scale The domain of Scale is comprised of the knowledge of the differences in the size of the Sun, Earth, and Moon, as well as the understanding that the Sun is much farther away from the Earth than the Moon. Twenty-two states included standards pertaining to either differences in size or distance between any two of the Sun, Earth, and Moon. More specifically, ten states consider both important aspects in understanding lunar phases. None of the 22 states who held expectations in the domain of Scale insisted that students understand the specific sizes or distances in terms of miles or kilometers, but rather in relation to other sizes and distances. For example, one standard from Kansas states that students need to model “spatial relationships of the earth/moon/planets/sun system to scale.” However, Delaware included the following standard

39


The Sun is much larger than the Moon. Although the Moon is closer to Earth than the Sun, the two appear to be the same size when viewed from Earth. This is because objects appear smaller as the distance from the viewer increases. This understanding would also support knowledge of solar eclipses. In addition to authors of state standards, researchers have also been interested in students’ conceptions of scale of the Sun-Earth-Moon system. For example, how do students compare the sizes of the Sun and Moon to the Earth’s size? How do students compare the distances from the Earth to the Sun and Moon? Scale misconception Scale is often a difficult concept to grasp for early adolescents. Jones and Lynch (1987) found a statistically significant difference existed between males and females in an understanding of the differences in size of the three celestial bodies of the Sun-Earth-Moon system. Males had a greater understanding than females. Results of the gender and grade level comparisons in knowledge of cosmic body size are provided in Table 2.6.

Table 2.6. Understanding of Cosmic Body Size by Third and Sixth Grade Males and Females (Jones & Lynch, 1987)

Grade level

Incorrect size

Correct size

Grade 3

13

3

Grade 6

11

5

Male

9

7

Female

15

1

Gender

40


It seems reasonable to hypothesize that students who do not have an accurate understanding of the sizes of the Sun, Earth, and Moon or the distances between the celestial bodies would be more likely to cling to the misconception that the lunar phases are the result of the Earth’s shadow. Fanetti (2001) conducted a study to see whether there was a correlation between students’ beliefs about the cause of lunar phases and their knowledge of Earth-Moon system scale. Participants were college freshman and sophomores; 192 received traditional instruction while 69 received special instruction that involved a 3-dimentional modeling activity. Fanetti found no statistically significant correlation between the control and the experimental groups. However, her study may have been compromised by the special instruction. Fanetti reported, During special instruction, students were asked about the shadows on the Moon from the Earth. The students at the time said there were no shadows and seemed to understand the cause of the lunar phases slightly better. When it came time to do [sic] answer the surveys, these students showed an increase in the Shadow category. This was definitely unintentional and will need to be remedied for future instruction (p. 69). It is important to note that 54% of the participants in her study were fairly close in their estimation of the ratio of the Earth to the Moon (i.e., placed the ratio between 1 and 5 when the correct ratio is 3.7). However, approximately 96% of the total participants thought the Moon is within ten Earth diameters of the Earth before instruction and approximately 75% held to that same understanding after instruction (both traditional and special instruction).

41


At the same time Fanetti was conducting research on college students’ understanding of scale with respect to aerospace, Trumper (2001) documented his findings on the notions of scale with adolescents. Trumper reported that students of his study (n=448) underestimated distances between celestial objects while overestimating the diameter of the Earth. In fact, only 20% correctly estimated the distance between the Earth and Sun while only 8% accurately estimated the diameter of the Earth. Phase Geometry The Phase Geometry domain consists of an understanding that the relative positions of the Sun, Earth, and Moon have observable effects (i.e., lunar phases). Forty percent of the states value the knowledge that lies within the domain of Phase Geometry. Although angles and perspective are difficult for early adolescents to grasp, these concepts are crucial to seizing the big picture of the cause of lunar phases. Delaware’s standard for eighth-grade students is quite simply stated as, “Moon phases occur because the relative positions of Earth, Moon, and Sun change, thereby enabling us to see different amounts of the Moon's surface.” While Arkansas wants their seventh-grade students to model the positions of the Sun, Earth, and Moon for a variety of moon phases, Connecticut expects their eighth-grade students to explain how the relative positions of the three celestial bodies results in lunar phases. Michigan’s fifth-grade students are encouraged to “Explain moon phases as they relate to the position of the Moon in its orbit around the Earth, resulting in the amount of observable reflected light.” Mississippi emphasized the

42


importance of their students’ relating the lunar phases to the phase angle formed by the Sun, Earth, and Moon. The ability to predict the lunar phase when given the relative positions of the Sun, a planet, and its Moon requires more advanced understanding than merely knowing that lunar phases result from a change in the Moon’s position as it orbits the Earth. Missouri and New Mexico have reserved this learning objective for their high school students. However, it must be noted that both states have a well-structured lunar phase framework so that their students are exposed to these concepts for several years at increasing depths. Phase geometry misconception Early adolescents often carry inaccurate mental models of the Sun-Earth-Moon system into their science classrooms. Interviews were conducted individually with 32 students of low, average, and high ability to ascertain their understanding of selected aspects pertaining to the Sun-Earth-Moon system (Jones & Lynch, 1987). One-half of the participants were enrolled in the third grade while one-half were sixth-grade students. To identify any differences between genders that might exist, one-half of the students were male and the other one-half were female. A variety of geometric shapes (e.g., circular and semi-circular discs, spheres and hemispheres, cylindrical rods) in various sizes were made available to facilitate student explanations. Students demonstrated an understanding of one of the five distinct models shown in Figure 2.4.

43


Figure 2.4. Models of the Sun-Earth-Moon System (Jones & Lynch, 1987).

44


Although there was a statistically significant difference between the number of third-grade students and sixth-grade students who had an accurate mental model of the Sun-Earth-Moon system, 38% of the older group still clung to their misconceptions. No difference existed regarding gender. In their analysis of a project-based, space science curriculum study, Barnett and Morran (2002) identified misconceptions in the domain of Phase Geometry acquired by 14 elementary students attending a rural school. Participants of their study were advanced fifth-grade students enrolled in a science class that was to meet three days each week for a total of ten weeks. Students were first asked to document their beliefs about the Sun-Earth-Moon system in a journal and then encouraged to participate in a group brainstorming activity which resulted in the compilation of a list of their beliefs. Prior to instruction, participants were allowed to communicate their knowledge verbally, with manipulatives (spheres), or through sketches in individual interviews. These advanced students could recognize and distinguish between the different lunar phases but experienced difficulty explaining their cause. When questioned about the Moon’s relative position to the Earth for each phase, six students believed that lunar eclipses were caused by the Earth blocking the Sun’s rays and solar eclipses were the result of the Moon blocking the Sun’s rays. However, this knowledge did not help them resolve the relative positions of the Sun, Earth, and Moon necessary to create a full moon and a new moon phase. Three children deduced that the three

45


celestial bodies must line up to orchestrate a lunar eclipse but then had no explanation for why this phenomenon does not occur every month. An alternative explanation for lunar phases that surfaced was that “as the Earth spins we get to see different sides of the Moon” (Barnett & Morran, 2002, p. 867). Three students claimed that the Moon needed to be located to the side of the Earth (where the Moon and Sun form a 90° angle with the Earth observer) in order for the Earth observer to see a full moon. Phase Sequence The domain of Phase Sequence includes more specific understanding than simply knowing that the Moon’s appearance changes a little from day to day. Students who possess understanding in the domain of Phase Sequence are not only familiar with the various lunar phases but also know the order in which the lunar phases appear. They know that immediately after a new moon phase, the Moon waxes (i.e., appears to thicken) until its face is entirely illuminated and then wanes (appears to thin) until the next new moon phase, all in a predictable sequence. Eight states expect their students to order the lunar phases in the sequence in which they occur. Delaware and South Carolina anticipate that their first-grade students will draw and illustrate the Moon’s appearance, in the order in which it appears, over a period of time. Tennessee hopes their third-grade students are able to identify and order the basic lunar phases after a period of Moon observations. Missouri’s students are asked to note the pattern of lunar phases in the third grade and then sequence images of the phases in the fifth grade. In spite of the number of times the term predictable is used in the states’ space

46


science standards, only Texas expects their eighth grade students to actually predict the sequence of lunar phases. Phase sequence misconceptions Many adolescents lack an understanding that the Moon’s appearance occurs in a cyclical, predictable pattern. Before they began a nine-week investigation that involved documenting daily Moon observations, nine of the 48 fourth-grade students in a study conducted by Trundle, Atwood, and Christopher (2007) believed that the Moon’s phases would not appear in a predictable sequence. Sixteen of the 25 who attempted to sketch the lunar phases in sequence, failed to scientifically illustrate a waning sequence. Consecutive drawings showing a waning phenomenon were considered to adequately illustrate a waning sequence, even if all waning phases were not included in the sequence. Twenty-six of 31 children who attempted to sketch a lunar phase sequence unsuccessfully illustrated a waxing sequence. Cardinal Direction The Cardinal Direction domain encompasses knowledge of the specific direction the Moon appears to move across the sky due to the Earth’s rotation on its axis and the directions from which the Moon appears to rise and in which the Moon appears to set. Only four states, Delaware, Minnesota, Missouri, and New York, specifically address cardinal directions in their space science standards with the use of the terms east and west. Missouri expects their students to know that the Moon rises in an easterly direction, appears to move across the sky from east to west, and sets in a westerly direction. Eleven other states expect their students to describe (1) the change

47


in location, (2) the change in position, or (3) the movement of the Moon but are not specific about how. It is assumed that students would need to satisfy this standard with the knowledge of cardinal directions. Oregon wants their third-grade students to trace the movement of celestial objects across the sky. Knowledge of cardinal directions is universal in that one’s Earthly location need not be a consideration when using cardinal directions to describe movement. Phase Time Three states, Delaware, Missouri, and Mississippi, established learning expectations that most easily fit into the Phase Time domain. This domain includes an understanding of the rise and set times of lunar phases. While the Moon is in a waxing crescent phase for several days, its first and last quarter moon phases can only be viewed one day (and night) in one lunar cycle. In fact, a first quarter moon rises close to noon and sets close to midnight. A last quarter moon rises near midnight and sets near noon. The Phase Time domain also includes the knowledge that the length of time between these two phases is approximately 14 days. Additionally, a new moon phase falls midway between the waning and waxing crescent moon phases with this 14 day sequence beginning with a last quarter moon and ending with a first quarter moon. Similarly, a full moon phase is sandwiched between waxing and waning gibbous phases with this 14 day sequence beginning with a first quarter moon and ending with a last quarter moon. Finally, the Phase Time domain encompasses the knowledge that the moon rises approximately 50 minutes later each day. This knowledge can be gained through a simple analysis of Moon observation data recorded in a daily journal.

48


Missouri wants their students to realize that the Moon’s rise and set times are different for each phase. More specifically, they expect their students to discover that the Moon appears to rise later each day (about 50 minutes) as a result of the simultaneous motions of the Moon and Earth (i.e., the Moon’s counterclockwise revolution about the Earth, as seen from above the North Pole, and the Earth’s counterclockwise rotation on its axis, as seen from above the North Pole). Additionally, students in the state of Missouri are to learn that the Moon remains above the horizon for approximately 12 hours in a 24-hour period. Although there is some degree of variability, the average number of hours the Moon is above the horizon and below the horizon is about the same in each lunar cycle. No state expects students to know the number of days between the appearance of a first-quarter moon and a full moon. Yet, several desire that students conduct regular observations over an extended period of time and take note of emerging patterns. If any pattern is to be noted, then it follows that students will need to record their observations. Numerous patterns are embedded within students’ observational data. If students are expected to invest their time in documenting the change in the Moon’s appearance over an extended period of time, it seems a waste to not make good use of the documentation. The length of time between certain moon phases can be easily identified with a quick analysis of regular Moon observations. Global Lunar Perspective The domain of Global Lunar Perspective encompasses the specialized knowledge that the Moon’s appearance as seen from the northern hemisphere on any

49


specific calendar day differs from the Moon’s appearance as seen from a location in the southern hemisphere on the same calendar day. Also, the geometric configuration of the Sun, Earth, and Moon for the lunar phases as seen from above the South Pole is a reflection of the configuration as seen above the North Pole. Finally, the direction of the Moon’s orbit is as seen above the South Pole is opposite to its orbit as seen above the North Pole. Although none of the states expect students to acquire knowledge in this domain of Global Lunar Perspective, items that assess understanding in the domain of GLP are included in the CMPA. Students must be prepared to live and work in a global society. The Internet has opened doors for adolescents to gain knowledge from diverse perspectives within international learning communities. For example, students from Australia share their Moon observations with students in the U.S. as coparticipants in the MOON Project. The differences in the Moon’s appearance due to the diverse hemisphere perspectives are often brought to light in the essays written by the students. It was necessary to include items in the CMPA to measure understanding in this domain of Global Lunar Perspective in order to provide teachers with a tool that will meet their future needs.

Summary The lunar phase science domains identified by Lindell and Olsen (2002) were invaluable in organizing the space science standards established by the 50 United States. The misconceptions to which early adolescents cling have been unveiled by previous researchers and informed the construction of some of the item distractors of

50


the CMPA. Finally, diverse global lunar perspectives guided the construction of some items in the CMPA.

51


CHAPTER III METHODOLOGY Purpose and Objectives This research was conducted to develop and analyze an instrument, called the Comprehensive Moon Phases Assessment (CMPA), that measures what early adolescents know about lunar phase concepts identified through three sources – (1) (American) state science standards, (2) literature about misconceptions student harbor about the Moon, and (3) ideas that evolve from studying the Moon’s behavior from a global perspective. The CMPA will enable teachers, nationally and internationally, to (1) assess the mastery of lunar phase concepts identified from these three sources and (2) compare learning outcomes across schools that use the CMPA. The specific objectives were to: 1.

Identify domains of lunar phase concepts from state standards, misconception literature and a global study of the Moon,

2.

Construct and refine draft items to assess these identified domains,

3.

Select refined items for content coverage and to achieve a balance of item-difficulty,

4.

Administer the instrument,

5.

Establish content validity of items through expert review, and

6.

Determine construct validity through statistical analysis.

52


Research Design Groundwork for Construction of Draft Items In order to construct draft items for the CMPA, domains of understanding in lunar phase concepts needed to be identified. Thorough reviews of research that focused on lunar phase misconceptions and the state standards, conducted to write Chapter II, also helped identify many of the domains on which the instrument was built. Forward thinking led the research team to define the additional domain of Global Lunar Perspective. The first step toward identifying lunar phase domains of understanding was to develop a database of space science standards from all 50 states of the United States. Earth and space science standards are combined at the state level. Thus a database was first created with the Earth and space science standards from all of the 50 states’ education agency websites. Next, the space science standards targeting lunar phase understanding were identified and moved to a separate database. The method used to analyze the state standards and identify lunar phase domains was the constant comparative method (Glaser, 1965; Strauss & Corbin, 1994) based on grounded theory (Glaser & Strauss, 1967). The state lunar phase standards were read multiple times, during which concepts were labeled and relationships discovered. In the development of the LPCI, Lindell and Olsen (2002) identified eight science domains in which college-age students’ understanding of lunar phases is measured. Six of the LPCI’s eight domains emerged in the state standards for K-12 students. A seventh domain that Lindell and Olsen described as “Effect of lunar phase

53


with change in location on Earth” is encapsulated in a domain that will be referred to as Global Lunar Perspective. This domain includes the knowledge that the same lunar phase is visible from every earthly location on the same day even though the Moon’s appearance is distinctly different from northern to southern hemisphere perspectives. Students who completed the CMPA were participants in the More Observations of Nature (MOON) Project (http://www.worldmoonproject.org/attachments /047_Teacher%20Handbook.pdf; Smith, 2003; Smith, 2007; Trundle, Willmore, & Smith, 2006). The MOON Project is an inquiry-based project in which middle school science students share their Moon observations (documented in their Moon journals) with other children from around the world via the Internet. Because students from both the northern and southern hemispheres participate in the MOON Project, it follows that the CMPA needed to include items that assess this specific understanding. A complete list of lunar phase concept domains for which items were constructed is found in Table 3.1.

54


Table 3.1. Lunar Phase Concept Domains and Number and Percentage of States that Established Standards for Each Domain Number of States

Percent of States

Periodicity

42

84

Motion

41

82

Phase Appearance*

28

56

Terminology*

27

54

Phase Geometry

25

50


21

42

Scale*

15

30

Phase Sequence*

5

10

Phase Time

2

4

Cardinal Direction

2

4

Global Lunar Perspective**

0

0

*Domains not found in the LPCI **Domain added to assess global perspectives of lunar behavior

The construction of multiple-choice items was carefully approached so that the range of content knowledge to be measured was limited to the content within the eleven domains of lunar phase knowledge. The taxonomy of rules for writing multiple-choice items compiled by Haladyna and Downing (1989) directed the construction of the CMPA items (see Appendix D). After the author constructed draft

55


items, she collaborated with a doctoral student in educational psychology, an expert scholar in psychometrics, and an expert in lunar phases to complete the task of refining the draft items. The wording of each item was scrutinized and items were modified as needed to improve item clarity. Item Selection The refined multiple-choice draft items were circulated among the development team. Each member offered feedback on conceptual accuracy and clarity on the item stems, correct answers, and distractors. Several face-to-face meetings were held to expedite the refinement process. The final assessment consists of 40 multiplechoice items, each of which were constructed to measure conceptual understanding of one to three lunar phase domains. An additional open-ended item was constructed to provide students the opportunity to explain in their own words their thinking about the cause of lunar phases. However, the open-ended item was not analyzed in this research study. Six additional questions were included to gather demographical information about the student. The items included in the final assessment were selected to achieve a balance between questions requiring lower-order and higher-order thinking according to Bloom’s Taxonomy (1956). Appendix E shows the abstraction level of each CMPA item. Administration Once the assessment was completed, the data collection phase began. Copies of the final CMPA were printed and mailed to ten teachers who agreed to administer the assessment as a pre-test. Copies were mailed shortly before the commencement of

56


the MOON Project (to be administered as a pre-test) and their expeditious return was requested. Return envelopes, prepared with return postage, were included in the packet to hasten the return of the assessments in order to control for overexposure and possible bias. The same procedure for disseminating the assessment was followed for the administration of the post-tests. This occurred at the culmination of the MOON Project, ten weeks after the pre-test was given. Eight teachers had agreed to administer the CMPA as a post-test. There were 490 students who completed the CMPA as a pretest and 375 students completed the CMPA as a post-test. Validation Four middle school science teachers and four university faculty, who have knowledge about lunar phase concepts, provided feedback on the content validity of each of the 40 items of the CMPA. None of the expert reviewers were currently participating in the MOON Project. A Likert-type scale was used to evaluate the extent to which 1) each item measured the construct for which it was designed, 2) each item was appropriately worded for an early adolescent student, and 3) the feasibility of the item’s distractors. Experts were asked to use a 5-point Likert scale to score each item, with 5 indicating high quality and 1 indicating poor quality. Reviewers were encouraged to provide suggestions for improving every item that did not receive a score of 5. The format of the evaluation rubric completed by the expert reviewers is attached as Appendix F.

57


Statistical Analyses An exploratory factor analysis was conducted to provide further evidence of validity by identifying the dimensionality of the 40 quantitative items of the CMPA. The validity to be determined by an exploratory factor analysis is construct validity, not to be confused with content validity established by the expert reviewers. Construct validity pertains to a test’s ability to measure underlying factors or latent constructs. Factor analysis examines the answers students marked on the CMPA and seeks to identify patterns in their responses. Patterns within the student responses indicate relationships among the items. The extent to which the CMPA items are related is quantified in a correlation matrix. It is assumed that items that correlate highly do so because of an underlying independent factor. Factor analysis then groups the items that correlate and ranks them according to the amount of variance in each item is explained by the underlying factor. Once subsets of items cluster together, an analyst must then closely examine the items within the cluster to make an objective evaluation of the nature of the underlying factor. In this case, it is believed that a subset of CMPA items correlate because of students’ common understanding in a specialized domain of lunar phases. This analysis provided answers to research questions 1 and 2: 1.

Does the CMPA measure the following lunar phase constructs found in the state standards: a. Periodicity b. Motion

58


c. Lunar Phase Terminology d. Phase Appearance e. Cause of Lunar Phases f. Scale g. Phase Geometry h. Phase Sequence i. Cardinal Direction j. Phase Time 2.


A factor analysis was chosen over principal components analysis. Principle components analysis does not distinguish between unique and shared variance while factor analysis isolates only the shared variance. Shared variance is a more accurate indicator of the strength of relationships among items and a better statistic to direct the categorization of the items. Therefore, factor analysis was preferred over principle components analysis to minimize the possibility of inflating the estimates of variance (Costello & Osborne, 2005). Maximum likelihood was the method used to extract the factors. Fabrigar, Wegener, MacCallum and Strahan (1999) claimed that maximum likelihood “allows for the computation of a wide range of indexes of the goodness of fit of the model [and] permits statistical significance testing of factor loadings and correlations among factors and the computation of confidence intervals” (p. 277). Although a correlation

59


matrix could easily be produced with the sample data, the maximum likelihood method of extraction requires that a correlation matrix be generated with Maximum Likelihood (ML) estimators for the parameters. These estimators are derived by maximizing the likelihood function (or log-likelihood function). This set of parameter values holds the greatest probability of reproducing the data generated by the sample. In other words, the likelihood function is used to obtain the maximum likelihood estimation (MLE) estimate of the parameter values of the model (or desired probability distribution) that underlies the data. The factor axes were rotated in an oblique fashion within the factor space so the nature of the latent constructs could be more easily recognized. An oblique rotation was chosen as opposed to an orthogonal rotation to allow the factors to correlate. After these factors were identified, it was necessary to decide which factors could sufficiently emulate the bulk of the assessment. This was accomplished by determining the factors that adequately account for the covariation among the items (DeVellis, 2003). Factor pattern coefficients quantify the importance of each item to a specific factor in the presence of all the other items; while factor structure coefficients measure the strength (and direction of) the linear relationship between an item and a factor controlling for all the other items in the assessment. The two correlation coefficients (i.e., factor pattern coefficients and factor structure coefficients) provided useful information in identifying the constructs measured by the CMPA. Finally, the

60


level of abstraction of each item was identified according to Bloom’s Taxonomy and included in Appendix E.

61


CHAPTER IV RESULTS AND DISCUSSION In order to develop a valid instrument that will measure what early adolescents know about lunar phase concepts it was necessary that the Comprehensive Moon Phases Assessment (CMPA) undergo an expert review and be submitted to an exploratory factor analysis. The expert review and the exploratory factor analysis provided evidence to answer the research questions: 1.

Does the CMPA measure the following lunar phase constructs found in the state standards:

2.

a.

Periodicity

b.

Motion

c.

Lunar Phase Terminology

d.

Phase Appearance

e.


f.

Scale

g.

Phase Geometry

h.

Phase Sequence

i.

Cardinal Direction

j.

Phase Time

Does the CMPA measure the construct of Global Lunar Perspective

(GLP)?

62


Statistical Analyses Exploratory Factor Analysis A Full Information Maximum Likelihood (FIML) factor analysis was conducted to determine the dimensionality of the CMPA and thus address the research questions. The software program used to conduct all the analyses in this study was MPlus version 5.2 (Muthén & Muthén, 1998-2009). Maximum likelihood was used to estimate the model parameters of the “true” model (i.e., minimum information lost with respect to the “true” model). These estimated parameters were then used to select the model that closely fitted the “true” model. Each item of the CMPA was loaded onto 5-factor, 6-factor, and 7-factor models. The Akaike Information Criterion (AIC; Akaike, 1974), an information theoretic criterion, was the measure used to estimate fit of the models. The AIC values for each of the three models are found in Table 4.1 shown below.

63


Table 4.1. AIC Values for Models Model AIC 5-factor

40,247.598

6-factor

40,214.805

7-factor

40,203.049

The minimum information theoretic criterion estimation or MAICE is found with the 7-factor model. In other words, the MAICE indicates that the least amount of information is lost with the 7-factor model. Items were considered valid if their factor loadings > .32 in the 7-factor model (Tabachnick & Fidell, 2001). Factor loadings with a minimum value of .32 translated to a minimum of “10% overlapping variance with other items in that factor” (Costello & Osborne, 2005, p. 4). Table 4.2 shows the factor loadings of items that loaded > .32 in the 7-factor model. Item 45 was the only item that cross-loaded onto two different factors. Factor correlations and their standard errors are shown in Table 4.4 below.

64


Table 4.2. Factor Pattern Coefficients for Items of the CMPA Item

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Factor 6

Factor 7

7

0.574

0.019

0.011

0.140

0.020

-0.093

0.081

8

0.420

-0.040

0.029

0.069

0.059

-0.059

0.015

9

-0.023

0.945

0.037

0.004

-0.050

-0.017

-0.032

10

-0.028

0.826

0.014

-0.004

0.079

-0.073

0.045

11

0.111

0.386

-0.051

0.154

-0.049

0.119

0.080

12

-0.159

0.071

0.055

0.182

0.082

0.066

-0.014

13

0.213

0.056

-0.007

0.279

0.259

0.054

-0.012

14

0.045

0.049

0.951

-0.024

-0.056

0.008

0.006

15

-0.162

-0.023

0.916

0.024

0.029

0.005

-0.005

16

0.035

0.017

0.135

0.000

0.128

-0.021

-0.228

17

0.111

0.244

0.174

-0.004

0.051

0.009

0.143

18

-0.056

0.043

-0.103

-0.233

0.021

0.065

0.351

19

0.002

-0.126

0.016

-0.164

-0.020

-0.027

0.284

20

-0.077

-0.071

0.017

0.119

-0.027

-0.071

-0.143

21

0.111

-0.025

0.019

-0.050

-0.005

-0.011

-0.020

22

0.112

0.044

-0.035

0.039

-0.032

0.134

0.004

23

0.108

-0.073

0.026

0.266

-0.002

-0.010

-0.092

24

0.151

0.022

0.061

0.034

0.399

-0.008

-0.080

25

0.026

0.170

0.038

0.371

-0.093

0.180

-0.009

26

-0.080

0.121

0.054

0.182

0.081

0.032

-0.010

27

0.133

0.134

-0.020

0.173

0.113

-0.030

0.014

28

0.112

0.233

-0.021

0.511

0.082

0.139

-0.079

29

-0.150

0.019

0.001

0.042

0.008

0.066

-0.027

65


30

-0.140

-0.053

0.044

0.145

-0.025

0.116

0.044

31

0.082

0.014

0.038

-0.101

0.165

-0.001

-0.003

32

0.002

-0.022

-0.033

-0.137

0.754

0.082

-0.017

33

0.016

0.028

0.051

-0.020

-0.060

-0.169

0.446

34

0.008

-0.005

0.078

0.104

0.136

0.140

0.062

35

-0.274

0.053

-0.039

0.042

0.340

-0.276

0.121

36

-0.198

0.036

0.014

0.077

0.197

-0.103

0.010

37

0.165

0.084

-0.015

0.101

0.093

0.049

0.388

38

0.202

0.023

-0.044

0.263

-0.022

0.096

0.294

39

-0.018

0.110

-0.029

-0.003

0.137

0.730

0.010

40

-0.119

-0.097

0.023

0.273

-0.032

0.468

0.180

41

-0.051

0.110

-0.053

0.554

-0.004

-0.217

0.101

42

-0.189

0.006

-0.082

0.170

0.000

0.006

-0.100

43

0.021

0.063

0.092

-0.098

-0.004

0.132

0.116

44

-0.106

-0.069

0.028

-0.078

-0.103

-0.010

0.045

45

0.014

-0.037

0.081

0.369

0.272

-0.063

0.380

46

0.003

-0.093

0.001

0.257

0.205

-0.058

0.070

66


Table 4.3. Factor Structure Coefficients for Items of the CMPA Item

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Factor 6

Factor 7

7

0.614

0.176

0.070

0.268

0.172

-0.071

0.150

8

0.440

0.069

0.064

0.152

0.146

-0.050

0.054

9

0.128

0.926

0.112

0.288

0.176

0.045

0.140

10

0.129

0.844

0.083

0.284

0.277

-0.018

0.201

11

0.199

0.461

0.001

0.312

0.114

0.162

0.181

12

-0.092

0.129

0.058

0.205

0.124

0.091

0.027

13

0.323

0.247

0.034

0.417

0.391

0.093

0.078

14

0.117

0.116

0.957

0.032

-0.024

0.016

0.053

15

-0.080

0.040

0.902

0.037

0.020

0.014

0.026

16

0.056

0.022

0.132

0.014

0.127

-0.018

-0.208

17

0.186

0.316

0.211

0.146

0.142

0.028

0.206

18

-0.073

0.024

-0.098

-0.162

-0.025

0.043

0.313

19

-0.034

-0.131

0.010

-0.164

-0.079

-0.052

0.232

20

-0.080

-0.081

0.003

0.042

-0.033

-0.064

-0.142

21

0.096

-0.027

0.022

-0.041

-0.006

-0.017

-0.025

22

0.119

0.074

-0.020

0.080

0.001

0.142

0.024

23

0.143

0.017

0.038

0.247

0.071

0.015

-0.052

24

0.233

0.146

0.081

0.173

0.438

0.003

-0.032

25

0.115

0.286

0.071

0.425

0.061

0.231

0.085

26

-0.005

0.191

0.068

0.233

0.149

0.060

0.046

27

0.209

0.239

0.013

0.273

0.219

0.000

0.084

28

0.260

0.431

0.031

0.633

0.302

0.211

0.064

29

-0.138

0.009

-0.008

0.024

-0.003

0.071

-0.026

30

-0.117

-0.016

0.038

0.116

-0.016

0.128

0.050

67


31

0.098

0.037

0.044

-0.032

0.155

-0.008

0.000

32

0.105

0.117

-0.026

0.076

.0.709

0.073

0.002

33

0.041

0.085

0.071

0.035

-0.029

-0.167

0.446

34

0.064

0.090

0.090

0.173

0.173

0.153

0.092

35

-0.192

0.104

-0.045

0.091

0.319

-0.266

0.137

36

-0.140

0.072

0.008

0.097

0.192

-0.091

0.029

37

0.243

0.241

0.030

0.259

0.197

0.070

0.438

38

0.271

0.194

0.000

0.363

0.114

0.129

0.355

39

0.028

0.186

-0.012

0.150

0.167

.0739

0.040

40

-0.070

0.030

0.030

0.292

0.018

0.491

0.202

41

0.076

0.282

-0.019

0.569

0.174

-0.149

0.209

42

-0.169

0.004

-0.094

0.114

0.007

0.022

-0.087

43

0.028

0.071

0.101

-0.036

-0.002

0.127

0.117

44

-0.147

-0.128

0.009

-0.143

-0.158

-0.024

0.006

45

0.162

0.225

0.119

0.499

0.395

-0.019

0.457

46

0.079

0.051

0.013

0.292

0.260

-0.032

0.110

68


Table 4.4. Factor Correlations and Standard Errors Factor

1

2

3

4

5

6

2

0.169 (0.174)

3

0.082 0.083 (0.081) (0.057)

4

0.195 0.327 0.048 (0.217) (0.135) (0.069)

5

0.181 0.244 0.020 0.286 (0.179) (0.087) (0.069) (0.131)

6

0.065 0.009 0.111 0.011 0.007 (0.134) (0.136) (0.065) (0.106) (0.158)

7

0.186 0.045 0.171 0.063 0.006 0.071 (0.177) (0.180) (0.075) (0.155) (0.156) (0.109)

69


Factor determinacies indicate how well the factor scores are estimated. The factor determinacy is measured in terms of a correlation. The closer the factor scores are to the estimated scores, the closer the factor determinacy is to one. Table 4.5 shown below provides the determinacies of the factors that emerged in the exploratory factor analysis.

Table 4.5 Factor Determinacies Factor

Determinacy

1

0.789

2

0.952

3

0.972

4

0.850

5

0.834

6

0.825

7

0.765

The figure of the 7-factor measurement model is shown in Figure 4.1 below. The typical double-headed arrows used to indicate correlations between factors were omitted for visual parsimony.

70


Item 7 Factor 1 Item 8

Item 24

Factor 5

Item 9

Item 10

Factor 2

Item 32

Item 35

Item 11

Item 39 Factor 6

Item 14

Item 40 Factor 3

Item 15

Item 18

Item 25

Item 33 Factor 7

Item 28

Item 37 Factor 4

Item 41

Item 45

Item 45 Figure 4.1. The 7-factor Measurement Model. 71


Confirmatory Factor Analysis A confirmatory factor analysis was conducted with the salient factors and items identified with the exploratory factor analysis. The correlation matrix used in the confirmatory factor analysis is shown below in Table 4.6. The confirmatory factor analysis was also conducted with the MPlus version 5.2 software program (Muthén & Muthén, 1998-2009).

72


Table 4.6. Correlation Matrix, Means, and Standard Deviation for Items of the 7-factor Model Item

9

10

11

14

15

18

24

25

28

9

1.00

10

.808

1.00

11

.431

.405

1.00

14

.122

.091

.055

1.00

15

.028

.089

.009

.870

1.00

18

.017

.062

-.027

-.113

-.049

1.00

24

.160

.136

.091

.081

.023

-.113

1.00

25

.276

.239

.310

.075

.043

-.123

.171

1.00

28

.379

.320

.353

.065

-.022

-.121

.173

.325

1.00

32

.071

.164

-.037

-.041

-.029

-.022

.377

-.042

.140

33

.088

.105

.029

.093

.051

.052

.035

-.068

.055

35

.100

.115

.079

-.076

-.014

-.011

.054

-.002

.022

37

.209

.233

.300

.039

-.025

.150

.124

.224

.214

39

.208

.117

.144

.001

.012

.057

.025

.216

.262

40

.055

.016

.046

.033

.016

.047

.001

.178

.188

41

.289

.274

.221

-.023

-.024

-.034

.118

.233

.387

45

.228

.270

.262

.108

.095

.024

.213

.211

.263

73

M

S

.486

.500

.465

.499

.497

.500

.484

.500

.395

.489

.299

.458

.436

.496

.326

.469

.798

.402


Item

32

33

35

37

39

40

41

32

1.00

33

-.034

1.00

35

.169

.168

1.00

37

.137

.243

.073

1.00

39

.193

-.081

-.075

.085

1.00

40

-.005

.014

-.102

.098

.452

1.00

41

.060

.140

.132

.189

-.063

.131

1.00

45

.257

.176

.135

.302

.022

.220

.351

45

1.00

M

S

.460

.499

.218

.413

.170

.376

.437

.496

.365

.482

.338

.473

.421

.494

.431

.495

Parameter estimates for the items of the 7-factor model are shown below in Table 4.7. R2 is the portion of variation in the item that is explained by the underlying factor.

74


Table 4.7 Standardized Loadings (Standard Errors) for 7-Factor Model Factor

Item

Standardized

R2

9

0.900 (0.034)

0.810

10

0.867 (0.035)

0.752

11

0.577 (0.043)

0.310

14

1.442 (0.579)

*

15

0.603 (0.243)

0.364

28

0.670 (0.049)

0.449

41

0.520 (0.054)

0.270

25

0.498 (0.057)

0.248

45

0.410 (0.080)

0.301

24

0.653 (0.090)

0.426

32

0.556 (0.079)

0.309

35

0.227 (0.078)

0.052

39

0.688 (0.104)

0.473

40

0.658 (0.100)

0.433

37

0.837 (0.161)

0.700

33

0.298 (0.078)

0.089

45

0.235 (0.092)

0.301

18

0.062 (0.070)

0.004

2

3

4

5

6

7

* Undefined

75


Factor correlations and their standard errors for the 7-factor model were calculated in the confirmatory analysis. The results are shown below in Table 4.8.

Table 4.8. Factor Correlations and Standard Errors for 7-Factor Model Factor

2

3

4

5

6

3

0.084 (0.055)

4

0.639 0.063 (0.057) (0.058)

5

0.247 0.012 0.403 (0.068) (0.055) (0.084)

6

0.188 0.020 0.395 0.098 (0.065) (0.053) (0.081) (0.085)

7

0.352 0.048 0.401 0.292 0.132 (0.085) (0.055) (0.106) (0.094) (0.083)

76


Because the exploratory factor analysis suggested that the intercorrelations among items were best explained with a 7-factor model, a confirmatory factor analysis was conducted with factors two through seven. The first factor (believed to be terminology) was excluded from the confirmatory factor analysis to minimize the degrees of freedom. Three values were used to test the model’s goodness of fit: model chi-square, Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA). Keeping in mind that the 7-factor model is an estimation of the “true” model, model chi-square, CFI, and RMSEA measure how far apart the two models are. The model chi-square, a sample-based index, is sensitive to sample size and measures overall error. The RMSEA “estimates the amount of error of approximation per model degree of freedom and takes sample size into account” (Kline, 2005, p. 139). The result of the model chi-square test was significant, χ2 = (75, N = 865) = 162.179, p < 0.001. The CFI was 0.964. The RMSEA was 0.037. Hu and Bentler (1999) recommended CFI > .95 and RMSEA < .06 for a good-fit model.

77


Identification of Factors The exploratory factor analysis uncovered seven latent factors that undergirded the CMPA and showed 19 items loading onto the 7-factor model. The follow-up confirmatory factor analysis substantiated that the 7-factor model is a good fit model. Table 4.9 lists the 19 items of the CMPA and the factors on which they loaded in the exploratory factor analysis. Items in Table 4.9 are listed from left to right in decreasing order of the magnitude of their factor loadings in the confirmatory factor analysis. The domains for which items were developed and yet did not emerge in the exploratory factor analysis are: Periodicity, Cause of Lunar Phases, Scale, and Phase Geometry.

Table 4.9. CMPA Items that Loaded on Factors Factor

Item

Terminology

7

8

Phase Appearance

9

10

Motion/GLP

14

15

Full/New Moon

28

41

25

Cardinal Direction

24

32

35

Phase Time

39

40

Phase Sequence

37

33

78

11

45

45

18


A summary of how each factor was identified, the stability was determined, and the results compare to the a priori theory follows. The names of the domains that emerged from the factor analysis are italicized to distinguish them from the theoretical domains. •

Factor 1/Terminology: The factor underlying items seven and eight was interpreted as Terminology because these items ask students to define a motion and a pathway, respectively. Indeed, items seven and eight were designed to measure the construct of Lunar Phase Terminology. However, these items moderately loaded with factor pattern coefficients of 0.574 and 0.420, respectively. This factor is considered unstable in that only two items loaded moderately. In other words, these items might produce different results if administered to a different sample of middle school students. A factor on which five items load is more stable than a factor on which two items load.

•

Factor 2/Phase Appearance: Items 9, 10, and 11 ask students to connect physical descriptions of the Moon’s changing appearance with scientifically accepted terminology. The original intent of items 9, 10, and 11 was to measure terminology and it could be argued that this factor is not distinct from the first factor (Terminology). However, these items specifically focus on the terminology associated with the appearance of the lunar phases while the first factor thought to be Terminology is comprised of two items that measure one abstract path and one concrete motion. Only three items loaded onto this factor of Phase Appearance, yet the factor pattern coefficients of items nine and ten

79


were 0.945 and 0.826. Since two of the three items loaded strongly onto this factor, it is thought to be stable. •

Factor 3/Motion/GLP: Items 14 and 15 ask students to describe the direction of the Moon’s orbit about the Earth from two different perspectives (from above the North and South Poles). These two items were designed to measure the construct of Motion and Global Lunar Perspective. As mentioned in Chapter II, motion can only be described once perspective is identified. Although only two items loaded onto this factor, both loaded strongly with factor pattern coefficients of 0.951 and 0.916, respectively. This factor of Motion/GLP is considered to be stable.

•

Factor 4/Full/New Moon: Items 41 and 28 loaded moderately (0.554 and 0.511) onto the fourth factor. Both items ask students to make selections based on their knowledge of the full or new moon phases. Item 41 measures understanding of time between the two phases, while item 28 measures knowledge of the phase in which the Moon is not visible. The former was designed to measure the construct of Phase Time. The latter was intended to measure Phase Appearance. It was difficult to identify the construct that items 25 and 45, which loaded weakly onto the fourth factor, shared with items 41 and 28. Item 25 asks students to connect a waxing crescent phase (northern hemisphere perspective) with its image. Item 45 measures students’ knowledge of the length of one lunar cycle. This factor is considered to be unstable because two items loaded moderately and the other two loaded weakly.

80


Furthermore, the weakly loading items do not seem to measure the same understanding as the strongly loading items. •

Factor 5/Cardinal Direction: Only item 32 loaded strongly (0.754) with the other two items loading weakly. However, the top two loading items in this factor measure students’ understanding of cardinal direction. These same two items were indeed designed to measure the construct of Cardinal Direction. Item 35, with the smallest factor pattern coefficient, was designed to measure understanding of Phase Geometry. The stability of this factor is questionable.

•

Factor 6/Phase Time: Items 39 and 40 loaded onto this factor, identified as Phase Time. Both items ask students to select the correct length of time between two given lunar phases. These two items were designed to measure the construct of Phase Time. Item 39 loaded stronger (0.730) than item 40 (0.468). The stability of this factor is also questionable.

•

Factor 7/Phase Sequence: Items 33, 37, 45, and 18 all loaded weakly to moderately. Item 33 asks students to describe the Moon three days later than the day on which the given image can be seen. Item 37 asks students to identify the phase that can be seen two days after a full moon. Both measure knowledge of the phase that comes next in the lunar cycle. While item 33 was designed to measure Phase Appearance, it clearly measures sequential knowledge. Item 37 was designed to measure the construct Phase Appearance. Item 45 loaded weakly onto Factor 4 (0.369) as well as Factor 7 (0.380). Item 45 was designed to measure periodicity because it asks the length of a lunar

81


cycle. Item 18 ascertains a student’s knowledge of the amount of the Moon’s surface that is illuminated during a full moon phase. The latter two items do not seem to share a common construct with the former two items. For these reasons, this factor of Phase Sequence also seems to be unstable. In summary, two of the seven factors appear to be stable. The two factors are Phase Appearance and Motion/GLP. The five other factors identified in the 7-factor model may not be replicable when administered to a new sample of middle school students. Parameter estimates are included in the figure of the 7-factor measurement model shown in Figure 4.2. Factors are labeled in light of the factor identification explanation previously mentioned.

82


Item 9 0.900

Phase Appearance

0.867 0.557

1.442

Motion/GLP

0.603

Item 10 Item 11 Item 14 Item 15 Item 28

0.670

Item 25 Full/ New Moon

0.520 0.498

Item 41

0.410

Item 45 Item 24

0.653

Cardinal Direction

0.556 0.227

Item 32 Item 35 Item 39

0.688

Phase Time

0.658

Item 40 Item 37

0.837

Phase Sequence

0.298

0.235 0.062

Item 33 Item 45 Item 18

Figure 4.2. Parameter Estimates for the Items of the 7-Factor Model. 83


Expert Review The instrument was inspected by classroom teachers and university professors who are experts on the topic of lunar phases and adolescent cognition. These experts provided feedback on the content validity of each item of the 40-item version of the CMPA. In order to retain their independence, the reviewers were chosen so that none were concurrently participating in the MOON Project. Expert reviewers were selected based on experience in (1) middle school classrooms, (2) teaching the topic of lunar phases, (3) research and publishing concentrated on lunar phase concepts or misconceptions. A 5-point Likert-type scale was used to evaluate the extent to which 1) each item measured the construct for which it was designed, 2) each item was appropriately worded for an early adolescent student, and 3) the feasibility of each item’s distractors. An item which received a score of 5 was recognized as strong in measuring the construct for which it was designed. Conversely, an item which received a score of 1 was seen as weak in measuring the construct for which it was designed. Reviewers were encouraged to support their rankings with comments and provide suggestions for improvement for any and all items of their choosing. The format of the evaluation rubric completed by the expert reviewers is attached as Appendix F. Four classroom teachers and four university professors, who possessed the expected qualities, were recruited, and agreed to evaluate the 40-item CMPA. The expert reviewers had diverse experiences with the topic of lunar phases. The classroom teachers had a total of 133 years experience teaching adolescents and a total

84


of 53 years experience teaching lunar phases. Two of the teachers had published numerous sections in science textbooks and reference books. The four university professors who reviewed the 40-item CMPA had a total of 38 years experience teaching at the university level in addition to a total of 38 years spent teaching science at middle schools or high schools. Three of the four university professors had a minimum of four years experience conducting research and publishing their findings on lunar phase understanding at the middle level. One researcher had a strong background in astronomy, including a combined doctorate in astronomy and education, and numerous publications on the topic. Content Validity For each item, the reviewers were provided a concise definition of the construct for which the item was designed to measure. For example, the definition of the construct of motion given to the reviewers was “the real movements of rotation and revolution, as well as the movement of the Moon which appears to rise, set, and move across the sky in a westerly direction as a result of the Earth’s rotation.” The rankings supplied by the reviewers are evidence of content validity (see Table 4.10). Breaks were added within the table to enhance the readability of the data. The items were sequentially listed according to mean, then median, and lastly mode. Blank cells within the table indicate the criteria on which the reviewer chose to omit a ranking for that particular item. An omitted ranking was dropped when means were calculated. Reviewers’ comments and suggestions for improving content validity can be found in Appendix G.

85


Table 4.10. Rankings for Content Validity by Expert Reviewers Expert Reviewer

Item

Mean

Median

Mode

5

4.86

5.00

5

5

5

4.86

5.00

5

5

5

5

4

4.75

5.00

5

5

4

5

5

4

4.75

5.00

5

5

5

5

5

5

4

4.75

5.00

5

5

5

5

4

4

5

5

4.75

5.00

5

4

4

5

5

5

5

5

4

4.63

5.00

5

33

4

4

5

5

5

5

5

4

4.63

5.00

5

28

5

4

5

5

5

5

5

3

4.63

5.00

5

37

4

3

5

5

5

5

5

5

4.63

5.00

5

38

4

3

5

5

5

5

5

5

4.63

5.00

5

11

5

3

5

5

5

5

5

4

4.63

5.00

5

24

5

4

5

5

3

5

5

4

4.50

5.00

5

32

4

5

5

5

3

5

5

4

4.50

5.00

5

7

5

3

5

5

5

5

5

3

4.50

5.00

5

46

5

4

5

5

2

5

5

5

4.50

5.00

5

8

5

3

5

5

2

5

5

5

4.38

5.00

5

17

5

4

5

4

5

4

5

3

4.38

4.50

5

A

B

C

D

E

F

G

19

4

5

5

5

5

5

26

5

4

5

5

5

13

4

5

5

5

16

5

5

5

34

5

4

47

5

23

86

H


25

5

4

5

5

5

5

3

3

4.38

5.00

5

27

5

4

5

5

5

5

2

4

4.38

5.00

5

35

5

5

5

5

1

5

5

4

4.38

5.00

5

36

5

5

5

5

1

5

5

4

4.38

5.00

5

45

5

4

1

5

5

5

5

5

4.38

5.00

5

12

5

4

5

5

2

4

5

4

4.25

4.50

5

18

5

5

5

5

4

4

5

1

4.25

5.00

5

22

5

2

5

3

5

5

5

4

4.25

5.00

5

42

5

4

5

5

2

4

4

5

4.25

4.50

5

29

5

3

5

5

3

5

3

4.14

5.00

5

21

5

2

5

2

5

5

5

4

4.13

5.00

5

10

5

1

5

4

5

5

5

3

4.13

5.00

5

39

5

4

1

5

3

5

5

5

4.13

5.00

5

40

5

4

1

5

3

5

5

5

4.13

5.00

5

43

5

3

5

5

4

2

5

4

4.13

4.50

5

15

5

4

5

4

3

5

3

4

4.13

4.00

5

9

5

1

5

3

5

5

5

3

4.00

5.00

5

31

5

3

5

5

3

2

5

4

4.00

4.50

5

41

3

4

3

4

3

5

5

5

4.00

4.00

3

20

4

1

5

4

3

5

5

4

3.88

4.00

4

30

4

5

1

4

3

5

5

4

3.88

4.00

4

44

4

1

5

4

3

4

5

5

3.88

4.00

4

14

5

4

5

2

2

5

3

4

3.75

4.00

5

87


There was no item in which the reviewers were in total agreement on content validity. For the most part, the reviewers were in agreement on the ranking of 16 items. That is to say that no more than two of the eight reviewers’ rankings differed from the other six on 16 of the items. Only three items (19, 26, and 29) were unranked on content validity by a reviewer. The first eight items in the table received high rankings of no lower than a four. The next six items received high scores (four or five) by all except one reviewer who was neutral, scoring a three on content validity. Except for two, the items between 46 and 42 in Table 4.10 received only one low score (one or two) from a reviewer. Eleven (9, 10, 18, 20, 30, 35, 36, 39, 40, 44, and 45) items drew a single low ranking of one by a reviewer. However, on four of the 11 items (18, 35, 36, and 45), all other reviewers had rated the item with either a score of four or five. Table 4.11 shows the nine items that loaded on the 7-factor model and received high ratings by six of the eight reviewers. The expert reviewers’ high evaluations of nine of the 19 items that loaded onto the 7-factor model are evidence that the CMPA measures understanding in the domains of Cardinal Direction and Phase Sequence.

Table 4.11. CMPA Items Loaded on Factors and Received High Ratings by Six of Eight Experts Factor

Item

Phase Appearance

11

Full/New Moon

28

Cardinal Direction

32

24

35

Phase Sequence

33

37

45

88

18


Many of the reviewers, on one item or another, were influenced by the appropriateness of the wording when they ranked the content validity. In other words, a low ranking on content validity was most often followed by a similar low ranking on wording appropriateness. Reviewers’ comments and suggestions for improving the wording appropriate for middle school students can also be found in Appendix G. Table 4.12 reports the rankings each item received for the appropriateness of the wording for middle school students. As in the previous table, breaks were added within the table to enhance the readability of the data, items were sequentially listed according to mean, then median, and lastly mode. Blank cells indicate the criteria on which a reviewer chose to omit a ranking for that particular item. An omitted ranking was dropped when means were calculated.

89


Table 4.12. Rankings for the Wording Appropriateness by Expert Reviewers Expert Reviewer

Item

Mean

Median

Mode

5

4.88

5.00

5

5

4

4.88

5.00

5

4

5

5

4.88

5.00

5

5

5

3

4

4.63

5.00

5

5

5

5

5

3

4.63

5.00

5

3

5

5

5

5

5

4.63

5.00

5

5

5

5

5

5

3

4.57

5.00

5

4

5

5

5

3

5

5

4

4.50

5.00

5

17

5

4

5

4

5

5

5

3

4.50

5.00

5

23

3

5

5

5

5

4

5

4

4.50

5.00

5

24

5

4

5

5

3

5

5

4

4.50

5.00

5

37

4

3

4

5

5

5

5

5

4.50

5.00

5

39

5

4

3

5

4

5

5

5

4.50

5.00

5

40

5

4

3

5

4

5

5

5

4.50

5.00

5

46

5

4

5

5

2

5

5

5

4.50

5.00

5

12

5

5

5

5

2

5

5

4

4.50

5.00

5

38

4

3

3

5

5

5

5

5

4.38

5.00

5

7

5

4

3

5

5

5

5

3

4.38

5.00

5

11

5

2

5

5

5

4

5

4

4.38

5.00

5

14

5

4

5

4

3

5

5

4

4.38

4.50

5

34

4

4

5

3

5

5

5

4

4.38

4.50

5

18

5

5

5

4

5

5

1

4.29

5.00

5

27

2

3

5

5

5

5

4

4.25

5.00

5

A

B

C

D

E

F

G

H

8

5

4

5

5

5

5

5

32

5

5

5

5

5

5

47

5

5

5

5

5

13

5

5

5

5

28

5

4

5

45

5

4

19

4

16

5

90


31

5

3

5

5

5

2

5

26

5

2

2

5

5

5

5

25

5

2

5

5

5

5

3

42

5

5

5

5

2

3

20

3

1

5

5

5

35

5

5

4

5

21

5

1

5

22

5

1

41

3

15

4.25

5.00

5

4.14

5.00

5

3

4.13

5.00

5

3

5

4.13

5.00

5

5

5

4

4.13

5.00

5

1

4

5

4

4.13

4.50

5

2

5

5

5

4

4.00

5.00

5

5

2

5

5

5

4

4.00

5.00

5

4

3

4

4

4

5

5

4.00

4.00

4

5

4

2

3

3

5

5

4

3.88

4.00

5

33

4

4

1

3

5

5

5

4

3.88

4.00

4

36

3

5

2

5

1

4

5

4

3.63

4.00

5

30

4

4

1

3

3

5

5

4

3.63

4.00

4

29

4

1

4

5

3

5

3

3.57

4.00

4

10

5

1

3

3

5

2

5

3

3.38

3.00

5

44

4

2

2

3

1

4

5

5

3.25

3.50

4

43

2

3

1

5

4

2

5

4

3.25

3.50

2

9

5

1

3

3

4

2

5

3

3.25

3.00

3

91

4


As with content validity, there was no item in which the reviewers were in total agreement on word appropriateness. For the most part, the reviewers were in agreement on the ranking of 12 items. That is to say that no more than two of the eight reviewers’ rankings differed from the other six on 12 of the items. Only three items (19, 26, and 29) were unranked on word appropriateness by a reviewer. The first three items in Table 4.11 received high scores of 5 from all reviewers except one who scored the items with a four. Items listed in Table 4.11 between 13 and 40 received only one neutral score with all other rankings of four or five. Five items received only one low score (one or two) with all other rankings of four or five. Thirteen items were rated low with a single score of one by one of the reviewers. The readability of the CMPA was calculated with the Flesch-Kincaid (Flesch, 1948) readability test. The results indicate that the CMPA is readable at an eighth grade level. Ease of readability was scored at 63. Results of reading ease suggest that the CMPA is easily comprehended by students as young as 13 years of age. Evaluation of Item Distractors In addition to rating the items on content validity, each expert reviewer was asked to rate the distractors for each item. The same 5-point Likert scale that was used to evaluate content validity was also used to rate the feasibility of the item distractors. Reviewers’ comments, explanations for the low ratings, and suggestions for improving item distractors can be found in Appendix H.

92


CHAPTER V SUMMARY AND CONCLUSIONS Introduction The purpose of this chapter is to discuss the findings of the research that focused on the development and validation of the Comprehensive Moon Phases Assessment. The topics that will be addressed are: (a) summary of the research, (b) summary of the findings, (c) conclusions, (d) limitations of the research, and (e) recommendations for future research.

Summary of the Research The purpose of this research was to develop and validate a multiple-choice instrument that would assess middle school students’ understanding of lunar phase concepts based on state science standards, students’ misconceptions about lunar phases, and students’ understanding of lunar phases from a global perspective. Lindell and Olsen (2002) created an instrument to measure lunar phase understanding for use with college students. No single, multiple-choice test, designed to assess middle level students’ understanding of concepts related to lunar phases had been developed. Yet, the domains of lunar phases that Lindell and Olsen (2002) asserted that college-age students need to master were helpful in the development of a similar instrument for adolescents. Since no instrument existed, the state standards plus misconceptions held by children were used to develop the instrument. The eight science domains assessed by the LPCI served as a filter through which the state standards were examined. Standards were first categorized into some 93


but not all of the eight domains. The review of research that targeted adolescent misconceptions in lunar phase knowledge provoked a second review of the space science standards to tease out state standards which addressed concepts that would challenge these misconceptions. Consequently, domains emerged that are not assessed by the LPCI. A third review of the standards was conducted to confirm the findings of the first two reviews and provided a last opportunity to identify concepts that might have previously gone unnoticed. This constant comparative method (Glaser, 1965; Strauss & Corbin, 1994) led to the identification of ten distinct domains of lunar phase understanding. Although no state standards addressed this issue, an eleventh domain, global lunar perspectives, was added to these ten domains, since the CMPA would first be used with students who would be studying the Moon worldwide. A 40-item multiple-choice assessment, known as the Comprehensive Moon Phases Assessment (CMPA), was developed to measure these 11 domains of lunar phase understanding and then administered to middle school students participating in the MOON Project (a forum in which students from around the world can share their self-collected lunar phase observational data). Table 5.1 shows the items of the CMPA and the domains for which they were designed to measure.

94


Table 5.1. CMPA Items Constructed for Each Lunar Phase Domain Domain

Item

Periodicity

17

45

46

Motion

14

15

16

Lunar Phase Terminology

7

8

9

10

11

12

Phase Appearance

25

26

27

28

33

34


13

47

Scale

20

21

22

23

Phase Geometry

18

19

29

35

Phase Sequence

37

38

43

Cardinal Direction

24

30

31

32

Phase Time

30

31

39

40

41

42

43

44

Global Lunar Perspective

14

15

25

26

30

31

33

34

36

35

36

43

The data collected from the middle school students underwent a factor analysis to identify latent variables followed by a confirmatory factor analysis. The CMPA was evaluated on content validity by four middle school science teachers and four university professors who all had broad and deep understanding of lunar phase concepts.

95


Summary of the Findings Exploratory Factor Analysis An exploratory factor analysis uncovered seven latent factors that undergirded the CMPA and showed 19 of the 40 items loading onto the 7-factor model. Table 5.2 shows the 19 items of the CMPA and the factors on which they loaded in the exploratory factor analysis. Items in Table 5.2 are listed from left to right in decreasing order of their factor loading magnitude. The domains for which items were developed and yet did not emerge in the exploratory factor analysis are: Periodicity, Cause of Lunar Phases, Scale, and Phase Geometry.

Table 5.2. CMPA Items Loaded on Factors Factor

Item

Terminology

7

8

Phase Appearance

9

10

Motion/GLP

14

15

Full/New Moon

41

28

25

Cardinal Direction

32

24

35

Phase Time

39

40

Sequence

33

37

96

11

45

45

18


Confirmatory Factor Analysis A confirmatory factor analysis indicates that the 7-factor model is a model of good fit. The CFI was 0.964. The RMSEA was 0.037. Hu and Bentler (1999) recommended CFI > .95 and RMSEA < .06 for a good-fit model. In other words, this 7-factor model is thought to closely replicate the observed data. Expert Review While no item received strong scores of five (on a 5-point Likert scale) from all eight reviewers, seven reviewers rated 21 of the 40 CMPA items with scores of four or five. These 21 items were evaluated as valid. Of the 19 items that loaded onto the 7-factor model, two or more reviewers ranked ten items with a score of three or lower on content validity. On the other hand, 13 items that did not load onto the 7factor model received high rankings (i.e., scores of four or five) from seven of the eight experts on content validity.

Conclusions Seven factors that emerged from the exploratory factor analysis were confirmed by the confirmatory factor analysis. In fact the 7-factor model was found to be a good fit model. Four factors were easily identified as Terminology, Phase Appearance, Motion/GLP, and Cardinal Direction. The identification of the factors, Phase Time and Sequence, was a bit more challenging because a commonality could only be found between the top two loading items. The factor Full/New Moon was the most difficult factor to pin down.

97


The items that loaded onto the 7-factor model are not the same items the expert reviewers believed to be strong on content validity. The results of the exploratory factor analysis and the ratings provided by the expert reviewers support the validity of nine (23%) of the CMPA items. The diagram below illustrates the overlap of 53% percent of the CMPA items that were considered strong on content validity by the experts (i.e., received scores of four or five by seven of the expert reviewers) and 48% percent of the CMPA items that loaded onto the 7-factor model in the exploratory factor analysis (see Figure 5.1 below).

23%

22%

53% 48%

Figure 5.1. Diagram of Percentage of CMPA Items Identified as Valid by Expert Reviewers and Factor Analysis.

98


The specific questions that directed this research were: 1.

Does the CMPA measure the following lunar phase constructs found in the state standards: a. Periodicity b. Motion c. Lunar Phase Terminology d. Phase Appearance e. Cause of Lunar Phases f. Scale g. Phase Geometry h. Phase Sequence i. Cardinal Direction j. Phase Time

2.


The analyses and expert review indicate that the CMPA does not measure all of the lunar phase constructs found in the state standards and the construct of Global Lunar Perspective. All 11 of the originally proposed domains of lunar phase understanding were not identified or confirmed by the analyses. However, the factor analyses provided evidence of construct validity in the domains of Lunar Phase Terminology, Motion, Phase Appearance, Cardinal Direction, and Global Lunar Perspective. The experts provided evidence of content validity in the domains of

99


Cardinal Direction and Phase Sequence. The constant comparative method used to identify the domains within the state standards strengthens the framework of the instrument.

Limitations of the Research Students that completed the 40-item CMPA were scattered across North America and even across the globe with some attending middle schools in Australia. The ages of the students (enrolled in grades four through eight) varied from nine to 13. Some of their teachers had a great deal of knowledge about lunar phases while others had very little knowledge but were eager to learn. In fact, this was the first time one teacher with 90 students had addressed the subject in her classroom. Because the state standards vary from state to state, the instruction and lunar phase activities in which the students were engaged varied from classroom to classroom. For this reason, some students focused on lunar phases for numerous hours each week while others merely touched on the topic. It is likely that the variability in the numerous factors previously mentioned produced inconsistent data. Students who spent three hours per week engaged in classroom discourse about their moon observations, under the guidance of a teacher who is knowledgeable of lunar phases would be expected to answer the items of the CMPA quite differently than students who spent 30 minutes per week discussing their observations under the direction of a novice. Because these factors (e.g., time on task, experience of teacher, age levels of children) influenced the data and were not held constant within the sample, they are viewed as confounding factors that mask

100


correlations between items and factors and thus camouflage useful information about the CMPA. Ten of the 11 domains of lunar phase understanding were identified in the state standards through the constant comparative method (Glaser, 1965; Strauss & Corbin, 1994). The eleventh domain of Global Lunar Perspective was defined and included to assess understanding that is expected to emerge from a global study of the Moon. While these 11 domains of knowledge do exist in the minds of adults, they may or may not exist in the schema of adolescents. In fact, middle school students may categorize their lunar phase understanding in a smaller, very different set of domains than the 11 described in this research study. The limited number of experts who evaluated the CMPA affected results of the content validity. With only eight experts in the review panel, items that received two low scores were evaluated as invalid. A greater number of experts may have come closer to a consensus on rating the items. Furthermore, decisions regarding item revision could be more easily made with one or two dissenting scores in an expert panel of 15 rather than in an expert panel of eight.

Recommendations for Future Research The CMPA was developed to measure lunar phase understanding held by adolescents in the lower 48 states of the U.S. The goal was to assess their comprehensive lunar phase knowledge yet, limit the number of items so that middle school students could thoughtfully complete the assessment in 45 minutes or less. A strong instrument would include four or five items per domain. While the most

101


effective multiple-choice test includes items that measure only one construct, the 11 domains of understanding would require the CMPA to be comprised of a minimum of 44 items. However, the various lunar phase domains found in the state standards are linked. Indeed, the factors that emerged were correlated. Therefore, a 30- to 35-item multiple-choice instrument could be constructed that will measure the broad range of lunar phase understanding the state education agencies expect their middle school students to know. It is suggested that the results of this study be used to revise the CMPA to more effectively measure the various aspects of lunar phase understanding the states expect their adolescents to know, the misconceptions commonly held by middle school students, and knowledge constructed from a global study of the Moon’s phases. Items that adequately loaded onto the factors Terminology, Phase Appearance, Motion/GLP, and Cardinal Direction could be retained in the revised version of the CMPA. Additional items need to be developed to assess understanding in the domains that were confirmed by the analyses and expert review in this research study. Other items should be constructed to address the domains that emerged from the state standards yet were not confirmed by this research study. A revised version should then be administered to a new sample of middle school students who have completed a similar comprehensive study of lunar phases with an experienced teacher. Expert feedback provided in this research study should be considered in the process of item revision and item construction.

102


REFERENCES Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York: Oxford University Press. Barnett, M., & Morran, J. (2002). Addressing children’s alternative frameworks of the Moon’s phases and eclipses. International Journal of Science Education, 24(8), 859-879. Baxter, J. (1989). Children’s understanding of familiar astronomical events. International Journal of Science Education, 11, 502-513. Bisard, W., Aron, R., Francek, M., & Burton, B. (1994). Assessing selected physical science and Earth science misconceptions of middle school through university pre-service teachers. Journal of College Science Teaching, 24, 38-42. Bloom, B., Engelhart, M., Furst, E., Hill, W., & Krathwohl, D. (Eds). (1956). Taxonomy of educational objectives. The classification of educational goals, Handbook I: Cognitive domain. New York, NY: David McKay Company, Inc. Boston, C. (2002). The concept of formative assessment. Practical Assessment, Research & Evaluation, 8(9). Retrieved February 27, 2009 from http://PAREonline.net/getvn.asp?v=8&n=9. Callison, P., & Wright, E. (1993). The effect of teaching strategies using models on pre-service elementary teachers' conceptions about Earth-Sun-Moon

103


relationships. Paper presented at the National Association for Research in Science Teaching Annual Meeting, Atlanta, GA. Cambridge, B. (1996). The paradigm shifts: Examining quality of teaching through assessment of student learning. Innovative Higher Education, 20(4), 287-297. Cameron, J., & Pierce, D. (1994). Reinforcement, reward, and intrinsic motivation: A meta-analysis. Review of Educational Research, 64(3), 245-281. Costello, A., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7). Retrieved April 10, 2009 from http://pareonline.net/getvn.asp?v=10&n=7. Crooks, T. (2001). The validity of formative assessments. Paper presented at the British Educational Research Association Annual Conference, University of Leeds, September 13-15. Dai, M. & Capie, W. (1990). Misconceptions about the moon held by preservice teachers in Taiwan. Paper presented at the National Association for Research in Science Teaching annual meeting, Atlanta, GA. DeVellis, R. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage Publications, Incorporated. Di Noia, J., Contento, I., & Schinke, S. (2007). Criterion validity of the healthy eating self-monitoring tool (HEST) for black adolescents. Journal of the American Diabetic Association. 107(2), 321-324.

104


Driver, R., Guesne, E., & Tiberghien, A. (1985). Children’s ideas in science. Philadelphia, PA: Open University Press. Erickson, E. L. (1979). Children’s conceptions of heat and temperature. Science Education, 63, 221-230. Fanetti, T. (2001). The relationships of scale concepts on college age students’ misconceptions about the cause of lunar phases. Unpublished master’s thesis, Iowa State University, Ames, Iowa. Flesch, F. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233. Glaser, B. (1965). The constant comparative method of qualitative analysis. Social Problems, 12, 436-445. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing Company. Gunning, R. (1968). The technique of clear writing. New York, NY: McGraw-Hill. Haladyna, T., & Downing, S. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37-50. Haupt, G. (1950). First grade concepts of the Moon. Science Education, 34(4), 224234. Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

105


Jones, B., & Lynch, P. (1987). Children’s conceptions of the earth, sun and moon. International Journal of Science Education, 9(1), 43-53. Kirschenbaum, H., Simon, S., & Napier, R. (1971). Wad-ja-get? The grading game in American education. New York, NY: Hart Publishing. Kiss, C., & Nikolov, M. (2005). Developing, piloting, and validating an instrument to measure young learners’ aptitude. Language Learning, 55(1), 99-150. Kluger, A., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284. Lawrenz, F. (1986). Misconceptions of physical science concepts among elementary school teachers. School Science and Mathematics, 86, 654-660. Lindell, R., & Olsen, J. (2002). Developing the Lunar Phases Concept Inventory. In S. Franklin, J. Marx, and K. Cummings (Eds.), Physics Education Research Conference Proceeding. New York, NY: PERC Publishing. Marshall, J. (11). Construct validity of multiple-choice and performance-based assessments of basic science process skills: A multitrait-multimethod analysis. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Lake Geneva, WI. McMillan, J. (2004). Classroom assessment: Principles and practice for effective instruction (3rd ed.). Boston, MA: Allyn and Bacon. Muthén, L., & Muthén, B. (1998-2009). Mplus user’s guide (5th ed.). Los Angeles, CA: Muthén & Muthén.

106


National Research Council (NRC). (1996). National science education standards. Washington, DC: National Academy Press. National Science Teachers Association (2008). Position statement. Retrieved March 15, 2009 from http://www.nsta.org/about/positions/aerospace.aspx Piazza, C., & Siebert, C. (2008). Development and validation of a writing dispositions scale for elementary and middle school students. The Journal of Educational Research, 101(5), 275-285. Popham, W. (2008). Classroom assessment: What teachers need to know. Boston, MA: Pearson Education, Inc. Plummer, J., & Krajcik, J. (2008). A learning progression for celestial motion. Paper presented at the Annual Meeting for the National Association for Research in Science Teaching, Baltimore, Maryland. Sadler, P. (1987). Misconceptions in astronomy. In J. D. Novak (Ed.), Proceedings of the Second International Seminar: Misconceptions and Educational Strategies in Science and Mathematics (Vol. 3, pp. 422-425). Ithaca, NY: Cornell University. Schoon, K. (1992). Students’ alternative conceptions of earth and space. Journal of Geological Education, 40, 209-214. Selley, N. (1996). Children’s ideas on light and vision. International Journal of Science Education, 18(6), 713-723. Skamp, K (Ed). (2004). Teaching primary science constructively. Southbank, Victoria: Thomson Learning Australia.

107


Smith, W. (2007). Creating global educational systems: The international virtual science, technology, engineering and mathematics (I.V. STEM) classroom. The Hoosier Science Teacher, 32(4), 103-108. Smith, W. (2003). Meeting the moon from a global perspective. Science Scope, 26(8), 24-28. Smith, W. (2009). Teacher handbook: January – April 2009. Retrieved April 4, 2009, from Moon Project website: http://www.worldmoonproject.org/attachments /047_Teacher%20Handbook.pdf Stahly, L., Krockover, G., & Shepardson, D. (1999). Third grade students’ ideas about the lunar phases. Journal of Research in Science Teaching, 36(2), 159-177. Stark, J., & Thomas, A. (1994). Assessment program evaluation. Needham Heights, MA: Simon & Schuster Custom Publishing. Stead, B., & Osborne, R. (1980). Exploring science students’ concepts of light. Australian Science Teachers Journal, 26(2), 84-90. Straatemeier, M., van der Maas, H., & Jansen, B. (2008). Children’s knowledge of the earth: A new methodological and statistical approach. Journal of Experimental Child Psychology, 100, 276-296. Strauss, A., & Corbin, J. (1994). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 273285). London: Sage Publications. Tabachnick, B., & Fidell, L. (2001). Using multivariate statistics. Boston, MA: Allyn and Bacon.

108


Trumper, R. (2001). A cross-age study of junior high school students’ conceptions of basic astronomy concepts. International Journal of Science Education, 23(11), 1111-1123. Trundle, K., Atwood, R., & Christopher, J. (2002). Preservice elementary teachers' conceptions of moon phases before and after instruction. Journal of Research in Science Teaching, 39(7), 633-658. Trundle, K., Atwood, R., & Christopher, J. (2007). Fourth-grade elementary students’ conceptions of standards-based lunar concepts. International Journal of Science Education, 29(5), 595-616. Trundle, K., Willmore, S., & Smith, W. (2006). The MOON project. Science and Children, 43(6), 52-55. van de Berg, E. & Sandura, L. (1990). Student ideas on the velocity of light. Australian Science Teachers Journal, 36(2), 72-75. Vosniadou, S. (1991). Children’s naïve models and the processing of expository text. In M. Carretero, M. Pope, R. Simons, & T. Pozό (Eds.), Learning and instruction: European research in an international context (Vol. 3, pp. 325336). Oxford: Pergamon. Vosniadou, S., & Brewer, W. (1994). Mental models of the day/night cycle. Cognitive Science, 18, 123-184. Wiggins, G. (1996). Embracing accountability. New Schools, New Communities, 12(2), 4-10.

109


Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass, Inc. World MOON Project. (2009). http://worldmoonproject.org/ zenith. (n.d.). The American Heritage® Dictionary of the English Language, Fourth Edition. Retrieved May 06, 2009, from Dictionary.com website: http://dictionary1.classic.reference.com/browse/zenith

110


APPENDIX A Comprehensive Moon Phases Assessment

111


112


113


114


115


116


117


118


119


APPENDIX B States Which Established Standards for Each Lunar Phase Concept Domain Assessed by the CMPA Lunar Phases Concept Domain Terminology Scale Phase Time Phase Sequence Phase Geometry Phase Appearance Periodicity Motion Cardinal Direction Global Lunar Perspective Cause of Lunar Phases

Lunar Phases Concept Domain Terminology Scale Phase Time Phase Sequence Phase Geometry Phase Appearance Periodicity Motion Cardinal Direction Global Lunar Perspective Cause of Lunar Phases

State AL AK AZ AR CA CO CT DE FL GA ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

●

●

State HI ID IL IN IA KS KY LA ME MD ● ● ● ● ● ● ● ● ●

● ● ●

●

●

● ●

● ● ●

●

● ●

●

120

●

● ● ● ●

● ● ●

●

● ● ●

● ● ● ●

●



State MA MI MN MS MO MT NE NV NH NJ ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●

● ● ●

● ●

● ● ●

●

●

●

●

● ●

● ●

● ●

● ●

● ●

●

●

●

●

Lunar Phases State Concept NM NY NC ND OH OK OR PA RI SC Domain Terminology ● ● ● ● ● ● ● ● Scale ● ● ● ● ● Phase Time ● ● ● Phase Sequence Phase Geometry ● ● Phase Appearance ● ● ● ● ● ● ● Periodicity ● ● ● ● ● ● ● ● ● ● Motion ● ● ● ● ● ● ● ● ● Cardinal ● Direction Global Lunar Perspective Cause of Lunar ● ● ● ● ● ● ● Phases

121



State SD TN TX UT VT VA WA WV WI WY ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

● ●

●

● ●

● ●

●

● ●

● ●

● ●

● ●

●

●

●

●

122

● ●

●

●

●

● ●

● ●

● ●

● ●

●

●

●


APPENDIX C State Standards Addressing Lunar Phases

STATE

DOMAIN

GRADE STANDARD

Alabama Phase Geometry Periodicity Terminology Motion

3

Describe the position of Earth, the moon, and the sun during the course of a day or month. Define axis, rotate, revolve, and orbit.

Alabama Phase Appearance Motion Terminology Phase Geometry

4

Describe the appearance and movement of Earth and its moon. Describe the positions of Earth, the moon, and the sun during the course of a day or month. Explain the relationship of Earth's moon and sun during the full moon. Identify the waxing and waning of the moon in the night sky. Identify lunar and solar eclipses.

Alabama Phase Appearance Periodicity Phase Geometry Terminology

6

Identify the moon's phases. Illustrate the phases of the moon during the course of a lunar cycle. Describe the positions of Earth, the moon, and the sun during the course of a day or lunar cycle. Describe lunar and solar eclipses.

Alabama Periodicity Motion Terminology

9-12

Explain the length of a day and of a year in terms of the motion of Earth. Distinguish between rotation and revolution.

Source: Retrieved May 15, 2008 from http://www.alsde.edu/html/ doc_download.asp?id=7342§ion=65

123


STATE

DOMAIN

GRADE STANDARD

Alaska

Periodicity Motion Phase Appearance

5

The student demonstrates an understanding of cycles influenced by energy from the sun and by Earth's position and motion in our solar system by observing a model of the Earth and moon determine the apparent shape (phases) of the moon over time. The student demonstrates an understanding of the theories regarding the origin and evolution of the universe by recognizing that the Earth is in regular and predictable motion explains the length of a day and a year.

Alaska

Periodicity Motion Terminology Scale

8

The student demonstrates an understanding of cycles influenced by energy from the sun and by Earth's position and motion in our solar system by recognizing the relationship between the seasons and Earth's tilt relative to the sun and describing the day/night cycle as caused by the rotation of the Earth every 24 hours. The student demonstrates an understanding of the theories regarding the origin and evolution of the universe by creating models of the solar system illustrating size, location/position, composition, moons/rings, and conditions.

Source: Retrieved May 17, 2008 from http://www.eed.state.ak.us/standards/ pdf/standards.pdf

124


STATE

DOMAIN

GRADE STANDARD

Arizona

Motion

1

Compare celestial objects (e.g., Sun, Moon, stars) and transient objects in the sky (e.g., clouds, birds, airplanes, contrails). Describe observable changes that occur in the sky, (e.g., clouds forming and moving, the position of the Moon).

Arizona

Periodicity Phase Appearance Terminology Motion Phase Geometry

5

Describe how the Moon’s appearance changes during a four-week lunar cycle. Describe how Earth’s rotation results in day and night at any particular location. Distinguish between revolution and rotation. Describe the change in position and motion of the following objects in the sky over time: • real motion – Moon, planets • apparent motion (due to the motion of the Earth) – Sun, Moon, stars.

Arizona

Motion Phase Geometry Scale Terminology

9-12

Describe the characteristics, location, and motions of the various kinds of objects in our solar system, including the Sun, planets, satellites, comets, meteors, and asteroids. Explain the phases of the Moon, eclipses (lunar and solar), and the interaction of the Sun, Moon, and Earth (tidal effect).

Source: Retrieved May 15, 2008 from http://www.ade.state.az.us/standards/science/ articulated.asp

125


STATE

DOMAIN

GRADE STANDARD

Arkansas Phase Appearance Motion Terminology

2

Illustrate four moon phases: full, half, crescent, and new. Model the movement of Earth and its moon. Contrast the visibility of the sun and moon.

Arkansas Motion Periodicity Terminology

3

Demonstrate the orbit of Earth and its moon around the sun. Relate Earth’s rotation to the day/night cycle.

Arkansas Periodicity Motion Phase Geometry Terminology

7

Identify and model the causes of night and day. Model moon phases demonstrating the position of Earth, moon, and sun. Compare and contrast solar eclipse and lunar eclipse.

Source: Retrieved May 15, 2008 from http://arkansased.org/teachers/ frameworks2.html#science

126


STATE

DOMAIN

GRADE STANDARD

California Periodicity Motion Phase Appearance

3

Objects in the sky move in regular and predictable patterns. Students know the way in which the Moon's appearance changes during the four-week lunar cycle. Students know that Earth is one of several planets that orbit the Sun and that the Moon orbits Earth.

California Terminology

8

Students know that stars are the source of light for all bright objects in outer space and that the Moon and planets shine by reflected sunlight, not by their own light.

Source: Retrieved May 15, 2008 from http://www.cde.ca.gov/be/st/ss/

127


STATE

DOMAIN

GRADE STANDARD

Colorado Terminology Motion Periodicity

3-5

The rotation of the Earth on its axis, in relation to the Sun, produces the day-andnight cycle and the orbit of the Earth around the Sun completes one year.

Colorado Motion Phase Geometry Phase Appearance Terminology

6-8

Relative motion, axes tilt and positions of the Sun, Earth, and Moon have observable effects (for example: seasons, eclipses, moon phases).

Colorado Scale

9-12

The scales of size and separation of components of the solar system are complex.

Source: Retrieved May 15, 2008 from http://www.cde.state.co.us/coloradoscience/ Science_Standards_July_2007.pdf

128


STATE

DOMAIN

GRADE STANDARD

Connecticut Periodicity Motion Terminology Phase Appearance Phase Geometry Cause of Phases

5

Most objects in the solar system are in a regular and predictable motion. Explain the cause of day and night based on the rotation of Earth on its axis. Describe the monthly changes in the appearance of the moon, based on the moon’s orbit around the Earth.

Connecticut Periodicity Motion Terminology Phase Appearance Phase Geometry Cause of Phases

8

The motion of the Earth and moon relative to the sun causes daily, monthly and yearly cycles on Earth. Explain how the regular motion and relative position of the sun, Earth and moon affect the seasons, phases of the moon and eclipses. Earth and moon affect the seasons, phases of the moon and eclipses.

Source: Retrieved May 15, 2008 from http://www.sde.ct.gov/sde/cwp/ view.asp?a=2618&q=320890

129


STATE

DOMAIN

GRADE STANDARD

Delaware Motion Periodicity Phase Appearance

K

From Earth many objects may be seen in the sky including the Sun, the Moon, stars, and manmade objects. The Sun and Moon appear to move slowly across the sky. The pattern of day and night repeats every 24 hours. The Sun can only be seen in the daytime. The Moon can be observed sometimes at night and sometimes during the day. The appearance of the Moon changes in a cycle that takes about a month.

Delaware Motion Periodicity Phase Appearance Phase Sequence

1

From Earth many objects may be seen in the sky including the Sun, the Moon, stars, and manmade objects. The Sun and Moon appear to move slowly across the sky. The pattern of day and night repeats every 24 hours. The Sun can only be seen in the daytime. The Moon can be observed sometimes at night and sometimes during the day. The appearance of the Moon changes in a cycle that takes about a month. Observe the Moon in the day sky over several months. Draw a sequence of pictures that show the repeating cyclic pattern of the Moon. Use simple models to demonstrate how Earth's rotation causes day and night.

130


STATE

DOMAIN

GRADE STANDARD

Delaware Motion Periodicity Phase Appearance

2-3

From Earth many objects may be seen in the sky including the Sun, the Moon, stars, and manmade objects. The Sun and Moon appear to move slowly across the sky. The pattern of day and night repeats every 24 hours. The Sun can only be seen in the daytime. The Moon can be observed sometimes at night and sometimes during the day. The appearance of the Moon changes in a cycle that takes about a month.

Delaware Phase Geometry Scale Cause of Phases

6-7

Moon phases occur because the relative positions of Earth, Moon, and Sun change, thereby enabling us to see different amounts of the Moon's surface. The Moon is a natural satellite of Earth and is different than the Earth in size, atmosphere, gravity, and surface features.

131


STATE

DOMAIN

Delaware Motion Periodicity Terminology Phase Appearance Scale Cardinal Direction

GRADE STANDARD 4

The apparent path of the Sun, as seen from Earth, is from east to west. Over the course of a day, half of the Earth is always illuminated by the Sun causing day, and the half not illuminated by the Sun experiences nighttime. The cycle from day to night is caused by the Earth's rotation. Earth undergoes one complete rotation about every 24 hours. The Moon orbits the Earth. The appearance of the Moon changes as it moves through its orbit. These changes are called phases. The Sun is much larger than the Moon. Although the Moon is closer to Earth than the Sun, the two appear to be the same size when viewed from Earth. This is because objects appear smaller as the distance from the viewer increases. Use models to describe how the Earth's rotation on its axis causes one half of the Earth to always be illuminated by the Sun (day) and one half to not be illuminated by the Sun (night). Apply this model of the rotating Earth to explain why the Sun appears to move across the sky each day from east to west. Using newspapers, the internet, and actual sky observations when possible, chart the appearance of the Moon in the night sky over the course of at least two months. Identify the basic pattern of the Moon's appearance. Classify the Moon's appearance by using the terms new, first quarter, full, last (third) quarter. Observe the size of the Sun and Moon in the sky. Use models to illustrate the approximate size and distance relationship between the Sun and Moon. Explain why the Sun and Moon appear to be similar in size when observed in the sky.

132


STATE

DOMAIN

Delaware Phase Geometry Scale Cause of Phases

Delaware Phase Geometry Scale Cause of Phases Phase Appearance Phase Time Terminology

GRADE STANDARD 6-7

Moon phases occur because the relative positions of Earth, Moon, and Sun change, thereby enabling us to see different amounts of the Moon's surface. The Moon is a natural satellite of Earth and is different than the Earth in size, atmosphere, gravity, and surface features.

8

Moon phases occur because the relative positions of Earth, Moon, and Sun change, thereby enabling us to see different amounts of the Moon's surface. The Moon is a natural satellite of Earth and is different than the Earth in size, atmosphere, gravity, and surface features. Analyze data on sunrise and sunset times (in terms of length of daylight) and describe patterns. Explain the reason for the patterns by using models or computer simulations of the Earth and Sun. Using internet, newspaper, and actual observations of night sky for at least two months, collect data on the Moon's appearance, and moonrise and moonset times. Analyze the data to describe the observable patterns (phases). Explain why the Moon's appearance changes in a repeating cyclical pattern. Use models to describe how the relative positions of the Sun, Moon, and Earth account for Moon phases, eclipses, and tides.

Source: Retrieved May 15, 2008 from http://www.doe.k12.de.us/ddoe/files/pdf/ science_Standard4.pdf

133


STATE

DOMAIN

GRADE STANDARD

Florida

Periodicity

K

Recognize the repeating pattern of day and night. Observe that sometimes the Moon can be seen at night and sometimes during the day.

Florida

Periodicity Phase Appearance Terminology Motion

4

Describe the changes in the observable shape of the moon over the course of about a month. Recognize that Earth revolves around the Sun in a year and rotates on its axis in a 24hour day. Relate that the rotation of Earth (day and night) and apparent movements of the Sun, Moon, and stars are connected.

Florida

Cause of Phases Phase Geometry Terminology

8

Explain the impact of objects in space on each other including: the Moon on the Earth, including phases, tides, and eclipses, and the relative position of each body.

Source: Retrieved May 15, 2008 from http://www.floridastandards.org/Standards/ FLStandardSearch.aspx

134


STATE

DOMAIN

GRADE STANDARD

Georgia

Periodicity

K

Students will describe time patterns (such as day to night and night to day) and objects (such as sun, moon, stars) in the day and night sky.

Georgia

Periodicity Phase Appearance

2

Students will investigate the position of sun and moon to show patterns throughout the year. Use observations and charts to record the shape of the moon for a period of time.

Georgia

Phase Geometry Terminology Motion Phase Sequence Periodicity Phase Appearance

4

Students will model the position and motion of the earth in the solar system and will explain the role of relative position and motion in determining sequence of the phases of the moon. Explain the day/night cycle of the earth using a model. Explain the sequence of the phases of the moon.

Georgia

Motion Phase Geometry Phase Appearance Terminology

6

Students will explain the motion of objects in the day/night sky in terms of relative position. Students will understand the effects of the relative positions of the earth, moon and sun. Demonstrate the phases of the moon by showing the alignment of the earth, moon, and sun. Explain the alignment of the earth, moon, and sun during solar and lunar eclipses.

Source: Retrieved May 15, 2008 from http://gadoe.georgiastandards.org/science.aspx

135


STATE

DOMAIN

GRADE STANDARD

Hawaii

Motion

K-3

Observe and describe the properties, locations, and movements of celestial objects in the sky. Describe the movements of the sun, moon and stars throughout the day.

Hawaii

Cause of Phases Terminology Periodicity

4-5

Explain the phases of the moon and eclipses. Describe the Earth’s daily rotation and annual revolution.

Hawaii

Periodicity

6-8

Give examples of objects in the solar system that are in regular and predictable motion.

Source: Retrieved May 15, 2008 from http://www.hcps.k12.hi.us/

136


STATE

DOMAIN

Idaho

Periodicity

Idaho

GRADE STANDARD 3

Explain the reasons for length of a day, the seasons, and the year on Earth.

4

Compare and contrast the basic components of our solar system (planets, sun, moon, asteroids, comets, meteors).

Source: Retrieved May 16, 2008 from http://www.sde.idaho.gov/ContentStandards/ sciencestandards.asp

137


STATE

DOMAIN

GRADE STANDARD

Illinois

Phase Appearance

K-3

Identify and describe characteristics of the sun, Earth and moon as familiar objects in the solar system. Identify daily, seasonal and annual patterns related to the Earth’s rotation and revolution.

Illinois

Periodicity Phase Appearance Cause of Phases

4-8

Identify and explain natural cycles and patterns in the solar system (e.g., order of the planets; moon phases; seasons as related to Earth’s tilt, one’s latitude, and where Earth is in its yearly orbit around the sun).

Source: Retrieved May 16, 2008 from http://www.isbe.state.il.us/ils/science/ standards.htm

138


STATE

DOMAIN

Indiana

Periodicity

2

Investigate by observing and then describe that some events in nature have a repeating pattern, such as seasons, day and night, and migrations.

Indiana

Motion Phase Appearance

3

Observe and describe the apparent motion of the sun and moon over a time span of one day. Observe and describe that the moon looks a little different every day, but looks the same again about every four weeks.

Indiana

Periodicity

4

Observe and report that the moon can be seen sometimes at night and sometimes during the day. Explain that the rotation of Earth on its axis every 24 hours produces the night-and-day cycle.

5

Understand concepts and relationships of the universe

Indiana

GRADE STANDARD

Source: Retrieved May 16, 2008 from http://dc.doe.in.gov/Standards/ AcademicStandards/StandardSearch.aspx

139


STATE

DOMAIN

GRADE STANDARD

Iowa

Periodicity Motion

K-2

Seasons of the year, day and night are events that are repeated in regular patterns. We are unable to see the sun at night because of the rotation of the earth.

Iowa

Periodicity Terminology Motion Cause of Phases

3-5

Most objects in the solar system are in regular and predictable motion. The rotation of the earth on its axis every 24 hours produces the day-and-night cycle. To people on the earth this turning of the planet makes it seem as though the sun, planets, and stars are orbiting the earth once a day. The moon’s orbit around the earth once in about 28 days changes what part of the moon is lighted by the sun and how much of that part can be seen from the earth – the phases of the moon.

Iowa

Periodicity Motion Terminology Cause of Phases

6-8

Seasons result from variations in the amount of the sun’s energy hitting the surface, due to the tilt of the earth’s rotation on its axis and the length of the day. Most objects in the solar system are in regular and predictable motion. Those motions explain such phenomena as the day, the year, phases of the moon, and eclipses.

Source: Retrieved May 16, 2008 from http://www.corecurriculum.iowa.gov/ Discipline.aspx?C=Science&D=Earth+and+Space

140


STATE

DOMAIN

Kansas

GRADE STANDARD 3

The student observes the moon and stars.

Kansas

Periodicity Phase Geometry Scale Terminology

5

The student models spatial relationships of the earth/moon/planets/sun system to scale. The student demonstrates and models object/space/time relationships that explain phenomena such as the day, the month, the year, seasons, phases of the moon, eclipses and tides.

Kansas

Periodicity Phase Geometry Scale

7

The student demonstrates and models object/space/time relationships that explain phenomena such as the day, the month, the year, seasons, phases of the moon, eclipses and tides.

Kansas

Phase Geometry Scale

8

The student understands the relationship between the earth, moon, and sun explains the seasons, tides and moon phases. The student understands the relative sizes and distances of objects in the solar system.

Source: Retrieved May 16, 2008 from http://www.ksde.org/Default.aspx?tabid=144

141


STATE

DOMAIN

GRADE STANDARD

Kentucky Motion Scale Periodicity Phase Appearance

3

Students will describe the properties, locations and real or apparent movements of objects in the sky (Sun, moon). Objects in the sky have properties, locations and real or apparent movements that can be observed and described. Observational data, patterns and models should be used to describe real or apparent movements. Students will understand that the moon appears to move across the sky on a daily basis much like the Sun. The observable shape of the moon can be described as it changes from day to day in a cycle that lasts about a month.

Kentucky Motion Periodicity Phase Appearance

4

Students will understand that the moon appears to move across the sky on a daily basis much like the Sun. The observable shape of the moon can be described as it changes from day to day in a cycle that lasts about a month.

Kentucky Periodicity Motion Phase Geometry

6

Students will explain and predict phenomena (e.g., day, year, moon phases, eclipses) based on models/representations or data related to the motion of objects in the solar system (e.g., earth, sun, moon). Observations and investigations of patterns indicate that most objects in the solar system are in regular and predictable motion. Evaluation of this data explains such phenomena as the day, the year, phases of the moon and eclipses.

Source: Retrieved May 16, 2008 from http://www.kde.state.ky.us/KDE/Instructional+ Resources/Curriculum+Documents+and+Resources/Core+Content+for+Assessment/C ore+ Content+for+Assessment+4.1/

142


STATE

DOMAIN

GRADE STANDARD

Louisiana Phase Appearance Motion Periodicity Terminology Cause of Phases

K-3

Students are to observe and describe the characteristics of objects in the sky. Students are to demonstrate how the relationship of the earth, moon, and sun causes eclipses and moon phases. Students are to observe and record the changing appearances and positions of the moon in the sky at night and determining the monthly pattern of lunar change. Students are to model changes that occur because of the rotation of the earth (alternation of night and day) and the revolution of the earth around the sun.

Louisiana Phase Appearance Motion Periodicity Terminology Cause of Phases

5-8

Students are to model the motions of the earth-moon-sun system to explain day and night, a year, eclipses, moon phases, and tides.

Source: Retrieved May 16, 2008 from http://www.doa.louisiana.gov/osr/lac/28v123/ 28v123.pdf

143


STATE

DOMAIN

GRADE STANDARD

Maine

Motion Phase Appearance

PreK-2

Maine

Periodicity Terminology Motion

3-5

Show the locations of the sun, Earth, moon, and planets and their orbits. Explain the effects of the rotation of Earth on the day/night cycle, and how that cycle affects local temperature.

Maine

Motion Terminology

6-8

Explain the motions that cause days, years, phases of the moon, and eclipses.

Describe how the sun and moon seem to move across the sky. Describe the changes in the appearance of the moon from day to day.

Source: Retrieved May 16, 2008 from http://www.maine.gov/education/lres/pei/

144


STATE

DOMAIN

GRADE

STANDARD

Maryland Phase Appearance

K

Identify and describe the sun, moon and stars.

Maryland Scale Periodicity Motion Phase Appearance

2

Observe and describe changes over time in the properties, location, and motion of celestial objects. Identify and record the apparent visible changes in the shape of the moon over two months of observations. Observe and record changes in the location of the sun and moon in the sky over time. Describe and compare the patterns of change that occur in the sun and the moon.

Maryland Periodicity Motion Phase Appearance Terminology Cause of Phases Phase Geometry

5

Describe the rotation of the planet Earth on its axis. Recognize and describe that the rotation of planet Earth produces observable effects: (1) the day and night cycle, and (2) the apparent movement of the sun, moon, planets, and stars. Verify with models and cite evidence that the moon's apparent shape and position change.

Maryland Scale Periodicity Motion

6

Construct models with accurate scale that represent the position of the Earth relative to the sun and to other planets. Identify and describe the general pattern of movement of all objects in our solar system.

Maryland Periodicity Terminology Phase Appearance Cause of Phases

8

Identify and explain the relationship between the rotation of a planet or moon on its axis and the length of the solar day for that celestial object. Identify and explain the cause of the phases of the moon. Describe how lunar and solar eclipses occur.

Source: Retrieved May 17, 2008 from http://mdk12.org/instruction/curriculum/ science/ index.html

145


STATE

DOMAIN

GRADE STANDARD

Massachusetts Periodicity

K-2

Identify some events around us that have repeating patterns, including the seasons of the year, day and night.

Massachusetts Motion Periodicity Terminology Phase Appearance

3-5

Recognize that the earth revolves around (orbits) the sun in a year's time and that the earth rotates on its axis once approximately every 24 hours. Make connections between the rotation of the earth and day/night, and the apparent movement of the sun, moon, and stars across the sky. Describe the changes that occur in the observable shape of the moon over the course of a month.

Massachusetts Terminology Phase Geometry

6-8

Describe lunar and solar eclipses, the observed moon phases, and tides. Relate them to the relative positions of the earth, moon, and sun.

Source: Retrieved May 17, 2008 from http://www.doe.mass.edu/frameworks/ current.html

146


STATE

DOMAIN

GRADE STANDARD

Michigan Scale Periodicity Phase Appearance Motion

4

Compare and contrast the characteristics of the sun, moon and Earth, including relative distances and abilities to support life. Explain that the spin of the Earth creates day and night. Describe the motion of the moon around the Earth. Explain how the visible shape of the moon follows a predictable cycle which takes approximately one month. Describe the apparent movement of the sun and moon across the sky through day/night and the seasons.

Michigan Motion Terminology Phase Geometry Cause of Phases

5

Describe the motion of planets and moons in terms of rotation on axis and orbits due to gravity. Explain moon phases as they relate to the position of the moon in its orbit around the Earth, resulting in the amount of observable reflected light. Recognize that nighttime objects (stars and constellations) and the sun appear to move because the Earth rotates on its axis and orbits the sun. Explain lunar and solar eclipses based on the relative positions of the Earth, moon, and sun, and the orbit of the moon.

Source: Retrieved May 17, 2008 from http://www.michigan.gov/mde/0,1607,7-14028753_33232---,00.html

147


STATE

DOMAIN

GRADE STANDARD

Minnesota

Terminology Motion Periodicity Scale

3

The student will recognize the difference between rotation and revolution and their connection to day, night, seasons and the year. The student will identify the planets in the solar system and their relative sizes, distances and basic characteristics.

Minnesota

Motion Cardinal Direction

4

The student will recognize that the stars in the sky appear to slowly move from east to west.

Minnesota

Motion Periodicity Cause of Phases Terminology

8

The student will use the predictability of the motions of the Earth, and sun to explain the length of day, length of year, phases of the moon, eclipses, tides and shadows.

Source: Retrieved May 17, 2008 from http://education.state.mn.us/MDE/ Academic_Excellence/Academic_Standards/Science/index.html

148


STATE

DOMAIN

GRADE STANDARD

Mississippi

Phase Appearance

K

Describe the appearance of the moon in the sky (visible by day; bright by night; different shapes).

Mississippi

Phase Appearance Motion Periodicity

1

Describe the changing position of the moon in the sky. Identify day and night as parts of a cycle of regular change.

Mississippi

Phase Appearance Periodicity

2

Describe the apparent shapes of the moon from week to week. Identify the moon’s phases as parts of a cycle of regular change.

Mississippi

Periodicity Motion Cause of Phases Terminology

5

Explore how the Earth’s motion defines the day and the year and influences the phases of the moon and eclipses.

Mississippi

Motion Terminology

6

Demonstrate how the Earth’s motion influences the day, year, phases of the moon, and eclipses.

Mississippi

Phase Appearance Phase Time Phase Geometry Terminology

7

Distinguish between radiating objects (the sun and the stars) and reflecting objects (the planets and their moons). Characterize lunar phases in terms of their appearance, their visibility at a given time of day or night, and their progression through time. Illustrate the relationship between lunar phases and the phase angle between the sun and the moon as seen from Earth. Illustrate the alignments of the Earth, the moon, and the sun, which give rise to solar and lunar eclipses and explain why these eclipses do not occur every month.

Source: Retrieved May 17, 2008 from http://www.mde.k12.ms.us/acad/id/curriculum/ Science/science_curr.htm 149


STATE

DOMAIN

GRADE STANDARD

Missouri

Motion Periodicity

K

Observe and describe the presence of the Sun, moon, and stars in the sky. Recognize the Sun appears to move across the sky from morning to night. Observe the moon can be seen sometimes at night and sometimes during the daytime. Recognize the moon appears to change shape over the course of a month.

Missouri

Motion Cardinal Direction Phase Appearance Phase Time Phase Sequence Periodicity

3

Describe our Sun as a star because it provides light energy to the solar system. Recognize the moon is a reflector of light. Illustrate and describe how the moon appears to move slowly across the sky from east to west during the day and/or night. Observe the change in the moon’s appearance relative to time of day and month over several months and note the pattern in this change. Recognize there is a day/night cycle every 24 hours.

Missouri

Motion Phase Appearance Phase Sequence Periodicity Terminology

5

Recognize the moon orbits the Earth. Sequence images of the lit portion of the moon seen from Earth as it cycles day-today in about a month in order of occurrence (Do NOT assess cause of moon phases). Recognize the Earth rotates once every 24 hours. Relate the apparent motion of the Sun, moon, and stars in the sky to the rotation of the Earth (Do not assess apparent motion of polar constellations).

150


STATE

DOMAIN

Missouri Scale Motion Cardinal Directions Terminology Phase Time Phase Appearance Sequence Phase Geometry Cause of Phases

GRADE STANDARD 7

Classify celestial bodies in the solar system into categories: Sun, moon, planets, and other small bodies (i.e., asteroids, comets, meteors), based on physical properties. Identify the relative proximity of common celestial bodies (i.e., Sun, moon, planets, smaller celestial bodies such as comets and meteors, other stars) in the sky to the Earth. Relate the apparent east-to-west changes in the positions of the Sun, other stars, and planets in the sky over the course of a day to Earth’s counterclockwise rotation about its axis. Observe the change in time and location of moon rise, moon set, and the moon’s appearance relative to time of day and month over several months, and note the pattern in this change. Recognize the moon rises later each day due to its revolution around the Earth in a counterclockwise direction. Recognize the Moon is in the sky for roughly 12 hours in a 24-hour period (i.e., if the Moon rises at about 6 P.M., it will set at about 6 A.M.). Recognize that one half of the Moon is always facing the Sun and, therefore, one half of the Moon is always lit. Relate the apparent change in the moon’s position in the sky as it appears to move east to west over the course of a day to Earth’s counterclockwise rotation about its axis. Describe how the appearance of the moon that can be seen from Earth changes approximately every 28 days in an observable pattern (moon phases). Illustrate and explain a day as the time it takes a planet to make a full rotation about its axis. Describe how the moon’s relative position changes as it revolves around the Earth. Recognize the phases of the moon are due to the relative positions of the Moon with respect to the Earth and Sun.

151


STATE

DOMAIN

Missouri Periodicity Motion Phase Geometry Terminology Phase Time

GRADE STANDARD 9-11

Relate units of time (i.e., day, month, year) to the regular and predictable motion of the planets and moons and their positions in the Solar system. Provide evidence that can be observed from Earth that supports the fact Earth rotates on its axis and revolves around the Sun. Predict the moon rise/set times, phases of the moon, and/or eclipses when given the relative positions of the moon, planet, and Sun.

Source: Retrieved May 17, 2008 from http://dese.mo.gov/divimprove/curriculum/ GLE/SCgle.html

152


STATE

DOMAIN

GRADE STANDARD

Montana

Periodicity Motion

K-4

Students will identify objects (e.g., moon, stars, meteors) in the sky and their patterns of movement and explain that light and heat comes from a star called the sun.

Montana

Periodicity Scale Motion

5-8

Students will describe and model the motion and tilt of earth in relation to the sun, and explain the concepts of day, night, seasons, year, and climatic changes. Students will describe the earth, moon, planets and other objects in space in terms of size, force of gravity, structure, and movement in relation to the sun.

Source: Retrieved May 17, 2008 from http://www.opi.mt.gov/PDF/Standards/ 06CSPDScience.pdf

153


STATE

DOMAIN

Nebraska

GRADE

STANDARD

K-1

Students will develop an understanding of the objects in the sky. Recognize objects in the sky (e.g., the sun, moon, and stars). Investigate that the sun provides heat and light.

Nebraska

Periodicity Motion

2-4

Students will develop an understanding of objects in the sky. Observe and describe how objects move in patterns (e.g., sun, moon, stars, and clouds). Students will develop an understanding of the changes in the earth and sky.

Nebraska

Periodicity Motion Terminology Cause of Phases

5-8

Investigate and list the components of the solar system. Investigate and describe the motion of objects in the solar system that support the concepts of day, year, eclipses, and phases of the moon.

Source: Retrieved May 17, 2008 from http://www.nde.state.ne.us/ndestandards/ documents/ScienceStandards.pdf

154


STATE

DOMAIN

GRADE

STANDARD

Nevada

Periodicity Phase Appearance Motion

K-2

Students know objects in the sky display patterns in how they look, where they are located, and how they move. Students know the Sun rises every day and the Moon can rise during the day and/or the night. Students know the Sun and Moon appear to move across the sky. Students know the Moon appears to change shape over the course of a month.

Nevada

Periodicity

3-5

Students know there are cyclical patterns of observable objects in the solar system.

Nevada


6-8

Students know regular and predictable motions of Earth around the Sun and the Moon around the Earth explain such phenomena as the day, the year, phases of the Moon, and eclipses.

Source: Retrieved May 17, 2008 from http://www.doe.nv.gov/Standards_Science.html

155


STATE

DOMAIN

GRADE

STANDARD

New Hampshire


K-2

Recognize the basic patterns of the Moon, including its appearance sometimes at night and sometimes during the day; and how it appears to change shape through the month. Recognize that the Sun, Moon and stars all appear to move slowly across the sky.

New Hampshire

Terminology Periodicity Motion Scale

3-5

Explain that night and day are caused by the Earth’s rotation on its axis; and that the Earth rotates approximately once, every 24 hours. Recognize that the Moon orbits the Earth. Recognize that astronomical objects in space are massive in size and are separated from one another by vast distances.

New Hampshire

Periodicity Motion Cause of Phases Terminology

6-8

Recognize and describe how the regular and predictable motions of the Earth and Moon explain certain Earth phenomena, such as day and night, the seasons, the year, shadows and the tides. Recognize and describe how the regular and predictable motions of the Earth and Moon account for phenomena, such as the phases of the Moon and eclipses. Explain the temporal or positional relationships between or among the Earth, Sun and Moon (e.g., night/day, seasons, year, tide). Compare and contrast planets based on data provided about size, composition, location, orbital movement, atmosphere, or surface features (includes moons).

New Hampshire

Periodicity Motion

9-11

Explain how gravitational force influenced the formations of the planets and their moons; and describe how these objects move in patterns under its continued influence.

Source: Retrieved May 17, 2008 from http://www.ed.state.nh.us/EDUCATION/doe/ organization/curriculum/CurriculumFrameworks/CurriculumFrameworks.htm#science 156


STATE

DOMAIN

GRADE

STANDARD

New Jersey

Periodicity Motion

K-2

Observe the patterns of day and night and the movements of the shadows of objects on the Earth during the course of a day. Recognize that the sun can only be seen during the day, but the moon can be seen sometimes at night and sometimes during the day.

New Jersey

Periodicity Terminology Motion Phase Appearance

3-4

Observe patterns that result from the Earth's position relative to the sun and rotation of the Earth on its axis. Recognize and describe the phases of the moon.

New Jersey

Periodicity Scale

5-6

Explain how the motions of the Earth, sun, and moon, define units of time including: days, months, years. Using models, demonstrate an understanding of the scale of the solar system that shows distance and size relationships among the sun and planets.

New Jersey

Motion Cause of Phases Terminology

7-8

Investigate the Earth, moon, and sun as a system and explain how the motion of these bodies results in the phases of the moon and eclipses.

Source: Retrieved May 17, 2008 from http://education.state.nj.us/cccs/ ?_standard_matrix;c=5

157


STATE

DOMAIN

GRADE

STANDARD

New Mexico

Motion

K

Observe that there are many objects in the night sky and that some are brighter than others. Describe the location and movements of objects in the sky (e.g., stars, sun, moon).

New Mexico

Motion Periodicity

1

Observe the changes that occur in the sky as day changes into night and night into day. Describe the basic patterns of objects as they move through the sky: (1) sun appears in the day, (2) moon appears at night but can sometimes be seen during the day, (3) sun and moon appear to move across the sky, and (4) moon appears to change shape over the course of a month. Recognize that the sun, moon, and stars all appear to move slowly across the sky.

New Mexico


2

Observe that the phase of the moon appears a little different every day but looks the same again after about four weeks.

New Mexico

Motion Scale

3

Describe the objects in the solar system (e.g., sun, Earth and other planets, moon) and their features (e.g., size, temperature). Describe the relationships among the objects in the solar system (e.g., relative distances, orbital motions).

New Mexico

Scale

5

Know that many objects in the universe are huge and are separated from one another by vast distances (e.g., many stars are larger than the sun but so distant that they look like points of light).

158


STATE

DOMAIN

GRADE

STANDARD

New Mexico


6

Identify the components of the solar system, and describe their defining characteristics and motions in space, including: sun as a medium sized star sun's composition (i.e., hydrogen, helium) and energy production nine planets, their moons, asteroids. Know that the regular and predictable motions of the Earth-moon-sun system explain phenomena on Earth, including: Earth's motion in relation to a year, a day, the seasons, the phases of the moon, eclipses, tides, and shadows, moon's orbit around Earth once in 28 days in relation to the phases of the moon.

New Mexico

Scale Phase Geometry Periodicity Phase Appearance Motion

9-12

Understand the scale and contents of the universe, including: range of structures from atoms through astronomical objects to the objects in the universe such as planets, stars, galaxies, and nebulae. Predict changes in the positions and appearances of objects in the sky (e.g., moon, sun) based on knowledge of current positions and patterns of movements (e.g., lunar cycles, seasons).

Source: Retrieved May 17, 2008 from http://www.ped.state.nm.us/MathScience/ scienceStandards.html

159


STATE

DOMAIN

GRADE

STANDARD

New York

Motion Periodicity Phase Appearance Phase Sequence

K-4

The universe is made up of many different objects. Students should observe and describe the motions of the Sun, Moon, and stars. The movement of these objects through space can be traced and measured over various time segments. By keeping daily records, students will learn to identify sequences of changes and look for patterns; this skill will be useful throughout their study of the natural world. Younger students should draw what they see. Older students should be encouraged to keep journals and use instruments to measure and record their observations. Describe patterns of daily, monthly, and seasonal changes in their environment.

New York

Scale Periodicity Cause of Phases Motion Terminology Cardinal Direction

5-8

The Sun is more than a million times greater in volume than Earth. Most objects in the solar system have a regular and predictable motion. These motions explain such phenomena as a day, year, phases of the Moon, eclipses, tides, meteor showers, and comets. Moons are seen by reflected light. Our Moon orbits Earth, while Earth orbits the Sun. The Moon's phases as observed from Earth are the result of seeing different portions of the lighted area of the Moon's surface. The phases repeat in a cyclic pattern in about one month. The apparent motions of the Sun, Moon, planets, and stars across the sky can be explained by Earth's rotation and revolution. Earth's rotation causes the length of one day to be approximately 24 hours. This rotation also causes the Sun and Moon to appear to rise along the eastern horizon and to set along the western horizon. Earth's revolution around the Sun defines the length of the year as 365 1/4 days.

160


STATE

DOMAIN

New York

Motion Periodicity Terminology

GRADE 9-12

STANDARD Most objects in the solar system are in regular and predictable motion. These motions explain such phenomena as the day, the year, seasons, phases of the moon, eclipses, and tides. Earth rotates on an imaginary axis at a rate of 15 degrees per hour. To people on Earth, this turning of the planet makes it seem as though the Sun, the moon, and the stars are moving around Earth once a day.

Source: Retrieved May 17, 2008 from http://www.emsc.nysed.gov/ciai/cores.htm

161


STATE

DOMAIN

GRADE

STANDARD

North Carolina

Periodicity Motion Phase Sequence Phase Appearance

3

Observe that light travels in a straight line until it strikes an object and is reflected and/or absorbed. Observe that objects in the sky have patterns of movement including: Sun, Moon, and Stars. Use appropriate tools to make observations of the moon. Observe and record the change in the apparent shape of the moon from day to day over several months and describe the pattern of changes.

North Carolina

Periodicity Terminology

6

Analyze the components and cycles of the solar system including: Sun, planets and moons, asteroids and meteors, comets, phases, seasons, day/year, and eclipses.

North Carolina

Motion Terminology

9-12

Analyze planetary motion and the physical laws that explain that motion: rotation; revolution; apparent diurnal motions of the stars, sun and moon; and effects of the tilt of the earth's axis.

Source: Retrieved May 17, 2008 from http://www.dpi.state.nc.us/curriculum/ science/scos/

162


STATE

DOMAIN

North Dakota

GRADE

STANDARD

K

Identify objects (e.g., sun, birds, airplanes, moon) in the sky.

North Dakota

Motion

1

Explain why the sun can only be seen in the daytime, but the moon can be seen sometimes during the day and sometimes at night.

North Dakota

Periodicity Phase Appearance Cause of Phases

2

Explain how the moon appears slightly different every day, but looks nearly the same every four weeks.

4

Identify components of our solar system (e.g., planets, moons, Sun).

North Dakota North Dakota

Motion Periodicity

5

Identify the objects in the sky that have predictable patterns of movement (e.g., sun, planets, moons, stars).

North Dakota

Motion Phase Geometry Cause of Phases Scale Terminology

8

Explain how phenomena on Earth (i.e., day, year, seasons, lunar phases, eclipses, tides) are related to the position and motion of the Sun, Moon, and Earth. Identify the composition (e.g., stars, galaxies) and scale of the universe.

Source: Retrieved May 17, 2008 from http://www.dpi.state.nd.us/standard/content/ science/index.shtm

163


STATE

DOMAIN

Ohio

GRADE

STANDARD

K

Observe that the sun can be seen only in the daytime, but the moon can be seen sometimes at night and sometimes during the day.

Ohio

Motion Periodicity Phase Appearance

2

Observe and describe how the sun, moon and stars all appear to move slowly across the sky. Observe and describe how the moon appears a little different every day but looks nearly the same again about every four weeks.

Ohio

Periodicity Motion

5

Describe how night and day are caused by Earth's rotation. Explain that Earth is one of several planets to orbit the sun, and that the moon orbits Earth.

Ohio


8

Describe how objects in the solar system are in regular and predictable motions that explain such phenomena as days, years, seasons, eclipses, tides and moon cycles.

Ohio

Motion Periodicity Cause of Phases

11

Analyze how the regular and predictable motions of Earth, sun and moon explain phenomena on Earth (e.g., seasons, tides, eclipses and phases of the moon).

Source: Retrieved May 17, 2008 from http://www.ode.state.oh.us/GD/Templates/ Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=1705&ContentID=834&Con tent=51519

164


STATE

DOMAIN

Oklahoma


GRADE 7

STANDARD Most objects in the solar system are in regular and predictable motion. Those motions explain such phenomena as the day, the year, phases of the moon, and eclipses.

Source: Retrieved May 17, 2008 from http://sde.state.ok.us/Curriculum/PASS/Subject/ science.htm

165


STATE

DOMAIN

GRADE

STANDARD

Oregon

Motion

3

Identify and trace the movement of objects in the sky.

Oregon

Motion Periodicity

5

Describe the Earth's place in the solar system and the patterns of movement of objects within the solar system using pictorial models. Recognize that the rotation of the Earth on its axis every 24 hours produces the night-andday cycle.

Oregon

Motion Cause of Phases Terminology

8

Explain the relationship of the Earth's motion to the day, season, year, phases of the moon, and eclipses.

Source: Retrieved May 17, 2008 from http://www.ode.state.or.us/

166


STATE

DOMAIN

GRADE

Pennsylvania

Periodicity Scale

4

STANDARD Illustrate patterns that regularly occur and reoccur in nature. • Use knowledge of natural patterns to predict next occurrences (e.g., seasons, leaf patterns, lunar phases). Know that scale is an important attribute of natural and human made objects, events and phenomena. • Explain the importance of scale in producing models and apply it to a model.

Source: Retrieved May 18, 2008 from http://www.pacode.com/secure/data/022/ chapter4/s4.83.html

167


STATE

DOMAIN

GRADE

STANDARD

Rhode Island

Motion Phase Appearance Periodicity

K-2

The sun can be seen only in the daytime, but the moon can be seen sometimes at night and sometimes during the day. The sun, moon, and stars all appear to move slowly across the sky. The moon looks a little different every day, but looks the same again about every four weeks.

Rhode Island

Motion

3-5

The earth is one of several planets that orbit the sun, and the moon orbits around the earth.

Source: Retrieved May 18, 2008 from http://www.ride.ri.gov/instruction/frameworks/ science/pdf/Science_Framework_Chapter_3B.pdf

168


STATE

DOMAIN

GRADE

STANDARD

South Carolina

Motion Periodicity Phase Sequence

1

Compare the features of the day and night sky. Recognize that the Sun and the Moon appear to rise and set. Illustrate changes in the Moon’s appearance (including patterns over time).

South Carolina

Scale Motion Terminology Phase Appearance

4

Compare the properties (including the type of surface and atmosphere) and the location of Earth to the Sun, which is a star, and the Moon. Explain how the rotation of Earth results in day and night. Illustrate the phases of the Moon and the Moon’s effect on ocean tides.

South Carolina


8

Summarize the characteristics and movements of objects in the solar system (including planets, moons, asteroids, comets, and meteors). Explain the motions of Earth and the Moon and the effects of these motions as they orbit the Sun (including day, year, phases of the Moon, eclipses, and tides).

Source: Retrieved May 18, 2008 from http://ed.sc.gov/agency/Standards-andLearning/Academic-Standards/old/cso/standards/science/

169


STATE

DOMAIN

GRADE

STANDARD

South Dakota

Periodicity

K

Students are able to describe simple Earth patterns (i.e., seasons, day and night, seasons) in daily life.

South Dakota

Motion Periodicity

3

Students are able to recognize changes in the appearance of the Moon over time. • Know that the Moon does not change shape, but at different times appears to change shape. • Explain the relationship between the rotation of the Earth on its axis and the day/night cycle.

South Dakota

Motion Periodicity Terminology Phase Appearance Scale

4

Students are able to describe the motions of Earth, Sun, and Moon. • Revolution and rotation • Use terminology to describe the phases of the Moon (i.e., waning moon or waxing moon). • Describe relative size and position of moons, planets, and stars. • Identify the characteristics of the planets (i.e., appearance, size, distance from the Sun).

South Dakota

Phase Geometry Terminology Cause of Phases

8

Students are able to differentiate the influences of the relative positions of the Earth, Moon, and Sun. • Lunar and solar eclipses, moon phases, tides, seasons

Source: Retrieved May 18, 2008 from http://doe.sd.gov/contentstandards/science/ newstandards.asp

170


STATE

DOMAIN

GRADE

STANDARD

Tennessee

Periodicity

K

Recognize that different objects appear in the day and nighttime sky. Identify objects that appear in the day and nighttime sky. Recognize that there are predictable patterns that occur in the universe. Classify pictures as representative of day or night.

Tennessee

Periodicity

1

Recognize that different objects appear in the day and nighttime sky. Distinguish between objects that appear in the day and nighttime sky. Recognize that the moon is the closest object in the sky. Recognize that there are tools for observing objects in the day and nighttime sky. Recognize that there are predictable patterns which occur in the universe.

Tennessee

Periodicity

2

Recognize that different objects appear in the day and nighttime sky. Recognize that there are predictable patterns which occur within the universe. Determine the approximate time of day, from the position of the sun in the sky. Recognize that the phases of the moon occur in a predictable pattern.

Tennessee

Periodicity Motion Terminology Phase Sequence Phase Appearance

3

Recognize that different objects appear in the day and nighttime sky. Recognize that a telescope serves as a tool for observing distant objects. Recognize that there are predictable patterns that occur in the universe. Explain how day and night result from the rotation of the earth relative to the sun. Observe, identify, and order the basic phases of the moon.

171


STATE

DOMAIN

GRADE

STANDARD

Tennessee

Motion Terminology Phase Appearance

4

Know that objects in space have identifiable characteristics (e.g., appearance, location, and apparent motion). Investigate the patterns and movement of objects in space. Demonstrate how the earth rotates and revolves. Simulate the changing shape of the moon.

Tennessee

Motion Terminology Phase Appearance Cause of Phases

5

Know that objects in space have identifiable characteristics (e.g., appearance, location, and apparent motion). Investigate the patterns and movement of objects in space. Demonstrate how moon phases occur. Explain why the moon appears to change shape. Explain the difference between rotation and revolution in the solar system.

Tennessee

Scale Terminology

6

Investigate the relative distances between objects in space. Describe the positional relationships among the earth, moon, and sun. Illustrate the positions of the earth, moon, and sun during solar and lunar eclipses.

Source: Retrieved May 18, 2008 from http://www.tn.gov/education/ci/sci/

172


STATE DOMAIN Texas

GRADE STANDARD K

The student is expected to observe and identify patterns including seasons, growth, and day and night and predict what happens next.

Texas

Periodicity

4

The student knows that change can create recognizable patterns. The student is expected to identify patterns of change such as in weather, metamorphosis, and objects in the sky.

Texas

Periodicity Phase Appearance

5

The student knows that some change occurs in cycles. The student is expected to identify events and describe changes that occur on a regular basis such as in daily, weekly, lunar, and seasonal cycles.

Texas

Motion Terminology Cause of Phases Periodicity

7

The student is expected to relate the Earth's movement and the moon's orbit to the observed cyclical phases of the moon.

Texas

Periodicity Phase Sequence

8

The student is expected to analyze and predict the sequence of events in the lunar and rock cycles.

Texas

9-10

The student is expected to observe and record data about lunar phases and uses that information to model the earth, moon, and sun system.

Source: Retrieved May 17, 2008 from http://ritter.tea.state.tx.us/rules/tac/ chapter112/index.html

173


STATE DOMAIN Utah

Motion Terminology Periodicity

GRADE STANDARD 3

Explain that the sun is the source of light that lights the moon. Describe the movement of Earth and the moon and the apparent movement of other bodies through the sky. Describe the motions of Earth (i.e., the rotation [spinning] of Earth on its axis, the revolution [orbit] of Earth around the sun). Use a chart to show that the moon orbits Earth approximately every 28 days. Use a model of Earth to demonstrate that Earth rotates on its axis once every 24 hours to produce the night and day cycle. Use a model to demonstrate why it seems to a person on Earth that the sun, planets, and stars appear to move across the sky. Utah Cause of 6 Explain patterns of changes in the appearance of Phases the moon as it orbits Earth. Describe changes in the appearance of the moon Phase Appearance during a month. Periodicity Identify the pattern of change in the moon’s Motion appearance. Use observable evidence to explain the movement Phase of the moon around Earth in relationship to Earth Geometry turning on its axis and the position of the moon changing in the sky. Design an investigation, construct a chart, and collect data depicting the phases of the moon. Demonstrate how the relative positions of Earth, the moon, and the sun create the appearance of the moon’s phases. Identify the difference between the motion of an object rotating on its axis and an object revolving in orbit. Compare how objects in the sky (the moon, planets, stars) change in relative position over the course of the day or night. Model the movement and relative positions of Earth, the moon, and the sun. Source: Retrieved May 17, 2008 from http://www.schools.utah.gov/curr/core/ page2.htm 174


STATE

DOMAIN

Vermont

GRADE STANDARD PreK-K

Students demonstrate their understanding of characteristics of the solar system by observing and recording the day and night sky. (1) The sun can be seen only at day time (2) The sun and moon are in the sky.

Vermont Periodicity Phase Appearance

1-2

Students demonstrate their understanding of characteristics of the solar system by observing and describing qualitatively how the sky looks at different times. Students demonstrate their understanding of characteristics of the solar system by keeping a journal record of the shape of the moon each night for a month. (1) The moon looks slightly different every day, but looks the same again about every four weeks. (2) The moon can be seen sometimes at night and sometimes during the day.

Vermont Motion Terminology Periodicity

3-4

Students demonstrate their understanding of characteristics of the solar system by creating a model of the planets and their correct order from the sun. Students demonstrate their understanding of characteristics of the solar system by drawing or building and then explaining a model of the earth rotating on its axis in relation to the sun and moon (i.e., day and night). (1) The earth is one of several planets that orbit the sun, and the moon orbits the earth. (2) Like all planets and stars, the earth is approximately spherical in shape. The rotation of the earth on its axis every 24 hours produces the night and day cycle.

175


STATE

DOMAIN

Vermont Motion Terminology Periodicity Scale Cause of Phases

GRADE STANDARD 5-6

Students demonstrate their understanding of characteristics of the solar system by creating a diagram or model of the orbit of the earth around the sun and the moon around the earth. (1) The earth orbits the sun in a near circular path that takes a year to complete. (2) The moon’s orbit around the earth once in about 28 days changes the portion of the moon visible to us, as a result of the sun’s reflected light. (phases of the moon). Students demonstrate their understanding of processes and change over time within systems of the universe by explaining (after viewing a picture or illustration with sun/moon showing true relative size) why the sun and moon appear to be the same size when seen from the earth. Students demonstrate their understanding of processes and change over time within systems of the universe by relating this phenomenon to a lunar and solar eclipses. (1) From earth the moon and the sun appear to be the same size, because the moon is so much closer to the earth than the sun. (2) Telescopes magnify the appearance of some very distant objects in the sky, including the moon and the planets. The number of stars that can be seen through telescopes is dramatically greater than can be seen by the unaided eye.

Source: Retrieve May 17, 2008 from http://education.vermont.gov/new/html/ pgm_curriculum/science/gle.html

176


STATE

DOMAIN

GRADE STANDARD

Virginia Motion Periodicity

1

The student will investigate and understand that night and day are caused by the rotation of the Earth.

Virginia Periodicity

3

The student will investigate and understand basic patterns and cycles of natural events (day and night, seasonal changes, phases of the moon, and tides).

Virginia Motion Terminology Cause of Phases Scale

4

The student will investigate and understand the motions of the Earth, moon, and sun (revolution and rotation). The student will investigate and understand the causes for the Earth’s seasons and phases of the moon. The student will investigate and understand the relative size, position, age, and makeup of the Earth, moon, and sun. The Earth completes one revolution around the sun every 365 days. The moon revolves around the Earth about once every month.

Virginia Motion Terminology Periodicity Cause of Phases

6

The student will investigate and understand the sun, moon, Earth, other planets and their moons, meteors, asteroids, and comets. The student will investigate and understand revolution and rotation. The student will investigate and understand the mechanics of day and night and the phases of the moon.

Virginia Terminology

7

The student will investigate and understand the sun-Earth-moon relationships (seasons, tides, and eclipses).

Source: Retrieved May 17, 2008 from http://www.doe.virginia.gov/VDOE/Instruction/ sci_resource.html

177


STATE

DOMAIN

GRADE STANDARD

Washington Periodicity Phase Appearance Motion

K-1

Many things can be seen in the sky. Some change minute by minute, while others move in patterns that can be seen if they are observed day after day. The Moon can be seen sometimes during the day and sometimes during the night. The Moon appears to have different shapes on different days.

Washington Periodicity Phase Geometry Terminology Scale Cause of Phases

6

The Moon's monthly cycle of phases can be explained by its changing relative position as it orbits Earth. An eclipse of the Moon occurs when the Moon enters Earth's shadow. An eclipse of the Sun occurs when the Moon is between the Earth and Sun, and the Moon's shadow falls on the Earth. Earth is the third planet from the sun in a system that includes the Moon, the Sun, seven other major planets and their moons, and smaller objects, such as asteroids, plutoids, and comets. These bodies differ in many characteristics (e.g., size, composition, relative position). Most objects in the Solar System are in regular and predictable motion. These motions explain such phenomena as the day, the year, phases of the moon, and eclipses.

Source: Retrieved March 12, 2008 from http://www.sbe.wa.gov/documents/ WAScienceStandardsDec9.pdf

178


STATE

DOMAIN

West Virginia

Motion

1

Students will recognize that the sun, moon, and stars appear to move. Students will observe and discuss the importance of objects in the day and night sky.

West Virginia

Terminology Motion Periodicity

2

Students will explain how the rotation of the Earth on its axis causes day and night. Students will understand that the moon has phases.

West Virginia

Motion

3

Students will recognize the relative movement of the Earth and moon in relation to the sun.

4

Students will explain the effects of alignment of earth, moon and sun on the earth.

West Virginia

GRADE STANDARD

West Virginia


6

Students will recognize the phases of the moon. Students will investigate models of earthmoon-sun relationships (e.g., gravity, time, or tides).

West Virginia

Motion Terminology

8

Students will explain phenomena associated with motions in sun-earth-moon system (e.g., eclipses, tides, or seasons).

Source: Retrieved May 17, 2008 from http://wvde.state.wv.us/policies/csos.html

179


STATE

DOMAIN

Wisconsin


GRADE STANDARD K-4

Identify celestial objects (stars, sun, moon, planets) in the sky, noting changes in patterns of those objects over time. Students describe observable objects in the sky and their patterns of movement.

Source: Retrieved May 17, 2008 from http://dpi.wi.gov/standards/ sciintro.html#content

180


STATE

DOMAIN

Wyoming

Phase Appearance Motion

GRADE STANDARD K-4

Students describe observable objects in the sky and their patterns of movement.

Source: Retrieved May 17, 2008 from http://www.unavco.org/edu_outreach/ docs/wy.pdf

181


APPENDIX D Taxonomy of 33 Guidelines for Creating Multiple-Choice Tests that Achieved Consensus from 46 Authoritative Sources Item-Writing Guideline 1. Use either the best answer or the correct answer format. 2. Allow time for editing and other types of item revisions. 3. Use good grammar, punctuation, and spelling consistently. 4. Minimize examinee reading time in phrasing each item. 5. Base each item on and educational or instructional objective. 6. Focus on a single problem. 7. Keep the vocabulary consistent with the examinee’s level of understanding. 8. Avoid cuing one item with another; keep items independent of one another. 9. Use the author’s examples as a basis for developing your items. 10. Avoid textbook, verbatim phrasing when developing the item. 11. Avoid items based on opinions. 12. Use multiple-choice to measure higher level thinking. 13. Test for important or significant material; avoid trivial material. 14. State the stem in either a question form or completion form. 15. When using the completion format, don’t leave a blank for completion in the beginning or middle of the stem. 16. Ensure that the directions in the stem are clear, and the wording lets the examinee know exactly what is being asked.

182


Item-Writing Guideline 17. Avoid window dressing (excessive verbiage) in the stem. 18. Word the stem positively; avoid negative phrasing. 19. Include the central idea and most of the phrasing in the stem. 20. Place options in logical or numerical order. 21. Keep options independent; options should not be overlapping. 22. Keep all options in an item homogeneous in content. 23. Keep the length of options fairly consistent. 24. Avoid, or use sparingly, the phrase “none of the above.” 25. Phrase options positively, not negatively. 26. Avoid distractors that can clue test-wise examinees; for example, avoid clang associations, absurd options, formal prompts, or semantic (overly specific or overtly general) clues. 27. Avoid giving clues through the use of faulty grammatical construction. 28. Avoid specific determiners, such as “never” and “always.” 29. Position the correct option so that it appears about the same number of times in each possible position for a set of items. 30. Make sure there is one and only one correct option. 31. Use plausible distractors; avoid illogical distractors. 32. Incorporate common errors of students in distractors. 33. Avoid technically phrased distractors. *Taxonomy of guidelines developed by Haladyna and Downing (1989, p. 44)

183


APPENDIX E Level of Abstraction of Each Item of the CMPA According to Bloom’s Taxonomy CMPA Item 7. What word best defines the movement of the moon on its axis? a. b. c. d.

rotation orbit reflection revolution

8. The path that the moon travels around the Earth is called its: a. b. c. d.

Knowledge

gibbous moon waxing moon waning moon crescent moon

11. A _____________ moon phase is when more than half of the moon appears to be lit in the sky. a. b. c. d.

Knowledge

gibbous moon waxing moon waning moon crescent moon

10. A moon that appears thinner today than it did yesterday is a: a. b. c. d.

Knowledge

spin revolution rotation orbit

9. A moon that appears thicker today than it did yesterday is a: a. b. c. d.

Level of Abstraction Knowledge

gibbous waxing waning crescent

184

Knowledge


CMPA Item 12. When the sun reaches its highest point above the horizon from where you are standing, that point in the sky is known as the sun's: a. b. c. d.

zenith peak median crest

13. What causes the illumination of the moon’s surface? a. b. c. d.

Knowledge

clockwise counterclockwise easterly westerly

16. Similar to the appearance of the sun moving through the sky, the moon appears to move across the sky during the day and night. What causes this? a. b. c. d.

Knowledge

clockwise counterclockwise easterly westerly

15. If you were in a spaceship looking down on the South Pole, in what direction would the moon be traveling around the Earth? a. b. c. d.

Knowledge

The moon reflects light from a comet. The moon radiates its own light. The moon’s light comes from the eye of the observer. The moon reflects light from the sun.

14. If you were in a spaceship looking down on the North Pole, in what direction would the moon be traveling around the Earth? a. b. c. d.


the rotation of the moon the revolution of the moon the rotation of the Earth the revolution of the Earth

185

Comprehension


CMPA Item 17. How many full rotations does the Earth complete in one calendar day? a. b. c. d.

1 rotation 7 rotations 28 rotations 365 rotations

18. How much of the moon’s total surface is lit during a Full Moon phase? a. b. c. d.

Knowledge

1 3 10 30

21. The distance between the moon and the Earth is approximately how many miles? a. b. c. d.

Comprehension

Waning Crescent Moon Waning Gibbous Moon Last Quarter Moon Waxing Crescent Moon

20. The distance between the moon and the Earth can be measured in Earth-diameters. How many Earth-diameters is the moon from Earth? a. b. c. d.

Comprehension

a small part of the moon’s surface 50% 100% none of the moon’s surface

19. We are able to see approximately 25% of the moon's total surface in a __________ phase. a. b. c. d.


38,400 miles 240,000 miles 384,000 miles 1,000,000 miles

186

Knowledge


CMPA Item 22. The distance between the sun and the Earth is approximately how many miles? a. b. c. d.

38,400 miles 150,000 miles 93,000,000 miles 150,000,000 miles

23. The size of the Earth’s diameter is approximately: a. b. c. d.

Knowledge

4 times smaller than the moon’s diameter. 4 times larger than the moon’s diameter. 20 times larger than the moon’s diameter. the same size as the moon’s diameter.

24. The moon rises closest to which direction? a. b. c. d.


Comprehension

North South East West

25. Circle the letter below the window that best illustrates what you might see from the northern hemisphere during a waxing crescent phase.

187

Synthesis


CMPA Item 26. Circle the letter below the window that best illustrates what you might see from the southern hemisphere during a Last Quarter Moon phase.

27. The moon’s shape:

Level of Abstraction Synthesis

Analysis

a. changes from day to day in an unpredictable way. b. appears to change from day to day in an unpredictable way. c. changes from day to day in a predictable way. d. appears to change from day to day in a predictable way. 28. In which phase is the moon not visible? a. b. c. d.

Comprehension

New Moon Full Moon First Quarter Moon Last Quarter Moon

29. Which one of the following reasons correctly explains why the moon is not visible one particular day of the moon's orbit? a. Something is blocking the sun’s light from striking the moon. b. The Earth blocks the view of the moon as it passes through the sun’s rays. c. Light from the observer’s eye is not striking the moon. d. All of the sunlight that reaches the moon is reflected away from the Earth.

188

Analysis


CMPA Item 30. Which of the following cannot be seen in the northern hemisphere?


a. A Waning Crescent Moon just before sunrise in the eastern sky. b. A Full Moon at noon in the southern sky. c. A Waxing Gibbous Moon in the early afternoon in the southern sky. d. A Last Quarter Moon around midnight in the southern sky. 31. If you are located in the northern hemisphere and see a Full Moon setting in the western sky at about 7 AM, which of the following is true?

Synthesis

a. The moon will set tomorrow at about 8 AM in the western sky. b. The moon will set tomorrow at about 8 AM in the eastern sky. c. The moon will set tomorrow at about 7 AM in the western sky. d. The moon will set tomorrow at about 8 AM but it is impossible to predict its direction. 32. If the moon is directly overhead at 4 PM, which of the following best describes the moon’s position 3 hours later? a. b. c. d. 33.

Comprehension

The moon has moved east. The moon has not moved in 3 hours. The moon has moved west. The moon’s position 3 hours later is unpredictable.

The window to the right illustrates the moon in the southern hemisphere tonight. What will it look like 3 days from now? a. The moon will be lit on the right and the crescent will appear thicker b. The moon will be lit on the left and the crescent will appear thicker. c. The moon will be lit on the right and the crescent will appear thinner. d. There is no way to be sure of what the moon will look like 3 days later.

189

Synthesis


CMPA Item 34. The window below illustrates the moon in the northern hemisphere tonight. What will it look like 2 days from now?


a. The moon will be lit on the left and will appear thicker. b. The moon will be lit on the left and will appear thinner. c. The moon will be lit on the right and will appear thicker. d. There is no way to be sure of what the moon will look like 3 days later. 35. Looking down on the North Pole, select the correct position of the moon when a waning crescent moon appears in the sky? Your choices of the position of the moon are shown in the circles numbered 1- 4. a. b. c. d.

1 2 3 4

36. Looking down on the South Pole, select the correct position of the moon when a waxing gibbous moon appears in the sky? Your choices of the position of the moon are shown in the circles numbered 1- 4. a. b. c. d.

Analysis

1 2 3 4

37. Which moon can be seen 2 days after a Full Moon? a. b. c. d.

Analysis

Full Moon Waning Gibbous Moon Waning Crescent Moon It is not possible to know.

190

Analysis


CMPA Item 38. Which moon can be seen 3 days before a First Quarter Moon? a. b. c. d.

Full Moon Waning Crescent Moon Waxing Crescent Moon It is not possible to know

39. If a First Quarter Moon appears today, how many days will it be until a Full Moon appears? a. b. c. d.

Analysis

approximately 3 days approximately 5 days approximately 9 days approximately 13 days

42. Which one of the following moon phases is least likely to be seen between sunrise and sunset? a. b. c. d.

Analysis

7 14 28 The number of days changes from time to time

41. It is 1 day before a Full Moon. How many days ago was the moon in a New Moon phase? a. b. c. d.

Analysis


40. If a Last Quarter Moon appears today, how many days will it be until a First Quarter Moon appears? a. b. c. d.

Level of Abstraction Analysis

First Quarter Moon Waxing Crescent Moon Waning Gibbous Moon Full Moon

191

Synthesis


CMPA Item 43. If you are located in the southern hemisphere and see a Full Moon rising in the eastern sky at about 8 PM, which of the following is true?


a. A Waning Gibbous Moon will rise 3 days later about 3 hours later. b. A Waning Gibbous Moon will rise 3 days later about the same time. c. A Waxing Gibbous Moon will rise 3 days later about 3 hours later. d. It is impossible to predict the way the moon will appear 3 days later. 44. When does a First Quarter Moon reach its zenith? a. b. c. d.

sunrise sunset noon midnight

45. How many days do you have to wait to see the moon just as it appears today? a. b. c. d.

Analysis


46. How many days does it take for the moon to complete one rotation on its axis? a. b. c. d.

Analysis

Knowledge

about 1 day about 7 days about 28 days about 365 days

47. Sometimes the moon looks like a glowing round ball in the sky and other times is looks like a thin fingernail. In your own words, why does the appearance of the moon change from day to day? (open-end item)

192

Synthesis


APPENDIX F Expert Evaluation of the Comprehensive Moon Phases Assessment (CMPA)

Construct Terminology associated with lunar phases Correct Answer rotation Please rank the above assessment item on the following criteria:

1 Weak

2

3

4

5 Strong

1) How well does the item above measure the construct listed? 2) How appropriate is the wording of the stem above and its answer choices for middle school students?

3) How reasonable are the following distractors?

Place an X in the one of the boxes below

Do you have any concerns about the item? Examples: There is more than one correct answer among the choices. There is a cue to the correct answer in the stem. The indicated correct answer is incorrect.

YES

If you marked yes, please type any concerns you might have here:

193

NO


APPENDIX G Expert Evaluation of Item Content Validity Expert evaluations of content validity for each CMPA item are provided below. Many of the reviewers, on one item or another, were influenced by the appropriateness of the wording when they ranked the content validity. In other words, a low ranking on content validity was most often followed by a similar low ranking on wording appropriateness. For this reason, the comments regarding the appropriate wording for early adolescents are presented with the comments on content validity. Reviewers used a 5-point Likert scale to rate the extent to which each item measures the construct for which it was designed. Item evaluations are grouped according to lunar phase domain. A table that displays the reviewers’ content validity ratings for each item in the domain follows the domain definition. Items are listed in each table from highest to lowest rankings. Each item in the domain is shown (with the correct answer marked) as it appears in the CMPA followed by the reviewers’ comments and researcher’s responses. Domains are presented in order of importance with respect to the number of states that established space science standards in the domain. _____________________________________________________________________

194


Periodicity The domain of Periodicity encompasses the knowledge that both the Moon’s changing appearance and day and night occur in a cyclical, predictable pattern.

Content Validity Rankings for Periodicity Items Expert Reviewer Item

Mean

Median

Mode

A

B

C

D

E

F

G

H

46

5

4

5

5

2

5

5

5

4.50

5.00

5

45

5

4

1

5

5

5

5

5

4.38

5.00

5

17

5

4

5

4

5

4

5

3

4.38

4.50

5

The domain Periodicity, as a unique factor measured by the items in the CMPA, was not confirmed by the factor analysis. Item 45, which was designed to measure understanding of the length of one lunar cycle, weakly cross-loaded onto the factors labeled Full/New Moon and Phase Sequence? _____________________________________________________________________ Item 46

Item 46 of the CMPA.

195


Six reviewers scored item 46 with fives, one gave it a four, and one ranked it weak with a score of two. The expert who scored item 46 with a two stated that many students do not believe the Moon rotates. Her comment seems to imply that the item would fail to identify those students who cling to that misconception. Indeed, this item assumes that students know the Moon rotates on its axis. The designers asserted that, as written, item 46 is a more difficult item than one that would identify students who maintain that the Moon does not rotate. _____________________________________________________________________ Item 45


Six reviewers scored item 45 with fives, one gave it a four, and one ranked it weak with a score of one. Item 45 was rated weak by one reviewer, who reminded the assessment designers, “It takes the moon approximately 28 days to orbit the earth but since the earth's position relative to the sun is also changing during that period of time the complete lunar cycle takes 29.5 days.” She concluded that the length of the lunar cycle was sometimes 29 days and sometimes 30 days. This reviewer is entirely accurate. The state standards that target this understanding, however, are not. Five states expect their students to know that the length of the lunar cycle is approximately 28 days. Perhaps the word approximately needs to be included in the first three answer

196


choices. The last choice was included to identify students who hold the misconception that lunar phases do not occur in any predictable pattern. _____________________________________________________________________ Item 17


Four reviewers rated the content validity of item 17 strong with scores of five, three ranked it with scores of four, and one was neutral with a score of three. Two reviewers questioned why the time frame in item 17 was specified as a calendar day. One warned that middle school students are instructed that the Earth completes one rotation in a 24-hour period. The second wondered what other kind of day there could be other than a calendar day. The designers’ thinking was that the use of calendar day as a time frame would require a higher-level of thinking than the use of “24-hour period.” However, the intention was not to trick or confuse students. A third reviewer believed the item was too easy for a middle school student. Another reviewer commented that the stem contained a grammatical error in that the article the should not precede the word Earth. While preceding the noun Earth with the article the might be grammatically taboo, the designers argued adolescents are more familiar with the Earth rather than Earth.

197


_____________________________________________________________________

Motion The domain of Motion incorporates the real movements of rotation and revolution, as well as the apparent movement of the Moon which appears to rise, move across the sky in a westerly direction, and then set along the western horizon, as a result of the Earth’s rotation.

Content Validity Rankings for Motion Items Expert Reviewer Item

Mean

Median

Mode

4

4.75

5.00

5

3

4

4.13

4.00

5

3

4

3.75

4.00

5

A

B

C

D

E

F

G

H

16

5

5

5

5

4

5

5

15

5

4

5

4

3

5

14

5

4

5

2

2

5

The domain Motion also emerged from the factor analysis. Items 14 and 15, which measure understanding of the Moon’s direction with regards to global lunar perspective, strongly loaded. _____________________________________________________________________ Item 16

Item 16 of the CMPA. 198


Item 16 received strong ratings on content validity from all eight reviewers: six scores of five and two scores of four. No comments on this item were included in their evaluations. _____________________________________________________________________ Item 15


The content validity of item 15 was evaluated as strong by three reviewers who ranked it with fives. Three other reviewers rated the item with fours and two reviewers were neutral in their rankings. The comments on content validity were offered by the reviewers for item 15. _____________________________________________________________________

199


Item 14


Three experts rated item 14 strong on content validity with scores of five. Two ranked it with scores of four. One was neutral in ranking the item. Two reviewers found the item to be weak with scores of two. The comments of the two reviewers, who scored item 14 low on content validity, pertained to the construct of GLP (to be presented later in this appendix). One of the reviewers asserted that item 14 would not identify students who believe that the Moon does not move. Someone observing the Moon’s motion from a spaceship located beyond the North Pole would be able to see the Moon complete its orbit over a lunar month. Likewise, another observer in a space station beyond the South Pole would also be able to witness the same cycle. Space can be divided into two sections separated by the Moon’s orbital plane. An observer watching the Moon from one side of the Moon’s orbital plane would describe the direction of the Moon’s motion differently than an observer watching the Moon from the other side of the plane. One might expect a student who resides in the northern hemisphere to select counterclockwise as the correct answer to item 14 even if a vantage point were not specified. Likewise, one

200


might expect a student who resides in the southern hemisphere to select clockwise as the correct answer. However, discerning students (regardless of residency) would be confused without a given vantage point. While the reviewers were instructed to “read the questions through the eyes of” middle school students in the continental United States, it was anticipated that this instrument would be used in other countries (including those located in the southern hemisphere). Indeed, the reviewers were correct in pointing out that a correct answer selected by a northern hemisphere student on this single item is not an indicator of understanding in the GLP domain. ___________________________________________________________________

Terminology The domain of Terminology consists of basic words associated with lunar phases and their meanings. This foundational understanding is needed in order to effectively communicate more complex knowledge. Content Validity Rankings for Terminology Items Expert Reviewer Item

Mean

Median

Mode

4

4.63

5.00

5

5

3

4.50

5.00

5

5

5

5

4.38

5.00

5

2

4

5

4

4.25

4.50

5

4

5

5

5

3

4.13

5.00

5

3

5

5

5

3

4.00

5.00

5

A

B

C

D

E

F

G

H

11

5

3

5

5

5

5

5

7

5

3

5

5

5

5

8

5

3

5

5

2

12

5

4

5

5

10

5

1

5

9

5

1

5

201


The domain Terminology also emerged from the factor analysis. Items seven and eight, which measure knowledge of the terms orbit and rotation, moderately loaded. _____________________________________________________________________ Item 11


Six reviewers found the content validity of Item 11 to be strong as indicated with scores of five. One rated the item with a four and a sixth was neutral in ranking item 11. Concerns about item 11 were expressed by two reviewers. One reviewer was concerned that the phrase more than half “might imply that the moon is getting bigger, hence waxing.” However, more than half of the face of the Moon (the surface that can be seen from Earth) is illuminated in a waning gibbous moon phase, yet this phase is not waxing but waning. In other words, a moon phase in which more than half of the face of the Moon is illuminated could be waxing (i.e., illuminated portion of the face is increasing) or waning (i.e., illuminated portion of the face is decreasing). Hence, as worded, this item requires that a student be able to distinguish between the two

202


concepts of change in illumination (increasing or decreasing) and nominal distinctions (crescent or gibbous). Another reviewer suggested that fill-in-the-blank items be avoided. However, the taxonomy of guidelines developed by Haladyna and Downing (1989) permit the inclusion of items that are constructed in “completion form” (p. 44). Although, the guidelines indicate that a blank is best situated at the end of the stem. The reviewer that scored item 11 with a three on content validity did not provide a comment. _____________________________________________________________________ Item 7

Item seven of the CMPA.

Item seven received two rankings of three on content validity; and the remainder of the ratings was fives. While one expert offered no comment regarding the neutral score, the second stated, The moon rotates once as it revolves once – a fairly abstract concept. It is good to keep together as one movement in my opinion. I know that rotation is the correct answer but in this case it is a more unique and complex movement. Two reviewers thought the wording of the stem might be confusing to middle school students. One suggested the item stem be modified to read, “The movement of the

203


moon on its axis is called …” The other recommended that the word movement be replaced with spin. The fact that the Moon is locked in synchronous rotation is a unique phenomenon. However, the designers’ challenge was to construct items that would measure morsels of understanding. A more difficult question that would identify students who have grasped the more complex phenomenon would fail to identify students who understand only one of the Moon’s motions (i.e., rotation or revolution). _____________________________________________________________________ Item 8

Item eight of the CMPA.

Item eight received one score of two and one of three on content validity. The other six reviewers ranked item eight with fives. However, comments were provided by neither the neutral nor low scoring reviewers on this item. _____________________________________________________________________

204


Item 12


Item twelve received scores of five from four reviewers and scores of four from three reviewers. One reviewer evaluated item twelve as weak with a score of two. The low scoring reviewer offered this explanation: I’m concerned with the fact that there are two meanings to the word zenith. I primarily use it in the sense of being the point directly above an observer (opposite the nadir). Thus if that is the definition held by the student (well, probably most middle schoolers don’t know the term zenith) AND they know that the sun is never at the zenith (alternate definition I indicated) from the northern hemisphere, then s/he will rule that answer out. Thus this may not measure something meaningful. The reviewer makes a good point. There are two meanings for the term zenith. Zenith is “the point on the celestial sphere that is directly above the observer” and also “the highest point above the observer's horizon attained by a celestial body” (zenith, n.d.). While both the Sun and the Moon reach their own zenith in their trek across the sky, both zeniths are distinctly different points. Indeed, neither may coincide with the sky’s zenith. Upon further examination of the stem’s wording, the item might be made clearer by eliminating the phrase from where you are standing. The ultimate question of whether this distinction is crucial for middle school students needs to be answered. _____________________________________________________________________

205


Item 10

Item ten of the CMPA.

Item ten received five scores of five, one score of four, and one low score of one on content validity. One reviewer indicated the content validity of item ten was neutral with a score of three. Six experts were concerned with the word thinner in the stem of item ten. One reviewer was concerned that the use of the word thinner would spawn misconceptions. She confirmed that the Moon’s size is not actually changing and suggested that the item ask students about the Moon’s reflected light or illumination. Another reviewer stated, “When we discuss the phases of the moon we talk about being able to see more or less of the lighted part of the moon. I'm not sure if they [middle school students] would relate the changes of the shape of the moon to changes in thickness.” A third reviewer explained that thickness is a concept used in mathematics to describe height, not width. She went on to say, “… when the students are learning about the crescent moon, they are introduced to waxing and waning to describe these phases. Since a waxing crescent is “thicker” than a waning, it could confuse students as well.” The term thickness is not restricted to height. On the contrary, thickness is often associated with layers. It was for this reason that thicker was specifically chosen to describe the

206


appearance of a waxing moon. Layers of illumination appear to be added to a waxing crescent moon while layers of illumination appear to be removed from a waning crescent moon. Also, the Moon in a waxing phase does not necessarily appear thicker than it would in a waning phase. A fourth reviewer commented that thinner “just struck me funny,” but went on to say that it would probably be adequate. A fifth reviewer offered the following modification: A reflective moon that is decreasing in size is waning. A fifth reviewer stated that he was comfortable with thinner if that is the way the concept is explained in the classroom. _____________________________________________________________________ Item 9

Item nine of the CMPA.

Item nine was ranked similarly to item ten on content validity. Five reviewers gave the item a score of five, two scored it with a three, and one reviewer evaluated it as weak with a score of one. Explanations and comments offered by the reviewers were the same as the ones that accompanied rankings of item ten previously mentioned. _____________________________________________________________________

207


Phase Appearance The domain of Phase Appearance embodies the visual descriptions and images corresponding to each lunar phase.

Content Validity Rankings for Phase Appearance Items Expert Reviewer Item

Mean

Median

Mode

4.86

5.00

5

4

4.75

5.00

5

5

3

4.63

5.00

5

5

5

4

4.63

5.00

5

5

5

3

3

4.38

5.00

5

5

5

2

4

4.38

5.00

5

A

B

C

D

E

F

G

26

5

4

5

5

5

5

5

34

5

4

5

5

5

5

5

28

5

4

5

5

5

5

33

4

4

5

5

5

25

5

4

5

5

27

5

4

5

5

H

The domain Phase Appearance also emerged from the factor analysis. However, items nine, ten and 11 loaded onto the factor identified as Phase Appearance. Items nine and ten strongly loaded. _____________________________________________________________________ Item 26


208


The content validity of item 26 was ranked as strong by six of the reviewer. One reviewer rated it with a score of four. The eighth reviewer did not evaluate item 26 on content validity. No comments were offered regarding the construct of Phase Appearance. _____________________________________________________________________ Item 34

Item 34 of the CMPA. Item 34 was ranked strong on content validity by six of the reviewers with scores of five. Two other reviewers rated it highly with scores of four. The reviewers offered no comments regarding the construct of Phase Appearance. _____________________________________________________________________ Item 28


209


Item 28 was ranked by six reviewers with scores of five, one expert with a score of four. A seventh reviewer scored item 28 with a three. Item 28 drew only one comment regarding the wording of the item stem. The reviewer who gave a neutral score for item 28 on content validity, clarified that all phases are not visible at one time or another. Therefore, he suggested that the stem be refined by adding the phrase “at any time” after “not visible.” _____________________________________________________________________ Item 33


The content validity of item 33 was ranked with scores of five by five reviewers and scores of four by three. In spite of the high scores item 33 received, one expert believed item 33 to be too difficult for middle school students. No other comments were offered regarding the construct of Phase Appearance. _____________________________________________________________________

210


Item 25


Item 25 received high scores of five for content validity from five of the expert reviewers. One reviewer scored the item with a four. Two reviewers rated item 25 as neutral on content validity. The two reviewers who neutrally ranked item 25 chose not to comment on the content validity of the item. None of the other reviewers provided comments on the construct of Phase Appearance. _____________________________________________________________________ Item 27


Item 27 earned scores of five from five reviewers on content validity. Two reviewers rated the item with scores of four. One reviewer evaluated the item as weak with a score of two. One reviewer asserted that item 27 does not measure the construct of Phase Appearance as defined in the evaluation form. The definition that was

211


provided to the reviewers for the construct of Phase Appearance was: the ability to distinguish between the visual images of the phases. Indeed, the item does not correspond with the construct as defined. However, the problem clearly lies with the concise construct definition provided the reviewers. Abbreviated definitions of the constructs were included on the reviewers’ evaluation forms for the sake of visual parsimony. Implied in the domain of Phase Appearance is that while the Moon’s shape appears to change, it never does. Although it might be tempting to accuse the CMPA designers of splitting hairs, 12 states included this understanding in their Earth and space science standards. For example, South Dakota states, “Students will know that the Moon does not change shape, but at different times appears to change shape.” _____________________________________________________________________

Cause of Lunar Phases The domain of Cause of Lunar Phases is a synthesis of knowledge within the domains of Periodicity and Motion. Students who have grasped an understanding in this domain are able to explain the cause of lunar phases based on knowledge of threedimensional objects, reflection of light, perspective, motion, and cycles.

Content Validity Rankings for Cause of Lunar Phases Items Expert Reviewer Item

Mean

Median

Mode

4

4.75

5.00

5

5

4.75

5.00

5

A

B

C

D

E

F

G

H

13

4

5

5

5

5

5

5

47

5

5

5

5

4

4

5

212


The domain Cause of Lunar Phases, as a unique factor measured by the items in the CMPA, was not confirmed by the factor analysis. _____________________________________________________________________ Item 13


Six of the reviewers ranked item 13 strong with scores of five on content validity. Two rated the item with scores of four. No comments or suggestions were included in the evaluations. _____________________________________________________________________ Item 47


The last item, the only open-ended item, received high scores of five from six of the expert reviewers on content validity. Two scored the item with fours. _____________________________________________________________________

213


Scale The domain of Scale is comprised of the knowledge of the differences in the size of the Sun, Earth, and Moon, as well as the understanding that the Sun is much farther away from the Earth than the Moon. Content Validity Rankings for Scale Items Expert Reviewer

Item

Mean

Median

Mode

A

B

C

D

E

F

G

H

23

4

4

5

5

5

5

5

4

4.63

5.00

5

22

5

2

5

3

5

5

5

4

4.25

5.00

5

21

5

2

5

2

5

5

5

4

4.13

5.00

5

20

4

1

5

4

3

5

5

4

3.88

4.00

4

The domain Scale, as a unique factor measured by the items in the CMPA, was not confirmed by the factor analysis. _____________________________________________________________________ Item 23


Item 23 received scores of five from five of the reviewers and scores of four from the other three. Only one reviewer commented on item 23. Her concern was that the item required a great deal of reading and a student who was reading avoidant

214


would have difficulty with this item. This comment certainly provoked the designers to reconsider the wording of item 23. Students who are discouraged by the wordiness of an item will more than likely make no attempt to either think about the concept or select an answer. In such a situation, student understanding cannot be measured, which runs counter to the purpose of this instrument. _____________________________________________________________________ Item 21

Item 21 of the CMPA. Item 21 received five scores of five, one score of four, and two scores of two. The two reviewers that rated item 21 as weak on content validity shared a common concern. One reviewer insisted that the distance be requested in Le Système International d'Unités (SI) units of measure instead of miles. Her rationale was that ”we should use the world’s scientific language of measures.” As mentioned earlier in this section, those who embrace situated cognition cannot argue with this reviewers’ reasoning. The second reviewer claimed that middle school teachers are encouraged to instruct their students using SI units of measure and therefore this instrument should measure that specific understanding. _____________________________________________________________________

215


Item 22


Item 22 received five scores of five, one score of four, one score of three, and one score of two on content validity. The two reviewers who expressed concerns regarding the unit of measure for item 21 echoed their concern for item 22. The decision to use miles instead of SI units of measure was a difficult one for the designers to make. That middle school teachers are encouraged to adopt this unit of measure is not disputed. However, a preference for a unit of measure could not be found in any of the state standards that targeted lunar phases. With this in mind, the designers agreed to measure understanding in this domain with a unit believed to be most familiar to middle school students attending schools in the U.S. _____________________________________________________________________ Item 20


216


Item 20 received three rankings of five, three rankings of four, one score of three, and one low score of one on content validity. The reviewer who rated item 20 as weak on content validity commented, “The assumption that students would know the Earth’s diameter is the construct not that the distance between objects in space is great.” On the contrary, one need not know the diameter of the Earth to have an accurate understanding of the relative distance between the Moon and Earth. In fact, Kansas prescribed that their eighth-grade students understand “the relative sizes and distances of objects in the solar system.” Ten other states have also set objectives for their students to learn the relative sizes of and distances between celestial objects (accomplished through modeling). Using an object within a system as a unit of measure to compare distances between objects within the system, helps students come to terms with the astronomical dimensions of their solar system. Another reviewer feared that this item might prove challenging for a student weak in mathematics. Once again the concept of distance between objects is more easily learned in relative terms by all students with the use of models. _____________________________________________________________________

217


Phase Geometry The Phase Geometry domain consists of an understanding that the relative positions of the Sun, Earth, and Moon have observable effects (i.e., lunar phases).

Content Validity Rankings for Phase Geometry Items Expert Reviewer

Item

Mean

Median

Mode

4.86

5.00

5

A

B

C

D

E

F

G

H

19

4

5

5

5

5

5

5

35

5

5

5

5

1

5

5

4

4.38

5.00

5

36

5

5

5

5

1

5

5

4

4.38

5.00

5

18

5

5

5

5

4

4

5

1

4.25

5.00

5

29

5

3

5

5

3

5

3

4.14

5.00

5

The domain Phase Geometry, as a unique factor measured by the items in the CMPA, was not confirmed by the factor analysis. ______________________________________________________________________ Item 19

Item 19 of the CMPA. Item 19 scored highly on content validity with six reviewers ranking it with fives and one reviewer scoring it with a four. One reviewer did not rate item 19 on content validity. Item 19 drew only one comment regarding content

218


and one regarding formatting. As with item 11, a reviewer suggested that fillin-the-blanks be avoided. The same reviewer claimed that total surface could easily be interpreted by a middle school student to mean total surface from an Earthly perspective. No other concerns were noted regarding the stem of item 19. _____________________________________________________________________ Item 35


Item 35 received strong scores of five from six reviewers, one score of four, and one score of one. The single weak score received by one reviewer was accompanied by a comment pertaining to the construct of GLP (to be presented later in this section). Another reviewer suggested that the North Pole be marked on the diagram. A third expert noted “If students do not know what a waning crescent moon looks like they cannot answer this question.” This reviewer concluded that item 35, as written, also measures the construct of Phase Appearance. He suggested that an image 219


of a waning crescent moon be provided instead of the phase name. It can be argued that a student would not necessarily need to know what a waning crescent moon looks like in order to know its position in the configuration. _____________________________________________________________________ Item 36


Item 36 received six scores of five, one score of four, and one score of one. Most of the concerns that were made regarding item 36 pertained to the construct of GLP (to be presented later in this appendix). One reviewer suggested replacing the phase name with an image of a waxing gibbous moon for the same reason as mentioned for item 35. _____________________________________________________________________

220


Item 18


Item 18 received five scores of five, two scores of four, and one score of one. Item 18 was thought to be ambiguous by one reviewer, who questioned the meaning of total surface. He added that from an Earthly perspective, 50% of the total surface area is all that can be seen and total surface would, therefore, be interpreted by some students as the total surface that is visible from the Earth. This reviewer was convinced (from student interviews) that perspective needs to be identified in this item. If the goal were to identify students who understand that the Moon is a threedimensional sphere and rays from a single light source reach only 50% of the total surface of a three-dimensional sphere, then the item is adequate as worded. However, the intent of item 18 was to measure understanding in the domain of Phase Geometry. Therefore, a student who knows that 100% of the face of the Moon (visible from Earth) is illuminated during a full moon phase may indeed have understanding of the linear (or somewhat linear) relationship of the Sun, Earth, and Moon during a full moon phase. With this in mind, revisions are in order for item 18. _____________________________________________________________________

221


Item 29


Item 29 received four scores of five, and three scores of three on content validity. One reviewer chose not to rank the item on any of the criteria but instead commented that the wording was awkward. Two reviewers who were neutral in rating the content validity of item 29 provided comments. One reviewer believed the phrase one particular day may miscue middle school students to think about a lunar eclipse rather than a new moon phase. The second expert claimed that the item implied both a new moon and a phase that occurs at night. The third neutral ranking reviewer left no comment regarding content validity. _____________________________________________________________________

222


Phase Sequence The domain of Phase Sequence includes knowledge of the various lunar phases the order in which the lunar phases appear. Content Validity Rankings for Phase Sequence Items Expert Reviewer Item

Mean

Median

Mode

5

4.63

5.00

5

5

5

4.63

5.00

5

5

4

4.13

4.50

5

A

B

C

D

E

F

G

H

37

4

3

5

5

5

5

5

38

4

3

5

5

5

5

43

5

3

5

5

4

2

The domain Phase Sequence also emerged from the factor analysis. Items 18, 33, 37, and 45 loaded weakly to moderately onto the factor. _____________________________________________________________________ Item 37


Item 37 received six rankings of five, one score of four, and one neutral score of three on content validity. Although one reviewer was neutral in ranking Item 37, no comment was included with this reviewer’s evaluation. _____________________________________________________________________

223


Item 38


As with item 37, item 38 received six rankings of five, one score of four, and one neutral score of three on content validity. The same reviewer that scored item 37 as neutral also ranked item 38 as neutral. Again, this reviewer failed to comment on the neutral ranking. One expert that ranked item 38 as strong on content validity felt that some students who know the correct answer might get confused by the given number of days prior to a first quarter moon. She suggested that the item be reworded to read: Which moon phase can be seen just before a first quarter moon. _____________________________________________________________________ Item 43


224


Item 43 received four scores of five, two scores of four, one score of three, and one score of two on content validity. The single reviewer that scored item 43 with a two for content validity left a comment on the distractors (to be discussed in the following section) but not on the stem of the item. A second expert stated that the item required too much reading and many adolescents would choose not to answer it and just skip to the next item that was easy to read. Finally, a third reviewer claimed that item 43 was developmentally too difficult for the age group it was intended to assess. _____________________________________________________________________

Cardinal Direction The domain of Cardinal Directions encompasses knowledge of the specific direction the Moon appears to move across the sky due to the Earth’s rotation on its axis and the directions from which the Moon appears to rise and in which the Moon appears to set. Content Validity Rankings for Cardinal Direction Items Expert Reviewer Item

Mean

Median

Mode

4

4.50

5.00

5

5

4

4.50

5.00

5

2

5

4

4.00

4.50

5

5

5

4

3.88

4.00

4

A

B

C

D

E

F

G

H

24

5

4

5

5

3

5

5

32

4

5

5

5

3

5

31

5

3

5

5

3

30

4

5

1

4

3

The domain Cardinal Direction also emerged from the factor analysis. Items 24, 32, and 35 loaded onto the factor. _____________________________________________________________________

225


Item 24


Item 24 received five scores of five, two scores of four, and one score of three on content validity from the reviewers. The only comment that accompanied the neutral ranking for construct validity on item 24 was that the item presumes that students believe that the Moon rises and sets. Indeed, the designers presumed that the teachers who will administer the instrument to their students will instruct or guide them through various activities to witness the rising and setting of the Moon. _____________________________________________________________________ Item 32


Item 32 received five scores of five, two scores of four, and one score of three on content validity. Although item 32 received high scores for content validity, one reviewer who was neutral in ranking the item reminded the assessment designers that 226


the Moon is never directly overhead for much of the northern hemisphere. This reviewer is entirely accurate. Furthermore, she believed item 32 measures understanding in more domains than Cardinal Direction. At first glance, one might suspect that student understanding of the Moon’s motion could be measured by this item. The item draws attention to the Moon’s position as opposed to the Moon’s motion. However, it cannot be assumed that a student who understands the change in the Moon’s position from one point in time to another also possesses an accurate understanding of the Moon’s apparent or real motion. _____________________________________________________________________ Item 31


Item 31 received four scores of five, one score of four, two scores of three, and one score of two on content validity. The only reviewer, who evaluated the content validity of item 31 with a score of two, only commented on the distractors (to be addressed in the next section of this paper). None of the other reviewers offered comments regarding the domain of Cardinal Direction. _____________________________________________________________________

227


Item 30


Item 30 received three scores of five, three scores of four, one score of three, and one score of one on content validity. Item 30 elicited concerns from three reviewers. One believed the complexity of the item might provoke middle school students to guess at the correct answer. Another who evaluated the item as weak stated that, although it appears to be complex, it actually only measures the “relative positions of the earth, moon, and sun during a full moon.” A third reviewer reminded the designers that negative wording might mislead students. _____________________________________________________________________

228


Phase Time The domain of Phase Time includes an understanding of the rise and set times of lunar phases, the length of time between phases, and that the Moon rises approximately 50 minutes later each day. Content Validity Rankings for Phase Time Items Expert Reviewer

Item

Mean

Median

Mode

A

B

C

D

E

F

G

H

42

5

4

5

5

2

4

4

5

4.25

4.50

5

39

5

4

1

5

3

5

5

5

4.13

5.00

5

40

5

4

1

5

3

5

5

5

4.13

5.00

5

43

5

3

5

5

4

2

5

4

4.13

4.50

5

31

5

3

5

5

3

2

5

4

4.00

4.50

5

41

3

4

3

4

3

5

5

5

4.00

4.00

3

30

4

5

1

4

3

5

5

4

3.88

4.00

4

44

4

1

5

4

3

4

5

5

3.88

4.00

4

Domain Phase Time also emerged from the factor analysis. Items 39 and 40 loaded onto the factor. _____________________________________________________________________ Item 42


229


One reviewer, who scored the content validity of item 42 with a two, was concerned that the phrase between sunrise and sunset would cause some students to stumble in their thinking. A second reviewer who gave a rank of four to this item agreed that the wording of the stem needed to be reworded. _____________________________________________________________________ Item 39


The content validity of item 39 received five scores of five, one score of four, one score of three, and one score of one from the reviewers. The reviewer, who scored items 39 with a one, commented that in her own research she had discovered some middle school students who “think that the moon’s phase can change from one phase to another in less than a night.” She asserted that, as worded, item 39 would fail to identify students who held this misconception. _____________________________________________________________________ Item 40


230


Item 40 received five scores of five, one score of four, one score of three, and one score of one on content validity. Item 40 elicited the same concern that was expressed with item 39. Students who believe more than one moon phase can be seen between moonrise and moonset will not be identified with this item, as written. _____________________________________________________________________ Item 43


Item 43 received four scores of five, two scores of four, one score of three, and one score of two. The only reviewer that scored item 43 with a two for content validity left a comment on the distractors but not on the stem of the item. A second expert stated that the item required too much reading and many adolescents would choose not to answer it but instead just skip to the next item that was easy to read. Finally, a third reviewer claimed that item 43 was developmentally too difficult for the age group it was intended to assess. _____________________________________________________________________

231


Item 31

Item 31 of the CMPA. Item 31 received four scores of five, one score of four, two scores of three, and one score of two. The receiver, who rated item 31 with a score of two, did not offer a comment. Another expert stated that the item fails to measure understanding in GLP. _____________________________________________________________________ Item 41


Item 41 received three scores of five, two scores of four, and three scores of three on content validity. Three reviewers gave item 41 a neutral evaluation on content validity. One of these experts commented that she did not like the item. She questioned the reasoning for requiring middle school students to subtract one day from the length of time that separates the first and last quarter moon phases. A second concurred that the phrasing 1 day before might be confusing for the students.

232


Strangely, the designers were unaware that asking middle school students to subtract one from a period of days would incite such opposition. The third expert echoed the concern she had with item 40 regarding those students who believe the moon phase can change within the length of one evening. Once again, modification to this item needs to be considered to accommodate these misconceptions. _____________________________________________________________________ Item 30


Item 30 received three scores of five, three scores of four, one score of three, and one score of one on content validity. Item 30 elicited concerns from three reviewers. One believed the complexity of the item might provoke middle school students to guess at the correct answer. Some students will undoubtedly guess on questions they perceive to be too difficult to answer. However, discerning students who have mastery of the content will make an attempt. This item is definitely discriminating. Another reviewer who evaluated the item as weak stated that, although it appears to be complex, it actually only measures the “relative positions of the earth,

233


moon, and sun during a full moon.” This same reviewer claimed “It has nothing to do with northern and southern hemispheres—the full moon is the same in both.” A third reviewer reminded the designers that negative wording might mislead students. This is a good point. _____________________________________________________________________ Item 44


Item 44 received three scores of five, three scores of four, one score of three, and one score of one on content validity. Item 44 received a weak score on content validity from a reviewer who stated that the item did not take seasonal changes into consideration. Another reviewer echoed this same concern. Granted in January a first quarter moon reaches its zenith approximately one and one half hours after sunset, while in June a first quarter moon reaches its zenith approximately one hour before sunset. In other words, a first quarter moon reaches its zenith within one and one half hours of sunset, regardless of the season. This translates to a marginal 10-15 degrees either side of the Moon’s zenith. Therefore, of the answer choices provided, sunset is the closest timeframe in which a first quarter moon reaches its zenith.

234


A third expert stated that one must know the definition of the term zenith to answer this item correctly. Therefore, this item measures terminology as well as understanding in the construct of Phase Time. Another reviewer remarked that she did not believe that rise and set times of specific lunar phases was an appropriate concept for middle school students. However, Delaware, Missouri, and Mississippi have included this domain in their state standards. _____________________________________________________________________

235


Global Lunar Perspective The domain of Global Lunar Perspective encompasses the specialized knowledge that the Moon’s appearance as seen from the northern hemisphere on any specific day (or night) differs from its appearance as seen from a location in the southern hemisphere on the same day. The reviewers’ rankings for all items assessing GLP were reported earlier in this chapter. However, the rankings for the GLP items will be repeated along with reviewers’ comments and suggestions regarding the construct of GLP. Content Validity Rankings for Global Lunar Perspective Items Expert Reviewer Item

Mean

Median

Mode

4.86

5.00

5

4

4.75

5.00

5

5

4

4.63

5.00

5

5

3

3

4.38

5.00

5

1

5

5

4

4.38

5.00

5

5

1

5

5

4

4.38

5.00

5

5

5

4

2

5

4

4.13

4.50

5

4

5

4

3

5

3

4

4.13

4.00

5

5

3

5

5

3

2

5

4

4.00

4.50

5

30

4

5

1

4

3

5

5

4

3.88

4.00

4

14

5

4

5

2

2

5

3

4

3.75

4.00

5

A

B

C

D

E

F

G

26

5

4

5

5

5

5

5

34

5

4

5

5

5

5

5

33

4

4

5

5

5

5

25

5

4

5

5

5

35

5

5

5

5

36

5

5

5

43

5

3

15

5

31

H

The domain Global Lunar Perspective also emerged from the factor analysis. Items 14 and 15 loaded strongly onto the factor. _____________________________________________________________________

236


Item 26


The content validity of item 26 (see Figure 4.46 below) was ranked as strong by six of the reviewer. One reviewer rated it with a score of four. The eighth reviewer did not evaluate item 26 on content validity. Concerns were raised regarding item 26. One reviewer wanted to remind the assessment designers of what she believed to be the heart of the concept which was that “the phase of the moon is the same in both the northern and southern hemispheres.” Furthermore, this reviewer asserted that the fact that the two images of the same moon phase as seen from these two vantage points are different from one another is too complex for students at this age level. The designers, however, disagree. Middle school students engaged in sharing their Moon journal observations with students from the opposite hemisphere (northern versus southern), as participants in the MOON Project, have quickly discovered this concept. _____________________________________________________________________

237


Item 34


Item 34 was ranked strong on content validity by six of the reviewers with scores of five. Two other reviewers rated it highly with scores of four. No comments regarding content validity pertaining to GLP were offered for item 34. _____________________________________________________________________ Item 33


The content validity of item 33 was ranked with scores of five by five reviewers and scores of four by three. In spite of the high scores item 33 received, one expert believed item 33 to be too difficult for middle school students. Another reviewer added that item 33 was “Great for students who live in the Southern Hemisphere. Northern Hemisphere students probably do not have to learn the Southern Hemisphere unless their curriculum calls for it.” The assessment designers 238


anticipated imminent standard revisions that would include knowledge in the domain of GLP as participation in global learning communities increased with Internet accessibility. _____________________________________________________________________ Item 25


Item 25 received high scores of five for content validity from five of the expert reviewers. One reviewer scored the item with a four. Two reviewers rated item 25 as neutral on content validity. The two reviewers who neutrally ranked item 25 chose not to comment on the content validity of the item. One reviewer stated that the item only indirectly measures the construct of GLP. This reviewer’s suggestion was instead to present an image of the Moon in a waxing crescent phase and reword the stem to read, “Imagine that you are in the northern hemisphere and looked up at the Moon and it looked like the picture below. What would the Moon look like to someone in the southern hemisphere?” Agreed, this item has greater potential of assessing knowledge in the domain of GLP of students residing in the southern hemisphere as opposed to students living in the northern hemisphere. _____________________________________________________________________

239


Item 35


Item 35 received strong scores of five from six reviewers, one score of four, and one score of one. The single weak score received was accompanied by the comment that item 35 does not measure GLP since this reviewer does not believe “you need to know that to answer this question.” The designers assumed that students from the southern hemisphere would be more likely to have learned the positions from a vantage point above the South Pole (the closest pole). With this in mind, it was postulated that southern hemisphere students might have greater difficulty answering this item from the prescribed perspective. Therefore, the designers believed that this item would indeed measure understanding associated with GLP, albeit not as the construct is defined. Reviewers were given the following concise definition (for visual parsimony) for the domain of Global Lunar Perspective: the Moon’s appearance as seen from the southern and northern hemispheres are mirror images. Perspective affects the geometry of objects as well as their appearance. The geometric

240


configuration of the Sun, Earth, and Moon as seen from above the South Pole is different than the configuration as seen from above the North Pole for the same lunar phase. Another reviewer suggested that the North Pole be marked on the diagram. _____________________________________________________________________ Item 36


Item 36 received six scores of five, one score of four, and one score of one. More concerns were made known regarding item 36 than the similar previous item. Three reviewers expressed concerns that the item was too difficult and not relevant to northern hemisphere students. One of the three reviewers was not convinced that the item measures GLP. A second echoed an earlier opinion that emphasis should be placed on the fact that the phases are the same in both hemispheres and less attention drawn to the difference in the phase images as seen from the two diverse perspectives.

241


However, this item (similar to item 35) measures the understanding that the geometry for the phases also differs between northern to southern hemisphere perspectives. _____________________________________________________________________

242


Item 15


The content validity of item 15 was evaluated as strong by three reviewers who ranked it with fives. Three other reviewers rated the item with fours and two reviewers were neutral in their rankings. The two neutral scores for item 15 were accompanied by comments from the reviewers. Both conceded that together items 14 and 15 does indeed measure the GLP construct, although one of the two believed GLP is measured indirectly by the two items. The other reviewer stated, “A student can memorize the answers and never make a connection between the two.” Unfortunately, that can be said for many assessment items requiring students to use only low-level thinking abilities. Yet, one of the goals established early in the development phase of the CMPA was to include some of these less complicated items. It was believed that an assessment that achieves a balance between easy and complex items, or requires students to access their lower- and higher-level thinking abilities, has a greater potential to measure all levels of understanding. One expert that gave item 15 a high score of five suggested that it not follow item 14. She also voiced a concern that students from the northern hemisphere might

243


have a low success rate when answering this item. Another reviewer insisted that the emphasis of GLP be placed on the fact that the Moon’s phases are the same regardless of one’s hemisphere location. She encouraged the designers to downplay the differences in the phase appearances since she believed this concept was too complex for middle school students to grasp. From early childhood, students are encouraged to identify differences as well as similarities. The differences seem to make the topic interesting to study for adolescents. The hope is that identifying the hemisphere differences and similarities will become an integral part of lunar phase inquiry in middle school science classrooms. _____________________________________________________________________ Item 43


Item 43 received four scores of five, two scores of four, one score of three, and one score of two. The only reviewer that scored item 43 with a two for content validity left a comment on the distractors but not on the stem of the item. Another reviewer claimed that item 43 was developmentally too difficult for the age group it was intended to assess. _____________________________________________________________________

244


Item 31

Item 31 of the CMPA. Item 31 received four scores of five, one score of four, two scores of three, and one score of two. The receiver, who rated item 31 with a score of two, did not offer a comment. Another expert stated that the item fails to measure understanding in GLP. _____________________________________________________________________ Item 30


Item 30 received three scores of five, three scores of four, one score of three, and one score of one on content validity. Item 30 elicited concerns from three reviewers. One believed the complexity of the item might provoke middle school students to guess at the correct answer. Some students will undoubtedly guess on questions they perceive to be too difficult to answer. However, discerning students

245


who have mastery of the content will make an attempt. This item is definitely discriminating. Another reviewer who evaluated the item as weak stated that, although it appears to be complex, it actually only measures the “relative positions of the earth, moon, and sun during a full moon.” This same reviewer claimed “It has nothing to do with northern and southern hemispheres—the full moon is the same in both.” _____________________________________________________________________ Item 14


Three experts rated item 14 strong on content validity with scores of five. Two ranked it with scores of four. One was neutral in ranking the item. Two reviewers found the item to be weak with scores of two. One of the two experts who ranked item 14 low on content validity was not convinced that the item measures the GLP construct. Another reviewer who scored item 14 with a three on content validity also stated that the item does not seem to measure the GLP construct; although all experts seemed to be satisfied that the item measured the construct of motion.

246


Someone observing the Moon’s motion from a spaceship located beyond the North Pole would be able to see the Moon complete its orbit over a lunar month. Likewise, another observer in a space station beyond the South Pole would also be able to witness the same cycle. Space can be divided into two sections separated by the Moon’s orbital plane. An observer watching the Moon from one side of the Moon’s orbital plane would describe the direction of the Moon’s motion differently than an observer watching the Moon from the other side of the plane. One might expect a student who resides in the northern hemisphere to select counterclockwise as the correct answer to item 14 even if a vantage point were not specified. Likewise, one might expect a student who resides in the southern hemisphere to select clockwise as the correct answer. However, discerning students (regardless of residency) would be confused without a given vantage point. While the reviewers were instructed to “read the questions through the eyes of” middle school students in the continental United States, it was anticipated that this instrument would be used in other countries (including those located in the southern hemisphere). Indeed, the reviewers were correct in pointing out that a correct answer selected by a northern hemisphere student on this single item is not an indicator of understanding in the GLP domain.

247


APPENDIX H Expert Evaluation of Item Distractors Each expert reviewer was asked to rate the feasibility of the distractors for each item of the CMPA. A 5-point Likert scale was used for this evaluation. Reviewers were encouraged to provide comments. Each item of the CMPA is presented just as it was presented to the middle school students who completed the CMPA along with a table displaying the distractor rankings given by the expert reviewers. The reviewers’ comments pertaining to the item distractors are also provided for each item. The researcher’s responses are included. _____________________________________________________________________ Item 7

Item 7 of the CMPA.

Two of the distractors for item seven received high scores of fours and fives. The distractor reflection received two scores of five and two low scores of one, while four reviewers were neutral in ranking the distractor.

248


Feasibility of the Distractors for Item 7 of the CMPA Expert Reviewer Criterion How reasonable are the following distractors?

Distractor A

B

C

D

E

F

G

H

4

4

5

4

5

4

5

4

1

3

3

5

3

3

5

1

5

5

5

5

5

4

5

5

The distractor, reflection, received a low score from two reviewers. One reviewer commented that it was the only distractor that was not motion related. The other low scoring reviewer did not explain his thinking about the low ranking. Reflection was chosen because it is a term included in the lunar phase unit of instruction. Furthermore, a student would need to know its definition in order to eliminate it as a distractor. Four other reviewers, who were neutral in ranking reflection did not offer a comment. A third reviewer asserted that the item forces a response from a student who does not believe the Moon moves on its own axis. This item fails, then, to clearly identify students with this mental model. _____________________________________________________________________

249


Item 8

Item 8 of the CMPA.

Two of the distractors for item eight received high scores of fours and fives from seven of the eight reviewers with one low score each. Spin received three scores of five, one score of four, two scores of two, and one score of one.


Distractor A

B

C

D

E

F

G

H

1

5

5

2

5

2

5

4

5

5

5

5

1

4

5

4

5

5

5

5

5

2

5

4

The reviewer who rated the distractor, spin, with a score of one commented that spin is the only distractor that is a verb while the other distractors are nouns. However, spin is a noun as well as a verb. The other low scores of two for spin were not accompanied by comments. Another reviewer predicted that “Many students would choose revolution because it is used to explain Earth’s movement around the sun. So the Moon has the same motion.” Still another reviewer was concerned that

250


because revolution is a term that is used to describe the orbit of an object, students might be confused in answering item eight. While revolution is defined as an orbital motion of an object about another object, it is not defined as the path taken by the object in orbit. An analogy used to shed light on the difference between the two terms is: revolution is to orbit as walk is to trail. Clearly, this item discriminates between students who have mastery of lunar phase terminology and those who do not. The weak scores of one for revolution and two for rotation were not accompanied by an explanation. _____________________________________________________________________ Item 9

Item 9 of the CMPA.

Four reviewers evaluated the distractors for item nine as strong. Another reviewer rated the distractors with scores of four. One reviewer rated all of the distractors for item nine as neutral. One expert scored the gibbous moon and crescent moon with threes and the waning moon with a score of four. Five reviewers were concerned with the use of the descriptor thicker in the stem of item nine. No comments were offered to explain the moderate scores received by some of the distractors. 251


Feasibility of the Distractors for Item 9 of the CMPA Expert Reviewer Criterion

Distractor

How reasonable are the following distractors?

A

B

C

D

E

F

G

H

5

3

5

5

5

4

5

3

5

3

5

5

5

4

5

4

5

3

5

3

5

4

5

3

_____________________________________________________________________ Item 10


Four reviewers evaluated the distractors for item ten as strong. Another reviewer rated the distractors with scores of four. One reviewer rated all of the distractors for item nine as neutral. One expert scored the gibbous moon and crescent moon with threes and the waxing moon with a score of four. Concerns expressed for item ten were similar to concerns expressed for item nine. Reviewers feared that adolescents might not associate the descriptive word thinner with a waning moon.

252


Feasibility of the distractors for item 10 of the CMPA Criterion

Expert Reviewer

Distractor


A

B

C

D

E

F

G

H

5

3

5

5

5

4

5

3

5

3

5

5

5

4

5

4

5

3

5

3

5

4

5

3

_____________________________________________________________________ Item 11


Six of the reviewers rated the three distractors for item 11 strong with scores of fives and fours. One reviewer felt waning and crescent were neutral distractors. One reviewer rated waxing and waning with scores of two, yet did not offer an explanation or comment on the two distractors with low rankings. A reviewer, who ranked all distractors for item 11 as strong, suggested that the word lopsided be used to describe a gibbous moon phase instead of more than half of the moon appears to be lit in the sky. It could be argued that lopsided does not adequately describe a gibbous moon.

253



Distractor


A

B

C

D

E

F

G

H

2

4

5

4

5

5

5

5

2

4

5

5

5

5

5

3

5

4

5

5

5

5

5

3

_____________________________________________________________________ Item 12


Three reviewers ranked all the distractors for item 12 with scores of five. While all three distractors received one or two neutral scores, peak received two low scores of two. One reviewer believed that item 12 offered too many answer choices that share the same meaning. For example, crest is defined as the highest point in a wave. While peak is most often associated with mountains and crest is most often associated with waves and mountains, neither are used when describing the highest point of a celestial object’s path across the sky. Median is defined as the middle, as oppose to the highest. No comments accompanied the low score for median or one of the low scores for peak.

254



Distractor


A

B

C

D

E

F

G

H

2

3

5

5

2

5

5

5

4

2

5

5

4

3

5

4

5

3

5

5

4

3

5

4

_____________________________________________________________________ Item 13


Three reviewers scored all of the distractors for item 13 with scores of five. Mixed reviews are displayed in the table below. One reviewer, who rated the last distractor as weak, was skeptical that any middle school student would select it. However, Selley’s (1996) investigation of children’s thinking regarding to light and vision revealed that some children do believe that the eye emits light to illuminate objects. Therefore, this distractor was included to identify students who cling to this dubious misconception.

255



Distractor


A B C D E

F G H

4

2

5

5

5

2

5

1

5

5

5

5

5

2

5

4

1

4

3

5

5

2

5

The reviewer who rated the first distractor as weak with a score of one doubted that an adolescent would believe it to be correct. One expert who scored fives on all the distractors for item 13 suggested that the word radiates be changed to produces in the second distractor. The reviewers who scored twos on the distractors did not comment on the low rankings they gave the distractors. _____________________________________________________________________ Item 14


256


Three reviewers ranked all the distractors for item 14 with scores of five. Easterly received two weak scores of one and one low score of two. Westerly received one low score of two and one weak score of one. One reviewer evaluated easterly and westerly as neutral.


Distractor


A

B

C

D

E

F

G

H

5

4

5

4

5

5

5

4

5

2

5

4

1

3

5

1

5

2

5

1

5

3

5

4

The reviewer who found westerly to be a weak distractor commented, “Students are taught that the moon moves in a westerly direction. In modeling moon phases, we use the term ‘counterclockwise’ to describe the direction of the movement. Since these are both answer choices, this could also be misleading to students.” The two reviewers who scored easterly as weak questioned why easterly is not correct in addition to counterclockwise. Many students are indeed taught that the Moon moves in a westerly direction. However, the Moon only appears to move westerly across the sky. In fact, from an Earthly vantage point, the Moon’s real movement is easterly. The Moon only appears to move westerly due to the Earth’s rotation on its axis. Thirty-one states have documented learning objectives that their students know the difference between the

257


Moon’s real and apparent motions. Although, easterly would be an accurate description of the Moon’s direction of movement from an Earthly vantage point, counterclockwise best describes the motion of the Moon from the vantage point above the North Pole. _____________________________________________________________________ Item 15


Three reviewers ranked all the distractors for item 15 with scores of five. Easterly received one low score of two and westerly received two weak scores of one. Two reviewers believed the two distractors easterly and westerly to be neutral. Feasibility of the Distractors for Item 15 of the CMPA Expert Reviewer Criterion How reasonable are the following distractors?

Distractor A

B

C

D

E

F

G

H

5

4

5

5

5

5

5

4

5

3

5

2

5

3

5

4

5

3

5

5

1

3

5

1

_____________________________________________________________________

258


Item 16


Similar comments accompanied item 15 as those comments that were offered for item 14. Five reviewers rated all the distractors for item 16 strong with scores of five. Two other reviewers evaluated the distractors highly with scores of four. One expert believed all the distractors to be weak as evidenced by the rankings of two. The reviewer who evaluated all the distractors for item 16 as weak offered this comment: The problem I see with this question (which otherwise is a useful question) is that it relies on their understanding of the terms “rotation” and “revolution.” If they do not understand those terms this does not test their underlying understanding. Possibly using more descriptive language would help, though it would be hard to do without giving away answers to other questions (i.e. a possible distractor listed as: the revolution (orbit) of the moon around the earth).


Distractor


259

A

B

C

D

E

F

G

H

5

5

5

5

2

4

5

4

5

5

5

5

2

4

5

4

5

5

5

5

2

4

5

4


_____________________________________________________________________ Item 17


Four reviewers scored all the distractors for item 17 as strong with scores of five. Two other reviewers evaluated the three distractors highly with scores of four. Two reviewers rated the last distractor as weak with scores of one. While on of the two reviewers who rated the last distractor with a score of one offered no comment, the other stated, “If you dropped ‘full’ you could add ‘1/2’ rotation. Seems too easy for middle school, but I may be wrong. Are you asking how long it takes to make one complete rotation? Then different time frames could be offered.” This reviewer further speculated this item seems too easy for middle school students. Yet, 49 states have included standards pertaining to the domain of Periodicity.

260


Feasibility of the Distractors for Item 17 of the CMPA Criterion

Expert Reviewer

Distractor


A

B

C

D

E

F

G

H

5

4

5

4

5

4

5

3

5

4

5

4

5

4

5

3

5

4

5

1

5

4

5

1

_____________________________________________________________________ Item 18


Three of the reviewers evaluated all the distractors for item 18 as strong with scores of five. One other reviewer rated all the distractors highly with scores of four. The one reviewer chose not to rank any of the distractors for item 18 stated that he believed the item to be ambiguous. The first distractor drew one low ranking with a score of two and the third distractor drew one weak ranking with a score of one. Neither of the two reviewers who scored the two distractors with low scores offered an explanation or comment.

261



Distractor A

B

C

D

E

F

G

5

3

5

2

5

4

5

5

5

5

5

4

4

5

5

1

5

5

5

4

5

H

A reviewer who rated all the distractors as strong suggested that this item could be improved if the distractors were consistent. In other words, she suggested that all distractors either describe the portion of the Moon’s surface that is illuminated or provide percentages of the Moon’s surface that is illuminated. _____________________________________________________________________ Item 19


Five reviewers rated all the distractors for item 19 with scores of five. One scored all distractors with fours and one rated the distractors as neutral. As with the previous item, the reviewer that did not rate the distractors claimed the item is ambiguous. The reviewer that scored all distractors with threes speculated that some middle school students would select the only distractor that was not a waxing or waning moon phase. 262



Distractor


A B C D E

F

G H

3

5

5

5

5

4

5

3

5

5

5

5

4

5

3

5

5

5

5

4

5

_____________________________________________________________________ Item 20


Although the second and third distractors for item 20 each received one weak score of one, both received high scores of fours and fives from six other reviewers. The first distractor, one, received three weak scores of one and two neutral scores of three. The reviewer who ranked all the distractors as weak limited her comments to the construct of the item rather than explaining her thinking regarding the distractors. The reviewer who rated the first and third distractors as neutral doubted that any middle school student would select the first distractor for this item. She did, however, admit that she had no evidence on which to base this belief. The third reviewer to rate the first distractor as weak suggested that a distractor greater than the correct answer (e.g.,

263


30 or 50) be offered instead of the answer choice one. Yet, 33 of 375 (9%) middle school students did indeed select the first distractor, one, after formal instruction in lunar phases.


Expert Reviewer

Distractor


A B C D E

F G H

4

1

5

3

1

3

5

1

4

1

4

4

4

4

5

3

4

1

5

5

4

3

5

4

Offering choices greater than and less than the correct answer does indeed help foster the identification of all ranges of student understanding. However, one of the guidelines for creating a good multiple-choice test is to order choices numerically. Therefore, to comply with both would result in placing no correct numerical answers as the first or last choices. Astute middle school students could easily pinpoint such a pattern in the design of an assessment. _____________________________________________________________________ Item 21


264


Three reviewers found the distractors of item 21 to be strong with scores of five. One expert rated them all with scores of four. Another reviewer believed the distractors to be neutral with scores of three. Two reviewers considered the first distractor, 38,400 miles to be weak with scores of one. One of these reviewers was displeased with the choice in the designers’ unit of measure. As mentioned earlier, two reviewers believe that this instrument should use the SI unit to assess students’ understanding in scale. This may indeed explain the weak scores for all distractors given by one of the two disgruntled reviewers.


Distractor A B C D E F G H


4

1

5

3

5

3

5

1

4

1

5

5

5

3

5

4

4

1

5

3

5

3

5

4

Although middle school science teachers may be encouraged to use SI units when teaching distances and dimensions, the designers believed the use of familiar units would more effectively facilitate the assessment of existing knowledge. The other reviewer who rated the first distractor as weak thought the distractor, 38,400 miles, is too obviously low and should be replaced with 80,000 miles or 100,000 miles instead. However, 72 of 375 (19%) middle school students did select the first distractor, 38,400 miles, after formal instruction in lunar phases.

265


_____________________________________________________________________ Item 22


Half of the reviewers rated the first distractor of item 22 either with scores of one or two. The second distractor received two weak scores of one and one low score of two. The last distractor also received one weak score with six other scores of four or five. The reviewer who rated all the distractors as weak was displeased with the units of miles, as previously mentioned. The reviewer who gave scores of two to the first and second distractors stated that because these two distances would place the Sun closer to the Earth than the Moon, students might be able to eliminate them too easily. Likewise, the reviewer who gave the same two distractors scores of one suggested that the first two distractors be replaced with distances closer to but less than the correct answer of 93,000,000 miles.

266





4

1

5

2

5 3

2

1

4

1

5

3

5 3

2

1

4

1

5

5

5 3

5

4

Only 15 of 375 (4%) middle school students selected the first distractor while 36 of 375 (10%) middle school students selected the second distractor. The designers may need to reconsider the first distractor and offer another choice that is a more reasonable (i.e., closer to the correct answer). _____________________________________________________________________ Item 23


While the second distractor of item 23 received high scores of fours and fives from all the reviewers, the first and third distractors received a weak score of one from the same reviewer. Another reviewer rated the first distractor with a low score of two. The reviewer who gave the first distractor a score of two pondered whether there was

267


any middle school student who thinks that the Moon is larger than the Earth. The reviewer who gave the first and their distractors a weak score of one was confident that the first and third distractors would be too obviously eliminated as incorrect answers by middle school students.




2

4

5

3

5 5

5

1

4

4

5

5

5 4

5

4

3

4

5

5

5 4

5

1

A look at the post-test data of 375 middle school students revealed that 30 (8%) students selected the first distractor, while only 14 (4%) students selected the last distractor. A greater number of students, 101 (27%), selected the second distractor. The last distractor may need to be revised. _____________________________________________________________________ Item 24


268


Seven of the eight reviewers rated all the distractors of item 24 as strong with scores of five. Only one reviewer rated the distractors with scores of four. While all the reviewers believed the distractors to item 24 were strong, the reviewer who rated the distractors with scores of four commented that this item presumes that students believe the Moon rises and sets. As stated earlier in the previous section, it is indeed presumed that students who will be asked to complete the CMPA will be exposed to instruct or inquiry which will provide them opportunities to witness the rising and setting of the Moon.




5

5

5

5

5

5

5

4

5

5

5

5

5

5

5

4

5

5

5

5

5

5

5

4

_____________________________________________________________________ Item 25


269


Four reviewers found all of the distractors of item 25 to be strong with scores of five. The second distractor (full moon image) received a low score of two and a weak score of one. The new moon image received two weak scores of one. The image of a waning crescent moon (as seen from a northern hemisphere perspective) received a single weak score of one. Likewise, the image of a first quarter moon (as seen from a northern hemisphere perspective) also received a single weak score of one. The reviewer who scored the distractors set in a blue window stated, “Why do some have a ‘blue’ background? Does this mean it appears in the day? This will be confusing to students.” A reviewer who evaluated all distractors as strong thought that all the distractors should have the same background color. One reviewer rated the distractors set in blue windows as neutral. The reviewer who rated the full moon and new moon images with scores of two speculated that for early adolescents, this might be too many distractors from which to choose. Her evaluation seems to imply that she thinks the item could be improved by removing the low scoring distractors. The reviewer who rated the new moon and full moon images as weak also stated that many of the images will be quickly eliminated by middle school students as incorrect choices.

270





5

4

5

5

5

5

5

3

5

4

5

5

2

5

5

1

5

1

5

5

5

5

3

3

5

4

5

5

5

5

5

3

5

1

5

5

2

5

3

1

5

4

5

5

5

5

5

3

5

4

5

5

5

5

5

3

5

1

5

5

5

5

3

5

It was originally decided that some of the answer choices for this item would be set in a blue window and others set in a dark window to minimize the fostering of the misconception that the Moon can only be seen in the evening. The question of one reviewer, “Does this mean it appears in the day?” is curious. Eliminating the choices of the new moon and full moon windows might easily allow the designers to set the remaining choices on backgrounds of similar color. The question remains – Which sky would be most appropriate for this assessment? In a recent study by Plummer and Krajcik (2008) 40% of the first-grade students they interviewed believed that the Moon only appears at night. However, 80% of the third-grade students had come to understand that the Moon can be seen in

271


the day. While their findings may not be generalized to all elementary students, it is safe to presume that middle school students who have completed a study of lunar phases should not be confused to see the Moon against a daytime background. _____________________________________________________________________ Item 26


Four reviewers rated all the distractors for item 26 as strong with scores of five. One reviewer was neutral in ranking the images set in blue windows, while another rated them as weak with scores of one. One reviewer rated the image of a last quarter moon (as seen from a northern hemisphere perspective) low with a score of two. The comments and the rankings for the distractors of item 26 were very similar to those for the distractors of the previous item. However, the reviewer, who rated the image of a last quarter moon (as seen from a northern hemisphere perspective) low with a score of two, took one more opportunity to persuade the designers to redirect the focus of GLP on the fact that the phases are the same in both hemispheres. It is unclear why one of the reviewers chose not to rank any distractors.

272





5

4

5

5

5

5

5

5

4

5

5

5

5

5

5

4

2

5

5

5

5

5

1

5

5

5

5

3

5

4

5

5

5

5

5

5

1

5

5

5

5

3

5

4

5

5

5

5

5

5

1

5

5

5

5

3

_____________________________________________________________________ Item 27


Four reviewers rated all the distractors for item 27 as strong with scores of five. One reviewer ranked all the distractors highly with scores of four. Another expert 273


was neutral in her ranking of the three distractors. The last distractor received a weak score of one from one of the reviewers. The reviewer that ranked the last distractor as weak expressed a concern that the distractor changes from day to day in a predictable way is too close to the correct answer (i.e., appears to change in a predictable way). She asserted that this distractor will easily mislead students. A reviewer who evaluated all the distractors as strong suggested that the order be changed. She suggested placing the answer choices beginning with changes from day to day … first, followed by those beginning with appears to change from day to day …. Indeed, this rearrangement might encourage students to first consider the difference between real and apparent change in shape and then decide between the choices of predictable or unpredictable.




5

3

5

5

5 3

5

4

5

3

5

5

5 4

5

4

5

3

5

1

5 3

5

4

_____________________________________________________________________

274


Item 28


Four reviewers rated all the distractors for item 28 strongly with scores of five. One reviewer ranked them highly with scores of four. One reviewer believed all the distractors were weak with scores of one. His only comment, however, was that the item was ambiguous. Another reviewer scored the Full Moon as weak and the other distractors as strong. Indeed, replacing this answer choice with a gibbous moon phase might be more feasible. It is unclear why one of the reviewers chose not to rank the last distractor for this item. No comments were offered specifically for the distractors of item 28.


Expert Reviewer

Distractor

A B C D E F G H How reasonable are the following distractors?

5

4

5

5

5 4

1

1

5

4

5

5

5 4

5

1

5

4

5

5

5

5

1

_____________________________________________________________________ 275


Item 29


The first and third distractors for item 29 received strong scores of five from four of the reviewers and a single weak rating. However, the second distractor received three strong scores of five and three weak scores of one. A reviewer who rated all the distractors with scores of five believed the wording of the correct response to be confusing and thus suggested that it be modified to read: All of the sunlight that reaches the moon is reflected away from the side not facing earth.




276

5

1

5

5

5 4

4

5

1

5

5

1 4

1

5

5

5

1

5 4

3


One reviewer stated, “I sense the intent, but the answers seem complicated and abstract as written. Even the right answer seems ambiguous to me. I’m not trying to be over critical, but the wordy answers strike me as complicating things.” No other comments were made about the low evaluations of the distractors. It is unclear why one of the reviewers chose not to rank the distractors. _____________________________________________________________________ Item 30


Three reviewers rated all the distractors for item 30 as strong with scores of five. Three reviewers rated the distractors highly with scores of four. One reviewer was neutral in her rankings. Only one reviewer rated the distractors as weak with scores of one. No comments or suggestions accompanied the reviewers’ rankings for the distractors in this item. The reviewer who scored all the distractors with ones was displeased with the item in general, as mentioned in the previous section. The reviewer who believed the distractors were neutral commented that the item was too complex. In the process of reporting these results, the author discovered that the last distractor is also a correct answer to the stem question.

277





5

4

1

4

3 5

5

4

5

4

1

4

3 5

5

4

5

4

1

4

3 5

5

4

_____________________________________________________________________ Item 31


Five experts ranked all the distractors for item 31 as strong with scores of five. One reviewer highly rated the distractors with scores of four. Two reviewers were neutral in their ranking of the distractors for item 31. One reviewer who rated the distractors as neutral contended the answer choices should be more exact. In fact, the Moon sets approximately 50 minutes later each day instead of one hour. No other comments were made regarding the distractors for item 31.

278





5

3

5

5

5 3

5

4

5

3

5

5

5 3

5

4

5

3

5

5

5 3

5

4

_____________________________________________________________________ Item 32


Five experts believed all the distractors for item 32 to be strong with scores of five. Two reviewers rated the distractors highly with scores of four. One reviewer rated the second and third distractors as neutral. However, he did not comment on the distractors.

279





4

5

5

5

5 5

5

4

4

5

5

5

5 3

5

4

4

5

5

5

5 3

5

4

_____________________________________________________________________ Item 33


Three experts highly rated the distractors for item 33 with scores of four and one reviewer rated all distractors strongly with scores of five. One reviewer found the first and second distractors to be weak, however did not comment or make suggestions to help improve them. The distractor which was moderately rated by one reviewer was not accompanied by a comment. It is unclear why one of the reviewers chose not to rank the last distractor. Another expert reiterated a concern expressed with previous items regarding the use of the descriptions thicker and thinner.

280





4

4

1

4

5 4

5

4

4

4

1

4

5 4

5

4

4

4

5

5 3

5

4

_____________________________________________________________________ Item 34


Four reviewers strongly rated all the distractors of item 34 with scores of five. Two experts highly rated the distractors with scores of four. The same reviewer who moderately rated the last distractor in the previous item rated the same distractor moderately in item 34. However, the reviewer who strongly ranked all of the distractors for item 34, weakly evaluated similarly worded distractors for the previous item. Curiously, the descriptions thicker and thinner did not evoke negative comments from five of the six reviewers who had earlier expressed concerns when the

281


descriptions were used. The sixth reviewer simply stated that the use of the two words could be misleading to adolescents.




5

4

5

4

5

4

5

4

5

4

5

4

5

4

5

4

5

4

5

5

5

3

5

4

_____________________________________________________________________ Item 35


Four experts strongly evaluated all the distractors for item 35 with scores of five. The last distractor drew a single weak score from one of the reviewers. The one reviewer who was neutral in ranking the distractors commented, “This was extremely

282


difficult to interpret. It took me a while to figure out why the circles with numbers next to them were along the left hand side.” However, none of her comments explained the moderate rankings of the distractors. The only reviewer who scored the third distractor as weak suggested that since students are most familiar with the positions of the quarter moon phases, the third position should be moved to the position of the waxing gibbous moon phase.




5

5

4

5

3 4

5

4

5

5

5

5

3 4

5

4

5

5

5

5

3 4

5

1

_____________________________________________________________________

283


Item 36

Item 36 of the CMPA

Four experts found all the distractors to item 36 to be strong with scores of five. Two reviewers highly ranked the distractors with scores of four. One reviewer was neutral in her ranking of the distractors. The second distractor was the only one to receive a low score of two from one of the reviewers. This distractor would be an accurate answer if the vantage point were from above the North Pole. As mentioned in the previous section, two reviewers expressed opinions that item 36 is too complex for middle school students. In line with the comment offered on the previous item, a reviewer suggested that the fourth position be moved to the position of a waxing crescent moon phase.

284





5

5

5

5

3 4

5

4

5

5

2

5

3 4

5

4

5

5

5

5

3 4

5

4

_____________________________________________________________________ Item 37


The reviewers’ ratings were mixed for the distractors of item 37. The first and third distractors received one score each of one and two, yet each received strong scores of five from three of the reviewers. The second distractor received only one low score of two with strong ratings of five from four of the reviewers. The low scorer for the first distractor commented, “Because it is difficult to discern subtle changes in the phase of the Moon, the Moon often appears full for 3-4 days. Since you are testing the order in which the phases appear, not the duration of each phase, you might want to change the full moon to another choice, perhaps waxing crescent.” Reviewer 3 makes

285


a good point. Just as the Moon often appears full for three or four days, it seems that the new moon phase lasts longer than one day. In other words, several days pass between the time when one can locate the smallest waning crescent and the smallest waxing crescent. The reviewer who scored the first distractor as weak suggested that the distractor Full Moon be replaced with Waxing Gibbous Moon. In fact, the waxing gibbous phase precedes the full moon phase, while the waning gibbous phase follows the full moon phase. The reviewer who believed the last distractor to be weak suggested that the CMPA designers avoid choices that are equivalent to “none of the above.” At first glance, the last distractor appears to be similar to an answer choice of “none of the above.” However, a selection of “none of the above” implies that there exists another choice that is correct but is not offered. A selection of the last distractor implies that any phase could randomly appear two days after a full moon is seen. The last distractor was purposefully included to identify students who do not believe that the Moon’s phases occur in a predictable sequence. Feasibility of the Distractors for Item 37 of the CMPA Expert Reviewer Criterion



4

3

2

5

5 4

5

1

4

3

5

5

5 4

5

2

1

3

5

3

5 2

5

4

_____________________________________________________________________

286


Item 38


The first and second distractors for item 38 received high scores of fours and fives from all the reviewers except one who rated them as neutral. The last distractor received one low score of two and one weak score of one. Two other reviewers thought the last distractor to be neutral, while four other experts strongly rated the last distractor. As with the previous item, one reviewer suggested the designers avoid choices that are equivalent to “none of the above.” No other comments were offered for the distractors of item 38.




5

3

5

5

5

4

5

4

5

3

5

5

5

4

5

4

1

3

5

3

5

2

5

5

_____________________________________________________________________

287


Item 39


Four expert strongly ranked the distractors of item 39 with scores of five. Two reviewers highly rated the distractors with scores of four. A single reviewer believed all the distractors for item 39 to be weak with scores of one. The reviewer who rated all the distractors of item 39 as weak offered this comment: The correct answer is the final choice. It takes the moon approximately 28 days to orbit the earth but since the earth's position relative to the sun is also changing during that period of time the complete lunar cycle takes 29.5 days. Therefore the time from full moon to full moon varies from 29 to 30 days and the time from first quarter to full moon is either 7 or 8 days. This year, in April it is 7 and in May it is 8. This reviewer is absolutely correct in the timing of the lunar cycle. The state standards, however, are not so exact. In fact, 14 states have established standards that describe the length of the lunar cycle in one of the following ways: (1) approximately four weeks, (2) 28 days, or (3) one month. The last distractor was included to identify students who do not believe that the Moon’s phases occur in a predictable sequence. These distractors may be improved by including the word approximately with each. One reviewer reported a discovery from her own research. She stated that some middle school children “think that the Moon’s phase can change from one phase to

288


another in less than a night.” Her concern was that the last distractor might not identify students with that specific mental model. The reviewer who lowly rated the last distractor reminded the designers that distracters which stand out (noticeably longer or shorter than the other distractors) are often quickly eliminated by students. Feasibility of the Distractors for Item 39 of the CMPA Expert Reviewer Criterion



5

4

1

5

5 4

5

5

5

4

1

5

5 4

5

5

5

4

1

5

5 4

2

5

_____________________________________________________________________ Item 40


Four experts strongly rated the distractors of item 40 with scores of five. Another reviewer highly evaluated the distractors with scores of four. A single reviewer believed the distractors of item 40 to be weak with scores of one. The last distractor drew two low scores of two and one weak score of one. Comments offered for item 40 were similar to those offered for the previous item. 289





5

4

1

5

5 4

5

5

5

4

1

5

5 4

5

5

5

4

1

5

5 2

2

5

_____________________________________________________________________ Item 41


Two reviewers strongly rated the distractors for item 41 with scores of five. Three reviewers highly ranked the distractors with scores of four. One reviewer evaluated the distractors as weak with scores of two. The reviewer who evaluated all the distractors as weak expressed her dismay that this item requires students to subtract, yet left no specific comment pertaining to the distractors. Another reviewer noticed that all three distractors were shorter time spans than the correct answer and concluded that students could too easily select the correct answer. This reviewer went on to suggest that the first distractor be replaced with one that was a time span longer than the correct answer.

290


A reviewer who highly rated the distractors reiterated a concern previously mentioned regarding students that believe the lunar phases can change over the course of one evening. Feasibility of the Distractors for Item 41 of the CMPA Expert Reviewer Criterion



2

4

1

1

5 4

5

4

2

4

4

3

5 4

5

4

2

4

4

4

5 4

5

4

_____________________________________________________________________ Item 42


Seven experts strongly rated all the distractors of item 42 with scores of five. The eighth reviewer rated the distractors highly with scores of four. The reviewer who ranked all the distractors with scores of four was concerned that students who had no idea or those who think that all phases are equally likely not to appear during the day would have no way to indicate their understanding.

291





5

5

5

5

5 4

5

5

5

5

5

5

5 4

5

5

5

5

5

5

5 4

5

5

_____________________________________________________________________ Item 43


The evaluations of the distractors for item 43 were somewhat mixed. While three reviewers highly rated the distractors for item 43 with scores of four, two reviewers believed the distractors to be strong with scores of five. Two other reviewers were neutral in their evaluations of the distractors. One reviewer thought the first and second distractor to be weak with scores of one. The reviewer who rated the first two distractors as weak commented that item 43 was developmentally too difficult for most middle school students. One reviewer who neutrally ranked the distractors repeated a concern previously mentioned that the distractors should be more exact with respect to time (i.e., the moon rises approximately 50 minutes later

292


each day). The other reviewer who neutrally ranked the distractors offered no comment. Feasibility of the Distractors for Item 43 of the CMPA Expert Reviewer Criterion



4

3

1

5

4 3

5

4

4

3

1

5

4 3

5

4

4

3

5

5

4 3

5

4

_____________________________________________________________________ Item 44


Four experts strongly rated the distractors for item 45 with scores of five. One reviewer highly ranked the distractors with scores of four. Another reviewer neutrally ranked the distractors. Although one reviewer rated all of the distractors as strong, she felt the concept was too specific for middle school students. Her concern led her on a search of the concept in her own state standards, the AAAS Benchmarks, and the

293


National Standards. She did not discover the concept mentioned at any level. The reviewer who rated the second distractor, noon, as weak did not leave a comment. The reviewer who neutrally ranked the distractors and the reviewer who rated the distractors highly claimed that this item does not take seasonal changes into consideration. Granted in January a first quarter moon reaches its zenith approximately one and one half hours after sunset, while in June a first quarter moon reaches its zenith approximately one hour before sunset. In other words, a first quarter moon reaches its zenith within one and one half hours of sunset, regardless of the season. Therefore, of the answer choices provided, sunset is the closest timeframe within which a first quarter moon reaches its zenith.




5

3

5

5

5 4

5

5

5

3

5

2

5 4

5

5

5

3

5

5

5 4

5

_____________________________________________________________________

294


Item 45


Five experts strongly rated the distractors of item 45 with scores of five. Another reviewer evaluated the distractors highly with scores of four. The last distractor received one low score of two and one weak score of one. The reviewer who moderately ranked the first two distractors of item 45 and rated the last distractor as weak repeated the same concern she had of item 39, regarding the length of the lunar cycle. The single rating of two on the last distractor was not explained in a reviewer comment.




5

4

3

5

5 4

5

5

5

4

3

5

5 4

5

5

5

4

1

5

5 2

5

5

_____________________________________________________________________

295


Item 46


Six experts found the distractors of item 46 to be strong with scores of five. The two other reviewers highly evaluated the distractors with scores of four. One of the reviewers who rated the distractors with score of four believed there to be a number of middle school students who maintain that the Moon does not rotate. His implication seems to be that item 46, in presuming that all adolescents believe that the Moon does indeed rotate on its axis, fails to identify adolescents that do not.




296

5

4

5

5

5 4

5

5

5

4

5

5

5 4

5

5

5

4

5

5

5 4

5

5

THE DEVELOPMENT AND VALIDATION OF AN ASSESSMENT THAT ...

THE DEVELOPMENT AND VALIDATION OF AN ASSESSMENT THAT ...

Suggest Documents

Development and Validation of the Personality Assessment ...

Development and Validation of the Personality Assessment ...

DEVELOPMENT AND VALIDATION OF AN

Development and Validation of an Assessment for Pretrial Conditional

Development and Validation of an Assessment for Pretrial Conditional ...

Development and Validation of the Patient Assessment of Chronic ...

Development and Validation of the Quality-of-Life Assessment System ...

Development and validation of the functional assessment of cancer ...

development and validation study - Driving Assessment Conference

Development and Validation of the Information Literacy Assessment in ...

Development and validation of an ultrasensitive ...

Development and Validation of an Ecofriendly Chemiluminescence

Development and Validation of an Individualism ...

Development and Validation of an Electrochemical ...

DEVELOPMENT AND VALIDATION OF AN ALTERNATIVE ...

DEVELOPMENT AND VALIDATION OF AN OPERATIONAL ...

Development and validation of an Entrepreneurial ...

Development and validation of an electrically ...

Development and validation of an ...

Development and Validation of an HPLC Method

Development and Large-Scale Validation of an

Development and Validation of an Energy ... - ScienceCentral

Development and validation of an UV ... - IngentaConnect

Development and Performance Validation of an ... - MAFIADOC.COM