Jun 15, 2015 - ing programs were examined using computer- based analytics, dating from. 1962 to 2013. Variables were Overall Text Complexity Level and ...
Has First-Grade Core Reading Program Text Complexity Changed Across Six Decades? Jill Fitzgerald
ABSTR ACT
The University of North Carolina at Chapel Hill, USA; MetaMetrics, Durham, North Carolina, USA
The purpose of the study was to address possible text complexity shifts across the past six decades for a continually best-selling first-grade core reading program. The anthologies of one publisher’s seven first-grade core reading programs were examined using computer-based analytics, dating from 1962 to 2013. Variables were Overall Text Complexity Level and nine text- characteristic operationalizations: number of Syllables in words and Decoding (word structure); Age of Acquisition, Abstractness, and Word Rareness (word meaning); and Intersentential Complexity, Text Density, Phrase Diversity, and Noncompressibility (discourse level). Multilevel modeling was conducted. There were three main conclusions: (1) Overall Text Complexity levels trended toward more complexity in more recent years. (2) For four of the nine text characteristics, program years were different in the text characteristic progression from the beginning to the end of the first-grade year. Initially in the fall, programs of later years exposed children to word structures (Syllables and Decoding) that were as easy or easier than in earlier years, but there was intense, and incomparable, control over gradually increasing the complexity of word structures throughout the year, ending the first-grade year with the most complex word structures of any year. Simultaneously, as compared with earlier years, there was a pronounced diminished emphasis in the later program years on selected aspects of repetition and redundancy (Text Density and Phrase Diversity) across the first-grade year. (3) Two of the six text characteristics, Age of Acquisition and Word Rareness, trended toward more complexity on average in the later program years.
Jeff Elmore MetaMetrics, Durham, North Carolina, USA
Jackie Eunjung Relyea The University of North Carolina at Chapel Hill, USA
Elfrieda H. Hiebert TextProject, Santa Cruz, California, USA; University of California at Santa Cruz, USA
A. Jackson Stenner MetaMetrics, Durham, North Carolina, USA; The University of North Carolina at Chapel Hill, USA
T
Reading Research Quarterly, 51(1) pp. 7–28 | doi:10.1002/rrq.115 © 2015 International Literacy Association.
he purpose of the present study was to address possible text complexity shifts across the past six decades for a continually best-selling first-grade core reading program (in former years, referenced as a basal reading program). The Common Core State Standards (National Governors Association Center for Best Practices & Council of Chief State School Officers [NGA Center & CCSSO], 2010a, 2010b) brought attention to the concept of text complexity in the United States as never before (Williamson, Fitzgerald, & Stenner, 2013). Standard 10 on text complexity provides a staircase of target complexity levels that students in grades 2–12 should be able to read independently to be on track to graduate college and career ready. The target levels are higher than current-day text complexity exposure levels (Williamson et al., 2013). The Standards authors based the need for the text complexity standard in part on interpretations of prior findings that suggested that texts used in U.S. schools have become less and less challenging over the years (e.g., Chall, 1977; Hayes, Wolfer, & Wolfe, 1996). The authors also argued that on the
7
whole, end- of- high- school text complexity level is significantly lower than college-level text complexity (cf. Williamson, 2008). They concluded that to rise to the required college/workplace levels, students should read increasingly complex text throughout the course of schooling. Although the Common Core authors intentionally excluded kindergarten through first-grade text complexity ranges from their staircase, since the text complexity standard was published, heated debates have arisen about its impact on the youngest students (e.g., Hiebert & Pearson, 2014). The importance of the first step in learning to read has been well documented (e.g., National Institute of Child Health and Human Development [NICHD], 2000). Because core reading programs have dominated center stage for first-grade U.S. reading instruction for many years (Education Market Research, 2014; Hiebert, Martin, & Menon, 2005), the complexity of the programs could have serious implications for students’ reading progress. The evidence for declining text complexity in general across elementary and high school is thin in that very few studies have been published. Also, extant claims of declining text complexity tend to pertain to textbooks generally rather than core reading programs specifically and more often to upper elementary, middle grades, and high school than early elementary. From an analysis of 800 elementary, middle, and high school books, many of which were content area texts, Hayes and colleagues (1996) concluded that especially for students in fourth grade and higher, text difficulty had been significantly reduced from 1919 to 1991. Similarly, an evaluation of best-selling sixth-and 11th-grade texts, including reading texts, literature anthologies, grammar and composition textbooks, and history textbooks from the 1940s through the mid-1970s, revealed signs of decreasing challenge over the period (Chall, 1977). However, for 258 third-and sixth-grade reading textbooks from the 1890s to 2008, some decline was evidenced at third grade during the early decades of the century, but a steady increase in difficulty was noted over the past 70 years (Gamson, Lu, & Eckert, 2013), although sixth-grade difficulty tended to be stable since the 1940s. Notably, the evidence for declining text complexity at the early grade levels is particularly scant, and discerning an overall trend for early grades’ text complexity across the 20th century and into the 21st century is quite difficult due to variation in years, programs studied, and variables examined. On the whole, the prior studies of different time periods lead to only one empirically based conjecture: that in the more recent program years, word structure text characteristics may have been emphasized more than in the comparative prior years (Chall, 1967, 1977, 1983; Hayes et al., 1996;
8 | Reading Research Quarterly, 51(1)
Hiebert, 2005; Hoffman et al., 1994; Hoffman, Sailors, & Patterson, 2002; Popp, 1975). Importantly, to our knowledge, no prior researchers have examined historical first-grade core reading program text complexity trends based on a broad theoretical exposition of the meaning of text complexity. Also, no prior study has covered as many as six decades, including the current decade, and little exploration of complexity shifts over time has occurred for text characteristics beyond the word level, with prior researchers focusing primarily on linguistic characteristics such as word frequency, number of types, number of tokens, type–token ratio, number of unique words, and to a limited extent, degree of orthographic/phonics regularity and sentence length. A more complete, theoretically based understanding of the text complexity shifts in core reading programs across several decades leading to the present could provide an important context for both instructional and policy decisions. In light of decade comparisons, educators who understand text complexity demands for first- grade students could consider whether and how they might supplement the texts of core reading programs with additional texts and forms of instruction. Policymakers who learn about first- grade text complexity trends could better envision the degree to which tougher or more lenient standards are in order.
A Theory of Early Grades’ Text Complexity Positions the Types of Text Characteristics of Major Interest in Relation to Student Reading Progress The outlook on text complexity in the present research is a relational one in which complexity arises among reader, printed text, and the situation and context during the whole reading act (cf. Mesmer, Cunningham, & Hiebert, 2012). When an individual is reading, complexity is in some manner relative to that individual and to contextual characteristics (e.g., age, developmental reading level, degree of instructional support from an expert reader). Similarly, complexity of particular texts is relative to groups of readers at different ages or reading ability levels (cf. Kusters, 2008). So, when viewed on a continuum of complexity in relation to a large group of students’ developmental levels, texts themselves can take on an emergent quality and be assigned a complexity level to situate them on a broad continuum. Optimal first-grade texts are configured so young students can construct rich meanings coupled with ease of effort (cf. Merlini Barbaresi, 2003). Consequently, texts for the early grades are generally written to
heighten certain facets of reading processes (e.g., word decodability) while at the same time requiring minimal effort for meaning creation (cf. Juel & Roper/Schneider, 1985). In fact, a plethora of research suggests that although comprehension and meaning creation are at the core of learning to read, cracking the code consumes the focal effort for beginners, and critical cognitions are phonological awareness and word recognition (e.g., Adams, 1990; Fitzgerald & Shanahan, 2000). Moreover, aligned with a relational theory of text complexity, considerable evidence supports the contention that type of text exposure, including characteristics of the texts themselves, can impact students’ reading processes and progress (e.g., Eason, Goldberg, Young, Geist, & Cutting, 2012; Juel & Roper/Schneider, 1985). Critical text characteristics at the word, sentence, and discourse levels that can support early word-reading learning and development include the following (Fitzgerald et al., 2015; Mesmer et al., 2012): At the word level, simple orthographic configurations can support development of phonics knowledge (e.g., Compton, Appleton, & Hosp, 2004; Juel & Roper/Schneider, 1985), and words with known meanings in oral language can ease the challenge of code breaking (e.g., Muter, Hulme, Snowling, & Stevenson, 2004). At the sentence and discourse levels, repetition of simple words and high- frequency words can facilitate sight-word growth and development of orthographic pattern knowledge (e.g., Vadasy, Sanders, & Peyton, 2005); rhyming words can advance phonological awareness (e.g., Adams, 1990); syntactical simplicity can be related to the ease or challenge for meaning creation (likely due to the working memory demands that are stretched when syntax is more complex; e.g., Graesser, McNamara, & Kulikowich, 2011; Mesmer et al., 2012); text cohesion can affect students’ comprehension and recall (e.g., Graesser, McNamara, & Louwerse, 2003); and other discourse- level characteristics, such as repeated refrains or phrase repetition, can reinforce phonological awareness and sight-word growth, as well as the development of varied word recognition strategies (e.g., Ehri & McCormick, 1998). As a consequence, to capture text characteristics that are associated with students’ emergent reading progress, it is theoretically important to analyze several text characteristics at multiple linguistic levels (Graesser et al., 2011; Mesmer et al., 2012). In the present study, linguistic levels are represented through nine text characteristics: for word structure, number of syllables and decoding levels of words; for word meaning, age at which words are known (age of acquisition), degree of word abstractness, and word rareness; and for sentence/syntax and discourse levels, characteristics that capture repetition and redundancy in text, that is, intersentential complexity, text density (information
load), diversity of phrases, and the degree to which information in the text is compressible.
What Is Known Historically About Trends in Reading Instruction Emphases and Their Reflection in Core Reading Programs? The potential sources of influence on the content and design of first-grade textbooks over the past 50 years are presented in the following section, after which the few prior findings that addressed changes over time in first-grade core reading texts are discussed. (More detail about historical influences on early grades’ core reading program emphases appears in Appendix A, which is available as supporting information for the online version of this article.)
Sources of Influence on First-Grade Core Reading Program Texts Core reading programs represented in the earliest decade of the present study, the 1960s, were influenced by the report of the National Commission on Reading (see Gray, 1925), which was published in the National Society for the Study of Education’s 24th yearbook. Reading proficiency, according to the report, was based on recognition of the high- frequency words that accounted for the majority of the words in texts. Such facility arose from repeated exposure to these high- frequency words that came to be described as the look- say approach. The perspective was implemented in the Elson–Gray Basic Readers (Elson, Gray, & Keck, 1936), a program that became the Curriculum Foundation Series published by Scott Foresman from the early 1940s through the early 1960s. The program was not only used widely in the marketplace but also became the model for a series of look-a like programs from other publishers (Smith, 1965). By the time Flesch (1955) identified the look-say method as a culprit in U.S. students’ reading achievement, all but a very small percentage of U.S. classrooms were using one of the so-called mainstream core reading programs (Austin & Morrison, 1963). Flesch’s (1955) critique of the emphasis on memorization of high-frequency words and the lack of attention to phonics in the first-grade core reading programs led to a series of reports from the 1960s through the 2000s: Learning to Read: The Great Debate (Chall, 1967, 1983), the First-Grade Studies (G.L. Bond & Dykstra, 1967), Becoming a Nation of Readers: The Report of the Commission on Reading (Anderson, Hiebert, Scott, & Wilkinson, 1985), Beginning to Read: Thinking and Learning About Print (Adams, 1990), Preventing
Has First-Grade Core Reading Program Text Complexity Changed Across Six Decades? | 9
Reading Difficulties in Young Children (Snow, Burns, & Griffin, 1998), and the Report of the National Reading Panel (NICHD, 2000). The authors of all of these reports concluded that students who are beginning to learn to read benefit from instruction on letter-sound patterns. Increasingly, the influence of research reports and educational trends has been manifest in the guidelines for textbooks issued by states and districts. In particular, states that limit textbook purchases to an adopted set of materials have impacted textbook content and design, especially the nation’s two largest states: California and Texas (Venezky, 1987). Over the period in the current study, the mandates of these two states have moved from cosmetic and trivial to pedagogical and profound (Hiebert, 2005). Special interest groups and organizations can also influence the creation of guidelines and interpretations of guidelines to texts, as was evident in the advocacy by NICHD staff for decodable texts in the Texas and California guidelines in the late 1990s (McDaniel, Sims, & Miskel, 2001).
Historical Changes in First-Grade Core Reading Program Texts A very small number of prior historical cross-decade analyses involved first-grade text challenge levels, although none investigated the sweep of six decades into contemporary times. Chall (1967) was not the first to analyze first-grade texts (cf. Bliesmer & Yarborough, 1965), but the substantial influence of her conclusions continues, as is evident in the Common Core (NGA Center & CCSSO, 2010a, 2010b). She combined Klein’s (1964) analyses of the first- grade readers of Scott Foresman from 1920–1940 with an additional analysis of the 1956 and 1962 Scott Foresman first-grade readers. Chall reported a steady decrease in new words from 1920 (2.40 new words per 100) to 1956 (1.40 per 100). In 1962, however, the pattern had reversed, trending upward (1.90 new words per 100). Also, for texts from 1936 to 1962, Chall (1977) arrived at a mixed conclusion: For two of three text characteristics, the texts were easier, but for number of new words per 100 running words, there may have been a reversal of the trend by 1962. Popp (1975) likewise suggested that in the 1970s, selected text aspects, such as decoding demand, were more challenging than previously and more challenging earlier. Hayes and colleagues (1996) provided further confirmatory evidence of a shift to greater word difficulty in first-grade texts from 1963 to 1991. Hayes and colleagues described changes in the mean lexical difficulty (of content words) within primer and first-grade readers across three clustered periods: 1919–1945, 1946– 1962, and 1963–1991. The mean lexical difficulty for
10 | Reading Research Quarterly, 51(1)
first-grade texts decreased from −47.30 in the first period to −59.70 in the second period. However, by the third period, the difficulty increased, indicating more challenging lexical difficulty of content words in texts (−50.30). Many of the analyses of first-grade reading texts that followed Chall’s (1967) study compared texts of different programs from a particular era on a specific feature, particularly orthographic regularity of words (e.g., Beck & McCaslin, 1978; Foorman, Francis, Davidson, Harm, & Griffin, 2004; Stein, Johnson, & Gutlohn, 1999). The analyses provided snapshots of textbook features at particular points in time but failed to provide an understanding of the changes over time. One exception to the pattern of comparing texts on a specific text-characteristic feature was a set of studies conducted on the first-grade core reading programs adopted by Texas from 1986 to 2000. In the first study, Hoffman and colleagues (1994) compared the 1993 programs with those of the previous adoption in 1986. The 1993 series showed substantial changes from the previous programs in both word-level and content features. In comparison with the earlier year, the vocabulary of 1993 was less controlled and more demanding, and the literature was more engaging and texts more predictable, possibly related to the implementation of a whole- language philosophy. Hoffman and colleagues (2002) found that first-grade textbooks changed again in 2000 when the Texas Education Agency called for attention to decodability of words in student texts. The texts included more decodable words, and the features of engagement and predictability of the previous adoption had diminished. Hiebert (in press) also examined the word-level demands of first-grade textbooks for 1962, 1993, 2000, and 2008. Similar to Chall’s analysis (1967, 1983), Hiebert considered the number of unique words per 100 and, additionally, the percentage of single- appearing words across four copyrights. The task for entry-level first graders, at least as based on number of new words per 100 and single-appearing words, could be judged substantially more challenging in 2008 than in 1962. The number of new words per 100 had increased from 10 in 1962 to 131 in 2008. Whereas none of the words in the 1962 beginning first-grade texts appeared a single time, 35% of the unique words in the first- grade text appeared a single time in the 2008 program. In sum, no prior study has broadly examined the continuous multilevel linguistic text complexity shift from the 1960s into the present decade. However, collectively, the prior study results suggest that from the 1960s to the late 2000s, research findings and policy appear to have substantially shaped early grades’ core reading program design emphases, with a movement
toward greater emphasis on word structure text characteristics coupled with less emphasis on word repetition. What is needed is a clearer, coherent, theoretically based understanding of text complexity trends into the contemporary decade that includes text characteristics at multiple linguistic levels.
Research Questions The three main research questions guiding this study were the following: 1. Did overall text complexity levels shift across the seven program years? 2. For each of nine text characteristics, did the progression of text-characteristic complexity from the beginning to the end of the first-grade year vary across program years? 3. On the whole, did text-characteristic levels vary as a function of program year? Based on the limited prior research on first-grade text-characteristic shifts across selected time periods, we predicted that word structure text characteristics would be more complex and that possibly word meanings might be more challenging in later years as compared with earlier years. However, because discourse-level characteristics were rarely studied in the past, there was not a sufficient empirical basis for predicting discourse-level trends. Also, because little prior evidence was available for assessing program shifts from the beginning to the end of a year, we were unable to predict whether there would be differences across program years for within-year text-characteristic shifts.
Methods First-Grade Core Reading Programs Seven first-grade core reading programs were collected from one publisher (Scott Foresman) for 1962 (Robinson, Monroe, & Artley, 1962), 1971 (Aaron et al., 1971), 1983 (Aaron, Jackson, Riggs, Smith, & Tierney, 1983), 1993 (Allington et al., 1993), 2000 (Afflerbach et al., 2000), 2007 (Afflerbach et al., 2007), and 2013 (Afflerbach et al., 2013). Across the six decades the seven published programs were most likely continuously among the top best-selling series in the United States (cf. Chall, 1967, 1983; Education Market Research, 2014; Foorman et al., 2004). Scott Foresman was s elected for the current study because it was one of the core reading programs examined by Chall (1967, 1983) in her landmark exposition
about the “great debate,” and Scott Foresman is the only one of the publishers in that study that is still active; also, the publisher’s programs are typical of current best- selling core reading programs (Foorman et al., 2004; Hiebert, 2005). The seven years were selected to represent the earliest years of a decade, with the exception of 2007. The 2007 program was the most recent publication prior to the release of the Common Core State Standards, and the 2013 program was after that release. Only the anthologies intended for use during reading instruction were included in our analyses. That is, no supplemental materials were included. The texts were scanned and digitized. To create equal text units within year, we divided each program year into 10 equal sections (similar to the 10 months of a school year) that had approximately equal numbers of words in them.
Measures Ten measures were employed, one for Overall Text Complexity and nine for individual text characteristics at multiple linguistic levels. (More complete detail about the development of all measures in the prior study is available in Fitzgerald and colleagues’ 2015 study and in supplemental materials that accompany that article. Also, interested parties may obtain computer code for measures from the second author.)
Overall Text Complexity Level The development of the Overall Text Complexity measure was described by Fitzgerald and colleagues (2015). In brief, because text complexity was defined as relational to students reading texts in a particular context, the Overall Text Complexity measure for early grades’ texts was based on student responses during a reading task and teachers’ ordering of texts according to complexity. That is, two parallel substudies were accomplished for its development: 1,258 first-and second-grade students in 10 U.S. states completed a maze task, and 90 practicing primary-grade teachers from 33 states and 75 school districts completed a text- ordering task. For the early grades texts, 350 were used. Using Rasch modeling (T.G. Bond & Fox, 2007), two logit scales were created: one from the student task and one from the teacher task. Cronbach’s α estimates of reliability for all test forms from the student task ranged from .85 to .96. Dimensionality assessments for text genre, student ethnicity, gender, and free or reduced-price lunch status suggested no evidence of measurement multidimensionality. Separation index reliability (Wright & Stone, 1999) of the teacher ordering was .99. The correlation between the two logit scales was .79 (p
