determining english language learners' response to

DETERMINING ENGLISH LANGUAGE LEARNERS' RESPONSE TO INTERVENTION: QUESTIONS AND SOME ANSWERS Sylvia Linan-Thompson,

Paul T. Cirino, and Sharon Vaughn

Abstract. Using an extant database, we examined three grade 1 criteria for identifying response to intervention (RTI) in English language learners (ELLs): (a) set benchmark criteria with a standard score above 95 (37th %ile) on both decoding and comprehension measures and a raw score of 40 or more correct words per minute (CWPM) on oral reading fluency; (b) discrepancy benchmark criteria, with performance on these measures at or above the mean of not-at-risk peers; and (c) discrepancy slope criteria, with growth during grade 1 on these measures at or above the mean of not-at-risk peers. The sample consisted of 81 students (41 intervention, 40 comparison) who were bilingual (Spanish/English) and were part of a supplemental reading instruction study during first grade. The three grade 1 criteria were evaluated in relation to a set benchmark criteria in grade 2. Results indicated that approximately 80% of the students did not meet any criteria in either year, but that the discrepancy slope criteria in grade 1 were most predictive of set benchmark criteria in grade 2. Recognizing that we applied highly stringent criteria, implications and issues are presented related to using RTI with ELLs to facilitate decision making about further intervention and referral for special education.

Sylvia Linan-Thompson, Ph.D., associate professor and fellow in the Mollie V. Davis Professorship in Learning Disabilities, The University of Texas at Austin. Paul T. Cirino, Ph.D., research assistant professor. The University of Houston. Sharon Vaughn, Ph.D., professor and H. E. Hartfelder/Southland Corp Regents Chair, The University of Texas at Austin.

A longstanding concern in special education has been the over- and under-representation of students from linguistically diverse groups in special education due to inappropriate assessment and instruction (Donovan & Cross, 2002). These concerns have not been unfounded. Disenchantment with the use of IQachievement discrepancy to identify children with learning disabilities (LD) has been growing, as increasing evidence has emerged that it does not discriminate

between IQ-discrepant and nondiscrepant low achievers and other subgroups of low-performing students (Fletcher et al., 1994; Stanovich & Siegel, 1994; Vellutino, Scanlon, & Lyon, 2000). Furthermore, consensus reports (Donovaan & Cross, 2002; Fletcher et al., 2002; President's Commission on Excellence in Special Education, 2002) agree that learning problems are common in schools, and that lack of appropriate instruction can lead to identification for disability in students.

Volume 30, Summer 2007

18S

Minority students and English language learners (ELLs) use the target skill and receive feedback as needed prior are at greater risk since many also live below the to independent practices. Instruction at the letter, poverty level - another factor that increases risk for aca- word, and connected text level was fast-paced to build demic failure. Combined these factors have con- automaticity. It also included both decoding and tributed to the disproportionate representation of encoding practice. Ensuring that students mastered minority students in special education. instructional elements provided them a foundation of In the case of ELLs, additional factors that must be reading knowledge and skills. Finally, interventions considered in making instructional and eligibility deci- had built-in review of all elements taught. sions are language of instruction and opportunity to Third, in interventions that interspersed language learn English as well as opportunity to learn in general. support activities to build oral language improvement Under No Child Left Behind (NCLB), there is now an intervention, students outperformed comparison stuemphasis on doing what is effective based on scientific dents on several reading measures (Gersten et aL, research and on the academic progress of every child, 2006). including those learning English. To ensure that ELLs Assessment and ELLs are making progress, students are assessed in reading There is also evidence that many of the basic reading and mathematics and are assessed annually to measure assessments and procedures for identifying ELLs at risk how well they are learning English (NCLB, 2002). for reading disabilities are valid and reliable. The most After several years of increased funding for research persuasive research has addressed measures that focus focusing on ELLs, the knowledge base on their instruc- on foundational literacy skills. These are the measures tional needs has grown exponentially. In particular, the most useful in identifying students early who will benfield has made strides in identifying effective instruc- efit from supplemental instruction associated with the tional and assessment practices (see Francis, Rivera, prevention of later reading difficulties. Measures of Lesaux, Kiefer, & Rivera, 2006). Therefore, we are now phonological awareness, alphabetic principle, and better poised to provide ELLs receiving English reading word reading are particularly useful. Identification in a instruction in the early grades instruction that is prevention model does not indicate that students have research-based. a learning difficulty but that they need additional instruction to develop early literacy skills (Francis et al., Research-Based Reading Instruction and ELLs Various intervention studies with ELLs have provided 2006; Gersten et al., 2006). findings showing that these students benefit from Prior research with ELLs has identified reading interinstruction that includes (a) the essential components ventions that are effective in improving reading outof reading, (b) features of effective instruction, and comes (Denton et al., 2004; Gunn et al., 2000; Gunn et (c) development of English language skills (Denton, al., 2002; Vaughn, Mathes et al., 2006). In these studAnthony, Parker, & Hasbrouck, 2004; Gunn, Biglan, ies, the effect of the intervention was determined by Smolkowski, & Ary, 2000; Gunn, Smolkowski, Biglan, comparing differences on outcome measures between & Black, 2002; Vaughn, Mathes et al., 2006). Effective students receiving a treatment and a comparison interventions are comprehensive when they appropri- group. Outperforming a comparison group provides ately address reading components based on students' relevant information about the effectiveness of the grade and learning needs: phonological awareness, overall intervention; however, questions regarding the phonics, fluency building, vocabulary and compre- number of students who continue to be at risk and prohension. cedures for determining the risk status of students are understudied and not specified for ELLs. Thus, the use In intervention studies that provided additional readof RTI as a prevention model and a data source for ing instruction to ELLs, students who received comprehensive instruction had better outcomes than students informing identification of ELLs requires further invesin comparison conditions in at least one area of read- tigation. ing. If, on the other hand, instruction focused on just Response to Intervention one element such as fluency building, there were no RTI as a preventive approach includes the use of studifferences with comparison students (Gersten et al., dents' learning rate and level of performance to make 2006), instructional decisions. Students exhibiting difficulty Second, instructional elements common to all the in learning to read are provided intervention and given interventions included explicit and systematic instruc- time to acquire reading skills before they are referred tion that incorporated elements of direct instruction. for special education. Under the recent reauthorization Additionally, instruction provided enough practice of IDEA, eligibility and identification criteria now items so that students had multiple opportunities to include the following: "The LEA may use a process that

Learning DisabiUty Quarterly

186

determines if a child responds to scientific, researchbased intervention as part of the evaluation" (IDEA, 2004). However, there is no guidance on how to measure RTI, and to date little research has addressed this issue with ELLs (Linan-Thompson, Vaughn, Prater, & Cirino, 2006). How Do We Determine RTI? Despite an examination of this question by various researchers (Fuchs, Fuchs, & Gompton, 2004; Fuchs, Mock, Morgan, & Young, 2003; McMasters, Fuchs, Fuchs, & Compton, 2005), currently, there is no consensus on how to determine response or which measures are most effective for discriminating responders from nonresponders. In the absence of consensus on this issue, performance level, most often measured with a benchmark score, and growth are frequently used. The benchmark is usually set by a norm group. In some cases these are minimal standards, meaning that if students can at least reach the benchmark, they will be at lower risk of experiencing reading difficulties. However, using benchmark set by a norming sample as criteria is particularly problematic in the case of ELLs since, with few exceptions, they are often not included in these samples. On neither performance nor growth is there convergence on the cut point to be used for determining responsiveness. A third approach, dual discrepancy, combines performance and growth and compares students to their peers (Fuch & Fuchs, 1998; McMasters et al., 2005; Speece & Case, 2001; Speece, Case, & Molloy, 2003). Although the cut-point question is resolved, there is no consensus on which measures yield the most accurate information regarding response to intervention. A previous study of RTI with ELLs used performance level to determine RTI (Linan-Thompson et aL, 2006). The cut point was a standard score of 85 on the Word Attack and Passage Comprehension subtests of the Woodcock-Johnson Reading Battery. Using these criteria, 97% of intervention students who responded at the end of first grade maintained their status as responders through second grade compared to students in our previous studies, in which as few as 66% of the students who received 20 weeks of intervention continued to "thrive" in the general education classroom without supplemental instruction (Vaughn, Linan-Thompson, & Hickman-Davis, 2003). Furthermore, results indicated that general classroom instruction was insufficient for students at risk for reading difficulties, since 91% of the students who received the English intervention met criteria at the end of first grade compared to 41% of the control students. These differences could be attributed to several factors. For example, the intervention provided was more

intensive both in terms of span over the year and length of each session. In addition, the use of standard scores on untimed measures rather than fluency measures may have contributed to the higher percentage of students meeting criteria, RTI and ELLs RTI offers a promising alternative for reducing disproportionate representation of culturally and linguistically diverse students in special education by identifying students at risk early and providing preventive instruction to accelerate progress. Furthermore, the emphasis on instruction and progress monitoring can potentially impact student outcomes for a population (i.e., ELL) that has historically not fared well on measures of student achievement. Yet, despite the generally positive response to RTI as a means for identifying students with learning disabilities, some researchers have questioned the validity of RTI with culturally and linguistically diverse groups (Klingner & Edwards, 2006). In particular, they emphasize the need to determine what works, for whom, and in what context. Although these questions are inherent in a multitiered RTI model, to date, with few exceptions, data have not been disaggregated by ethnicity, for students who are ELLs, or for students who have other disabilities even when these groups of students are included in the data set (Donovan & Cross, 2002). In other studies, they are excluded from participating, often because of their limited proficiency in English. Yet, there is a growing database to suggest that many of the interventions to improve early reading for monolingual English students are used in classrooms with ELLs and that when ELLs are included they respond similarly to non-ELL students (Shanahan & Beck, 2006). As increasing numbers of schools implement RTI practices based on recommendations in the recently authorized legislation (IDEA; December, 2004), questions about how to accurately identify EL learners in need of special education require data-based answers. To minimize the possibility of continuing to inappropriately identify EL learners, more research is needed on the use of RTI with EL learners. This study examined the RTI of EL learners identified as at risk for reading problems in the fall of first grade and provided extensive and systematic intervention for seven months. To identify a viable means for determining RTI, we examined the relative number of EL learners at risk for reading difficulties who met RTI criteria after an intensive seven-month intervention using three different approaches: benchmark, growth, and discrepancy. Rather than focusing on students who did not respond to the intervention, the goal was to identify students who had responded well and were, therefore.


187

no longer at risk for later reading difficulties. The data were part of a larger intervention study, the results of which were provided previously (Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006).

Students in the treatment condition received daily 50minute interventions in small groups hy teachers trained hy the research team only during first grade. This instruction was provided in addition to the core reading instruction. The core curriculum selected hy the METHODS schools for reading during the classroom reading Participants instruction for one site was Language Enrichment Selection criteria for the original sample. Students (Carreker, 1999) and in the second, McGraw Hill {N = 142) were selected from two sites and four effective Reading (2001), hilingual schools in Texas. Schools were considered The school selected the language of instruction for effective for hilingual students hecause their state-level reading as English; thus, the intervention was in English reading achievement tests at third grade indicated that to align with the school's decision. Additional details of 80% or more of students passed the test. To ensure that sites, schools, teachers, and the selection process for the poor performance in English was not due solely to a lack samples screened for inclusion, as well as findings from of exposure to English language or print material, the intervention studies, are availahle in previous researchers screened students in hoth English and reports (Vaughn, Cirino et al., 2006; Vaughn, Mathes et Spanish. Two tests in hoth Spanish and English for a al., 2006). total of four measures were used: (a) The Letter Word Findings for original sample. At the end of first Identification (LWID) suhtest from the Woodcock grade, across two cohorts, the intervention group outLanguage Proficiency Battery (English and Spanish, performed the comparison group on measures of described helow) and (h) the first five words from a word phonological awareness, word reading fluency, reading reading list (English and Spanish) used to assess initial comprehension, and spelling (Vaughn, Cirino et al., word reading ahility. Students who scored helow the 2006; Vaughn, Mathes et al., 2006). Longitudinal find25th percentile for the first grade on the Letter Word ings from the English interventions revealed significant Identification subtest in hoth Spanish and English, and differences in favor of treatment students in oral lanwho were unable to read more than one word from the guage, decoding, spelling, fluency and, comprehension simple word list were eligihle for the study and were ran- (median d = +0,41) at the end of second grade (Cirino et domly assigned to treatment or comparison conditions. al,, in review).

Table 1 Means and Standard Deviations by Group

Intervention Time 1 M SD

Time 2 M SD

Comparison N = 40 Time 3 M SD

Time 1 M SD

Time 2 M SD

Time 3 M SD

OL

53,7

21,5

63,4

18,7

69,4

17,8

59,0

18,5

62,3

17,5

65,9

15,2

WA

85,8

8,8

100

13,6

100

19,5

83,6

6,0

93,5

11,4

93,5

12,7

PC

80,8

10,4

92,7

13,4

91,3

17,2

81,4

11,8

86,4

11,9

87,4

13,0

,6

1,9

20,2

16,4

53,3

33,5

1,7

13,1

13,0

40,1

29,3

ORF

,73

Note. Time 1 = Begin of second grade. OL = Orai Language composite; WA = Word Attack subtest of WL PB-R; PC = Passage Comprehension subtests of WLPB-R; ORF = DiBELS Orai Reading Fiuency,

Learning Disability Quarterly

188

Current participants. One hundred and eleven of the original 142 students (78%) were availahle at followup (end of grade 2) for further assessment (see Cirino et al., in review). The present study focused on students for whom all RTI criteria data were availahle at followup (see RTI Criteria in next section). In all, 81 students met criteria for this study (41 treatment, 40 comparison). All students were hilingual (Spanish/English), with an average age of 8.16 (0.4 SD) at the grade 2 followup point. The sample was 44% female. At the end of grade 1, English and Spanish oracy (from the Woodcock Language Proficiency BatteryRevised, see below) were 63.41 (18.8 SD) and 64.66 (20.4 SD) for treatment students, and 62.35 (17.6 SD) and 68.18 (28.8 SD) for comparison students, respectively. Means and standard deviations at each time point (heginning of first grade, end of first grade, and end of second grade) for the three measures used for RTI criteria are provided in Table 1 as well as the means and standard deviations for oral language. Measures Woodcock Language Proficiency Battery-Revised (WLPB-R; Woodcock, 1991). The WLPB-R is a well-standardized instrument with internal consistency values for the suhtests utilized at the age range of the current study's participants ranging from .77 to .95 (median r = .89). The WLPB-R was used to assess reading decoding and comprehension outcomes. Test development, scaling, and the norming process for the assessment are descrihed in detail in the WLPB-R manuals (Woodcock, 1991). The Word Attack and Passage Comprehension suhtests were used, and both standard and W (a Raschscaled ability score) scores for these suhtests were utilized, depending on the RTI criteria. In addition, an Oral Language composite was computed, based on the Picture Vocabulary, Verbal Analogies, and Listening Comprehension suhtests of the WLPB-R (reported above). These measures were administered at the beginning and end of the intervention year, and at the end of the followup year. Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002)/Indicadores Dinamicos del Exito en la Lectura (Good, Bank & Watson, 2003). The DIBELS suhtest utilized was Oral Reading Fluency (ORF). To assess oral reading fluency for connected text, students were administered four measures of reading fluency requiring them to orally read a passage. Reliability was determined by correlating scores from two stories administered to a subset of the larger parent sample {N = 766) with data on these measures for students from Texas at the end of grade 1. Values were quite high, r = 0.95, for the English versions. The ORF test was timed, allowing students a

maximum of 3 seconds per word and 60 seconds for each passage. Students were administered one story at the heginning of the intervention year, two at the end of the intervention year, and four at the end of the followup year, in English. The number of passages was increased as the students' reading skills improved. Particularly at the heginning of first grade, the students were unahle to read, so administering more passages would not have been appropriate. At the end of second grade, students were administered two grade 1 and two grade 2 beginning-of-year passages to determine growth on hoth grade-level text (second grade) and on text with level held constant over time (first grade). The dependent measures were the average of the number of words read correctly within the time limit across all stories administered at the end off each year. RTI Criteria We were interested in the response to intervention of the target students in first grade. These students were at risk for reading difficulties and received a supplemental reading intervention for 50 minutes a day, 5 days per week, during most of their first-grade year. The RTI of this ELL sample was examined at the end of first grade and the end of second grade. Four different criteria were evaluated to define "responsiveness" to intervention, each in comparison to students who did not receive intervention. The henchmarks used were set benchmarks from the normative sample of the WLPB-R, or expectations for the DIBELS ORF passages. These henchmarks correspond to "placement level models" (Foorman, Francis, Fletcher, Schatschneider, & Metha, 1998; O'Connor, 2000; Torgesen et al., 1999; Vellutino et al., 1996). Discrepancy benchmarks and slopes, on the other hand, were established relative to a sample of students from the same schools as those in the intervention studies (hut who were not identified as struggling readers and therefore not a part of the intervention studies). These criteria correspond to a "local norms approach" (Fuchs & Fuchs, 1998; McMaster et al., 2005; Speece & Case, 2001; Speece et al., 2003). The longitudinal sample consisted of approximately 430 grade 1 students in immersion programs in four sites in Texas and California from 16 schools. Benchmarks were highly similar when obtained from a smaller sample (approximately 100 students) of only the four specific schools from which intervention students were selected. Therefore, the larger sample from all 16 schools was utilized. The reason for utilizing set henchmarks as well as benchmarks and slopes based on peers was that set henchmarks have not previously been used to determine adequate progress in ELLs in early


189

elementary school, and the discrepancy benchmarks were derived from a large sample size consistent with the intervention study students' peer group. A preliminary examination of the methods was conducted using only a percentage of students who met criteria. In this examination, the percentage of students identified as responders were compared with and without fluency and using benchmark, growth, and dual discrepancy to extend the work that we had done previously. When we examined the students who were identified as responders without and with the fluency benchmark as part of the criteria, the percentage who met the response criteria was 40% to 7%, respectively. Considering the importance of fluency in identifying students with reading difficulties and with the goal of not over-identifying students as responders who were not, we included fluency as part of the criteria. We also chose to use the more stringent criteria of standard scores of > 95 on both Word Attack and Passage Comprehension rather than the lower criteria we had used in the past of a standard score of > 85 on both Word Attack and Passage Comprehension. In our previous study, the less stringent criteria identified 91% of the students as responders compared to the 64% who were identified with the more stringent criteria of standard scores of > 95 on both measures. Therefore, for this study, the first of three grade 1 RTI criteria, set benchmark, was a WLPB-R standard score at or above 95 (37th %ile) on both the Word Attack and Passage Comprehension subtests in English and a raw score of at least 40 CWPM on an appropriate DIBELS ORE passage at the end of the intervention year (grade 1). The second criterion, discrepancy benchmark criteria, was a W score at or above the mean of the longitudinal sample at the end of grade 1 on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and a raw score for words correctly read per minute on an appropriate DIBELS ORE passage that was at or above the mean of the longitudinal sample. The actual criterion values for Word Attack and Passage Comprehension were W = 481 and W = 459, respectively. The actual CWPM criterion for the ORE was 52 words. The third criterion, discrepancy slope criteria, was a W score gain at or above the mean gain of the longitudinal sample during grade 1 on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and the student obtained a raw score gain for words correctly read per minute during grade 1 on an appropriate DIBELS ORE passage at or above the mean gain of the longitudinal sample. The actual criterion W score increases for Word Attack and Passage Comprehension were 15 and 28, respectively. The actual CWPM

gain criteria for the ORE were 27. In each case, students who met these criteria were considered to have responded to the intervention. These three criteria were compared to a final outcome criterion in grade 2, which was essentially the same as the set benchmark criteria from grade 1: a standard score at or above 95 (37th percentile) on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and a raw score of at least 70 words correctly read in one minute on an appropriate DIBELS ORE passage at the end of the followup year. If a student received a score below these levels on any of the three measures at the end of grade 2, this final outcome criterion was not considered to have been met. In the remainder of this article, the three grade 1 RTI criteria are referred to as Set Benchmark 1, Discrepancy Benchmark 1, and Discrepancy Slope 1, and the grade 2 RTI outcome criteria are referred to as Set Benchmark 2. Analyses Each of the grade 1 criteria were first evaluated against the grade 2 criteria through simple frequency tables. Eor example, in comparing Set Benchmark 1 to Set Benchmark 2, the proportion of intervention students who met or did not meet criteria in both grades, only in grade 1, and only in grade 2 can be derived. In addition, struggling readers and their controls can be compared. Next, logistic analyses were run with Set Benchmark 2 as the dependent variable, and Set Benchmark 1, Discrepancy Benchmark 1, and Discrepancy Slope 1 used as predictors, respectively. Models were also considered that included intervention status, or oral language proficiency in grade 1, as a predictor for each of the criteria. Erom these analyses, various measures of effectiveness of the RTI criteria were obtained (sensitivity, specificity, odds ratio, d, etc.). RESULTS Table 2 presents the results of the frequency analyses for each of the grade 1 criteria against the grade 2 set benchmark criteria by group (intervention or comparison). These results indicate that 88% of comparison students and 75% to 80% of intervention students did not pass any of our stringent grade 1 or grade 2 criteria. Thirteen students (16%, 8 intervention and 5 comparison) met the grade 2 criterion, and varying numbers of students met the various grade 1 criteria. Only five students (4 intervention and 1 comparison) met the Set Benchmark 1 criteria, and only one intervention student met the Discrepancy Benchmark 1 criteria. Given the low number of students who met the grade 1 benchmark criteria, and the fact that a higher number of students (9 and 12) met the grade 2 criteria in spite of not meeting the grade 1 criteria, further analyses were not conducted on the two benchmark criteria. However, 11


190

Table 2 Grade 1 Criteria v. Set Benchmark 2 (Grade 2) Criteria Intervention Students SB 1 V. SB 2

DB 1 V. SB 2

DS 1 V. SB 2

Met grade 1 but not grade 2 criteria

1 (2.4%)

0 (0%)

2 (4.9%)


6 (14.6%)

7 (17.1%)

2 (4.9%)

Met both grade 1 and grade 2 criteria

2 (4.9%)

1 (2.4%)

6 (14.6%)

32 (78.1%)

33 (80.5%)

31 (75.6%)

DB 1 V. SB 2

DS 1 V. SB 2

Met neither grade 1 nor grade 2 criteria

Comparison Students N=4l SB 1 V. SB 2

0 (0%)

(0%)

0


3 (7.5%)

5 (12.5%)

2 (5.0%)

Met both grade 1 and grade 2 criteria

2 (4.9%)

0 (0%)

3 (7.5%)

35 (87.5%)

35 (87.5%)

35 (87.5%)

Met neither grade 1 nor grade 2 criteria

(0%)

0


Note. SB2 = Set Benchmark 2 (standard score of 95 on both Word Attack and Passage Comprehension subtests of WLPB-R and reading rate of 70 CWPM on DIBELS ORF at the end of Grade 2). SBI = Set Benchmark 1 (standard score of 95 on Word Attack and Passage Comprehension and a reading rate of 40 CWPM on DIBELS ORF at the end of Grade 1). DBl = Discrepancy Benchmark 1 (Word Attack, Passage Comprehension, and DIBELS ORF levels at or above the mean of EL peers receiving English instruction). DB2 = Discrepancy Slope 2 (students' growth during grade 1 on the Word Attack, Passage Comprehension, and DIBELS ORF subtests are at or above the mean growth of EL peers receiving English instruction).

Students (8 intervention and 3 comparison) met the Discrepancy Slope 1 criteria; these results were further explored via logistic regression. When Discrepancy Slope 1 criteria were evaluated against the grade 2 criteria, Discrepancy Slope 1 was a significant predictor, x^(l), 21.17, p < .0001, which generated a point estimate (odds ratio) of 74.2, which indicates that those who passed this grade 1 criterion were 74 times more likely to pass the grade 2 criteria than if they did not. This odds ratio corresponds to the more typical Cohen d value of +2.38, which is a very large effect. Overall classification was 93%, with a specificity of .97. Thus, passing these grade 1 criteria is a clear indi-

cator of success at the end of grade 2 (9 of 11, positive predictive value of .82); further, not meeting these criteria is a clear indicator of also not meeting grade 2 criteria (66 of 70, negative predictive value of .94). These values are interpreted within the context of relatively few students meeting either set of criteria, such that a model predicting all failures would result in an overall classification rate of 84%. Sensitivity was .69, indicating that four of the 13 students who met grade 2 criteria did so without meeting Discrepancy Slope 1 criteria. Additional models were evaluated in which the Discrepancy Slope 1 was evaluated in the context of treatment group and oral language. In these cases, there


191

were no interactions of these predictors with Discrepancy Slope 1, and in each case, only the Discrepancy Slope 1 variable had a significant effect.

criteria, in other ELL students who present with similar characteristics (i.e., significantly at risk) at the beginning of grade 1.

DISCUSSION Despite increased attention and knowledge about effective instructional practices related to early reading (McCardle & Chhabra, 2004), there is still much we do not know about the reading development of ELLs and the use of RTI with this population (August & Shanahan, 2006). For example, we have effective universal screening measures to facilitate early identification of students at risk for reading problems, but we do not have an accurate means for determining the level of RTI needed to prevent further difficulties or to warrant more extensive interventions or services (e.g., special education). While practices for using RTI to determine student progress and to assist in decision making about further interventions is an issue that is still being examined for all students, the situation is more complex when language learning is also considered. This study provided an initial examination of various commonly used assessment practices of RTI with ELLs and their implications with respect to the number of students who respond to intervention and their outcomes a year later. Since we are in the preliminary stages of understanding issues related to RTI and ELLs, the results of this study raise more questions than they provide answers. How Should We Determine Response to Intervention? Of the three approaches examined here, only the Discrepancy Slope criteria at grade 1 predicted students who would meet the grade 2 criteria. This approach assessed whether growing at the same rate as typically achieving peers on measures of non-word reading, oral reading fluency, and reading comprehension resulted in a greater likelihood of meeting accepted standards in these areas one year later. It clearly did, albeit in the context of relatively few students meeting any of the criteria. In contrast, almost none of the at-risk students were able to meet either the benchmarks set by norming samples or those of their typically achieving peers. Although not the focus of the current study, it should be noted that the benchmark for oral reading fluency of the longitudinal sample was higher, 52 CWPM, than that of the norming sample, 40 CWPM. Thus, in a large sample, ELLs not at risk who were receiving reading instruction in English were able to perform as well as English monolingual students on measures of oral reading. Although discrepancy slope criteria best predicted the grade 2 criteria, these results clearly need replication, both in terms of these particular grade 1 and grade 2 RTI criteria and alternative RTI

Why Do So Few Students Meet Any of the Grade 1 or Grade 2 Criteria? The answer to this question could be that the criteria we used were too stringent in terms of performance level as well as the number of measures. However, the performance-level criterion used for the non-word reading and comprehension subtest, the 37th %ile, was in the middle range compared to the performance level used in other studies (Al Otaiba & Fuchs, 2005). While we know from our past research (Linan-Thompson et al., 2006) that adding a fluency measure to the criteria reduces the percentage of students who meet the criteria, the benchmark of 40 CWPM is a commonly accepted end-of-first grade benchmark even for ELLs (Baker, 2003). The criteria may have been too stringent in that students needed to meet the target performance levels on every measure (i.e., rather than any single measure or any two measures). The current study emphasized meeting standards in all areas since the goal of instruction and intervention includes success in each of the areas of decoding, fluency, and comprehension. Clearly, the choice of performance level and measures affected the results in that in an earlier study (LinanThompson et al., 2006), response rates were much higher, even though some of the same participants were included. Instead of a performance level of a standard score of 95, as in the current study, Linan-Thompson et al. considered a student to have met criteria with a standard score of 85. In addition, they examined only decoding and comprehension performance as opposed to also including fluency. Another factor could be that these students started out too low and, therefore, despite an intensive intervention, were not able to reach the benchmark. Of particular note are the low scores in language skills in English and Spanish, with participants scoring, on average, two standard deviations below the norm. These students may be a group of low oracy students, and their results may not generalize to bilingual students participating in interventions with a higher level of oracy skills. The focus should be on continuing to develop interventions that will improve outcomes for ELLs so that they can meet the criteria regardless of their starting point. It is reasonable to think that even with intensive interventions provided during first grade, students may require additional interventions in subsequent grades to compensate for their overall very low start and low levels of English language. Students may have difficulty passing the grade 2 criterion, in part, for the some of the same reasons that


192

caused them to have difficulty meeting the grade 1 criteria: the stringency of the criteria and the low language and literacy start points prior to treatment. A recent study hy Chiappe and Siegel (2006) demonstrated that ELLs may acquire foundational literacy skills at the same rate as non-ELLs. However, they did not disaggregate the results for the lowest performing students so it may he that these students need more time. Regardless of their start point, it could he that in general ELLs need more time to solidify the skills they acquired in first grade and/or extend their English knowledge through instruction and experience. Once they have mastered basic reading skills in first grade, they have the opportunity to practice their literacy skills in second grade. Additionally, if they are reading, they have more opportunity to encounter new words and build their vocahulary and oracy. Further studies with ELLs are needed to evaluate the effectiveness at potentially earlier start points and with longer follow-up periods. It may he that students receiving intensive interventions early enough would demonstrate hoth improved grade 2 RTI response rates and continued improvement in later grades. Why Don't Intervention and Comparison Students Differ in Their Grade 1 Response Rates Despite the General Effectiveness of the Intervention? We know from previous reports of the data that intervention students outperformed comparison students immediately after intervention, as well as one year after intervention (Cirino et al., submitted; Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006). This occurred in a context in which it is not easy to do so, given that the participating schools were doing well according to state accountability measures. Therefore, one could expect that all students were receiving generally effective reading instruction from their classroom teacher. Though many of the comparison students also received supplemental reading instruction, the inclusion of actual supplemental instruction as a predictor did not change the fact that intervention students outperformed comparison students (Vaughn, Cirino et al., 2006). It is important to consider that when it comes to RTI, the focus is on individual students meeting criteria and ensuring that they will continue to profit from general education reading instruction without additional intervention or referral for special education. This contrasts with studies that examine the effectiveness of the intervention overall, in which the performance of the treatment group is compared with the control providing evidence ahout what works hut not evidence of whether it resolves all further difficulties in reading for

a given student. This is a critical distinction when decisions are made ahout identification and eligihility for special education. Euture research might take a closer look at individual students and specific variables that might impact learning outcomes and rate of response to intervention. Eor example, exposure to English print outside of school or the extent to which students have opportunities to practice using English may impact their response to the intervention they are receiving at school. If a Yearlong, Intensive Intervention Is Not Enough to Ensure Response to Intervention, Then What Is? Notwithstanding the issues surrounding which criteria to use and which levels on which measures to define response, it is clear that hased on relatively stringent criteria the rate of students' responsiveness in this study was low. An obvious answer to what might improve this responsiveness could be simply more years, but a more precise answer might he more time with high-quality instruction and opportunities to extend literacy and oracy skills. Students need more time acquiring basic foundational skills and practicing them. They need more time listening to, reading, speaking, and writing in English. They need more time developing academic language and higher order thinking skills. However, more time does not necessarily mean more years. It might he a longer instructional day, or grouping and instructional practices that provide students more opportunities to actively use English, or it could he an extended school year. CONCLUSIONS AND FURTHER QUESTIONS A final decision on the criteria for determining whether ELLs' performance is sufficient for them to profit adequately from classroom instruction alone and/or to determine whether students would qualify for special education services cannot yet be made. In order to arrive at an accepted decision that resolves many of the prevailing questions about the application of RTI, further research is required. Many ELLs will profit from additional intervention, and for some it will serve as a protective factor, reducing the likelihood that they will require further intervention in the future. Eor others, the immediate henefit from instruction will he significant though inadequate to reduce the gap hetween their reading performance and grade-level expectations. Eor still others, when (what grade level) and how intensive (intervention for how long and under what conditions) is far less apparent. We are aware that the findings of this study are limited to the approaches to response to intervention we selected and the criteria we established. While we considered the most prevailing practices for identifying


193

response to intervention to test, there are other ways to conceptualize and implement RTI that we did not address. Furthermore, we recognize that if we adjusted the criteria for RTI by lowering them, considerably more students would be classified as responders. Criteria for determining whether ELLs' performance is sufficient to profit adequately from classroom instruction alone and/or to determine whether students would qualify for special education services require additional research.

Gersten, R., Baker, S. K., Shanahan, T., Linan-Thompson, S., Chiappe, P., & Scarcella, R. (2006). Effective literacy and language instruction for English learners in the elementary grades. An IES practice guide. Washington, DC: IES, Department of Education. Good, R. H., & Kaminski, R. A. (2002). Dynamic indicators of basic early literacy skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Good, R. H., Bank, N., & Watson, J. M. (Eds.), (2003). Indicadores dindmicos del exito en la lectura. Eugene, OR: Institute for the Development of Educational Achievement. Gunn, B., Biglan, A., Smolkowski, K., & Ary, D. (2000). The effiREFERENCES cacy of supplemental instruction in decoding skills for Hispanic August, D., & Shanahan, T. (Eds.). (2006). Developing literacy in second-language learners: Report of the national literacy panel on lan- and Non-Hispanic students in early elementary school. The Journal of Special Education, 34(2), 90-103. guage-minority children and youth. Mahwah, NJ: Lawrence Gunn, B., Smolkowski, K., Biglan, A., & Black, C. (2002). Eribaum Associates. Baker, S. (2003). Enhancing core reading instruction for English lan- Supplemental instruction in decoding skills for Hispanic and non-Hispanic students in early elementary school: A follow-up. guage learners in kindergarten and Grade 1. Presented at the Journal of Special Education, 36(2), 69-79. Institute on Beginning Reading II, Salem, Oregon. Individuals with Disabilities Education Improvement Act of 2004 P L Carreker, S. (1999). Language enrichment. Bellaire, TX: Neuhaus 108-446, 20 U.S.C. §1400et5e^. Education Center. Chiappe, P., & Siegel, L. S. (2006). A longitudinal study of reading Klingner, J., & Edwards, (2006). Cultural considerations with Response to Intervention models. Reading Research Quarterlv development of Canadian children from diverse linguistic 41(1), 108-117. backgrounds. The Elementary School Journal, 107{2), 135-153. Linan-Thompson, S., Vaughn, S., Prater, K., & Cirino, P. T. (2006). Cirino, P. T., Vaughn, S., Linan-Thompson, S., Cardenas-Hagan, The response to intervention of English language learners atE., Fletcher, J. M., & Francis, D. J. (submitted). One-year followup outcomes of Spanish and English interventions for English lanrisk for reading problems. Journal of Learning Disabilities 39(5) guage learners at-risk for reading problems. 390-398. McCardle, P., & Chhabra, V. (Eds.). (2004). The voice of evidence in Denton, C. A., Anthony, J. L., Parker, R., & Hasbrouck, J. (2004). reading research. Baltimore: P. H. Brookes Pub. Effects of two tutoring programs on the English reading development of Spanish-English bilingual students. The Elementary McGraw Hill Reading. (2001). New York: McGraw Hill School Division. School Journal, 104{4), 289-305. Donovan, M. S., & Cross, C. T. (2002). Minority students in special McMaster, K. L., Fuchs, D., Fuchs, L.S., & Compton, D. L. (2005). Responding to nonresponders: An experimental field of trial of and gifted education. Washington, DC: National Academy Press. identification and intervention methods. Exceptional Children Fletcher, J. M., Reid, L. G., Barnes, M., Stuebing, K. K., Francis, D. 71(4), 445-463. J., Olson, R. K., Shaywitz, S. E., & Shaywitz, B. A. (2002). In R. Bradley, L. Danielson, & D. Hallahan (Eds.), Identification of No Child Left Behind Act of 2001, Pub. L. 107-110, 20 U.S.C. § 6301 learning disabilities: Research to practice (pp. 185-239). Mahwah, et seq. (2002)(enacted). Retrieved November 26, 2006, from NJ: Lawrence Eribaum. http://www.ed.gov/policy/elsec/leg/esea02/107-110.pdf Fletcher, J. M., Shaywitz, S. E., Shankweiler, D. P., Katz, L., O'Connor, R. E. (2000). Increasing the intensity of intervention in Liberman, L Y., Steubing, K. K., et al. (1994). Cognitive profiles kindergarten and first grade. Learning Disabilities Research and of reading disability: Comparisons of discrepancy and low Practice, 15{l), 43-54. achievement profiles. Journal of Educational Psychology, 86(1), 6- President's Commission on Excellence in Special Education. (2002). A new era: Revitalizing special education for children and Foorman, B. R., Francis, D. J., Fletcher, J. M., Mehta, P., & their families. Washington, DC: Author. Schatschneider, C. (1998). The role of instruction in learning to Shanahan, T., & Beck, 1. (2006). Effective literacy teaching for read: Preventing reading failure in at-risk children. Journal of English language learners. In D. August & T. Shanahan (Eds.), Educational Pscyhology, 90(1), 37-55. Developing literacy in second-language learners: Report of the Francis, D. J., Rivera, M., Lesaux, N., Kiefer, M., & Rivera, H. national literacy panel on language-minority children and youth (2006). Practical guidelines for the education of English language (pp. 415-488). Mahwah, NJ: Lawrence Eribaum Associates. learners: Research-based recommendations for instruction and aca-Speece, D. L., & Case, L. P. (2001). Classification in context: An demic interventions. Portsmouth, NH: RMC Research alternative approach to identifying early reading disability. Corporation, Center on Instruction. Journal of Educational Psychology, 93(4), 735-749. Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying Speece, D , Case, L. P., & Molloy, D. E. (2003). Responsiveness concept for reconceptualizing the identification of learning disto general education instruction as the first gate to learning abilities. Learning Disabilities Research and Practice 13{i) 204disabilities identification. Learning Disabilities Research and 219. Practice, 18(3), 147-156. Fuchs, D., Fuchs, L. S., & Compton, D. L. (2004). Identifying readStanovich, K. E., & Siegel, L. S. (1994). Phenotypic performance ing disabilities by responsiveness-to-instruction: Specifying profile of children with reading disabilities: A regression-based measures and criteria. Learning Disability Quarterly, 27(4), 216. test of the phonological-core variable-difference model. Journal Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). of Educational Psychology, 86(1), 24-53. Responsiveness to intervention: Definitions, evidence, and Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A., implications for the learning disabilities construct. Learning Voeller, K. K., & Conway, T. (2001). Intensive remedial instrucDisabilities Research & Practice, 18{3), 157-171. tion for children with severe reading disabilities: Immediate

Learning DisabiUty Quarterly

194

and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34(1), 33-58. Torgesen, J. K., Wagner, R. K., Rasbotte, C. K., Rose, E., Lindamood, P., Conway, T., & Garvan, C. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual response to instruction. Journal of Educational Psychology, 91, 579-593. Vaughn, S., Cirino, P. T., Linan-Thompson, S., Mathes, P. G., Carlson, C. D., Cardenas-Hagan, E., et al. (2006). Effectiveness of a Spanish intervention and an English intervention for English language learners at risk for reading problems. American Educational Research foumal, 43, 449-488. Vaughn, S., Linan-Thompson, S., & Hickman-Davis, P. (2003). Response to instruction as a means of identifying students with reading/learning disabilities. Exceptional Children, 69(4), 391410. Vaughn, S., Mathes, P., Linan-Thompson, S., Cirino, P., Carlson, C, Pollard-Durodola, S., Cardenas-Hagan, E., & Francis, D.J. (2006). Effectiveness of an English intervention for first-grade English language learners at risk for reading problems. The Elementary School Journal, 107, i53-180. Vellutino, F. R., Scanlon, D. M., & Lyon, J. (2000). Differentiating between difficult-to-remediate and readily remediated poor readers: More evidence against the IQ-achievement discrepancy definition of reading disability. Journal of Learning Disabilities, 33(3), 223-238.

Vellutino, F. R., Scanlon, D. M., Sipay, E. R., Small, S. G., Pratt, A., Chen, R., & Denckia, M. B. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88(4), 601-638. Woodcock, R. W. (1991). Woodcock Language Proficiency BatteryRevised. Chicago: Riverside. Woodcock, R. W., & Johnson, M. B. (1989). Woodcock-Johnson Psycho-Educational Battery-Revised. Chicago: Riverside.

NOTE This research was supported in part by grants POl HD39521 and R305U010001, both jointly funded by the National Institute for Child Health and Human Development and the Institute of Education Sciences. The attitudes and opinions expressed in this manuscript are those of the authors and do not necessarily reflect those of the funding agencies. The authors wish to thank their many collaborators, coworkers and students, parents, teachers, and school and district officials who made this research possible.

Please send correspondence to: Sylvia Linan-Thompson, The University of Texas, College of Education, 1 University Station, SZB D53OO, Austin, TX 78712; [email protected]


195