Using Language Sampling to Measure Language ...

Unless otherwise noted, the publisher, which is the American Speech-LanguageHearing Association (ASHA), holds the copyright on all materials published in Perspectives on Language Learning and Education, both as a compilation and as individual articles. Please see Rights and Permissions for terms and conditions of use of Perspectives content: http://journals.asha.org/perspectives/terms.dtl

Using Language Sampling to Measure Language Growth Raúl Rojas and Aquiles Iglesias Temple University Philadelphia, PA

Abstract This article illustrates how speech-language pathologists (SLPs) can use language sampling and growth curve modeling (GCM) to examine the language growth rates of English Language Learners. GCM data on language samples provides SLPs with powerful, new tools to evaluate actual progress over time instead of relying on single, static measurement endpoints to determine typical development. Alejandro and Luis, children from Spanish-speaking households, passed the school administered English as a second language (ESL) proficiency screening during the fall semester of second grade and were placed in a regular English-only classroom. After being in the regular classroom for a couple of months, the teacher became concerned that the two children were performing below their classroom peers and recommended a speech-language evaluation. At this juncture, the clinician is faced with a range of options: 1. Conduct the evaluation in Spanish, the children’s presumable native and more proficient language, to rule out a speech-language disorder in the native language 2. Evaluate in English, the language of academic instruction, comparing Alejandro and Luis to monolingual English speaking children 3. Evaluate in English, but compare the second language skills of each child to the English of other bilingual children It is important to note that these clinical options are not mutually exclusive. The speech-language pathologist (SLP) opted to administer the evaluation in English (the language of instruction and the language spoken by the SLP) and to compare the children’s results to those of monolingual English speakers. Formal testing results indicated that both children were performing below expected levels. Thw scenario described above is common: children are placed in regular English classes based on brief ESL screenings; they struggle to keep up with their peers; they are referred for a speech-language assessment; they are found to perform poorly on standardized English tests; and they become eligible for speech-language services. Regrettably, many of these children are misdiagnosed and should not be receiving speech-language services. Was the ESL screening a valid measure of language proficiency in English? Did the speech-language evaluation in English provide a representative picture of Alejandro and Luis’ language skills? What can monolingual SLPs (or bilingual SLPs who do not speak the children’s native language) do to ensure they are providing services to children who truly have language disorders? Resolution of the first two issues will requires, as discussed below, long-term changes in district policy and practice.

24

Since, in many districts, monolingual SLPs have limited access to personnel who can conduct native language assessment, information needed to make an appropriate diagnosis is often lacking. In the absence of information on both languages, SLPs must begin to utilize alternative assessment approaches that can supplement traditional protocols. The purpose of this article is to illustrate one of such approaches. The alternative approach we are recommending is to use language sampling to assess language growth; determining whether the children’s language skills are increasing at the same rate as typically developing children who are in the process of acquiring a second language. We will first address the two fundamental issues that resulted in the children being placed on the SLPs caseload. We will then provide a brief example of how SLPs could use information on language growth rates to assist them in their management of English Language Learners (ELLs). Some background information needs to be provided in order to address the issue of whether ESL proficiency screenings validly measure English proficiency. Every state in the U.S. identifies children as ELL using a home language survey and an ESL screening assessment, except for South Dakota, which does not require a home language survey (Swanson, 2009). Once children have been identified as ELL, they are assigned to a particular type of Englishlanguage instructional program with or without ESL support, depending on their level of English proficiency. Reliance on ELL screening assessment (instead of in-depth testing) to categorize children as ELL by the majority of school districts (90.1%) is problematic (Hopstock & Stephenson, 2003). Researchers (Abedi, 2006; Abedi, Hofstetter, & Lord, 2004) have questioned the limited content, construct, and concurrent validity of English proficiency screening assessments. These ELL screeners categorize children, but there is little or no relationship between the results of the screeners and the language skills needed to adequately function in an English classroom. Therefore, ESL screenings, in general, are questionable and should not be relied on as the sole measure of proficiency in a language, especially the language proficiency or skills needed to function in a classroom. Assessment in only one language, whether it is the first or the second language learned by a bilingual child, provides only partial information needed to make an appropriate diagnosis. Best practices literature (Bedore & Peña, 2008; Goldstein, 2004; Kohnert, 2008) and Federal Law (U.S. Department of Education, 2009) recommend testing in the native language. However, if the child is assessed in one language (whether the native or second language) and results indicate typical development, then no further testing is required (Iglesias & GutiérrezClellen, 1998). If the child is assessed in one language and results indicate below average performance, one cannot conclude the child has language impairment without testing in the other language. Best practices and reality, however, are not necessarily aligned, given that most SLPs who work with ELLs are not proficient in languages other than English, and they often assess bilingual children using standardized tests in English (Caesar & Kohler, 2007; Roseberry-McKibbin, Brice, & O’Hanlon, 2005). Speech-language evaluations in English (in the absence of native language assessment) do not provide a representative picture of bilingual children’s language skills. We do not condone or support not testing in the native language when necessary. However, we are mindful of the misalignment between best practices and reality. Resolution of the prevalent practices of only assessing ELL proficiency by using screening tools and testing in the majority language will require policy and practice changes at the district and school level. Districts should require SLPs to test ELL children using a combination of formal assessment tools and alternative assessment practices, such as language sampling. In addition, districts should ensure that the placement criteria for ELL students, as well as the tools to assess whether the children meet the criteria, validly reflect the skills needed to function adequately in specific instructional programs (e.g., English immersion, transitional bilingual) along with ESL support when required. Until these two

25

issues are resolved, many SLPs will continue to make clinical decisions based on inappropriate and insufficient information. This does not need to be the case. Over time, typically developing children will demonstrate a language growth rate that is equal to or greater than that of the average language learner. If we were able to demonstrate that Alejandro and Luis learn language at the same rate as their ELL peers, then we have provided evidence indicating that they are typically developing children in the process of acquiring a second language. The approach of using language sample data to examining language growth rate is the focus of this paper.

Alejandro and Luis: Change Over Time Alejandro and Luis were diagnosed as having a moderate receptive and a moderate-tosevere expressive language disorder in English, even though both passed their ESL screening. The results of formal testing in English indicated several areas of deficit, but the limited English vocabulary of both children was of particular concern. Therefore, a critical intervention goal for the children was to expand their vocabulary, particularly to increase their lexical diversity (i.e., number of different root words). Language sampling, amenable to measuring a wide range of expressive language skills (e.g., mean length of utterance, verbal fluency, overall intelligibility) and difficulties (e.g., repetitions, revisions, grammatical omissions), is an excellent alternative to track the growth of lexical diversity over time. Although language sampling was not conducted as part of the speech-language evaluation, the SLP can administer it in English during initial treatment to establish baseline oral language measures, including number of different words (NDW)—a measure of lexical diversity (Paul, 2006). Having done this, while following recommended language sampling practices for bilingual children (Rojas & Iglesias, 2009a), the SLP obtains the following baseline NDWs for the two children: •

Alejandro NDW-English = 52.3

•

Luis NDW-English = 60.2

After comparing the English NDWs to the Bilingual English Story Retell Database contained in the Systematic Analysis of Language Transcripts (SALT; Miller & Iglesias, 2010), which is based on 2,070 U.S. bilingual (Spanish-English) children (kindergarten to 2nd grade) retelling Mercer Mayer’s (1969) wordless picture book, Frog, Where Are You?, in English, the SLP determines that the English lexical diversity of both children fell below expected levels (i.e., SD > -1.5) when compared to the English NDWs of other age-matched, bilingual children. It should be noted that the comparison group is other ELL children, not monolingual Englishspeaking children. Although the possibility that the children’s lexical diversity might be age appropriate in their native language cannot be ruled out in the absence of language sampling in Spanish, the clinician nevertheless can focus on expanding lexical diversity in English within therapy. Given sufficient treatment time (e.g., a semester), the SLP can then re-assess the English lexical diversity of Alejandro and Luis using narrative language sampling to measure the progress each child has achieved with therapy. The procedure of initial assessment, intervention, and re-assessment is fundamental to clinical approaches such as dynamic assessment (Gutiérrez-Clellen & Peña, 2001), which focus on change over time. Assessment and initial intervention, respectively, are achieved by the SLP during initial language sampling (i.e., baseline oral language measures, including NDW) and during focused intervention to increase each child’s vocabulary over time. Follow-up language sampling would be considered re-assessment, to once again measure Alejandro and Luis’ NDW in English. However, is progress best measured by the results of the second assessment (i.e., follow-up language sampling) alone or by considering the actual rate of progress from initial testing to follow-up testing? Even though neither child may meet the target NDW range of typically developing children, each child’s individual rate of progress would provide the SLP with valuable information regarding their ability to learn.

26

Preliminary, longitudinal l l research da ata from ourr laboratory,, based on narrative n e samples in n Spanish an nd English frrom 208 biliingual childrren (followed d from language kinderga arten through h second gra ade), have allowed us to o determine typical t lexical diversity growth ra ates for ELL Ls like Alejan ndro and Luiis (Rojas & Iglesias, I 200 09b). Growth h curve modeling (GCM; Siinger & Wille ett, 2003) wa as used to determine d th he trajectory,, or shape, of o NDW grow wth over time e. Our prelim minary data from bilingu ual children (See Figure 1) suggest that t the prototypiical growth trajectory t off English lexical developm ment from kindergarten k n to second grade g is not lin near, but rath her that it is s a shifting rate r of growtth that seem ms to accelerrate during Kinderga arten (18.9 new n differentt words), slow down durring first gra ade (2.7 new different wo ords), and then n accelerates s again durin ng second grrade (17.2 new n differentt words). Figure 1. Prototypical growth trajjectory for ND DW English (NDW-E): Kin ndergarten to t 2nd grade

O data als Our so indicate significant in ndividual diffferences as a function of the childre en’s initial En nglish NDW (i.e., initial NDW N range: 8-to-113) and a their respective grow wth rates (i.e e., non-linea ar trajectorie es of growth h and decline e over a 3-ye ear period). Further, F the e data indica ate that although girls’ English E NDW W tends to develop fasterr than boys’’ English ND DW over time e, the r of grow wth are non--significant. Clearly, GC CM data wou uld provide SLPs S differences in these rates with pow werful, new tools to evalu uate actual progress p (in this case, th he growth off English lex xical diversity over time) over o time, instead of sole ely relying on n the single, static meas surement endpointts provided by b standardiized tests to determine typical t devellopment. In other words s, while tra aditional assessment me ethods will co ontinue to be b implemen nted, measurring progress over time e can enhan nce clinical practice p for all a children, regardless of o their langu uage(s). A Assuming tha at our prelim minary longitudinal data a are confirm med, the SLP P servicing Alejandro o and Luis would w now have h two valu uable clinica al benchmarrks to evalua ate each chilld’s English NDW: N (1) the e Bilingual English E Story y Retell Data abases within SALT tha at can be use ed to compare each child’s s English ND DW at baseliine (i.e., initiial assessme ent) and at follow-up f (i.e e., re-assess sment), afterr focused inttervention; (2) ( the overa all growth rate of English h NDW for bilinguall (Spanish-E English) child dren during second grad de (i.e., 17.2 NDW increa ase). Therefo ore,

27

after using baseline and follow-up narrative language sampling, the SLP could have the information outlined in Table 1. Table 1. Language sampling and growth curve modeling (GCM) data: NDW English Reference

Reference GCM

Test

Database

Retest

NDW

NDW

Database

NDW

NDW

Change

NDW

Child NDW

Ratea (SD) Alejandro

(SD) 66.1

52.3 89.4

13.8 108.3 17.2

Luis

60.2

(18.3)

81.8

(18.9)

21.6

a = 2nd Grade

If the SLP relied exclusively on Alejandro and Luis’ English NDWs to compare them to those of age-matched, bilingual children from the Bilingual English Story Retell Database, both children would appear to have performed below age expectations at test and retest. Therefore, the SLP could conclude that even with focused treatment, the absolute gains made by Alejandro and Luis were insufficient to consider the possibility of either child being typically developing. After all, even in the absence of native language testing, at least their English NDW was compared to that of other bilingual children. This would be a premature conclusion, however, because the role of progress over time has not been considered fully. The SLP can determine the amount of NDW change (i.e., additional different words) by calculating the difference between English NDW at initial assessment and at re-assessment, which provides a measure of progress for each child. The amount of individual NDW change is then compared to the prototypical NDW growth rate to evaluate whether the lexical growth rate of each child is faster or slower than that of other bilingual children in English. Examination of these results indicate that, even though Luis’ English NDW is below expectations at retest (> -1SD), he is actually gaining different words at a faster rate (21.6) than other bilingual children in second grade (17.2). In contrast, Alejandro’s NDW is not only below expectations at retest (> -2SD), but he is adding different words at a slower rate (13.8) than other bilingual second graders (17.2). Looking ahead, the SLP could make an informed prediction that Luis’ rate of progress may eventually allow his English lexical diversity to be on par with his bilingual peers, or perhaps even exceed it. Further, Luis’ accelerated vocabulary growth rate suggests that he could actually be typically developing, and at minimum, provides sufficient evidence to warrant native language testing. Moreover, since Luis achieved considerable progress with focused clinical intervention, he is a prime candidate for ESL support. On the other hand, Alejandro’s reduced rate of progress indicates that he is not likely to “catch up” to his bilingual peers without further clinical intervention, and perhaps also ESL services. Although both children passed their ESL screenings, Alejandro’s relatively limited vocabulary growth over time warrants more in-depth testing. It is important to emphasize that, although this article focused on using language samples to assess language growth rates to determine typical development in ELL children, this approach can also be applied to monolingual-English speakers. A variety of databases based on monolingual-English speakers of different ages are available to clinicians and researchers (Breit-Smith, Cabell, & Justice, 2010; Hammer, Farkas, & Maczuga, 2010; Hart, Petrill, & Kamp Dush, 2010; Heilmann, Miller, & Nockerts, 2010; Mashburn & Myers, 2010; Mullen &

28

Schooling, 2010; Tomblin, 2010), and growth rates can be obtained on a variety of language and literacy measures. The Bilingual English Story Retell Database, along with its counterpart —the Bilingual Spanish Story Retell Database, currently represent the most comprehensive set of language sample analysis databases for Spanish-English bilingual children, which have been used in prior research (e.g., Miller, Heilmann, Nockerts, Iglesias, Fabiano, & Francis, 2006). Further, the sensitivity (69-78%) and specificity (84-85%) of these and other SALT databases has been evaluated to be appropriate for identification of language impairments (Heilmann, Miller, & Nockerts, 2010).

Speech-Language Services and ESL Instruction Speech-language services and ESL instruction are not parallel in that the former focuses on treatment of disordered speech and or language, while the latter focuses on helping ELL speakers master using the English language. Although the American-Speech-LanguageHearing Association (ASHA) explicitly prohibits SLPs from providing ESL instruction, with the exception of SLPs who have completed ESL professional preparation, it does encourage collaboration between SLPs and ESL instructors during pre-assessment, assessment, and intervention (1998). In actual practice, however, the demographic demands of the school-aged ELL population relative to district-level resources often blur the distinction between speechlanguage services and ESL instruction. Moreover, inherent shortcomings in how ELL children are generally dealt with by providers of speech-language and ESL services further confound one service type with the other. As previously discussed, the majority of SLPs do not speak the native languages of ELL children, and they tend to assess ELL children using standardized tests in English. The absence of native language testing probably contributes to overidentification (and occasionally, under-identification) of ELL children as having language disorders. The general approach to identify children as ELL and to assign children to specific instructional programs and/or ESL services is equally as flawed. The majority of school districts rely on home language surveys and ESL screening assessments to do so, and the validity of both tools is in question. As a result, some ELL children who need ESL-only support qualify instead for speech-language services. Although this type of misidentification is problematic, equally troublesome is that speech-language and ESL services are often considered to be mutually exclusive; therefore, active collaboration between SLPs and ESL instructors is not commonplace. Perhaps most concerning, however, are ELL children who are candidates for both services—students who have speech and/or language disorders (identified in both languages) and who are in the initial stages of acquiring English as their second language. These are the children that need the most support and, therefore, require the highest degree of collaboration between speech-language and ESL service providers.

Conclusion Alejandro and Luis, the two ELL children in our story, are examples of children who got “pushed” to speech-language services secondary to ESL screenings that classified them as English proficient. Yet, each child represents a distinct profile of language skills in English, which was clarified by narrative language sampling. Available data from SALT (Miller & Iglesias, 2010) and preliminary growth curve modeling data (Rojas & Iglesias, 2009b) of bilingual children’s narrative language samples in English and Spanish provided appropriate benchmarks from where to anchor baselines of oral language skills and how to measure progress of these language skills over time. The use of SALT data (i.e., Bilingual Spanish/English Story Retell Databases) and of growth curve modeling data based on bilingual children’s language development are valuable tools for SLPs to better manage ELL caseloads and to foster ongoing collaboration with ESL instructors.

29

References Abedi, J. (2006). Psychometric issues in the ELL assessment and special education eligibility. Teachers College Record, 108(11), 2282-2303. Abedi, J., Hofstetter, C. H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28. American Speech-Language-Hearing Association. (1998). Provision of instruction in English as a second language by speech-language pathologists [Technical report]. Available from www.asha.org/policy Bedore, L. M., & Peña, E. D. (2008). Assessment of bilingual children for identification of language impairment: Current findings and implications for practice. International Journal of Bilingual Education and Bilingualism, 11(1), 1-29. Breit-Smith, A., Cabell, S. Q., & Justice, L. M. (2010). Home literacy experiences and early childhood disability: A descriptive study using the national household education surveys (NHES) program database. Language, Speech, and Hearing Services in Schools, 41, 96-107. Caesar, L.G., & Kohler, P. D. (2007). The state of school-based bilingual assessment: Actual practice versus recommended guidelines. Language, Speech, and Hearing Services in Schools, 38, 190-200. Goldstein, B. (2004). Bilingual language development and disorders in Spanish-English speakers. Baltimore: Paul H. Brookes. Gutiérrez-Clellen, V. F., & Peña, E. (2001). Dynamic assessment of diverse children: A tutorial. Language, Speech, and Hearing Services in Schools, 32, 212-224. Hammer, C. S., Farkas, G., & Maczuga, S. (2010). The language and literacy development of headstart children: A study using the family and child experiences survey database. Language, Speech, and Hearing Services in Schools, 41, 70-83. Hart, S. A., Petrill, S. A., & Kamp Dush C. M. (2010). Genetic influences on language, reading, and mathematic skills in a national sample: An analysis using a national longitudinal survey of youth. Language, Speech, and Hearing Services in Schools, 41, 118-128. Heilmann, J. J., Miller, J. F., & Nockerts, A. (2010). Using language sample databases. Language, Speech, and Hearing Services in Schools, 41, 84-95. Hopstock, P. J., & Stephenson, T. G. (2003). Descriptive study of services to LEP students and LEP students with disabilities [Special Topic Report #1]. Washington, DC: U.S. Department of Education, Office of English Language Acquisition, Language Enhancement and Academic Achievement of LEP Students. Iglesias, A., & Gutiérrez-Clellen, V. (1988). The cultural-linguistic minority student. In D. Yoder & R. Kent (Eds.), Decision making in speech-language pathology. Burlington, Canada: B. C. Decker. Kohnert, K. (2008). Language disorders in bilingual children and adults. San Diego, CA: Plural Publishing. Mashburn, A.J., & Myers, S.S. (2010). Advancing research on children with speech-language impairment: An introduction to the early childhood longitudinal study-kindergarten cohort. Language, Speech, and Hearing Services in Schools, 41, 61-69. Mayer, M. (1969). Frog where are you? New York: Dial Press. Miller, J. F., Heilmann, J., Nockerts, A., Iglesias, A., Fabiano, L., & Francis, D. J. (2006). Oral language and reading in bilingual children. Learning Disabilities Research & Practice, 21(1), 30-43. Miller, J. F., & Iglesias, A. (2010). Systematic Analysis of Language Transcripts (SALT), Bilingual SE Version 2010 [Computer Software], SALT Software, LLC. Mullen, R., & Schooling, T. (2010). The national outcomes measurement system for pediatric speechlanguage pathology. Language, Speech, and Hearing Services in Schools, 41, 44-60. Paul, R. (2006). Language disorders from infancy through adolescence: Assessment and intervention (3rd ed.). St. Louis. MO: Mosby. Rojas, R., & Iglesias, A. (2009a, March 3). Making a case for language sampling: Assessment and intervention with (Spanish-English) language learners. The ASHA Leader, 14(3), 10-13. Rojas, R., & Iglesias, A. (2009b, November). Language growth modeling in bilingual children. Poster session presented at the American Speech-Language-Hearing Association Convention, New Orleans, LA.

30

Roseberry-McKibbin, C., Brice, A., & O’Hanlon, L. (2005). Serving English language learners in public school settings: A national survey. Language, Speech, and Hearing Services in Schools, 36, 48-61. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press. Swanson, C. B. (2009). Perspectives on a population: English-language learners in American schools. Bethesda, MD: Editorial Projects in Education. Tomblin, B. J. (2010). The EpiSLI Database: A publicly available database on speech and language. Language, Speech, and Hearing Services in Schools, 41, 108-117. U.S. Department of Education. (2009). Building the legacy: IDEA 2004. Available from http://idea.ed.gov/explore/home

31