Pragmatic Versus Morphological! Syntactic Criteria for

Pragmatic Versus Morphological! Syntactic Criteria for Language Referrals Jack Damico and John W. OlleT, Jr.

Two methods of identifying language disordered children are examined . Traditional approaches require attention to relatively superficial morphological and surface syntactic criteria, such as, noun-verb agreement, tense marking, pluralization . More recently, however, language testers and others have turned to pragmatic criteria fo cussing on deeper aspects of meaning and communicative effectiveness, such as, general fluency, topic maintenance, specificity of referring terms. In this study, 54 regular K-5-teachers in two Albuquerque schools serving 1212 childre n were assigned on a roughly matched basis to one of two groups . Group S received in-service training using -traditional surface criteria for referrals, while Group P received similar inservice training with pragmatic criteria. All referrals from both groups were reevaluated by a panel of judges following the state determined procedures for assignment to remedial programs. Teachers who were taught to use pragmatic criteria in identifying language disordered children identified significantly more children and were more often correct in their identification than teachers taught to use syntactic criteria. Both groups identified significantly fewer children as the grade level increased.

Teacher participation in the referral process has been advocated for more than 20 years (Power, 1956) and has been discussed frequently (Ainsworth, 1965; ASHA, 1974; and Deal,- McClain and Sudderth, 1976). As recently as January 1978, Nodar mscussed in this journal the use of teacher judgements as a basis for identifying children with hearing loss. Results show that teachers are capable of making reliable judgements (Clauson and Kopatic, 1975) and improving the overall effectiveness of certain referra~ procedures (Pickering and Dopheide, 1976). Moreover, extensive research in language testing demonstrates the utility of subjective judgements of communicative effectiveness (Oller, 1973; 1979; Spolsky, 1978; and Muma, 1978b). In this study, we focus on the criteria used by teachers to aid the clinician in identifying language disordered children. Although the use of teacher referrals was recommended by the American Speech-Language-Hearing Association more than five years ago (ASHA Task Force, 1974), disappointingly little information is available on precisely what teachers might be encouraged to look for in terms of specific referral characteristics. Clauson and Kopatic (1975) Jack Damico is presently affiliated with the A lbuquerque Public School System. Requests{or reprints may be addressed to him at East Area Office, Albuquerque Public Schools, 2611 Eubank N.E., Albuquerque, NM 87112. John W. Oller, Jr. il' an associate professor at the University of New Mexico (Albuquerque) .

Reprinted fi'om Language, Speech, and Hearing Services in Schools Aplil 1980, Vol. Xl, No.2 CopYllght © 1980 by the Amellcan Speech-Language-Hearing Association

reported that teachers were able to correctly identify "language delayed" children with a high rate of success, but specific language characteristics of such children were not made explicit. Similarly, Pickering and Dopheide (1976) proposed that aides could be trained to screen children for language difficulties, but no information on particular criteria was offered . Where diagnostic language tests in speech-language pathology are used, or where specific remediation procedures are recommended, the emphasis is frequently on relatively superficial aspects of utterance-forms which rarely have a profound effect on the child's ability to communicate (Baratz, 1969; Politzer, Hoover, and Brown, 1974; Oller, 1979). In fact, it is deeply distressing from a linguistic perspective that such an emphasis is very likely to result in the referral of children from certain non-majority language backgrounds who are quite normal in terms of language development (Labov, 1976; Wolfram and Christian, in press; D. K. Oller, in press). Their only deviancy may be speaking a variety of English that differs from that spoken by the majority. Not only is there good reason to believe that by such surface oriented tVethods there will be a higher number of false alarms (where normal childr~ are identified as disordered, and a probable high proportion of these being minority children), but there will also be a higher number of misses (where truly disordered children are overlooked by the screening methods). Current literature in language testing, psycholinguistics, and applied linguistics suggest that a disproportionate amount of attention has been directed to surface form at the expense of meaning (Oller , 1973; Abkarian, 1977; Spolsky, 1978). Developmental psychologists have also tended to refocus their attentio n from surface elements to embrace the broader and deeper aspects of meaning and communication (Bloom, 1976). , In a discussion of "the ontogenesis of speech acts," Bruner (1975) offers an 6nsightful and persuasive rationale for his preference for "a pragmatically oriented approach." His thinking accords well with that of Bates (1976), M uma (1978a), Oller and Perkins (1978), Rees and Shulman (1978), and Prutting (1979). In all these approaches, the emphasis is away from surface form and is focused on the efficiency with which meaning is conveyed between sources and audiences . Would there be a significantly larger number of referrals overall with a shift in attention from the surface forms of speech to the deeper aspects of meaning? Would this shift result in greater accuracy of referrals? How will the change in focus affect accuracies across different grade levels, if at all? ! ' It wou ld have been desirable to assess the differentia l effect of a change in method for referra ls across mainstream children and child ren from ethnic backgrounds . However, thi s was not possible d ue to the fact that the ch ildren in the schools studied are nearly all from homes where a majority variety of English is used . It is to be hoped, however, that future stud y will examine the probable discriminatory effects on eth nic minority children of referra l methods that focus on surface form. . 86

Langu~ge,

Speech, and Hearing Services in Schools

XI

85-94

Apri l 1980

Method The subject population for this study consisted of 1212 children in two middle class ?uburban schools in the Albuquerque area. The ethnic composition of the schools was predominantly anglo and nearly all children were native speakers of the majority variety of English. From the pool of 54 teachers in the two schools, two groups were formed. The only a priori criterion for group matching was grade taught. Otherwise the assignment to groups was random. However, after the fact comparison of teachers on criteria of age, teaching experience, level of education and number of children taught, revealed a close match across groups. (See Table 1.) TABLE 1. Means and ranges of the two experimental groups in age and years of experience. educational data . number of children in classes. and number of teachers assigned to each group. Group P (Pragmatic Focus)

Group S (Suiface Focus)

Age Mean Range

45.57 71 -25

44.00 60-22

Teach ing Experience Mean Range

\5.17 40-02

13.56 38-01

5 7 5 10

8 5 4 10

604 22.37

608 22.51

27

27

Characteristic

Level of Education BA BA 15 hours BA 45 hours MA Number of Children Taught Total Mean Number of Teachers

Group S received in-service training in the use of traditional morphological and syntactic characteristics frequently used in current language testing procedures (Kirk, McCarthy, and Kirk, 1968; Lee, 1969; Carrow, 1973; 1974) and remediation methods (Coughran and Liles, 1976; Fokes, 1976) in speech-language pathology . Referral criteria for Group S included frequent errors in any of the following: (1) noun-verb agreement; (2) possessive inflections; (3) verb tenses; (4) auxiliary verbs; (5) irregu lar verbs; (6) irregular plurals; (7) pronoun case or gender; (8) reflexive pronouns; and (9) syntactic transpositions. The teachers were given examples from the aforementioned tests and ex planaDAMICO AND OLLER, JR: Language Referrals

87

tions were offered for each characteristic. Taped samples of child speech illustrating the errors in question were presented and discussed. Group P received in-service training that was similar in all respects except the criteria of interest were pragmatic 2 in nature. Such characteristics are known to have greater impact on the total effectiveness of communication attempts (Evola, Marner and Levty, in press; Bacheller, in press; and Hendricks, et ai, in press). Further, they tend to be strongly interrelated with each other. Referral criteria for Group P included the following:

linguistic norifluency-the child's speech production is disrupted due to a disproportionately high number of repetitions, unusual pauses and excessive use of hesitation forms; (2) revisions - speech production is broken-up by numerous ,false starts or self-interruptions. The child revises what he has already l said as if he keeps coming to dead ends in a maze; \ (3) delays before responding-attempts at communication initiat~ by others are followed by pauses of inordinate length; (4) nonspecific vocabulary - in this instance we are concerned with the use of deictic expressions such as "this" or "that," "then," "he," "over there," and the like when no antecedents have been provided by the speaker and when the listener has no way of knowing what is being referenced. Also, children displaying this characteristic will tend to use generic terms such as "thing," "stuff," "these" and "those" when more specific referring expressions would seem to be required; (5) inappropriate responses-these are easy to spot but difficult to explain. It is as though the child were operating on an independent discourse agenda-not attending to the prompts or'probes of the adult or others; (6) poor topic maintenance-the child makes rapid and inappropriate changes in the topic without providing transitional clues to the listener; (7) need for repetition-multiple repetitions arf requested without any indication of improvement in comprehension. (1)

Participating teachers were not informed of the experimental issues. They saw the division into groups as a practical necessity motivated by the number of teachers to receive the in-service training and the availability of staff and space. Four 45 minute sessions were conducted by the first author on four 20ur definition of pragmatics is intentionally broad. For extensive discussion see Oller, 1973: 1979: Bates, 1976; and Muma, 1978a. The latter defines pragmatics as "psychosocial dynamics of language use." We are particularly concerned with the pragmatic mapping of utterances onto experiential contexts. The temporal aspect of such mappings is stressed in criteria 1,2,3, and 6. Criterion 4 is especially concerned with the connecting of specific utterance forms to known spatio-temporal coordinates in ongoing experience and the categorical specificity of referring terms. Criteria 5, 6 and 7 also encompass, somewhat loosely, strategies of interaction.

~

Langunge, Speech, and Hearing Services in Schools

XI

85-94

April 1980

separate days. At School A, one session was held for Group P and one for Group S. This procedure was repeated for School B. Exactly one week after each session a 35 minute follow-up was conducted. There were four followup sessions on four separate days. These were held to remind teachers of the criteria for referrals and to answer any questions. Accuracy Criterion. As referrals were received (during the academic year 1977 -78), an objective evaluation was conducted by a clinician holding the Certificate of Clinical Competence (the first author). These evaluations conformed to the standard testing procedures specified in the Plan for the Delivery of Special Education Services in New Mexico: Regulations (1976). This plan describes a set of objective tests that must be administered and specific. guidelines for the classification of a child as "language disordered" by the Educational Appraisal and Review Committee (Appendix). Completed evaluations were discussed by the Educational Appraisal and Review Committee, consisting of two speech-language pathologists (the first author and his supervisor), two special education teachers, a counselor, a nurse, and the school principal. To insure objectivity concerning the accuracy of referrals during this session, the first author's participation was strictly limited to reporting scores on the objective tests required by the state. (See the Appendix.) Only this information was used by the committee to evaluate referrals both from Group P and Group S (following the state guidelines) to determine in each case whether or not the referred students should be placed in a program for remediation. Results The contrasts between groups and across grades were examined by a two-way analysis of variance (Nie, Hull, Jenkins, Steinbrenner, and Bent, 1975). As can be seen in Table 2, the number of referrals fell off with inTABLE 2. Number of referrals, number correct, and percent of accuracy by grade between the two experimental groups. Group S (Surface oriented)

Group P (Pragmatically oriented) Grade

Kindergarten First. Grade Second Grade Third Grade Fourth Grade Fifth Grade Tota l

Number of Referrals

Number Correct

Accuracy

9 14 10 5 7 3 48

7 13 8 5 5 2 40

78% 93% 80% 100% 71% 67% 83%

Number of Referrals 12

8 8 2 0 I 31

DAMICO AND

Number Correct 7 6 4

I

Accuracy 58% 75% 50% 50%

0

I

100%

19

61%

OLLER, JR: Language Referrals

89

creasing grade level. Presumably this is due to a tendency to identify language disordered children relatively early in the school program. However, we cannot rule out the possibility that language disorders may be harder to spot at the higher grade levels. The data in Table 2 were subjected to a two-way analysis of variance. T he pragmatically oriented group referred significantly more children (p < 0.03) and the decrease in referrals with increase in grade level was also significant (p < 0.001). The interaction between group and grade was not significant (see Table 3). The.number of correct referrals was also subjected to a twoway analysis of variance. Again Group P made more correct referrals (p < 0.001) and the referrals decreased with the increased grade level (p < 0.001). These analysis of variance results are presented in Table 4. The interaction again was not significant. It is interesting to note that Group P TABLE 3. Two-way classical analysis of variance Wilh number of referrals by group and by grade .

S ouree

0/ V ariatio n

Main effects Group Grade 2- Way in teraclions Grade Group

Signif o/F (p < value givm)

Sum of Squares

df

M ean Square

F

51.70 5.35 46.35

6 I 5

8.62 5.35 9.27

8.80 5.47 9.47

0.03

6.61 6.6 1

5 5

1.32 1.32

1.35 1. 35

0.26 0.26

5.42

Explained

58.31

II

5.30

Residual

4 1.12

42

0.98

Total

99.43

53

1.88

TABLE 4. Two-way classical analys is of variance with number of co rrect refelTa ls by gro up and by grade.

Source of Variation

Main Effects Group Grade 2-Way interaclions Gn;up Grade Explained

df

Mean Square

F

32.595 8. 167 24.429

6 I 5

5.433 8.167 4.886

9.04 13.58 8. 13

0.001

2.692 2.692

5 5

0.538 0.538

0.90 0.90

0.49 (J.49

35287

II

3.208 ·

5.34

Residual

25.25 0

42

0.6(JI

Total

6(J.537

53

1.1 42

90

Signi! 0/ F (P < value given)

Sum o/Squares

Language, Speech, and Hear ing Servias in Schools

,':'

XI

85-94

April 1980

referred more children above the second grade level and were more accurate in their referrals . Group Preferred 15 children (grades 3-5) with an accuracy of 80% while Group S referred only 3 children from the same grades with an accuracy of only 67% . Overall, the accuracy of Group P was 83% as agamst 61 % for Group S (Table 2). These figures may underestimate the contrast because Group S not only had a higher rate of false alarms (referrals of children who were in fact normal), but may have failed to identify children who were language disordered. 3 Group Preferred 48 children . Of these referrals, 40 were judged to be correct. Group S had the same number of children and the same number of teachers but made only 31 referrals , with 19 judged to be correct. If we assume that there were approximately equal numbers of disordered children in the classes taught by Group S teachers (608 children) as in the classes taught by Group P teachers (604 children), it is reasonable to suppose that Group S teachers overlooked about 21 children who really had language difficulties. The pragmatic criteria for referrals seems to be superior for identifying language disordered children. Teachers were able to identify language disordered children more readily by using these pragmatic criteria than they can by using morphologicaVsyntactic criteria. This difference is not because the teachers were untrained in linguistics. Results from many studies (see especially the 24 empirical studies reported in Oller and Perkins, in press; and for earlier references, see Oller, 1976; and the bibliography to Spolsky, 1978) show that even persons who are untrained in linguistics make less reliable judgements about discrete points of surface form than they do about more global aspects of communicative effect. Muma (1978b) has suggested that the best screening procedure is the informed judgement of classroom teachers. Assuming that he is correct, the results reported here would suggest that such judgements are more apt to be correct in cases where teachers are encouraged to look for pragmatic characteristics pertaining to communicative effects Yather than morphologicaV syntactic errors. This finding is not only consistent with the research on pragmatic language testing, but it is also supported by the intuitive reactions of teachers to the pragmatically oriented criteria. When these were being discussed during the in-service training sessions, teachers reacted both verbally and non-verbally indicating that they had already noticed some of the

3In general we object to the use of the terms "normal"' and "disordered" in this and similar contexts. However , we are referring here to the widely institutionalized practice of making this differentiation in a quite practical sense. We realize that many of the children identified as "disordered" may be as normal as can be. It is, in fact, in the hope of appropriately limiting the use of these terms that this whole study is conducted.

DAMICO AND OLLER, JR:

\

Language Referrals

.

91

pragmatic difficulties in certain children. No such natural insight was indicated in response to the presentation of the surface linguistic criteria. References ABKARIAN. G. G., The changing face of a discipline : Isn't it "romantic"? I Speech Hearing Dis., 42, 422-435 (1977). AINSWORT H, S., The speech clinician in public schools: "Pa rticipant" or "separatist"? Asha, 7 , 495-503 ( 1965). AMERICAN SPEECH AND HEARING ASSOC IATIO N, Standards and Guidelines for Comprehensive Language, Speech and Hearing Programs in the Schools. Washington , D.C.: American Speech and Hearing Association (1974). BACHELLER, F. , Communicative effectiveness as predicted by judgements of the severi~f learner errors in dictations. In Oller and Perkins (in press). BARATZ, j., A bidialectal task for determining language proficiency in economically disadvan taged Negro children. Child Development, 40, 889-901 (1969). BATES, E., Language and Context: the acquisition of pragmatics. New York: Academic Press (1976). BLOOM, L., Language development. In R. Wardhaugh and H. D . Brown (Eds .) A Survey of Applied Linguistics. Ann Arbor: University of Michigan Press (1976). BRUNER, j. S., The otogenesis of speech acts .I Child Lang., 2 , 1-19 (1975). CARROW, E ., Test of Auditory Comprehension of Language. Austin: Learning Conce pts (1973). CARROW, E., Carrow Elicited Language In ventory. Austin: Learning Concepts (1974) . CLAUSON, G. M. and KOPATIC, N. j. , Teacher attitudes and knowledge of remedial speech programs. Lang. Speech Hearing Sem Schools, 6, 206-211 (1975) . COUGHRAN, L. and LILES , B. Z., Developmental Syntax Program. Austin: Learn ing Concepts (1976). DEAL, R. E. , MCCLAIN, B. and SUDDERTH, J. F., Identification, eva luation, therapy and follow -up for children with vocal nodules in a public school setting. I Speech Hearing Dis., 41, 390-398 ( 1976). I . EVOLA, J. , MAMER, E. and LENTZ, B., Discrete point versus global scoring for cohesive devices. In Oller and Perkins (in press). FOKES, J., Fakes Sentence Builder. Boston: Teaching Resources (1976). HENDRtCKS, D., SCHOLZ. G ., SPURLING, R., JOHNSON, M. and VANDENBURG, L., Oral profici ency testing in an intensive English language program. In Oller and Perkins (in press). KIRK, S., MCCARTHY , J. and KRIK, The Illinois Test of Psycholinguistic Abilities (rev. ed.). Urbana: Un ivers it y of Illinos Press (1968). LABOV, W.:Systematically misleading data from test questions. Urban Review, 9, 146-169 (1976). LEE , L., Northwestern Syntax Screening Tests. Evanston, Ill.: Northwestern Un iversin' Press (1969). MUMA , j. R., Language Handbook: Concepts, Assessment, !nteTvention. Englewood Cliffs. N .J.: Prentice- Hall, Inc. (1978a). MUM A. J. R., Language Assessment and [nterventinn: IEP ing it. Workshop presented to the Ora l Language, Speech and Hea ring Special ists of Albuquerque Public Schools (1978b). NEW MEXICO STATF. DEPARTMENT OF EDUCATION, A Plan faT the Delivery of Special Education Seruices in New MexUo: Regulatioill. Santa Fe: New Mexico State Department of Education (1976). NIE, N. H., HULL, C. H. , JENKDIS , J. G., STIF.NBRENNF.R, K. and BENT. D. H., Statistical Packagefor thR Social Sciences. (2nd Ed.). New York: McGraw I-Iill (1975). NODAR, R. H., Teacher identification of elementary school chi ldren with hearing loss. Lang. Speech Hearing Servo SchooLI, 9, 24-28 (1978). OLLER, D. K .. Phonological theory and communication disorders: The role of non-standard language or dialect. Sympos iulll paper for the American Associat ioll of PhOlwtic Sciences. San Francisco. California. NO\'ember 1978, in press. Ol.LER, J. W., Pragmati c la n guage resting. Lrwguagr ScUollcrs. 28. 7-12 ( 1973). OLLER, J. W. , Language Tesring. In R. Wardhaugh and H. D. B]'()lm (Eels.) S1I11'I'-" of" Applird Lillguistics. Ann Arbor: Un ivers itl' of Michigan Press (1976). OLLE~. J. W. and PERKrNs. K., Lallglltlgr ill Ed1lcatioll: testillg the tests. Rowley. :'-.Iass.: Newbury House (1978).

92

Language, Speech, and Hearing Senlices in SchooL,'

XI

85-94

April 1980

OLLER, J. W. and PERKINS, K., Research in Language Testing. Rowley, Mass.: Newbury House, in press. OLLER, J. W., Language Tesls al School: A Pm!{'rl.atic Approach. London: Longman (1979). PICKERING, M. and DOPHEIDE, W., Training aides to screen children for speech and language problems. Lang. Speech HearingServ. Schouls, 7,236-241 (1976). POLITZER, R. L., HOOVER, M. R. and BROWN, D., A test of proficiency in black standard and non-standard speech, TESOL Quarterly, 8, 27-35 (1974). POWER, M. H., What makes an effective public school speech therapist?]. Speech Hearing Dis., 21,461-467 (1956). PRUTTING, C. A., Process/pras/sesln: the action of moving forward from one point to another on the way to completion.]. Speech Hearing Dis., 44, 3-30 (1979). REEs, N. S. and SHULMAN , M., I don't understand what you mean by comprehension.]. SPeech Hearing Dis., 43, 208-219 (1978). . SPOLSKY, B. Advances in Language Testing: Series 2, Approaches to Language Testing. Papers 111 applied linguistics, Center for Applied Linguistics, Arlington, Virginia (1978). WOLFRAM, W. and CHRISTIAN, D., On the application of sociolinguistic information: test evaluation and dialect differences in Appalachia. In T. Shopen (Ed.) Variation in thl' Structure and Use of English. New York: Winthrop, in press. Received January 31, 1979 Accepted Septem bel' 27, 1979

Appendix State mandated procedure jar placement uf children into language disordered remediation programs as specified by the Plan/or the Deliver), of Special Education Services in New Mexico: Regulaticms (1976).

5.1

Definition

5.2

d. Language disorder-a disability in verbal learning resulting in markedly impaired ability to acquire, use, or comprehend spoken andlor written language where no significant degree of sensory or motor incapacity, mental retardation or emotional malacljustment is present as the primary disabling condition.

5.3

Reljuired Evaluation Data

5.3.1

Screening a. Hearing b. Vision (near, bll' and color discrimination)

5.3 .2

Case History a. Educational b. Family (inclusive of language dominance of the home) c. 'vledical information

5.3.3

Ed ucational Assessment by qualified personnel d. Language disorder -individual intelligence test (one reljuired) -appropriate Wechsler Intelligence Scale -Stanf()rd-Binet Intelligence Scale -Leiter International Performance Scale -education achievement (one reljuired) -Peabod y Individual Achievement Test -Wide Range Achievement Test

DAMICO AND OLLER, JR: Language Referrals

93

-language assessment (receptive and expressive .vocabulary, auditory discrimination, auditory memory, understanding and expression of syntax, and conversation speech) [Only one of the following is req uired by law , but two were adm inistered to each child d ependin g on age level. A spontaneous lan guage sample was also obtained and analyzed by Developmental Sentence Scoring.] - lIlinois Test of Psycholinguistic Abilities -No rthwestern Syntax Screening Test - UtahTest of Language Development - Detroit Test of Learning Aptitude -Assessment of Children's Language Comprehension -Boehm Tests of Basic Concepts 5.4

Analyzing the data for Education Needs

5.4.3

A la nguage disorder is considered significant only when the language process and academic test results are below the student's chronological and mental age expectancy.

94 Language, Speech, and Hearing Services in Schools

XI

85-94

April 1980