Cloze, Discourse, and Constraints on Native

Prepublication draft of Oller, J. W., Jr. (1994). Cloze, discourse, and approximations to English. In J. W. Oller, Jr., & Jon Jonz (Eds.), Cloze and coherence (pp. 119-134). Cranbury, New Jersey: Bucknell University Press.

CHAPTER SEVEN

Cloze, Discourse, and Constraints on Native Speakers of English John W. Oller, Jr. (1975a, revised 1992)

Editors’ Introduction This chapter—a revised version of Oller (1975a)—addresses two closely interrelated questions. First, is cloze procedure sensitive to constraints ranging over passages of prose longer than five-to-ten words? Or, stated differently, are meaning constraints over such passages helpful to subjects in filling in cloze items in those passages? Second, is there a point of diminishing returns such that providing additional context beyond that point will not help a language user to guess the missing words? Stated differently, is it harder to guess a missing word on the basis of five surrounding words of context than to guess the same missing word on the basis of an additional five words, or an additional ten words, or twenty-five, or fifty? What is the point of diminishing returns if there is one? This is a study of the expectancy hypothesis (cf., Oller, 1983b): in its most general form it says that grammar-based expectancies ranging over all levels of discourse will help the language user to infer the plausible limits of intended meanings and to supply reasonable guesses about missing, distorted, or erroneous material in any stream of meaningful semiotic forms. More specifically, as applied to cloze procedure, the hypothesis predicts that cloze items (embedded in coherent discourse/text) will be sensitive to procedures that interfere with constraints ranging over segments longer than five-to-ten words. This prediction contradicts what some specialists have maintained—namely that cloze items are usually sensitive to only five-to-ten words on either side of a blank. The experimental design employed here avoids a supposed flaw of the study reported in the previous chapter and yet its results are entirely consistent with the findings of that study. Implications are discussed. If in a cloze test it is generally true that grammar-based expectancies ranging beyond five-to-ten words on either side of the blanks help subjects guess missing words, then cloze procedure must be sensitive to discourse constraints a fortiori.1 In

The author wants to thank Lisa Kahn, Nancy Morse, and especially Molly

1

Harvey for help with the handling of data reported here. Dennis Muchisky and Merryl Kravitz also helped in the administration of tests to students at the University of New Mexico. The research

SEVEN—Cloze, Discourse, and Native Adults 120 addition, if cloze procedure is subject to the expectancy hypothesis, it follows that the point of diminishing returns will be much farther out than has often been claimed (cf. MacGinitie, 1961; Carroll, 1972; Alderson, 1979a-b, 1980, 1983). In fact, a point of diminishing returns, if it exists, will not be a limitation on text per se, but on the capacities of human producers and interpreters. While a constraint on human ability is a constraint on the representations we are capable of producing and/or understanding in a sense, this does not reduce the logically independent objective status of any representamen per se. A representamen (the material, usually physical, stuff of a text or discourse) is as different from an act of using it as a baseball is from an act of being thrown or caught (though the size, weight, etc. of the ball depends greatly on the size, shape, and strength of a human hand). In any case, the expectancy hypothesis suggests that contexts beyond the five-to-ten word level will generally be helpful in inferring intended meanings and supplying missing material 〈〉. All of this seems tantamount to the trite and apparent truism that context is important to language use. However, there is plenty of disagreement about how context affects language use; how it is related to internalized grammars of language users; and how it can be measured. For instance, Carroll (1972) argued that cloze procedure is probably not an appropriate device for the measurement of a person’s ability to utilize discourse constraints, i.e., constraints ranging over passages of prose longer than 5-to-10 words. This conclusion, incidentally, was weakly refuted by the results of Carroll, Carton, and Wilds (1959). A well-known and often cited paper supporting Carroll’s skepticism about the applicability of cloze is the classic study entitled “Verbal Context and the Recall of Meaningful Material” by Miller and Selfridge (1950). They said that contextual dependencies extending over 5-to-6 words permit positive transfer, and it is these familiar dependencies, rather than the meaning per se, that facilitate learning.... When short range contextual dependencies are preserved in nonsense material, the nonsense is as readily recalled as is meaningful material (p. 184) .

A little over a decade later, however, Coleman (1963) argued that it was counterintuitive for scores on normal prose not to be greater than scores on nonsense. He found that in two attempted replications of the Miller-Selfridge results, approximations to English prose beyond the 5th order were more difficult both on a recall task (Sharp, 1958) and on a cloze task (Deese, 1961) than lower order approximations. Coleman reasoned that the explanation might lie in the way the Miller-Selfridge experiment defined orders of approximation to English prose. The technique was as follows: At the second order...a common word, such as ‘he’, ‘it’, or ‘the’ is presented to a person who is instructed to use the word in a sentence . The word he uses directly after the one given is then noted and later presented to another person, and he, in turn, is asked to use that word in a sentence. The word he uses directly after the word given him is then noted and later given to yet another person...(p. 180).

Higher orders of approximation can then be derived by using the second word to get a 3rd order of approximation, or the third to generate a 4th, and so on. Coleman believed that higher orders of approximation created by this method would be likely to deviate farther and farther from normal sequences. To test his reported here was originally presented at the Annual Convention of Teachers of English to Speakers of Other Languages held in Los Angeles in 1975.

SEVEN—Cloze, Discourse, and Native Adults 121 prediction, he submitted 1st through 9th order approximations generated by the MillerSelfridge method to twelve linguists and asked them to rank them for grammaticalness. The result was that 6th order approximations were judged less grammatical than 5th order approximations and 7th order approximations dropped even more dramatically on the scale of grammaticalness 〈〉. The dip in the curve corresponded precisely to the results discovered by Deese (1961) and Sharp (1958) using similar orders of approximation. This suggested to Coleman that, in fact, the reason nonsense conforming to short range contextual dependencies in English is as easy to recall as higher order approximations created by the Miller-Selfridge method (or even easier) is that the higher order approximations are not actually closer to normal English but rather farther away from it. In two follow-up experiments he also made an important discovery about the Miller-Selfridge scoring procedure. In the first experiment, subjects heard thirty separate strings of words each of which ranged from 1st through 6th order approximations. They also heard normal prose. In the second experiment, subjects read passages of 50 to 150 words conforming to the same criteria. In both cases subjects tried to recall as much as they could of what they had heard or read. Coleman found that if he scored the data by counting all of the words subjects correctly recalled, irrespective of sequence, his findings paralleled those of Miller and Selfridge (1950) and several other investigators who had replicated their findings. However, if he scored the data by looking at correct sequences of words, i.e., sequences of 2, 3, 4, and so on up to sequences of 15 words, the results markedly favored normal prose and the advantages of the higher order approximations tended to increase as recall was “scored in longer and longer sequences” (p. 246). The advantage of normal prose over the 5th and 6th order approximations generated by the Miller and Selfridge method was dramatic 〈〉. Of course, there are other ways of investigating approximations to normal prose. Deese (1961) used the cloze procedure as a method of testing ease of processing though he generated orders of approximation by the Miller-Selfridge method. If the cloze procedure is a suitable device for measuring ability to take advantage of longrange constraints over prose, this would demonstrate yet another application of the technique to the problems of assessing language proficiency. A lot of prior research could be brought to bear (for reviews see Klare, 1974-75, and the annotated bibliographies of Riley, 1973, and Oller, 1975c).

Experiment Method While Miller and Selfridge (1950) constructed orders of approximation to normal English by adding words one at a time to similarly generated strings of a predetermined length, there is no guarantee that the person continuing a string at any given length actually had anything definite in mind. However, it is possible to construct orders of approximation assuring the sensibleness of higher and higher orders of approximation by starting with meaningful material at the beginning and working away from it in a backward direction rather than trying to build it up one word at a time with a new participating originator after each word is added. The trouble with asking a dozen different individuals to build a 12th order of approximation to normal prose by the Miller-Selfridge method is that the method is far from natural. Each person contributes

SEVEN—Cloze, Discourse, and Native Adults 122 only a single word to any particular string at a given order of approximation and no one of the n contributors to an nth order of approximation necessarily has any notion of what the previous n - 1 other contributors might have had in mind. In fact, it may be supposed that each person had nothing at all in mind other than the nth - 1 preceding word that alone provides a constraint on the nth word which the final contributor must dutifully dredge up from a strained imagination. Instead of trying to build up towards normal prose through an artificial method that very soon (after 5 or 6 words) results in a peculiar variety of nonsense, we may begin with a passage of normal prose and work backwards by simply chopping it up into segments of various lengths and arranging them in more or less random orders 〈〉. Randomized 100-word segments might be considered a 100th order of approximation, 50-word segments a 50th order of approximation, and so forth. The shuffled segments could then be used to produce cloze test material. By segregating the segments of the same lengths, it is possible to define 100th, 50th, 25th, 10th, and 5th orders of approximation to normal prose in an intuitively more natural scale than could ever be produced by the Miller-Selfridge method. These more natural orders of approximation, arranged in ascending order, would actually progress toward full-fledged normal prose in a strict mathematical and logical sense. Cloze items could be systematically inserted to give a measure of the differential effects of given orders of approximation. The design could also be systematically counterbalanced to neutralize effects of variations in passage difficulties and skills of subjects.

Materials For the purpose of this experiment, 5 passages of prose were selected. Four were excerpted from selections in a reader edited by Gorrell, Laird, and Freeman (1970). The fifth was a passage taken from a novel by Victoria Holt (1973). (All of the texts used in Appendix 7 following this chapter.) Selected materials ranged from (I) a description of a carry-out restaurant of the 1940s or 50s, to three expository passages covering such diverse topics as (II) how to get an education, (III) how to construct a paragraph, (IV) what a sentence is, and finally (V) a brief fictional narrative about the death of Sir Edward Travers. The type of text employed proves to be an important element in subsequent research by Jonz (Chapter 19). He stresses the importance of narrative-like sequencing.2 In the present study, each passage was at least 100 words in length not counting a lead-in sentence in each passage, and a lead-out sequence of at least 5 words beyond the 100th word. By deleting the 3rd word of the 2nd sentence of each passage and every-5th word thereafter, 5 cloze tests of 20 items each were constructed over each of the 5 passages. These cloze tests yielded 5 tests at the 100th+ approximation to the actual prose these segments of more than 100-words each were taken from.3 Each 100-word segment contained exactly 20 items spaced exactly 4 However, it should be noted that according to Peircean semiotic theory, all representational forms

2

that have any meaning are ultimately related to the sort of human experience that involves the passage of time and a narrative-like sequencing of events. This is not merely a consequence of a ‘western’ point of view, as some (notably Mark Clarke, et al., 1984) have maintained, but rather a fact of human physiology as it interacts with the physical world of experience. In other words, if the whole passages could be considered normal prose, then an excerpt of 100 words 3

would be a 100th order approximation to the complete text in question. The selections used here, in

SEVEN—Cloze, Discourse, and Native Adults 123 words apart (i.e., falling in the position of the 5th word) except for the last item which had somewhat more than 2 words following it. To achieve the 50th order of approximation, each of these initial texts of 100+ words was duplicated and cropped at the ends so that it was exactly 100 words long. Then it was divided in the middle to produce two 50-word segments. To get the 25th, 10th, and 5th, orders of approximation, the 50-word segments were duplicated 3 more times and cut 3 more times into 25-word, 10-word, and 5-word segments. This yielded a pool of 100 cloze items, each item appearing in 5 conditions (orders of approximation) to normal prose (100th+, 50th, 25th, 10th, and 5th). For example, following the lead-in sentence in passage I, in the 100th+ condition, there were 20 cloze items of the form: For those who hang out there, the Carry-out offers a wide array of sounds, sights, smells, tastes, and tactile experience which titillate and sometimes assault the five senses. The air (1) warmed by smells from (2) coffee urns and grill (3) thickened with fat from (4) deep-fry basket. The jukebox (5) up a wide variety (6) frenetic and lazy rhythms. (7) ...

The 5th order approximations consisted of the same items only in 5 word segments, such as, fat from (4) deep-fry basket. wide variety (6) frenetic and The air (1) warmed by...

and so forth. The 10th order approximations consisted of the same items but arranged in randomized 10-word segments as in The jukebox (5) up a wide variety (6) frenetic and The air (1) warmed by smells from (2) coffee urns ...

Therefore, 100 distinct items times the 5 conditions of arrangement (i.e., the 5 orders of approximation to normal prose) gave a grand total of 500 items from which to construct 5 distinct 100 item cloze tests. Each of these tests would contain one of the 5 passages in the 100th+ condition, another in the 50th word condition, another in the 25th, another in the 10th, and the remaining one in the 5th. Table 7.1 Schematic Illustration of Card Stacking for the Initial Pool of 500 Cloze Items Over Five Orders of Approximation to Normal Prose Text

I

II

100th+

(1-20)1

(1-20)2

50th

25th

10th

5th

(1-10)1 (11-20)2

(1-5)1 (6-10)2 (11-15)3 (15-20)4

(1-2)1 . . . (19-20)10

(1)1 . . . (20)20

(1-10)3 (11-20)4

(1-5)5 (6-10)6 (11-15)7 (15-20)8

(1-2)11 . . . (19-20)20

(1)21 . . . (20)40

order to find suitable break points were a bit longer than 100 words, hence, they are referred to as 100th+ approximations.

SEVEN—Cloze, Discourse, and Native Adults 124

III

IV

V

(1-20)3

(1-20)4

(1-20)5

(1-10)5 (11-20)6

(1-5)9 (6-10)10 (11-15)11 (15-20)12

(1-2)21 . . . (19-20)30

(1)41 . . . (20)60

(1-10)7 (11-20)8

(1-5)13 (6-10)14 (11-15)15 (15-20)16

(1-2)31 . . . (19-20)40

(1)61 . . . (20)80

(1-10)9 (11-20)10

(1-5)17 (6-10)18 (11-15)19 (15-20)20

(1-2)41 . . . (19-20)50

(1)81 . . . (20)100

To systematize the scrambling technique for the several orders of approximation, each separate segment from each of the 5 passages was typed on 5 different cards. The cards were originally stacked as shown in Table 7.1. Each distinct segment of text (and therefore each cloze item) appeared exactly once in each of the 5 respective orders of approximation to normal prose (i.e., one time each in the 100th+, 50th, 25th, 10th, and 5th orders of approximation). The 500 items in Table 7.1 are represented as ranges of numbers in parentheses on the columns; passages are shown as Roman numerals on the rows; and each distinct card on which a given range of items appeared in a given order of approximation to normal prose based on a specific passage and a particular order of approximation to normal prose is indicated as a separate line in the respective cell of the matrix. The number of cards in each stack is indicated by the number of lines appearing in each column of Table 7.1 as shown in the subscripts attached to lines. Reading the table from top to bottom (column by column) and proceeding from left to right (across the rows): (1) The first stack of 5 cards contained 100+ words of text on each card and the segment of text in each case was drawn from one of the 5 different passages (I, II, III, IV, or V; see Appendix 7 at the end of this chapter). Items appearing on the cards in this stack were considered to correspond to the 100th+ order of approximation to normal prose. (2) The second stack contained the same 100 cloze items on 10 cards with each card having exactly 50 words of text on it with 10 cloze items appearing in each 50-word segment. These 50-word segments were identical to the 100+ segments of the previous stack except for the fact that the 50-word segments abruptly began exactly two words before the first cloze item and abruptly ended exactly two words after cloze item 10. The second 50-word segment resumed at the next word and ended abruptly again 2 words after cloze item 20. This procedure was repeated proceeding down the column for each of the 5 passages. (3) The third stack of 10 cards contained the same 100 cloze items as the first two except that the 20 cards in this stack each contained exactly 5 cloze items appearing in a 25-word segment. These 25-word segments were derived by dividing the previous 50-word segments in half. (4) The fourth stack contained 50 cards and each card in that stack had 10 words of text with 2 cloze items inserted in each 10-word segment. These segments were

SEVEN—Cloze, Discourse, and Native Adults 125 obtained by dividing each of the 50-word segments (see column 2 of Table 7.1) into 5 equal segments. (5) The last batch of 100 cards contained exactly one each of the 100 distinct cloze items, and each of the items appeared in exactly 5 words of text. These 5-word segments were obtained by exactly dividing each of the 10-word segments in half. Next, on the basis of the permutations shown in Table 7.2, the stacks were systematically rearranged. Column one, containing the complete passages, was left alone; then, column two was reordered so that the cards from passage II in the 50th condition appeared at the top followed by III, IV, V, and I at the bottom. Each column was similarly rearranged until all of the rows contained exactly 20 items from each of the 5 original passages. Each row in fact contained exactly 100 items, 20 in each of the 5 conditions and each set of 20 was from a different one of the 5 passages. From each 100-item row, a cloze test was constructed. The normal prose passage was placed first, followed by card 20 in the 5th order of approximation for that row, card 10 in the 10th order of approximation, card 4 in the 25th, two in the 50th, nineteen in the 5th, 9 in the 10th, and so forth until only 10 cards in the 5th condition remained. Then the drawing continued first from the top of the stack then from the bottom until all cards were used up. Thus, items 1-20 on cloze test 1 were the 20 items ranging over the normal prose of passage I; item 21 was the last item from passage V; items 22-23 were the next to the last and last item from passage IV; and so forth. In each separate test there were 100 different cloze items: there were 20 items in the normal prose context (that is, a lead-in sentence followed by 100 consecutive words with 20 cloze blanks inserted followed by at least 5 words of lead-out ending at a period); and 20 items each in the 4 other conditions shuffled together as described earlier. Table 7.2 Permutations of Passages Over Orders of Approximation in Constructing Five 100-Item Cloze Tests Cloze Tests

100th+

50th

25th

10th

5th

Test 1

I

II

III

IV

V

Test 2

II

III

IV

V

I

Test 3

III

IV

V

I

II

Test 4

IV

V

I

II

III

Test 5

V

I

II

III

IV

Subjects Since the objective of the experiment was to contrast the test conditions rather than subjects, the only demographic data asked of subjects was their native language. Ninety-three adult native speakers of English each completed one of the 5 cloze tests. Subjects were drawn from introductory linguistics and English classes at the University of New Mexico and from a group of participants at the Sixth Annual Michigan Conference on Applied Linguistics in February 1975. Subjects ranged in age from the late teens to over 50. There was a nearly equal balance of males and females.

SEVEN—Cloze, Discourse, and Native Adults 126

Procedure The tests were administered to randomly selected subgroups of approximately equal numbers on each occasion by pre-stacking the tests in the order of 1,2,3,4,5; 1,2,3,4,5; 1,2,... The tests were handed out in this same order, thus ensuring an equal chance for any given subject being assigned to any one of the 5 tests 〈〉. The first subject and every fifth thereafter received Test 1, the second subject and every fifth + 1 received Test 2, and so forth. Thus the possibility of comparing answers was discouraged and the assignment of subjects to groups was randomized. Tests were scored by two methods. First, the exact-word scoring method was used, and second, words other than the exact-word which fit the total context of the original passage were counted correct. The latter method is referred to here as the ‘contextually acceptable’ (A) criterion and is contrasted with the ‘exact-word’ (E) criterion.

Results and Discussion

Figure 7.1. Contrasts over all passages.

SEVEN—Cloze, Discourse, and Native Adults 127 If the hypothesis stated at the outset were correct, it should be easier to guess the missing words in the optimal context condition (that is, the 100th+ order of approximation to normal prose) than in any of the other conditions which in fact should be ordered in descending rank from 50th, to 25th, to 10th, to 5th. Figure 7.1 shows that this prediction is sustained for all conditions and for both the exact-word and contextual scoring methods provided all groups/tests and all passages are considered. Table 7.3 Means and Grand Means for Cloze Tests (i.e, Subject Groups) and Orders of Approximation with Both the Exact-Word (E) and Contextually Acceptable (A) Scoring Criteria Test/Group

Scoring

100th+

50th

25th

10th

5th

Test Means

A

17.5

12.3

9.6

5.8

6.3

10.3

E

8.3

4.2

6.0

4.2

4.6

5.6

A

15.5

13.6

11.1

10.5

5.8

11.3

E

5.3

7.4

8.1

6.3

3.0

6.0

A

16.3

12.5

12.6

9.7

4.2

11.1

E

8.5

9.0

8.4

4.3

1.3

6.3

A

16.6

14.4

12.4

8.2

6.0

11.5

E

12.7

8.3

5.5

3.6

4.3

6.9

A

19.2

15.3

11.5

6.8

5.3

11.6

E

11.3

6.5

4.3

4.9

3.9

6.2

A

17.0

13.6

11.4

8.2

5.5

11.6

E

9.2

7.1

6.5

4.7

3.4

6.2

1 (n = 20)

2 (n = 19)

3 (n = 19)

4 (n = 18)

5 (n = 17)

Grand Means for Orders

In Table 7.3, the means over each condition for each test and grand means over both tests and conditions are given. It is clear without going into any further analysis that the main differences are due to conditions rather than to any differences that may have existed between the tests/ groups. This can be seen by reading across the last two rows and down the last column. The last column presents means over conditions and shows differences between groups/tests. The 5 distinct cloze tests and the 5 groups tested are completely confounded (intentionally), but it is apparent that groups/tests are not contributing much to the contrasts in the data. However, it is possible to look at the passages independently of groups/tests by rearranging the means in accord with the permutations listed in Table 7.2. The result of such an

SEVEN—Cloze, Discourse, and Native Adults 128 arrangement is given in Table 7.4. (Of course, the grand means for the columns would be the same as in Table 7.3 so they are not given in Table 7.4.) Table 7.4 Means Rearranged by Passages Passage I

Scoring

100th+

50th

25th

10th

5th

Means

A

17.5

15.3

12.4

9.7

5.8

12.1

E

8.3

6.5

5.5

4.3

3.0

5.5

A

15.5

12.3

11.5

8.2

4.2

10.3

E

5.3

4.2

4.3

3.6

1.3

3.7

A

16.3

13.6

9.6

6.8

6.0

10.5

E

8.5

7.4

6.0

4.9

4.3

6.2

A

16.6

12.5

11.1

5.8

5.3

10.3

E

12.7

9.0

8.1

4.2

3.9

7.6

A

19.2

14.4

12.6

10.5

6.3

12.6

E

11.3

8.3

8.4

6.3

4.6

7.8

II

III

IV

V

Figures 7.2-7.6 display graphically the contrasts between means for Table 7.4. From the figures it is clear that even on individual passages, the contrasts between means generally conform to the experimental hypothesis in spite of the fact that the contrasts are confounded by any differences between groups/tests. There actually are two reversals in predicted contrasts. The contrasts between the 50th and 25th orders of approximation favor the 25th over the 50th for the exact word scoring on passages II and V by .1 in each case (see Figures 7.3 and 7.6). A two-way analysis of variance with the 5 groups/tests as the first variable and the 5 conditions as the second variable was computed for both scoring methods. For the exact word scoring, the condition Figure 7.2. Passage I. contrasts were significant with F 1, 4 = 1560, p < .001 and the interaction between the groups/tests variable and the conditions variable was also significant, F 16, 352 = 30, p < .001. However, the contrast between groups/ tests as expected (due to the randomization of subject assignment to groups/ tests) failed to reach significance at the p < .05 level, F 4, 88 = 2.4, p > .05 . By the more lenient scoring method the result was similar. The overall effect of conditions was significant, F 1, 91 = 2723, p < .001 and the interaction between Figure 7.3. Passage II. groups/tests by conditions was also significant, F 16, 352 = 8.31, p < .001. Contrasts between groups/tests however (again as expected), failed to reach significance at the p < .05 level, F 4, 88 = 1.20, p > .31.

The fact that subjects attained higher scores on cloze items embedded in higher order contexts than on the same items embedded in lower order contexts is merely another special demonstration of a very general fact—namely, that long-range context (beyond five to ten words on either side of a blank is important to the processing of verbal material 〈〉. A number of other studies have provided other special demonstrations of this fact. For example, Dooling and Lachman (1971) showed that understanding the topic of a passage of prose aided in the recall of the words used in the passage. Bransford and Johnson (1972) and Dooling and Mullet (1973) demonstrated that advance knowledge of the topic or ‘theme’ of a passage of prose aids in the retention of the passage whereas knowledge of the topic after presentation does not improve retention. Thus it is clearly during the storage process that knowledge of the topic is helpful. This conclusion Figure 7.4. Passage III. accords well with the main hypothesis stated at the outset in this paper concerning grammar-based expectancies. If the language user is able to anticipate information to be presented or at least the limits of the range of meanings, processing will thus be facilitated. Johnson, Doll, Bransford, and Lapinski (1974) showed that this is true even when the material to be presented and tested for recall consists of short, referentially vague sentences—e.g., “ʽThe crowd expectantly waited for the steam (geyser); ‘He kicked twice but got no change’ (vending machine); ‘The eye is comparatively calm’ (hurricane); and ‘The bird was too small for the family’ (Thanksgiving dinner)” (p. 358). Using a different experimental technique, Klein and Klein (1973) showed that performance on a word recognition task was enhanced as contextual constraints were increased. Word sequences were typed without punctuation or spacing. Subjects were required to segment the words within a limited amount of time. Sequences presented to subjects included 1st, 3rd, and 6th orders of approximation and normal prose. (The orders of approximation were generated by the Miller-Selfridge method.) Subjects also were presented with sequences that were ungrammatical, or anomalous, in relation to normal grammatical prose. In all cases contrasts favored the material that conformed more closely to normal prose. In accord with the Coleman (1963) argument, however, there was no significant contrast between the 3rd and 6th orders. As Coleman insisted, this lack of contrast is probably due Figure 7.5. Passage IV. to the inherent strangeness of the material at the 6th order (and beyond) generated by the Miller-Selfridge method. In conclusion, the data from the present study show that the cloze procedure is sensitive to discourse constraints, and the possible point of diminishing returns for increasing orders of approximation must be well beyond 50 words of context, not at 5 or 6 words, as previously argued by Miller and Selfridge (1950). Neither are cloze items generally limited in sensitivity to material within the immediate sentence context as claimed by MacGinitie (1961) or by Carroll (1972) . A.Key Points and Comments〈 The expectancy hypothesis, of course, does not derive from any statistical evidence from applications of cloze procedure. It derives from a theory of human intelligence—more particularly, semiotic and linguistic capacities. A.〈 Linguists ranked 6th and 7th order approximations to normal prose as less

Figure 7.6. Passage V.

SEVEN—Cloze, Discourse, and Native Adults 130 grammatical than 5th order approximations on account of the strangeness of the Miller-Selfridge method. By the time 6 or 7 people had each contributed a separate word to the 6 or 7-word string, it had become bizarre. B.〈 Coleman found that tasks scored by the number of words recalled in their original sequence favored sensible prose over the several orders of approximation (increasingly nonsensical sequences) generated by the Miller-Selfridge method. C.〈 The method of generating approximations to normal connected prose by starting with normal discourse and cutting it into segments of varying lengths and scrambling them systematically guarantees that longer segments will in fact approximate more and more closely the original text(s) up to the limit of being that original text. D.〈 The procedure of constructing tests and distributing them as described here virtually guarantees that variance attributable to the first position (authors of the 5 passages in question), as well as the second position (the various subject-groups tested) will be reduced to nothing more than unsystematic error just so long as we consider all the passages in all the various conditions of presentation. The advantage is to focus attention on variance owed to the character of the contexts in which the cloze items themselves are embedded (this last sort of variance, incidentally, is abstracted third position variance owed to long-range contextual constraints). E.〈 When the undesirable variance attributable to differences between passages (first position variance) which is confounded by whatever differences may exist between subject-groups/tests (second position variance) are equally distributed over all tests and conditions, a clear difference (third position variance) emerges between orders of approximation as predicted.

A.Research Questions and/or Applications Does it not seem strange to suggest that a sequence of nonsensical phrases will be as easy to recall as the elements of a sensible story? For instance, imagine trying to recall a random sequence of phrases, say, as long as the screen play for a full-length feature film. Why is it that we are certain in advance that reasonably intelligent people will do better with, say, two hours worth of sensible material than with a random arrangement of phrases or sentences? B. Notice the oddity of beginning a string with a pronoun or a definite article when the user has no idea of who or what ‘he’, ‘she’, ‘it’, or ‘the’, might conceivably indicate. Discuss the pragmatic problem presented by the Miller-Selfridge method of generating approximations to connected discourse. Why is it that such a method can never actually approximate what we do when we use language meaningfully? C. Consider why it is essential to demonstrate that the contrasts between groups/ tests on the whole are negligible. Examine experiments in the cloze literature. Pick out one that does not control group/test variance and critique or redesign it. D. Suppose it is argued that contrasts are due to false expectations set up by the scrambling rather than the mere removal of existing constraints. Does this weigh in favor of or against the sensitivity of cloze items to long-range discourse constraints? What about the fact that sentence boundaries are ignored in the cutting and scrambling procedure used in this study (also see Chapter 6 above by Ramanauskas and Chapter 8 immediately below by Chihara, et al.)? Is this an advantage or disadvantage to the procedure?

SEVEN—Cloze, Discourse, and Native Adults 131

Appendix 7 Passage I4 For those who hang out there, the Carry-out offers a wide array of sounds, sights, smells, tastes, and tactile experiences which titillate and sometimes assault the five senses. ∥ The air (1) IS warmed by ∣ smells from (2) THE coffee urns ∣ and grill (3) AND thickened with ∣ fat from (4) THE deep-fry basket. ∣ The jukebox (5) OFFERS up a ∣ wide variety (6) OF frenetic and ∣ lazy rhythms. (7) THE pinball machine ∣ is a (8) STANDING challenge to ∣ one’s manipulative (9) SKILL or ability ∣ to will (10) THE ball into ∣ one or (11) ANOTHER hole. Flashing ∣ lights, bells (12) AND buzzers report ∣ progress or (13) ANNOUNCE failure. Colorful ∣ signs exhort (14) CUSTOMERS to drink ∣ Royal Crown Cola and (15) EAT Bond Bread. On ∣ the wall, (16) ABOVE the telephone, ∣ a long-legged (17) BLOND in shorts ∣ and halter (18) SMILES s21 a fixed ∣ wet-lipped smile (19) OF unutterable delight ∣ at her Chesterfield (20) CIGARETTE, her visage ∥ unmarred by a mustache or scribbled obscenities. In the background, a sleek ocean liner rides a flat blue sea to an unknown destination (Gorrell, Laird, and Freemen, 1970, p. 29).

Passage II It is possible to get an education at a university. ∥ It has (1) BEEN done; not ∣ often, but (2) THE fact that ∣ a proportion (3) HOWEVER small, of ∣ college students (4) DO get a ∣ start in (5) INTERESTED, methodical study, ∣ proves my (6) THESIS, and the ∣ two personal (7) EXPERIENCES I have ∣ to offer (8) ILLUSTRATE it and ∣ show how (9) TO circumvent the ∣ faculty, the (10) OTHER students, and ∣ the whole (11) COLLEGE system of ∣ mindfixing. My (12) METHOD might lose ∣ a boy (13) HIS degree, but ∣ a degree (14) IS not worth ∣ so much (15) AS the capacity ∣ and the (16) DRIVE to learn, ∣ and the (17) UNDERGRADUATE desire for ∣ an empty (18) BACCALAUREATE is one ∣ of the (19) HOLDS the educational ∣ system has (20) ON students. Wise ∥ students some day will refuse to take degrees, as the best men (in England, for instance) give, but do not themselves accept, titles (Gorrell, Laird, and Freemen, 1970, pp. 33-34).

Passage III The statement of the main idea, discussed in the preceding chapter, helps the writer organize the subdivisions of his paper. ∥ As he (1) WRITES, he also ∣ recognizes and (2) UNIFIES material within ∣ the subdivisions, (3) USING the paragraph ∣ as his (4) UNIT of composition. ∣ Paragraphs can (5) BE written in ∣ many ways, (6) BUT one basic ∣ type is (7) SO useful for ∣ expository writing (8) THAT we are ∣ calling it (9) THE standard paragraph. ∣ It includes (10) THE following: (1) topical ∣ material to (11) INTRODUCE the subject; ∣ (2) development to (12) ILLUSTRATE or support ∣ or extend (13) THE subject, material ∣ that may (14) BREAK into divisions ∣ or even (15) SUBDIVISIONS; and (3) sometimes ∣ a conclusion.

In this Appendix, double bars mark the beginnings and ends of the 100-word segments while single

4

bars mark the boundaries between 5-word segments. There were six cases where proper nouns were counted as one word. For instance, in Passage I, “Royal Crown Cola” was counted as one word; also “Bond Bread”. In Passage III, parenthesized numbers were not counted as words. In Passage V, “Edward Travers” was counted as one word on its first appearance, and “Sir Edward Travers” was counted as one word on the next appearance. “Ralph Bodrean” and “Keverall Court” were each counted as one word. Otherwise, blanks were spaced with exactly four words between them in each case. Hyphenated forms were, as is customary in cloze applications, counted as one word.

SEVEN—Cloze, Discourse, and Native Adults 132 (16) USING this pattern, ∣ the most (17) OBVIOUS and most ∣ useful way (18) TO construct a ∣ paragraph is (19) TO write a ∣ topic sentence (20) AND add specification ∥ to support it (Gorrell, Laird, and Freemen, 1970, p. 26).

Passage IV On some occasions, a single word—“Fire!” or “Murder!” for example—is a complete message; context and tone of voice supply the unspoken information. ∥ Usually, however, (1) COMMUNICATION requires more, ∣ that we (2) NOT only name ∣ a topic (3) BUT say something ∣ about it—“(4) THE fire is ∣ spreading” or “(5) THE fire was ∣ caused by (6) FAULTY wiring”. A ∣ simple sentence, (7) THE horse is ∣ eating, gets (8) UTTERED not because ∣ we think (9) OF the topic ∣ horse and (10) THEN search for ∣ something to (11) SAY about it ∣ but because (12) WE want to ∣ report on (13) WHAT we have ∣ observed, want (14) TO say something ∣ about the (15) HORSE. If we ∣ have a (16) GREAT deal more ∣ to say (17) ABOUT s21 a horse, ∣ the result (18) MAY be, not ∣ a simple (19) SENTENCE but an ∣ article or (20) A book. Writing ∥ a simple sentence or a longer composition includes at least a topic and a comment about it, although the line between the two may not be precise (Gorrell, Laird, and Freemen, 1970, p. 1).

Passage V When Sir Edward Travers died suddenly and mysteriously there was consternation and speculation, not only in our neighborhood but throughout the country. ∥ One newspaper (1) HEADLINE ran: SIR ∣ EDWARD TRAVERS VICTIM (2) OF CURSE? Another ∣ reported: SUDDEN (3) DEATH OF EMINENT ∣ ARCHAEOLOGIST BRINGS (4) ABRUPT END TO ∣ EXPEDITION. A (5) PARAGRAPH in our ∣ local paper (6) STATED, “The death ∣ of Sir Edward Travers, (7) WHO recently left ∣ this country (8) TO carry out ∣ excavations among (9) THE tombs of ∣ the pharaohs, (10) HAS caused us ∣ to wonder (11) IF there is ∣ any truth (12) IN the ancient ∣ belief that (13) HE who meddles ∣ with the (14) RESTING place of ∣ the dead (15) INVITES their enmity.” ∣ Sir Ralph Bodrean (16) AT Keverall Court, our ∣ local squire (17) AND Sir Edward’s ∣ closest friend,(18) HAD given financial ∣ aid to (19) THE expedition, and ∣ when, a (20) FEW days after ∥ the announcement of Sir Edward’s death, Sir Ralph had a stroke, it was hinted that his misfortune was the result of the same curse (Holt, 1973, p. 325).