Cognitive Neuropsychology Assessing semantic ...

This article was downloaded by: [the Bodleian Libraries of the University of Oxford] On: 02 May 2014, At: 08:04 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Cognitive Neuropsychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/pcgn20

Assessing semantic comprehension: Methodological considerations, and a new clinical test a

Dorothy Bishop & Sally Byng a

b

The University , Newcastle-upon-Tyne, U.K.

b

Birkbeck College, University of London , U.K. Published online: 16 Aug 2007.

To cite this article: Dorothy Bishop & Sally Byng (1984) Assessing semantic comprehension: Methodological considerations, and a new clinical test, Cognitive Neuropsychology, 1:3, 233-244 To link to this article: http://dx.doi.org/10.1080/02643298408252024

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

COGNITIVE NEUROPSYCHOLOGY, 1984, 1(3) 233-243

Assessing Semantic Comprehension: Methodological Considerations, and a New Clinical Test Downloaded by [the Bodleian Libraries of the University of Oxford] at 08:04 02 May 2014

Dorothy Bishop The University, Newcastle-upon -Tyne, U.K.

Sally Byng Birkbeck College, University of London, U.K. Most tests designed to assess semantic comprehensioninvolve confronting the patient with an array of pictures including a target and a set of semanticallyrelated items, and asking him to select the one which matches a spoken word. These tests are defective on several counts. First, they do not give the patient the opportunity to make errors other than semantic errors, so that a patient who makes errors because of auditory perceptual problems, or because of nonlinguistic factors such as reluctance to scan the full array, will appear to have a “semantic” disorder. Second, many of the semantic distractors used in such tests are visually similar to the target (e.g., knife and spoon, sheep and goat), so that errors might arise for perceptual rather than linguistic reasons. The test materials described here have proved useful in distinguishingreceptive semantic problems from perceptual deficits.

INTRODUCTION In the 1950s and 1960s, analysis of the linguistic disorders of dysphasic patients went little beyond drawing a broad distinction between receptive and expressive dysphasia. In the 1970s, however, aphasiologists began to recognise that by applying concepts from linguistics to the study of their patients they could obtain new insights into the nature of language disorders, and go far beyond the old receptive/expressive distinction. In particular, psychologists and linguists became interested in considering separately

Requests for reprints should be sent to D. V. M. Bishop, Department of Speech, The University, Newcastle-upon-Tyne, NEI 7RU. The authors would like to thank Professor Brian Matthews and Dr Freda Newcombe, from the Department of Neurology, Radcliffe Infirmary, Oxford, and the Speech Therapy Department, Queen Mary Hospital, Sidcup, Kent, who enabled the authors to see patients attending their departments for assessment or treatment. This work was carried out when the first author was working at the Neuropsychology Unit, Oxford, and was supported by MRC grant no. G973/144/C.

0 1984 Lawrence Erlbaum CN 1.3--8

Associates Limited

Downloaded by [the Bodleian Libraries of the University of Oxford] at 08:04 02 May 2014

234

BISHOP AND BYNG

disorders of phonology, semantics and syntax in both receptive and expressive language. Specific interest was kindled in the question of whether one could find evidence for selective breakdown in one component of language. For example, could one reconceptualise Broca’s dysphasia as a phonological and/or syntactic disorder, and Wernicke’s dysphasia as a semantic disorder? Such new concepts required new means of assessment, and in 1970 Pizzamiglio and Parisi reported three new Italian comprehension tests designed to assess understanding of phonology, semantics and syntax. All three tests used a multiple-choice format in which the patient listened to a word or sentence and had to select from an array the picture which corresponded to the spoken message. In the phonological test, the distractors were words of similar sound, in the semantic test they were words of similar meaning, and in the syntactic test the distractors corresponded to sentences differing from the test sentence in terms of word order, function word or inflection. Results obtained by Parisi and Pizzamiglio (1970) and Pizzamiglio and Appicciafuoco (197 1) suggested that dysphasic disorders did not neatly correspond to selective breakdowns in one component of language. Scores on the three tests were positively correlated, so that a patient who had a comprehension deficit on one test tended to be impaired on the others. Furthermore, the difference between Broca’s and Wernicke’s dysphasics seemed to be one of degree rather than pattern of comprehension disorder. Such results suggested that the fashionable linguistic approach to dysphasia might be less useful than supposed in characterising language disorder, and that the old view of receptive versus expressive disorders might correspond more closely to what was actually observed. Lesser (1974) gave English versions of the Italian tests to a group of dysphasic patients and a group of patients with right-hemisphere lesions. She found the dysphasic patients had clearcut impairments on all tests, but she also found a smaller degree of impairment on the semantic test in patients with right hemisphere lesions. Further evidence for impaired semantic comprehension with right hemisphere lesions was found by Gainotti, Caltagirone, Miceli and Masullo (198 1). These studies suggested two important conclusions. First, it seemed that understanding of word meaning need not be all or none. When confronted with a target word, patients may select a word of related meaning. This suggests that the patient might retain an association between a phonetic string (i.e., a word) and the concept it represents, but that either the concept itself, or the association between word and concept, is degraded, so that the meaning associated with the word is less rich and more generalised than in a normal person. The work of Pizzamiglio and Appicciafuoco (197 1) suggested that such errors are very common in aphasics. Only six of their sixty aphasic patients scored within normal limits on their Semantic Comprehension test. Second, the studies by Lesser (1974) and Gainotti et al. (1981) appeared to indicate that such degradation of word meaning may occur even when the patient is not overtly dysphasic. The deficit found in patients with right hemisphere lesions has


SEMANTIC COMPREHENSION

235

been interpreted in terms of right hemisphere involvement in representation of word meaning. However, the validity of such conclusions rests on the validity of the tests used to assess semantic comprehension, and there are several reasons for questioning this. The Italian tests all suffer from a fundamental design fault in that the patient has no opportunity to make any errors other than those the test is designed to measure. For example, in the semantic test, all distractors are semantically related to the target: if the patient makes an error, then this is bound to be a semantic error. Suppose the patient had a receptive phonological disorder, so that he had difficulty in accurately perceiving spoken words. In such a case, we would expect the patient to make many errors, so on this “semantic comprehension test” he would have a deficit. It would be rash, however, to conclude on the basis of such evidence that the patient had a selective difficulty in discriminating between words close in meaning. One could only draw such a conclusion if the patient reliably confused target words with semantically close distractors, but seldom confused words which were very different in meaning. Similar logic can be applied to the other Italian tests. Thus we would expect a patient who had a genuine problem with understanding word meaning to do poorly on both the “phonological test” (where he has to match a spoken word to a picture: he may perceive the phonological string quite adequately but not know what it means), and the “syntactic test” (where failure could arise because the patient does not know the meaning of the content words, even if he has a good grasp of syntax). Considered in this light, it is hardly surprising that deficits on the three tests are highly correlated: they are simply far less pure measures of specific skills than they appear. As Lesser (1974) points out, even generalised, nonlinguistic deficits might result in apparently specific “semantic” errors. She noted that some of her subjects failed to scan the array of pictures in detail. In her task, some of the “semantic distractors” could be regarded as poor exemplars of the target word; e.g., one of the distractors for the word “paper” was the word “write”, depicted by a pen writing, presumably on paper. A patient who did not consider all pictures, but stopped scanning when he found one which seemed a possible response might make many “semantic errrors” for perceptual or motivational reasons, rather than because of linguistic disability. If one is to distinguish between generalised linguistic or nonlinguistic disorders, and specific semantic difficulties, then two measures must be taken. First, each test item should include unrelated as well as semanticallyrelated distractors. One can then tell from the nature of the patient’s errors whether the problem is specifically semantic. Second, the distractors should be totally unambiguous, so that the correct picture is not simply the best exemplar of the target word in the array, but it is the only picture in the array which could be a valid response to the target word. Even with these controls, one difficulty would remain. Pictures of objects


236

BISHOP AND BYNG

close in meaning are often also similar in appearance. For example, dog and sheep, cup and mug, coat and jacket, knife and spoon are not only similar in meaning but also are similar in appearance. The findings of impaired semantic comprehension in patients with right hemisphere lesions could have a different explanation from the impairment seen in those with left hemisphere lesions. Visual perceptual problems are associated with right hemisphere lesions, and it could be that the errors made by such patients on the “semantic test” are perceptual rather than linguistic. All semantic comprehension tests reported in the literature fail to take this into account, and so run the risk of confounding linguistic and perceptual deficits. We report here the development of materials for testing semantic comprehension in a way which overcomes these methodological problems.

METHOD Materials The test of Lexical Understanding with Visual and Semantic Distractors (LUVS) is a multiple choice test in which the patient is confronted with an array of four or eight pictures, and must point to the one which corresponds to a target word. The test was designed so that the same 80 picture arrays could be used for either of two parallel forms. (This enables one to compare performance depending on whether the target word is spoken or written). For each form, the test is divided into four sets, each of 20 picture arrays, as follows: Set V (visual): the array includes one distractor picture designed to be visually, but not semantically, confusable with the target. See Table 1 for examples. Set S (semantic): the array includes a distractor which is semantically related to the target, but which looks quite dissimilar when pictured. Set SV (semantic-visual): the array includes a distractor which is both semantically and visually similar to the target. Set R (random): all distractors are unrelated to the target. The four word sets were matched on the mean frequency of the target word, and on the range of frequencies of target words, using the Carroll, Davies and Richman (1971) word frequency data. Visual relatedness of words in the V and SV set was judged subjectively. A reaction time experiment, carried out with 14 normal subjects, provided objective evidence of the semantic relatedness of the target-distractor pairs in the S and SV conditions’. ‘In this experiment, the subject’s task was to decide whether a pair of written words, shown on a VDU, were synonymous, by pressing a “yes” or “no” key. The experiment was controlled by an Apple I1 microcomputer which automatically timed responses and randomised the


237

TABLE 1 Targets and Distractors used in Form A of LUVS


Visual thistle/shaving brush scre.wdriver/sword radiator/fence butterffylbow hammer/totem-pole balloon/magnifying glass flag/axe wheel/lifebe.lt bed/trunk purse/shell pliers/tuning fork snowman/panda bottle/lighthouse funnel/cone button/coin drum/sieve pipe/axe boat/cradle gun/drill globe/shaving mirror

Semantic nunlchurch octopus/seahorse hose/watering can apple/banana bird/feather tiger/snake arrow/Indian needle/thread chair/desk finger/ring igloo/Eskimo pineapple/chemes stool/armchair caterpillar/butterfly violin/trumpet glasses/eye candle/lightbulb star/moon key/padlock bab&ram

Semantic

+ Visual

slug/centipede pear/lemon worm/caterpillar necklace/ring jacket/coat donkey/zebra cup/basin lion/bear radio/television cat/fox swan/duck dartlarrow giraffe/gazelle knee/arm parrot/owl nail/screw goat/sheep child/teddy rose/daisy piano/x yiophone

A short experiment was carried out on 14 normal subjects to assess the degree of semantic relatedness of the word sets used in LUVS. In each array the pictures were arranged in an oval, with the position of target and distractor being controlled so that both were equally likely to occur in each of the eight positions. The same spatial arrangement was used for four-choice arrays, with four of the spaces (randomly selected) being left blank. A sample array is shown in Fig. 1. sequence of stimuli. The positive stimuli (to which subjects were required to press a “yes” key) were 160 synonym pairs, whilst the negative stimuli (to which subjects were required to press a “no” key) were the total set of 160 target-distractor pairs from LUVS (20 from each condition in each parallel form). Mean latencies of correct “no” responses, and percentages of erroneous “yes” responses were compared across the four types of negative item (i.e. V, S, SV and R). The latencies (response times) for correct “no” responses differed significantly across the four sets of items (mean response time: V-items = 1439ms; S-items = 1495ms; SV-items = 1527ms; R-items = 1375ms. F = 3.06; df = 3, 36; p < 0.05). A Neuman-Keuls test showed that the latency for SV items was significantly greater than for R items. The percentage of correct responses also differed between item types (mean percentage correct: V-items = 95%; Sitems = 87% SV-items = 82%; R-items = 99%. F = 8.46; df = 3, 39; P < 0.01). The percentage correct was significantly lower for both S and SV items compared to R items, but V and R items did not differ significantly, and the difference between S and SV items was not significant (Neuman-Keuls test). To sum up, if one regards mean latency of correct responses, and percentage of correct responses in this synonym judgement task as measures of degree of semantic relatedness of word pairs, then both S and SV word sets are more strongly reIated than V and R sets, but S and SV sets do not differ in degree of semantic relatedness.


238

BISHOP AND BYNG

FIG. 1 . Sample item from LUVS. Form A: S-item: target = “pineapple”; distractor = “cherries” Form B: V-item: target = “parachute”; distractor = “umbrella”.

Procedure Patients were told that they would see a set of pictures, and should point to the one which corresponded to a word spoken by the tester. (In the written


239


condition, they were told to point to the picture which matched the written word.) Each patient was given three practice arrays, followed by the LUVS test arrays presented in random order. When spoken presentation was used, the tester spoke the target word while turning the page to present the array. When written presentation was used, the target word was written in the centre of each array.

Patients Tested Using LUVS Two types of studies have been conducted with LUVS. First, the test has been given to a group of patients who were selected on the basis of their having unilateral cerebral lesions. This group consisted of 31 men who had sustained penetrating gunshot wounds of the brain some 35-40 years previously. A control group of 10 patients undergoing minor surgery was also tested. This latter group was given both forms of the test, once with written presentation of target words, and once with spoken presentation. The second set of studies consisted of individual case studies of patients who might be expected to have disorders of semantic comprehension, because they were suffering from dysphasia, dyslexia, or visual agnosia.

ResuIts 1. Group Study. Errors in each condition were totalled for each patient. If the patient spontaneously corrected an error, this was counted as half an error. The data are shown in Table 2, with scores rounded to the nearest whole error. It is clear that very few errors are made by either neurological or control patients, and those errors that do occur tend to occur on V and SV items, rather than S items. The three groups did not differ significantly in error rate, and there was no interaction between groups and conditions. Response latencies were measured in this study, and it was found that although the patients were in general slower than the controls, there was no interaction between groups and conditions: i.e., there was no evidence that any type of distractor was particularly difficult for one group. Ten of the patients in the left hemisphere group had had a history of dysphasia at time of injury but were no longer dysphasic. Three patients had persisting dysphasia, which was severe in one case. The patient with severe dysphasia was the only patient to make an error on the S condition, though this was the only error he made. The two patients with milder dysphasia were among those left hemisphere patients who made three errors on the SV condition. The performance of the control patients was identical on written and spoken versions of the test. 2. Single Case Studies. Results from single case studies are shown in Table 3.

240

BISHOP AND BYNG TABLE 2 Distribution of Errors on LUVS

Left Hemisphere Condition

Lesion (n = 17)

Right Hemisphere Lesion (n = 16)

Control

(n = 10)


V

0 errors

12

1 error 2 errors

4 1

S 0 errors 1 error

16 1

11 5

0

10 0 0

16 0

10

9

7

6 2

2 0

0

16 0

10 0

0

sv 0 errors 1 error 2 errors 3 errors R 0 errors 1 error

16 1

2

DISCUSSION Considering first the results from the group studies, the one striking finding is the absence of semantic comprehension problems in patients with lateralised cerebral lesions. It had been anticipated that one might find an increase in errors or a slowing of responses to S and SV items, certainly among those dysphasic patients with left hemisphere lesions, and perhaps also among nondysphasic patients with lesions of either hemisphere. In this study, there was no evidence of any impairment on LUVS in patients with right hemisphere lesions, and only a nonsignificant trend for patients with left hemisphere lesions to be impaired on items which were both semantically and visually confusing. This result contrasts with the studies of Lesser (1974) and Gainotti et al. (1981), who found evidence of impaired semantic comprehension, not only among dysphasics, but also among nondysphasic patients with right hemisphere lesions. Two conclusions seem possible. Either these patients do have a real semantic Comprehension deficit, and the test used here was not sensitive enough to detect it, or specific semantic comprehension problems are rare in brain-damaged patients, and the apparently high incidence of such problems reported by previous studies is an artifact arising from the tendency to confound general with specific impairments, and visual with semantic confusability. Consider first the possibility that the patients do have a real semantic comprehension deficit, but that it is not detected by this task. It could be


241

TABLE 3 Single Case Results With LUVS

Errors Patient

V

S

S

V

R

Source


~

GR: deep dyslexia/Broca’s aphasia spoken test written test

0 0

2 7

2 6

0 2

D E deep dyslexia/Broca’s aphasia spoken test written test

0 0

0 0

0 0

0 0

spoken test written test

0 0

0 0

0 0

0 0

JG: deep dyslexia/Broca’s aphasia spoken test written test

0 1

3 0

2 3

0 0

0 0

2 1

1

1

0 0

0

0 0

0

0

0

1

0

0 0

3

4

0

2

4

0

2 2

3

0 0

Newcornbe & Marshall (1980) Patterson & Besner (in press)

PW:deep dyslexia/Broca’s aphasia

Bishop & Byng (this paper)

VS: deep dyslexia/Broca’s aphasia spoken test written test

R W deep dyslexia/fluent aphasia spoken test written test BN: deep dyslexia/fluent aphasia spoken test written test

R€?deep dyslexia/global aphasia spoken test written test M C global aphasia spoken test

R F global aphasia spoken test written test BB: global aphasia spoken test written test

Patterson (1981) 0

0

4

MS: visual agnosic spoken test

Ratcliff & Newcombe (unpub.)

J R visual agnosic spoken test

Davidoff & Wilson (unpub.)

On all items in the V, S, and SV conditions the patient could make either a related error (e.g., selecting “bird” for “feather”), or a random error (e.g., selecting “bicycle” for “trumpet”). The probability of such errors occurring by chance was 1 in 4 (4-choice items) or 1 in 8 @-choice items) for related errors, and 2 in 4 or 6 in 8 for random errors. Most patients made no random errors. Where they did so, the number of random errors is shown in brackets after the number of related errors.


242

BISHOP

A N D BYNG

that the nature of the semantic relationships considered here affected accuracy. The design requirement that the S set of items be visually distinctive, whilst the SV set were not, meant that the majority of the S item pairs were semantic associates (obtained from word association norms), and the majority of the SV item pairs were different members of the same superordinate category (see Table 1). One wonders whether the former type of relationship is less confusing than the latter, and thus there were few errors on the S set. However, two points go against such an explanation. First, Pizzamiglio and Appicciafuoco (197 1) and Lesser (1 974) relied exclusively on word association norms as an index of semantic relatedness in their tasks, and yet found semactic comprehension deficits. Second, both S and SV item sets were associated with slow responses and high error rates in a synonym judgement task used with normal subjects, suggesting they are of equivalent semantic confusability (see above). However, one could nevertheless argue that the items used here were simply not difficult enough. Most concrete nouns which are very close in meaning refer to objects which also look alike. By requiring that items in the S set should be visually distinct, one may be making the test too easy, and insensitive to subtle semantic disturbances. Although one cannot rule out this possibility, it seems less plausible when we consider that patients with left hemisphere lesions not only made very few errors on S items, but also showed no signs of responding more slowly to such items. If a patient had a very mild difficulty in discriminating between words related in meaning, then we would expect this to be reflected in latency scores, even if no errors were made. Turning to the results from single case studies, we find here ample evidence that the test is sensitive to semantic comprehension problems. There is wide variation in individual response patterns. If one considers deep dyslexic patients, it is clear that one cannot generalise about their comprehension. Some of these patients (GR and JG) confuse semantically related words with both written and spoken presentation. Others (VS and RP) have little difficulty with spoken words, but make semantic confusions between written words. The remaining patients who have been studied have no problems with either written or spoken presentation. The results on the three globally dysphasic patients are of especial interest. MC is a patient who would appear to have considerable difficulty with semantically related words if he were tested on one of the traditional “semantic comprehension” tests. However, LUVS suggests that his difficulties are perceptual. He never confused semantically related words when the pictures were visually distinct, whilst he did confuse visually similar but semantically distinct items. R F and BB, however, clearly demonstrate problems of semantic comprehension which cannot be explained away in terms of perceptual factors. LUVS has also proved useful for elucidating the nature of visual agnosia. Both the agnosic patients reported in Table 3 make a substantial number



243

of errors on visually confusable items, as one might expect. However, in both cases there is also a tendency to make semantic errors, even when items are visually distinct, indicating that their disorder is not purely perceptual, but appears to involve confusion between semantic categories. These case studies demonstrate the importance of distinguishing between visual and semantic sources of comprehension errors. It seems likely that previous claims that patients with right hemisphere lesions might have semantic comprehension problems are mistaken, and have arisen because studies have confounded these two types of errors.2

CONCLUSIONS 1. Tests designed to measure “semantic comprehension” frequently confound generalised linguistic or nonlinguistic deficits with poor semantic comprehension, and invariably confound perceptual and linguistic effects on performance. 2. When one uses a test which overcomes these problems, semantic deficits are found to be rare, and there is no support for the view that right hemisphere lesions are associated with semantic deficits.

3. Single case studies with the test indicate that it is useful in pinpointing receptive semantic disorders in patients suffering from conditions such as deep dyslexia.

4. Semantic confusions are also found in patients with visual agnosia, although the main source of errors for such patients is perceptual confusability . Manuscript received 12 March 1984

REFERENCES Carroll, J. B., Davies P., & Richman, B. (1971) Wordfrequency book. New York: American Heritage Pub. Co. Ltd. Gainotti, G., Caltagirone, C., Miceli, G., & Masullo, C. (1981) Selective semantic-lexical impairment of language comprehension in right-brain damaged patients. Brain and Language, 13, 201-211. Lesser, R. (1974) Verbal comprehension in aphasia: An English version of three Italian tests. Cortex, 10, 247-263. Newcombe, F., & Marshall, J. C. (1980) Response monitoring and response blocking in deep dyslexia. In M.Coltheart, K. Patterson, & J. C. Marshall (Eds.), Deep dyslexia. London: Routledge & Kegan Paul. Parisi, D., & Pizzamiglio, L. (1970) Syntactic comprehension in aphasia. Cortex, 6, 204-215. *An updated version of LUVS, suitable for clinical use, is available from Lawrence Erlbaum Associates Ltd, Chancery House, 319 City Road, London, EClV ILJ.

244

BISHOP AND BYNG


Patterson, K. E. (1981) Neuropsychological approaches to the study of reading. British Journal of Psychology, 72, 151-174. Patterson, K. E., & Besner, D. (In press) Is the right hemisphere literate? Cognitive Neuropsychology . Pizzamiglio, L., & Appicciafuoco, A. (1971) Semantic comprehension in aphasia. Journal of Communication Disorders, 3, 280-288. Pizzamigho, L., & Parisi, D. (1970) Studies on verbal comprehension in aphasia. In G. B. Flores d‘Arcais, & W. J. M. Levelt (Eds.), Advances in psycholinguisfics.Amsterdam: North Holland Publishing Co.

REFERENCE NOTES 1. Davidoff, J., & Wilson, B. (submitted for publication)

A case of visual agnosia showing a disorder at post -semantic visual classification. 2. Ratcliff, G., & Newcombe, F. (unpublished). An “apperceptive” deficif in “associative” visual agnosia. Paper presented at 10th Annual Meeting of the International Nei ropsychological Society, Pittsburgh, 1982.