APHASIOLOGY, 2002, 16 (4/5/6), 587–594
Using eye movement responses to index auditory comprehension: An adaptation of the Revised Token Test Brooke Hallowell Ohio University, Athens, OH, USA
Robert T. Wertz VA Tennessee Healthcare System and Vanderbilt University, Nashville, USA
Hans Kruse Ohio University, Athens, OH, USA Background: Tracking spontaneous eye movement responses may improve the accuracy of comprehension assessment in patients with neurological impairments. Eye movement methods provide an on-line response mode that does not require an overt planned motoric response, tax participants’ understanding of instructions, or interrupt the comprehension process with intervening instructions or prompts. Aims: The purpose of this study was to examine an adaptation of a standardised test of auditory comprehension for adults with neurological impairments, using eye movement responses. Individuals with normal language comprehension should demonstrate a consistent pattern of eye movement responses indicating comprehension. Methods & Procedures: Nineteen adults with no history of neurological involvement were presented with auditory comprehension stimuli from the Revised Token Test (RTT) (McNeil & Prescott, 1978) in each of three conditions: traditional, pointing, and eye movement. In the traditional condition, the standardised version of the RTT was administered. In the pointing condition, RTT stimuli were manipulated to fit a multiple-choice format; participants responded to modified verbal stimuli for each item by pointing to one of four images shown in a printed test manual. In the eye movement condition, participants viewed the same stimuli presented in the multiple-choice pointing version, shown on a computer monitor rather than in a test manual. A remote pupil centre/corneal reflection system was used to monitor eye movements. Participants were not instructed to look at anything in particular, and were not told that there were target images within the displays. The eye movement condition was presented first to avoid participants’ perceptions that they were expected to select a target image with their eyes. Dependent variables included the proportion of total viewing time that participants fixated on target images and non-target foils. Outcomes & Results: Traditional RTT scores indicated normal comprehension for all participants, according to published norms. Likewise, pointing condition scores indicated good comprehension. For each of the subtests in the eye movement condition, the proportional amount of time that fixations were allocated to target images significantly exceeded chance expectations. Address correspondenc e to: Brooke Hallowell, PhD, School of Hearing, Speech and Language Sciences, Ohio University, W231 Grover Center, Athens, OH 45701, USA. Email:
[email protected] This work was supported in part by grant number DC00153-01A 1 from the National Institute on Deafness and Other Communication Disorders. The authors wish to thank Pro-Ed, Inc. for permission to use and modify stimuli from the RTT, Dr Malcolm McNeil for helpful suggestions regarding the auditory presentation of RTT stimuli, and James Herpy, Carol Basilio, and Marsha Courtney for assistance in stimulus preparation. # 2002 Psychology Press Ltd http://www.tandf.co.uk/journals/pp/02687038.html
DOI:10.1080/02687030244000121
588
HALLOWELL, WERTZ, KRUSE
Conclusions: The language-normal data reported here suggest that the experimental form of the RTT and the application of the eye movement method yield results consistent with more traditional assessments of auditory comprehension. Thus, application of the method with patients with neurogenic disorders is warranted. Ultimately, the use of eye movement methods may help indicate comprehension abilities in patients whose comprehension status might otherwise be invalidly assessed.
Distinguishing competence from performance may be problematic when appraising communication deficits in patients with neurological impairments. Motoric and perceptual deficits can be confounding factors when traditional linguistic comprehension test items requiring overt verbal, gestural, or graphic responses are administered. Thus, incorrect responding or failure to respond on traditional tests of linguistic comprehension does not necessarily indicate a failure to comprehend. Deficits in linguistic competence, therefore, may be overestimated (Rosenbek, Kent, & LaPointe, 1984). Use of a spontaneous non-volitional eye movement response mode may improve the accuracy of comprehension assessment in patients who are difficult to assess. Development of methods for assessing comprehension through eye movement responses may have important implications for research and clinical practice. Potential advantages of applying eye movement methods are that they: . offer an alternative response mode that requires no talking, writing, or gesturing to provide information about intact comprehension ability in patients who have difficulties in these response modalities; . permit stimulus adaptations to control for perceptual, attentional, and oculomotor deficits; . minimise reliance on participants’ understanding of instructions prior to the presentation of testing stimuli; and . permit a continuous record of processing, simultaneous with the processing task, without interruption of the comprehension process with intervening verbal instructions, prompts for responses, or demands for participants’ conscious planning of responses. Moreover, eye movements are often preserved even in patients with severe motoric and cognitive deficits (Leigh & Zee, 1983). Like some other means of examining auditory comprehension, e.g., the Revised Token Test (McNeil & Prescott, 1978), eye movement methods do not tax auditory memory by requiring participants to wait to respond to an auditory verbal stimulus until the stimulus presentation is completed, and they permit the testing of a range of sentence types, lengths, and grammatical structures. The feasibility of assessing auditory and reading comprehension with eye movement responses has been demonstrated previously (Hallowell, 1999; Hallowell & Katz, 1999). The purpose of the current study was to examine an adaptation of the Revised Token Test (RTT) (McNeil & Prescott, 1978), a standardised test of auditory comprehension for neurologically impaired adults, using eye movement responses in a sample of normal adults. Those with normal language comprehension, as assessed by case history, observation during conversation, and administration of the traditional RTT, should demonstrate a consistent pattern of responses indicating comprehension when assessed via the eye movement version. Demonstration of the method’s validity with data derived from language-normal participants will support subsequent exploration and response comparisons with patient populations (McCauley & Swisher, 1984).
EYE MOVEMENTS TO INDEX COMPREHENSION
589
METHODS Participants A total of 19 adults, 11 males and 8 females, with no history of neurological involvement served as participants in the study following a process of informed consent. Participants were selected randomly from a list of those who responded to newspaper and flyer announcements in a university community. Mean age was 22 years with a range from 18 to 30 years and a standard deviation of 3 years. Inclusion criteria included normal speech, language, and memory functions as indicated by self-report and observation of communication during an extensive case history interview, use of English as a native language, completion of a minimum of high-school education, lack of academic training in the field of communication sciences and disorders, and lack of knowledge of the purposes of the present study. All participants passed a vision screening to demonstrate 100% acuity for reading text at the same distance and visual angle as the test stimuli, using glasses or contact lenses if necessary for corrected vision. All participants passed a hearing screening to demonstrate 100% acuity for 500-, 1000-, and 2000-Hz pure tones at 25 dB HL. One participant was African American, one was of Hispanic origin, and the others were Caucasian.
Procedure RTT stimuli were presented to participants in each of three conditions: traditional administration, pointing, and eye movement. The eye movement condition was always presented first in order to avoid participants’ perceptions that they should select a target image with their eyes. The order of the traditional and pointing conditions was counterbalanced across participants. Traditional. The standardised version of the RTT (10 items in each of 10 subtests) was administered according to the published instructions. Participants manipulated plastic tokens that varied in size, shape, and colour according to the standard verbal protocol. Example verbal stimuli are: ‘‘Touch the big black square and the little red circle’’, and ‘‘Put the little green circle to the left of the big red square’’. Traditional 15-point multidimensional RTT scores were obtained independently by two examiners using published instructions. Examiners were two graduate students who had extensive training in RTT administration and ample previous practice with videotape and inperson scoring of the RTT. For each participant, one examiner scored the RTT during actual administration, while another used video recordings of the same session. Overall inter-examiner agreement for the 580 scores obtained for each participant was 97% for initial scoring. One hundred percent consensus was obtained after video analysis and discussion. Pointing. A multiple-choice version of the RTT was created to allow for comparison with spontaneous eye movement responses. Stimuli were based on the first eight subtests of the RTT, with 10 items in each subtest. Subtests 9 and 10 of the RTT were not included because they could not be replicated in a multiple-choice format. Verbal stimuli were adapted from the original RTT by deleting instructional portions of the RTT commands and converting them to descriptive phrases. For example, the command ‘‘Touch the blue circle’’ was modified to ‘‘blue circle’’. Likewise, the command ‘‘Put the red square in front of the white circle’’ was modified to ‘‘The red
590
HALLOWELL, WERTZ, KRUSE
square is in front of the white circle’’. For each item, a ‘‘target’’ image, corresponding to semantic content of the verbal stimulus, was included in a display along with three non-target foils. Non-target foils varied in terms of the number of visual elements in common with the semantic elements of the verbal stimulus. Foil images varied from target images in terms of colour, relative location (corresponding to preposition in the verbal stimuli), shape, and size. Participants were presented with the visual stimuli in a printed test manual and were instructed to point to the image that corresponded to a digitised verbal message. For example, the verbal stimulus ‘‘The little green circle is to the left of the big red square’’, accompanied the visual stimulus shown in Figure 1. Similarly, the verbal stimulus ‘‘big black square and the little red circle’’, accompanied the stimulus shown in Figure 2. Eye movement. Participants viewed the same stimuli presented in the multiplechoice pointing version, shown on a computer monitor rather than in a test manual. A remote ISCAN RK426 pupil centre/corneal reflection system was used to monitor eye movement as participants viewed the visual stimuli. The system entails the projection of a near-infrared light onto one eye, creating two points of reflection, one on the pupil and one on the cornea. These reflections are then recorded digitally and vector calculations are used to determine eye position relative to the visual display. Calibration procedures involved participants’ viewing of a series of five blinking dots on the computer monitor. Given a viewing distance of 34 inches between participants
Figure 1. Sample stimulus: visual matrix to accompan y the verbal stimulus, ‘‘The little green circle is to the left of the big red square’’. Note: colour words are added only for readers of this black-and-whit e text.
EYE MOVEMENTS TO INDEX COMPREHENSION
591
Figure 2. Sample stimulus: visual matrix to accompany the verbal stimulus, ‘‘Big black square and the little red circle’’. Note: colour words are added only for readers of this black-and-whit e text.
and the test monitor, accuracy was plus or minus 0.5 degrees of visual angle, far exceeding the level of accuracy required to differentiate fixations on the images within each display. Target images within a display were viewed at 20 degrees of visual angle apart to eliminate the possibility of clearly identifying target and non-target images via peripheral viewing, thus requiring eye movement for clear viewing of images within the display and ensuring appropriate separation of areas of interest for analysis. No components of the system came into contact with participants, who sat in a soft padded high-back chair while viewing test stimuli. Although participants were encouraged to remain in one relaxed position against the soft chair back to stabilise head movement, no head restriction was required. The ISCAN RK426 system enables compensation for head movement via a pan-tilt lens unit that moves during eye tracking in response to head movement. Custom analysis software was used to determine fixation coordinates and duration, and to eliminate blink artifacts. Dependent variables derived from the ISCAN output included the proportion of total viewing time (total scan duration minus blink time and the time the eyes were transitioning between fixations) that participants fixated on images that matched the verbal stimulus (the target) and on each of the non-target foils. To discourage conscious planning of intentional eye movement responses, participants were not instructed to ‘‘look at’’ anything in particular, and were not told that there were target images within the displays. Verbal stimuli, the same as those used in the pointing condition, were presented simultaneously with the visual stimuli. Visual stimuli were presented for twice the duration of verbal stimuli, plus one second, as proposed by McNeil (M. McNeil, personal communication, 7 June1999), to allow adequate time for comprehension.
592
HALLOWELL, WERTZ, KRUSE
RESULTS Group mean performance and standard deviations for the subtests in each condition are shown in Table 1. Traditional RTT scores indicated normal comprehension in all participants, according to norms reported by McNeil and Prescott (1978). Mean performance across subtests ranged from 14.54 to 14.98 out of a possible score of 15.00. Overall mean performance was 14.85 (sd = 0.23). Mean performance in the pointing condition ranged from 97.9% to 100% correct across the eight subtests. Overall percent correct was 98.68 (sd = 3.29). Pointing version and traditional scores were significantly correlated (r = .53, p < .02), even with the conservative effect of high performance limiting variability of performance in both conditions. The mean proportion of the total duration of eye fixations within a scan allocated to target images ranged from 58% to 75% across the eight subtests. Overall mean proportional fixation duration on the target stimuli was 71% (sd = 0.10). For each of the eight subtests in the eye movement condition, the proportional amount of time that fixations were allocated to target images significantly exceeded chance expectations (p < .001), as shown in Table 2. These results are shown graphically in Figure 3.
DISCUSSION Results support the feasibility of employing eye movements to index linguistic comprehension. All subjects displayed normal comprehension on the traditional RTT, consistent with observation by examiners and self-report of normal language and memory functions. All subjects displayed similarly high levels of auditory comprehension on the experimental pointing version of the RTT. In addition, all subjects’ eye movement fixations implied accurate comprehension by a significant proportion of fixation duration on target images. The language-normal data reported here suggest that the experimental form of the RTT and the application of the eye movement method yield results consistent with more traditional assessments of auditory comprehension. Thus, application of the method with patients who have neurogenic disorders is warranted. In addition to the TABLE 1 Summary of scores by subtest within conditions (N = 19)
Traditional RTT score (of possible 15 points per subtest and for overall score)
Pointing score (percent correct)
Eye movement score (proportion of fixation duration on target)
Subtest
Mean
SD
Mean
SD
Mean
SD
1 2 3 4 5 6 7 8 9 10 Overall
14.96 14.96 14.98 14.97 14.93 14.74 14.94 14.78 14.54 14.68 14.85
.1232 .0078 .0034 .0056 .1183 .7325 .0099 .7340 .7260 .5698 .2275
100.00 97.90 97.90 98.95 98.42 98.42 99.47 98.42 N/A N/A 98.68
.00 4.19 4.19 3.15 3.75 3.75 2.29 5.01 N/A N/A 3.29
.6966 .5759 .7173 .7513 .7153 .7458 .7333 .7505 N/A N/A .7108
.150 .120 .130 .142 .009 .009 .148 .105 N/A N/A .102
EYE MOVEMENTS TO INDEX COMPREHENSION
593
TABLE 2 Analysis of variance for eye movement results by subtest (df = 1,17)
Subtest
MS error
F
Significance
.02249 .01340 .01565 .01782 .00965 .00923 .02327 .01167
159.61 151.50 263.14 267.90 410.53 475.30 178.23 386.98
