Auditory Encoding in Visual Short-Term Recall - Science Direct

JOURNAL OF VERBAL LEARNING AND VERBAL BEHAVIOR 19, 722--735 (1980)

Auditory Encoding in Visual Short-Term Recall: Effects of Noise Intensity and Spatial Location HERBERT A. COLLE Wright State University

According to the primary memory masking hypothesis, speech noise increases serial recall errors made on visually presented lists because the list items are recoded and stored as auditory memory representations which are masked by the noise. In four experiments, both the spatial location of speech noise and its intensity were systematically varied to determine how they influenced recall. The results were consistent with the assumption that primary memory masking takes place in the preperceptual auditory store, which has been inferred from backward recognition masking studies, but were inconsistent with the assumption that primary memory masking takes place in the preeategorical acoustic store, which has been inferred from modality and suffix studies.

Early short-term memory models (e.g., Sperling, 1967) assumed that visually presented items were recorded and stored in auditory memory as auditory representations. Despite early supporting evidence from the phonological similarity effect (Baddeley, 1972) and from the phonological intrusion phenomenon (Conrad, 1964), this auditory recoding assumption has fallen into disfavor. The present paper describes three different problems that led to the demise of the auditory recoding assumption as a key element in models of short-term recall. It discusses the implications of incorporating the auditory recoding assumption into current models, and it presents experimental evidence from noise research which is relevant to these implications. The first problem for the auditory recoding assumption was the failure to gain empirical support for a key prediction that acoustic noise should mask the recoded auditory representations, and thereby reduce These experiments were supported in part from a grant from the University Research Council, Wright State University. I thank Raymond Baird, F. Thomas Eggemeier, and Larry Kurdek for critical comments on earlier drafts, as well as Patricia Wimmers and Joy Wright for assistance in data collection and analyses. Requests for reprints should be sent to Herbert A. Colle, Department of Psychology, Wright State University, Dayton, Ohio 45435. 722 0022-5371/80/060722-14502.00/0 Copyright© 1980by AcademicPress, Inc. All ~ghts of reproductionin any form reserved.

recall performance even for visually presented lists (Hintzman, 1965; Murray, 1965; Sperling, 1963). Colle and Welsh (1976) called this prediction the primary memory masking hypothesis. Noise studies are crucial tests of the auditory recoding assumption because if representations of list items are created in an auditory memory that is a part of the auditory system, then other stimuli that use this part of the auditory system must interact with these representations. Without evidence that acoustic stimuli affect visual lists, phonological coding could be attributed to nonauditory sources such as the articulatory system. Recently, Colle and Welsh (1976) reported evidence supporting the primary memory masking hypothesis. Irrelevant speech sounds increased the number of errors that were made when visually presented lists were serially recalled. They suggested that previous studies (Hintzman, 1965; Murray, 1965; Sperling, 1963) did not find a noise effect because gaussian noise was used, whereas Colle and Welsh had used a speech noise. To buttress the primary memory masking hypothesis, they showed that the speech noise only increased errors in the component of recall attributable to primary memory, or shortterm memory. Furthermore, the interpretation that the noise o p e r a t e d upon

A U D I T O R Y E N C O D I N G IN SHORT-TERM RECALL

phonologically encoded representations was reinforced by the finding that noise and phonological similarity had redundant effects. That is, the error increase produced by combining the presence of noise with the use of highly similar lists did not differ significantly from the effect each had individually. The status of phonological encoding is the second problem facing a model which incorporates the auditory recoding assumption. The saliency of phonological encoding has been reduced because nonauditory encoding of visually presented items has been demonstrated in a number of different experimental paradigms (Brooks, 1968; Healy, 1977; Kroll, 1975; Meudell, 1972; T v e r s k y , 1969; Salthouse, 1974; 1975). Nonauditory encoding is inconsistent with Sperling's (1967) assumption that all visual stimuli must be given an auditory representation in order to be stored in short-term memory. Nevertheless, a more flexible model of memory processing, such as Baddeley's (1978) speech loop model, could be adopted to deal with nonauditory encoding. According to Baddeley's (1978) model, the speech loop is a small phonologically coded articulatory buffer memory that is used in conjunction with a working memory. A central executive routine controls the extent to which the speech loop is used. In particular, the speech loop is used optionally in only some short-term memory tasks, serial recall in particular. Unlike Sperling's (1967) speech loop, Baddeley's did not include the auditory recoding assumption. Sperling assumed that rehearsal could be described as a two-stage process in which implicitly executing the articulatory buffer would create representations in auditory memory which, in turn, could be read so that representations could be recreated in the articulatory buffer. In contrast, Baddeley assumed that the speech loop accomplished rehearsal without utilizing auditory memory. The modified speech loop model will refer to a version of Baddeley's

723

model which incorporates Sperling's twostep rehearsal process. Implications from auditory memory research create the third set of problems for the auditory recoding hypothesis. Auditory memory properties and processes that are inferred from research on the primary memory masking hypothesis should agree with those properties and processes that are inferred from studies of auditory memory. If existing auditory memory models are the only ones considered, then either Crowder and Morton's (1969) precategorical acoustic store or Massaro's (1972) preperceptual auditory store would have to be incorporated into the modified speech loop model. When incorporated, both models make predictions about the conditions which produce primary memory masking, as well as predictions about other data.

Masking Level Difference Hypothesis Crowder and Morton (1969) incorporated their precategorical acoustic store into a model whose assumptions are contradictory to the auditory recoding assumption. They argued that short-term memory is distinct from auditory memory, and that information from lists presented acoustically is stored both in auditory memory and in short-term memory, whereas information from lists presented visually is stored only in short-term memory. Accordingly, their model predicts that acoustically presented lists will be recalled better than visually presented ones because lists presented acoustically have an additional memory store upon which to draw. Crowder and Morton's (1969) model also predicts that acoustic masking should occur, but they predict that it should be present only for acoustically presented lists. The predictions made from Crowder and Morton's model have received a considerable amount of empirical support (Crowder, 1976). Specifically, when serial recall was examined for lists presented via the auditory modality, it was found that fewer errors were made at the end of these lists

724

HERBERT A. COLLE

compared with the number of errors made at the ends of lists presented visually (the modality effect). This a d v a n t a g e that acoustically presented lists had over visually presented ones could be reduced or even eliminated by appending an irrelevant acoustically presented item to a list (the suffix effect). The suffix increased errors at the end of acoustically presented lists, but had little or no effect at the end of visually presented ones. One way to resolve the contradiction b e t w e e n Crowder and Morton's (1969) model and the auditory recoding assumption is to reject Crowder and Morton's model entirely. For example, attentional grouping could be used to account for the suffix effect (Kahneman & Henik, 1977). A consequence of rejecting their model entirely is that modality and suffix effects would no longer be attributed to auditory memory. Therefore, modality and suffix effects could not be used as converging sources of data to independently verify auditory memory properties that are inferred from the primary m e m o r y masking interpretation of noise effects. The noise effect obtained by Colle and Welsh (1976) does show some similarities to the suffix effect. For example, both effects do not occur if lists with phonologically similar vowels are used. Also, gaussian noise neither produced much of a suffix effect (Morton, Crowder, & Prussin, 1971) nor much of a noise effect (Colle, Note 1). Although the suffix and noise effects appear different because the suffix effect increases errors mainly at the end of lists whereas noise increases errors at all serial positions, Odgers and Colle (in press) have pointed out that an end-of-the-list effect is not inherent in Crowder and Morton's model. Instead it is derived from the temporal position of the suffix. In Colle and Welsh's noise experiment, noise stimuli were presented repeatedly throughout list presentation as well as during the retention interval. The masking level difference hypothesis is a version of the modified speech loop

model in which the precategorical acoustic store is used as the auditory memory. It retains both the logic behind Crowder and Morton's interpretation of modality and suffix effects, as well as the logic behind the primary memory masking interpretation of the noise effect. The masking level difference hypothesis focuses on the importance of spatial localization of information in auditory memory, and its dependence upon intensity. From suffix experiments, it has been inferred that information in the precategorical acoustic store can only be masked by incoming information if both sets of information share the same spatial locations (Morton et al., 1971). Crowder (1978) has argued that each spatial location is a channel and that channels are independent information repositories. Psychoacoustic experiments have shown that the release from masking produced by spatially separating target and masking stimuli in simultaneous masking experimenfs depends upon their relative intensity (Jeffress, 1972). A masking level difference refers to the increase in intensity which makes a target stimulus that is spatially separated from a masking stimulus as detectable as it is when it cooccurs in space with the masking stimulus. For the masking level difference hypothesis, it is assumed that visually presented items and acoustically presented items do not share the same spatial locations. Thus, the modality effect is produced because acoustically presented items reach both sets of spatial locations, whereas visually presented items only reach those spatial locations that are accessible to the speech loop. Normally, acoustic suffixes would mask only information from similar spatial locations. Thus, the representations of items at the end of acoustically presented lists are the only ones available for a suffix to mask. Early list items already have been masked by later list items, and visually presented items are located elsewhere in space and, therefore, escape masking. Visual suffixes have little effect because use of the speech

A U D I T O R Y E N C O D I N G IN S H O R T - T E R M R E C A L L

725

loop is optional. There would be no reason are not stored independently at different spatial locations in the preperceptual audito enter irrelevant items into it. According to the masking level difference tory store (Hawkins & Presson, 1977; Pihypothesis, acoustic noise could increase soni, Note 2). Instead, this memory inhererrors on visually presented lists if noise ently can store only one item at a time, reintensity exceeds the masking level differ- gardless of stimulus characteristics. If more ence needed to compensate for the spatial than one stimulus is presented, stimulus separation of the noise and speech loop rep- interaction must occur. resentations. The noise used by Colle and Massaro (1975) also has argued that Welsh (1976) was at least 15-20 db louder masking in the preperceptual auditory store than a comfortable listening level, and, is intensity independent. He argued that hence, the level of most acoustic suffixes. backward recognition masking is a twoIn comparison, masking level differences of p r o c e s s s y s t e m analogous to the twoabout 11 db have been reported for detec- process system found in vision (Turvey, tion e x p e r i m e n t s (Jeffress, 1972), and 1973). Turvey has described two different somewhat smaller ones have been reported types of visual masking. For central maskfor intelligibility e x p e r i m e n t s (Tobias, ing, patterned masking stimuli produced 1972). masking functions which were time depenThe masking level difference hypothesis dent but not intensity dependent, and which makes two predictions which are relevant occurred both for stimuli presented monopto the experiments reported below. (a) It tically and dichoptically. For peripheral predicts that there exists a critical noise in- masking, unpatterned and patterned masktensity below which noise does not increase ing stimuli produced masking functions visual serial recall errors. This critical in- which were intensity dependent, and which tensity must be greater than the intensities had to be presented monoptically in order of suffixes which have failed to increase vi- to be effective. sual serial recall errors. (b) It also predicts Intensity-independent backward recogthat the critical noise intensity should in- nition masking results have been reported crease as the spatial separation of the noise (Pisoni, Note 2). Intensity-dependent reand the speech loop representations is in- sults also have been reported (Repp, 1975a, creased. Data obtained by asking subjects 1975b), but the results of these studies to localize their inner speech suggest that could have been produced by peripheral speech loop representations are localized masking because relatively long-duration near the midline (Weber & Bach, 1969). target stimuli were used so that the masking Therefore, a noise should have a lower and target stimuli overlapped in time for all critical intensity if it is localized near the but one interval. Unfortunately, it is still midline than if it is localized elsewhere. not clear how to functionally separate Equivalently, a midline noise should pro- peripheral from central masking effects in duce more errors than one, as intense, lo- audition, because of differences between cated elsewhere. the auditory and visual systems. The central masking hypothesis is a verCentral Masking Hypothesis sion of the modified speech loop model in Massaro (1972, 1975) has described an which the preperceptual auditory store is auditory memory, the preperceptual audi- used as the auditory memory in the speech tory store, which has properties that differ loop. This hypothesis predicts that primary from those that describe the precategorical memory masking should be independent of acoustic store. In particular, backward rec- the spatial location of the noise and its inognition masking studies have led to the tensity. It assumes that primary memory conclusion that auditory representations masking is central masking. It should be

726

H E R B E R T A. C O L L E

noted in this regard that Turvey's (1973) distinction between " c e n t r a l " and "peripheral" masking was based upon the functional criteria that were described previously. He suggested that peripheral processing included some cortical processing, and was not restricted to effects in the sensory receptors. In this regard, the masking level difference hypothesis implies that noise has peripheral effects. The present experiments were designed primarily to evaluate whether or not noise at midline spatial locations produced more recall errors than noise located elsewhere. In the process of exploring this question, evidence was collected about the effect of intensity. Experiment 1 compared the effectiveness of monaurally, binaurally, and dichotically presented speech noise. With binaural presentation, the same passage is presented to both ears so that it is localized in the midline. With monaural presentation, the passage is presented to only one of the ears so that it is localized on that side. With dichotic presentation, passages are presented to both ears, but each ear receives a somewhat different one so that the two passages do not fuse. Each of the passages is simultaneously localized near the ear to which it is presented. Thus, the masking level difference hypothesis predicts that binaural noise should produce more recall errors than monaural or dichotic noise. On the other hand, the central masking hypothesis predicts that binaural, m o n a u r a l , and dichotic noises should produce equivalent recall decrements. If dichotic noise produces more recall errors than binaural noise, then neither auditory memory assumption can describe the data. This result would be evidence against the primary memory masking hypothesis. EXPERIMENT 1

Method Subjects. The subjects were 72 female undergraduates from Wright State University who received extra credit in introduc-

tory psychology courses. The subjects reported having no hearing losses, and were not familiar with either German or Yiddish. Also, subjects who spoke nonstandard English were not used. All the subjects were right-handed. Procedure. Each subject was tested individually in a single-walled, sound-attenuated room for three separate blocks of trials. The first block of four trials was a practice block during which the noise stimuli were never presented. The second and third blocks were test blocks, consisting of 22 trials each. In one block, a speech noise was presented and in the other it was not. The noise and quiet blocks were counterbalanced across subjects. The sequence of events that occurred on each trial was the same in all four experiments which are described below. The memory stimuli were the letters F, K, L, M, Q, R, and Y, presented visually. All seven letters were presented sequentially, in random order. Each appeared for 0.8 second and was followed by a 0.4-second interstimulus interval. Following the presentation of all of the letters, the subjects had to wait 10 seconds before initiating recall. The word "recall" appeared on the screen and signaled the end of the delay. The subjects recalled the letters serially in the correct order by typing them on an electronic keyboard on which the letters appeared as ordered above. Following the seventh key press, the word " r e a d y " app e a r e d on the screen and the subject pressed the space bar to start the next trial. The stimuli were presented on a Techtronics 604 monitor using a 35-dot character generator under computer control. The first 6 trials of each test block were treated as practice trials and the data were discarded. The first 14 of the remaining 16 trials were used as the test trials. The last 2 trials were presented to all subjects, but they only were used as replacement trials. The data from the first replacement trial were set aside to be used if a subject spoke

A U D I T O R Y E N C O D I N G IN SHORT-TERM RECALL

on one test trial. Both replacement trials were to be used if a subject spoke on 2 test trials. Five subjects spoke aloud during the experiment. All of them spoke aloud only once. Four spoke aloud during the noise block and one spoke aloud during the quiet block. Noise conditions and measurements. The speech noise was a German passage from " A Hunger Artist" by Franz Kafka (1922). A female native German speaker read the passage as naturally as possible while attempting to minimize variations in voice intensity. The speaking rate was determined by measuring the time needed to speak 20 consecutive words. Based upon the entire passage, the mean time was 9.09 seconds (2.2 words/second) with a standard deviation of 1.11 seconds for the 168 successive samples of 20 words. There were six different types of noises. In order to produce the binaural noise, the German passage was recorded onto both tracks of a tape using a TEAC tape deck (No. 7030 GSL). The dichotic tape was made similarly except that two passes were made. After the left channel was recorded, the right was recorded with a lag of 2 minutes. The intensity of both the fight and the left earphone was set at 70 db(A) for both the binaural and the dichotic conditions. There were four monaural noises, arising from the factorial combination of noise intensity (low vs high) and ear of presentation (right vs left). To produce the monaural noise, a single c h a n n e l f r o m the twochannel tapes was connected to the appropriate earphone. The noise was either presented at 70 db(A) (low) or at 76 db(A) (high). T w o i n t e n s i t i e s w e r e u s e d to evaluate the possibility that m o n a u r a l binaural differences might depend upon loudness d i f f e r e n c e s . Output of either channel of the tape deck led to a separate step a t t e n u a t o r and then to a separate channel of a power amplifier, which in turn led to a Telephonics TDH-39 earphone mounted in a MX41/AR cushion. Recorded at the beginning of each stimulus tape was a 1000 Hz tone, which was used to set inten-

727

sity levels before each experimental session by adjusting the voltage across 1 ohm at the earphones. The intensity generated at each earphone was m e a s u r e d using a s t a n d a r d 6-cm 3 coupler (Type 1460-P83) and a General Radio sound level meter (Type 1565-A) set on the slow A and C scales. All measurements were made at least 20 db above the noise level of the meter. An attempt was made to estimate the intensity while speech was present. The dichotic and binaural stimuli used in Experiment 1 produced approximately 70 db(A) and 75 db(C) at each earphone. The corresponding values for the lower intensities that are marked on the abscissa of Figure 1 were generated by increasing the circuit attenuators. In addition to these two measures, an attempt was made to estimate the loudness levels and the s e n s a t i o n levels of the binaurally presented stimuli. Eight inexperienced listeners obtained from the same population as the experimental subjects matched the intensity of a 1000-Hz tone to various intensities of the binaurally presented German. An 800-millisecond segment of the tape at a given intensity was followed after 250 milliseconds by an 800-millisecond 1000-Hz tone. The tone was either increased or decreased at the listener's request until she indicated that they matched. Following a match, the same segment of the tape was presented again at a different intensity together with a new starting intensity of the tone. Three widely spaced segments of the German passage were evaluated in this way at eight different intensities, spanning a 60-db range. The starting intensities of the tone varied from 5 to 20 db above and below the anticipated final matching level. The values of loudness level shown on the abscissa of Figure 1 were obtained by regressing the mean (in db SPL) of the tone settings from the three German segments for each subject against the relative intensity of the German (r = .99). As the abscissa values reveal, the slope was very close to one. The last four listeners were also asked to

728

HERBERT A. COLLE

indicate the point at which they could no by intensity interaction. A fourth contrast longer hear the German. An ascending and evaluated performance with binaural versus descending method of limits was used twice dichotic presentations, and the fifth comin each direction while the tape ran con- pared the average of the four monaural tinuously. The mean of all 16 intensities conditions with the average of the binaural was used as the sensation level. The means and dichotic conditions. The interactions of for the ascending and descending directions these five contrasts with the noise- and differed by less than 2 db. The abscissa of list-order factors also were obtained. Figure 1 also shows the intensities of the Results German in terms of the sensation level. Responses were scored as correct only if Experimental design. Three betweensubjects factors were used. A type of noise the correct letter was recalled in the correct factor with six levels (four monaural, one position. For each subject, the total number binaural, and one dichotic) was factorially of errors made on all 14 lists was obtained. combined with the two methodological The planned orthogonal comparisons, defactors (noise order and list order). There scribed above, were computed using the was a total of 24 experimental combina- noise minus quiet difference scores. The tions, each of which had three subjects in- means for the four monaural conditions, the binaural condition, and the dichotic condidependently and randomly assigned to it. Five planned orthogonal contrasts were tion are presented in Figure 1 at the intenused to analyze the data. Three of the con- sity levels of 70 and 76 db(A), which are the trasts were a factorial analysis of the intensity levels that were used in Experimonaural conditions. One of them eval- ment 1. The other data points in Figure 1 uated the ear effect, one evaluated the in- come from Experiments 2 - 4 , which are detensity effect, and one evaluated the ear scribed below.

o o

~

24

D

22

2o

Eo18

D 16

CI~ 10 ¢0 C 8

"~ "~ z

/

4 2 0

dB above the lowest intensity Loudness Level, dB(phons) Sensation Level, dB

dB(C) dB(A)

DR -L•-

/ / Bd

Dichotic Right Ear Left Ear Mean-

D

29 12 25 20

2b

48 32 45 40

58 42 55 50

4'o

i i

77 62 75 70

s6 60 83 68 81 76

Speech Noise Intensity FIG. 1. N o i s e - e f f e c t m a g n i t u d e a s a f u n c t i o n o f t h e i n t e n s i t y a n d s p a t i a l l o c a t i o n o f t h e n o i s eNoise . intensity was manipulated across experiments. The data at 70 and 76 db(A) are from Experiment 1. The data at 50, 40, and 20 db(A) are from Experiments 2, 3, and 4, respectively. Alternative measures of noise intensity are presented for comparison on the abscissa.

AUDITORY ENCODING IN SHORT-TERM RECALL

The speech noise significantly increased the number of recall errors that were made. Recall errors increased from 31.7% on the quiet trials to 48.5% on the noise trials, F(1,48) -- 88.9, p < .01. As the following analyses will show, this 16.8% increase in errors did not depend upon the type of noise that was used. To evaluate the importance of spatial location, the binaural and dichotic noises were compared. The error increase was not reliably different for these two conditions, F(1,48) = 1.98, p > .05. Errors increased 15.6% during the binaural noise and 24.2% during the dichotic noise. The error increase from these two conditions also did not differ reliably from the mean monaural error increase of 15.1%, F(1,48) = 1.55, p > .05. The monaural noises also did not differentially affect recall performance. There was no ear effect, F(1,48) < 1.0, no intensity effect, F(1,48) < 1.0, and no ear x intensity interaction, F(1,48) < 1.0. The noise minus quiet error difference did not depend upon noise order or list order, F(1,48) < 1.0 and F(1,48) = 3.75, p > .05, respectively. These two factors also did not interact with any of the orthogonal comparisons. To evaluate serial position effects, the six noise types were treated as a single factor with independent measures. The two factors of serial position and noise vs quiet were treated as factors with repeated measures. Conservative degrees of freedom, which reduce the degrees of freedom by a factor of six, were used to avoid problems of covariance heterogeneity. There was a large serial position effect, F(1,66) = 78.6, p < .01, whose shape did not depend upon either the presence of noise, F(1,66) = 1.75, p > .05, the noise types, F(1,66) = 1.26, p > .05, or the interaction of the noise types with the presence vs absence of noise, F(1,66) < 1.0. No other factor interacted with serial position. The serial position curves were typical visual serial position curves with a large primacy effect and little or no recency effect (Cone & Welsh, 1976; Morton, 1970).

729

Discussion Binaural noise, which is localized in the midline, did not reliably produce more recall errors than either monaural or dichotic noises, which are not localized in the midline. These results support the central masking hypothesis rather than the masking level difference hypothesis. According to the masking level difference hypothesis, binaural noise which is closer to the midline speech loop representations should produce more masking. On the other hand, these data do not provide very strong evidence against the masking level difference hypothesis, because a plausible alternative explanation for the results is that the noise level was still too high and, therefore, all masking effects were asymptotic. Intensities were reduced in Experiments 2-4. A noise level of 50 db(A) was used in Experiment 2 in order to increase the likelihood of producing a b i n a u r a l - d i c h o t i c noise difference. For the audio system which was used in the present experiments, I estimated that comfortable listening levels ranged from 50 to 60 db(A). Thus, most suffix intensities probably have been within this range. Therefore, 50 db(A) should be close to the predicted lower bound for noise effects, according to the masking level difference hypothesis. I expected to find either a binaural-dichotic difference or no noise effect and planned to use an iterative bracketing procedure to search for binaural-dichotic differences. As it turned out, Experiments 2 - 4 produced evidence against the masking level difference hypothesis because a noise effect was found at a noise intensity which was substantially below any suffix that would have been used.

EXPERIMENTS 2--4

Method Subjects. Altogether 80 female undergraduates who did not participate in Experiment 1 served as subjects in Experiments 2 - 4 . They were obtained from a university subject pool as in Experiment 1.

730

H E R B E R T A. C O L L E

There were 24, 32, and 24 subjects in each of the respective experiments. The subjects reported no hearing losses, and were not familiar with German or Yiddish. Subjects who spoke nonstandard English were not used. Handedness was not determined. Procedure. Experiments 2 - 4 were replications of Experiment 1, except for two changes. First, only the binaural and the dichotic noises were used; no monaurally presented noise was used. An equal number of subjects received each type of noise. Each subject again received two blocks of test trials, one without noise and one with the noise present. Both noise and list order were c o u n t e r b a l a n c e d across subjects. Second, noise intensity was reduced. In Experiments 2 - 4 , respectively, circuit attenuators were increased 20, 30, and 50 db above the settings that were used to produce the 70 db(A) noise in Experiment 1, thereby, producing noises of 50, 40, and 20 db(A). E a c h of the t h r e e e x p e r i m e n t s was analysed with the same analysis of variance design. There were three between-subjects factors, noise type (binaural vs dichotic), noise order (quiet-noise vs noise-quiet), and list order, and there was one repeated-measure factor, noise (quiet vs noise). The dependent variable in the analyses was the total number of errors made by each subject on the 14 test lists. Responses were scored as correct only if the correct letter was recalled in the correct position. Replacement trials were used only for one subject who was in E x p e r i m e n t 3. She spoke aloud on two test trials during the noise block.

> .05. Errors increased 12.2% when noise was binaural and 6.3% when it was dichotic. Figure 1 shows these data at the 50db(A) abscissa value. The noise factor interacted with the noise order factor, F(1,16) = 5.97, p < .05. Because a counterbalanced design was used, a significant noise by noise order interaction would be produced if more errors were made on block 1 than were made on block 2. The data were consistent with this pattern. No other effects were statistically significant. An analysis of variance, including serial position as a factor and using conservative degrees of freedom, indicated that errors depended upon serial position, F(1,22) = 35.9, p < .01. The curves were typical visual serial recall curves. As in Experiment 1, no variables interacted with serial position. Experiment 3. At 40 db(A) the speech noise sounded as quiet as a whisper. Yet, it still increased recall errors. Errors rose from 30.5% during the quiet blocks to 43.9% during the noise blocks, F(1,24) = 27.4, p < .01. Binaural noise did not reliably produce more recall errors than dichotic noise, F(1,24) = 2.34, p > .05. The binaural and dichotic error increases are shown in Figure 1 at the 40-db(A) abscissa value. The data points in parentheses show the means based upon only the first 24 subjects instead of all 32 subjects. In both cases the results of analyses of variance were very similar. Noise order did not interact with the noise factor, F(1,24) = 1.28, p > .05. There were no other statistically significant main effects or interactions. Results The conservative serial position analysis Experiment 2. Although the intensity of r e v e a l e d large serial p o s i t i o n e f f e c t s , the speech noise was reduced to 50 db(A), F(1,30) = 44.6, p < .01. The curve again the noise still p r o d u c e d an i n c r e a s e d was raised uniformly by the presence of number of recall errors. During the quiet noise, independent of the type of noise block, errors were made on only 31.7% of used, F(1,30) = 2.79, p > .05, F(1,30) < the responses, compared to 41.4% during 1.0, respectively. As in the previous two the noise block, F(1,16) = 14.9, p < .01. experiments, these were typical visual seNevertheless, binaural and dichotic noises rial recall curves. Experiment 4. The 20-db(A) noise did not did not differ statistically, F(1,16) = 1.51, p

A U D I T O R Y E N C O D I N G IN S H O R T - T E R M RECALL

produce a statistically significant increase in recall errors. The noises produced an error increase of only 0.6% above the 32.4% error that was made during the quiet blocks, F(I,16) < 1.0. The increases produced by binaural and dichotic noise were not statistically different, F(1,16) < 1.0. Figure 1 shows the error increases. Noise order did not interact with the noise factor, F(1,16) = 2.15, p > .05, and no other main effects or interactions were statistically significant. The conservative serial position analysis again showed the same visual serial recall pattern that was found in the previous three experiments. Serial position was statistically significant, F(1,22) = 29.6, p < .01, but it did not interact with any other factors. Combined analyses. In addition to the analyses of the individual experiments, noise intensity effects were explored by analyzing all four experiments together. A completely balanced factorial design was created by discarding the monaural conditions of Experiment 1 and the fourth replication from each cell in Experiment 3. The between-subjects factors were noise type (binaural vs dichotic), i n t e n s i t y (four levels), plus noise order and list order. The repeated-measures factor was noise (quiet vs noise). More errors were made during the noise blocks than were made during the quiet blocks, F(1,64) = 49.1, p < .01. Binaural noise produced about the same error increase as dichotic noise, F(1,64) < 1.0. The error increases produced by binaural versus dichotic noise did not depend upon intensity, F(3,64) = 1.08, p > .05. The noise effect was not the same at all intensities, F(3,64) = 6.66, p < .01. Apparently, this differential noise effect across intensity was produced by the absence of a noise effect at the lowest intensity used in Experiment 4. In a similar analysis using only Experiments 1 through 3, the intensity by noise interaction was not statistically significant, F(2,48) = 2.33, p > .05. Thus

731

the results indicate that as intensity was decreased, the noise effect remained intact at full strength over at least a 30-db range, dropping out only at a noise intensity somewhere between 20 and 40 db(A). The only other factor to affect the magnitude of the noise effect was noise order, F(1,64) = 8.16, p < .01. Although the effect was statistically significant only in Experiment 2, the analysis revealed no evidence of an interaction with intensity, F(3,64) < 1.0. The overall pattern was that subjects made 4.4% fewer errors in the second block of trials regardless of experimental conditions. Concerns about carryover effects of noise from block one to block two can be alleviated by an analysis of block one only. The subjects tested with noise in the first block produced 13.7, 8.5, 25.3, and 1.5% more errors than did subjects tested without noise in the first block in Experiments 1 through 4, respectively. This considerably less powerful analysis still yielded a noise effect, F(1,64) = 13.7, p < .01. However, the analysis was not sensitive enough to detect a differential noise effect across intensity, F(3,64) = 2.28, .05 < p < .10. A similar analysis on Experiments 1 through 3 yielded a noise effect of F(2,64) = 15.6,p < .01, and a noise × intensity interaction of F(2,64) = 1.54, p > .05. In addition to the analyses of the total number of errors made per block, corresponding analyses were performed on the number of lists per block recalled without error. The results of these analyses were basically the same as the total error analyses.

Discussion These four experiments provide substantial evidence against the masking level difference hypothesis. Neither of its predictions were confirmed. Binaural noise did not impair recall performance more than dichotic noise at any intensity. Thus spatial localization appears to be unimportant for primary memory masking. More important, a noise effect was found for an intensity of

732

HERBERT A. COLLE

40 db(A). Suffixes that quiet would not normally be used. Therefore, the results of suffix and noise experiments cannot be reconciled by attributing the differences to the relative intensities of suffixes and noises. The results support the central masking hypothesis. According to this hypothesis, noise impairs visual recall performance because information in the speech loop must pass through the preperceptual auditory store and the information in this store gets masked by the acoustic noise. As outlined in the introduction, this store should be insensitive to differences in the relative spatial location of stimuli. According to the central masking hypothesis, masking of information in the preperceptual auditory store also should not depend upon mask intensity. This prediction was supported by the above experiments which found that the magnitude of the noise effect was constant over a 36-db range from 40 to 76 db(A). The weighted mean error increase was 14.6% which is indicated by the horizontal dashed line in Figure 1. On the other hand, the above experiments were not designed to evaluate intensity effects and, therefore, a powerful test of noise constancy could not be conducted. Small increases in the range 40 to 76 db(A), which would be consistent with a negatively accelerated function, might not have b e e n d e t e c t e d by the analysis. Nevertheless, small performance changes between 40 and 76 db(A) must be considered within the context of the intensity range which was used. At 76 db(A) speech sounds very loud, whereas at 40 db(A) it sounds as quiet as a whisper. This 36-db range corresponds to a power ratio of over 4000. In comparison, Turvey (1973) varied intensity by a factor of 2. The fact that primary memory masking did not occur at 20 db(A) does not create any serious problems for the assumption that primary memory masking is a central masking process. All patterned masking

processes imply that the intensity-independence range has a nonzero lower bound, because at low but still detectable intensities pattern characteristics will not be extracted, making the patterned mask effectively unpatterned. G E N E R A L DISCUSSION

The present experiments are consistent with the hypothesis that noise disrupts serial recall because it masks information in the preperceptual auditory store. Colic and Welsh's (1976) experiments implied that noise masked an auditory memory. The present experiments suggest that this auditory memory is the preperceptual auditory store (Massaro, 1972), and not the precategorical acoustic store (Crowder & Morton, 1969). On the other hand, although the data provide strong evidence against the incorporation of the precategorical acoustic store into the modified speech loop, the evidence for the incorporation of the preperceptual auditory store is not as strong. The experiments were designed primarily to test the masking level difference hypothesis. More convincing evidence that noise produces recall errors by masking the preperceptual auditory store would be obtained if variables which influence the magnitude of backward recognition masking also were found to have similar effects upon the magnitude of the noise effect. Turvey (1973) argued that central masking in vision only was obtainable if a patterned masking stimulus was used. Analogously, i f primary memory masking is a central masking process as the above data suggest, then only patterned masks should produce primary memory masking. Colle and Welsh (1976) used a patterned masking argument to explain why they had found a noise effect, whereas others had failed. S p e e c h noise is a p a t t e r n e d masking stimulus, whereas gaussian noise is not. Recently, Colle and Welsh's argument has been challenged by the conclusions of Poulton (1977). Poulton argued that noise

A U D I T O R Y E N C O D I N G IN S H O R T - T E R M R E C A L L

can impair performance in complex visual tasks by masking "inner speech," an explanation that appears to be essentially the same as Colle and Welsh's. Many of the studies Poulton used to support his "inner speech" argument had used gaussian noise. Thus, he implied that unpatterned gaussian noise was an effective primary memory masking stimulus. In his review, Poulton did not cite the three studies which were most similar to Colle and Welsh's, and which did not find a gaussian noise effect (Hintzman, 1965; Murray, 1965; Sperling, 1963). In a direct comparison of gaussian and speech noises, Colle (Note 1) found that gaussian noise did not impair visual serial recall, although speech noise did. In Poulton's (1977) summary analysis, it was assumed that if visual tasks were complex, then they required short-term memory and, hence, inner speech. Baddeley's (1978; Baddeley & Hitch, 1974) studies of working memory and short-term memory point out the dangers of assuming a simple correspondence between the two. In fact, Baddeley has argued that not even all short-term memory tasks use the speech loop. Poulton provided no corroborating evidence, such as evidence that phonological coding was used, that the complex tasks he described used the speech loop, or inner speech. Poulton (1977) reviewed several other hypotheses which had been proposed to account for the effects of noise on visual performance. Three different types of hypotheses have been proposed: (a) methodological problems, (b) selection of input stimuli, and (c) arousal states. Poulton argued that noise may mask switch clicks and other sounds which provide useful feedback to subjects. The response mechanisms used in the present study make that explanation particularly unlikely. Besides, Colle and Welsh used written recall. On the other hand, it is possible that Poulton overlooked a source of acoustic feedback which could potentially explain some of the gaussian noise results

733

he described. In the present experiments subjects were not allowed to talk aloud because subject-produced overt speech can produce a modality effect (Murray, 1965). None of the studies Poulton reported controlled subjects' overt speech. Thus it is possible that in these studies noise masked "outer speech" not "inner speech." The reported effects were small, and not many trials would have to have been affected to produce them. Noise effects also have been attributed to the loss of input information. The internal blink hypothesis and attentional funneling are two more specific input loss hypotheses. Colle and Welsh (1976) provided evidence against attributing speech noise effects to input losses. The present experiments also provide evidence against the input loss hypotheses, although the data are less definitive than Colle and Welsh's. The input loss explanation would predict that dichotic noise should produce more errors than either binaural or monaural noise. The present experiments do provide substantial evidence against the arousal hypothesis. According to the arousal hypothesis, for each situation there is an arousal level that leads to optimal performance. Noise is assumed to increase the arousal level. Thus the arousal hypothesis predicts that there is a U-shaped function relating recall errors to noise intensity. In many noise studies only two levels of noise intensity were used so that this hypothesis can predict any result that is obtained. In the present study, the error function was not U-shaped. Only an inverted U-shaped function would be consistent with the data, and it would be in the direction opposite that predicted by the arousal hypothesis. In summary, speech noise produced a short-term recall decrement that was relatively insensitive to the intensity of spatial location of the noise. Baddeley's (1978) speech loop model can be made consistent with these results, if it is assumed that the speech loop cycles through an auditory memory and in the process recodes visually

734

H E R B E R T A. COLLE

presented items, giving them auditory representations. According to the primary memory masking hypothesis, speech noise masks the auditory memory representations. The auditory memory in this modified speech loop model was found to have properties that are consistent with those that have characterized Massaro's (1972) preperceptual auditory store but are inconsistent with those that have characterized Crowder and Morton's (1969) precategorical acoustic store. REFERENCES BADDELEY, A. D., Retrieved rules and semantic coding in short-term memory. Psychological Bulletin, 1972, 78, 379-385. BADDELEY, A . D . , The trouble with levels: A reexamination of Craik and Lockhart's framework for memory research. Psychological Review, 1978, 85, 139-152. BADDELEY, A. D., & HITCH, G., Working Memory. In G. H. Bower (Ed.) The psychology of learning and motivation (Vol. 8). New York: Academic Press, 1974. BROOKS, L. R. Spatial and verbal components of the act of recall. Canadian Journal of Psychology, 1968, 22, 349-368. COLLE, H . A . , ~z WELSH. A. Acoustic masking in primary memory. Journal of Verbal Learning and Verbal Behavior, 1976, 15, 17-31. CONRAD, R. Acoustic confusions in immediate memory. British Journal of Psychology, 1964, 55, 75 -83. CROWDER, R. G. Principles of learning and memory. Hil/sdale, N.J.: Erlbaum, 1976. CROWDER, R. G. Mechanisms of auditory backward masking in the stimulus suffix effect. Psychological Review, 1978, 85, 502-524. CROWDER, R. G., & MORTON, J. Precategorical acoustic storage (PAS). Perception and Psychophysics, 1969, 5, 365-373. HAWKINS, H. L., & PRESSON, J. C. Masking and Preperceptual selectivity in auditory recognition. In S. Dornic (Ed.) Attention and performance V1. New York: Wiley, 1977. HEALY, A. F. Pattern coding of spatial order information in short-term memory. Journal of Verbal Learning and Verbal Behavior, 1977, 16, 419437. HINTZMAN, D. L. Classification and aural coding in short-term memory. Psychonomic Science, 1965, 3, 161-162. JEFFRESS, L . A . Binaural signal detection: Vector theory. In J. V. Tobias (Ed.), Foundations of modern auditory theory (Vol. 2). New York: Academic Press, 1972.

KAFKA,F. Ein hunger-kunstler. Die Neue Rundschau, 1922, 33,983-992. KAHNEMAN, D., & HENIK, A. Effects of visual grouping on immediate recall and selective attention. In S. Dornic (Ed.) Attention and performance VI. New York: Wiley, 1977. KROLL, N. E. A. Visual short-term memory. In D. Deutsch & J . A . Deutsch (Eds.), Short-term memory. New York: Academic Press, 1975. MASSARO, D . W . Preperceptual images, processing time, and perceptual units in auditory perception. Psychological Review, 1972, 79, 124-145. MASSARO, D . W . Understanding language. New York: Academic Press, 1975. MEUDELL, P . R . Short-term visual memory: Comparative effects of two types of distraction on the recall of visually presented verbal and nonverbal material. Journal of Experimental Psychology, 1972, 94, 244-247. MORTON, J. A functional model for memory. In D. A. Norman (Ed.), Models of human memory. New York: Academic Press, 1970. MORTON, J., CROWDER,R. G., & PRUSSIN, H. A. Experiments with the stimulus suffix effect. Journal of Experimental Psychology Monograph, 1971, 91, 169- •90. MURRAY, D. J. The effects of white noise on the recall of vocalized lists. Canadian Journal of Psychology, 1965, 19, 333-345. ODGERS, R. P., • COLLE, H. A. The status of spatial location in auditory sensory memory: Evidence from rapidly presented lists. Journal of Experi-

mental Psychology: Human Learning and Memory, in press. POULTON, E. C. Continuous intense noise masks auditory feedback and inner speech. Psychological Bulletin, 1977, 84, 977-1001. REPP, B. H. Dichotic forward and backward "masking" between CV syllables. Journal of the Acoustical Society of America, 1975, 57, 483-496. (a) REPP, B. H. Dichotic masking of consonants by vowels. Journal of the Acoustical Society of America, 1975, 57, 724-735. (b) SALTHOUSE, T. A. Using selective interference to investigate spatial memory representations. Memory and Cognition, 1974, 2, 749-757. SALTHOUSE, T. A. Simultaneous processing of verbal and spatial information. Memory and Cognition, 1975, 3, 221-225. SPERLING, G. A model for visual memory tasks. Human Factors, 1963, 5, 19-35. SPERLING, G. Successive approximations to a model for short-term memory. Acta Psychologica, 1967, 27, 285-292. TOBIAS, J. V. Curious binaural phenomena. In J. V. Tobias (Ed.), Foundations of modern auditory theory (Vol. 2). New York: Academic Press, 1972. TURVEY, M. T. On peripheral and central processes in vision: Inferences from an information-processing

AUDITORY ENCODING IN SHORT-TERM RECALL analysis of masking with patterned stimuli. Psychological Review, 1973, 80, 1-52. TVERSKY, B. Pictorial and verbal encoding in a short-term memory task. Perception and Psychophysics, 1969, 6, 225-233. WEBER, R. J., & BACH, M. Visual and speech imagery. British Journal of Psychology, 1969, 60, 199-202.

735

REFERENCE NOTES 1. COLLE, H. A. Masking of auditory imagery in short-term memory. Paper presented at the meeting of the American Psychological Association, Toronto, Canada, August 1978. 2. PISONI, D. B. Perceptual processing time for consonants and vowels. Haskins Laboratories Status Reports on Speech Research, SR-31/32, pp. 83-92, 1972. (Received June 25, 1979)