779415 research-article2018
MSX0010.1177/1029864918779415Musicae ScientiaeHerff et al.
Article
Context effects of background babbling on memory for melodies
Musicae Scientiae 1–17 © The Author(s) 2018 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav https://doi.org/10.1177/1029864918779415 DOI: 10.1177/1029864918779415 journals.sagepub.com/home/msx
Steffen A. Herff Western Sydney University, Australia Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
Roger T. Dean Western Sydney University, Australia
Nora K. Schaal
Heinrich-Heine University Düsseldorf, Germany
Abstract Disruptive effects of background noise on memory have been thoroughly investigated. Potential beneficial context effects of background noise on memory, however, have received far less attention. Here, we explore whether and how unintelligible multiple talker background babbling affects melody recognition. Participants continuously listened to melodies, each of which appeared twice during the experiment. After each melody participants were asked to indicate whether they had heard the melody previously in the experiment. The presence or absence of background noise during first or second melody presentation was manipulated in four conditions (Clear-Clear, Clear-Noise, Noise-Clear, Noise-Noise). We measured recognition performance as well as cumulative disruptive interference between first and second melody presentation. Mixed effects models revealed that recognition in Clear-Clear was significantly better compared to Noise-Clear and Noise-Noise, but not compared to Clear-Noise. Clear-Noise showed descriptively better recognition performance than Noise-Noise, however this comparison did not reach statistical significance. Recognition performance in Clear-Noise was significantly better than in NoiseClear, suggesting that noise during encoding affects recognition performance more strongly than noise during retrieval. Furthermore, cumulative disruptive interference was stronger in mismatching contexts. Our results suggest that if possible, background noise should be avoided as it negatively affects memory performance. However, if encoding is likely to take place in a noisy environment, then presenting background noise during retrieval may be beneficial. This is because matching background noise during encoding and retrievals appears to reduce cumulative disruptive interference.
Keywords background noise, interference, memory, music perception, recognition
Corresponding author: Steffen A. Herff, The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Post: Locked Bag 1797, Penrith, NSW 2751, Australia. Email:
[email protected]
2
Musicae Scientiae 00(0)
When we think about auditory background noise, we usually focus on its disruptive effects. Previous research follows this trend and demonstrates the negative effects of noise on perceptual tasks (Gilbert, Chandrasekaran, & Smiljanic, 2014; see Mattys, Davis, Bradlow, & Scott, 2012, for a review). This intuitively makes sense; when we are in a noisy environment such as a restaurant or a bus we may become distracted from the conversation at hand or unable to hear certain words, all of which impedes on our recognition performance later. However, the literature on perceptual context effects draws a different picture. Work on context effects suggests that memory performance is increased when information is accessed in the same context as it was learned in (Baddeley, Eysenck, & Anderson, 2009, pp. 176–180; Smith & Vela, 1992; see Smith & Vela, 2001, for a review). This suggests that background noise during memory encoding might not always be disruptive but could potentially act as a context cue and be beneficial, leading to enhanced memory performance when similar noisy retrieval conditions are present. While a noisy environment during the first encounter of a stimulus seems to have a destructive impact on encoding every detail, it might also be beneficial in the light of a contextdependent memory when retrieval is likely to occur in a noisy environment as well. Here, we specifically investigate context effects of background talker babbling on memory for melody since music is often encountered in noisy environments and it shows intriguing properties in the context of memory. In the following we will discuss the two main motivators of this study: context-dependency and the cumulative disruptive interference in memory for melody.
Context-dependency Context-dependency in recognition is not as strong as it is in recall, yet is still present (Godden & Baddeley, 1980; Grant et al., 1998; Smith & Vela, 1992, 2001). Indeed, research in the language domain indicates that new words learned in a noisy environment show a recognition advantage over words learned in a clear environment when tested in a noisy context (Mattys, Bradlow, Davis, & Scott, 2013, pp. 75–82). For example, Creel, Aslin, and Tanenhaus (2012) presented their participants with spoken words, either clear or embedded in white noise. Following the learningphase, a testing-phase assessed recognition performance on either clear words or words embedded in white noise. This enabled all four possibilities (clear-clear, clear-noise, noise-clear, noise-noise) as a within-subject factor. In line with context-specific recognition predictions, Creel et al. found that performance was better when the background noise condition during retrieval matched the background noise condition during encoding. The authors suggest that this indicates listeners’ memory representations of words are sensitive to the spectral background context. The present study aims to further investigate potential effects of background noise on memory. Here, we were specifically interested in potential beneficial effects of real-world background noise on melody recognition. Using background noise with real-world implication such as background babbling rather than white noise, we aim to investigate auditory induced, perceived or imagined environmental context effects beyond purely spectral background effect. Background babbling is a commonly used stimulus to induce auditory interference in disrupting memory performance (e. g., Heinrich, Schneider, & Craik, 2008). Whether it may also induce environmental context effects that are reflected in melody recognition is the subject of the present investigation. Music is often encountered in noisy environments such as concerts, trains, cars, and in the background of a movie or advertisement; yet, we can recognize a melody in our favorite pub through the vocal chaos of a Friday evening. In the context of the present study, we investigate if hearing a melody for the first time in an adverse or noisy condition may help in recognizing the melody later if it is also encountered during adverse or noisy conditions.
Herff et al.
3
Memory for melody and cumulative disruptive interference The present study is also motivated by the many intriguing properties of memory for melody. Memory for melody can be rather persistent, as exemplified by “sticky earworms” (Bailes, 2007; Halpern & Bartlett, 2011; Williams, 2015). Memory for melody also seems to be spared in some forms of dementia (Cuddy & Duffin, 2005) and, recently, it has been shown that it does not seem to be affected by the number of intervening items (Herff, Olsen, & Dean, 2017; Herff, Olsen, Dean, & Prince, 2017; Herff, Olsen, Prince, & Dean, 2017). When melodies are presented, one after another, and participants are asked to respond after each melody whether or not they have heard this melody in this experiment before, then recognition performance is stable between 1 and up to 195 intervening melodies (Herff, Olsen, & Dean, 2017). This is quite remarkable as most other stimuli such as words or numbers demonstrate cumulative disruptive interference as recognition performance cumulatively decreases with each additional intervening item (Bui, Maddox, Zou, & Hale, 2014; Campeanu, Craik, Backer, & Alain, 2014; Friedman, 1990; Hockley, 1992; Konkle, Brady, Alvarez, & Oliva, 2010; Poon & Fozard, 1980; Rakover & Cahlon, 2001; see Sadeh, Ozubko, Winocur, & Moscovitch, 2014, for a review). As this research shows that memory for melodies has specific properties that differ from other domains, we here aimed to specifically investigate some of these properties, in particular the magnitude of cumulative disruptive interference, and implement them as an additional measurement to assess potential context effects of background noise on melody recognition. Previously, it has been shown that the resilience of memory for melody towards cumulative disruptive interference is stimulus sensitive. For example, while memory for melodies in a familiar tuning system is resilient to cumulative disruptive interference, memory for melodies in an unfamiliar tuning system is not (Herff, Olsen, Dean, et al., 2017). A proposed mechanism that provides memory for melody with resilience towards cumulative disruptive interference is the formation of multiple memory representations, each reflecting a perceptual experience (e.g., pitch, rhythm, intervals, phrases, a whole melody) (Herff, Olsen, Dean, et al., 2017). As these memory representations code partially redundant information, they provide protection against cumulative disruptive interference. Listeners are well experienced with the rules underlying music in a familiar tuning system and perceive and integrate melodies accordingly. This is not the case with melodies in unfamiliar tuning systems, disrupting integration of individual notes into larger structures such as a melody and thereby preventing resilience towards cumulative disruptive interference (Herff, Olsen, Dean, et al., 2017; Herff, Olsen, Prince, et al., 2017). Theoretically it is possible that background noise could also be integrated as part of the perceptual experience. However, in the case of mismatching context, this additional incongruency between memory and stimulus could have a detrimental effect during recognition, reducing resilience towards cumulative disruptive interference in the process. Whether the cumulative disruptive interference phenomenon is sensitive to context matching between memory encoding and retrieval is still under debate. In the present work, we investigated whether and how matching or mismatching background noise conditions during retrieval and encoding affect cumulative disruptive interference in memory for melody. The presence or absence of cumulative disruptive interference therefore provides an additional assessment of potential beneficial effects of background noise in form of context-dependency in memory for melody. Taken together, the present study tests for potential context effects of background noise on melody recognition performance, as well as cumulative disruptive interference in memory for melody. We hypothesized that there are context effects of background noise on melody recognition. Furthermore, we hypothesized that these context effects are observable in reduced melody
4
Musicae Scientiae 00(0)
recognition performance as well as in stronger cumulative disruptive interference in contextmismatching conditions.
Method Participants Forty participants were recruited. For power analysis and sample size estimation, please refer to the Statistical Approach section. One participant had to be removed from the sample for pressing the same button (“new”) in every single trial. The 39 remaining participants (Mage = 24.4 years, SDage = 6.0 years, 30 female, 9 males) had an average of 1.7 years of musical training (SD = 3.1 years) with a majority of participants (23) reporting that they had never received any formal musical training. Twenty-five students were recruited from the Heinrich Heine University in Düsseldorf, Germany, and 14 students were recruited from Western Sydney University, Australia.1 The study was approved by the Human Research Ethics Committee of Western Sydney University (H10847).
Stimuli In total, 40 melodies were randomly selected from the melody corpus previously used in Herff, Dean, and Olsen (2017) and Herff, Olsen, and Dean (2017). All melodies were 12 seconds in duration and tonal, composed in 12-tone equal temperament. Half of the melodies were composed in major and the other in minor keys and the key for each melody was randomly chosen before composition. All tones were sounded with the same grand piano timbre at the same velocity. The meter was balanced across the melodies between 4/4 and 3/4, and the tempi were pseudo-randomized between 80–165 beats per minute. Further information about the stimuli, including sounds files and musical feature analyses can be found in Herff, Olsen, and Dean (2017). Previously, an uninvolved expert listener (Ollen Musical Sophistication Index of 856, see Ollen (2006), where on a scale of 0–1000, > 500 is deemed to be musically sophisticated) described this corpus as like everyday tunes from TV programs, films, or adverts. Figure 1 shows notations of representative examples. All stimuli used in the present study, as well as detailed musical feature analysis of the stimuli can be found in Appendix S1 in the Supplemental Material Online section. Depending on the experimental condition, sometimes a background-talking (babbling) file was played simultaneously with the melody. The background noise consisted of continuous unintelligible multiple talker babbling, similar to background noise in a restaurant or bar. It is important to note that while the background noises in a real-world environment such as a restaurant are similar across an evening, the spectral context is not identical. Therefore, the background noise during encoding and retrieval should be similar, but not identical. We aimed to avoid using the same short babbling clip in each trial, which would have led to having an acoustically identical background sound played every time. Furthermore, we aimed to avoid using fundamentally different babblings sound files for each trial, which might provide further context information or may have limited generalizability to real-world environments such as a restaurant. Consequently, we chose a relatively long sound file (15 minutes) and played snippets of the same unintelligible multiple talker babbling file throughout the experiment (Auditec, St. Louis, MO; also used in Davis & Kim, 2010; Davis, Kim, Grauwinkel, & Mixdorff, 2006; Kim, Davis, Vignali, & Hill, 2005; Munhall, Jones, Callan, Kuratate, & Vatikiotis-Bateson, 2004). As a result, participants listened to the same background context throughout all noise conditions in the experiment,
Herff et al.
5
but never heard the same passage twice. This is important to note, because it means that the background noise during encoding and retrieval of the same melody was similar, yet not identical. Appendix S1 in the Supplemental Material Online section contains an example of a melody with and a melody without background noise. The level of the babbling file was adjusted to be 15 dB louder than the melodies, using a custom Max MSP patch (Cycling74, 2014).2
Figure 1. Examples of the melodies used in the present experiment.
Procedure Participants provided informed consent and were instructed that they would hear different melodies, one after another. The entire experiment was presented in a custom-made Max MSP patch (Cycling74, 2014) and is available from the first author upon request. Each melody was presented twice during a continuous recognition paradigm in random order (Shepard & Teghtsoonian, 1961) for a total of 80 trials. The number of intervening melodies until a target melody recurred was randomized. After each melody, participants were asked to indicate whether they had heard the melody in the experiment before.3 A continuous recognition paradigm has the advantage that there is no artificial division between a testing and a learning block (Dowling, 1991). Instead, similar to the real-world music experience, listeners perform the same task in every single trial regardless of whether or not they have heard the current melody before. Simultaneously, the number of intervening melodies until a target melody recurs can be easily manipulated, as well as whether or not some melodies are presented with background noise. Participants were instructed to pay no attention to the background babbling which was task irrelevant to them. There were four within-subject experimental conditions defined by whether or not background babbling (noise) was present. The order of background noise conditions was randomized for each trial as part of the continuous recognition paradigm. Each participant listened to 20 melodies in each experimental condition. The experimental conditions are illustrated in Table 1. Table 2 exemplifies the procedure in a few trials.
6
Musicae Scientiae 00(0)
Table 1. Experimental conditions. Second presentation First presentation
Clear
Noise
Clear
Clear-Clear
Clear-Noise
Noise
Noise-Clear
Noise-Noise
Note. Clear refers to no background babbling present during melody presentation. Noise refers to background babbling being present during melody presentation.
Table 2. Example of the procedure. Trial number
Melody
Presentation
Noise
Noise condition
Intervening items
1
Melody A
First
No Noise
Clear-Clear
−
2
Melody B
First
Noise
Noise-Clear
−
3
Melody C
First
No Noise
Clear-Noise
−
4
Melody A
Second
No Noise
Clear-Clear
2
5
Melody D
First
Noise
Noise-Noise
−
6
Melody C
Second
Noise
Clear-Noise
2
7
Melody B
Second
No Noise
Noise-Clear
4
8
Melody D
Second
Noise
Noise-Noise
2
Note. This is just an example of the procedure, the actual experiment consisted of 80 trials and each participant received an individual randomized trial list, as well as randomized melodies to the noise conditions. In this example, Melody A is presented in trial 1 and 4, hence two intervening trials between the first and second presentation of Melody A. Melody A is also in the Clear-Clear condition, so both presentations are without background noise. Melody B is presented in trial 2 and 7, so four intervening trials. Melody B is in the Noise-Clear condition, so there is background noise during the first presentation of the melody, but not during its second presentation.
Statistical approach In general, we used linear mixed effects models (Baayen, 2008; Baayen, Davidson, & Bates, 2008; Judd, Westfall, & Kenny, 2012; Kass & Raftery, 1995; Kruschke, 2010, 2013; Nathoo & Masson, 2016) to investigate potential context effect of background noise on melody recognition. The models were implemented in the R software platform (R-Core-Team, 2013) using the lme4 package (Bates, Maechler, Bolker, & Walker, 2013) and included the experimental fixed factor NoiseCondition together with other potential predictors as now described. Random Participant variation was taken into account in the form of a random intercept. Coefficients and p-values for the Noiseconditions are reported. A conservative approach was chosen by using Kenward–Roger approximations to estimate the degrees of freedom and corrected p-values are reported throughout (Kenward & Roger, 1997, 2009). Where applicable, we further assessed the significance of fixed factors with a direct model comparison approach using likelihood-ratio tests (Wilks, 1938). Melody recognition performance was assessed in terms of log linear corrected (Hautus, 1995) d-prime values (d’). d’ is calculated by subtracting the z-transformed false alarm rate from the z-transformed hit-rate (Kopiez, Wolf, Platz, & Mons, 2016; Macmillan & Creel, 2005). Here, false alarms are defined as first melody presentations where participants falsely indicate that this melody has been presented before. Hits are defined as second melody presentations
Herff et al.
7
where participants accurately indicate that this melody has been presented before. We calculated d’ separately for each participant for each of the four conditions. For the Clear-Noise and Clear-Clear, we calculated the false alarm rate on the combined data, since both conditions are indistinguishable during first melody presentation. The same goes for the Noise-Noise and Noise-Clear condition. To assess cumulative disruptive interference we deployed a method closely following previous work (Herff, Olsen, & Dean, 2017; Herff, Olsen, Dean, et al., 2017; Herff, Olsen, Prince, et al., 2017). Specifically, we used a generalized mixed effects model to predict binary recognition responses (“old” or “new” during second melody presentation). Mixed effects models provide a plethora of advantages over a more traditional ANOVA-type analysis as described, such as taking model complexity and cross random effects between participants and stimuli into account (Baayen, 2008; Baayen et al., 2008; Judd et al., 2012; Kass & Raftery, 1995; Kruschke, 2010, 2013; Nathoo & Masson, 2016). The models consisted of a fixed factor Number of Intervening Melodies, coding the number of trials between first and second presentation of a target melody, and random effects on Participant and Melody intercepts (Barr, Levy, Scheepers, & Tily, 2013). Coefficient p-values are reported as calculated by lme4 for generalized mixed effects models. To ensure that significance was not simply due to an increase in model complexity, significant coefficients were also further assessed in the form of a model comparison approach (Kruschke, 2011). For this, each model with a significant predictor was compared with the same model but without the significant predictor using likelihood-ratio tests (Wilks, 1938). To assess significance in light of increase in model complexity, we used differences in Bayes information criteria (Schwarz, 1978) and reported ΔBIC (Kass & Raftery, 1995). A ΔBIC of two or greater is considered “positive” evidence in favor of the model with lower BIC. A ΔBIC difference of six or greater is considered “strong” evidence (Kass & Raftery, 1995). Many memory paradigms such as the continuous recognition paradigm are subject to response tendency shifts (Berch, 1976; Donaldson & Murdock, 1968; Snodgrass & Corwin, 1988). These shifts describe changes in the response bias as an experiment progresses such as changes in response tendencies due to fatigue. To account for individual participant response tendencies and for how these tendencies might change over the course of the experiment, we trained participant-wise generalized mixed effects models on “old” responses on first presentations (False Alarm rates) based on trial number. The fitted model was then used to predict the probability of pressing “old” on a repetition trial, based solely on trial number. In turn, these predictions were then implemented in all models as a fixed Dynamic Response Tendency to account for response tendency shifts (see Herff, Olsen, & Dean, 2017; Herff, Olsen, Dean, et al., 2017, for further detail on dynamic response tendencies). Power analysis and sample size estimation. We used a simulation-based approach to estimate sample size. Since the present design closely follows previously published experiments we were able to obtain relatively precise coefficient estimates for data simulation. To simulate data we used model coefficients for sampling error, random participant variation, random melody variation, and random cumulative interference variation from Herff, Olsen, Prince, et al. (2017) and Herff, Olsen, and Dean (2017). With a maximum of 78 intervening melodies and 40 melodies in the present design as well as an alpha level of .05, 10000-fold simulation showed that we can capture cumulative disruptive interference with a coefficient of −.01 (the smallest observed in Herff, Olsen, Prince, et al., 2017) with a probability greater than 0.85 using the statistical analysis described above, if we recruit a sample of 36 participants. The R routine used for data simulation and power analysis can be obtained from the first author upon request.
8
Musicae Scientiae 00(0)
Figure 2. Participant-wise hit rates and false alarm rates in all four experimental conditions. The reference line represents chance level. Overall, participants performed significantly above chance (see text for more detail).
Results Overall, participants performed significantly above chance with overall d’ significantly above zero, t(38) = 14.08, p < .001, Cohen’s d = 4.51. Figure 2 shows participant-wise performance for each of the four conditions. Background noise conditions affected melody recognition performance. A mixed effects model predicting d’ with a random intercept for Participant (BIC = 291.30, LogLik = −138.07) improved significantly when provided with a fixed factor for Noise-Condition (BIC = 286.25, LogLik = −127.97, p < .001). Performance was significantly better in Clear-Clear compared to Noise-Clear (Coef = −.49, SE = .11, t(38) = − 4.49, p < .001, Cohen’s d = .87) and compared to Noise-Noise (Coef = −.33, SE = .11, t(38) = − 2.99, p < .005, Cohen’s d = .50), however, there was no significant difference between Clear-Clear and Clear-Noise (Coef = −.21, SE = .11, t(38) = − 1.88, p = .06, Cohen’s d = .33). Clear-Noise outperformed Noise-Clear (Coef = −.28, SE = .10, t(38) = − 2.76, p < .01, Cohen’s d = .52) but not Noise-Noise (Coef = −.12, SE = .12,
Herff et al.
9
Figure 3. d’ in all four conditions. Error bars represent a 95% confidence interval.
Figure 4. Probabilities of responding “old” on a second melody repetition, based on the number of intervening items between first and second presentation of a melody. Only the Noise-Clear condition showed significant cumulative disruptive interference than cannot be attributed to an increase in model complexity. Error bars represent a 95% confidence interval.
t(38) = − 1.00, p = .32, Cohen’s d = .19). Noise-Noise showed only at trend level better performance than Noise-Clear (Coef = .16, SE = .11, t(38) = 1.47, p = .15, Cohen’s d = .28). Figure 3 shows d’-values for all four experimental conditions.
Cumulative disruptive interference A generalized mixed effects model (BIC = 458.55, LogLik = −217.34) predicting “old” responses on second melody presentations using a random intercept for Melody and Participant, and a fixed factor controlling Dynamic Response Tendencies improved significantly (BIC = 459.72, LogLik = −214.94, ΔBIC = 1.06) in the Clear-Clear condition when provided with the Number of Intervening Items (Coef = −.015, SE = .007). However, this effect can be attributed simply to an increase in model complexity as indicated by the positive ΔBIC of 1.06.
10
Musicae Scientiae 00(0)
In the Clear-Noise condition (BIC = 497.58, LogLik = −236.86) the model showed no significant improvement (BIC = 500.04, LogLik = −235.10, p = .061, ΔBIC = 2.46) when provided with the Number of Intervening Items (Coef = −.014, SE = .007). Similarly, the Noise-Noise condition BIC = 476.26, LogLik = −255.53, p = .247) showed no significant effect (BIC = 480.89, LogLik = −225.53, ΔBIC = 4.63) of the Number of Intervening Items (Coef = −.009, SE = .008). The Noise-Clear condition was the only condition that showed a reliable disruptive effect of the number of intervening items that cannot be simply attributed to an increase in model complexity. The generalized mixed effects model (BIC = 523.14, LogLik = −249.64) improved significantly (BIC = 515.28, LogLik = −242.72, p < .001, ΔBIC = −7.86) when provided with the Number of Intervening Items (Coef = −.025, SE = .007). Figure 4 shows the effects of cumulative disruptive interference in all four noise conditions.
Discussion We explored potential context effects from background noise, here unintelligible multi-speaker babbling, on melody recognition. Usually, background noise is associated with disruptive effects on memory (Gilbert et al., 2014; Mattys et al., 2012), however, previous research has also shown beneficial context effects, when learning and testing context match (Baddeley et al., 2009, pp. 176–180; Mattys et al., 2013, pp. 75–82; Smith & Vela, 1992, 2001). Here, we did not find compelling support for general, universally beneficial context effects of background noise on melody recognition. Indeed, the present melody recognition experiment suggests that in general, background noise is not beneficial. However, we did find indications that when encoding already took place in suboptimal circumstances (multi-talker babbling background), then matching these conditions can be beneficial for retrieval. In the following, we discuss the present findings in terms of the effects of background noise on overall melody recognition performance, encoding and retrieval, as well as cumulative disruptive interference The study showed that melody recognition performance was best and significantly better when first and second melody presentation took place without background noise (clear-clear) compared to noise during first but not second melody presentation (noise-clear) as well as compared to background noise during both melody presentations (noise-noise). This is in line with a large body of previous literature showing disruptive effects of background noise on memory (Gilbert et al., 2014; Mattys et al., 2012). However, the clear-clear condition was not significantly different from the clear-noise condition, suggesting suboptimal conditions during memory encoding have a stronger disruptive effect than suboptimal conditions during retrieval. This finding is also similar to previous research using musical stimuli in divided attention paradigms showing worse melody recognition under divided attention (Herff & Czernochowski, 2017), as well as similar to non-musical stimuli in divided attention paradigms, which shows stronger destructive effects of disruption during encoding than during retrieval (Anderson, Craik, & Naveh-Benjamin, 1998; Craik, Naveh-Benjamin, Ishaik, & Anderson, 2000; Fernandes & Moscovitch, 2000; Iidaka, Anderson, Kapur, Cabeza, & Craik, 2000; Naveh-Benjamin, Craik, Guez, & Dori, 1998; Naveh-Benjamin, Craik, Perretta, & Tonev, 2000; Naveh-Benjamin, Kilb, & Fisher, 2006; Park, Smith, Dudley, & Lafronza, 1989). A likely explanation is that encoding and retrieval simply are two fundamentally different processes: encoding being a process that is strongly negatively affected by any form of disruption and retrieval being a process that is more resistant against many forms of divided attention (Naveh-Benjamin et al., 2006). Further support can be found in a PET study that showed separate, though partly overlapping, brain systems for encoding and retrieval of auditory stimuli (Fletcher et al., 1995). The finding that clear-noise led to significantly better performance than noise-clear further supports this.
Herff et al.
11
Interestingly, while performance in the clear-noise condition was significantly better than the noise-clear condition, it was statistically similar to noise-noise rather than being substantially better, despite the additional disruption during encoding. This suggests that background noise during second presentation of a melody can provide some form of a beneficial context effect on recognition performance, given that the first presentation also took place in a suboptimal, noisy environment. This supports that context effects do indeed influence memory performance. However, to conclude that background noise during retrieval provides a beneficial effect given that encoding also took place in suboptimal conditions, we would have needed to find significantly better performance in noise-noise over noise-clear. This comparison only yielded a result at trend level, but no statistically significant difference was observed. As a result, the noise-noise condition produced performance intermediate between clear-noise and noise-clear, not significantly different from either. A similar comparison between noise-noise over noiseclear has previously yielded significant results using words rather that melodies as stimuli and white noise rather than babbling as background noise (Creel et al., 2012). Potentially, we just lacked the statistical power to produce the same significant results, considering that we found the same tendency within our data. Furthermore, the range of musical training in the present sample (mean of 1.7 years of musical training, and a standard deviation of 3.1 years, with 23 participant that have never received any musical training) might have introduced additional noise in the data. Future studies could investigate differential effects of background noise on musical novices and experts. Alternatively, the context effect might be less pronounced in musical stimuli compared to words, or the somewhat more ecologically valid background babble used in the present study generally produces a less strong context effect than the white noise used in previous study. Further research could disentangle these possible explanations. More conclusive results, however, were observed by the analysis of cumulative disruptive interference. Previous studies have found a remarkable resilience of memory for melody towards cumulative disruptive interference (Herff, Olsen, & Dean, 2017). Memory for most stimuli rapidly deteriorates as the number of intervening items between first and second presentation increases (Bui et al., 2014; Campeanu et al., 2014; Deutsch, 1970, 1975; Donaldson & Murdock, 1968; Friedman, 1990; Hockley, 1992; Konkle et al., 2010; Nickerson, 1965; Olson, 1969; Poon & Fozard, 1980; Rakover & Cahlon, 2001; see Sadeh et al., 2014, for a review), memory for melody, however, shows only minimal disruptive effects of such kind (Herff, Olsen, & Dean, 2017). Here, we used the magnitude of cumulative interference as another indicator of the effects of background noise on melody recognition. Similar to previous research we did not find overall cumulative disruptive interference in memory for melodies in a familiar tuning system. When analyzing the noise conditions separately we still observed that clear-clear, clear-noise, as well as noise-noise show no significant cumulative disruptive interference. Noise-clear, on the other hand, showed significant cumulative disruptive interference that cannot be attributed to an increase in model complexity. This suggests that the mechanism underlying the destructive effects of background noise during encoding could be cumulative disruptive interference. The initial memory trace deteriorates over time and with each additional item, however only when background noise disrupted memory formation during encoding. As a result, it could be predicted that the disruptive effects of suboptimal encoding increases over times, when compared with melodies that have been encoded without background noise. This provides a compelling testable prediction for further studies. However the question remains, why does noise-clear enable cumulative disruptive interference, whereas noise-noise does not? A possible answer may be found in a recent regenerative multiple representations (RMR) conjecture (Herff, Olsen, & Dean, 2017; Herff, Olsen, Dean, et al., 2017; Herff, Olsen, Prince, et al., 2017).
12
Musicae Scientiae 00(0)
The RMR conjecture assumes that prior experience influences perception, and perception mediates memory formation (Herff, Olsen, & Dean, 2017). In general, the conjecture assumes that memory representations are subject to decay and interference (similar to Norman, 2013; Oberauer & Lewandowsky, 2011; Oberauer, Lewandowsky, Farrell, Jarrold, & Greaves, 2012). However, if we have formed multiple representations of the same stimulus, then these multiple representations can regenerate each other and provide additional protection against decay and interference (similar to Paivio, 1969). This may be the underlying mechanism that provides memory for melody with its resilience against interference, as we seem to perceive melodies as underlying components (notes, intervals, phrases) as well as integrated, coherent melodies simultaneously (Deutsch, 1986; Herff, Olsen, Prince, et al., 2017; Krumhansl, 1991, p. 295; Schneider, 1997, p. 119). This also explains why cumulative disruptive interference is stronger in melodies in an unfamiliar tuning system compared to melodies in a familiar tuning system (Herff, Olsen, Dean, et al., 2017). Prior experience with a tuning system enables the listeners to integrate a series of notes into a cohered melody and thus forms an additional memory representation that a melody in an unfamiliar tuning system, without prior experience, lacks. In the present experiment the additional background noise during encoding has a disruptive effect on later recognition, however, it may also provide an additional percept for the listener that forms a memory representation. If the second presentation now also has background noise then, similar to prior knowledge with a tuning system, the additional representation may help in regenerating the target melody representation, providing some resilience against cumulative disruptive interference. In other words, background noise influences and changes melody perception and thus becomes part of the memory. If the background noise is not present during second melody presentation, then the melody may indeed be perceived differently, possibly impeding melody recognition. A similar mechanism has already been considered in speech (Creel et al., 2012). For example, Creel, Aslin, and Tanenhaus (2008) suggest that extra-phonemic acoustic information (i.e., talker voice) is stored in addition to words themselves and affects word learning. In melody recognition, the difference between context matching and no context matching is not as strong as the difference between melodies in familiar and unfamiliar tuning systems, but it seems to be large enough to soften the slope of cumulative disruptive interference. An alternative explanation may lie in music’s intimate relationship with emotion and its possible implications for memory (Eschrich, Münte, & Altenmüller, 2008; Nguyen & Grahn, 2017). The arousal-mood hypothesis states that music influences arousal and changes the mood, which in turn affects cognitive performance (Husain, Thompson, & Schellenberg, 2002). It is possible that background noise may affect melody recognition by modulating valence and arousal. Unfortunately, the current experiment does not shed light on how background noise may effect emotions, which in turn may affect melody recognition. Instead, we strongly encourage future research to investigate the link between context, emotion, and melody recognition. Regardless of possible explanations provided here, the observation that context effects and background noise shape the nature of cumulative disruptive interference is interesting and warrants further exploration. The present findings demonstrate that there are auditory context effects in memory for melodies beyond purely identical spectral backgrounds. The context effect induced by multi-talker background babbling may bear implications for composers who strive to compose memorable motives or melodies (e.g., in movies or advertisement). Based on the present findings, reinstating the auditory background context could improve recognition performance when the first melody presentation took place in a noisy environment, such as an action-packed movie scene or busy advertisement. For example, in a video game that introduces a melody for a new character it would be important to consider likely later soundscapes
Herff et al.
13
in which this character may be encountered and adjust the background noise accordingly during first melody presentation. Theoretically, the present findings may inform domains outside of music. Future studies could replicate the present design with visual or speech stimuli to provide further insight into the effects of auditory background noise on study context. In particular, a further investigation of how auditory background noise modulates cumulative disruptive interference in recognition could possibly reveal intriguing similarities and differences between the stimuli.
Conclusions There is ample evidence that background noise negatively effects memory. However, the possibility that background noise may have beneficial properties in the form of a context effect has received less scientific attention. In the context of melody recognition we found no general beneficial effects of background babbling on recognition performance. However, we observed that encoding is more prone to disruption than retrieval and that noise during encoding, but not retrieval, leads to cumulative disruptive interference, whereas background noise during encoding and retrieval does not. Our results suggest that, if possible, background noise should be avoided as it negatively affects memory performance and introduces cumulative disruptive interference. However, if encoding is likely to take place in a noisy environment, then presenting background noise during retrieval may be beneficial. This is because matching background noise during encoding and retrievals appears to reduce cumulative disruptive interference. Acknowledgements We would like to thank Lauren Fairley for constructive feedback on an earlier version of this manuscript. We would also like to thank the Australia–Germany Joint Research Cooperation Scheme funded by German Academic Exchange Service (DAAD) and Universities Australia (UA) for the financial support. The Supplemental Material Online section contains the stimuli used in the present study, detailed musical feature analysis, and an example of a melody with and a melody without background noise.
Funding This study was supported by the Australia–Germany Joint Research Cooperation Scheme, funded by German Academic Exchange Service (DAAD) and Universities Australia (UA).
Notes 1. The experiment reported here is part of a larger investigation of context effects in memory for melody, funded by an Australian–German exchange grant. Overall performance was not significantly different between the Australian and the German sample. As a result, the present analysis combines the data of both samples. 2. While there is a plethora of research on speech recognition under adverse conditions, literature on melody recognition of healthy subjects in noisy environments is rather sparse. In speech recognition, a speech-shaped noise masker is known to not affect intelligibility until the noise becomes loud enough to energy mask the speech signal (around a signal-to-noise ratio of −3 dB) (Brungart, 2001). At around −15 dB signal to speech-shaped noise performance approaches chance level. For the present study we intended to use background noise with real-world implication and therefore chose unintelligible multiple talker babbling. We piloted our multiple talker babbling noise 15 dB louder than our target melodies and found that the melodies were still audible and memorable (average d’ = .9). 3. Participants could also indicate confidence on a 100 point vertical visual analogue scale. Data not reported here.
14
Musicae Scientiae 00(0)
ORCID iD Steffen A. Herff
https://orcid.org/0000-0001-8396-1306
References Anderson, N. D., Craik, F. I. M., & Naveh-Benjamin, M. (1998). The attentional demands of encoding and retrieval in younger and older adults: I. Evidence from divided attention costs. Psychology and aging, 13, 405–423. Baayen, R. H. (2008). Analyzing linguistic data: A practical introduction to statistics using r. New York, NY: Cambridge University Press. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. Baddeley, A., Eysenck, M. W., & Anderson, M. C. (2009). Memory. New York, NY: Psychology Press. Bailes, F. (2007). The prevalence and nature of imagined music in the everyday lives of music students. Psychology of Music, 35, 555–570. Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. Bates, D. M., Maechler, M., Bolker, B., & Walker, S. (2013). Lme4: Linear mixed-effects models using eigen and s4. R package version, 1, 2–23. Berch, D. B. (1976). Criterion change in continuous recognition memory: A sequential effect. Bulletin of the Psychonomic Society, 7, 309–312. Brungart, D. S. (2001). Informational and energetic masking effects in the perception of two simultaneous talkers. Journal of the Acoustical Society of America, 109, 1101–1109. Bui, D. C., Maddox, G. B., Zou, F., & Hale, S. S. (2014). Examining the lag effect under incidental encoding: Contributions of semantic priming and reminding. Quarterly Journal of Experimental Psychology, 67, 2134–2148. Campeanu, S., Craik, F. I. M., Backer, K. C., & Alain, C. (2014). Voice reinstatement modulates neural indices of continuous word recognition. Neuropsychologia, 62, 233–244. Craik, F. I. M., Naveh-Benjamin, M., Ishaik, G., & Anderson, N. D. (2000). Divided attention during encoding and retrieval: Differential control effects? Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 1744–1749. Creel, S. C., Aslin, R. N., & Tanenhaus, M. K. (2008). Heeding the voice of experience: The role of talker variation in lexical access. Cognition, 106, 633–664. Creel, S. C., Aslin, R. N., & Tanenhaus, M. K. (2012). Word learning under adverse listening conditions: Context-specific recognition. Language and Cognitive Processes, 27, 1021–1038. Cuddy, L. L., & Duffin, J. (2005). Music, memory, and Alzheimer’s disease: Is music recognition spared in dementia, and how can it be assessed? Medical Hypotheses, 64, 229–235. Cycling74. (2014). Max-msp. Retrieved from http://cycling74.com/products/max/ Davis, C., & Kim, J. (2010, December). Is speech produced in noise more distinct and/or consistent? In M. Tabain, J. Fletcher, D. Grayden, J. Hajek, & A. Butcher (Eds.), Proceedings of the 13th Australasian International Conference on Speech Science and Technology, Melbourne, Australia (pp. 46–49). Canberra, Australia: ASSTA. Davis, C., Kim, J., Grauwinkel, K., & Mixdorff, H. (2006, May). Lombard speech: Auditory (a), visual (v) and av effects. Paper presented at the Proceedings of the Third International Conference on Speech Prosody, Dresden, Germany. Deutsch, D. (1970). Tones and numbers: Specificity of interference in immediate memory. Science, 168, 1604–1605. Deutsch, D. (1975). The organization of short-term memory for a single acoustic attribute. In D. Deutsch & J. A. Deutsch (Eds.), Short-term memory (pp. 107–151). New York, NY: Academic Press. Deutsch, D. (1986). Auditory pattern recognition. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (Vol. II, Cognitive Processes and Performance, pp. 32– 49). New York, NY: Wiley.
Herff et al.
15
Donaldson, W., & Murdock, B. B. (1968). Criterion change in continuous recognition memory. Journal of Experimental Psychology, 76, 325–330. Dowling, W. J. (1991). Tonal strength and melody recognition after long and short delays. Perception and Psychophysics, 50, 305–313. Eschrich, S., Münte, T. F., & Altenmüller, E. O. (2008). Unforgettable film music: The role of emotion in episodic long-term memory for music. BMC Neuroscience, 9, 48. doi:10.1186/1471–2202–9–48 Fernandes, M. A., & Moscovitch, M. (2000). Divided attention and memory: Evidence of substantial interference effects at retrieval and encoding. Journal of Experimental Psychology-General, 129, 155–176. Fletcher, P. C., Frith, C. D., Grasby, P. M., Shallice, T., Frackowiak, R. S. J., & Dolan, R. J. (1995). Brain systems for encoding and retrieval of auditory–verbal memory: An in vivo study in humans. Brain, 118, 401–416. Friedman, D. (1990). ERPs during continuous recognition memory for words. Biological Psychology, 30, 61–87. Gilbert, R. C., Chandrasekaran, B., & Smiljanic, R. (2014). Recognition memory in noise for speech of varying intelligibility. Journal of the Acoustical Society of America, 135, 389–399. Godden, D., & Baddeley, A. (1980). When does context influence recognition memory. British Journal of Psychology, 71, 99–104. Grant, H. M., Bredahl, L. C., Clay, J., Ferrie, J., Groves, J. E., McDorman, T. A., & Dark, V. J. (1998). Contextdependent memory for meaningful material: Information for students. Applied Cognitive Psychology, 12, 617–623. Halpern, A. R., & Bartlett, J. C. (2011). The persistence of musical memories: A descriptive study of earworms. Music Perception, 28, 425–431. Hautus, M. J. (1995). Corrections for extreme proportions and their biasing effects on estimated values of d’. Behavior Research Methods, 27, 46–51. Heinrich, A., Schneider, B. A., & Craik, F. I. M. (2008). Investigating the influence of continuous babble on auditory short-term memory performance. Quarterly Journal of Experimental Psychology, 61, 735–751. Herff, S. A., & Czernochowski, D. (2017). The role of divided attention and expertise in melody recognition. Musicae Scientiae. Advance online publication. doi:10.1177/1029864917731126 Herff, S. A., Dean, R. T., & Olsen, K. N. (2017). Inter-rater agreement in memory for melody as a measure of listeners’ similarity in music perception. Psychomusicology: Music, Mind & Brain, 27, 297–311. Herff, S. A., Olsen, K. N., & Dean, R. T. (2017). Resilient memories for melodies: The number of intervening melodies does not influence novel melody recognition. Quarterly Journal of Experimental Psychology. Advance online publication. doi:10.1080/17470218.2017.1318932 Herff, S. A., Olsen, K. N., Dean, R. T., & Prince, J. (2017). Memory for melodies in unfamiliar tuning systems: Investigating effects of recency and number of intervening items. Quarterly Journal of Experimental Psychology. Advance online publication. doi:10.1080/17470218.2017.1333519 Herff, S. A., Olsen, K. N., Prince, J., & Dean, R. T. (2017). Interference in memory for pitch-only and rhythmonly sequences. Musicae Scientiae. Advance online publication. doi:10.1177/1029864917695654 Hockley, W. E. (1992). Item versus associative information – further comparisons of forgetting rates. Journal of Experimental Psychology: Learning Memory and Cognition, 18, 1321–1330. Husain, G., Thompson, W. F., & Schellenberg, E. G. (2002). Effects of musical tempo and mode on arousal, mood, and spatial abilities. Music Perception, 20, 151–171. Iidaka, T., Anderson, N. D., Kapur, S., Cabeza, R., & Craik, F. I. M. (2000). The effect of divided attention on encoding and retrieval in episodic memory revealed by positron emission tomography. Journal of Cognitive Neuroscience, 12, 267–280. Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103, 54–69. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773– 795.
16
Musicae Scientiae 00(0)
Kenward, M. G., & Roger, J. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997. Kenward, M. G., & Roger, J. H. (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics and Data Analysis, 53, 2583–2595. Kim, J., Davis, C., Vignali, G., & Hill, H. (2005, July). A visual concomitant of the lombard reflex. Paper presented at the International Conference on Auditory-Visual Speech Processing (AVSP ’05), Vancouver Island, Canada. Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139, 558–578. Kopiez, R., Wolf, A., Platz, F., & Mons, J. (2016). Replacing the orchestra? The discernibility of sample library and live orchestra sounds. PloS One, 11, e0158324. doi:10.1371/journal.pone.0161911 Krumhansl, C. L. (1991). Music psychology – tonal structures in perception and memory. Annual Review of Psychology, 42, 277–303. Kruschke, J. K. (2010). Doing Bayesian data analysis: A tutorial introduction with r and BUGS. Burlington, MA: Academic Press. Kruschke, J. K. (2011). Bayesian assessment of null values via parameter estimation and model comparison. Perspectives on Psychological Science, 6, 299–312. Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142, 573–603. Macmillan, N. A., & Creel, S. C. (2005). Detection theory: A user’s guide. Mahwah, NJ: Lawrence Erlbaum Associates. Mattys, S. L., Bradlow, A. R., Davis, M. H., & Scott, S. (2013). Speech recognition in adverse conditions: Explorations in behaviour and neuroscience: Abingdon, UK: Psychology Press. Mattys, S. L., Davis, M. H., Bradlow, A. R., & Scott, S. K. (2012). Speech recognition in adverse conditions: A review. Language and Cognitive Processes, 27, 953–978. Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science, 15, 133–137. Nathoo, F. S., & Masson, M. E. J. (2016). Bayesian alternatives to null-hypothesis significance testing for repeated-measures designs. Journal of Mathematical Psychology, 72, 144–157. Naveh-Benjamin, M., Craik, F. I., Perretta, J. G., & Tonev, S. T. (2000). The effects of divided attention on encoding and retrieval processes: The resiliency of retrieval processes. Quarterly Journal of Experimental Psychology, 53, 609–625. Naveh-Benjamin, M., Craik, F. I. M., Guez, J., & Dori, H. (1998). Effects of divided attention on encoding and retrieval processes in human memory: Further support for an asymmetry. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1091. Naveh-Benjamin, M., Kilb, A., & Fisher, T. (2006). Concurrent task effects on memory encoding and retrieval: Further support for an asymmetry. Memory & Cognition, 34, 90–101. Nguyen, T., & Grahn, J. A. (2017). Mind your music: The effects of music-induced mood and arousal across different memory tasks. Psychomusicology: Music, Mind, and Brain, 27, 81. Nickerson, R. S. (1965). Short-term-memory for complex meaningful visual configurations – a demonstration of capacity. Canadian Journal of Psychology, 19, 155–160. Norman, D. A. (2013). Models of human memory. New York, NY: Elsevier. Oberauer, K., & Lewandowsky, S. (2011). Modeling working memory: A computational implementation of the time-based resource-sharing theory. Psychonomic Bulletin & Review, 18, 10–45. Oberauer, K., Lewandowsky, S., Farrell, S., Jarrold, C., & Greaves, M. (2012). Modeling working memory: An interference model of complex span. Psychonomic Bulletin & Review, 19, 779–819. Ollen, J. E. (2006). A criterion-related validity test of selected indicators of musical sophistication using expert ratings (Doctoral dissertation). The Ohio State University, Columbus, OH. Retrieved from https://etd. ohiolink.edu/!etd.send_file?accession=osu1161705351&disposition=inline Olson, G. M. (1969). Learning and retention in a continuous recognition task. Journal of Experimental Psychology, 81, 381–384.
Herff et al.
17
Paivio, A. (1969). Mental imagery in associative learning and memory. Psychological Review, 76, 241– 263. Park, D. C., Smith, A. D., Dudley, W. N., & Lafronza, V. N. (1989). Effects of age and a divided attention task presented during encoding and retrieval on memory. Journal of Experimental PsychologyLearning, Memory and Cognition, 15, 1185–1191. Poon, L. W., & Fozard, J. L. (1980). Age and word-frequency effects in continuous recognition memory. Journal of Gerontology, 35, 77–86. R-Core-Team. (2013). R: A language and enviornment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/ Rakover, S. S., & Cahlon, B. (2001). Face recognition: Cognitive and computational processes (Vol. 31). Amsterdam, the Netherlands: John Benjamins Publishing. Sadeh, T., Ozubko, J. D., Winocur, G., & Moscovitch, M. (2014). How we forget may depend on how we remember. Trends in Cognitive Sciences, 18, 26–36. Schneider, A. (1997). “Verschmelzung”, tonal fusion, and consonance: Carl Stumpf revisited. In M. Leman (Ed.), Music, gestalt, and computing: Studies in cognitive and systematic musicology. New York, NY: Springer. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. Shepard, R. N., & Teghtsoonian, M. (1961). Retention of information under conditions approaching a steady-state. Journal of Experimental Psychology, 62, 302–309. Smith, S. M., & Vela, E. (1992). Environmental context-dependent eyewitness recognition. Applied Cognitive Psychology, 6, 125–139. Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203–220. Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: Applications to dementia and amnesia. Journal of Experimental Psychology-General, 117, 34–50. Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio for testing composite hypotheses. The Annals of Mathematical Statistics, 9, 60–62. Williams, T. I. (2015). The classification of involuntary musical imagery: The case for earworms. Psychomusicology: Music, Mind, and Brain, 25, 5–13.
