Brain Imaging and Behavior DOI 10.1007/s11682-011-9116-5
ORIGINAL RESEARCH
The perception of harmonic triads: an fMRI study Takashi X. Fujisawa & Norman D. Cook
# Springer Science+Business Media, LLC 2011
Abstract We have undertaken an fMRI study of harmony perception in order to determine the relationship between the diatonic triads of Western harmony and brain activation. Subjects were 12 right-handed, male non-musicians. All stimuli consisted of two harmonic triads that did not contain dissonant intervals of 1 or 2 semitones, but differed between them by 0, ±1, ±2 or ±3 semitones and therefore differed in terms of their inherent stability (major and minor chords) or instability (diminished and augmented chords). These musical stimuli were chosen on the basis of a psychoacoustical model of triadic harmony that has previously been shown to explain the fundamental regularities of traditional harmony theory. The brain response to the chords could be distinguished within the right orbitofrontal cortex and cuneus/posterior cingulate gyrus. Moreover, the strongest hemodynamic responses were found for conditions of rising pitch leading from harmonic tension to modal resolution. Keywords fMRI . Harmony . Major . Minor . Orbitofrontal cortex . Psychoacoustics . Sound symbolism . Frequency code . Harmony map
No animals were used in this research, which was partially supported by university research grants and does not involve any financial relationship between the authors and those institutions. T. X. Fujisawa Graduate School of Biomedical Sciences, Nagasaki University, Nagasaki, Japan N. D. Cook (*) Department of Informatics, Kansai University, Takatsuki, Osaka, Japan 569-1095 e-mail:
[email protected]
Introduction The relative consonance (dissonance) of two-tone musical intervals has been studied psychophysically since Helmholtz (1877) and quantitative models have successfully explained the experimental pattern of interval perception, as reported by children and adults, musicians and non-musicians, and peoples from the East and the West (e.g., Plomp and Levelt 1965; Kameoka and Kuriyagawa 1969). The key insight that has allowed for successful modeling of interval perception is consideration of the role of the higher harmonics (~upper partials). Unfortunately, application of the same “interval perception” model to three-tone triads has not been successful in explaining the phenomena of musical harmony. If only interval dissonance effects (including those entailed by higher harmonics) are considered, the distinction between resolved and unresolved triads cannot be explained, and the different affective valence of major and minor chords remains a mystery. Such difficulties have led us to develop a model of harmony perception that includes a three-tone “tension” factor. “Harmonic tension,” as defined by Leonard Meyer (1956), is a consequence of three-tone pitch patterns where the middle-tone lies exactly midway between the upper and lower tones. We have converted that musical insight into a psychophysical model by proposing a theoretical tension curve that can be used to calculate the tension effects of all combinations upper partials. Details of the model can be found in the literature (Cook 2001, 2002, 2007, 2009, 2011; Cook and Fujisawa 2006; Cook et al. 2006, 2007; Cook and Hayashi 2008; Fujisawa 2004). Suffice it to say that, when both 2-tone dissonance and 3-tone tension are included in theoretical calculations, the well-known perceptual regularities of the harmonic triads and the incidence of their historical usage in classical music (Eberlein 1994)
Brain Imaging and Behavior
can be explained psychophysically without borrowing qualitative notions from traditional harmony theory and without resorting to cultural explanations of harmony perception. On the basis of that model, we have undertaken an fMRI study of harmony perception in order to determine the relationship between the common harmonic triads— psychophysically-defined—and brain activation. The psychophysics of interval perception and harmony perception Psychophysical models from the 1960s coherently explain the regularities of interval perception (Sethares 1999) by postulating: (i) the presence of a critical band of roughness (dissonance) in the vicinity of 1–2 semitones (Fig. 1a), and (ii) the cumulative effects of the dissonance among all combinations of fundamental frequencies and upper partials (Fig. 1b). The resulting “dissonance curve” for all intervals within one octave shows notable decreases in the total dissonance at intervals corresponding to most of the tones of the diatonic scales (explained in terms of the physiology of the cochlear membrane rather than on the basis of Renaissance ideas concerning integer ratios). Experimental data on interval perception match this theoretical curve reasonably well (e.g., Kameoka and Kuriyagawa 1969) (Fig. 1b) and suggest why diatonic scales and their subsets (principally, pentatonic scales) are used worldwide in so many different musical traditions. Although the dissonance curve (Fig. 1) continues to be an important success in the science of music perception, the total dissonance of triads (as calculated from all combinations of partials in the triads) does not explain the results from behavioral experiments on the perception of such chords (Table 1; Fig. 2). The usual explanation of this theoretical failure is that there is a firm (perhaps universal) psychophysical basis for the perception of 2-tone intervals (Fig. 1b), but that the perception of more complex musical stimuli—starting with 3-tone triads—is dominated by learning effects (musical traditions, training, etc.) that make the psychophysics of interval perception relatively unimportant in the perception of real music. Fig. 1 The psychophysical model of 2-tone interval perception (Plomp and Levelt 1965) a and the total dissonance curve b obtained when upper partials are also included in the calculations. The theoretical curve in (b) and the experimental data (filled circles) are from Kameoka and Kuriyagawa (1969)
Empirically, the main objection to a “cultural” explanation of harmony is the fact that normal subjects from various musical cultures and very young children with only minimal exposure to music and without musical training, distinguish among the common triads, and perceive the resolved/unresolved character and affective valence of major and minor chords in a consistent way. Specifically, augmented and diminished chords are perceived as being rather unstable, as compared to major and minor chords (Roberts 1986; Cook et al. 2007). Similarly, keeping all other variables constant, major chords are perceived as being relatively “strong,” “happy” and “bright” with a positive affective valence, compared to the slight negative affect of the minor chords (Kastner and Crowder 1990). These common perceptions remain inexplicable on the basis of the calculated consonance/dissonance of intervals. We have consequently introduced a three-tone “tension” factor to the psychophysical model specifically to solve the theoretical difficulty of explaining diatonic chord perception solely on the basis of interval dissonance. By considering the relative size of the two neighboring intervals in any triad (Fig. 2d) (Meyer 1956), the total “instability” can be calculated as the sum of two-tone dissonance and three-tone tension, and the results compared against experimental data on chord perception. As shown in Table 1, the model predictions of the relative stability of the most important triads in Western diatonic music (last column) agree well with both the results of behavioral experiments (columns 3 and 4) and historical usage (column 2). The circle of fifths and the cycle of modes By making a distinction between interval dissonance (Fig. 1a) and triadic tension (Fig. 2d), we have found that it is possible to explain the regularities of traditional diatonic harmony on a strictly psychophysical basis. That is, any triad containing a whole-tone or semitone dissonance will be “unstable” (requiring harmonic resolution through the movement of one or more tones to produce a stable triad) solely as a consequence of the dissonant
Brain Imaging and Behavior Table 1 The empirical and theoretical sonority of the common triads Empirical Sonority
Major Minor Dim Sus4 Aug
Theoretical Sonority Predicted by
Incidence in classical music
Evaluation in laboratory experiments
Various “interval models”
Eberlein (1994)
Roberts (1986)
Plomp and Levelt (1965)
1 2 3 4 5
(51%) (37%) (9%) (2%) (
