schema driven properties in melody cognition

SCHEMA DRIVEN PROPERTIES IN MELODY COGNITION: EXPERIMENTS ON FINAL TONE EXTRAPOLATION BY MUSIC EXPERTS Jun-ichi Abe Etsuko Hoshino Hokkaido University Two experiments were conducted to investigate the final-tone extrapolating behavior of two music experts, one of Western classical music and the other of Japanese traditional music. The first experiment verified schema-driven properties of the experts' melody cognition. Both experts' final tone extrapolation was highly rule-governed, but the underlying properties of their melodic schema seemed quite different. The Western classical music expert tended to process given melodies within the Western diatonic tonal frame, while the Japanese traditional music expert tended to operate "bimusically." That is, the latter processed some stimulus tone sequences based on the Japanese traditional tonal frame, but processed others based on the diatonic tonal frame. It was also confirmed that the melodic cognition of the Western music expert was more harmony-oriented, while that of the Japanese music expert was relatively contour-oriented. The second experiment conducted a more detailed investigation of the Western music experts' rules in extrapolating thefinaltones. Only the relationship between the responses of this expert and the Western diatonic scale structures was focused on. The response structures of this expert were analyzed in contrast to her subjective tonal structures for stimulus melodies and are discussed in terms of her tonal schema. What kind of internal processings are executed when people listen to a melody? As these internal processes of melody cognition are not directly observable, appropriate behavioral indices which reflect the cognitive processes are necessary. In an earlier study (Hoshino and Abe, 1981), the authors attempted to identify characteristics of tone structures observed within each category of melody, and to identify properties of melody schemata which could be considered to be internal rules of melody cognition. The study utilized two kinds of response measures: whether a melody was acceptable as being "melody-like" and whether the melody was "memorable." Results indicated that there was a high positive correlation between acceptability and memorability of melodies. Moreover, the results suggested that acceptability and memorability of melodies were possibly related to the tonality of the melodies. However, the authors could not find any systematic relationship between characteristics of tone structures and acceptability or memorability of melodies. It was also found that the response measures employed in the experiments were not so acute that they could be reflective of individual differences in melody cognition. Because of this, the authors decided to investigate tonality and used melodic expectancy as the experimental measure. The definition of tonality is quite ambiguous. According to the Shin-ongagujiten (New Music Dictionary) (1977, p. 367) tonality is defined as that which "indicates predominance of a certain tone over others in a piece of music." Recent theories of music claim that the concept of tonality results from human cognitive expePsychomusicology, 1990 Volume 9, Number 2

161

Copyright 1990 Fsychomusicology

Schema in Melody Cognition rience of music, not vice versa. Hoshino and Abe (1984) tried to verify this psychological reality of "predominance of a certain tone in music" by using a paradigm for tone expectancy called final tone extrapolation. The method used involves presenting subjects with a tone sequence (a melodic phrase), and having them select a certain tone which they consider would be "the best ending" for the melodic phrase.1 For stimulus materials, in addition to 35 different six-tone melodies employed in previous experiments (Hoshino & Abe, 1981), fragmentary melodies which consisted of the first two, three, four, and five tones of each of the six-tone melodies were also employed. Subjects, 12 nonmusic major college students and 12 music major college students, were asked to close the phrase of each stimulus melody by adding a final tone. Results indicated that subjects' responses to all the fragmentary melodies derived from the same six-tone melody tended to fall into a very small number of certain tones when the original six-tone melody was either acceptable as "melodylike" or was considered memorable. On the other hand, when the six-tone melody was of low acceptability and memorability, the subjects' final tone responses tended to be varying and unstable through its fragmentary melodies. Figure 1 shows two extreme examples of those cases. (a) High-acceptable melodies Partial melodies o "

o-o-

|(**)—*C^)—^(^oTT^)—>(*°u)

$

QsolSJ

!»/sol4j

-or-e-

^z?) ^sol7J

(b) Low-acceptable melodies Partial melodies

^ (*si 9 ) o

-o

«» II o

>^si 4 J

o

^do6J Mni4j

®

>^do7J

(*mr4j

Figure 1. Examples of the results of final-tone extrapolating responses for each fragmentary tone sequence derived from the melody with high acceptability (a) and the melody with low acceptability (b). Tone names shown in ovals are denoted in terms of "movable do." The number following each tone name indicates the number of the subjects who selected that tone as the most appropriate final tone for each fragmentary melody. The number of all subjects was 24. (From Hoshino & Abe, 1984.) 162

Abe and Hoshino In the analyses of these results, the tone in which the subjects' responses were concentrated was considered to serve as a reference tone to unify sequentially presented tones into a melodic phrase. The results were interpreted to mean that people would accept a tone sequence as a melodic melody when its reference tone was easy to discover, but would not when its reference tone was difficult to find. The authors' next concern dealt with the kind of pitch positions, relative to each tone sequence, that were extrapolated as final tones by these subjects. As Figure 1 (a) indicates, the relative position of the final tone most selected within a certain scale was the tone do? This particular tone is equivalent to the tonic (or the key note) of the Western diatonic scale or the Yonanuki major mode,3 one of the Japanese pentatonic scales. Similar results were obtained for many of the subjects' responses to stimulus melodies which were highly accepted as melody-like. These results suggest that Japanese college students process melodies within the tonal frame of the Western diatonic (or the Yonanuki) scale. It was also found, however, that there was variability within this diatonic melody cognition. For instance, there were some cases where the same melodic sequence elicited different diatonic final tones among individuals (Figure 2a). On the other hand, in other melodies, most subjects agreed with one specific tone as the final tone despite the theoretical possibility for multiple selection of diatonic final tones (Figure 2b). (a)

m ft

U m

ffa vsv

(b)

Final tone

Final tone

h

v M jj

**ZSL «» o

4% t t ^ «' «' o n

c*

" ci o

J

" Y

c»

3 E

a

r*

" o

id>d>r I td>i i id> Yonanuki minor mode

$

0

\>0-

Figure 4. Western diatonic scales and Japanese scales. O indicates the tonic or nuclear tone. $ indicates other tones within each scale. Each scale structure is indicated in terms of its intervallic structure on the horizontal line divided into half steps. In the cases of Ritsu and Miyakobushi, the fifth scale-tone is raised in its ascending pattern which is parenthesized in the figure. A more specific difference between the two experts could be recognized in the intervallic relationships between a responded final tone and the preceding tone. In the case of the Western classical music expert, a half-step ascending sequence (+1) was frequently observed. In the case of the Japanese traditional music expert, a whole-step ascending sequence (+2) was more apparent. Such a difference was probably derived from each subject's sensitivity to a leading note, which was originated from his or her background in music training. In Western classical music, the half-step interval of (ti-do) is frequently used at a cadence. Therefore, this particular interval becomes very different from any other interval for listeners of Western diatonic music in respect to its psychological effects on closing a phrase. On the other hand, various pentatonics used in Japanese traditional music do not contain a 167

Schema in Melody Cognition half-step interval (except the "In" scale), and the tone of a whole-step below is generally used as a leading tone toward the tonic or the nuclear tone. Characteristics of each expert's melody cognition can be observed also in the responses categorized as "Others" in Figure 3. In the case of the Western classical music expert, she appeared to have processed a nonscale tone in a tone sequence as a passing tone or an ornament of its surrounding scale tones. For instance, when the authors interpreted the extrapolated final tone as do (the tonic of a major mode in terms of the concept of "movable do"), the most frequently observed nonscale tones were re#,fa#, and sol# which occurred between do and mi, between do and sol, or preceding do, mi, or sol (e.g., [do-re#-mi —> do], \fa#-sol-do —> do]). Thus, the subject appeared to adopt do, mi, and sol, which constitute the tonic chord, as a scaffolding in her melody cognition, and thereby assimilated nonscale tones of a stimulus melody within their diatonic tonal surroundings. In the case of the Japanese traditional music expert, the authors could not find any particular regularity in his responses in the category of "Others," even though most of his responses were categorized here (34,9%). There are two factors which may explain this irregularity in his responses: the level of his expertise in Japanese traditional music and the lack of perceptual clues in such short stimulus melodies which were presented in this experiment. Concerning the former, this subject had not studied Japanese traditional music for as long as nor as much as the other subject had studied Western classical music, and so he might not have acquired a firm tonal schema for Japanese traditional music. If that were the case, he could not undertake the final tone responses as regularly as the Western classical music expert did. In present-day Japan, it is impossible for one to be exposed to only Japanese traditional music. In fact, people generally have more experience in listening to diatonic music than Japanese traditional music in their daily lives. In such a music environment, it is impossible to develop a strict "traditional Japanese" tonal schema, but rather, one may develop a himusical tonal schema which can be utilized in listening to both Western diatonic music and Japanese traditional music. Concerning the latter factor, the Japanese traditional music expert may have had consistent difficulties in processing melodies, because the stimulus melodies were not so long as to have sufficient clues for him to determine their music origin. Also, the use of a Western music instrument as a means for response may have contributed to his confusion between the two music systems. In any case, it would be desirable to investigate the processing rules of a Japanese traditional music expert again, by using a subject who has more expertise in that music and also by using a response means which was more relevant to his/her music. Experiment II: Thorough investigation of the Western classical music expert From the results of Experiment I, it was found that the final tone extrapolation by the Western classical music expert was highly rule-governed. It was also confirmed that the rules observed in her responses were probably related to the diatonic tonal structure of Western classical music. 168

Abe and Hoshino In Experiment II, the authors conducted a more detailed investigation of her rules in extrapolating the final tones. The authors focused only on the relationship between the responses of this expert and the Western diatonic scale structures. Method The subject was the same expert in Western classical music as in Experiment I. Stimulus materials and the response procedure were the same as Experiment I except that the stimulus materials consisted of all possible 469 types of three-tone sequences of which pitch range was restricted to one octave. To investigate reliability of the subject's responses, the whole experiment was conducted twice. In both sessions, the absolute pitch position of each stimulus tone sequence was randomly assigned to some position within the range from A3 (220.00 Hz) to G5 (783.99 Hz). The presentation order of those stimulus melodies was also random. Results and Discussion Reliability of the subject's responses was verified by the fact that she selected the same final tone for 326 out of 469 types of stimulus tone sequences through the two sessions. The following analyses were based on the results obtained in the first session. The authors categorized the tone structures of the 469 stimulus and response sequences as follows: those of the three-tone stimulus sequences were categorized as "the stimulus structures," those of the four-tone response sequences were categorized as "the response structures" and the subjective scale structures of the fourtone response sequences as "the cognitive structures." The authors then analyzed the relationships among those tone structures. The results are shown in Table 1. The stimulus structures were defined in terms of whether or not all three tones within a stimulus tone sequence could be interpreted as the diatonic scale tones in the major or natural minor modes. For instance, there are four different diatonic interpretations for the stimulus tone sequence ([+7, +2]: i.e., [do-sol-la], [re-la-ti], [fa-do-re], and [sol-re-mi]). The number of possible interpretations in the diatonic scale ranges from 0 (when the stimulus tone sequence does not fit in the diatonic scale, e.g., [+1, +1]) to 8 (e.g., [0,0], [+12,0]). The number of possible interpretations for each stimulus sequence are shown under the first column of the "Stimulus Structures" in Table 1. For instance, there are 60 types of stimulus tone sequences which have 0 possible diatonic interpretation. Of these 60 types, 42 can fit into the melodic minor or the harmonic minor scale, both of which were derived from the natural minor scale. The remaining 18 types do not fit into even these sub-diatonic scales. Results under the column of the "Response Structures" indicate the number of four-tone response sequences for which all tones can be interpreted as diatonic scale tones when each extrapolated final tone is regarded as do or la. The whole category was subdivided into five sub-categories: the cases when the four-tone response sequences fit only the major scale (M), the cases when they fit only the natural minor scale (m), the cases when they fit both (3vl & m), the cases when they fit either me169

Schema in Melody Cognition Table 1 Classification of tone structures of all the 469 three-tone stimulus sequences and those of the corresponding "response structures" and "cognitive structures" of the four-tone response sequences produced by the Western classical music expert in Experiment II. Stimulus Structures

Response Structures

Cognitive Structures

Number of possible diatonic Number of interpreta- tone tions sequences

Objective structures of the four-tone response sequences

Subjective structures of the four-tone response sequences

7 24

Number of tone sequences

M&m M&m M M&m M tm M&m

7 23 1 32 4 6 6

M M M M M m M m M m m M .? M m [M m M m m M Miyako M m m M m Miyako Min'yo m M m Miyako ? M m Miyako

Number of tone sequences

7 23 1 42 32 "5" 4 "4" 6 3 60 3 M 29 29 m 20 20 2 mhm 2 2 none 3 1 M 52 "3" 52 78 m 24 24 none 2 1 1 M 58 58 ••2" 108 m 35 35 1 1 mhm 14 2 none 12 M 90 49 49 "1" m 7 7 10 mhm 10 24 12 none 2 9 1 "0" (mhm) 42 33 33 mhm 4 9 none 3 1 1 6 (Impossible) 18 18 1 3 8 469 Total 469 469 Note: M = major scale, m = minor scale; mhm = either melodic or harmonic minor scale; Miyako = Miyakobushi scale, Min'yo = Min'yo scale, ? = unrecognizable "6"

170

Abe and Hoshino Iodic or harmonic minor scale (mhm), and finally the cases when they fit none of the scales (none). The number of tone sequences belonging to each type of these scales are indicated in this column. The column of the "Cognitive Structures" indicates the subject's own recognition of the scale to which she judged that each response sequence would belong. Five types of scales, the diatonic major mode (M), minor mode (m), Miyakobushi (Miyako), Min'yo (Min'yo), and unrecognizable (?) were reported. The number of tone sequences belonging to each type of these scales are indicated in this column. The results show no contradiction between the objective response structures and the subjective cognitive structures. This suggests again that this expert's melody cognition is strongly limited by the Western diatonic tonal framework. As seen in Table 1, there were only 36 types of tone sequences which were not subjectively recognized as Western diatonic tone sequences. Among these nondiatonically recognized tone sequences, 25 types were recognized as Miyakobushi, one type was recognized as Min'yo, and the remaining 10 types were unrecognized. Many of these 36 types have such stimulus tone structure as a half-step interval followed by a fourth. Such characteristics might force the subject to recognize the sequences nondiatonically. Even of the 18 types of tone sequences whose stimulus structures could not fit into any of the diatonic and the subdiatonic scales, seven of them were subjectively recognized as diatonic sequences. General Discussion Through Experiments I and II, it was confirmed that music experts' melodic cognition is highly schema-driven. Although the same kind of schema-driven properties could be observed to some extent in the melody cognition of the novice (Hoshino & Abe, 1984), those properties were evidently highly stable, elaborative, and distinctive in the melody cognition of the expert. The results of the present experiments verified that prominent properties of the tonal schemata are related to theoretical concepts such as key, scale, and mode. But, unfortunately, these theoretical concepts are considered to be only reflections of "static" properties of the internal schema for melody cognition. Therefore, the authors still need to investigate the "dynamic" properties of the schemata which have not yet been clarified. For example, though the stimulus tone sequence (+7, +2) has four different diatonic tone combinations available, the Western classical music expert selected only the interpretation [sol- remi —» do]. How did this expert recognize this sequence as such? More basically, how did she process the stimulus tone sequence? How did she decide upon a particular tone as the final tone? How did she internally represent the tone sequence as such? How did she interpret the tone sequence as a melody? To answer such questions, the authors are currently developing a dynamic model (i.e., a computer simulation model) of her final-tone extrapolation processes (see Abe, et al., 1984). The authors believe that such an attempt will allow clearer understanding of the dynamic properties of the internal tonal schemata. Also, further exploration of a variety of music schemata is needed; not only the Western diatonic schema but also the schemata for the music of other cultures, thebimusical ormultimusical schemata, and the novice's 171

Schema in Melody Cognition undeveloped schema. It is to be hoped that investigations of such a variety of schemata will give more knowledge about music cognition and acquisition from a universal viewpoint. References Abe, J., Hoshino, E., & Yamamoto, M. (1984). Merodi ninchi sukima ni tsuite no kenkyu: Shushi-on-doshutsu-katei no shimyuresyon [A study of the schema in melody cognition: Simulating the final-tone extrapolation behavior of a music expert]. (Tech. Rep. No. MA84-3). Musical Acoustics Group, Acoustical Society of Japan. Hoshino, E., & Abe, J. (1981). Merodi ninchi ni okeru "chosei-kan" to "patan no matomarisei" ["Tonality" and "coherence" in melody cognition]. Hokkaido Behavioral Science Report, Series P (Supplement), No. 23. Hoshino, E., & Abe, J. (1984). Merodi ninchi ni okeru "chosei-kan" to shushi-on doshutsu ["Tonality" and the final tone extrapolation in melody cognition]. The Japanese Journal of Psychology, 54,344-350. Koizumi, F. (1964). Nihon dento ongaku no kenkyu [A study of Japanese traditional music]. Tokyo: Ongaku-no-Tomo-sha. (Original work published 1960) Shin-ongagu-jiten (New Music Dictionary). (1977). Tokyo: Ongaku-no-Toma-sha. Yanagida, T. (1932). Senritsu no jikken-teki kenkyu [An experimental study of melody]. The Japanese Journal of Psychology•, 7,559-596. Footnotes A similar procedure was employed in a part of Yanagida's experiment (1932). 2 In this paper, absolute pitch names are denoted as C, D, E, etc., and relative pitch names (tone names in relation to tone sequences within the diatonic frame) are denoted as do, re, mi, etc. 3 In this paper, the terms "scale" and "mode" may not be strictly distinguished. 4 The reason that the Yonanuki scale is classified as a Japanese scale here is because it has been commonly used in Japanese popular music since its origination during the Meiji period [1868-1912]. It has the same scale structure with the Yoh found in the Min'yo and Ritsu modes. The scale structure was thought to be derived from Western scale structures which were imported in the beginning of the Meiji period, and was then combined with Japanese traditional scale structures. 1

Author Notes This article is a translation by Mayumi Adachi of an article by the same title in The Japanese Journal ofPsychonomic Science, 1985,4( 1), 1 -9, and is printed here by permission of that Society. This study was supported by the Ministry of Education, Science, and Culture of Japan under Grant-in-Aid for Scientific Research, No. 59810002 (Jun-ichi Abe, principal investigator). Parts of the study were presented at the seminar of the Musical Acoustics Group, the Acoustical Society of Japan, on May 14,1984. The affiliation of the second author, Etsuko Hoshino, is now the Junior College Department, Ueno Gakuen College, Tokyo. Ms. Adachi is a doctoral student in the Systematic Musicology Program at the University of Washington, Seattle. Requests for reprints should be sent to Jun-ichi Abe, Division of Psychology, Department of Behavioral Science, Faculty of Letters, Hokkaido University, Sapporo 060, Japan.

172