40
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
Towards a Semantic Web Based Model for the Tonal System in Standard IEEE 1599 Lorenzo Sommaruga SUPSI, Scuola Universitaria Professionale della Svizzera Italiana Semantic and Multimedia Systems Laboratory (LSMS) CH-6928 Manno, Switzerland
Email:
[email protected]
Denis Baggi SUPSI, Scuola Universitaria Professionale della Svizzera Italiana Semantic and Multimedia Systems Laboratory (LSMS) CH-6928 Manno, Switzerland
Email:
[email protected]
Abstract—XML based representations of music, such as the standard IEEE 1599, usually describe in detail all features of music as found in scores and/or synchronizes different music layers to allow navigation. The enhancement proposed here adds to this kind of definitions musicological concepts, allowing the future realization of inferences for an analytical understanding of a piece of tonal music. To this aim, an ontology about tonal music is defined and two initial prototypes, concerning the application of the semantic web technology, have been developed. The prototypes here describe in detail tonal music element formalizations and an example of the application of an inference rule about intervals. Index Terms — semantic web, ontology, tonal system, musicology, music analysis, notes, modes, intervals, chords, four-part harmony
I. INTRODUCTION The distinctive features of the new XML-based standard IEEE 1599 [1, 2] are: the possibility of describing all musical entities found in a score expressed in Common Western Notation (CWN); its ability, to synchronize different music layers [3], so that music can be heard, seen at the level of score and libretto, of additional images and videos, and even graphical representations for music that cannot be expressed in CWN. And, lastly, the possibility of extending the standard, using the intrinsic property of XML, for special purposes such as proprietary score writing, research projects and experimental music. This property is studied in the present work, which proposes how to gradually build a set of clauses that describe the key items used in composition and analysis of pieces of tonal music, as known in the West since the 17th century, and still in use today [4]. The kernel of the extension is a set of music entities or resources, starting
© 2009 ACADEMY PUBLISHER
from notes to advanced harmonic devices, which can be used as a dynamic knowledge base by a set of inferences, which can be as size-able and flexible as a given project needs. An ontological model is exploited here to capture the features of tonal music. Few illustrating examples of how to model the resources involved by means of a semantic web will be given here, after a short introduction of the basic concepts.
II. TONAL MUSIC It has been the music of the West for the last four centuries and beyond, that for which the singer or performer moves within a frame of reference, called tonality, from which derives the concept of key. The key is, eventually, one note which is heard as representing the first step of a scale ordering of the notes of a piece, and often it is the first and the last note of the piece. The next paragraphs introduce gradually these concepts, together with their encoding in XML, in a way compatible with IEEE 1599. A. The Notes An international standard defines note a as a sound with the frequency of 440 Hz. From it, all other notes are derived in order in the following row: a, (a#), b, c, (c#), d, (d#), e, f, (f#), g, (g#). For historical reasons, there are 5 of the 12 notes in parenthesis which correspond to the black keys of a piano, while the other 7 are the white keys, called natural. All notes are derived from the preceding one by multiplying the frequency by the 12th root of 2, or 1.059463094. Hence, note c has the frequency of 440 * [(12th root of 2) to the third power], or 523.25 Hz. This
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
41
short discussion on the frequencies has been given here for the sake of completeness, and is not part of the tonal model. In tonal music, one starts with the seven natural notes, which can be ordered in different ways called modes by starting on a given note: ionian, on c (sounds like Western major) dorian, on d (popular in modal jazz of the 60’s, e.g. by Miles Davis and John Coltrane; also popular in Brittany, Ireland, etc.) phrygian, on e (found in some Neapolitan songs) lydian, on f (basis of a theory by jazz musicologist George Russell [4]) mixolydian, on g (used among others by Claude Debussy in Fêtes) aelian, on a (similar to Western minor) locrian, on b (used in jazz improvisation on diminished chords). In Western tonal music, only the major and minor modes have been retained, which enjoy a special relationship explained below. The others, which relate to ancient pre-tonal music, have been mentioned for the sake of completeness. B. The Intervals It is a peculiar characteristic of Western music that of having a vertical dimension, namely, that several notes can be played at the same time – while about two thirds of World music is monodic, namely, only one note is played at a given time. This happens thanks to the definition of intervals, music entities consisting of two notes played together. Among the most important intervals we can mention the: unison, second, third, fourth, fifth, sixth, seventh, and octave. For example, an interval of a second with 2 semitones is c-d. C. Chords More than two intervals can be played together, superimposed in thirds. This gives rise to chords with three or four notes. For example, a vertical assembly of g-b-d-f is a dominant-seventh chord in the key of C-major. Another simpler example of a chord is the simultaneous playing of c-g-e. D. Four-Part Harmony This is a model used in Western music schools to emphasize the aspect of harmony, which is the ability of assembling sounds or notes sounding together and of concatenating them. In this model, meters are very simple and generally two halves or three halves, no expression such as forte is given, melody plays a limited role, and music color is non-existent. Pieces consist merely of sequences of four-note chords – often, one note is repeated in the vertical assembly – of the same duration. The four voices are called, from top to bottom, soprano, alto, tenor and bass. The following Fig. 1 is a trivial example of a piece in four-part harmony.
© 2009 ACADEMY PUBLISHER
Figure 1.
A trivial music piece in four-part harmony.
The importance of this model consists of the fact that, at least in theory, it is always possible to analytically reduce a piece of tonal music into its four-part structure, to extract its purely harmonic contents. Hence, for any piece of music, no matter how many voices and instruments it is played with, it is possible to identify the structural soprano and the bass. Various analytical techniques - Schenkerian analysis, metric reduction – could be implemented with a set of XML inferences. E. The Special Relationship between a Major Mode and its Corresponding Minor If one considers the key of C with note c-d-e-f-g-a-b, C-major, it can be seen that it contains the same note as the set a-b-c-d-e-f, which is called its corresponding minor, in this case a-minor. Western tonal music has used this fact in order to let a piece move freely between one key and its corresponding minor, the first degree of which is found a minor third below that of the major. Hence, the corresponding minor of, say, F-major is d-minor. The notes are, as stated, the same, except that, for melodic reasons, in the minor mode some notes may be raised, i.e. g and f may become g# and f#. This fact is used in the determination of the key of a piece, which could be e.g. both F-major and d-minor (and even g-dorian, which is beyond the scope of this article). III. AN ONTOLOGICAL MODEL FOR TONAL MUSIC Within the context of above, an interesting and powerful extension to the previous IEEE 1599 model, which also overcome other music ontologies (e.g. [7]), can be based on a semantic web where the set of music entities are modeled as RDF resources [8], starting from notes to advanced harmonic devices, that can be used as a dynamic knowledge base by a set of inferences, which can be as sizeable and flexible as a given project needs. An ontological model is exploited here to capture tonal music features, working at the basic granularity level of notes, etc., differently from the Music Ontology Specification [7] which covers mainly a higher level about composition, recording, artist, medium, etc. Few illustrating examples of how to model these resources by means of a semantic web will be given here, after a short introduction of the basic concepts.
42
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
A. The Ontology Formalization for Tonal Music The semantic web is a new form of web content which provides standard languages for expressing both data and rules for reasoning about the data. These languages encompass RDF (Resource Description Framework) [8] for metadata representation, OWL (Ontology Web Language) [9] for ontology representation, and other various formats for rules representation (RuleML, TRIPLE, N3, RDQL, RIF). An important common feature of most of these semantic web languages is that they are based on the Extensible Markup Language (XML). Therefore, RDF and OWL languages inherit all the beneficial features of XML including the definition of computable, self-descriptive, re-usable, non-proprietary documents. The Semantic Web is a net of interconnected resources which define the contents and their relationships. The same contents may be shared and processed both by humans and computer programs. A number of operations can be effectively executed on this semantic net including re-use, association/aggregation, integration, visualization, and automated processing /inferences. In particular, semantic web technologies are mature enough to be exploited and applied to the development of the tonal music model (Fig. 2). A semantic web-based environment provides a support for a shared representation of different notes facilitating their combinations in various modes. A key point in this modeling is the definition of a computable OWL ontology about tonal music, which represents a structural model of the resources world and which is detailed below. On the basis of this ontology, a number of resources, i.e. notes, intervals, etc, can be described in XML as a semantic web of interconnected RDF resources in RDF document. Within the ontology definition, the granularity level of the resource basic elements (notes) is a determinant aspect of their representation and will heavily impact on their aggregation. The user can interact with the resources which can be presented in an active way, facilitating the functionalities knowledge discovery in the tonal music, e.g. about specific intervals or chords.
An OWL-DL (Description Logics) expressive language [10] for the ontology model has been chosen to represent the tonal music model. The resulting formalization is outlined in Appendix A by detailing some of the main features contained in the .owl XML document. Once the ontology has been defined, as mentioned above, a number of note instances can now be described in RDF as Note resources (individuals), on the basis of the ontology.
Figure 3.
The definition of the ontology and instances in
Protégé
In this prototype, the main 12 notes have been formalised initially with the help of the Protégé tool (http://protege.stanford.edu/) for the definition of the ontology and instantiating its individuals (see Fig. 3). In a second step, they have been manually refined. The notes defined include, according to the ionian mode: c, c-sharp, d, d-sharp, e, f, f-sharp, g, g-sharp, a, b. It is worth noting that the previously named “c#” note (and similar ones) is here and in the rest referred to as “c-sharp” for simplicity in the XML representation. They are formalized in the form of XML RDF resource instances as shown in the following excerpt:
Figure 2.
A semantic web tonal music model.
© 2009 ACADEMY PUBLISHER
c Do white c-sharp black
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
43
[...] b Si white
B. Examples A number of examples of an interval and some possible inferences on the notes are sketched here. On the basis of the definition of a primitive set of resources, which represent all the notes and their combinations, a further step is an inference system where sizeable and flexible rules allow asking for desired derivations in the form of intervals and chords. 1) Intervals as written in Jazz Recalling the previously mentioned interval of a second, namely c-d, we can now represent it by explicitly adding the relation which links the note c to the note d with the expression:
Figure 4. The semantic web prototype interface with the description of resources (notes) explicitly existing in the semantic web, with no inferences.
The complete definition of the c note is now: c Do white
Alternatively, we could express such an interval property by means of an inference rule, as described in the next section. C. Examples of inferences Our prototype, based on Jena2, includes a general purpose rule-based reasoner. This allows a rule to be basically defined with an optional name, a list of body terms (premises), and a list of head terms (conclusions) according to a simple syntax template: [ Rule name: premise, premise, … -> conclusion, conclusion, … ] 1) Intervals by Inference Intervals can be deduced by means of an inference obtained by applying the following rules to a tonal music semantic web world: [IntervalRule: (?Note http://.../TonalMusic#is-one-semitone-above ?AnotherNote) -> (?AnotherNote http://.../TonalMusic#makes-interval ?Note)]
© 2009 ACADEMY PUBLISHER
Figure 5.
After the application of the Interval inference rule, some intervals are inferred from the semantic web world, e.g. the highlighted c-sharp-d.
In this rule, it is asserted that if a note (the ?Note variable) is-one-semitone-above another note (?AnotherNote), it is possible to conclude that the ?AnotherNote makes an interval with the first note (?Note). The first prototype for a graphical interface presents, in Fig. 4 and Fig. 5 respectively, the status of the semantic web world before and after the application of the rule. In fact, in Fig. 4 no rules are chosen to be applied (a so called raw model is used) and there are no inferences in the list of existing facts about the world. On the other hand, in Fig. 5 IntervalRule is selected and the list of inferred intervals appears below. Similarly, it is possible to define other generic or more specific rules which extend the explicit world of knowledge about the notes by adding derived conclusions.
44
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
The Note resource, which represents the basic element of the tonal music ontology, is defined as an OWL class:
A number of relationships are defined for dealing with intervals, etc. The makes-interval relation, which links two notes, is defined in owl as an ObjectProperty:
The makes-chord relation, which links more than two notes, is defined in OWL as an ObjectProperty:
Figure 6. The graphical interface of the NoteEngine prototype for tonal music. The semantic web world of resources and inferences about tonal music that can be structured according to the ontological model introduced here represents a first step for a system which, after the configuration of appropriate rules, can answer various questions such as:
Question: Which chord does the vertical assembly of note d - f - g – b correspond to, if any? It can answer with a conclusion like: Answer: Reordered as g - b -d - f, it is a dominantseventh chord in the key of C-major. A further development of the prototype consists of considering chords, progressions, and harmonies, and is deployed with a graphical user interface as the NoteEngine interactive web application based on web services, as shown in Fig. 6. II.
CONCLUSIONS
The work presented in this paper is only a first step to represent something as vast and complex as the tonal system. Nevertheless, it shows the power of a semantic web based extension to other XML based approaches such as IEEE 1599, which lends itself to representing musicological entities that were not considered by its designers. APPENDIX A THE TONAL MUSIC ONTOLOGY The tonal music ontology consists of an RDF-XML document and is presented here as a commented excerpt of the OWL specification.
It is an W3C's OWL ontology:
© 2009 ACADEMY PUBLISHER
A number of other attributes (properties) are useful to describe note objects (i.e,. instances of the note class) and are defined in OWL as DatatypeProperty. The name of the note: … [from c to b] …
The frequency of a particular note in Hz, which will assume for instance a value of 440 Hz for the note a: Frequency of the note 16- 20,000 Hz. Measure Unit: Hz.
A number of relationships are formalized in order to express the dependency of a note in another in a scale such as the ionian mode. The one semitone distance between two consecutive notes in a scale:
The one tone distance between two notes:
The two semitones relationship, which is equivalent to the one tone distance between two notes: […]
JOURNAL OF MULTIMEDIA, VOL. 4, NO. 1, FEBRUARY 2009
45
ACKNOWLEDGMENT
[9]
The authors wish to thank Nicola Boehny, for the work of its Bachelor Degree thesis in Computer Science at University of Applied Sciences of Southern Switzerland (2006-07) on Tonal Music in XML and the implementation of the NoteEngine prototype.
[10]
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
Baggi, D.: An IEEE Standard For Symbolic Music, IEEE COMPUTER, pp.100-102, November 2005. Haus, G., Longari, M.: A Multi-Layered Time-Based Music Description Approach based on XML, Computer Music Journal, MIT Press, Spring issue, 2005. Ludovico, Luca A.: An XML Multi-layer Framework For Music Information Description, Tesi di Dottorato di Ricerca, Univ. Degli Studi di Milano, Dec. 2006. Baggi, D.: Capire il Jazz, le strutture dello Swing, Surveys of CIMSI/SUPSI, Scuola Universitaria Professionale della Svizzera Italiana, www.supsi.ch, 2001, multimedia book with CD-ROM, in Italian. Catenazzi, N., Sommaruga, L.: "A Semantic Web Resource Navigator for Competence Based Learning" International Conference on Methods and Technologies for Learning ICMTL 2005, Palermo Italy, 9-11 March 2005, in Methods and Technologies for Learning, G. Chiazzese, M. Allegra, A. Chifari, and S. Ottaviano editors, pp. 527532, ISBN 1-84564-155-8, WIT Press (UK), 2005. Sommaruga, L., Roberti Foc, I.: "PeC: Persons and Competencies Navigator for the Semantic Web" SWAP04 Semantic Web Applications and Perspectives, Ancona (Italy), December 2004, URI: http://semanticweb.deit.univpm.it/swap2004/cameraready/ sommaruga.pdf Music Ontology Specification Specification Document 21 March 2007 http://purl.org/ontology/mo/ RDF Resource Description Framework, http://www.w3.org/RDF/
© 2009 ACADEMY PUBLISHER
OWL Ontology Web Language, http://www.w3.org/TR/owl-features OWL Web Ontology Language Guide W3C Rec. 10 February 2004, 1.1 The Species of OWL http://www.w3.org/TR/owl-guide/#OwlVarieties.
Lorenzo Sommaruga graduated in Computer Science at the University of Milan (Italy, 1989) and received the PhD in Psychology from Nottingham University (UK, 1993) on studies about Distributed Artificial Intelligence. He worked four years as lecturer and researcher at Univ. Carlos III of Madrid (Spain) and then in an Italian Government funded research project on elearning. He is currently a lecturer and researcher within the Innovation Technologies Department at the University of Applied Sciences of Southern Switzerland (SUPSI), where he is the head of the Semantic and Multimedia Lab. within the Information Systems and Networking Institute of the Innovative Technologies Dept., and he is co-director of the e-learning Lab. (eLab). He has been teaching and disseminating XML technology since 1998 and is actively applying it to different domains, from e-learning to intelligent web. His main interests and research areas concern the XML technology and new web standards, the Semantic Web, elearning systems and standards, knowledge representation, AI and agents, and innovative information and communication technologies. He is author of more than 40 international publications on journals, conferences and books in these areas. For the CV of Denis Baggi, see the Editorial “The New Standard IEEE 1599 To Encode Music with XML Symbols” in this issue.
