This talk is about the mathematical structure. TREE and. MUSIC .... Strings | Tree
state of the art | Construction | Applications | Conclusions ...
MaMuX March, 2012
David Rizo Univ. Alicante
Metrical trees music comparison, classification and composition
Trees & Science
Newton Tree
Trees & Music
This talk is about the mathematical structure TREE and MUSIC
HUMAN MUSICAL PROCESS
Creation
HUMAN MUSICAL PROCESS
Based on an abstraction / structure Creation
HUMAN MUSICAL PROCESS
Based on an abstraction / structure Symbols: score Creation
HUMAN MUSICAL PROCESS
Based on an abstraction / structure Symbols: score Creation Interpretation: produces
HUMAN MUSICAL PROCESS
Based on an abstraction / structure Symbols: score Creation Music Sound
Interpretation: produces
COMPUTER MUSICAL PROCESS
Creation
COMPUTER MUSICAL PROCESS
Creation
Based on an abstraction / structure
COMPUTER MUSICAL PROCESS
Creation
Based on an abstraction / structure
Symbols: score
COMPUTER MUSICAL PROCESS
Creation
Based on an abstraction / structure
Symbols: score
Interpretation: produces
COMPUTER MUSICAL PROCESS
Creation
Music Sound
Based on an abstraction / structure
Symbols: score
Interpretation: produces
COMPUTER MUSICAL PROCESS
Analysis
Music Sound
Based on an abstraction / structure
Symbols: score
Interpretation: produces
What role can trees play in music ?
Based on an abstraction / structure
What role can trees play in music ?
What role can trees play in computer music ?
REPRESENTATION • Internal
intermediate representation dramatically conditions the application
REPRESENTATION • Internal
intermediate representation dramatically conditions the application • e.g. Which
is better for printing a score?
REPRESENTATION • Internal
intermediate representation dramatically conditions the application • e.g. Which
a) Sequence of frequency and onset time in milliseconds
is better for printing a score?
REPRESENTATION • Internal
intermediate representation dramatically conditions the application • e.g. Which
a) Sequence of frequency and onset time in milliseconds
is better for printing a score?
b) Sequence of note names and figures
REPRESENTATION • Internal
intermediate representation dramatically conditions the application • e.g. Which
a) Sequence of frequency and onset time in milliseconds
is better for printing a score?
b) Sequence of note names and figures
REPRESENTATION
REPRESENTATION • Which
is best
REPRESENTATION • Which
is best
• depends
on the application
REPRESENTATION • Which
is best
• depends
on the application
• Composition
REPRESENTATION • Which
is best
• depends
on the application
• Composition • easily
generate new content
REPRESENTATION • Which
is best
• depends
on the application
• Composition • easily
generate new content
• Analysis
and retrieval
REPRESENTATION • Which
is best
• depends
on the application
• Composition • easily
generate new content
• Analysis • easily
and retrieval
capture important features
SYMBOLIC REPRESENTATIONS • Strings • n-grams • Graphs • Point
sets
• Geometric • Hidden • Spiral
Markov Models
arrays, etc....
• Trees
_
ABSTRACT DATA TYPE TREE B4
• Suitable E5 B4 h
h
I
• music V
fT
fD
AM
_
_
D5
A4
intermediate representation for: B4 D5 D5 D5 A4 h
h
h
h
information (MIR) C#5 V V retrieval V V
AM • composing
B4
C#5
h
n
V
V
fD
fD
AM
AM
A4
h
h
VI
VI
fD
fD
fD
fD
fT
fT
AM
AM
AM
AM
AM
AM
OVERVIEW •
Background •
•
•
Intermediate representations The origin: strings •
•
Previous uses of trees in computer music
Our tree approach •
metric tree construction
•
probabilistic k-testables
Applications •
classification and generation
•
composition
Strings | Tree state of the art | Construction | Applications | Conclusions
String = sequence of symbols
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch Rhythm
STRINGS n1 •A
n2
n3
···
n4
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch Rhythm
G
C
B
Note name
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch Rhythm
67
72
71
MIDI pitch
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch
67
72
71
MIDI pitch
Rhythm
1
1/2
1/2
Duration
STRINGS n1 •A
n2
n3
···
n4
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch
67
72
71
MIDI pitch
Rhythm
0
1
3/2
Onset
STRINGS n1 •A
n2
n3
···
n4
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note
! 2 " 4 ! !
Pitch
67
72
71
MIDI pitch
Rhythm
0
1
3/2
Onset
G,0
C,1
B,3/2
A combination
[Pitch, Onset]
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note or a relation 4th
! 2 " 4 ! !
2nd
interval
STRINGS n1 •A
n2
n3
···
n4
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note or a relation
! 2 " 4 ! !
4th
2nd
interval
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note or a relation UP
! 2 " 4 ! !
DOWN
contour
STRINGS n1 •A
n2
n3
n4
···
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note or a relation
! 2 " 4 ! !
UP
DOWN
contour
DECOUPLED STRINGS n1 •A
n2
n3
···
n4
nm
melody is represented as a sequence of symbols ni
• each
symbol ni may represent a note or a relation
G
0
C
1
B
3/2
onset
pitch
onset
pitch
onset
" 42 ! ! !
pitch
any of these dimensions
STRING LANGUAGES
STRING LANGUAGES • String
= sequence of symbols
STRING LANGUAGES • String • Each
= sequence of symbols
symbol belongs to an alphabet ∑
STRING LANGUAGES • String • Each
= sequence of symbols
symbol belongs to an alphabet ∑
• String
= sequence of ∑
STRING LANGUAGES • String • Each
= sequence of symbols
symbol belongs to an alphabet ∑
• String
= sequence of ∑
• denoted
by ∑* (meaning: 0 or more symbols)
STRING LANGUAGES • String • Each
= sequence of symbols
symbol belongs to an alphabet ∑
• String
= sequence of ∑
• denoted • ∑+
by ∑* (meaning: 0 or more symbols)
denotes 1 or more symbols
STRING LANGUAGES
STRING LANGUAGES • Language
= set of strings
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
• e.g. Let
∑ = {A, B, C},
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
• e.g. Let
∑ = {A, B, C},
the language AB+C is the set of strings
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
• e.g. Let
∑ = {A, B, C},
the language AB+C is the set of strings starting with A A
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
• e.g. Let
∑ = {A, B, C},
the language AB+C is the set of strings starting with A
containing 1 or more B A
B
STRING LANGUAGES • Language
= set of strings
automaton can generate and recognize this kind of language following states and transitions
• An
• e.g. Let
∑ = {A, B, C},
the language AB+C is the set of strings starting with A
containing 1 or more B A
B
C
ending with C
STRING LANGUAGES
STRING LANGUAGES •A
music language can be built on strings
STRING LANGUAGES •A
music language can be built on strings
• if
only notes are represented,
STRING LANGUAGES •A
music language can be built on strings
• if
only notes are represented, ∑ represents a combination pitch (∑p) and rhythm (∑r) ∑ = ∑p ⊗ ∑r
• alphabet
STRING LANGUAGES •A
music language can be built on strings
• if
only notes are represented, ∑ represents a combination pitch (∑p) and rhythm (∑r) ∑ = ∑p ⊗ ∑r
• alphabet
• Coupled
∑ = ∑p x ∑r
STRING LANGUAGES •A
music language can be built on strings
• if
only notes are represented, ∑ represents a combination pitch (∑p) and rhythm (∑r) ∑ = ∑p ⊗ ∑r
• alphabet
• Coupled
∑ = ∑p x ∑r
• Decoupled
∑ = ∑p U ∑r
PITCH ALPHABETS ∑p Absolute magnitudes ! Scientific pitch:
F4
C 5 D5 D5
G4
R
G4 A4 B4
! Pitch class:
5
0
2
2
7
R
7
9 11
! Contour:
x
+
+
=
-
=
+
! High Def. Contour
x
+2 +1
0
-2
0
+1 +1
! Intervals:
x
+7 +2
0
-7
0
+2 +2
! Relative to tonic:
5
2
7
Relative magnitudes
0
2
etc.....
R
7
9
+
11
|!p| = 89 |!p| = 13 |!p| = 3 |!p| = 5 |!p| = 49 |!p| = 13
RHYTHM ALPHABETS ∑r Absolute magnitudes ! Durations: n c c n ! Inter-onset intervals (IOIs): 1 1/2 1/2 1 Relative magnitudes • Duration relative to previous x 1/2 1 • Duration relative to most frequent 2 1 1 • Relation between IOIs (IORs): 1/2 1 2 ! Rhythmic contour: x - =
2
n
n
2
c
b
1/2 1/2 (2) IOIn = on+1 - on
1
1
2
2
2
2 +
1/4 =
=
etc.....
c
1/2 1 1
1
4
4 IORn = IOIn+1/IOIn 1 4 x - = +
MONOPHONIC
POLYPHONIC
STRINGS - POLYPHONY • Chord
sequence approach:
Onset group based
STRINGS - POLYPHONY • Chord
sequence approach:
Onset group based requires segmentation
∑ = (∑r x ∑+p )
{C,F,A}, 0
{G,D,F,B}, 2
{C,E,G,B}, 4
STRINGS - POLYPHONY • Chord
sequence approach:
Onset group based requires segmentation
{C,F,A}, 0
{G,D,F,B}, 2
{C,E,G,B}, 4
∑ = (∑r x ∑+p ) Named chord: requires chord analysis ∑ = set of chord names
IV
V
I
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc...
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
A
B
C
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
A
B
C
F
F
G
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
A
B
C
F
F
G
C
D
E
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
A
B
C
F
F
G
C
D
E
G
C
STRINGS - POLYPHONY • Individual
note approach
Not interleaved: requires voice analysis, deal with divisi, etc... sequential arrangement of all monophonic lines
∑ = note alphabet + voice
A
B
C
F
F
G
C
D
E
G
C
STRINGS - POLYPHONY • Individual
note approach
Interleaved
STRINGS - POLYPHONY • Individual
note approach
Interleaved
STRINGS - POLYPHONY • Individual
note approach
Interleaved
STRINGS - POLYPHONY • Individual
note approach
Interleaved notes sorted lexicographically (e.g. first order by onset, them by pitch)
STRINGS - POLYPHONY • Individual
note approach
Interleaved notes sorted lexicographically (e.g. first order by onset, them by pitch)
C,0
F,0
A,0
G,2
D,2
F2
B,2
C,4
E,4
G,4
C4
STRINGS - POLYPHONY • Individual
note approach
Interleaved notes sorted lexicographically (e.g. first order by onset, them by pitch)
∑ = ∑r x ∑p
C,0
F,0
A,0
G,2
D,2
F2
B,2
C,4
E,4
G,4
C4
Evolve Strings to Trees
Evolve Strings to Trees • Many
possible note representations (many alphabets ∑)
Evolve Strings to Trees • Many
possible note representations (many alphabets ∑)
• But...
difficult to represent structure
String
n1
n2
n3
n4
···
In strings each note /symbol is linked to | from just other note/symbol
nm
String
n1
n2
n3
n4
···
nm
In strings each note /symbol is linked to | from just other note/symbol If some structure is required? e.g. n1
n2
n3
n4
···
nm
STRUCTURE • e.g. “Alberti
C
G
E
G
Bass” like piano left hand accompaniment
C
G
E
G
D
G
F
G
C
G
E
G
STRUCTURE • e.g. “Alberti
C
G
E
G
Bass” like piano left hand accompaniment
C
G
E
G
D
G
F
G
C
G
E
G
STRUCTURE • e.g. “Alberti
Bass” like piano left hand accompaniment
C
I
C
G
E
G
G
E
G
D
G
F
G
C
G
E
G
STRUCTURE • e.g. “Alberti
Bass” like piano left hand accompaniment
I
C
G
E
G
I
C
G
V7
E
G
D
G
I
F
G
C
G
E
G
STRUCTURE • e.g. “Alberti
Bass” like piano left hand accompaniment
I
C
G
E
G
I
C
G
V7
E
G
D
G
I
F
G
C
G
E
G
STRUCTURE
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Production: Rewrite I as C G E G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Production: Rewrite I as C G E G
I
C
G
E
G
I →C G E G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Production: Rewrite I as C G E G
I
C
G
E
G
I →C G E G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Production: Rewrite I as C G E G
I →C G E G V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Production: Rewrite I as C G E G
I →C G E G V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Melody → I I V7 I
Production: Rewrite I as C G E G
I →C G E G V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Melody → I I V7 I
Production: Rewrite I as C G E G
I →C G E G
Terminals
V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
STRUCTURE Melody → I I V7 I
Production: Rewrite I as C G E G
I →C G E G
Terminals
V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
∈∑
GRAMMAR Melody → I I V7 I
Production: Rewrite I as C G E G
I →C G E G
Terminals
V7 →D G F G
I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
∈∑
GRAMMAR Melody → I I V7 I
Production: Rewrite I as C G E G
I →C G E G
Terminals
V7 →D G F G Parsing tree I
C
G
E
G
I
C
G
V7
E
G
D
G
F
I
G
C
G
E
G
∈∑
GRAMMAR • Generalizing
the grammar
Melody → ChordRealization+ ChordRealization → Note Note Note Note Note ∈ any ∑
TREE PROPERTY
Trees capture more easily structure than strings
Strings | Tree state of the art | Construction | Applications | Conclusions
MUSIC TREE REPRESENTATION previous uses Formal architecture Grammars Composition Analysis
MUSIC TREE REPRESENTATION Formal architecture
Musical form Grammars Song
1.Period
Composition O-Phrase
1.Motive
C-Phrase
Analysis
3.Period
2.Period
O-Phrase
2.Motive
C-Phrase
O-Phrase
1.Motive
C-Phrase
MUSIC TREE REPRESENTATION Lee’85 grammars
CHAPTER 3. MUSIC SIMILARITY WITH TREES
Formal architecture
$ # # #( # #
"!% ' &
Explain rhythm perception with grammars (a) First two bars of“Auld Lang Syne”
Grammars Composition
S→ ¯ |