Deep Predictive Models in Interactive Music

Deep Predictive Models in Interactive Music arXiv:1801.10492

Charles Martin - UiO Dept. Informatics

https://folk.uio.no/charlepm

https://charlesmartin.com.au

Music Generation

Interactive Music

Where can prediction fit into real-time interactive musical performance?

What is a prediction?

non-temporal:

temporal:

Now...

Models

Then...

Music Generation: Predicting note-by-note

Mozer (1994)

Eck & Schmidhuber (2003)

Magenta (Google) 2016-2017

Representations of Music Thin

Medium?

Thick

folkRNN

Performance RNN

WaveNet

Predicting the Input

MySong

Mixture Density RNN

Predicting “Physical” Input

Prediction as Processing

Wekinator

Predicting the Response

Continuator / AI Duet

(Trains variable-order Markov model on-line)

(Pre-trained RNN: memory state conditioned on-line)

Neural iPad Ensemble

Benefits of Prediction -

Temporality -

-

Proactivity -

-

Instruments are reactive. Performers are not - predict others in ensemble, results of own actions. New instruments could be proactive as well, changing mappings, or producing sounds in advance of performer/user commands.

Adaptability -

-

Interactive music framework not temporal, but music is. Deep models can introduce temporal predictions into these systems.

Musical interfaces can adapt to the needs, or expressions of the performer/user.

Generation -

Possibility for new layers of music that enhance solo performances.