... limitations. In its latest version it can be connected through .... are currently developing a networked laptop environment called Frequencyliator. [7], which ...
Network Performance: Strategies and Applications
Alain Renaud Pedro Rebelo Sonic Arts Research Centre Queen’s University Belfast BT7 1NN, Northern Ireland
Introduction The development of high-speed networks provides a medium, which is becoming increasingly usable for real time media applications. The development of network music performance (NMP) and composition systems presents challenges with technical and cultural implications. Due to various technical factors, research in NMP systems has traditionally been restricted to the work of a handful of groups. However, recent technical developments coupled with a nearly ubiquitous broadband infrastructure seems to have given a second life to NMPs. The Sonic Arts Research Centre (SARC) in Belfast has started to work on developing innovative approaches to the use of networks in music performance in composition, with the aim to make NMP systems as widespread and usable as possible. Much network performance research has as its goal the replication of existing performance conditions. This paper explores some possibilities in redefining relationships between musicians, performers and composers, performers and audience, performers and spaces. The aims of this research is to identify performance conditions that are only possible through the network and to reflect on how the network is changing the way we create and consume music. The paper describes different scenarios that take advantage of real-time high quality audio exchange over the internet. Distributed Ensembles Although the network is a natural extension for computer music systems and has been adopted as a model for platforms such as SuperCollider, instrumental ensembles often remain outside the possibilities of network performance. A distributed ensemble consists of a group of musicians that is distributed between two or more locations. Numerous experiments, addressing different types of repertoires have been carried out with view to answer the fundamental question of whether it is possible to engage in a coherent music performance situation with musicians in different sites. High-speed internet infrastructure such as Internet2 has been explored through projects such as SoundWire [1], led by Chris Chafe at CCRMA (University of Stanford). SoundWire acts as a platform for distributing audio between two or three locations in order to create a shared acoustic space within which musicians can play together. Such systems use audio to UDP software applications to transmit and receive high quality uncompressed audio signals through standard internet infrastructure. There are currently two main software applications, which are suitable for distributed ensemble performance. Firstly, the audio-to-UDP adapter originally developed by the SoundWire project allows up to three peers to be connected simultaneously and is able to support multiple channels of audio up to the network’s physical limitations. In its latest version it can be connected through
the Jack Audio Server [2] application which makes it a versatile streaming application should it be interfaced with other applications. Secondly, the Soundjack application, developed by Alexander Carôt at ISNM in Germany which allows a very direct connection from the input of the soundcard to the network interface. The application therefore transforms a computer into a dedicated UDP streamer, which is well suited for purely acoustic network performances (Soundjack is described in the Soundjack position paper [3]). SARC has been using both streamers in various collaborations with CCRMA on one hand, and ISNM and IRCAM on the other. Initial tests between SARC and CCRMA have been quite promising and have recently led to a three-site tele-concert between SARC, CCRMA and University of Washington. This consisted of a traditional Irish music seven-piece band distributed across the three sites (2 Fiddles, Celtic Harp, Flute, Uillean Pipe at SARC, a Fiddler was at CCRMA and a Uillean Piper at University of Washington). The audience was located at CCRMA and hence this acted as a central node for mixing and broadcasting of the individual signals. The local latency was reduced to a minimum on each local machine by using a low latency Linux kernel coupled with very high quality hardware. The latency induced by the network itself varied between 180ms and 220ms. This is quite high but physically unavoidable if we consider that the sites were more than 8000km apart. Musical strategies to overcome latency included a leader-follower which identifies a leader in one site who sets the tempo without trying to compensate for the delay. A second experience took place between SARC (Piano), ISNM (Bass) and IRCAM (Drums) using the Soundjack application. The ensemble played a selection of Jazz standards. The network latency was acceptable (around 40ms) and it led to a more interactive and dynamic performance. With relatively low and stable latency it was possible to focus on some more subtle modes of listening within the ensemble. It is clear that the way each instrument is amplified within each site plays a fundamental role in establishing the necessary conditions for ensemble playing. Musicians participating in these performances have reflected on the challenges and possibilities of network performances. While issues relating to latency can make certain music situations problematic, it was noted that musicians can easily adapt to new types of listening situations, particularly if these remain relatively stable. Even though playing music in real time over the Internet is fascinating there are still major steps to be taken not only to make the network better but also to find ways to increase the level of interactivity in distributed ensembles. Informal surveys with musicians who have taken part in networked performances have suggested the incorporation of additional communication mediums such as video as not having visual communication is often considered as a major drawback. It has also been suggested that the inclusion of visual and/or sensory cues could truly increase the level of interaction between performers. We are currently investigating the possibility of multiple audio and video streams which can be dedicated to broadcast particular cue-systems (e.g. ensemble leader, conductor). It is clear that some of the most exciting prospects lie in using the network as a medium for new types of performance and purpose-created music. As Tanaka illustrates, each network inherits a “specific temporal characteristic” [4] and it is this characteristic that needs be considered as a new medium for music composition and performance. The relationship between latency and distance is not unlike the relationship between instrumental source and acoustic reflections in a hall. Whereas standard ensemble performance relies on a common acoustic
space between performer and audience, distributed ensemble performances need to take into account the superimposition of acoustic spaces that is implicit in two or three way networks. A piece for instrumental sextet distributed between three sites with distinct acoustic characteristics is currently being developed. Rather than concentrating on the network itself, this project intends to concentrate on the spaces that become connected through the network. The use of space as a compositional element can be traced back to the antiphonal music of Giovanni Gabrielli in 17th century Venice, through to Iannis Xenakis, Karlheinz Stockhausen and John Cage. The implications of space for music composition have more recently been explored from the point of view of site-specificity [5] and electroacoustic spatialization [6]. Chamber music practice tends to rely on the acoustics of the same performance space for articulating relationships between various instruments. The placement of each individual player in a string quartet or the positioning of orchestral groups in Stockhausen’s Gruppen for Three Orchestras (1957) create particular conditions for composition. A performance condition which emphasizes distance, the dispersion of instrumental forces and the juxtaposition of different acoustic spaces, poses new and challenging questions for a composer. It is timely and important to question the role of composition in a context which has been dominated by the concerts of real-time interaction. It is important to observe and analyze the process through which a tradition, such as chamber music can be mediated and distributed in space. Remote Acoustics Remote Acoustics is the phenomenon of using a remote site as acoustic chamber for a sound source. Having several spaces networked together doesn’t only provide a medium for exchanging musical ideas but also allows for the sharing and superimposition of acoustic spaces. The combined latency of the network with the reverberation of remote acoustic spaces suggests a unique acoustic situation. An initial experiment was made using the multi-channel capabilities offered by the latest version of SoundWire (up to four channels simultaneously between the two sites) to map the strings of an electric cello at CCRMA to an array of loudspeakers at SARC’s SonicLab. The acoustic result of the acoustically dry electric cello being amplified and reverberated at SARC was sent back to CCRMA. The use of remote acoustics and the idea of exchanging acoustic spaces suggests the network acting as an extension of a closed acoustic space. The combination of several geographically displaced acoustic spaces is only possible in a networked situation. This approach deserves to be studied further as it can potentially lead to new types of acoustic conditions for performance, such as one instrumental source being transmitted to multiple halls for reverberation. Remote Soundscapes Soundscape composition is characterized by the use of sound environments as its main source. These environments are pre-recorded and subsequently manipulated, edited and mixed. The use of remote soundscapes aims to explore the unpredictability of real-time soundscapes. It involves the implementation of several points of presence at several locations with various sonic particularities. These points capture sound environments in real time and broadcast it to a central server. Composers, artists and audiences can access the server and navigate through the different environments while mixing and manipulating streams from different locations.
This application promotes an investigation into the relationships between multiple sound environments. The sonic material itself can be used as a composition tool or live improvisation. This can potentially lead to the notion of improvised soundscapes and would provide the electroacoustic soundscape-composer with an ever-changing palette of sonic environments. The notion of navigation is implemented as an interface which allows for selection and mixing through sound rather than reference to geographical location alone. It is intended that a webbased interface acts as a portal to access the multiple sound environments. As well as allowing for conventional mixing and selection, this interface will introduce interactive behaviors to be applied to each soundscape. These behaviors represent ways in which one soundscape can modulate, trigger or filter another. Due to the ever-changing and somewhat incidental nature of the sound materials, a simple set of behaviors can result in complex multi-layered sound environments that constantly re-invent themselves. Networked Laptop Improvisation The implementation of networked technologies into a laptop improvisation environment is one of the main aspects currently being investigated at SARC. We are currently developing a networked laptop environment called Frequencyliator [7], which combines networked Open Sound Control (OSC) [8] messaging, networked audio and interactive behaviors. The system is currently based on a central server which allows local and remote peers to interconnect. Traditional instrumental performers often recur to visual cues and gestures for communication in a group context while laptop performers rarely use this mode of communication to coordinate events. The idea behind the Frequencyliator is to recreate some of those cues to increase the level of interaction of networked laptop performers whether located in the same physical space or not. This environment attempts to offer an ensemble performing on software instruments certain musical structures that are very much taken for granted in an instrumental ensemble. A non-exhaustive list would include the following: • • • • •
Potential for common pulse to emerge through rhythmic interaction. Synchronization of events that require negotiation and agreement from two or more performers. Balanced spectral structure with possibilities for both masking and extreme spectral separation. Identification of performative roles which suggest performer-to-performer as well as audience-performer interactions. Up beats or the ability to anticipate and precede events across the ensemble.
The system is based on a central score, which acts as a shared timeline for the ensemble. The timeline resembles a proportionate score that defines the form of a piece. Its generation can be pre-determined and implemented manually or generated in real-time by an algorithm. For example, a function can define variations in tempo whilst a secondary envelope can send frequency interpolation messages to a filter bank. Based on this timeline several dynamic modules can be implemented. One of the modules that has so far been implemented detects amplitude thresholds as analyzed after bandwidth allocations. The crossing of these thresholds causes a change in bandwidth allocations, effectively resulting in a “stealing” of bandwidths. This type mechanism adds a level of interactivity within the ensemble that resembles game structures.
In its current version, the Frequencyliator is implemented through the use of a central server, which broadcasts OSC messages to several laptop performers. It is possible to have local and remote participants interacting. We are planning to turn the Frequencyliator into an easy to use product which will be accessible to a wider community. Even though for practical reasons, the system will always use a central point of reference to broadcast message information and receive incoming audio signals, we are planning to introduce a bi-directional communication mechanism for both messages and audio data. This approach intends to increase the level of interaction since it will allow, for a performer to send messages to another performer or even to the server. Conclusion This paper exposes a number of approaches to network performance with the intention of forcing a re-think of traditional relationships between musicians, audiences and spaces. As most music cultures have strong on-line presences effectively acting as one of the main modes of dissemination, it is important to reflect on the implications of the network for music that is distributed through performance. Aspects of performance which go beyond the musical play a central role in defining practice; these include ritual, social hierarchies, scripts, play, environment etc… In his analysis of performance, Richard Schechner [9] puts forward a web-diagram which exposes links between historical, behavioural and societal aspects of performance practice. It is important to understand performance in its widest cultural sense if we are to take advantage of the network. Research on network music performances can ultimately inform our understanding of the role of sound, site and play in digital and online societies. REFERENCES [1] SoundWire Project Webpage http://ccrma.stanford.edu/groups/soundwire/ Accessed on April 8, 2006. [2] Jack Audio Connection Kit Webpage http://jackit.sourceforge.net/ Accessed on April 8, 2006. [3] Carôt, A, Renaud, A, Verbrugghe, B. Network Music Performance (NMP) with Soundjack. NIME 2006 Network Performance Workshop. June 2006. [4] Tanaka, A. (2006) Interaction, Experience, and the Future of Music. Computer Supported Cooperative Work (vol. 35), pages 267-288, Springer. [5] Rebelo, P. (2003) Performing Space. Organised Sound: Vol. 8 no. 2. Cambridge: Cambridge University Press: 181-186. [6] Harrison, J. (1999) Diffusion: theories and practices, with particular reference to the BEAST system. eContact 2.4, ix 1999. URL: http://cec.concordia.ca/econtact/Diffusion/Beast.htm. Accessed on 1st March 2006
[7] Rebelo, P. Renaud, A. (2006) The Frequencyliator – Distributing Structures for Networked Laptop Improvisation, Nime 2006 [8] OpenSoundControl Webpage http://www.cnmat.berkeley.edu/OpenSoundControl/ Accessed on April 13, 2006. [9] Schechner, R. (1988) Performance Theory. Routledge
