REMOTE 3D-AUDIO PERFORMANCE WITH SPATIALIZED DISTRIBUTION
Winfried Ritsch
Thomas Musil
Alois Sontacchi
Johannes Zmölnig
Franz Zotter
[email protected]
[email protected]
Institute of Electronic Music and Acoustics Graz, Austria
[email protected]
[email protected]
[email protected]
ABSTRACT Today's electro-acoustics concert halls are usually equipped with a multi speaker setting. Some of these environments have the ability to play music on an spatialised 3D space including virtual acoustics. For remote performances, a concert in a source place is transmitted to a remote place where the audience is located. The concert's “audio signature” is replicated in the destination. The needed streams over the network should include an abstract description of the sound environment to be interpreted correctly to a different speaker setting in another place. Implementing this scenario we realized the lack of a common streaming technique to realize with a limited number of audio streams and control data to be transmitted. Here we suggest using an Ambisonics approach to stream complex audio environments including extra channels for monitoring, and special audio signals in a time accurate way. The description of the audio signals streamed with metadata use different containers, also storing them in common file formats as PCM-WAV or SDIF. As a proof of concept we implement this on our general purpose production software CUBEMixer, originally designed to be used for performances as spatialisation-system, a 3D-Audio mastering application including additional virtual acoustics in a concert-room. 1 The entire system can be controlled with OSC [10]. Keywords : Spatialisation, mixer, virtual room acoustics streaming, post-production. 1.
INTRODUCTION
In electro-acoustic concerts in the western tradition we think of sound as a spatial, sculptural phenomenon. Prerecorded audio signals are rendered in an 3D-AudioEnvironment, mixed with live signals. Since playback situation differ from each other and concert halls have different acoustics, we need to transport more or less dry signals of the sound environment to be rendered in 1
Open Sound Control Protocol
another, possibly virtual, acoustic environment, from big halls, via small rooms to binaural production systems. To achieve this goal, a 3D-Mixer is needed that has the possibility to produce an adequate spatialized stream. This could be done with audio and parallel time aligned control data like OSC or to stream the mix for a given speaker setting in the target room. The latter cannot do complex spatial environments and is inflexible for different performance places without adaptation. The main target is to use an higher order Ambisonics system as an abstract base of spatialisation. The 3Dspatialized data can be interpreted in different targets with their own decoding master sections for multi speaker systems, or special Ambisonics decoders to render the mix for example in 5.1 surround or even plain stereo for headphones. As a first implementation and proof of concept, The CUBEMixer[13] is adapted. The goal is to find a common performance standard for streaming audio, mixed in the 3D-domain. 2.
STREAMING FORMAT
Since there is no commonly used format for Ambisonics streams in 3D, their is the need to define one for the exchange of Ambisonics art works and streams. In the Ambisonics XChange meeting in Graz specialists for Ambisonics works in Europe met to find one general solution where full or reduced sets of Ambisonics channels can be mixed with additional audio channels for eg. click tracks, sub woofer, monitoring, ... . 2.1. Audio Channels The Ambisonics channels represent the Spherical Harmonics of a sound field. The Ambisonics order defines the number of channels needed. On 2D audio it is 2n+1, on 3D (n+1)^2 and on mixed order there are different solutions.
the time of writing an 3D Ambisonics up to 5th order[1,2,3] is used. In the Ambisonics domain, the 25 Ambisonics signals can be used for storage and as a stream source for complex spatialized audio.
Figure 1. Spherical harmonics for 3D Ambisonics
2.1.1.
Naming
The name space for description of the channels are traditionally “WXYZRSTUVKLMNOPQ...” which is not sufficient for higher order Ambisonics. Therefore a common naming convention should be defined. 2.1.2.
Number of Channels Needed
Not all channels are needed for most situations because of symmetries or other partially covered areas of sound distribution and some of them can be calculated from a linear combination of others. 2.1.3.
Monitor Channels
Monitor channels can also be calculated from Ambisonics signals which filters audio from a certain position in the 3D signal, like a magnifying glass on an audio scene. 2.1.4.
Combining Ambisonic Material
For mixing Ambisonics streams sometimes only one channel is needed to be spatialized from a certain direction and covering area distribution. Therefore needed Ambisonics channels can be calculated. 2.1.5.
Special Purpose Channels
These are needed for click tracks, special effects, etc and should be added to the multi-channel stream without being part of the Ambisonics encoded soundfield. 2.1.6.
Container
The audio channels and additional streamable audio data have to be transmitted in container. One container could be Broadcast WAV or for better streaming behaviour SDIF[13], which can also include some control streams like OSC. 2.2. Tools The mixer is split split into a configurable number of input sections assignable to individual input signals, with different optional encoders and a master section with extensions like different decoder and subwoofersystems, effects, 3D-reverbs, sound file players and recorders, a binaural rendering stage and other tools. At
Figure 2. Mixer Organisation
3.
CONCLUSION
The need for a common format is shown above. With the proposal of the IEM Ambisonics XChange Format we hope to have made a step forward solve this problem. At the end of this process a rough consensus for the used format should be found and as proof of concept, streaming actions should be done. This will help in the preservation of Ambisonics artworks as well as raise the standard of 3D-Audio environments to be shared. 4.
REFERENCES
[1] Gerzon, M. A. (1985). “Ambisonics in Multichannel Broadcasting and Video”, J. Audio Eng. Soc., 33(11), pp. 859-871. [2] Malham, D. G. (1992). Experience with Large Area 3D Ambisonic Sound Systems. In Proc. Inst. Acoust. [3] Musil, T. and J. Zmölnig, M. Noisternig, A. Sontacchi, Robert Höldrich: AMBISONIC 3DBeschallungssystem 5.Ordnung für PD, IEM Report 15/2003 [4] Musil, T.: IEMLIB für PD, IEM Report 12/2003. [5] Noisternig N and A. Sontacchi, T. Musil, R. Höldrich: A 3D Ambisonic based Binaural Sound Reproduction System" AES 24th
International Conference, 26-28 June 2003, Banff, Canada. [6] Puckette, M. 1996. "Pure Data: another integrated computer music environment." Proceedings, Second Intercollege Computer Music Concerts, Tachikawa, Japan, pp. 37-41. [7] Ritsch W. and R. Höldrich, C. Internet Archive for Electronic iARS internet Audio Rendering 24th International Conference, 2003, Banff, Canada.
Frauenberger: Music IAEMSystem, AES 26-28 June
[8] Ritsch W. and Musil T., Zmölnig J., Höldrich R. (1995):Implementation eines kombinierten Ambisonic- und Bus-Mixers für den Einsatz in 3D Audio Environments, IEM Report 28/05. [9] Sontacchi, A.: Neue Ansätze der Schallfeldreproduktion, Dissertation der TU Graz 2001. [10] Wright, M. and A. Freed 1997. Open Sound Control: A New Protocol for Communicating with Sound Synthesizers. Proceedings of the International Computer Music Conference, Thessaloniki, Hellas, pp. 101-104. [11] Zmölnig, J and W. Ritsch, A. Sontacchi: "Der IEM CUBE – ein periphones (Re) Produktions system", 22.TMT,Produktforum, Jahrestagung des Vereins Deutscher Tonmeister, Hannover, November 2002. [12] Zmölnig J. and W. Ritsch, A. Sontacchi: "The IEM CUBE", ICAD - International Conference on Auditory Display, July 7-9, 2003, Boston University, Boston, MA. [13] M. Wright, A. Chaudhary, A. Freed, D. Wessel, and X. Rodet. New Applications of the Sound Description Interchange Format. In Proceedings of the International Computer Music Conference, pages 276–279, Ann Arbor, US, 1998.
