Examining How Musicians Create Augmented Musical ... - Bristol CS

Examining How Musicians Create Augmented Musical Instruments Dan Newton and Mark T. Marshall Interaction and Graphics Group, Department of Computer Science, University of Bristol, UK. [email protected], [email protected]

ABSTRACT

2.

This paper exams the creation of augmented musical instruments by a number of musicians. Equipped with a system called the Augmentalist, 10 musicians created new augmented instruments based on their traditional acoustic or electric instruments. This paper discusses the ways in which the musicians augmented their instruments, examines the similarities and differences between the resulting instruments and presents a number of interesting findings resulting from this process.

The design process for the Augmentalist took an iterative, user-centred approach. This involved numerous consultation, testing and design sessions with musicians. The overall goal of this process was to ensure that the system was useful to the musicians themselves. The consultation process began as soon as the project itself was conceived. It began with a series of short meetings with a number of different musicians. These meetings included sessions with single musicians and also with groups of musicians. The aim of these initial sessions was simply to gauge interest in the project itself and to attempt to determine what features of the system would be useful to a variety of musicians. From these sessions, we arrived at a design that allowed the musicians to attach sensors the their instruments and then map the output from these sensors to MIDI signals. These MIDI signals could then be used to control parameters of audio software with which the musicians are familiar. The remainder of this section discusses the hardware and software implementation of the Augmentalist.

Keywords Augmented Instruments, Instrument Design, Digital Musical Instruments, Performance

1.

INTRODUCTION

Augmented musical instruments are created by the addition of sensors to existing acoustic or electric instruments to allow the performer to control additional digital audio effects or sound synthesis processes through their gestures. Such instruments offer numerous possibilities for musical performance [5], but also create issues with regard to the musicians ability to control these extra effects [2]. Based on the idea that musicians themselves would best know how to augment their musical instruments, both in terms of gesture potential and cognitive load, we created the Augmentalist [6]. The Augmentalist is a system to allow performers to easily augment their musical instruments. It consists of a combination of hardware (sensors and a sensor interface) and an easy to use mapping software. As part of the design process for the Augmentalist system we worked in collaboration with 10 musicians, developing the system in an iterative user-centred manner. This resulted not only in a robust and easy to use system, but also a number of new augmented instruments developed by these performers over the course of the project. This paper details these instruments, the design decisions that took place in their development, the similarities and differences between them and a number of interesting findings that resulted from this work. We begin with an overview of the Augmentalist system itself, to allow for a better understanding of how the system works and how it could be used. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NIME’11, 30 May–1 June 2011, Oslo, Norway. Copyright remains with the author(s).

2.1

THE AUGMENTALIST

Hardware

After experimenting with different available sensors and sensor systems, we decided to use Phidgets [3]. These sensors require no soldering or programming on behalf of the users. Thus they are ideal for a system designed for musicians due to their plug and play capabilities1 . The choice of the Phidgets system also allows for a large range of sensors to be available to the user, with dozens of sensors currently available from the manufacturers that plug directly in to the interface with no electronic skills required to use them. The sensors are connected via USB 2.0 to a computer. For our initial implementation we used a 2.53GHz MacBook Pro Running OS X 10.6 using a Stanton Scratch Amp firewire audio interface for audio input/output.

2.2

Software

To convert the sensor data into MIDI signals we used the Max/MSP programming environment. This had the advantage of being easy to use, as well as being fully compatible with the Phidget sensors. It is also an environment with which some of our musicians were familiar. The interface in Max/MSP allows the user to choose which sensors to map to specific MIDI channels, as well as setting the desired input range from the sensor, output range for the MIDI channel and the mapping between them. The software allows the user to select which sensor is mapped to to which MIDI signal using a simple graphical interface. The range of sensor values to be mapped can 1 Note however that due to the modular nature of the system, other sensor interfaces could be used in place of the Phidgets.

be selected by demonstration, with the performer moving the sensor through its desired range. The MIDI output range can also be limited to a specific range in the software. Finally, the user can specify the mapping function used to convert sensor data to MIDI data. This can be selected from a range of presets (linear increasing, logarithmic increasing, linear decreasing, etc.) or by drawing a mapping function in the interface. The MIDI output could then be mapped to parameters in audio software chosen by the musicians. In our development sessions we primarily used Apple’s Logic Pro 9.

3.

4.1.1

Gestures and Sensing

Most interestingly, we found that all 3 guitarist used a tilting of the guitar body as a control gesture. This gesture was sensed using an accelerometer, mounted to either the body or the headstock of the guitar, depending on the performer. One guitarist used the position of the picking hand over the guitar body as a control gesture. This was sensed using a slider mounted to the guitar body, below and parallel to the strings, as shown in Figure 1.

INSTRUMENT DESIGN AND TESTING SESSIONS

Over the course of the development of the Augmentalist a group of 10 musicians spent numerous hours working with the system, creating and testing new augmented musical instruments and mappings. This group of musicians was made up of 3 guitarists, 3 bass players, 2 DJs, a saxophonist and a vocalist. Interestingly, examples of augmented instruments for each of these types of performers can already be found in the literature [5, 1, 7, 4]. This would seem to indicate that these types of musicians have the necessary spare “bandwidth” to allow them to successfully play an augmented instrument. For the design and testing sessions, the musician was free to choose the sensors used, the attachment of the sensor to the instrument, the effect being controlled and the mapping of the sensor to the effect. This gives the musician total control over how the system is designed and used. Each session followed the same format, as follow: 1. Presentation of software including any updated features. 2. Participant uses software with researcher present for short time researcher helps participant with any issues that arise. 3. Participant left to use software for a longer period of time. 4. Participant fills in feedback form at the end of the session. 5. Researcher performs a short interview of participant to gather any additional thought, problems etc. The aim of this session format was to allow us to inform the users of new developments in the system and to receive as much feedback from the users as possible, without causing them to feel under pressure. The solo portion of the session, in which the participant used the system without supervision, was designed to allow them to explore the system with as much freedom as possible, and without the pressure of having an audience that could arise from our presence.

4.

DEVELOPED INSTRUMENTS

In this section, we discuss the instruments and mappings developed by the musicians. In particular we look at the choices of sensors and gestures that the performers used.

4.1

Guitarists

Three guitarists took part in the development of the Augmentalist system. Each guitarist worked with the system for multiple 1 hour sessions over the course of the development. Each guitarist was allowed to develop their own instruments and mappings. This allowed us to look for similarities between the instruments developed by different performers based on the same traditional instrument.

Figure 1: An example of a guitar augmented using the Augmentalist system. Another interesting gesture/sensor combination that was developed involved the use of an infrared distance sensor to detect strumming rhythm. A number of possibilities for detecting strumming rhythm were discussed by the guitarists, including attaching an accelerometer to the performers strumming hand, and trying to determine strumming rhythm from the sound output. However, one guitarist decided to detect strumming rhythm using an infrared distance sensor, which was mounted on the body of the guitar, under the strings. This sensor was set up so that when the guitarist strummed the strings their hand would pass over it. It was then configured as an on/off switch which triggered whenever the guitarist’s hand passed over it. This switch between on and off then provided a measurement of the strumming rhythm. One other possibility that guitarists examined for control gestures was the use of head and body movements. Suggestions included the use of head mounted accelerometers to detect head tilting and the use of accelerometers on the body to detect weight shifting. However, these were found to be too cumbersome and/or restrictive for use when playing.

4.1.2

Audio Effects and Mapping

For each of the guitarists, the control gestures just described were mapped to a number of audio effects in Apple’s Logic Pro. The choice of gestures, effects and mappings were left to the individual guitarists. Logic Pro was chosen as it is a software package that many of the participants were familiar with and also offers a large number of possible effects to control. All of the guitarists chose to use effects that they were already somewhat familiar with and that are commonly used by guitarists playing electric guitar. These effects included distortion, delay, chorus, flanger, and master volume. Example mappings included the control of delay using the tilt of the guitar, controlling distortion using the picking position and mapping strumming rhythm to master volume.

4.2

Bassists

Two of the participants in the design process were bassists. The bass guitar is interesting due to its similarity in form factor to the electric guitar, while having a different playing style. This allows us to look at any similarities between the instruments developed by the guitarists and bassists.

4.2.1

Gestures and Sensing

The sensing of the gestures for the bass was similar to the guitar. In particular, both bassists also used the tilt of the bass guitar as a control parameter, again detected using an accelerometer mounted to the headstock or body of the instrument. One of the bassists also tried to use body movements as a control parameter. As with the guitarists, he attempted to use head tilt (detected with an accelerometer on the head) as a control. While finding this somewhat difficult to control, he also found it extremely enjoyable and kept it as part of his instrument.

4.2.2


The effect that the bassists had the most fun with was the wah effect mapped to the accelerometer measuring the tilt of the neck. This is essentially a bi-pass cutoff where the cutoff frequency is set by the sensor. The wah has existed for many years as a foot pedal for guitarists and bassists alike but transferring this concept to the angle of the neck proved quite difficult for one bassist who had little experience with effects. Instead he ended up playing the bass as normal, with a few slight body movements in time with the music. This created very subtle changes to the ambience of the bass as the wah moved in time with the music. Other effects tested by the bassists included distortion and filter effects. These were often mapped to the tilt of the instrument, allowing a subtle, graduated control of the effect.

4.3

DJs

Two DJs took part in the development sessions. One of the DJs primarily played Techno, while the other was a Drum and Bass DJ. These two styles of music make use of different beat styles, which may be suited to different audio effects.

4.3.1

in the same place. The DJ activates the force sensor by pressing on it. The force of this pressing is then measured by the sensor. Figure 2 shows the system in use.

Sensing and Gestures

Although a DJ tends to have their hands full much of the time, we found that the DJs preferred to use their hands to control the sensors, rather than finding some unused performance gesture. This meant that they were often simply utilising the properties of the sensor directly, rather than attempting to use the sensor to sense a gesture. As such the sensors often became extra controls for their mixer. In testing, one DJ who used the system made extensive use of 3 sensors: an accelerometer, a slider and a force sensor. The accelerometer was attached to the performer’s hand and used as a tilt sensor. This allowed them to control effects by tilting their hand in 2 axes. This was the only sensor which the DJ used to sense movement, rather than as a direct control. Sliders are extremely common in DJ equipment and are used for volume, turntable speed, as well as many effects. The DJ quickly picked up on the advantage of the slider. Retaining its position and it’s location next to the pitch control on the turntable allowed for quick and easy adjustments whilst mixing. When a DJ mixes, a large proportion of his time is spent focusing on the pitch control which is located on the turntable next to the tone arm and so the DJ was able to quickly switch to this slider to control effects. The force sensor was attached to the opposite turntable

Figure 2: DJ playing with a slider and force sensor on the turntables The second DJ made use entirely of sliders, using them as additional effect controls on top of their turntable decks.

4.3.2


DJs can make use of a large number of effects during a performance, switching effects during a track or when changing tracks. As one of our DJs made use only of sliders as additional effect controls, the mapping is not particularly interesting. As such, we will concentrate on the other DJ who made use of a number of sensors and effects. This DJ who tested the system tried a large number of effects with each sensor, before settling on several options for each sensor. The result is that for each sensor the DJ has a number of effects, which can be controlled one at a time or even several at once. For the accelerometer, the effects chosen were a bandpass filter effect and a beat repeating effect called Beatmasher. While the bandpass filter resulted in controlled, predictable effects, controlling the Beatmasher resulted in interesting but more random results. Interestingly the DJs found that the Beatmasher effect was more useful when used on Techno music than Drum and Bass. For the slider, the DJ chose the distortion drive level and the Transpose Stretch effect, which pitch shifts and time stretches the audio. Finally, for the force sensor the DJ chose to control a number of effect mix levels, including reverb mix, delay mix, flanger mix and phaser mix. The nature of the force sensor, which returns to a zero value output as soon as the performer stops pressing on it, allowed the DJ to add and effect by pressing the sensor, increasing the effect by increasing pressure and then instantly stop the effect by releasing the sensor.

4.4

Vocalist

A single vocalist also took part in the design process for the Augmentalist. Vocalists offer an interesting contrast to the other performers as their hands are generally free when performing (except when holding a microphone or mic stand). This allows them to use a great range of finely controlled mappings utilising the dexterity of the hands, as well as other parts of the body. Note that in the case of the vocalist, the instrument could be considered to be their own body, or the microphone and/or mic stand.

4.4.1


The vocalist (an MC who ’rapped’ rather than sang) made use of hand gestures to control effects. This included sens-

ing of the tilting of his hand in two dimensions. This was accomplished through the use an accelerometer strapped to the back of his hand. As with the other participants, the vocalist also considered the use of head movements, again sensing head tilt using an accelerometer. However, these movements were found to be too disconcerting to use in performance. The vocalist also examined the augmentation of the microphone. Gestures used included the sliding of the hand along the microphone (measured using a slider attached to the microphone body) and grip pressure on the microphone, detected using an FSR attached to the microphone body. Most MCs hold the microphone to perform, instead of using a stand and so to put controls on the microphone itself proved to be intuitive. Furthermore, by mapping the tightness of the grip on the microphone to an effect mix, the mapping was a natural extension of emotive performance as with the accelerometer on the guitar.

4.4.2


This particular vocalist was not as well versed on all the various effects and their parameters as, for example, the guitarist. This meant that often effects were discovered by accident as more experimentation took place, rather than attempting to achieve a specific sound. Something that the vocalist was keen to try straight away was a pitch shifter. This effect simply changes the pitch of the input by an amount specified on a discrete bidirectional scale. After trying with the accelerometer and struggling to maintain a steady hand (i.e. keep the pitch normal) he requested that we be able to limit the MIDI output at half so the he could keep the sensor at 0 more easily. After this he found it very intuitive to map a drop in pitch to the downwards movement of his hand and keep the pitch at 0 with his hand up. The pressure sensor with its ’return to zero’ style of operation worked really well with effects that made the sound messy as when released the effect would return to normal. Delay mix and reverb mix as well a flanger intensity worked well to accent and in some cases twist quite dramatically the sound before snapping back to a dry signal when released.

4.5

Saxophonist

The final participant in the development process for the Augmentalist was a saxophonist. In general, we would not expect saxophonists to be overly familiar with the use of electronic effects and audio processing. However, the interview stage revealed that this saxophonist was also a keen producer of electronic music and as such was very well versed and comfortable with standard studio style controls. As will be seen, this may have influenced his choice of sensing and control gestures.

4.5.1


In a similar way to the DJs that worked with the system, the saxophonist talked more about the sensors as extra controls rather than a medium for interpreting gestures. Perhaps influenced by his familiarity with studio sound equipment, the first sensor he chose to use was the slider. A slider mounted on the saxophone body was used to control a variety of effects. A more interesting idea that came out of his sessions was utilising the free thumb of the right hand to control effects. The saxophone has a thumb rest for holding the instrument as show in Figure 3. By placing a force sensor around this area, the saxophonist was able to squeeze the saxophone with his spare thumb to invoke an effect. The saxophonist also experimented with the use of an

Figure 3: The thumb rest on a saxophone accelerometer to measure tilting of the saxophone in the vertical plane and the placement of a slider on his body at the hip.

4.5.2


As previously mentioned this saxophonist was also a music producer, and as such experimented with more intriguing and less popular effect parameters. One of the benefits of the Augmentalist system is its generality and the way that you can map to virtually any parameter. This allows you to add controls to parameters that you would not normally move during performance. The saxophonist was keen to explore these possibilities, and although they did not always work as expected, they were always interesting. One effect that appeared to work was a tape delay time. The amount of time that a delay takes to repeat is usually not moved, or only moved by small amounts to keep it in time. However when used aggressively by the saxophonist, some unusual and interesting sonic effects were created. Other effects looked at included a reverb effect and a number of filter effects. While the saxophonist experimented with all of these effects using both the slider and the thumb rest pressure sensor, we found that he significantly preferred to use slider as the main control. From interviews we determined that this was due to a combination of the ”return to zero” nature of the FSR not allowing values to be maintained easily and the difficulty of manipulating thumb pressure while playing. While this second difficulty could be overcome with practice, the first is inherent to the sensor design.

5.

DISCUSSION

This work has raised a number of issues regarding the development of augmented instruments by musicians. In this section we discuss in more detail some of the more interesting points raised during the development process and their implications for research into augmented instruments.

5.1

Similarities in Control Gestures

Most of the participants in the development process used or tried to use head movements and/or center of mass movements as a control parameter, irrespective of the instrument they played. This happened for a number of different performers including the guitarist, bassist, DJ and vocalist. It seems that these movements are considered by many people to be useful as additional controls in instrumental performance. Interestingly however, while most of the participants tried to use these gestures as controls, after some practice only one of them kept these gestures. This may indicate that what seems to be the most naturally useful gestures are not so useful in practice.

We also found that tilting the instrument was a commonly used control. The guitarist, bassists and saxophonist all made use of this gesture. This gesture has also been used with other wind instruments by other researchers and performers [10, 7], which would seem to indicate that this is a generally useful gesture for many instrumental performers.

5.2

Musicians as developers

The Augmentalist system was designed from the start to allow musicians to become the developers of their own augmented instruments. We believe that it is musicians who know the most about their instruments and about the sounds and music they wish to create and so it is the musician who should make the decisions on how the instrument should work. Over the course of the development of this system so far, the participating musicians developed hundreds of different gesture to sound mappings. While this is a large number of different mappings, what is interesting is that the musicians themselves considered far more of these mappings to be successful than not. This paper has covered only a selection of those that were considered best by the performers and that they continued to use across multiple sessions. The system not only enabled them to develop new mappings, but resulted in mappings the musicians found to be interesting, useful and musical. The use of the Augmentalist also resulted in some interesting discoveries. Most notable is the use of the infrared distance sensor as a switch to detect strumming by one guitarist. This provides a very simple and robust method of detecting strumming, and is one that we have been not previously seen in the literature on augmented instruments. Secondly, we found that one of the musicians developed a mapping for their guitar that mirrored that presented in [5] and did so within the first 2 hours of using the system. This emphasises how quickly interesting and usable instruments can be developed by a musician when given access to such a system.

5.3

Focus on technology

Our initial idea of how musicians would develop their instruments was that they would first decide on a gesture to detect and a sonic output to control with that gesture, and then on how to detect this gesture. However, over the course of this project we found that many of the musicians instead focused on the technology itself. They started by examining the sensors that were available to them, the parameters these sensors could detect and where on the instrument they could be easily mounted. Only then would they think about the gestures that the sensors could be used for. Such a focus on the technology is a somewhat interesting finding. It seems that the musicians consider the sensor technology to be the weakest link in the system and so allow themselves to be guided by the limitations of the sensors. While musicians’ creativity can often thrive off such boundaries and limitations [8], if we wish to develop a system that truly focuses on the gestures and sounds then we must alter the users’ perception of such limitations of the sensors.

5.4

Potential for exploration and mastery

Wessel and Wright state that a goal for designing new digital musical instruments should be for them to have a “low entry fee” together with “no ceiling on virtuosity” [11]. This means that such instruments should be simple to begin playing, but complex and engaging enough to offer the possibility of exploration and mastery. One of the advantages of augmented instruments is that they are based around existing musical instruments. A

guitarist will still be able to play an augmented guitar as though it is a regular guitar. This makes the instrument easy to begin using. The additional sensors then extend the performance possibilities of the instrument, thus allowing for more potential for creative exploration. When speaking with the musicians who worked with the system, we found they commonly expressed the belief that they could gain full control of the instrument and make best use of the system given enough time. This shows that the Augmentalist offers potential for further exploration and creativity.

5.5

Subtle Sonic Effects

When working with the bassists that took part in this research, we noticed that they seemed to produce very musical results when the effects used were quite subtle. The mapping features of the Augmentalist software, combined with the use of sensors to detect relatively large range movements combined to allow small movements to produce very subtle effects. This resulted in effects that were subtle enough to allow the musician to focus on making music rather than making the effects. We found that when the bassists started to focus on the music and not the effect mapping the effects became more subtle and natural by virtue of not being purposefully moved. This result aligns with the work presented by Lahdeoja et al [5].

5.6

Transferability of developed instruments

We have already mentioned that one musician developed a mapping for their instrument that directly mirrored a system discussed in the literature. This indicates that there is some common pool of gestures that are suggested by the design of the instrument itself, similar to those discussed by Wanderley [9]. As such, we would also expect that it is possible to transfer developed instruments between performers that play the same instrument. To examine this we asked another guitarist to try and perform using the mappings developed by one of the guitarists working with our system. The new guitarist found it easy to begin performing using any of these mappings. In each case, it took only a few minutes of practice before they were able to utilise the gestures in performance. The guitarist also made a number of comments on how “easy” and “natural” the mappings were to use. It is possible therefore to transfer mappings between performers of the same instrument. This means that the potential exists to share instrument designs and mappings across users. One possibility would be the creation of a community to promote such sharing between musicians interested in developing augmented instruments. We are now beginning to investigate this possibility.

5.7

Use in Ensemble Performance

One of the goals of the Augmentalist system was to allow musicians to create new augmented instruments that they could use in their own musical performances, as part of their performance careers. While our testing and development sessions focused on working with individual musicians, we also encouraged the musicians to take their augmented instruments with them for use in both alone and in conjunction with other musicians. In every case, the musicians were happy to continue working with their augmented instruments in private. However, one of the guitarist, a member of a 3-piece rock band, also asked to demonstrate the system to his band mates. As a result, he performed with a drummer and bassist, using his augmented guitar. All the musicians found the experience enlightening and fun, and felt that the performance was enriched because of the system. The band have stated their

intention to use the system in future live performances and are currently in the process of incorporating the system into their act.

5.8

Creativity and enjoyment

5

One of the main aims of this project was to produce a system that facilitated both creativity and enjoyment for the musicians using it. As has already been discussed, the musicians who used the system expressed the belief that the Augmentalist allows for new musical possibilities and offers much potential for further exploration and mastery. The question that then arises is: did the musicians enjoy using the system? Throughout the development of the Augmentalist there were regular testing, development and performance sessions involving musicians. At the end of each session, we ask each musicians to fill out a short questionnaire, which involved rating the system on a 1-to-5 scale on a number of criteria. Perhaps the most interesting result of this was that every musician gave the system a maximum rating of 5 (Very High) for enjoyment at the end of every session. Another interesting finding is that the average rating given by the performers to the system across all the measured criteria (ease of use, enjoyment, controllability, and expressive potential) increased over time. Figure 4 shows the mean performer rating of the system over the 9 weeks of development. As can be seen the mean rating rises from 3.3 to 4.6 over this period of time.

●

●

7

8

●

4

● ●

●

●

●

3

4

5

1

2

3

●

0

Combined User Rating

“I haven’t stopped smiling for ages, that was ******* awesome. When can I come back?”

Performance Bandwidth and Practice

As discussed by [2], some musicians have “spare bandwidth” when it comes to performing. This means that for these musicians, it is possible to extend their performance technique without putting too much of a load on their capabilities and reducing the quality of their performance. For most of our musicians this seemed to be the case, with some small exceptions. We found that the vocalist and saxophonists had difficulties with some performance gestures. Concentrating on hand movements (for the vocalist) or thumb pressure (for the saxophonist) distracted from the performance. The vocalist even found that concentrating on hand movements could result in him forgetting the lyrics. This may be a case of the vocalist having exceeded his available “bandwidth”. However, in both cases we noticed some improvement with time and practice, so this may also be a problem that could be overcome in time. The effect of practice on this sort of performance will form an interesting area of further study for us.

5.9

quote, received from one participant several hours after a session, fully illustrates the level of enjoyment felt by those using the system:

1

2

6

9

Week

Figure 4: Performer rating of the Augmentalist system over the 9 weeks of development While these ratings show that musicians definitely enjoyed working with the system, we think that the following

6.

CONCLUSION

The main goal of the Augmentalist system was to enable musicians to begin experimenting with digital musical instruments through augmenting their existing musical instruments. Our belief was that by focusing on existing instruments and augmenting them with sensors, musicians could produce new instruments with extended interaction and performance possibilities. Such instruments would also have the advantage of reducing the performer-instrument and audience-instrument disconnect that can be present with many new digital musical instruments. In this paper we have described a number of the instruments and mappings that our group of musicians have created using the Augmentalist system. By examining these instruments we have seen the similarities and differences between instruments designed by performers, whether playing the same instruments or difference ones. We have looked at issues such as the longer term development of these instruments by musicians, the possibility of sharing and exchanging ideas and mappings for such instruments and the innovative performance and interaction techniques that musicians develop as part of this process. Overall, the Augmentalist allows musicians to explore new musical techniques, while also allowing them to design and create their own instruments. It opens a number of performance possibilities for these musicians and we hope in the future to be able to work with our musicians to integrate the system permanently into their performance careers, whether as soloists or as part of ensembles.

7.

REFERENCES

[1] T. Beamish, K. MacLean, and S. S. Fels. Manipulating music: Multimodal interaction for djs. In Proc. CHI 2004, pages 327–334. ACM, 2004. [2] P. Cook. Principles for designing computer music controllers. In Proc. NIME 2001. ACM, 2001. [3] S. Greenberg and C. Fitchett. Phidgets: easy development of physical interfaces through physical widgets. In Proc. UIST 2001, pages 209–218. ACM, 2001. [4] D. Hewitt and I. Stevenson. E-mic: extended mic-stand interface controller. In Proceedings of NIME03, pages 122–128, 2003. [5] O. Lahdeoja, M. M. Wanderley, and J. Malloch. Instrument augmentation using ancillary gestures for subtle sonic effects. In Proc. SMC 2009, pages 327–330, 2009. [6] D. Newton and M. T. Marshall. The augmentalist: Enabling musicians to develop augmented musical instruments. In Proc. of ACM TEI2011, 2011. [7] S. Schiesser and C. Traube. On making and playing an electronically-augmented saxophone. In Proceedings of NIME06, pages 308–313, 2006. [8] A. Tanaka. Musical performance practice on sensor-based instruments. In M. M. Wanderley and M. Battier, editors, Trends in Gestural Control of Music, pages 389–405. IRCAM - Centre Pompidou, 2000. [9] M. M. Wanderley. Quantitative analysis of non-obvious performer gestures. In I. Wachsmuth and T. Sowa, editors, Gesture and Sign Language in Human-Computer Interaction, pages 241–253, 2002. [10] M. M. Wanderley and P. Depalle. Gestural control of sound synthesis. Proceedings of the IEEE, 92(4):632–644, 2004. [11] D. Wessel and M. Wright. Problems and prospects for intimate musical control of computers. Computer Music Journal, 26(3):11–22, 2002.