Integrating Theatrical Strategies into Sonic Interaction ...

Integrating Theatrical Strategies into Sonic Interaction Design Sandra Pauletto, Department of Theatre, Film and Television, The University of York, [email protected] Daniel Hug, University of the Arts, Zurich, [email protected] Stephen Barrass, University of Canberra, [email protected] Mary Luckhurst, Department of Theatre, Film and Television, The University of York, [email protected]

Abstract. The Workshop on 'Sonic Interaction Design and its relation to Film and Theatre Sound Design’ was organised as part of the European COST project on Sonic Interaction Design (COST-SID). The aim of the workshop was to explore how existing practices in theatre can inform the design and evaluation of sonic interfaces. In this paper we describe the process of creating the sound design for a short theatre scene and the process of directing and creating the final performance, which involved a high degree of improvisation. We also describe the listening process constructed for the workshop. The scene was played to the audience in the dark and they were asked for responses regarding understandings of the scene, the sources of the sounds, the emotional content and their expectations based on the sounds alone. The sound design was then played again but this time with an accompanying performance in which the actor interacted with various objects. More feedback was gathered after this second performance looking at how the perception of the sounds, the audience expectations and the emotional content of the piece was changed. Finally the Sound Designer, the Theatre Director and the Actor explained their processes and a discussion gathered comments on this “theatrical” method for testing and evaluating sonic interaction designs. We finish with reflections on what we have learnt from this workshop about the ways that theatre can be used to experiment, prototype and sketch sonic interaction designs, and conclude with directions for future work.

1.

Introduction

Interaction Design along with suggestions for further work in this area.

Sonic Interaction Design (SID) investigates ways of conveying information, data and object characteristics through sound in interactive contexts. This field of research has become increasingly important and urgent as the technology available has become more powerful and mobile due to the process of miniaturization. Research on the design of sonic interaction is relatively new but there is an already well established theory and practice of designing sounds to accompany, represent or evoke actions and objects in Theatre and Film. This knowledge and practical experience is therefore an important basis for new research in the field of Sonic Interaction Design. This paper reports on the events and discussions that took place during the Workshop on "Sonic Interaction Design and its relation to Film and Theatre Sound Design" (funded by the Cost Action IC0601 Sonic Interaction Design), at the Department of Theatre, Film and Television at The University of York (UK) on April 2nd 2009 [1]. The workshop brought experienced sound practitioners working in Film, Theatre, and Computer Games together with researchers in Sonic Interaction Design to discuss and exchange knowledge. The workshop was a rare opportunity to bring together a highly interdisciplinary group of people who usually work in very different contexts but who are interested in similar issues involving the design of interactive sounds. The workshop was initiated through a Call for Sound Designs for a short theatrical scene. This call was motivated by the difficulty in Sonic Interaction Design to identify ways in which design ideas could be sketched, tested, represented and discussed. The call proposed the setting of a theatrical scene in which a performer would interact with various objects as a means to investigate how theatrical strategies might be used to sketch and prototype sonic interaction designs. We report in particular on a version of the performance titled "Anny in the Kitchen/Mark in the Bathroom" and the discussion that followed. In the next section we introduce literature and works which form the background to the approach taken in the preparation of this workshop. The following section gives details of the Call for Sound Designs that was used to motivate the workshop. After that is a description of the sound design and the way in which the theatrical performance was created. Finally we report on the discussions that took place during the workshop and draw conclusions about the use of theatrical strategies in Sonic

2.

Background

Designing sounds for interactive artefacts requires a deeper understanding of the nature of such artefacts, of how they relate to individuals and society and how they become part of sociocultural narrative and experience [2]. This is even more true for computerised artefacts which are procedural and interactive, are often anthropomorphised, and may even seem ”magical” [3]. These kinds of interactive computerised artefacts are becoming commonplace in our everyday experience. Sound supports peripheral use, small screens and multi-modal interaction, and so can have an important part in the design of the interfaces for such commodities. However, there is still little knowledge about how to design sound for interactive physical artefacts. Everyday objects are a particular challenge for sound design, because they have to work in listening conditions that differ greatly from artistic and musical use. Researchers into sonic interaction design have drawn on techniques from other areas of design such as scenarios [4] and prototypes [5, 6]. A first approach to the theory of sound design for such devices proposes that the design of sound for interactive commodities could be informed by narrative and fictional design strategies that are based on the procedural nature of interactive artefacts and their role in personal everyday experience [7]. This proposal is also in line with conceptualisations of interactive media as cyber-narratives [8]. The sound designers for fictional media, and in particular film, have developed a theory and body of practice about how to convey meaning through the narrative qualities of sounds. Moreover sound designs in films, theatre and computer games are often highly elaborated, semantically rich and subtly tuned to the identity of the protagonists, objects and processes depicted. The narrative possibilities of cinematic sound were demonstrated in an audio-only film where scenes, characters and action were conveyed by sound alone, in the absence of any visuals [9]. Principles of diegesis from film sound have been extended to interactive sounding objects such as a cocktail glass and a teapot that produce narrative sounds in response to haptic exploration [10]. These observations suggest that the design of sound for film and theatre can also inform the design of sounds for interactive commodities that are to become a meaningful, enriching and an appreciated part of everyday life.

-1-

Integrating Theatrical Strategies into Sonic Interaction Design

3.

Experts in film, theatre and computer games were invited to the Workshop on “Sonic Interaction Design and its relation to Film and Theatre Sound Design” in order to leverage their knowledge of the existing approach to sound design in these mediums. Annabelle Pangborn is Head of Editing, Sound and Music, at the National Film and TV School (UK), and has worked extensively as a composer, sound designer and supervising effects editor in film. Prof. Mary Luckhurst is Director of the MA in Theatre: Writing, Directing and Performance at the Department of Theatre, Film and Television (The University of York), and is an experienced playwright and theatre director. Paul J. Abbott is a composer who regularly collaborates with Mary Luckhurst in sound design for theatre productions. Jerry Ibbotson is the founder of the York-based company Media Mill that develops sound for interactive computer games.

Method

The main idea for the workshop was that sound designers would create sounds for a short theatrical scene. The text of the scene was fixed and given in the Call for Sound Designs. The proposed sound designs would then be played as part of a performance of the scene in front of the workshop participants. In the performance the actor would respond and react to the sounds (and by inference to the objects that caused them). The audience would then be asked to observe the overall scene and discuss, with the sound designer and the actor, how well the sounds communicated the intended experience. Moreover, the goal was to take advantage of the Sound Design competence available in the communities of Theatre, Film, Games and Sonic Interaction Design and to stimulate discussions and reflections on the potential of theatrical performance and improvisation for sketching and prototyping sonic interaction designs. The aim of this exploration of the theatrical representation of sonic interaction designs was: - to discuss the effectiveness of the sonic interaction designs in light of the original designer’s intentions and the feedback produced by the actors and the audience; - to evaluate the advantages and limitations of using theatrical setups as a tool for testing and evaluating sonic interaction designs. The workshop was organised in three stages, the Call, the Performance and the Evaluation. In the Call the text of the scene was provided to the sound designers as a context, a setting and objects on which to base a sound design. The scene that was provided was "The New Anny" from the play "Attempts on Her Life" by Martin Crimp [19]. This scene was chosen because it includes explicit interactions with sounding objects, but also has many other concepts and images that make it open to different interpretations and performances. The scene is in the style of a script for an automobile advertisement (for a car called the New Anny) and describes actions by the driver and objects in the car. As the scene progresses there is a build up of dark concepts and images that transform it into a work of satire and tragedy. The sound designers were asked to submit two documents 1) a description relating each sound with actions or objects in the scene and 2) a description of their interpretation and setting of the scene, their intentions in the design of the sounds, and the technical strategies they used in the sound design. The actors were asked to rehearse the scene with the selected sound designs on the day before the workshop. This gave time to think about the sounds and respond to them in the performance. It also allowed for a high degree of improvisation and a great input on the performance by the actors. After the performance the audience, actors and designers were asked to discuss their interpretation of the sounds. The sound designer was then asked to describe the original intentions to the actors (interacting directly with the objects) and the audience (observers of the interaction). Two sound designs for the scene were selected for performance in the workshop. The first, by Stephen Barrass, was a sonic approach focussed on the sounds of interactions with objects. The second, by Prof. Mary Luckhurst and Paul J. Abbott, took a musical approach based on extensive experience with sound design for theatre. In this paper we will analyse the first performance with the sonic approach because it is more closely related to the topic of Sonic Interaction Design that we are exploring. Mary Luckhurst directed both scenes and the actor, Mark, performed in both scenes. They both participated in the discussions and were able to comment on the differences between the sonic and the more traditional musical approach.

2.1. Theatrical Metaphors and Methods in Interaction Design In addition to the narrative aspects above, interactivity also entails aspects of performance. There has been a developing discourse about performance, dramatic and theatrical processes in the field of Human Computer Interaction (HCI) as it has become more integrated with design disciplines. In her classic Computers as Theatre [11], Brenda Laurel extended the theatre metaphor to include the computer system itself, arguing that both are about the representation of actions with multiple agents. Laurel further defined dramaturgic criteria for developing interactions. More recently, interaction designers have begun to incorporate methods inspired by theatre into their creative process [12]. Some examples of this approach include scenariobased design [13], role-playing [14] and body-storming [15]. The general idea is that users play different roles, and the interactive system to be designed is considered in terms of interactions mediated by computers. 2.2. Sound Making as “Dramatic Performance” Focusing in from interactions with computer interfaces in general to systems that involve sound we notice how phenomenological properties of sound complement reflections about performance and theatre. Sound is often the result of an activity, of sound-making. We have to move our bodies, press our lungs, and interact with objects to produce sounds. The term “Ergo Audition”, coined by Michel Chion, decribes soundmaking as an expressive act [16]. It entails the experience of hearing oneself acting, of the acoustic manifestation of influence on the world. This notion is useful for designers because it goes beyond purely psychoacoustic and functional aspects of sound production and perception. It involves sound-making from a personal perspective, as a manifestation of agency and presence, always taking place in a socio-cultural context. This concept can be linked to the discussion of the acoustic community by Murray Schafer [17] and in particular Barry Truax [18]. Given these considerations, we suggested taking the performance arts as an additional field of practice and reflection into account, both in the design process as well as in theoretical reflection of phenomena related to sound in interactive contexts. This suggestion was incorporated into the workshop structure as follows : - Narrative sound design strategies were addressed through the invited talks from film, game and theatre sound experts and through the submissions for the call on Sound Design for a Theatrical Scene - Theatrical methods were used in the preparation and interpretation of the sonic interactions presented through the performance of the theatrical scene.

-2-


4.2. Mark in the Bathroom: The Plasticity of Meaning in Sound

There was a specific process employed for the presentation of the pieces. First the audience listened to the sound design in the dark, and suggestions were gathered regarding the interpretation of the scene, the understanding of the causes and meanings of the sounds, the expectations and the emotional content of the scene from sound alone. Then the scene was performed on stage with the actor. More feedback was gathered after this second performance looking at changes in the perception and interpretation, expectations and the emotional content. Finally the sound designer, the director and the actor explained their processes and a final discussion gathered comments on this “theatrical” method for testing and evaluating sonic interaction designs.

4.

In keeping with the intention for a great degree of improvisation, only a short rehearsal of the scene was planned the day before the workshop. On the day of the rehearsal it was found that there were no female actors available for the performance. This presented a challenge given that the sound design was based on a female actor interacting with kitchen implements. How could the situation be resolved if the actor was male instead? This led to a brainstorming session between everyone who was present Mary Luckhurst (theatre director), Stephen Barrass (sound design), Mark Smith (actor), Sandra Pauletto (workshop director and author of the sound design call), Daniel Hug (workshop collaborator and expert in sound design for interactive commodities), Andrew Fravigar (Theatre Officer) and Paul Ryan (Postproduction Officer). After some discussion it was decided that the way forward was to work out how to reinterpret the sound design through the theatrical actions of a male character. The process of creation of the final performance was a collaborative one in which ideas emerged through listening to the proposed sound designs, watching the actor improvise actions inspired by the sounds and context, and further brainstorming between the theatre director and all people present. We listened to the original sound design a few times. Kitchen implements were available for the actor to use. The element of the sound design that most struck the imagination of the people present was the surreal superposition of the dramatic images proposed in the text of the scene, and the normality of the actions proposed by the sonic interpretation in the kitchen. The fact that the performer was male together with the nature of the sounds suggested the morning ritual of a male character in the bathroom, adding another layer of surrealism to the performance (see Figure 1).

Results

4.1. The Anny in the Kitchen: the initial approach Below is the description of the Sound Design provided by Stephen Barrass in his submission titled “The Anny in the Kitchen”. The play portrays a woman named Anny through scenes with her friends and lovers, but Anny herself never appears. This particular scene is a spoof car advertisement that describes Anny in exotic/erotic marketing terms. The submitted sound design is an auditory sketch for actions to be performed by a female actor using kitchen implements and appliances to produce the sounds live on stage. This woman is at home in the kitchen listening to the radio on which we can hear a voice speaking the script of the 'The New Anny'. The voice on the radio was produced from the script with the Vicki voice of the Text to Speech Synthesiser on the MacIntosh computer. The music was produced using Microsoft Songsmith [20] which is a tool that automatically generates musical accompaniments for spoken or sung vocals. The sound design takes an asynchronous approach in which there are two distinct sound worlds that make occassional contact through syntagmatic 'coincidences', auditory puns, and semiotic similarities and contrasts. The first is the kitchen where sounds are a consequence of physical actions with implements. The second is the world of imagined sounds conveyed by the narrator on the radio. This approach was inspired by the French Director Rene Clair's manifesto on film sound [21] in which he argues against a literal soundtrack and introduces the concepts of asynchronous and contrapuntal sounds of objects and events that are not visible on the screen. Asynchronous sounds were generated from the New Anny text by substituting verb/noun pairs present in the script with verb/noun descriptors from a database of kitchen sounds. This substitution was inspired by the French Oulipo movement (founded by French Writer Raymond Queneau and François Le Lionnais in 1960) [22] in which rulebased constraints are used as creative devices for poetry, and which has been extended to sound works in Ourapo [23]. An example is to connect the sentence "the car twists along the Mediterranean road" with the sound of a cork twisting out of a wine bottle. Although this approach was fruitful, the script includes many non sounding objects and this strategy was difficult to maintain throughout. However the application of this process suggested other literary devices – such as paradigms (door slam for «shot»), metaphors (fizz for «gleam»), metonyms (phone ring for «a mobile phone»), contrapuntal (chopping for «happy») and puns (pest spray for «dirty bastards»). The dynamics of some sounds, such as the blender and the chopping were used to build tension. In the end there were a lot of different ways that the sounds related to the text, but they could all be produced with kitchen implements.

Figure 1: The Anny in the Kitchen: Mark Smith actor (photo by Daniel Hug) In particular the water sounds suggested setting the scene in the bathroom and various other mechanical and electrical sounds were re-interpreted as coming from implements usually found in the bathroom. It was also decided to use the sound design as submitted and rehearse the actor's actions in synchronisation to it. The actor and the theatre director worked a few hours on the performance creating a sequence of actions and gestures that were surreal and caricatural and in synchronisation with the soundtrack. Various other bathroom implements were provided to the actor who could then interact with them during the scene. The video of the final performance can be found at [24].

-3-


5.

Discussion

audience. This perception of synchronicity made the associations work well within the surrealistic theme of the performance. A member of the audience commented that the piece reminded of the films by French Director Jacques Tati in which the sounds are implausible for the actions and objects they portray, and noted that this fact often made the audience doubt that the objects are working properly. An additional factor that accentuated the unrealistic characteristic of the sounds was the fact that they were not placed in a sonic space in the mix (for example no reverberation was added), and they were played back from a point in space (the loudspeakers) separate from the object that should be the cause of the sound. This characteristic added to the surreal effect. However, while in the first listening condition we clearly heard three streams of different types of sound (music, voice and sound effects) coming from the two loudspeakers, during the performance the sounds seemed to be “magnetised” to the performer and objects present on stage. The synchronicity of actions with the sounds helped this “magnetisation”, i.e. the sounds were attributed to particular objects even if they were coming from a very different place in the space. The effect of "magnetisation" was described by Chion [25] with regard to spatial sound in the cinema. It was noted that, in some way, the performer seemed to become the "spatial" mediator between the audience and the sounds. The audience could now believe that the sounds came from the objects because the performer, with his expressiveness and timing, made us believe the overall space of the scene.

An audience of about 30 people was present for this theatrical performance at the workshop. The majority of the audience were experts in sonic interaction design involved in the COST-SID project with a range of both artistic and scientific backgrounds. As mentioned above, a specific listening method was devised for experiencing the theatrical performance. First the sound design was heard in the dark. A discussion followed on the nature of the scene conveyed by the sounds. These reflections were based exclusively on the sound effects, music, treatment of the voice and the text of the scene. Then the theatrical scene was performed and a second discussion followed on ways that the meaning of the sounds changed in this second listening experience. 5.1. Listening to the sound design in the dark This section is a summary of comments made by members of the audience in response to the sound design played on its own in the dark. In this listening condition three layers of sounds were recognised: the "cheesy euro-pop style" music, the voice and the sounds of objects and actions. The causes of a variety of sound effects were recognised: for example the door banging, the lighter, the spray sound and the sound of water being poured. The causes of some of the sounds were considered ambiguous. For instance a particularly noisy sound was considered to be either a blender or a hair dryer. The voice was recognised as processed, unnatural, but did not sound as if coming from a radio (i.e. it was not filtered like a voice from a radio would be). The text from the script was not always easy to understand due to the superposition of other sounds, however various connections between the words in the text and the sound effects were considered interesting (e.g. a gun shot is mentioned in the text and a loud door bang is heard in the sound mix). Some text/sound associations were clear (the mention of a mobile phone was followed by the sound of a ring tone) and created expectations for similar literal associations to be repeated. Those expectations were however often frustrated. Most people thought that the main character in the scene was a woman in the kitchen, or performing a morning ritual involving the bathroom and the kitchen, but the overall narrative and sequence of events was unclear. The clear change in meaning present in the text, i.e. the fact that it started as a positive advert and ended with tragic warnings, was understood. Most of the sound effects were also considered emotionally or expressively neutral and for this reason no clear narrative could be deduced from listening to them.

5.3. Comments from the sound designer, the director and the actor After the second discussion, the sound designer revealed his process and then the theatre director and the actor commented on the process. Mary Luckhurst, director of the scene, observed that the process of designing theatre from sounds was innovative and liberating. She noted that theatre directing traditionally works on the assumption that the meaning comes principally from the understanding of the words. However she found that starting from the non-verbal sounds to generate the actions, gestures and expressions was very interesting. The sounds suggest actions, gestures and meaning, and therefore influence the creation of the action on stage. She also noted that choosing actions, which literally describe the sounds, would have diminished the power of meaning of the sounds. To challenge the references the sounds provoke in the imagination of the audience seemed to be a much more interesting creative process. The actor commented that it took a lot of practice to synchronise his actions with the sounds and some were missed due to the fast pace of the piece. He also mentioned that objects and sounds were deliberately mismatched (i.e. electrical sound (blender) attached to a manual toothbrush, instead of electrical) to increase the surreal effect and the surprise. Finally one audience member commented that the piece was engaging because the audience was at particular moments required to be passive (for example accepting that a sound not appropriately filtered was supposed to be coming from the radio) and sometimes active (for example the mismatch of certain action/objects and sounds surprised people and made them laugh). The energy going back and forward between the stage and the audience created curiosity. Not only did the sound design require a theatrical resolution, it also caused the audience to become engaged in order to resolve it.

5.2. Listening and watching the performance The piece was then performed by the actor who performed an animated morning ritual in the bathroom in which various events were sychronised with the sound design. A second discussion took place after the performance and the narrative of the scene was now much clearer. A man goes through his morning rituals in the bathroom while listening to the radio. It was noted that the sense of time was extremely altered and that the overall scene seemed much shorter. The comments about the synchronicity between actions and expressions with the sounds showed that the effect of synchresis (the association of a sound with a synchronous image and vice versa) [25] extends well beyond the medium of film. Some of the sounds were considered realistic for the action performed, while others were clearly unrealistic and this created a comic effect. This oscillation between plausible and implausible associations between sound and action created expectation and surprise, which were engaging for the

6.

Discussion of method

One main observation from this process is that the meaning of a sound heard on its own, and the understanding of its cause, is

-4-


7.

greatly influenced by experiencing the action and the actual object that are supposed to produce the sound. This means that creating sound designs without ‘testing’, ‘prototyping’ or ‘sketching’ the action and object is not sufficient as design method for interactive artefacts. This result reinforces the belief that methodologies are needed in Sonic Interaction Design that allow to explore and evaluate sound designs holistically. Only in this way the interaction between sound, object and action in terms of meaning making can be experienced, analysed and understood. The workshop has shown the power of theatrical methods to bring sonic interaction scenarios to life and to explore various possible associations of objects, sounds and actions. Theatrical methods allow to explicitly involve non-represented action by humans in the design process. It provides a setting which is both controlled and relatively close to an "in situ" setting. The method's main strength lies in its ability to support a rich discourse among many stakeholders. The actor, the director, the sound designer, the audience and the moderator all provide different interpretive standpoints. The most extreme are the role of the observer (the audience) and the actor. The role of the director, the sound designer and the actor are ambivalent in that they can experience both the process of creation as well as the observation and execution of action (as in the case of the actor). This leads to a process of interpretation which is characterised by its iterativeness and limited predictability. Meanings are reestablished and re-interpreted several times. The method was particularly successful in activating the audience and fostered a lively discussion about the use of the sounds. Altogether such a procedure is ideal for research and design practice that relies on rich data. It is not very suitable for reductive analysis of causal relationships, but can even support this kind of more focused investigations by helping to point out "blind spots" in the research agenda. This richness of standpoints, accounts and documented experiences can balance one obvious shortcoming of the method, which lies in the relatively big efforts required for its preparation. It is not a lightweight, agile approach as it requires an elaborate interplay of place (theatre stage), theatre experts and an audience of a reasonable size. However, comparable lightweight versions of theatrical methods have been successfully applied already (see, e.g. [5]) and, as mentioned in the background section, are part of several interaction design methods already. The stage worked quite well as a proxy for the world and although the spatial separation between the actual sound source (speakers) and the interaction can be disorienting and can break the believability of the sound/interaction coupling, the "magnetisation" of the sounds to an action was quite strong. However, there are obvious limitations to the theatre setting as a replacement of a real-world experience. In particular, the black box theatre with a stage set in front of the public still creates an artificial environment with a strong hierarchy, which has little resemblance to everyday social situations. An alternative could be to look at younger forms of theatre and performance that break out of the traditional stage. However theatre, extended to performance arts in general, offers many interesting aspects that are useful for the design and evaluation of interactive sounding artefacts: it is accessible (we all have done role-plays in our lives), it provides an intermedial space that shares some properties with interactive artefacts (representations of representations), there is a spatial and corporeal presence of both objects and people (in comparison to film), and last but not least, humans are the driving force of both action and interpretation.

Conclusion and future work

We have presented the results from the Call for Sound Designs created for the Workshop on "Sonic Interaction Design and its relation to Film and Theatre Sound Design" at the University of York. We have described the theoretical background to the Workshop and Call and their realisation. We have described one Sound Design submission and its preformance. Finally we have presented the method used to present the sound design to the audience for discussion, and we have reported our comments regarding the use of theatrical methods to sketch, prototype and evaluate sonic interaction designs. Summarising some main points, we observed that synchronicity and spatial relationships between the sound and the object/action are very important aspects that allow us to believe a sound (plausible or not) to be produced by a certain action or object. The meaning of sound in itself is not necessarily a static given and changes in the interactive condition or may even become meaningful in this condition only. Many levels of interplay between sounds and interactions were observed: some combination was considered very neutral and easily accepted, on the other hand, if a sound was perceived as very unrealistic as a sound of a particular interaction then the effect was usually comical and surprising. The method is paricularly strong in providing rich data on prototypes of sounding artifacts at a early stage of development as well as it is very suitable for inspiring new ideas. It allows to test design hypotheses under conditions that include interaction with actual artefacts. The involvement of theatre experts, in particular professional actors, brings an inspirational quality to the process that is ideal to verify or explore emerging meanings of sounding interactive artefacts. The theatre thus seems to be a useful setting for developing designs at an early, explorative stage, where a specific effect is not known yet or a high level of innovation is required and can be a useful addition to experimental and ethnographic design, and evaluation methods in the context of SID. The process described here also provides material to fuel the theoretical discourse related to the relationship of sound and action and thus supplements comparable methods from film and game sound studies and action analysis. Future work will both include reapplication of the method in education and research as well as further evaluation of the outcomes with the aim of further developing the young field of Sonic Interaction Design.

References [1] S. Pauletto, Sonic Interaction Design and its relation to Film and Theatre sound design [2] Mihaly Csikszentmihalyi and Eugene Rochberg-Halton, The meaning of things - Domestic symbols and the self, Cambridge University Press, Cambridge, (1981) [3] D.Hug, Genie in a bottle: Object-sound reconfigurations for interactive commodities, in Proceedings of Audiomostly 2008, 3rd Conference on Interaction With Sound, (2008b) [4] S. Barrass, EarBenders: Using Stories About Listening to Design Auditory Interfaces, in Proceedings of the First AsiaPacific Conference on Human Computer Interaction APCHI'96, Information Technology Institute, Singapore (1996) [5] K. Franinovic, D. Hug, and Y. Visell, Sound embodied: Explorations of sonic interaction design for everyday objects in a workshop setting, in Proceedings of the 13th international conference on Auditory Display, Montreal, (2007) [6] D. Hug, Using a Systematic Design Process to Investigate

-5-


Narrative SoundDesign Strategies for Interactive Commodities in Proceedings of the 15th international conference on Auditory Display, Copenhagen, (2009) [7] D. Hug, Towards a hermeneutics and typology of sound for interactive commodities in Proc. of the CHI 2008 Workshop on Sonic Interaction Design, Firenze, (2008) [8] Janet H. Murray, Hamlet on the Holodeck - The Future of Narrative in Cyberspace, MIT Press, Cambridge, Mass., (1997) [9] M. Lopez and S. Pauletto, The design of an audio film for the visually impaired, in the Proceedings of the International Conference on Auditory Displays, Copenhagen (2009) [10] S. Barrass: Haptic-Audio Narrative: From Physical Simulation to Imaginative Stimulation. HAID 2006: 157-165 (2006) [11] Brenda Laurel, Computers as Theatre, Addison-Wesley Professional, 2nd edition, (1993) [12] Brenda Laurel, Design Improvisation - Ethnography Meets Theatre, in Design Research – Methods and Perspectives, MIT Press, Cambridge, Mass., (2003) [13] John M. Carroll, Making Use - Scenario-Based Design of Human-Computer Interactions, MIT Press, Cambridge, Mass., (2000) [14] Eva Hornecker, Hal Eden, Eric Scharff, “In MY situation, I would dislike THAAAT!” - Role Play as Assessment Method for Tools Supporting Participatory Planning, Proceedings of PDC 2002: 7th Biennial Participatory Design Conference, Malmö, (2002) [15] Antti Oulasvirta, Esko Kurvinen, Tomi Kankainen, Understanding contexts by being there: case studies in bodystorming, Personal Ubiquitous Computing, pp. 125-134, Springer, London, UK, (2003) [16] Michel Chion Le Son Editions Nathan, Paris, (1998) [17] R. Murray Schafer, The Soundscape: Our Sonic Environment and the Tuning of the World, Destiny Books, New York, (1977) [18] B. Truax, Acoustic Communication, 2nd ed., Greenwood Press, Westport, CT (2001) [19] Crimp M., Martin Crimp: Plays, Faber and Faber, London (2005) [20] Microsoft Songsmith, [21] Clair, R The Art of Sound (1929) in Weis E. and Belton J. eds Film Sound: Theory and Practice, Columbia University Press (1985) [22] Oulipo [23] Ourapo, 2004 [24] The Anny in the Kitchen/Mark in the bathroom video [25] M. Chion, Audio-Vision: sound on screen New York: Columbia University Press, (1994)

-6-