Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008
EXPLORING SONIC INTERACTION WITH ARTIFACTS IN EVERYDAY CONTEXTS Karmen Franinovic
Lalya Gaye
Zurich University of the Arts Daonk! Collective Zurich, Switzerland Goteborg, Sweden
[email protected] [email protected] ABSTRACT This workshop aims to introduce ICAD participants to the use of creative interaction design methods when exploring the design of sonic interactions with computational artefacts. Specifically, focus will be on physical interactions that rely on continuous sonic feedback. Participants will generate future scenarios and concepts for such interactions, and everyday sounding objects in context will be taken as a starting point. By taking part in the workshop activities, participants will get an embodied understanding of the challenges of designing for meaningful and engaging physical interaction with computational sonic artefacts. Methods employed in the workshop will thus be good complements to the cognition or technology-based approaches to designing sounding objects that are mainstream within the ICAD community.
Frauke Behrendt University of Sussex Brighton, UK
[email protected]
such contexts might be changed. They will do so by trying out a sample of creative and physically engaging interaction design techniques such as bodystorming [5] and interaction relabeling [6]. Originating from the field of interaction design, such playful and collaborative approaches to the development of interaction concepts are meant to allow the workshop participants to directly engage in everyday sonic experiences an embodied way. This will inform the design of tangible computational artefacts where interactive sound will provide meaningful contribution to overall experience. Therefore, these methods constitute a good complement to the scientific and technological approaches to auditory displays that the ICAD audience is accustomed to.
1. INTRODUCTION Designing for continuous feedback rather than a discrete trigger response opens up a range of new design opportunities, questions and problems [1], [2], [3] . Usually, when designing an interaction with a button on a certain product, different keys are indexed by the means of symbols and icons, as in the case of computer keyboard. However, designers can also make use of in the way the key is touched (e.g. how fast, how long and how strong). How can these physical aspects of interaction be meaningfully related to sonic feedback in tangible artifacts? In order to address this question, we designed a workshop that consist of explorations of everyday sounding objects in context. Its goal is to collaboratively, through design exercises, develop concepts and scenarios for future products that use continuous sound feedback. Through the latter, everyday objects naturally respond to their physical manipulation. Pouring a cup of tea generates the sound of continuously flowing water, while the act of walking is accompanied by the rhythm of the footsteps. Sounds produced when interacting with everyday objects communicate about this physical coupling: the richness of gestures it is made of (e.g. how fast and from which distance we pour the waters) as well as the context surrounding the interaction (e.g. what kind of pavement we walk on). Therefore, they are good starting points in exploring the design of tangible computational artefacts with continuous sonic feedback. Rather than exploring the question by addressing technical issues such of gestural data acquisition, sensing techniques, finding appropriate sound tools or using physical modelling (e.g. [4]), this workshop aims to embrace the design opportunities emerging from the application of such interactive sound technologies to everyday life. Participants will explore the meanings and importance of tangible sonic interactions in variety of contexts and imagine how
Figure 1: Everyday objects afford simple actions: Turning the handle in circle, squeezes and purees vegetables generating a range of sounds that change during the activity.
2. WORKSHOP STRUCTURE, TECHNIQUES AND METHODS This one day, hands-on workshop will be divided in fours parts: warm-up exercises, creative idea generation, concept exploration through bodystoming and final presentation and discussion. Workshop participants and facilitators will use no digital technology. By focusing on the experience rather than technology this workshop allows to foreground the contextual and embodied aspects of sonic
ICAD08-1
Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008
interactions [7], [8], [9], [10], [11], [12]. The complexity of multisensorial and temporal interactions with objects and sounds guides the workshop activities. The participants will engage in the phenomenological explorations of existing sonic experiences and this will inform the development of their own ideas and the creation of new sonic interaction concepts. The workshop is aimed at ICAD audience, but welcomes a wide range of participants with various backgrounds such as artists, designers, musicians, programmers, cultural and media studies scholars, etc. Working with an interdisciplinary group will be inspiring for all participants and enables to draw on the wide-ranging expertise of the group members. No previous technical knowledge or design experience is required. The workshop authors draw on their previous experience with such events [13], [14] .
the movement of the legs and the sound of the heels touching the ground. In addition we hope this will raise awareness that the situated sonic artefact can create a completely different experience due to the interaction with sounds and people in the context. In the final warm up exercise we will address the tactile and haptic experience of sonic artefacts. The workshop organizers will prepare a Sonic play table equipped with rubber bands, tubes, fabric and everyday objects (see Figure 2). These objects and materials will be explored by the blindfolded participants. Each will describe the haptic and tactile sensations caused through manipulation of the artefact and the way it is linked to sound. We will discuss sound as vibration and the ways in which the shape of the object affects its propagation as well as the way in which it is manipulated.
2.1. Sonic and Tactile Experiences in the Context The workshop will begin with a number of warm-up exercises. In order to engage participants in the thinking process on topics such as sounds, objects and haptics, even before the workshop begins, we will ask them to bring along an inspiring sonic object. Participants will introduce themselves and their everyday object, describing its sound and tactile qualities. We will discuss the reasons the specific object was selected for this workshop. Once everybody is introduced, all participants will reproduce the sound of their object without using the object itself. This exercise is an initiation of participants to quick and dirty sound prototyping by the means of their voice, their own bodies or in interaction with other objects; a skill that will be used in the ideation and presentation stages of the workshop.
Figure 3: Design matrix (on the wall) being filled with sketches of design concepts through interaction design speed-dating.
2.2. Idea Generation
Figure 2: Sonic play table (in the background) and interaction speed-dating. The following step will generate an understanding of the relationship between the listener and the surroundings and is inspired by the Sonic Postcard project [15]. The participants will be given two postcards, walk to nearby urban areas close their eyes and listen to their surroundings for few minutes. They will be asked to create a visual representation of the sound around them on a sound postcard. Subsequently, the participants will be asked to look around for interactions between people and artefacts that are silent and sketch them on the action postcard. In addition to presenting the participants with the complexity and richness of everyday soundscapes, this contextual exploration allows participant to link physical phenomena to the sound they generate such as
The second phase of the workshop encourages the participants to generate their own sonic ideas. In order to generate a large number of ideas, we will use idea matrix speed-dating method [16] . It is based on the idea of conceptualizing new ideas through multidimensional matrix of preselected design parameters. Three dimensions linked to potential design problem will be defined. These will include location (for example : prison, beach, kitchen, street), artefact (for example: glass, chair, bottle, tile) and activity (for example: walking, blowing, cutting, drinking). Their couplings will define the space of design opportunities that will be filled by participants’ concepts of interactive sonic artefacts. The organizers will prepare a number of cards with different actions, locations and artefacts based on the warm-up exercises. The participants will choose random cards by picking them from the three bags. Working in pairs, they will note the three instances of the categories they work with and sketch a design concept. After ten minutes, partners will change, new cards will be taken from a bag and new concept developed and added to the space of design ideas (Figure 3 shows a two dimensional matrix filled with sketched design concepts). In this way participants interact with each other through conceptualization constrained only by time. Each person will present his or hers favorite concept and discussed it with a group. 2.3. Developing Ideas through Embodied Explorations In this phase of the workshop specific concepts are explored in more detail in small group sessions. Groups of three to four randomly selected participants will be joined by the workshop orga-
ICAD08-2
Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008
nizer who will act as a facilitator for the concept development. Each group will choose several brainstorm ideas, discuss and chose one of them. The group will then have time to focus on this selected idea. It will be further elaborated through the use of bodystorming [5] and interaction relabeling [6] techniques. Bodystorming
Figure 4: Bodystorming techniques are used to physically experience and simulate imagined sonic interaction scenarios. is a methods in which future scenarios of use and interaction are enacted. Designers take on the roles of the users or artefacts in order to understand and reveal challenges and opportunities of their ideas. Such embodied design exploration is present in the interaction relabeling methods as well. In it ”possible interactions with a known mechanical device are mapped to the functions of an electronic device to be designed; and extreme characters, in which fictional users with exaggerated emotional attitudes are taken as the basis of design to highlight cultural issues.” [6]. We adapt these techniques for the sonic, haptic and situational focus of the workshop.
each other. This will allow them to test if the sounds they have chosen are appropriately communicating the intended meaning, sensation or emotion. For the final presentation of their scenarios, the groups have to enact the scenario in front of the other groups without images or verbal explanation: the actors have to communicate their ideas by enacting a scenario with their bodies and gestures and by producing sounds themselves. The variety of scenarios developed in the groups will then kick-start the final discussion of the workshop. We will conclude the workshop with a final discussion that will be informed by the various stages o the workshop: the warmingup, the idea generating and the idea development. We will discuss the new sonic potential in everyday life and debate the design challenges we encountered. The difficulty of verbalising these experiences, and lack of a common language to describe these complex interactions could be another topic of interest for the final discussion. The very activities and methods used in the workshop, and how they facilitated a focus on sonic and haptic interactions will also be productive for the discussion, asking how activities and methodologies can be developed further in this emerging area of interest. The discussion might also touch upon different sound design techniques - designing interaction through physical sound models vs. sample-based algorithms; the use of everyday sounds, etc. 3. CONCLUSION The workshop will give participants the opportunity to explore key aspects of tangible interaction and continuous sonic feedback, all in an activity-based way: the fact that the workshop will consist of low-tech small group exercises and interaction-focused creative activities instead of paper presentations or technology tutorials, will invite participants to have an active role and to engage all senses in the exploration of this type of situated sonic interaction. Thereby, and as focus is taken away from technology itself and turned instead towards the situated activities of using technology, the different workshop activities will raise awareness of the importance of multi-sensory and contextual aspects of interactions that are often overlooked within the ICAD research field. Acknowledgments The photographs used in this paper are from the previous workshops: From Tangible to Intangible and Back Again, a workshop (2007, Montreal) conducted together with Yon Visell and Michelle Valigura and Mobile Music Workshop co-organized together with Lars Erik Holmquist and Atau Tanaka (2005, Vancouver). 4. REFERENCES [1] Davide Rocchesso and Pietro Polotti, “Designing continuous multisensory interaction,” in Proc. of Sonic Interaction Design workshop at Computer-human interaction conference, Firenze, 2008.
Figure 5: Participants creating props.
[2] C. Cadoz and M. M. Wanderley, “Gesture-music,” in Trends in Gestural Control of Music, M. Wanderley and M. Battier, Eds. IRCAM - Centre Pompidou, Paris, France, 2000.
2.4. Presentation and Discussion For the final stage of the workshop the small groups will come back together to share their project ideas by acting them out for
[3] J. Lagarde and S. Kelso, “Binding of movement, sound and touch: multimodal coordination dynamics,” Exp Brain Res, vol. 173(4), pp. 673–688, 2006.
ICAD08-3
Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008
[4] Matthias Rath and Davide Rocchesso, “Continuous sonic feedback from a rolling ball,” IEEE MultiMedia, vol. 12, no. 2, pp. 60–69, 2005. [5] Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen, “Understanding contexts by being there: case studies in bodystorming,” Personal Ubiquitous Comput., vol. 7, no. 2, pp. 125–134, 2003. [6] J.P. Djajadiningrat, W.W. Gaver, and J.W. Frens, “Interaction relabelling and extreme characters: Methods for exploring aesthetic interactions,” in Proc. of Designing Interactive Systems, Brooklyn, New York., 2000. [7] M. de Certeau, The Practice of Everyday Life, U California Press., Berkeley, 1984. [8] B. Schulz, “The whole corporality of hearing. an interview with bernhard leitner,” in Resonanzen : Aspekte der Klangkunst. Resonances : aspects of sound art., pp. 81–88. Heidelberg: Kehrer, 2002. [9] N. Brown, “The flux between sounding and sound: Towards a relational understanding of music as embodied action,” Contemporary Music Review, vol. 25(1/2), pp. 37 – 46, 2006. [10] C. Spence and M. Zampini, “Auditory contributions to multisensory product perception,” Acta Acustica united with Acustica, vol. 92(17), pp. 1009–1025, November/December 2006. [11] Ludden G. D. S., Schifferstein H. N. J., and Hekkert P., “Surprising the sense,” Senses and Society, vol. 2(3), pp. 353– 359, 2007. [12] G. Ouzounian, “Embodied sound: Aural architectures and the body,” Contemporary Music Review, vol. 25(1/2), pp. 69 – 79, 2006. [13] K. Franinovic, Y. Visell, and D. Hug, “Sound embodied: A report on sonic interaction design in everyday artefacts,” in Proceedings of the 13th International Conference on Auditory Display, Montreal, Canada 2007. [14] Lalya Gaye, Lars Erik Holmquist, Frauke Behrendt, and Atau Tanaka, “Mobile music technology: report on an emerging community,” in NIME ’06: Proceedings of the 2006 conference on New interfaces for musical expression, Paris, France, France, 2006, pp. 22–25, IRCAM; Centre Pompidou. [15] Sonic Arts Network, “Sonic postcards project,” http://www.sonicpostcards.org/. [16] Eric Paulos, Ken Anderson, Michele Chang, and Anthony Burke, “Metapolis and urban life,” in Proceedings of the 2005 conference on Ubiquitous Computing, 2005.
ICAD08-4
