Evolving creative practice: a reflection on working with ...

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

Evolving creative practice: a reflection on working with audience experience in Cardiomorphologies George Poonkhin Khut & Lizzie Muller

Introduction This paper is an account of the early stages of the process of creating and exhibiting the artwork Cardiomorphologies. It is very much a reflection on work in progress, written from the midst of a developing collaboration and attempting to identify the emerging new directions in our creative practices which it has engendered. In many ways the paper invites input and discussion at an unusually early and tender point, but this is in-keeping with the spirit of the process we will go on to describe, which seeks to integrate dialogue and collaboration with audiences and peers at every stage of development. Cardiomorphologies is a computer-based interactive installation that uses bio-sensing and multimedia technologies to create real-time visual and sonic representations of the audience’s breath and heart rate. As you will see the form and detail of the work is continually developing throughout the process of creation we describe here. However in order to give an imaginative starting point we will describe the basic set-up that has so-far remained more or less constant in its several iterations. Individual participants are seated before a large screen in a dimly lit and enclosed space. The participant is fitted with a breath sensor (usually a stretch-sensor fastened around the body below the rib cage) and heart rate sensors (either attached to the arms or held in the hand). The pattern and frequency of breath and heart rate is projected onto the screen as animated visualizations in various ways such as expanding and contracting circles or changing colour fields. A surround sound system plays back a complex soundscape with amplified heart sounds that beat in time with the participants own heart-rate. During the course of their interaction participants are encouraged to use the work as a feedback system to observe and experiment with their own breath and heart-rate patterning. While the elements that constitute the artwork are unfixed the creative aims which underpin it are more constant. The goal is to create a work that allows participants to explore the embodied nature of their subjectivity and to experience a visceral and highly personal insight into the interrelation of cognition, emotion and physiology. There are two propositions that lie behind these aims. First that biofeedback interaction opens up a very different experience and representation of the “body-mind” to everyday experience, and second that interacting with transformed images of ourselves through computer mediation makes us aware (in a possibly unique way) of our embodied subjectivity. As artist and curator, but also as practice based researchers we needed to develop a methodology for understanding and demonstrating whether or not actual audience experiences supported these propositions, but also a way to achieve, and to demonstrate that we were achieving our aims. The realisation of Cardiomorphologies depends entirely on the physical and psychological involvement of individual participants. The audience’s embodied experience is both the site and the content of the work. Therefore working with the material of audience experience needs to be at the heart of both our creative and research processes. The approach that we are developing to meet these needs draws together our respective curatorial and artistic practices and their related contexts of studio and gallery to create an iterative development process structured on ongoing dialogue with audiences. It is based around the use of the exhibition as a site of collaborative public research. Our challenge throughout is to find ways to record, understand and integrate relevant and authentic audience experience. To meet this challenge we have sought methodological approaches that share our commitment to the primacy of lived, embodied experience. We have found them in the theories, tools and techniques of phenomenologically influenced practitioners in Human-Computer Interaction. This paper presents the development of Cardiomorphologies as a case study of the application of the methods we adopted from HCI as a means of creating interactive art experiences that involve novel

1

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

and highly specialised forms of audience participation. We also take this opportunity to reflect on the implications of the integration of these techniques for our ongoing practices.

Exploring self: physiologically interactive artworks Bio-feedback and art The use of physiological sensing and biofeedback technologies is by no means new, and artists started using these technologies as early as 1965 (Lucier 1965; Rosenboom 1976; Frieling 2004). Bioelectrical sensing technologies introduced during the early 1960’s provided a new way of experiencing the living body in ways that not previously been available, and heralded a broader movement towards a more systems oriented analysis of natural and social phenomena. In terms of the body and its representation this can be observed as move from cadaver-based representations of the human body toward a more process oriented analysis of physiology as a dynamic and intelligent self-regulating system. The motivation for developing Cardiomorphologies stems from a curiosity about the unusual experiential spaces opened up by these bio-sensing technologies, and the ways in which they render our subjectivity as an overtly active form of psychological and physiological orientation. Physiologically responsive artworks provide a means for us to observe and enact subtle changes in body state and nervous system arousal. This ability to enter into the processes of our own physiology through an art experience provides the basis for a radically altered perspective on bodily representational and practice in contemporary culture. Beyond their immediate charm as highly meditative aesthetic experiences, physiologically responsive works focusing explicitly on the actual processes of psycho-physiological interaction also seek to engage participants in a broader critique of their habitual psycho-physiological reflexes and modes of orientation, calling into question conventional representations of subjectivity and perception. Through detailed observations and conscious psycho-physiological adjustments, the participant’s experience becomes a form of expressive act.

A phenomenological understanding the experience of Cardiomorphologies The phenomenology of Merleau-Ponty, which investigates embodied subjectivity and active perception, can help us to understand the experience interactivity in art, and particularly a physiologically responsive work like Cardiomorphologies. In Merleau-Ponty's account of human being-in-the-world there is no such thing as passive on-looking, our relation to the world is not that of an inner thinking subject looking out at an external world. We are part of the world and our experience is always situated and embodied; it is necessarily contingent upon our being in a particular place, and being a particular body. The fact of our embodiment is what organises and shapes our experience of the world through our perception. Perception itself is active, creative and generative of meaning. Paul Crowther describes how, for Merleau-Ponty this process of active embodied perception involves a kind of mirroring with the world: “We give [Otherness] contour, direction, and meaning; thus constituting it as world. On these terms, the structure of embodied subjectivity and of the world are directly correlated. Each brings forth and defines essential characteristics of the other." (Crowther 1993). He argues that it is this “ontological reciprocity” that gives the experience of artwork special place in Merleau-Ponty's philosophy. The artwork is a kind of mirror that: "…reflects our mode of embodied inherence in the world, and by clarifying this inherence it brings about a harmony between subject and object of experience - a full realisation of the self." (1993: 3). Crowther's argument is based on a weaving between Merleau-Ponty's early philosophy as shown in Phenomenology of Perception (1962) with his writing on painting in the essay Eye and Mind (1964). But the argument readily offers itself for extension to other artistic forms – and particularly physiolgocially interactive art works like Cardiomorphologies. The unusual experiential space that

2

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

physiological reflection in art opens up can be understood as the “privileged moments” MerleauPonty noted where we become aware of the reciprocity of perception, of the relationship between self and world, “when we see ourselves in the act of seeing” (Crowther 1993).

Experience as material: The design challenge of Cardiomorphologies Cardiomorpologies is intended as a kind of provocation, to challenge our conventional experiences and representations of body-mind. This provocation is communicated to audiences through their own conscious intervention into bodily processes that are normally conceived of as unconscious. Beyond this initial provocation, the work is also intended as a highly specialised kind of place that encourages a sense of curiosity and detailed experimentation concerning the involvement of our mental and bodily processes in our moment-to-moment experience of our selves. The full significance of the work emerges primarily through the psycho-physiological processes that are enacted during the course of their interaction with the artwork. Put simply, you have to do it before you can know it. Whether or not these intentions translate into actual experiences on the part of the people interacting with the work depends on how effectively the design as a whole is able to facilitate this kind of highly specialized observation and interaction. As the 'site' within which the work is located the qualities of the audience's subjective and bodily experience of the work is critical to it's ultimate effectiveness. The highly nuanced and selfconscious sensory abilities that these works draw-on are not common features of everyday social interactions. Whilst we might suppose that these sensory abilities are innate in all of us (and constantly at work at an unconscious level) bringing them into play in an unfamiliar social setting in the presence of total strangers, and an unusual, complex interface, requires a great deal of research and preparation. The challenge is to design an experience for countless individual people, which begs the questions: How can we design a general set-up that will work for specific participants? What assumptions have we made as artist/curator about the physiologies and experiences of the people interacting with this work? Were these assumptions grounded in practical experiences, or have we just assumed that everyone thinks and behaves more-or-less just like us?

HCI and Interactive Art A conflicted convergence In 2003 Phoebe Sengers and Chris Csikszentmihalyi presented a panel at the ACM CHI New Horizons Conference called “HCI and the Arts: A Conflicted Convergence” (Sengers 2003). Citing the small but growing body of published research in this area, they pointed to its emerging significance and the potential benefits to both the domains. They also identified some serious conflicts in the underlying goals of HCI and art; HCI, they argued, is concerned with “usability” while artists “revel in the nonutilitarian nature of their work”. In another paper reporting on their study of the process of applying HCI techniques to artwork Sengers, along with Hook and Andersson, elaborate on this conflict, tracing its origin to a fundamental difference in underlying philosophy: "Grossly speaking the major conflict between artistic and HCI perspectives on user interaction is that art is inherently subjective, where HCI evaluation, with a science and engineering inheritance, has traditionally strived to be objective" (Hook 2003). The perception of this opposition comes from both sides. Artist and theorist Simon Penny calls for a new aesthetic approach to what he calls "computer automated cultural artifacts” which takes account of interaction. He acknowledges the role of HCI in this new aesthetic approach but claims it will be limited because HCI, as essentially "fordist and taylorist", deals with the "mechanics of interaction" rather than the "poetics"(Penny 2004). Due to this perceived opposition which caricatures both art and HCI the two fields have in previous studies informed one another but remained strongly demarcated. Hook, Sengers and Andersson conclude that there is scope, and

3

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

necessity to go further than they themselves have gone, and to “more fully integrate the perspectives of art and HCI”. The emerging body of phenomenological HCI research offers one way to draw the two fields more closely together by offering a common philosophical foundation, and a set of shared problems and questions

The phenomenological approach in HCI In his book “Where the Action Is” (2001) Paul Dourish heralds a new direction in HCI research with its roots in phenomenological philosophy. He concludes that a greater understanding of phenomenology can enlighten technology design. This belief echoes a number of other researchers (Moran 1990; Svanaes 1999; Robertson 2002) who exemplify this phenomenological approach. Their work is characterized by an emphasis on the lived experience of technology users, and is often built on empirical ethnographic research. A common aspect of the work of these researchers is the recognition of the importance of uncovering, exploring and progressing the philosophical underpinnings of technological research and design. The detailed and attentive way in which these researchers relate interactivity to primary phenomenological texts establishes a common philosophical foundation with research in interactive art, from which shared questions, goals and methodologies can develop. The work of HCI researchers motivated by a primary commitment to the technology-user can be described as human-centred interaction design. Their methods include a variety of tools and techniques used to understand and work with user experience. The use of these tools has an impact not just on the structure of the design process but on its orientation to the world around it. Researchers such as Robertson et al (2005) and Lehn et al (2001) have shown the effectiveness of applying human-centred analysis and design methods to interactive art exhibits. Lehn et al use video cameras in an ethnographic approach to observing visitor behaviour in galleries and museums, and have identified the socially situated way in which people organize their interaction with exhibits. In the work of Robertson et al human-centred design techniques were used as “tools to think with” in the design process of an immersive interactive installation. They note that the use of these techniques had a political dimension in balancing the desires of the artists and the demands of the technology with the power of the audience to shape their own experience.

The process of working with audiences in cardiomorphologies Using tools and techniques from human-centred interaction design we have developed an iterative approach to creating and curating Cardiomorphologies in a cycle that links the studio and the public exhibition context. The process has focused on 3 stages of intense activity, each using different tools and resulting in a new iterative development of the work, but it is best conceived, not as linear or sequential, but rather as a set of intertwining techniques feeding off and informing one another throughout,. The format of the stages and the major changes in each iteration are: • first iteration: a 5 day public exhibition at the Performance Space, Sydney, June 2004. Initial visulisations made by John Tonkin based on a concentric circle pattern. •

second iteration: intensive development period at Performance Space Sydney including two week workshop with software programmer Greg Turner and 3 days of invited public participation, May 2005. Major concerns for this stage included developing the visualisations, improving the mapping from bio-input to multimedia output and the induction/interpretation process. This included the development of a flexible set of visualisation tools with Greg that enabled changes to be made more or less on the fly in order to be more responsive to audience feedback and allow more improvisation and testing. It also included experimentation and testing with ambient qualities of the installation such as heat, noise and light.

•

third iteration: two day technical workshop and one week public display at Beta_space in the Powerhouse Museum, September 2005. Of the numerous developments planned for this iteration the most significant are the introduction of more stable and less intrusive sensors, and the development of more abstract, impressionistic and responsive

4

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

visualisations. Questions of sound bleed and participation by children lead to numerous interface and induction modifications.

Collecting experiences The first attempt to record audience experience was a large-scale gathering of verbal data through post-experience interviews alongside the earliest iteration of the work. Over a period of five days eighty audience interviews were recorded on audio tape. The interviews formed the corpus of a rich data source of different viewpoints to which we referred during the early stages of the development process. The interviews had some limitations in that they tended to produce analytical, rather than experiential responses that often seemed to mirror the questions asked rather than evoking a close account of the actual lived experience of the artwork. We wanted to get more natural and detailed verbal accounts of the experience so we turned to a technique called video-cued recall. Video-cued recall is a method for investigating situated action in which participants are recorded during an event (in this case their interaction with Cardiomorphologies), and then asked to give an account of their experience whilst watching the video of themselves. We asked participants to recount every detail of their experience, no matter how mundane it may seem to them, and to stick carefully to what they remember of their experience as it unfolded rather than speculating or analysing. The video helps the participant recall the detail of their experience and avoid selective interpretation. For a full description of this method, and an example of its use in documenting audience experience of another artwork see Costello (2005). The audience accounts form the basis of the design process. One of the strengths of these detailed, in depth accounts is in revealing the trajectory of the participants’ experiences, which include many common elements such as moments of uncertainty and realisation, moments of abandonment and absorption and key sticking points. There is a great deal of scope for further analysis of this data, including comparative experiences of different iterations, and comparisons with experiences of other artworks.

Weaving audience experience into the design process: Personas and Scenarios Our initial data gathering gave us a multilayered picture of the various ways that different people approached and interpreted the artwork. To integrate this knowledge into our ongoing design process we used Personas and Scenarios; imaginative, generative and flexible techniques for representing users in the design process, based around characterisation and narrative (Bodker 2000; Grudin 2002). Personas are composite characters, based on user research (such as our initial interviews) that summarise characteristics of the audience. They include rich amounts of detail including goals and motivation, knowledge and experience, physiological characteristics and sometimes social relationships. The key to their value lies in their power to mobilise observed characteristics in a believable and at times unexpected and living way. The usefulness of personas as a tool to support the design process of artworks has been piloted by Toni Robertson and her team in the Bystander Field Project (Kan 2005; Robertson 2005). We built our personas on models created by Robertson et al which were constructed from numerous observations of audience behaviour in a range of different interactive environments (Kan 2005), adding detail based on our own audience accounts. Scenarios are narratives in which personas conduct situated activities. They are used as a tool for revealing and solving design problems and possibilities through a focus on the user experience. They provide examples of action that can be useful at any stage of the design development lifecycle. The narrative structure provides an internal consistency and flow that creates a powerful representation of the real possibilities of user experience and action. We use scenarios as generative and collaborative thinking tools by writing in relay; first concentrating on the curatorial aspects of how the persona’s experience would be shaped until their physical encounter with the work, then continuing their experience through their interaction. In this way we integrate our own different perspectives into the scenarios – building a composite picture which reveals unexpected problems, including such factors as what aspects of the work they found

5

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

engaging, confusing or uninteresting, along with specific physiological data that could or could not be monitored by the work. Personas and scenarios form an ongoing strand of the process, continually being revisited, revised and developed at different stages of the project. They provide a way to reliably think beyond our own perspectives as makers, and to integrate our different areas of concentration into one cohesive account of an experience.

Working with peers It was clear from our initial data gathering that a large proportion of the audience for Cardiomorphologies had a professional interest in art, and often in experimental and new technology art. At first we had sought ways to avoid this, thinking that we needed rather to concentrate on “real” audiences. But we realised that in fact, avoiding this section of the audience would mean we were basing our design process on unrealistic and non-representative version of audience experience. We were also interested in the insights that might come from the experiences of this more specialised sub-group. One problem we foresaw was that specialists might only be able to give professional opinions of the work, which might not exactly reflect their actual experience. We decided to try and develop a collaborative tool that could enable us to work with representatives of this specialist group on a professional level but keeping as closely accountable as possible to their lived experience of the work.

Drawing together artistic aims and audience experience The tool we developed was adapted from the Future Workshops model (Kensing 1991), with most of the actual techniques created specifically for our needs. A group of six professionals working with art, interaction design and sound individually experienced Cardiomorphologies over the course of the day, and then joined together for a structured group discussion. We conceptualised the group discussion as a kind of conciliation – in which the actual experiences of the participants and the artistic experiential aims would be drawn together. We began with the generation of shared descriptive language. For this phase participants took it in turns to describe the experience of the work as it unfolded over the course of their interaction. They were asked to stick as closely as possible to what they did or felt at the time. The rest of the group listened and noted down on two different coloured stacks of paper key phrases that concurred with or contradicted their own experience of the work. They marked the phrases as either positive or negative. In the second stage we introduced a list of eight artistic experiential goals: 1) close fitting (reflecting as accurately as possible changes in physiology); 2) sensual and kinaesthetic (generating sensations of changing weight, motion and patterning within the body); 3) quiet, concentrated, inwardly attentive focus; 4) explorative, 5) simple/minimal aesthetic; 6) enabling (developing an ability to physically sense changes in heart rate/inter-beat-interval, and how these patterns interact with different mental/emotional states); 7) instructive (participants have a sense that they have learned something); 8) meaningful (provoking some consideration of mind-body processes). Participants were asked to assign words and phrases generated in the first stage to different experiential goals. The aim was to find out whether any of the experiential goals coincided with the participants’ own understanding of their experience. If so we were interested in what language they used to describe those experiences and what aspects of them were positive or negative. The categories “Quiet, Concentrated and Inward” and “Close Fit” were extremely meaningful for participants, the former attracting a large number of positive comments and the latter a mixed response. We judged that we had got the atmospheric approach to the work right, but that the visualisations and sound needed to reflect people’s physical feelings more closely and respond more exactly and rapidly to changes. On the other hand the experiential objectives “Enabling” and “Instructive” did not resonate with the participants. Although we considered the possibility that these goals may work over a longer period it was clear was that they were not being delivered in an observable way by the experience as it stood. Interestingly though our data suggested that they were being delivered by the research process surrounding the experience. The majority of the participants’ accounts – both from the peer discussion and the video-cued recall - showed that framing the audience approach and expectations to the work in the context of research actually helps people to orient themselves to the

6

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

work and to reflect on and deepen their engagement with it. The notion of framing the artwork more performatively as a kind of experiment in which the audience is invited to participate, suggests itself as a design response to this conclusion.

Evolving practice The experimental tools we have described above are steps in the early stages of the development of an iterative approach to making and showing interactive art. There is still a great deal of work to be done and potential for development. However the process we have initiated is gathering a momentum of its own which is beginning to generate unexpected directions, and we have reached a point where it is useful to reflect on the implications of our use of these tools both for Cardiomorphhologies as an artwork, and for our ongoing practices. Firstly it seems important to try and describe this sense of momentum. Throughout the process we have slowly been drawing myriad voices, accounts and perspectives into our thinking. Each account is surprising and idiosyncratic, and each reveals aspects of the work that we have not predicted or even conceived of. At the same time they do not pull away from one another. Without losing their individuality they form a tide, a cumulative force of possibilities, needs and enthusiasms, which sweeps the design forward. The design process has taken on a kind of evolutionary character, where contradictory accounts resolve into new and unforeseen directions. The personas and scenarios continue to develop and become richer; generating ideas, problems and solutions that seem to come from beyond our own thinking. As we move towards framing the artwork performatively as an experiment in which the audience is invited to participate, a process that began as a strand of associated research is becoming intertwined with the nature and character of the artwork itself. The momentum generated by working with audience experience is eroding the boundaries between research and practice, creative process and resulting artwork. The techniques we have borrowed from human-centred design have begun not only to inform but transform our creative practice. Studio practice has traditionally been conceived as a somewhat isolated process, unsullied by external demands and expectations of the audience. Artists working with interactive and physically responsive systems depend on maintaining close rapport with their audiences in ways that differ substantially from other more traditional art-forms. In our practices the principal object of considerations is not the art object per se, but a self-conscious appreciation of the behaviours/subjectivities that these objects bring forth in us as participants. In these cases artistic and curatorial practice begin to merge as the public exhibition becomes the principal site for the creation of the work. The iterative process we have developed for the creation of Cardiomorphologies was conceived and framed within the context of practice based research, but we have begun to question whether, beyond this context, we can evolve a kind of enquiry-based practice that continues to put codiscovery with audiences in a central position.

References Bodker, S. (2000). Scenarios in user-centred design - setting the stage for reflection and action. Interacting with computers. 13: 61-75. Costello, B., Muller, L., Amitani, S. and Edmonds, E. (2005). Understanding the Situated Experience of Interactive Art: Iamascope in Beta_space (forthcoming). Interactive Entertainment. Sydney, Australia. Crowther, P. (1993). Art and embodiment : from aesthetics to self-consciousness. Oxford, Clarendon Press, Oxford University Press: 210 p. Dourish, P. (2001). Where The Action Is. Cambridge, MA, MIT Press. Frieling, R. (2004). Reality / Mediality. Hybrid processes between art and life. Retrieved April 25th, 2005, from http://www.medienkunstnetz.de/themes/overview_of_media_art/performance/

7

Vital Signs: Creative Practice & New Media Now, 7 - 9 September 2005, Melbourne, Australia

J., Pruitt, John (2002). Personas, Participatory Design and Product Development: An Infrastructure for Engagement. Participatory Design Conference. Malmo, Sweeden: 144-161. Hook, K., Sengers, Phoebe, Andersson, Gerd (2003). Sense and Sensibility: Evaluation and Interactive Art. Conference on Human factors in computing systems. Ft. Lauderdale, Florida, USA, ACM Press, New York, NY, USA: 241 - 248. Kan, M., Robertson, T., Muller, L., Sadler, K. (2005). Designing a Movement-Based Interactive Experience Using Empirically Derived Personas and Scenarios. Critical Computing. Aarhus. Kensing, F., Halskov Madsen, Kim (1991). Generating Visions, Future Workshops and Metaphorical Design. Design at work : cooperative design of computer systems. M. Kyng. Hillsdale, N.J, L. Erlbaum Associates: 155-168. Lucier, A. (1965). Music for Solo Performer. USA, Rose Art Museum, Brandeis University. Moran, T. P. a. A., R.J (1990). The Workaday World as a Paradigm for CSCW Design. Conference on Computer Supported Cooperative Worrk. Los Angeles, California, ACM Press, New York: 381-393. Penny, S. (2004). Towards an Aesthetics of Behaviour. ISEA. Baltic Sea. Robertson, T. (2002). The Public Availability of Actions and Artefacts. Computer Supported Cooperative Work. 11: 299-316. Robertson, T., Mansfield, T., Loke, L. (2005). "Tools to Think With" - Critical Engagements in the Design of an Immersive Environment for Public Use (Forthcoming). Rosenboom, D., Ed. (1976). Biofeedback and the Arts: Results of Early Experiments. Vanouver, BC, CANADA, Aesthetic Research Cetre of Canada (A.R.C.). Sengers, P., Czikszentmihalyi, Chris (2003). HCI and the arts: a conflicted convergence? Conference on Human Factors in Computing Systems. Ft Lauderdale, Florida, USA, ACM Press, New York, NY, USA: 876-877. Svanaes, D. (1999). Understanding Interactivity: Steps to a Phenomenology of Human-Computer Interaction. Trondheim,. 2005: PhD Thesis. vom Lehn, D., Heath, C., Hindmarsh, J. (2001). Exhibiting Interaction: Conduct and Collaboration in Museums and Galleries. Symbolic Interaction. 24: 189-216.

8