Interactive Documentaries Modes of interaction as crucial factor for the expansion of cognitive experience to physical, emotional and social engagement Anna Wiehl University of Bayreuth Bayreuth, Germany
[email protected]
Abstract The linear filmic documentary genre has developed differentiated strategies of representing, ordering and negotiating 'reality'; 'New Media' offer interaction, collaboration, navigation and participation. Interactive documentaries (idocs) combine 'the best of both'. Thus they expand the primarily cognitive understanding of the world 'represented' to physical, emotional and social engagement. This shift, however, challenges basic assumptions of documentary-making – especially the notion of 'authorship.' The aim of this paper is to give an overview over the spectrum of major options of interaction currently employed in idocs. Combining Nichols' modes of representation with theories of interaction, it proposes a matrix that allows us to analyze the implications of different strategies as to the role of the 'author', the degree of participation of the 'viewer/user', and the potential of idoc as a revealing, preserving, interrogating and experience-shaping genre.
1
Introduction: 'i-what?' – or: 'interactive documentaries' as a fuzzy emerging genre
Interactive documentaries (idocs), a hypermedia, most often multiplatform genre that first appeared in French-Canadian mediascapes, can be described as rather 'fuzzy': Still lacking academic consideration we are missing clear definitions, operative systems of classification and analysis. From documentary platforms in the internet and educational CD-ROM material,
browsable archives and wikimentary to serious games, G4C, history games and locative media; from non-fiction database narratives, web-reportage to participatory social community projects, multimedia art and AR-museum guides – the label 'idoc' is applied to a vast spectrum of genres that somehow use digital, interactive, so-called 'New Media' to make account of 'reality'. So – what artifacts are we dealing with when we are speaking of 'interactive documentaries' in the following? Two factors are – in our opinion – crucial in qualifying an artifact as an 'idoc':1 First of all, digital technology must be employed in a way to incite some kind of interaction between the 'author', the 'user'2 and the 'artifact'. And secondly, the artifact must intend to document a factual or emotional situation – i.e. establish a more or less indexical relation with 'reality'.
2
Theoretical Background – Or: 'Traditional Documentary' meets 'New Media''
2.1
Essential logics of linear documentary: Nichols' 'modes of representation'
The (filmic) documentary genre has developed differentiated ways to 'represent' reality. Bill Nichols, one of the founding-fathers of documentary theory, describes these different strategies "of organizing texts (i.e. documentaries) in relation to certain recurrent features or conventions" by distinguishing between six basic 'modes of representation':3 poetic mode: emphasizes visual associations, tonal or rhythmic qualities; expository mode: emphasizes verbal commentary and an argumentative logic; observational mode: emphasizes a direct engagement with the everyday life of subjects as observed by an unobtrusive camera; participatory mode: emphasizes the interaction between filmmaker and subject;
1
I am hereby concordant with Sandra Gaudenzi who applied comparable categories in order to compile a corpus for her research on idocs as 'Living Documentary'. Cf. Gaud2012. 2 The terms 'author' and 'user' (respectively viewer) must be read as a kind of wild-cards. As we will see later, the interactive dimension of idocs leads to a fusion of some of these entities, enabling former (more or less) passive 'viewers'/'consumers' to become active 'prosumers' and 'co-authors'. 3 In Representing Reality (1991), Nichols unfolds only four 'modes'. In Introduction to Documentary (2001), he elaborates this basic concept further and distinguishes six modes (cf. Nich1991, pp. 33-34).
reflexive mode: calls attention to the assumptions and conventions that govern documentary filmmaking; performative mode: emphasizes the subjective aspect of the filmmaker's own engagement with the subject; These 'modes' (and their actual combination) provide a convincing framework when dealing with two of the three main 'agents' in the documentary experience: the 'author' (i.e. filmmaker) and the filmic 'text' – and respectively their relation to the 'reality' observed and documented. Yet Nichols' modes of representation are rather vague as to one 'player' in this 'game': Where is the 'viewer' – and what is his part in this system? 2.2
Essential logics of 'New Media': interactivity and participation
The 'viewer' respectively 'user' becomes decisive when dealing with 'New Media' – and thus when studying idocs, as interaction and interactivity4 can be considered as essential features5 that distinguish 'Old' and 'New Media' – hence inducing new cultural practices6 that have been called 'Convergence' and 'Participatory Culture'.7 Henry Jenkins first used this term to contrast this new (?) logic with older notions of media spectatorship: "In this emerging media system, what might traditionally be understood as media producers and consumers are transformed into participants who are expected to interact with each other according to a new set of rules which none of us fully understands." Although Aarseth's (1997) classification of the 'functions of the user' may not lead to a 'full understanding' of the rules of participation, they provide a cogent system of categorization: explorative function: the user decides which path to take within preset options; role-playing function: the user assumes strategic responsibility for a character in a world described by the text;
4
Interaction' in our understanding has both a technological and a psychological dimension. Operationally, it is composed of three principal entities: properties of technology, attributes of communication contexts, and the user. Cf. Kiou2002. 5 Most 'New Media' theorists such as Manovich, Bolter, Grusin and Jenkins unanimously consider interactivity, hypermediality, infinitivity, variability and the ability to create sensory-rich virtual environments as decisive for the logic of 'New Media'. 6 'Cultural practices' is to be understood as an umbrella term covering the whole range of industrial, social, political, institutional, economic, … practices that are involved when dealing with the production, distribution, reception and consumption of (hyper-)media artifacts. 7 "Convergence is a word that manages to describe technological, industrial, cultural, and social changes, depending on who's speaking and what they think they are talking about". (Jenkins, online; italics by AW)
configurative function: the user can create or design part of the narrative; poetic function: the user's actions, dialogue or design are aesthetically motivated.8 2.3
Towards a matrix of 'modes of interaction' – or: 'Blending the Best of Both'
Nichols' 'modes of representation' designed for describing linear documentary (and thus mainly considering the two entities 'author' and 'text') provide obvious congruities with Aarseth's 'user's function' in 'New Media' – employing even similar terminology (e.g. as to poetics). Thus these concepts are almost predestined to build the framework of an operable analysis model that encompasses all 'players' in the idoc experiences and their interdependencies by considering basic 'modes to interaction'.9 Conversational mode: the interaction follows the rules 'conversation' (sensu Lipman) being characterized by smooth transitions, limited look-ahead, interruptablity and the impression of infinite database; Hypertext mode: the interaction with the artifact is like hitchhiking through a branching structure of fixed lexias; Participative mode: through the interaction, the materials of the artifact evolve like in building a house, everybody adding an item; Experiential mode: through the live interaction between artifact, user/prosumer and/or author a new kind of experience is created.
3
Matrix of modes interactions and exemplifying case studies – or: Tour d'horizon through the broad spectrum of the idoc-universe
The following table presents a matrix that allows us to analyze how modes of interaction interdepend with modes of representation, the role of the author as well as the role of the user. Three things are especially remarkable:
8
Cf. Aars2001. As a point of departure (that we will later develop further) we combine Ryan's reflections on "Layers of Interactivity in Digital Narrative Texts" (2005) and Gaudenzi's terminology in "The Living Documentary" (2012). Cf. Ryan2011; Gaud2012. 9
First of all, with the amplification of complexity of the artifact (i.e. its (sensory) 'richness', the possibilities of (social) interaction and the spectrum of the different functions the user assumes), the degree of participation rises; Secondly, the increasing degree of participation incites a blurring of the role of the author and the user who become inter-actors, collaborators and co-creators; And thirdly, the amplification of possibilities of interaction and participation stimulates the expansion of the former primarily cognitive 'documentary experience' towards physical, emotional and social engagement. basic mode of interaction
conversational mode
basic mode of representation // means of realization expository observational
primary role of the 'author'
primary function of the user
create a world create rules ± control
role playing configurative control avatar choose among offered options
= "conversation" // (with closed 'idoc- ± 'serious game' universe') using avatars for immersion hypertext mode
poetic expository
create possible paths
explorative
// database hypertext on DVD or one site
within a closed database ± control
within in the 'idoc-universe' control choose
PLUS
PLUS
PLUS
examples
Collapsus
= "hitchhiking"
catalyze possible within the infinite further research universe of the Internet and interaction forums & social in the infinite control media platforms Internet choose collaborate
Journey to the End of Coal
------------
hypertext of the infinite Internet
Prison Valley
participative mode = "building"
participatory poetic reflexive performative
create conceptual framework
explorative configurative
catalyze participa- contribute existing // tion and collabo- and/or user generinternet ration ated content social media The Echo Chamcollaborate archive of the site ber Project (linear) doc-film DVD experiential mode = "living"
performative participatory poetic
// locative media AR
database hypertext archiving tools (cloud pad)
curate content create conceptual explorative framework for role playing conexperiences in a figurative dynamic envipoetic ronment choose catalyze control (inter-) action & create participation contribute collaborate curate content
Rider Spoke
Riders have Spoken
Table 1: Matrix of different modes of interaction
An exemplifying tour d'horizon through the broad spectrum of idocs illustrates these theoretical assumptions: Collapsus can be considered as a typical example of the conversational mode: This idoc, which shares many characteristics with 'serious game' or G4C (e.g. the use of avatars), is set in a dystopian hypothetical year 2012 that is heading towards a global energy crisis. The multi-linear semi-fictional docu-narrative is told/played/experienced from the perspective of ten young people who become involved in a net of lobbyism and conspiracies. The highly immersive multimedia project mixes animation, features of expository linear documentary (e.g. 'real' interviews with scientists, lobbyists and politicians) and fiction (e.g. faked news websites) in a choicebased interactive narrative. However, almost all auctorial control rests on the side of the designers: They create a 'playable' 'idoc-universe' and the rules that reign in this finite virtual space. Nonetheless, the immersive, emotionally involving experiences within this virtual environment
can have an impact on the user's decisions and actions in the 'real' physical world: By gaining insight into the issues at stake (i.e. the exploitation of natural resources and the fatal consequences if these existential goods become prone to profit-maximizing speculation) the users might change their own (everyday) behaviors and raise their voices as citizens in the elections.
Journey to the End of Coal can be characterized as a mainly hypertext-mode based web documentary. The 'story' about the misery of Chinese coal miners unfolds in a mixture of factual, expository and partly poetic mode. Via 300 photographs, three hours of audiovisual material and 10 hours of sound materials (interviews with miners but also immersive, atmospheric audios), the users navigate at their own pace through a finite, but extensive database of 'bits of information'. Choosing, for example, between different possible interview-questions, they conduct 'their own' conversations with miners in preset situations. As was the case with Collapsus, it is the author who controls the main direction of the narrative flow since he creates a (limited) number of possible paths through the material. However, by enabling the user to actively push the narration forward and to self-reflexively reconsider her relation with the people she 'encounters' and interviews, she becomes a (simulated) co-author and co-actor within the narrative. This does not only stimulate immersion but also enriches the experience by promoting aspects of Nichol's performative and participatory mode – i.e. calling attention to the conventions of interview-making in documentary filmmaking and to the filmmaker's own commitment with the subject.
In Prison Valley, the documentary experience is further amplified: At first sight, this idoc about the US 'prison industry' is mainly based on the hypertext-mode. Guided by a rather strong authorial 'narrative voice', the user 'hitchhikes' through linear video-sequences in which she 'meets' different parties concerned: young prisoners, a sheriff, journalists and prison guards. At this level – within the closed space of the idoc-narrative – the user's agency is only minimal. Yet this expository and observational mode of information is supplemented by options of interactivity and participation: Links on the official website invite the user to access a chat room and a forum. There, she cannot only exchange her experiences when 'surfing' the idoc; moreover, she can get into contact with the characters of the idoc and express and document her opinion about the issues negotiated. Hence – at this level – Prison Valley can be considered as a collaborative
project, expanding the primarily knowledge-driven, ontological documentary experience to (social) interaction and exchange.10
This collaborative potential of idocs is further expanded in the participative mode. One of the most complex and ambitious examples of this mode of interaction is The Echo Chamber Project. The open source, investigative documentary site wants to sensitize its audience to how news media became an uncritical echo chamber11 when covering the political decision-making process that led to the Iraq War 2003. This intention is persecuted by different options of interaction and different degrees of agency and participation. At the 'lowest level' of inveracity, the viewer merely browses the website's archive of audiovisual news-bits and (probably) makes up her personal opinion on the issue. Hence, apart from hitchhiking through the archive and 'interpreting' the videos, she stays rather passive. The possibility of commenting on the content of the statements documented, on the material as such, as well as on other participants' contributions via blogs or vlogs offers a more active participation. The 'highest level' of interaction and participation is reached when the 'consumer' joins the creative production cycle: The site enables the user to become co-author of this collaborative opensource project. She is invited to add her own 'found footage' (i.e. audiovisual news material covering the Iraq War) or even create content herself – e.g. by interviewing experts or people concerned. This vast corpus of audiovisual material is finally to be assembled to a collaborative, expository and participatory (linear) documentary film that nevertheless bears the traces of the collaborative film-making procedure:12 One of the major aims of the innovative project is to bring a "broader range of voices and perspectives into the mainstream media"13 and to discard the eponymic echo-chamber effect. Therefore, the process of conceptualization, writing, editing and post-production is transparently documented and – again – commented on. This permanent re-
10
Apart from that, Prison Valley is paradigmatic of the emergence of multi-platform ventures and cross-media projects: The launch of the web-based interactive documentary (a cooperation of arte with upian) was accompanied by a book, the screening of a one-hour linear documentary film-version on television (arte), an exhibition in Paris and an iphone app featuring the introductive video of the doc as well as the possibility of browsing audiovisual material of the idoc and of following flicker 'conversation' and tweets centered on the topic. 11 The echo chamber effect describes a situation in which opinions, beliefs and (sometimes false) information are reinforced as they are constantly repeated or 'echoed' – especially in mainstream media. Such unreflected repetition hinders critical discourse and autonomous, independent thought. 12 The project is still ongoing. Therefore, an encompassing evaluation would be precipitate. 13 Kent2013.
consideration and transparent discussion adds a reflexive and performative dimension to the whole project. In summary, the formerly strict borders between authorial producer/author/filmmaker and rather passive, consuming/interpreting user/consumer/viewer are challenged: The user becomes cocreator, being at par with the 'author' who designs conceptual frameworks of the site, catalyzes collaboration and gives impulses to curate contributions; however, she remains only one enactor of many in the creative collaborative documentary-making process. Thus, The Echo Chamber Project represents a pioneering type of collaborative idoc in which the increase of the prosumer's agency leads to the expansion of her virtual cognitive experience to personal engagement with a socially relevant issue; it incites a self-reflexive reconsideration of her individual sense-making when dealing with documentaries and factual news-programs and it induces a reexamination of collaborative sense-making processes in media society.
This self-reflexive and interrogatory capacity of idocs brings us the most complex type of idocs: Within the spectrum of the existing documentary genre, experiential idocs are probably the most multifaceted representatives. First of all, they combine all four modes of representation (performative, participatory, poetic, (self-)reflexive); moreover, they are almost always designed as multi-platform, multi-media projects, employing different devices – generally including locative media (e.g. smartphone, GPS, GIS, google-maps, …). Thus they are able to create hybrid spaces of virtual and physical reality – which alone already expands the 'space of experience'. And finally, they are most often conceptualized to inspire collective experiences – thus engaging social exchange and collaborative sense-making. In Rider Spoke, a mixture of locative game, emotional map and archive of mixed reality performances, the user rides her bike in the center of any city, equipped with a smartphone or WiFi mobile device. From time to time, she is encouraged to search for a 'hiding place' and insert a 'pin' in the virtual map her device shows. Then she is asked to answer a prerecorded question – e.g. to describe her awareness of her surroundings, her feelings at the present moment or also recall memories and associations with the chosen locations and – finally – to make a rather intimate avowal as to her future dreams 'triggered' by her trip. All these statements can be recorded. If the user decides to share them with others, her recordings will be organized in a (curated)
archive and reassembled in a kind of emotional map of the city.14 Thus, whenever a rider during her random tour crosses some other 'Rider's' route or 'hiding place', the device notifies her and offers her to listen to other participants' experiences. Accordingly – as to the role of the author respectively of the user – we are witnessing a fundamental paradigm shift: Apart from the general conceptual framework of the artifact (that is designed to be as 'open' as possible because it has to be applicable to very different and highly dynamic surroundings), the 'author' leaves almost all agency to the 'users'. Hence, those become the 'main' authors of this idoc, exploring their surroundings, creating lexias and – if they 'meet' other 'authors' – collaborate with them. Accordingly, Rider Spoke consists of multiple independent layers of representation, interaction and participation: First of all, it blurs public, physical space with a virtually augmented dimension of experience; secondly, it combines introspective personal awareness of space with the 'documented' experiences of other co-authors and the real-time interaction with bustling public space. And last but not least, it combines a diachronic axis of experience (i.e. different layers of previously documented experiences) with her synchronic involvement (her live, real-time awareness at the present moment). Thus, the pluri-voiced, multi-dimensional documentation and negotiation of reality exploits the participatory, collaborative, experiential and interactive possibilities of idocs and their potential to become a reality-shaping experience at its fullest.
4
Summary and Outlook – Or: Challenges and potential of idocs as a reality-shaping experience
As we have seen, the use of the options of interaction, collaboration and participation offered by 'New Media' are predestined to expand the primarily cognitive 'documentary experience' to physical, emotional and social engagement. At the same time, these possibilities call into question some basic assumptions of documentary making– especially the notion of 'authorship'. In linear documentaries, the filmmaker was (more or less) in control of the finite, closed, teleolog-
14
Moreover, an interdisciplinary research team uses this archive and the adjoined project Riders Have Spoken to solve the technical as well as conceptual, economic and aesthetic challenges involved in recording, remediation and replaying these lexias of interactive experiences. And last but not least, being part of the CREATive Organisations Research, scholars and creatives explore in how far such interactive projects enable new creative practices and can serve as databases for collaborative, interdisciplinary research.
ical and intentionally directed artifact. With the increase of user-agency, interaction, participation
and
collaboration
that
culminates
in
the
blurring
of
author
and
view-
er/user/prosumer/interactor, we face similar jeopardies that already challenged interactive narrative – namely the so called 'interactive paradox'. In fact, especially highly participative and collaborative idocs run the risk of ending up as randomly meandering, exhausting stories lacking an Aristotelian curve of rise and fall in tension. One essential piece of the puzzle to solve this paradigmatic paradox consists in the development of a well-considered technological, aesthetic and conceptual framework – and at this juncture, the main responsibility to catalyze interaction and participation and to curate the collaboratively produced content still remains with the author. As to these essential frameworks, it is crucial to consider the complexity of the issue/subject negotiated, the (situational) context in which the idoc is to be situated and the addressees' prior knowledge as well as motivation and willingness to actively participate: If prospective users are either well-versed or extremely avid to learn more and at the same time highly motivated to actively participate in collaborative processes, the framework as well as the architecture might be more complex and 'demanding'; if the addressees and potential users still have to be encouraged to engage with the idoc or if they are primarily seeking experience, more immersive, semifictional, playful approaches such as applied in Collapsus or Rider Spoke's 'hide&seek' might be more advisable to reach the audience; and if the probable users are semi-amateur filmmakers, (web-)activists or (web-)journalists etc., the affordances of interaction/participation and the (co)authorial responsibilities and competences might be rather encompassing as it is the case in The Echo Chamber Room Project. Above all, one should keep in mind that different modes of interaction and representation are not mutually exclusive as we have seen in Prison Valley. Maybe, it is exactly the well-balanced combination of diverse platforms, of different mediadevices and complementary modes of interaction and representation that solves the paradox – i.e. for example the combination of primarily information-led linear sequences featuring a rather strong authorial voice within the narrative framework with highly self-reflexive, interactive, participatory elements situated on social platforms or within physical 'reality' by employing locative media. Although such exceedingly complex idocs are still the exception to the rule, one hypothesis can be retained: Due to their hypermedial richness and depth PLUS the possibility of including multiple voices and alternative points of views PLUS the option of instant feedback, participation
and collaboration, digital artifacts such as idocs are extremely suited to encourage approaches to reality as being subjective, plurivocal and dynamic. Therefore, they deserve further attention as a highly potential revealing, preserving, interrogating, experience-shaping and transformative emerging genre.
Literature [Aars2001]
Aarseth, J. E.: Cybertext, perspectives on ergodic literature. The Johns Hopkins Press, Baltimore 2001.
[Gaud2012]
Gaudenzi, Sandra: The Living Documentary. From representing to co-creating reality in digital interactive documentary. Dissertationsentwurf 2012.
[Jenk2013]
Jenkins,
Henry:
"Welcome
to
Convergence
Culture".
Online:
http://henryjenkins.org/2006/06/welcome_to_convergence_culture.html; 19.06.2006; 2903.2013. [Kent2013]
Bye,
Kent:
Welcome
to
The
Echo
Chamber
Project.
Online:
http://www.echochamberproject.com/; 29.03.2013. [Kiou2002]
Kiousis, Spiro: Interactivity: a concept explication. In: Sage Publications 4 (3), 2002, S. 355–383.
[Mano2001] Manovich, Lev: The Language of New Media. The MIT Press, Cambridge, Massachusetts; London 2001. [Nich1991]
Nichols, Bill: Introduction to Documentary. Indiana University Press, Bloomington, Indian 1991.
[Ryan2011]
Ryan, Marie-Laure: Peeling the Onion. Layers of User Participation in Digital Narrative Texts. In: Page, R.et.al. (Hrsg.): New Narratives: Stories and Storytelling in the Digital Age. University of Nebraska Press, Lincoln 2011, S. 35-62.
Web-Material - idocs:
Bye,
Kent:
The
Echo
Chamber
Project.
2006.
Online:
2007.
Online:
http://www.echochamberproject.com/node; 29.03.2013. Brady,
Hannah:
Rider
Spoke.
Blast
Theory
http://blasttheory.co.uk/bt/work_rider_spoke.html; 29.03.2013. Dressen, Arnaud; Bollendorff, Samuel; Ségrétin, Abel: Journey to the End of Coal. honkytonk,
curiosphere.TV
2007.
Online:
http://www.honkytonk.fr/index.php/thebigissue/; 29.03.2013. Dufresne, David; Brault, Philippe: Prison Valley. arte TV und Upian, 2010. Online: http://prisonvalley.arte.tv/?lang=en/; 29.03.2013. Pallotta, Tommy: Collapsus. submarine 2008. Online: http://www.collapsus.com/; 29.03.2013.
