Sensemaking as Narrative: Visualization for Collaboration Chris Baber, Dan Andrews, Tom Duffy, Richard McMaster School of Electronic, Electrical and Computer Engineering University of Birmingham Birmingham UK Contact:
[email protected] ABSTRACT
In this paper we explore the role of ‘narrative’ as a means of capturing the schemata that are essential to sensemaking practice. ‘Narrative’ offers a convenient structuring of information that is germane to Intelligence Analysis. Developing this notion further, it is proposed that techniques for Narrative Visualization can support collaboration in sensemaking activities. The paper proposes recommendations for the use of Narrative Visualization (and related applications) in the practice of collaborative sensemaking. Author Keywords
Sensemaking, Visualization.
Collaboration,
Narrative,
Narrative
ACM Classification Keywords
H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.
“In short, what is necessary for sensemaking is a good story.” [ [1] p. 60] SENSEMAKING AS NARRATIVE
The ideas presented in this paper developed from work on Crime Scene Examination. It was noted that the acquisition and sharing of information in the criminal investigation process often involved some form of ‘narrative’ [2]. The process of sensemaking is, in effect, the collation of stories, and the product of sensemaking is, in effect, the resulting narrative. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. VAW2011, September 2011, London Copyright 2011 ACM xxxx.
In this paper it is assumed that ‘stories’ might be the informal (i.e., ambiguous, unstructured, biased, incomplete) accounts that are provided by different stakeholders, and ‘narrative’ is the formal (i.e., structured, plausible, complete) account that constitutes an official or, at least, sharable, report. Taking this notion a little further, we use a four-level distinction between archetype, story, argumentation and narrative. Broadly, each stakeholder involved in an investigation (victim, eye-witness, investigator, analyst, manager etc.) creates a ‘story’ that reflects key entities from their point-of-view. Furthermore, some collections of entities might represent generic versions of a story, e.g., an ‘archetype’ for a domestic burglary might include point of entry, route followed through house, items stolen etc. A story that follows this archetype would, therefore, be expected to contain a given collection of entities (and this could serve as a prompt for gathering additional information relating to the entities). Some of the entities might be consistent across the stories, while others might be conflicting or ambiguous. A process of reconciling these inconsistencies (argumentation) could lead to an ‘agreed’ version, i.e., narrative. This leads to the proposal that there are two broad forms of visualization underlying these processes: (i.) visualization of the entities and their relationships that constitute different stories (and that could be used to explore variation across different stories), and (ii.) argumentation that constitutes the ‘testing’ of the strength between the different relationships. The focus of this paper is on the definition of the ‘entities’ and the visualization of stories; argumentation will be dealt with only in passing. “A story is a powerful abstraction used by intelligence analysts to conceptualize threats and understand patterns as part of the analytical process.” [3, p.19]. The notion that Intelligence Analysis might involve ‘stories’ might appear antithetical to the aims of producing coherent, objective reports of intelligence that can be used as the basis for action. However, such analysis often requires the ability to spot links between disparate pieces of intelligence, and this ability could be supported by the abilities involved in constructing stories and narratives.
SENSEMAKING AND SCHEMA
In their account of sensemaking, Pirolli and Russell [4] propose that contemporary theories can be divided into three broad perspectives. The first perspective involves the transformation of a representation, “…from its raw state into a form where expertise can apply…and then out to another form suited for communication.” [4, p.3-4]. In this account, the representation is typically ‘external’ to the analyst and working with the representation involves manipulating a physical artifact. The second perspective involves the mapping of data to frames. “Frames can be expressed in a variety of forms including stories, maps, organizational diagrams or scripts.” [4, p.5]. In this account, the representation is typically ‘internal’ and working with the representation involves manipulating mental models. The third perspective involves the collaborative search-after-meaning. This involves establishing common-ground between two parties and the compilation of information into a shared account. “Analysts ultimately ‘tell stories’ in their presentations.” [5, p. 100]. From this discussion, it is proposed that contemporary theories of sensemaking rely heavily on some notion of representation and, often, the ‘internal’ version is termed a schema. These notions of ‘schema’ provide a means of representing the knowledge, information, expectations etc. of people who are engaged in sensemaking. In broad terms, a ‘schema’ is a means of organizing information. While this use of schema might seem uncontroversial, it remains an open question as to what form the schema might take. While we accept that different domains of application might draw on different types of information that require different forms of organization, we propose that Intelligence Analysis can be considered in terms of a narrative. ENTITIES, SCHEMA AND NARRATIVE STRUCTURE
Early attempts at theories of how narratives were formed drew on Russian Folk Tales [6, 7]. These Formalist approaches identified archetypal plots and actants (characters). In this approach, the basic unit of the narrative is the Action or Event (defined in terms of its relevance to the plot). Characters are the stereotypical roles associated with the performance of the Actions or who are affected by the Events. The development of Plot, thus, becomes the elaboration of Relations (between Actants and Actants, between Actants and Objects, and between Actants and Actions or Events). In this Formalist approach, the narrative can be recounted in terms of the mapping of these relations. The Formalist approach has several problems associated with it. The first is that, because Actants fulfill stereotypical roles, there is little opportunity to consider motive. Thus, one development involved expanding upon the set of entities involved in defining narrative structure. For example, [8] offers a pentad of key terms: Act (what was done), Scene (where and when was it done), Agent (who did it), Agency (how did they do it), Purpose (why did they do it), and [9] elaborated on these as follows:
1. Agent (who did what) 2. Act-type (what did he or she do?) 3. Modality of action (how did he or she do it?) a. modality of manner b. modality of means 4. Setting of Action (in what context did he or she do it?) a. Temporal aspect b. Spatial aspect c. Circumstantial aspect 5. Rationale of action (why did he or she do it?) a. Causality (cause) b. Finality (aim) c. Intentionality (state of mind) The gathering of entities, however, only forms one part of the challenge of constructing narrative. The Formalists recognized the importance of defining the relationships between actants, objects, actions, events. We contend that key to Intelligence Analysis is the ability to explore the relationships between entities; indeed, we would suggest that (in the initial analyses at least) it is more important to produce credible and coherent relations than to focus on entities per se. Taking the entities listed above, we propose that there are definable relations between each pair of them, e.g., in terms of {capability, dependency, goal, impact, location, motive, objects, opportunity, options, performance, relations, sequence, state-of-mind}. Some of these relationships are much easier to visual than others, e.g., Location can be shown on a map, Sequence could be shown on a timeline, Relations could be shown in Social Network diagrams. While the other relations are more difficult to visualize, they are still vital to the ongoing narrative. A second problem with Formalism is that it limits the interpretation of the narrative to solely the information contained within it. Structuralist theorists [10, 11] recognized that the narrative not only presents formulaic relations but also provides opportunity for the reader to make certain inferences by bringing in their own knowledge, experiences and expectations. This recognizes that narratives inevitably have ‘gaps’, either because the author omits the banal or common-sensical, or because the author is withholding information to reveal later on, or because narrative is always concerned with ‘canonicity and breach’ [12], i.e., with the situating of interesting occurrences against stereotypical expectations of the world. Similiarly, the process of sensemaking could be triggered when you experience a ‘gap’ in your understanding of the current context [13, 14]. The context-as-experienced could conform to an archetype, i.e., canonicity, or it could result in some gap, i.e., breach, of these expectations. The breach, then, motivates the search-after-meaning. This breach involves an initial response of gathering information that forms the basis of a story, which is then finessed into a narrative.
Thus, we propose that the Intelligence Analysis maintains a ‘gist’ (to keep track of many pieces of information) and manages ‘goals’ in this reading. “If accuracy is nice but not necessary in sensemaking, then what is necessary? The answer is something that preserves plausibility and coherence, something that is reasonable and memorable, something that resonates with other people, something that can be contrasted retrospectively but also can be used prospectively, something that captures both feeling and thought, something that allows for embellishment to fit current oddities, something that is fun to contrast. In short, what is necessary in sensemaking is a good story.” [1, p. 60-61, emphasis added]. VISUALISING NARRATIVE WITH STORY GRAMMAR
“…collaboration is made easier if it is organized around entities [people, places, things, times etc.] rather than freeform text.” [3, p.99]. From the ongoing discussion, we propose that a story’s structure creates the ‘slots’ in which specific entities can be placed. If specific values of entities are not known, then archetypal values can be used in their place. The story’s structure also incorporates generic relationships between entities. Each ‘story’ could emphasize different aspects of these relationships, depending on the perspective of the teller or the sources on which the story is developed. An approach to exploring these relationships is through the concept of a ‘story grammar’ [15], which offers a convenient means of combining entities. A ‘story grammar’ can be presented in terms of a hierarchy of story elements. A top-level element, such as ‘Story’, is built from () combinations of low-level elements (Setting + Theme + Plot + Resolution). Table 1 shows how these map ont to the entities proposed above. We posit two possible approaches to sensemaking using such levels within a grammar, which have been defined as ‘Bottom-Up’ and ‘Top-Down’. A ‘Bottom-Up’ approach would result in stories being created from information available at the higher levels, and is potentially the least subjective, as the story should be determined by the information available. An example of this approach would be an investigation in which the evidence available from a series of crime scenes helps determine the general pattern(s) of a criminal’s behaviour. In our terms, this uses the Archetype to specify the entities that will be required to explain an event, and the likely relationships between these entities (based on previous experiences). One disadvantage of this approach, however, is that ‘gaps’ in the information available could adversely affect the construction of the story. In such situations, a ‘Top-Down’ approach may facilitate sensemaking, as it allows for gaps in information by allowing the user to test hypotheses against whatever supporting evidence is available. The main disadvantage of this approach, of course, is that it is subjective, and there is a danger of overlooking details within the story if evidence to support an incorrect narrative is pursued instead.
Table 1: Defining Relations in Story Grammars [16] Thorndyke’s Grammar
Mapping to Entities
Story Setting + Theme + Plot + Resolution
Story Setting + Theme (archetype) + Plot + Resolution
Setting Characters + Location + Time
Setting 1 + 4 {a-c}
Theme (Event) + Goal
Theme 5a + 5b
Plot Episode
Plot Episode
Episode Subgoal + Attempt + Outcome
Episode 5b + 2 + 3
Providing the visualization with more dimensions (in terms of illustrating relationships between the entities in the narrative) can accommodate more stories (i.e., alternative readings of the narrative) which provide a basis for negotiation and collaboration as part of the sensemaking process. This comes at a cost in that the complexity of multiple dimensions increases the challenge of initial comprehension. Our proposal is that providing a grammar not only provides the key for the reading of the visualization, but also allows for the recognition of those entities which are not directly represented. We consider two very different approaches to the challenge of visualizing a narrative. The Hyper-Comic [17] allows readers to navigate the telling of a character’s actions, and to move from a linear track to explore motives and background of characters or digressions in the story (figure 1). Grammar
‘Hyper-comic’
Story Setting + Theme + Plot + Resolution
Setting Characters + Location + Time Theme (Event)* + Goal Plot Episode Episode Subgoal + Attempt + Outcome
Figure 1: Hyper-Comic and Story Grammar
In contrast, one of the best developed approaches to visualizing Intelligence is GeoTime Stories [5]. This provides combinations of visualizations to overly different material to reveal patterns for the analyst (figure 2).
ACKNOWLEDGEMENTS
The work in this paper is supported through an award from the Centre for Defence Enterprise [DSTLX1000054441] REFERENCES
Grammar
GeoTime Stories
Story Setting + Theme + Plot + Resolution
Setting Characters + Location + Time Theme (Event)* + Goal Plot Episode
Episode Subgoal + Attempt + Outcome
Figure 2: GeoTime Stories and Story Grammar CONCLUSIONS
This paper presents three proposals: i.
sensemaking involves developing a narrative built through the combination of a set of stories;
ii.
collaborative sensemaking involves the creation of artifacts to allow these stories to be visualized and shared between collaborators;
iii.
visualizations can be ‘read’ in terms of a grammar that highlights relationships between entities.
There are a variety of ‘genres’ of narrative visualization [18]. Even with visualization that captures all relationships between entities, there is a need for an approach that allows the key aspects to be appreciated. We propose that such an approach involves defining a narrative to allow reading of the visualization.
1. Weick, K.A. (1995) Sensemaking in Organizations, London: Sage 2. Baber, C., Smith, P., Cross, J., Hunter, J and McMaster, R. (2006) Crime scene investigation as distributed cognition, Pragmatics and Cognition, 14, 357-385. 3. Bier, E.A., Card, S.K. and Bodnar, J.W. (2008) Entitybased collaboration tools for intelligence analysis, IEEE Symposium on Visual Analytics Science and Technology, 99-106 4. Pirolli, P. and Russell, D.M. (2011) Introduction to this special issue on sensemaking, Human-Computer Interaction, 26, 1-8 5. Eccles, R., Kapler, T., Harper, R. and Wright, W. (2007) Stories in GeoTime, IEEE Symposium on Visual Analytics Science and Technology, 19-26 6. Propp, V. (1968) The Morphology of the Folktale, Austin, TX: University of Texas Press 7. Greimas, A-J. (1983) Structural Semantics: an attempt at a method, Lincoln, NE: University of Nebraska Press 8. Burke, K. (1969) A Grammar of Motives, Berkeley, CA: University of California Press 9. Rescher, N. (1966) Aspects of action, In N. Rescher (ed.) The Logic of Decision and Action, Pittsburgh, PA: University of Pittsburgh Press, 215-219 10. Barthes, R. (1977) Introduction to the structural analysis of narratives, Image Music Text New York: Hill and Wang, 79-124 11. Todorov, T. (1990) Genres in Discourse, Cambridge: Cambridge University Press 12. Bruner, J. (1991) The narrative construction of reality, Critical Inquiry, 18, 1-21 13. Dervin, B. (1998) Sense-making theory and practice: an overview of user interests in knowledge seeking and use, Journal of Knowledge Management, 2, 36-46 14. Bjørking, K. (2010) Sensemaking in Practice: reducing and amplifying equivocality through storytelling, OLKC 2010, 15.Thorndyke, P. (1977) Cognitive structures in comprehension and memory of narrative discourse, Cognitive Psychology, 9, 77-110 16. Andrews, D. & Baber, C. (2009) A Discussion of Using Narrative to Inform the Development of Serious Games for Criminal Investigation and Community Policing, CGAMES, Louisville, USA, 29th July-2nd August 17. http://e-merl.com/pocom.htm 18. Segal, E. and Heer, J. (2010) Narrative visualization: telling stories with data, IEEE Transactions on Visualization and Computer Graphics, 6, 1139-1148
