Collaborative Sensemaking Tools for Task Forces

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 53rd ANNUAL MEETING—2009

439

Collaborative Sensemaking Tools for Task Forces Eric A. Bier, Dorrit Billman, Kyle Dent, Stuart K. Card Palo Alto Research Center, Inc., Palo Alto, CA 94304

10.1518/107118109X12524441081901

{bier, billman, kdent, card } @parc.com Our work addresses the needs of multiple information workers collaborating on joint projects, which typically require finding, analyzing, and synthesizing information from heterogeneous sources. We report on iterative design, implementation, and assessment of collaborative tools for sensemaking tasks. Our goal is flexible, lightweight tools that both facilitate the activities done individually and lower the costs of effective collaboration. We suggest several approaches to enhance such collaborative sensemaking tools. These approaches include explicit representation of multiple team activities, integrated support for synchronous communication, and views of collected information that are tuned to both the reading and organizing phases of sensemaking. We present an integrated pair of tools, ContextBar and ContextBook, which illustrate these approaches, and describe the results from a formative evaluation of these tools. information. The Rooms system exploited this fact to create a INTRODUCTION virtual multi-desktop workspace reducing contention for screen space by displaying only task-relevant windows Much contemporary knowledge work involves (Henderson et al). The interface design lead the user to define collaboratively gathering and interpreting information that is and shift explicitly among activities. We can use this idea needed for a team project or activity. Such sensemaking work during collaborative work by associating a view of may include using search engines, browsing file systems and information with an activity or task, or indeed with any human repositories, reading documents, taking notes, adding context that conditions the probability of use of information. annotations, categorizing documents, sharing document In this paper, we report on a pair of tools, ContextBar information with co-workers, tracking activity status, and and ContextBook, for aiding the collaborative collection and communicating with co-workers about the activity. organization of information by a knowledge task force. In organizations, information foraging and sensemaking ContextBar supports teams in the process of finding and are often part of the activity of a task force, a set of individuals collecting documents. ContextBook supports teams in who come together for a limited time for a specific purpose. organizing the document content that has been found. The But task force-oriented collaborative knowledge work also tools allow users to tag, annotate, and organize documents occurs in more informal settings: family members trying to relative to a named task context. understand a disease, a debate team preparing a case, a group We have designed these tools to embody several of friends planning a vacation, or staff members assessing principles. (1) Easy transition from existing work practice: our ways to reduce lighting costs. The work may involve foraging tools should work alongside existing knowledge-worker tools, for information, organizing, re-representing and interpreting rather than replace them. This is a key factor in adoption. (2) information, and creating a knowledge product. Task force Efficient companion: our tools should be available throughout knowledge work is a natural application for recent social much of the sensemaking activity, both when an individual networking technologies, which have made it much easier to document is read and when information about multiple form ad hoc groups and to collaboratively share and structure documents is organized. (3) Continuous collaboration: information (Shirky, 2008). support collaboration throughout the activity and through Many current systems (Del.icio.us, 2009; SparTag.us, changes in collaborative configuration. Ideally, the same Hong, Chi, Budiu, Pirolli, & Nelson 2000) support system can be used effectively working alone or in a group. collaborative foraging by letting users tag information with (4) Individual privacy: the tools give team members control relevant topics. These systems focus on the structure of over information sharing. (5) Generalized annotation: the information rather than the structure of the activity to which tools leverage commonalities between the popular concepts of the information is relevant. Attempts to inject activity tagging, annotating, bookmarking, and note-taking. Userrelevance, for example by using personal activity tags, can added comments play a variety of roles in sensemaking and result in semantic pollution of the information space for others are bound to a variety of entities. (6) Task-based structure: (Golder & Huberman, 2006) and make it difficult to separate take advantage of the association between gathered process from content. information and specific teams and tasks. System information However, there is evidence that conditioning information about task organization can reduce costs of task interruption, on task structure can be effective. On a computer desktop, use multi-tasking, and group coordination and help the system of information follows a locality of reference pattern, in which deliver the right (task-relevant) information at the right time. most references are to recent information and the references Below, we describe the tools we have built, report on a are clustered by activities that often correlate with a task or formative evaluation of the technology, and discuss our design project (Henderson, Austin, & Card, 1986). Thus, while the principles for collaborative sensemaking tools. user works on the activity, the probability of her wanting to access activity-related information increases relative to other Downloaded from pro.sagepub.com at NASA AMES RESEARCH CENTER on January 8, 2016

This work is not subject to U.S. copyright restrictions.


CONTEXTBAR & CONTEXTBOOK

ContextBar and ContextBook are integrated tools supporting individual and collaborative work. Figure 1 shows both tools and an active document. ContextBar supports the user in reading and commenting on documents. We call it a per-document tool. ContextBook displays information about all of the documents the team has found in a task and allows that information to be organized. We call it a per-task tool. The two tools communicate with each other, placing little burden on the user to move information from one to the other. Information gathered in ContextBar can be viewed and organized in ContextBook and vice-versa. ContextBar displays as a vertical column containing an extensible set of gadgets that provide different functions and information. Supporting our goal of easy transition from existing work practice, it works with existing document applications, including Web browsers, Microsoft Office™ tools, email readers, and Adobe™ PDF tools. It connects to the currently active application window in two ways. First, it positions itself (docks itself) next to the window. It can move from a first application to a second when the second application becomes the active one. Second, it changes its display (syncs itself) to show information relevant to the document in the active application. The display may change either as a result of a change of active application or as the result of a change of document within the application. Turning to the gadgets within the ContextBar, the Task Gadget (near the top of Figure 2) allows the user to view and select the current task (activity and team), thus supporting our principle of task-based structure. Existing task names can be

Figure 2. The Task Gadget and Annotation Gadget

440

selected from a pull-down menu. New task names can be added as needed. While task selection does take a little time, it has several benefits. Because each task is associated with a

particular team of users, it allows the user to specify which users will receive the information being collected. It also allows ContextBar and ContextBook to produce a filtered view of information, screening information that is not relevant to the task at hand. The Generalized Annotation Gadget (bottom of Figure 2) lets the user associate information with the current document in several ways: The user can add an annotation to the document by typing or copy-and-paste; the resulting annotation is time-stamped and associated with the current user, task, and document. The user can press the Bookmark button. This operation associates a “bookmark” annotation with the document. The user can categorize an annotation by tagging it. ContextBook uses the tags to organize material. Tags that have been used in the current task are available in a pull-down menu, making most tagging fast and helping teams converge on a shared tagging vocabulary. Currently, ContextBar annotations are bound to a whole document. We plan to support annotation at the page and paragraph levels as well. To support our privacy principle, information about the documents a user visits is only shared for those documents that a user explicitly bookmarks or annotates; we also plan to support private bookmarks and annotations that are just for the use of their author. ContextBook is our per-task tool (Figure 3). It helps users track both document and comment information that has been produced by all members of the task team. It displays information entered from ContextBar and allows users to add content directly from ContextBook as well. Its two components, a chat region and a shared notebook, allow content to be assembled and organized like a jointly authored document or outline. The chat region supports synchronous

Downloaded from pro.sagepub.com at NASA AMES RESEARCH CENTER on January 8, 2016


441

the tools and asked them to perform research tasks using the tools and prepare a report on their results. Six pairs used the tools in the earlier phase, working on their tasks for two hours. Two pairs of partners on a high school debate team worked on their debate task and on an “interruption problem” about traveling with a teenage diabetic friend. Four pairs of friends or family worked on a problem of their choice and on an interruption problem concerning medical options for a diabetic uncle. In the most recent evaluation, two triples used ContextBar and ContextBook for an hour investigating the feasibility of changing office lighting to use LEDs; in each triple, two people gathered information for the third group member to use. Summaries or final reports were written with a standard text processor. Log data and debriefing interviews were included. RESULTS Figure 3. The ContextBook

and asynchronous communication among collaborators. Chat text persists in the chat area and is also entered as content in the notebook. This provides a real-time, task-level communication channel, while also allowing users to view messages later. In addition, users can chat to comment about the information being built up, because such comments persist and can be tagged, just like document annotations. Content entered via chat is treated like annotation content but associated only with a user and task, not a document. The notebook displays all document annotations and chat content with their tags, in a vertical column that looks like a document. It provides display in tag mode and time mode. The tag mode organizes information by tag and inserts section headers for each tag. A special section “Other” holds untagged material. If a more hierarchical organization is desired, users can create a hierarchy-like structure by subordinating tags. Clicking on a tag lets the user subordinate it, to act as a subheading similar to sub-folders in a folder hierarchy. The sub-section then appears under all sections in which it occurs, effectively creating multiple sub-folders at once. Sections can be expanded and collapsed to customize the display. Users can contribute to both private and shared section organizations. In the second mode, all comments and annotations are sorted by time, making it easy to find recent documents and comments, for example. Our current ContextBook design displays all of the annotations for a task in a single view. One useful consequence of this design is that task work can be exported as a single document. This document in turn may be the basis for a final product and may be emailed, printed, searched, and so on. Each ContextBook is also assigned a URL so that it can be bookmarked or linked from Web pages. Users can also filter to see only the annotations from a particular user or only the annotations marked with a particular tag. FORMATIVE EVALUATION METHOD To get early feedback on ContextBook and ContextBar, we performed formative evaluations of them at two points in our project. In each case, we trained pairs or triples of users on

All 8 groups worked collaboratively gathering and organizing substantial, relevant material for the task. User response guided our ongoing design throughout. For example, user feedback lead to a design that made it easier for users to distinguish their own comments from those of team members. The system allowed groups to adopt widely differing strategies. For the six pairs, the number of unique URLs visited by the team varied from 23 to 103 URLs visited on the primary task. Revisit patterns also varied considerably with revisits comprising 23%-63% of all views. See Figure 4, which shows the visit and revisit rates for six teams for both the main and interruption tasks.

Figure 4. URL viewings by collaborating pairs. Collaboration

Coordination methods varied greatly. One pair spent considerable time reading and discussing the same Web page, while members of another pair systematically avoided looking at the same document as their partner. For many but not all groups, chat was valued and heavily used. As we expected, some groups used chat to plan activities and allocate sources or issues to each partner. In contrast, one highly interactive group adopted a race model where they competed to find material and joked about “stealing my source”. However, subjects also used annotations in ContextBook to coordinate activity and drive ongoing interaction. In several groups, one partner changed activity because of information contributed by the other; these shifts included changing one’s own tags to be more informative, assessing what questions were now



adequately answered, and pursuing a new, specific question after seeing the partner was finding overview information. Perhaps the most interesting interactions concerned defining or refining the problem to be solved. Some partners chatted to explore and resolve unclear aspects: one pair resolved whether “atomic bombs” were “nuclear weapons” and hence within scope. Working on the diabetic uncle problem, one pair read about different risks for different ethnic groups; they discussed whether they would consider the uncle as Asian or not. In researching LED bulbs the last-joining member asked “Have you guys found a definition for ‘warm’ and ‘cool’?” One form of collaboration support created a tension between awareness of team activity and human spatial memory. ContextBar automatically updates when information from either partner is tagged, redisplaying the information in view. On the one hand this redisplay attracted attention to the availability of updates; on the other, users reported becoming “visually lost” following redisplays. Creating Structure Structuring collected material can help users “look backward” to access information encountered before and “plan forward” to create an organization relevant to the final product, e.g., an outline for a final report. Our tools provided several methods for creating structure and context: Bookmarking the document in ContextBar; tagging an annotation in ContextBook; annotating a document by copying material from the document itself; and annotating a document by typing comments. These methods have overlapping functions and relative reliance on them varied greatly. Some users copied extensively making copy-and-paste “annotations” from a large part of the document. Others never used document content in annotations. With respect to typed-in annotations, some users commented extensively, including chat annotations, while others did this rarely. The use of tagging varied similarly. DISCUSSION ContextBar and ContextBook address the challenge of providing a relevant, appropriately shared set of information and operations both to individuals and groups. They support collaboration and organization of work by task and provide additional support for reading, gathering, annotating, and organizing information by communicating with existing applications. ContextBar aids teams in gathering and annotating information, while sharing the results without additional cost. ContextBook helps teams collaboratively tag and organize the gathered information while also providing a chat tool for explicit, synchronous, persisting communication. We summarize the contribution of our system by pointing out how it embodies the six design principles listed in the introduction, and by relating our system to the prior art on which it builds.

442

The Six Principles and Our Relation to Prior Work

(1) Easy transition from existing practice. ContextBar and ContextBook integrate with current tools by attaching to established applications (such as browsers and editors) as a sidebar, allowing easy communication with those applications. Existing tagging and collaboration tools also strive to integrate with users’ tools; however, typical approaches are limited to linking to a single application, usually a browser as in the case of Malibu (Geyer, Brownholtz, Muller, Dugan, Wilcox, & Millen, 2008), SparTag.us (Hong et al, 2008), and SearchTogether (Morris et al, 2007) or a document editor as in Anchored Conversations (Churchill, Treor, Bly, Nelson, & Cubranic, 2000). Our design approach extends this to allow coupling with multiple applications. This is important because it gives the user a consistent user interface in different contexts and allows teams to combine information from many different sources. (2) Efficient companion. We emphasize the availability of the same tool (with its supporting data base) across multiple phases of complex information work as well as with multiple applications. This increases efficiency because users do not need to manually coordinate sets of information required by different applications as they progress through the many components of a complex sensemaking task. For example, the components for bookmarking URLs and drafting an outline use the same, collaboratively shared database. In contrast, prior collaboration systems typically support a single component, often search, which they support through tagging as in del.icio.us (Del.icio.us, 2008) and dogear (Millen, Feinberg, & Kerr, 2006). Other existing systems support document revision by shared annotation (Churchill et al, 2000), or a shared notebook as in Google Notebook and Evernote (Evernote, 2000). Our emphasis on supporting a large part of the sensemaking process rather than just search is a key difference between our Context tools and SearchTogether. While both provide some similar components, specifically persistent chat and page-specific commenting, SearchTogether focuses exclusively on search. It does not support the broader set of collaborative sensemaking activities such as organizing and outlining the search results, or adding user-generated content to construct a collaborative document. (3) Continuous collaboration. Just as the tool should be available throughout an activity, it should also support collaboration throughout. Further, it should smoothly and flexibly support individual work, synchronous and asynchronous collaboration, and variations in collaboration strategy. Our Context tools were designed to work well for a single individual, to benefit actively collaborating groups with little cost, and even to benefit passive collaboration of larger groupsa design property of many tagging systems. In contrast, conventional chat tools have a ‘sweet spot’ specifically for small collaborating groups; without careful coordination of group, task, and document, chat systems can become confusing as more participants enter (Churchill et al, 2000), nor are they effective annotation tools for an individual working alone.



Both Context tools and SearchTogether are designed primarily to support small groups of individuals actively cooperating on a shared project. For SearchTogether’s task domain of search, it provides two encapsulated strategies for dividing a collaborative task in its Split-Search and MultiEngine Search buttons. Beyond the value of any particular collaboration strategy, this design raises the interesting question of what collaborative strategies might be identified and offered to collaborating groups as “ready to use” options. (4) Individual Privacy. Within a single collaborative task, ContextBar allows users to search and view sites and documents with or without leaving a shared trace. This allows both privacy and sharing while working within the same task context. Many tools for active collaboration assume sharing of all activity within a task (Churchill et al, 2000; Morris & Horvitz, 2007) while tools for passive collaboration allow effective anonymity. An active research area is how task and group characteristics influence which control policies are useful, concerning granularity of access rights, who can change access rights, and when this can be done. (5) Generalized annotation. We generalize the notion of annotation because user-added commentary serves several functions and is bound to several types of elements. Within our Context tools, users may add comments bound to elements of different granularity or scope—to the task as a whole (through the task chat), to files and web pages (through the Generalized Annotation Gadget), and even to the comments themselves (through the “tag” button associated with every comment). User commentary serves varied purposes: task coordination (through the task chat), later retrieval (through document bookmarking), extracting particular pieces of information (though cut & paste entries in the document’s ContextBar), and building a new structure or draft (through insertion of document and task comments and through tagbased outlines generated in ContextBook). The Generalized Annotation Gadget in the ContextBar supports both bookmarking and annotations bound to a URL or to a file and also allows annotations on annotations. Currently, document annotations are only bound to the URL or file as a whole, rather than to a particular part or location within the document. SparTag.us (Hong et al, 2008) provides highlighting and tagging of web documents at the paragraph level with a UI that appears in place in the browser; a companion notebook shows a summary of gathered passages from this or another user. We are in the process of integrating SparTag.us into our system, so as to better support those aspects of sensemaking that require in-place, paragraph-level highlighting and tagging. Systems designed for shared document annotation often allow users to add comments bound to a specific location within a document (Churchill et al, 2000; Weng & Gennari, 2004) but do not also support bookmarking functions. Systems designed for shared bookmarking of URLs allow users to add tags and comments, bound to a URL as a whole, as with del.icio.us (Del.icio.us, 2008) on the public Internet and dogear in the enterprise (Millen et al, 2006). (6) Task-based structure. ContextBar/Book uses the context structure of tasks and teams to facilitate gathering and using information. It is primarily a collaborative sensemaking

443

tool organized by tasks, rather than a task management tool for coordinating activities, as are Malibu (Geyer et al, 2008) and RADAR (CMU, 2008). Like Rooms (Henderson & Card, 1986), ContextBar/Book uses the organization of a task to provide a task-specific workspace, but ContextBar shares the information that participants gathered and generated. Our use of task as an organizing principle is similar to the persistent “sessions” in SearchTogether, but ContextBar/Book allows users to pivot among tasks, for example, viewing the same document from the perspective of different tasks. Future Directions

We are expanding the notion of context beyond the task. It may include the current document, members of the team working on that task, and other conceptually close information. If a user accesses a document, she is more likely to be interested in its versions, authors, or other task forces also annotating it. However context is defined, collaborative information foraging and sensemaking can be improved by reducing the cost of accessing and distributing information that is more likely to advance the user’s goals, given the current context. To better support the user, we are working to combine ContextBar and ContextBook with additional support for identifying and using context. We plan to integrate our tools with natural language processing, image recognition, and activity detection. REFERENCES Carnegie Mellon University (accessed 2008). RADAR - Reflective Agents with Distributed Adaptive Reasoning. http://radar.cs.cmu.edu/index.jsp?s=2 Churchill, E. F., Trevor, J., Bly, S., Nelson, L., & Cubranic, D. 2000. Anchored conversations: chatting in the context of a document. In Proceedings of Human Factors in Computing Systems. CHI '00. 454-461. Del.icio.us, (accessed 2008) http://del.icio.us/ Evernote Corporation (accessed 2008) http://evernote.com/ Geyer, W., Brownholtz, B., Muller, M., Dugan, C., Wilcox, E., & Millen, D. R. 2007. Malibu personal productivity assistant. In CHI '07 Extended Abstracts on Human Factors in Computing. 2375-2380. Golder, S. &Huberman, B. A. (2006). Usage patterns of collaborative tagging systems. Journal of Information Science 32(2): 198-208. Henderson, D. Austin, Jr., and Card, Stuart K. (1986). Rooms: The use of multiple virtual workspaces to reduce space contention in a windowbased graphical user interface. ACM Transaction on Graphics 5 (3), 211241. Hong, L., Chi, E. H., Budiu, R., Pirolli, P., & Nelson, L. 2008. SparTag.us: a low cost tagging system for foraging of web content. In Proceedings of the Working Conference on Advanced Visual Interfaces '08. 65-72. Millen, D. R., Feinberg, J., & Kerr, B. 2006. Dogear: Social bookmarking in the enterprise. In Proc. of Human Factors in Computing Systems. CHI '06. 111-120. Morris, M. R., & Horvitz, E. 2007. SearchTogether: an interface for collaborative web search. In Proceedings of the 20th annual ACM Symposium on User Interface Software and Technology, UIST 2007, 312. Shirky, C. (2008). Here Comes Everybody: The Power of Organizing Without Organizations. New York: Penguin Press. WikiPedia (2008). Virtual desktop. http://en.wikipedia.org/wiki/Virtual_desktop, on 2008/09/18. Weng, C. & Gennari, J. H. (2004). Asynchronous collaborative writing through annotations. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, CSCW '04. 578-58.
