Designing Effective Notifications for Collaborative Development ...

Designing Effective Notifications for Collaborative Development Environments Joanna McGrenere Department of Computer Science, UBC

Jin Li, Elena Litani, and Jimmy Lo IBM Toronto Lab

Abstract We describe research conducted to improve the design and management of notifications in the Jazz collaborative development environment. Scenario-based design was used in conjunction with focus groups that included eight representative Jazz users. The end result is a proof of concept prototype implementing a new notification architecture.

1 Introduction Software development teams have learned that software is best developed by a team of people working together, reacting and responding to each other. Jazz is an IBM initiative to help make software delivery teams more effective [3]. It is a collaborative development environment (CDE) designed to transform software delivery, making it more collaborative, productive and transparent for development teams using the agile development process. Instead of an integrated development environment (IDE), such as Eclipse, which focuses on supporting an individual software developer, a CDE puts the team first, with the assumption that team productivity will increase [1]. Agile development uses a less structured development process relative to more traditional software development, and thus has a greater need for awareness of team member activities. Agile encourages rapid peer-to-peer communication, relying much less on official top down communication channels, such as team meetings. Jazz supports informal communication and lightweight awareness through a number of different

Figure 1: The Jazz CDE, showing the events list (left), linking to work item data (right). mechanisms, which include notifications of events that are fired based on many user activities (such as completing a work item) as well as system activities (such as the outcome of a build). The events are queued temporally, as shown in Figure 1, and developers sift through the events list to make sense of what is happening with the team. Jazz developers also rely heavily on other communication channels such as email and chat. While the agile process necessitates lightweight notifications as a means of keeping awareness of the constant and rapid changes happening on the team, the notifications often interrupt and overwhelm jazz developers, making it hard for them to focus on getting their work done. The goal of the work reported in this paper was to explore and validate this problem. The main contributions of our work are threefold: (1) We validated and prioritized of the pain points experienced by developers and other team members with respect to notifications and awareness of team activity; (2) We generated design requirements for the high priority pain points we identified; and (3) We created a preliminary design for the “Inbox” UI widget, and implemented a proof of concept prototype.

Copyright © 2009 Dr. Joanna McGrenere, IBM Canada Ltd.. Permission to copy is hereby granted provided the original copyright notice is reproduced in copies made.

1

Jazz CDE, resulting in poor awareness of team activities. The pain points that emerged in the first session and that were ranked as most problematic in the second session were as follows: 1. Awareness of changes by others that affect my current work 2. Prioritizing all incoming communication related to my current work 3. Notifying others who are (may be) affected by changes in my work 4. Blocked communication path to others Further, our main findings can be categorized within the following four themes: #1 Individual needs come first, team needs second. Despite the intention behind a CDE to support and benefit a team, the team is not the first priority for users. Design should focus on benefits to the individual, with the expectation that improved notification and awareness for the individual will in turn lead to a productivity improvement for the team. #2 Notification customization presents unique opportunities and challenges. There is no one size fits all solution. Users wanted to be able to customize to remove irrelevant notifications, but they were concerned with blocking all notifications of a certain type because some might be important. There is a lot of opportunity to leverage the CDE “knowledge”. For example, if it knows that a user is working on a particular work item right now, any events related to that work item could be made more salient to grab the user’s attention. #3 Notification UI should not block users: control must remain with the user. Consistent with other human-computer interaction research, our users were leery of the system taking over too much control [4]. For example, a programmer might get notified that the work item she is currently working on is being pushed to the next iteration. By no means should that notification also try to assist the programmer by automatically pushing it into future work. A programmer may want to finish a conceptual piece of the work item before putting shelving it. #4 Don’t want technology “watching you.” Related to the above points, users don’t want to feel like they are being watched. Notifications can serve different purposes for different roles. Project managers may like to see many notifications; this gives a sense of activity and progress. On the other hand, some developers choose not to follow team process (e.g., deliberately do not change the status of work items to “in progress”)

2 Research Conducted We took a scenario based design approach [2] and conducted a focus group user study with eight representative Jazz users to validate the scenarios and generate design requirements. There were four phases to the research, each outlined below.

2.1 Scenario Development A scenario is a story-like description of an idealized but detailed instance of humancomputer interaction. In our research we used scenarios to show notifications in context. The scenario development brought the academicindustry research team together, clarifying our own understanding of notifications and interruptions in Jazz. We created two scenarios: the As-Is scenario which reflected our assumptions about the current problems with notifications in Jazz, and the To-Be scenario which addressed those problems with specific design solutions. Here is a sample excerpt from our As-Is scenario: Patricia (Project Manager) and Marco (Team Lead) are working together to plan the next iteration and need to verify the availability of all the developers for the next week. Marco instant messages Deirdre (Component Developer) to confirm that she has completed an assigned feature, as that will impact the scheduled planning. Deirdre has blocked all IMs, so that she can focus on her work; however, as her boss, Marco’s IM gets through and she replies immediately.

2.2 Scenario Validation Through a User Study We conducted a three-session focus group study, each session lasting 1.5 hours, with eight representative participants, all IBMers who use Jazz. Our high-level goal was to elicit requirements. The specific goal of the first session was to present and validate the As-Is scenario and to document specific pain points experienced by our participants. After the first session we modified the To-Be scenario, reflecting what we had learned. The goal of the second session was to validate the revised To-Be, and to prioritize the pain points elicited in the first session. The third focus group session is described in Section 2.3. From these two sessions we learned that our scenarios were valid, although some minor adjustments were required. Our participants told us that they are over exposed to notifications in the

2

because this often generates many notifications that only help others to track their progress.

2.3 Exploration of Design Requirements The goal of the third phase of the research was to prioritize the requirements gathered in the second phase and explore design ideas. To do this we conducted brainstorming sessions within our research team, generated design sketches, and solicited input in the third focus group session. The key design goal that emerged was: To create a central place to view and prioritize work items and high-priority events. The latter were defined as events relevant to a specific user’s work. An example of one preliminary sketch of our design ideas is shown in Figure 2. The core design elements are as follows. Unification of all notifications relevant to the user, through a notification “Inbox.” All notifications (events, email, chats) arrive in a single place, by default in temporal order. The full set of events (“All Events”) are still available to the user. User control over notification management. User can delete notifications or move them individually to Current Todo or Future Todo areas. Notifications are automatically grouped. All events relating to a particular work item, for example, appear together, enabling the user to easily digest the current activities related to a particular work item. Additional filters/sorting is available. Small amount of automated assistance is provided. For example, when a chat appears in the inbox, the user can choose to move it directly to one of the todo areas without responding, and an automated response is sent to the sender indicating that the recipient has queued the chat request. Visual design supports the user. Link highlighting between the events in the Inbox and All Events allows the user to see where in the stream of full events his/her specific events are found. Colour coding of events by type supports easy scanning. Blinking chats makes time critical request more visually salient.

Figure 2. Sketch of UI inbox widget. Full integration with email and chat is yet to be completed.

3 Conclusions This work represents a first step in creating a new notification architecture for the Jazz CDE. It is the outcome from scenario-based design and the participation of eight Jazz users. Next steps involve evaluating our prototype with those same users as well as users new to our project. Based on what we learn, we expect to iterate on the design, before deploying it in the next release of the Jazz WebUI.

4 References [1] Booch, G., and Brown, A.W. (2002). Collaborative Development Environments. Online: http://www.booch.com/architecture/blog/artifacts/CDE. pdf, retrieved 10/15/2009. [2] Carroll, J.M. Ed. (1995). Scenario-based design. John Wiley. New York, New York.

2.4 Prototype Creation

[3] Jazz. http://www01.ibm.com/software/rational/jazz/, retrieved 10/15/2009.

The fourth phase of the research involved creating a proof of concept working prototype for the Jazz WebUI, which is the web version of Jazz, currently in its first release. It supports many of the features and design elements described above.

[4] McGrenere, J., Baecker, R., and Booth, K. (2007). Trans. on Human-Computer Interaction, v. 14, n.1, p.3.

3