The Project Browser: Supporting Information Access for a Project Team Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post TNO Human Factors, P.O. Box 23, 3769 ZG Soesterberg, The Netherlands {anita.cremers, peter.groenewegen, wilfried.post}@tno.nl,
[email protected]
Abstract. The objective of our study was to design and evaluate a user interface concept for a so-called Project Browser, following a user-centered design method. Previous work has suggested that users prefer to access project-related information instrumental to the task they have to carry out. First, a domain analysis was carried out, followed by an extensive user requirements study, eliciting input from people experienced with working in project teams. This formed the basis of a prototype user interface design of a Project Browser, which provides direct access to action points and decisions taken, irrespective of the source of the information (documents, messages or meetings). The information from meetings was extracted from digital (multi-media) recordings of meetings, through the application of automatic recognition and information processing technologies. The user interface concept was evaluated with a number of prospective users which resulted in positive and promising findings. Next, the browser will be further evaluated in a scenario-based project setting. Keywords: user interface, meeting browser, decisions, action points, documents, messages, meetings.
1 Introduction People who are collaborating in a project team typically spend their time working individually on their tasks and meeting at regular intervals to discuss the progress so far. During both individual work and meetings, they need access to information regarding the project to be able to carry out their tasks in an accurate, efficient and satisfactory manner. Unfortunately, this information is usually scattered over many different sources, such as messages (e-mails), documents and presentations. Information regarding meetings is usually laid down in minutes. Due to this scattering of information it is difficult for project team members to know whether the information they have found is complete and up-to-date. In particular information on meetings is problematic, since minutes tend to be concise and consequently hard to understand, especially for people who did not attend the meeting. In order to provide better access to meeting information, previous work has been carried out on meeting browser concepts. A meeting browser is an application that supports users in finding elements of interest in (multi-media) digital recordings that have been captured during meetings [1]. This is made possible through the application
2
Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post
of novel multi-media and automatic recognition and processing technologies that create digital, annotated recordings of meetings. Existing research prototypes of these innovative meeting browsers, such as Ferret [2] provide synchronized access to, among other things, videos of participants, presentations held, loggings of who speaks when, transcripts of dialogues and minutes (Figure 1). However, the user interface design of Ferret in its present form has not taken into account the user perspective yet, so it is not clear whether it suits the needs of the prospective user. Evaluations with newly developed user interface concepts for meeting browsers have suggested that users prefer to access the information in a task-oriented manner [3]. In particular, they are highly interested in decisions taken during the meetings and want to have an overview of their own action points. In other words, they want the information to be instrumental for carrying out the project. It can be expected that this also applies to other than meeting-related information which is available regarding the project.
Fig. 1. Research prototype of a meeting browser (Ferret) providing synchronized access to (from left to right, top to bottom) presentations, close-up videos of participants, minutes (abstract, actions, decisions, problems), speaker activity logs (who spoke when), and transcript. All information presented here is extracted from a meeting held in the context of playing a scenario in which a four-member project team was engaged in designing a new television remote control [4]. The meeting is part of the AMI corpus, available through: http://corpus.amiproject.org.
The Project Browser: Supporting Information Access for a Project Team
3
2 Objective The objective of the present study was to design and evaluate a user interface concept for a so-called Project Browser, using a user-centered design method. The Project Browser was designed in order to provide access to all available project information, irrespective of the source of the information, in a task-oriented manner, i.e. instrumental for carrying out the project. The central research question was how users experience this way of accessing project-related information, including meeting information. This novel browser can be seen as an extension of existing research prototypes of meeting browsers. Ultimately, it could form an integral part of a project management tool. All information presented in the concept browser is extracted from a project carried out by a four-member project team playing a scenario in which they were engaged in designing a new television remote control ([4]; [5]).
3 Method A user-centered design method was applied to support the design process of the Project Browser [6]. First, a requirements analysis was carried out with nine participants, who had experience working in design projects and participating in meetings. The analysis consisted of the following parts: 1. A pre-questionnaire dealing with background information, technology experience, meeting experience and design projects of the participants. 2. Interviews with all participants to gain insight into how they experience communication and information exchange in design projects and what problems they encounter there. The following main subjects were addressed: − Meetings: when is the goal (not) reached, when is time (not) well spent? − Collaboration, communication: what works well, what are obstacles? − Supportive technologies: positive and negative experiences? − Project support tools: comments on short descriptions and screenshots of the Ferret browser (Fig. 1) and Basecamp (Fig. 2), a representative example of a (web-based) project management tool (http://www.37signals.com). Functionalities offered in Basecamp are posting messages and gathering feedback, assigning todo’s and tasks, scheduling (milestones), sharing files and tracking people’s time spent on the project. 3. An assignment in which participants were asked to think of the role they usually fulfill in a design team, and imagine they had to replace a person with the same role in a project team. First, participants were asked to write down five to ten questions about the project and its team they would like to ask this person, in order to prepare for the next project meeting, and to sort them in order of importance. Second, they were asked to sort twelve similar predefined questions, again in order of importance. These questions dealt with topics that could be addressed in novel meeting browsers, e.g., emotions expressed by team members. 4. A post-questionnaire, asking about requirements for a future meeting room, instrumented with multimedia, automatic recognition and processing technologies.
4
Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post
Fig. 2. Basecamp overview screen: messages, to-do’s, milestones, writeboards, time, files and people.
Findings from the analysis were written down in the form of user requirements, which have formed the basis for designing the main screens of the proposed Project Browser. The design was implemented into a ‘click-through’ version, which allowed limited navigation through the various screens. Finally, the Project Browser concept was evaluated by four of the nine participants in the requirements analysis, in order to get user feedback on the concept, the overall information structure, functionalities, and specific design details. Each participant first received an explanation of the concept and functionalities. Then the participants ‘walked through’ the various screens of the Project Browser, meanwhile thinking aloud about issues such as their expectations, what they saw, what they understood and did not understand, what they liked and did not like, and what they missed.
The Project Browser: Supporting Information Access for a Project Team
5
4 Results
4.1 Requirements analysis Nine people, whose ages ranged from 23 to 47, participated in the requirements analysis. Six participants were professionals, three were students. Participants either worked or studied at TNO Human Factors, Philips Electronics and the Utrecht School of the Arts, offering sufficient variety in fields of work and team roles. 4.1.1 Pre-questionnaire All nine participants filled in the pre-questionnaire. All of them had extensive experience with computer and internet use. Most participants had a considerate amount of experience participating in meetings; all attended meetings at least on a weekly basis. All participants had average to a lot of experience in design projects. Participants felt that objectives for their meetings were attained most of the times (8) or sometimes (1). They felt that the time for meetings was most of the times (6) or sometimes (3) well-spent. They indicated that most of the times (6) or sometimes (3) they liked to participate in meetings. The five means they used most to prepare for a meeting were the agenda, related documents, personal recollection, contacting other participants and means to prepare a presentation. Least used were contacting external people, consulting external information sources, and pictures, audio recordings or video recordings of previous meetings. The ‘top 3’ of means used during a meeting were personal notes, give/discuss a presentation and the agenda. Least used were minutes of the previous meeting, consulting external information sources, making pictures, audio or video recordings, and using audio or video conferencing tools. The ‘top 5’ of means used after a meeting were personal recollection, personal notes of the previous meeting(s), related documents, contacting other participants and the agenda. Pictures, audio and video recordings of previous meetings, and consulting external information sources were hardly used. The ‘top 3’ of items to include in personal notes were things to do, reminders, and reference materials. When a meeting is missed most participants would like to catch up through reading meeting minutes and asking other participants. 4.1.3 Interviews The main results of the nine interviews that were conducted are summarized below. Meetings: One-third of the participants mention the distinction between meetings for project management and meetings for brainstorming or creative sessions. They assume most meetings belong to the first category. They feel objectives of meetings are not attained when not all necessary decisions have been made. This is caused either by people not being present, not having sufficient information at the time of the meeting or elaborating on a subject for too long. Participants felt meetings are unpleasant to attend when the content is not relevant to them. When they miss a meeting most participants consult their team members to catch up.
6
Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post
Collaborating in design teams: Tools for communication and information exchange most often used in design projects are telephone and e-mail, and sometimes Skype (http://www.skype.com), which can also be used for group meetings. People prefer e-mail to exchange files, which has the advantage of knowing the other person has received your file. File servers, Wiki’s or document management systems are also used to exchange files, but, unfortunately, some of these tools do not offer version management. Obscurity about requirements and deliverables is mentioned as an important pitfall in design projects. Quality of communication and information exchange depends on their frequencies and whether team members work in the same location or in different locations. Supporting technologies: Generally, participants who had worked with a collaboration tool had negative experiences. The main problem was the extra time and energy it took to use the tool, resulting in added extra strain to daily work. Also, not all team members used the tool on a regular basis, so messages and files remained unread. Finally, they felt not all information provided through the tool was reliable. Project support tools: Participants found the amount of data produced in Ferret quite overwhelming, and probably hard to process. The possibility of reviewing missed meetings was most useful to them. Suggestions for useful functionalities were retracing ideas from brainstorm sessions and finding arguments for design decisions (in current or previous projects). Participants expressed their concern about having to rely on recordings and summaries which are automatically generated, consequently missing the human interpretation and nuance. Further, privacy issues were mentioned, in particular the observation that some meetings may not be appropriate to record. Basecamp’s centralized project administration was appreciated most by the participants. They expected it to reduce time spent on administrative issues in meetings, such as changes in time planning. Also, the possibility of looking up to do’s was seen as useful. Basecamp could be improved by better accommodating needs of people working asynchronous and at different locations. 4.1.4 Assignment The questions generated by nine participants in the context of replacing a person in a design team were distributed over the following predefined categories fairly evenly: meetings, individual team members, team as a whole, task, time control, information control. In the categories organization, project environment and budget relatively few questions were asked. No questions were generated in the category quality control. Most often rated as most important question was: “What is the goal of the project?”. Other important questions were about individuals in the team, and difficulties or obstacles in the project. Further questions concerned milestones and deadlines, documentation of previous meetings and agenda of upcoming meetings. Mentioned only once were questions about which team members performed well and which did not, about average working hours and about what parties are involved in the project. The ordering of predefined questions was similar to the generated ones. Again the task, project goal and design decisions are considered most important. Arguments leading to these decisions are also considered important, followed by other team members and their roles, and the plan for the next meeting. Novel topics, such as expressed emotions were not considered important.
The Project Browser: Supporting Information Access for a Project Team
7
4.1.5 Post-questionnaire Post questionnaires were filled in by eight participants. Participants were asked to imagine a ‘smart meeting room’ that could automatically create multimedia recordings of meetings. They were asked what information they would like these recordings to include. The resulting ‘top 5’ out of a list of twelve possibilities was: agenda, transcribed speech, shared notes, used documents and presentations and personal notes/information on participants. Personal notes include points to improve in your work, ideas and design concepts other then discussed, individual goals, atmospheres, attitudes, personal to do's and points of attention. Participants were also asked how they would like to search for specific information within ‘smart minutes’, if these were available. This resulted in the following ‘top 5’ out of a list of sixteen possibilities: arguments for decisions, topics and things to do, agenda items and participants/speakers. Participants also indicated what sources they would use to catch up on a missed meeting, resulting in the following ‘top 5’: get an overview of things to do, get an automated summary, get a gist (the essence) of the contents of the meeting, browse through the ‘smart’ minutes and get a gist of the atmosphere during the meeting. Most participants think these types of information could be extracted from meeting recordings, and they trust ‘smart minutes’ to provide an accurate and adequate representation of the meeting. Participants indicated that team members should have access to all information, whereas clients and managers should have limited access only (for instance planning, hours logged, meeting agenda, people present). 4.2 User requirements Project Browser From the requirements analysis the user requirements for the Project Browser were identified. In these requirements some future developments of technologies are assumed to be available, for instance automatic topic tracking and speech recognition. • The Project Browser should provide access to project information and documentation, such as agenda, minutes, documents, presentations, internet, personal notes and meeting recordings. • It should provide a first step in processing this large amount of raw meeting data, automatically generating items such as todo’s, ideas, decisions and arguments, summaries (minutes). • It should generate (most of) its content automatically, and not require additional time or effort from its users (by filling in data such as todo’s). • It should clearly communicate the goal and the status of the project, and support consensus on planning, requirements, milestones and deliverables. • It should be sufficiently flexible to support a variety of design work settings, in particular remote cooperation with the possibility of exchanging messages. • It should consider privacy issues. It should have the possibility to authorize certain persons to view the information (for instance only team members and not management or clients).
8
Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post
4.3 The Project Browser
Fig. 3. The Meetings tab, listing all available digitally recorded meetings. When selecting one of the meetings from the list in the left section, meeting Details appear in the middle, including direct access to digital recordings of meetings, including automatically created minutes, and transcripts of the meeting dialogues.
The Project Browser provides direct access to three different information sources via the tabs Meetings (Fig. 3), Documents and Messages. In addition, it is possible to access these sources indirectly via three task-oriented tabs: Project (project details, people involved, the current design status, milestones, and a timeline), Todo’s (all Todo’s and my Todo’s, open and completed Todo’s, Fig. 4) and Decisions (including a listing of the arguments that have lead to the decision). All information items are hyperlinked to the original sources. This makes it possible to, for instance, immediately view a meeting clip in which a specific information item, such as an action item, is being discussed. In addition, it is possible to search for certain information over all available sources. Not included yet in the concept are: adding personal notes, automatically recognizing and presenting ideas, extensive project planning, and authorizing users.
The Project Browser: Supporting Information Access for a Project Team
9
Fig. 4. The Todo’s tab, listing both automatically created and manually added todo’s. When selecting one Todo from the list in the left section, the Details appear in the middle, and the Sources in which the Todo has originally appeared (in this case a meeting, a document and an email) can be previewed on the right.
4.4 User evaluation Four participants of the user requirements study evaluated the Project Browser ‘clickthrough’ concept. The opinion of participants was generally positive, although they experienced the amount of information in the Project Browser still to be overwhelming and they suggested reducing it as much as possible. The participants liked the synergy of the three information sources in one concept, though. They also appreciated the concept of generating information from meetings and making it available to team members. The fairly consistent information structure of all tabs helped participants to understand the application. Nevertheless, they also provided detailed suggestions for improvement of the various tabs related to functionality, information presentation and interaction. Finally, they mentioned it was hard for them to judge how such a tool would function in a real design project, and that it would probably be most suitable for larger teams.
10
Anita Cremers, Inge Kuijper, Peter Groenewegen, Wilfried Post
5 Conclusions and further work The concept of the Project Browser seems to be a promising tool for use by project team members, in particular since it allows access to all available information sources through one system. However, care should be taken to limit the information to only the most necessary, in order to avoid overwhelming the user. Although it is promising, it is still not clear whether this type of browser is more useful than existing meeting browsers and provides added value to existing project management tools, in the context of a design project. To investigate this, the project browser will be implemented and compared to other types of, non task-oriented, browsers, in a near real-life scenario-based experimental environment [6]. In this environment, the project behavior, in terms of efficiency and participant satisfaction, can be measured during as well as after project execution, resulting in insight into the added value of browsers to project success. Acknowledgments. This work was partly supported by the European Union 6th FWP IST Integrated Project AMI (Augmented Multi-party Interaction, FP6-506811, publication AMI-235).
References 1. Tucker, S. & Whittaker, S.: Accessing multimodal meeting data: systems, problems and possibilities. Fourth Joint AMI/PASCAL/IM2/M4 workshop on multimodal interaction and related machine learning algorithms, Martigny, June 21-22 (2004) 2. Wellner, P., Flynn, M., Guillemot, M.: Browsing Recorded Meetings with Ferret. In: Machine Learning for Multimodal Interaction. Lecture Notes in Computer Science 3361 (2005) 12-21 3. Cremers, A.H.M., Hilhorst, B., Vermeeren, A.P.O.S.: “What was discussed by whom, how, when and where?”: personalized browsing of annotated multimedia meeting recordings. HCI International, Las Vegas, July 22-27 (2005) 4. Post, W.M., Cremers, A,H.M., Blanson Henkemans, O.: A research environment for meeting behavior. In: Nijholt, A., Nishida, T. (eds.): Proceedings of Social Intelligence Design (2004) 159-165 5. AMI: Augmented Multi-party Interaction (AMI). Sixth Framework Programme, Priority 2, Information Society Technologies, Integrated project Annex 1 – “Description of Work” (2003). http://www.amiproject.org 6. Kuijper, I.: A user-centred approach to developing groupware. Master thesis Professional School of the Arts, Utrecht (2006) 7. Post, W., Elling, E., Cremers, A., Kraaij, W.: An experimental comparison of multimodal meeting browsers. This volume (2007)
