Ethnography, Scenario-Based Observational ... - Semantic Scholar

INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 17(3), 403-426 Copyright © 2004, Lawrence Erlbaum Associates, Inc.

Ethnography, Scenario-Based Observational Usability Study, and Other Reviews Inform the Design of a Web-Based E-Notebook Yolanda Jacobs Reimer Departnnent of Computer Science University of Montana

Sarah A. Douglas Department of Computer and Information Science University of Oregon

As users turn to the World Wide Web to accomplish an increasing variety of daily tasks, many engage in information assimilation (IA), a process defined as the gathering, editing, annotating, organizing, and saving of Web information, and the tracking of ongoing Web work processes. The process of IA, which is similar to traditional note taking but in the Web environment, emerges from a literature review and an ethnographic field study, as presented in this article. Despite strong evidence which suggests that IA is critical to many Web users, however, a scenario-based observational usability study and a heuristic evaluation indicate that it is currently not well supported by existing software applications. This article, which culminates in the presentation of NetNotes—a Web-based e notebook developed specifically to support the process of IA—illustrates how design requirements can be effectively extracted and synthesized from a variety of complementary background user studies.

1. INTRODUCTION More and more people are turning to the World Wide Web on a daily basis to accomplish a wide variety of tasks. Unfortunately, despite the explosive growth, popularity, and accessibility of the Web, users do not have the proper softwâre support to help them accomplish many of their tasks effectively and efficiently. In particular. Web users cannot readily engage in the process of information assimilation (IA), which for the purposes of this article is defined at a high level as the ability to gather, edit, annotate, organize, and save information from multiple, disparate This research was conducted at the University of Oregon. Requests for reprints should be sent to Yolanda Jacobs Reimer, Department of Computer Science, University of Montana, Missoula, MT 59812. E-mail: [email protected]

404

Reimer and Douglas

Web pages. In addition, because Web users' tasks are often protracted and extend beyond one continuous browsing session, IA also includes the ability of users to track their ongoing Web work processes so that they can easily recall and rejoin previous work. All too often. Web users are forced to depend on ad hoc and inconsistent methods to complete their critical IA tasks. The process of IA just defined, although Web-based, is quite similar to the traditional process of note taking with which many people are already familiar. Years of use indicate that note taking and its traditional implements (pen and paper) are integral to many daily tasks. Among other reasons, people take notes to help them remember things, to document events, and to organize information. However, because people now rely on the Web to help them with many of these same tasks, the traditional paper-based note taking paradigm, which is suddenly cumbersome and inefficient in the electronic environment, must be reexamined. It is time to step back, examine evolving Web activities, and work toward developing appropriate software tools to support these activities. The goal of this research was to incorporate key aspects of the user-centered design (UCD) process into the development of a Web-based e-notebook (i.e., NetNotes). This article describes a nun:iber of user studies that support the definition of IA, provide evidence that IA is a critical process for many Web users (and scientists in particular), and contributed directly to the development of NetNotes. The research described in this article is noteworthy because it does the following: 1. Offers a case study illustrating how design requirements can be extracted and synthesized from a variety of complementary background studies (i.e., literature review, ethnography, scenario-based observational usability study, heuristic evaluation). 2. Defines a theory of IA. 3. Illustrates that studying users' traditional note taking tasks can be an effective way to identify design requirements for electronic note taking software. 4. Suggests ways in which the process of note taking changes in the digital environment. 5. Provides evidence that note taking is critical to n:iany people—and scientists in particular—but that current software fails to adequately support the process of IA. 6. Introduces and describes NetNotes, a Web-based e-notebook designed specifically for lA. Section 2 begins with an empirical literature review of the process of traditional note taking where the focus is on why people take notes and what constitutes "good" notes. This review is followed by an ethnographic field study that examines the ways in which scientists regularly engage in traditional note taking. Section 2 concludes with a discussion of the possible paradigm shift that occurs when transitioning from traditional note taking to note taking on the Web and with a list of e-notebook functional requirements that emerge from the analyses discussed up to that point. Section 3 describes the current state of software support for IA beginning with a scenario-based observational usability study of how a group of 10 biologists perform a set of critical IA tasks with their standard software applications. This

Reviews And Web-Based E-Notebook Design

405

usability study is followed by a heuristic evaluation of other Web-based e-notebook applications. The studies described in Sections 2 and 3—a literature review, ethnography, the scenario-based observational usability study, and heuristic evaluation—provide ample evidence that the process of IA is critical to many Web users (certainly to scientists), and that adequate software support is currently lacking. Section 4, then, presents the NetNotes prototype, a Web-based e-notebook designed specifically to support key IA tasks.

2. PEN AND PAPER: AN EMPIRICAL LOOK AT TRADITIONAL NOTE TAKING

The research presented in this article revolves around the theory of IA. IA is defined as a Web-based process whereby users gather, edit, annotate, organize, and save information from multiple disparate Web pages, as well as keep track of long-term, ongoing Web work processes. As previously mentioned, the process of IA is analogous to the traditional process of note taking except that IA is Web-based and traditional note taking is not. The presumption that because people currently use traditional note taking to help manage many daily tasks, they will therefore also need to rely on some form of note taking when they move to the online environment summons new questions. For example, what evidence is there to support the belief that note taking is an important part of Web use? Does the process of note taking change when people move to the Web environment? What kinds of tools are needed to support Web-based note taking tasks? In this section, these and other questions that arise in the discussion of online note taking are addressed. Other researchers have identified and defined processes that are similar to IA, but with decidedly different foci. For example, Marshall and Shipmann (1997) defined information triage as "the process of sorting through relevant materials" (such as those retrieved from the Web) "and organizing them to meet the needs of the task at hand" (p. 124). However, whereas IA centers around the capture and subsequent editing and integration of smaller units of Web elements (e.g., a paragraph containing formatted text, hyperlinks, and images), information triage seems to focus more on the retrieval, manipulation, categorization, and scanning of collections of documents. Furthermore, Marshall and Shipmann acknowledged that the activity of information triage is "often time-constrained, and requires quick assessment based on insufficient knowledge" (p. 124); this is in contrast to IA, whose processes are often prolonged and extended over a period of time. This section presents research that is central to the definition of IA, beginning with a summary of an empirical review of the process of traditional note taking. Although recording notes in the physical world (i.e., with pen and paper) and in the virtual world (i.e., electronically on a computer) are significantly different processes that incorporate very different implements and mediums, they are both linked by similarities in user intent. For this reason, it seems reasonable to begin research on electronic note taking with what is already known about traditional note taking. Complementing this empirical review is an ethnographic study that focuses on how a real group of scientists takes notes. Whereas the empirical review

406

Reimer and Douglas

reveals some general reasons why people take notes and what "good" notes are, the ethnographic study highlights the current note taking practices of an actual user group, including specific components of notes, their organization, and how they are retrieved and used. The literature review and the ethnographic field study resulted in a preliminary list of tasks that note taking software should support. However, before this list of design requirements could be finalized, it was necessary to first revisit the process of Web-based note taking and consider the ways in which traditional note taking and IA differ. Once these differences were understood, it was finally possible to define a low-level list of IA requirements that should be addressed by software in the form of a Web-based e-notebook. This list is presented at the end of this section and is used in Section 3 to evaluate how well existing software applications support IA. 2.1. Summary of Traditional Note Taking

This literature review seeks answers to two main questions: Why do people take notes? and What are "good" notes composed of? Although the reasons why people engage in note taking are varied, most of them can be classified as belonging to one or more of the following categories: 1. Note taking to assist recall: People record notes to assist both their prospective memory, or the memory that helps them to remember something that will occur in the future, and their retrospective memory, which helps them to remember something that has occurred in the past (Baddeley & Wilkins, 1984; Meacham & Leiman, 1982). 2. Note taking is thinking: When people think, they participate in an internal monologue with themselves; when they write, and as they consider other voices and writings—those that either support or dissent from their own ideas—that internal monologue becomes an external dialogue; it becomes a discourse community (Gage, 1991). 3. Note taking to organize information: Note taking helps people organize information by drawing together related information into one centralized location— a convergence that at least aims at a well-structured and cohesive final product— and to facilitate consistency and logical progression in both writing and thought. 4. Note taking to process information: Notes also serve to process information in ways that can increase comprehension and memorability. Certain factors, such as the speed of the lecture (Aiken, Thomas, & Shennum, 1975), how familiar the subject matter is to the note taker (Peper & Mayer, 1986; Shrager & Mayer, 1989), and whether or not the notes are reviewed (Dyer, Riley, & Yekovich, 1979; Hartley & Davies, 1978; Shrager & Mayer, 1989; Wittrock, 1974), can impact how effective notes are in helping the note taker process and learn the material. 5. Note taking to document events: Notes are often used to document events for legal, recreational, or professional purposes. "Good" notes are those that serve their intended purpose. As such, it is critical for note takers to consider how, when, and why the notes they record will later be


407

used. In other words, the production of notes and the perception, or later consideration, of those notes are closely interlinked {Monty, 1990). For example, consider a student who records notes during a lecture with the intention of using then^ to study for a future exam. In addition to making the notes as complete and accurate as possible, which prompt review and possible revision might facilitate, the student should also be guided by important cognitive tools, or heuristics, when recording the notes. According to Monty, some of these cognitive heuristics include metamemory (Flavell & Wellman, 1973), or the knowledge of how one's own memory works; the importance of the information; memorability; and review context. If considered and applied conscientiously during their production, cognitive heuristics serve to improve the quality of notes and can improve efficiency when the time comes for using the notes. Notes are also judged by how easily they can be reviewed, browsed, and searched (Kanare, 1985; Malone, 1983; Monty, 1990; Parunak, 1989; Treisman, 1982). Again, how accessible notes are, particularly in relation to their intended purpose, is a key factor in their ultimate usefulness. 2.2. An Ethnographic Study

Although the review of note taking just presented provides important clues as to why people take notes and what good notes are, it is not sufficiently detailed for identifying note takers' specific tasks and requirements. Therefore, an ethnographic field study—which looks at how a group of biologists at the University of Oregon incorporate note taking into their daily work practices—complements and extends the genera! empirical summary. This ethnography produced an abundance of specific information from an actual group of scientific note takers about how they take notes, why they take notes, what the components of their notes are, how they organize their notes, how they use their notes, and so forth. Scientific note takers, as opposed to general note takers, were the subject of this ethnography, and indeed of this entire research, for the following reasons: • The process of note taking is critical to scientists. • Using the Web for information gathering purposes has become an integral part of the scientific work process. • The Zebrafish Information Network (ZFIN) is a Web accessible network and relational database housing genetics information for the zebrafish and other organisms. ZFIN was developed and is currently maintained by the University of Oregon, the institution where this research was also conducted, and therefore provides a real-world application upon which a prototype e-notebook can be built. • The University of Oregon biologists were accessible for consultation and feedback during the application of UCD methodology. Methodology

The ethnographic field study reported here involved meeting over the course of a couple of weeks with 4 geneticists working in the biology department at the University of Oregon. Each geneticist was consulted individually at his or her normal

Reimer and Douglas

work space (as opposed to conducting the meetings and intervie'ws in a neutral location), which provided the unique opportunity of observing note taking practices from within. As the researcher and geneticist sat together in the geneticist's work space, each geneticist retrieved and pointed out notes from his or her particular storage location (notehooks on shelf, file cabinets, etc.)- In each case, they discussed the origination and composition of the notes, the components of the notes, the rationale for their organization, and the typical methods used for retrieval. In addition to the analysis of physical notebooks, some geneticists also demonstrated other software and archival systems that they frequently use. These software systems included digital imaging software, such as Adobe Photoshop, and image archives stored on optical disks. Although the biologists were not videotaped, findings were recorded in a notebook. Discoveries The biologists included in this ethnography, besides being researchers in the field of genetics, are highly educated and proficient with computers. In addition to using the software previously mentioned, the biologists were also active with e-mail and were generally familiar with spreadsheet applications and charting software. Also significant was the discovery that the biologists rarely collaborate on note taking and are possessive and guarded with their notes. This vigilance, which is common among the scientific community in general, is quite understandable, as notes are typically the only means a scientist has of proving his or her experimental results. As illustrated in Figure 1, the biologists routinely maintain the following notebooks: experimental notebooks, publication notebooks (or folders), protocol note-

; Protocol notebook i ' refcrencei

TriT el notebook ' dzifefences ' nuiflty tactual notes

MiiceUineoas netebsols • bven tor; notebook * Gene nap votAock

meeting notAook 3iie notebook • pagei of slidw * ldenbfiad by fUh nams • atE»fii2«i by project

FIGURE 1

Summary of biologists' notebooks.


409

books, group meeting notebooks, travel notebooks, and slide notebooks. A summary of the more interesting of these notebooks follows, along with a brief discussion of other miscellaneous archiving systems used by the biologists. The results of this ethnography expose evidence that supports the empirical literature review and, ultimately, provides a link between the current note taking habits of an actual community of users and the functional requirements for an e-notebook.

The experimental notebook. The experimental or process notebook is the most common type of notebook kept by the geneticists, and it contains a collection of notes that detail a variety of experiments. The notes, which are contained in three-ring binders, spiral wire bound notebooks, or traditional stitched lab notebooks, are generally delineated by project. Within each subsection, the notes are stored in chronological order, and the page titles are usually derived from the combination of the date (in YYMMDD format) that the experiment was conducted on plus an additional code if necessary for multiple processes (e.g., a page title of 97042501 indicates the first embryonic process conducted on April 25, 1997). Enumerated lists detail the step-by-step procedures that were followed for a particular process, and information such as sample names, references, protocols, probes, elapsed time, temperature, equations, results, and comments are carefully recorded. Free-form diagrams are often interspersed among the handwritten notes, as are post-it notes, photographs, computer-generated graphs and tables, notes from other people, and various other printouts. Many attachments found within the notebook pages, such as photographs or graphs, are further annotated by the note taker. Often, the spatial layout of a note page is utilized in a meaningful way. For example, multiple columns of numbers might be listed side by side to facilitate a natural left to right comparison of the data. Occasionally synopsis pages that describe the overall purpose, summary, and follow-ups for an experiment are created and displayed prior to the process notes. These synopsis pages are easily inserted into a three-ring binder but must be carefully anticipated by the note taker when using a spiral bound or stitched lab notebook. Finally, notes contain cross-references to related elements in other notebooks or archives. Because it can be tedious and sometimes impossible to insert certain objects (such as pictures) into the page of a physical notebook, cross-referencing becomes an important tool for maintaining continuity between a set of related notes and objects.

The publication notebook (folder). Most geneticists also keep notebooks or folders to organize material related to in-progress or anticipated publications. Ttiese folders serve as a catch-all for notes, diagrams, pictures, experimental results, related research, memos, e-mails, and rough drafts. The protocol notebook. The protocol notebooks that most geneticists maintain contain lists of proper steps that must be followed for various experiments, as well as a reference to the developer of the protocol. These protocols, which are

410

Reimer and Douglas

mostly typewritten and inserted into a three-ring binder, are sometimes organized according to the type of experiment they refer to. Even though many of the protocols are standard procedures, a central group protocol notebook is not kept by any of the labs. The slide notebook. The slide notebooks contain pictures of different fish, the names of which are written on the slide's outer casing. Some geneticists segregate the slides in these notebooks by project, and others have tabs within the notebook indicating different projects. Occasionally, photographs will also be found intermixed with the slides in these notebooks. Other miscellaneous archiving systems. In addition to the notebooks shown in Figure 1, many geneticists record various notes on to-do lists, desk calendars, post-it notes, and other random pieces of paper strewn about their work space. Optical disks, which store fish images (but no text) and are indexed by franiie, also provide an important medium for storage and retrieval. As mentioned previously, these frames are sometimes cross-referenced in experimental notebooks. These images are frequently downloaded from the optical disk to digital imaging software (e.g., Adobe Photoshop), where finishing touches are applied prior to publication.

2.3. Paradigm Shift? From Traditional Note Taking to lA In contrast to what is known about traditional note taking, little is understood about how people actually take notes from the Web. This is the result, perhaps, of inadequate software support. Prior to finalizing a list of functional requirements for a Web-based e-notebook, it is necessary to first consider how the process of note taking and the conriponents of notes may change when people move into the digital Web environment. It is only by recognizing these differences that Web note takers' tasks and requirements for support can be fully understood. Less Homogeneity and More Complexity One of the most significant differences between traditional notes and the notes that result from the process of IA on the Web lies with the components of those notes. In many respects. Web notes are less homogeneous and more complex than traditional notes because Web information is typically displayed as a variety of diverse element types. For example. Web pages routinely contain formatted text, images, imagemaps, lists, tables, hyperlinks, form fields, pull-down menus, and buttons. Notes referring to such a Web page could potentially consist of any or all of these element types. Furthermore, there are a variety of different ways in which users may wish to save such components in their e-notebooks, including statically.


411

dynamically, linked, or programmed. Suggested definitions for each of these save categories follow, and ultimately it is concluded that Web-based e-notebooks are potentially complex both in terms of the variety of elements they may contain and in the way in which those elements might be saved and used. Static. A static copy-and-save of Web components would reflect a snapshot of the information at a particular point in time and would not be affected by future modifications to those elements. The constantly changing nature of the Web makes the static recording of Web information critical; because users cannot rely on Web information's staying stable from a given moment to the next, they should be able to effectively capture this information and, in some instances, verify its original existence {i.e., published on a particular Web page—or URLât a particular date and time).

Dynamic. In a dynamic copy-and-save of Web components, all future changes to the information on the Web would be automatically reflected in the e-notebook. Users may use the dynamic copy as a way of monitoring current Web information without having to revisit pages periodically. The Internet Scrapbook application {Sugiura & Koseki, 1998) contains an automatic update function and provides a good example of how users can maintain a dynamic view of Web information.

Linked. Linked Web elements would retain underlying hypertext information in the e-notebook. For example, a list of hyperlinks copied from a Web page into an e-notebook would retain their functionality and still be active from within the notebook. Because much of what gives the Web its meaning can be attributed to its underlying hypertext, it is critical that users be able to preserve this structure. Programmed. Programmed elements that might be copied from a Web page and saved in an e-notebook include applets. Javascript functions, and search forms. For example, a copied and pasted search form would retain its functionality and could be activated from within the e-notebook. In this case, the original Web site database would be searched, and the results would appear either in the Web browser window or in the notebook, depending on user preference.

Tracking Work Processes Becomes Criticai

Another major difference between traditional note taking and IA is an increased focus on the users' ability to represent and track their ongoing work processes. During traditional note taking, it is relatively easy for a note taker to remember and resume work after the process has been suspended. Cues and techniques—such as putting a reviewed or "looked-through" pile of books or papers off to one side of a

412

Reimer and Douglas

desk or leaving books open to the current page—can assist the note taker. However, when users conduct IA on the Web, it is much more difficult for them to note where they are in their current work process. The dynamic nature of the Web {whereby pages are constantly changing), along with its complex underlying structure, exacerbates the problem. It is awkward, confusing, and cognitively overwhelming to keep multiple Web pages open simultaneously, and network connections are often dropped unexpectedly. Some of the most telling evidence that Web users need support for their ongoing work processes—-and that this functionality is currently lacking in standard Web browsers—appears in a study conducted by Abrams, Baecker, and Chignell (1998). In their study, Abrams et al. analyzed 322 survey forms and 50 bookmark files in an attempt to discover key elements of bookmarking behavior. A review of user quotes from the survey forms indicates that, in addition to using bookmarks for their designed purpose {i.e., to mark significant pages for future accesses), these users also bookmark pages as a way of representing and saving their long-term work activities. For example, one participant said that bookmarks tell "what I was doing over a period of many browsing sessions. I can keep track of what I was doing lately and a few weeks earlier" {p. 43), while another looked at bookmarked pages "as a history of using the Web" {p. 42). Abrams et al. concluded that "in the absence of such {inter-session history) functionality, users are bookmarking pages to enable access to previous browsing sessions" {p. 43). Even more evidence that tracking previous work processes is a critical function for many Web users can be found in another notable study, this one conducted by Tauscher and Greenberg {1997). In their research, Tauscher and Greenberg analyzed 6 weeks of Web usage data from 23 users. One of their key discoveries was that there is generally a high number of page revisits per individual {a 58% recurrence rate). Perhaps one of the reasons why users return to previously visited Web pages so frequently is because those pages are part of a longer term work process; this, in him, suggests that these same users would greatly benefit from tools that help them track, remember, and rejoin their ongoing work.

A Mixed Bag

With the transition from traditional note taking to IA on the Web, there are many aspects of note taking that computer use may facilitate. For example, e-notebooks are advantageous for almost all aspects of organizing and cross-referencing notes. Users should be able to insert and delete pages in their e-notebooks, move sections of text around, create headings and subheadings, section off groups of notes, and create outline views. Many of these same functions, however, can be quite difficult to accomplish with ink and bound notebooks, particularly those tasks that involve moving notes around and changing the initial structure of a set of notes. In addition, as users begin to accumulate an increasing volume of personal notes (a process that may well be facilitated by effective e-notebooks), linking those notes together via cross-references may become essential for efficient organization and retrieval.


413

However, although there are numerous examples of how note taking may improve in the electronic environment, some aspects of note taking that people have come to depend on may also be lost. For example, it may be more difficult to integrate nondigital elements (e.g., pictures and memos) with electronic notes. Certain note takers, such as the biologists showcased in the ethnographic field study, not only routinely staple or paste these types of elements into their notebooks but they often annotate them as well. It is known from other research that electronic annotation alone is a complex and difficult function to achieve effectively (Chang, Mackinlay, Zellweger, & Igarashi, 1998). Drawing free-form diagrams may also prove more difficult in an e-notebook than in a traditional paper notebook. Finally, other intangible elements that traditional notes might reflect—such as cognitive cues and change over time—are lost given the homogeneity of electronic notes. For example, accidental cues such as a coffee stain that helps the note taker remember the situation of recording the notes and incidental cues such as worn paper edges that reflect heavy use (Monty, 1990) are unachievable in electronic note taking.

2,4. IA Functional Requirements for an E-Notebook

Using what has been learned from the studies presented thus far—â literature review, an ethnographic field study, and an analysis of how note taking changes in the Web environment—it is now possible to identify a detailed list of functional requirements for a Web-based e-notebook designed to support IA. These requirements are presented in Table 1. It should be noted that some functions have been omitted from this list because they are not the highest priority for the first version of an e-notebook implementation. For example, whereas it is crucial that users be able to copy and paste text, images, lists, tables, and hyperlinks from the Web into their e-notebooks, it is not as important initially that they be able to gather imagemaps, forms, pull-down menus, and buttons. Other noticeable omissions from Table 1 include the ability of users to create specialized spatial layouts in their e-notes and to generate free-form diagrams. The next section takes a closer look at the tasks listed in Table 1 to see how well each of them is supported by existing software. This review includes a scenario-based observational usability study involving the use of standard software applications and a heuristic evaluation of other Web-based e-notebooks. 3. SUPPORT FOR iNFORMATION ASSIMILATION: WHERE ARE WE NOW?

Having reviewed the process of traditional note taking from an empirical perspective as well as by conducting an ethnographic field study, and by speculating about how note taking becomes IA on the Web, I now take the next logical step, which is to examine the current level of software support for the process of IA. This section presents a detailed scenario-based observational usability study that shows how scientists complete key IA tasks using currently available software and a heuristic review of other Web-based e-notebooks.

Table 1: Functional Requirements for a Web-Based E-Notebook Designed to Support lA Tasks Users Should he Able to Perform

Gather 1. Copy and paste text (both plain and formatted) statically from multiple, disparate Web pages into an e-notebook while retaining formatting. 2. Copy and paste images statically from Web pages into an e-notebook while retaining formatting. 3. Copy and paste lists and tables statically from Web pages into an e-notebook while retaining formatting. 4. Copy and paste hyperlinks from the Web into an e-notebook while retaining formatting and functionality (i.e., hyperlinks should remain "active" in e-notebook). 5. Archive Web information by having the URL, date, and time of the original source information automatically included in their e-notes. Users should not be able to modify the source or the authentication stamp for such archived information. EeHt 6. Delete any content from their e-notebooks, including original Web elements. 7. Modify (change text, format text, etc.) any content in their e-notebooks (except images), including original Web elements. Annotate 8. Add text to or delete text from their e-notebooks. 9. Emphasize or differentiate text in their e-notebooks by choosing between different font styles (e.g., bold, italic, underline) and sizes, 10. Create automatic cross-references (i.e., links) from one section of their e-notes to another section of their e-notes. 11. Create automatic cross-references (i.e., links) from their e-notes to any Web page. Organize 12. Have multiple pages in their e-nofebooks and copy Web information into any page. 13. Move text (plain and formatted) around in their e-notebooks while retaining formatting. 14. Move images around in their e-notebooks. 15. Move lists and tables around in their e-notebooks while retaining formatting. 16. Move hyperlinks around in their e-notebooks while retaining formatting and functionality. 17. Create separations between groups of notes. 18. Name, insert, and delete e-notebook pages. Save 19. Save text (plain and formatted) in their e-notebook while retaining formatting. 20. Save images in their e-notebook. 21. Save lists and tables in their e-notebook while retaining formatting. 22. Save hyperlinks in their e-notebook while retaining formatting and functionality. 23. Save archived Web information in their e-notebook. Track ongoing work 24. Track an ongoing Web work process in their e-notebooks so that they can easily remember the work they were doing at a later time. 25. Track their current progress in an ongoing Web work process (i.e., users should be able to see how much of their initial work goals they have completed, and they should be able to gauge how much work is outstanding). 26. Annotate an ongoing work process. 27. Edit an ongoing work process (e.g., delete some portion of it, insert text into it, etc.). 28. Restart and rejoin an ongoing Web work process from within their e-notebooks with minimal repeated work (i.e., users should not have to relocate Web pages of importance).

414


415

3.1. Scenario-Based Observational Usability Study To assess the current state of software support, a scenario-based observational usability study analyzed how a group of biologists were able to complete critical IA tasks using the software currently available to them. For the following reasons, this usability study focuses on how biologists—as opposed to general Web users—use the ZFIN Web site: Note taking is critical to scientists, biologists at the University of Oregon routinely use the ZFIN Web site, necessary access to the ZFIN implementation was provided, and the biologists were available for testing and feedback.

ZFIN

The ZFIN Web site (http://zfin.org/ZFIN/) and associated relational database is a multimedia repository of genetics information related to the zebrafish species. ZFIN was developed by a group of biologists and computer scientists resident at the University of Oregon, and it now supports an international community of researchers interested in isolating and understanding the effects of particular genes on the development of the zebrafish. ZFIN includes access to the following types of data and information: fish, genes, genomics, publications, people, labs, and companies. Unregistered guests are allowed to searcb and browse the database, and registered users can also update certain information in the database.

Procedure Ten geneticists in the University of Oregon's biology department participated in this usability study. In an initial overview, participants were told the following: • They would be given two ZFIN-related research scenarios to complete. • Their goal was to use tbeir typical note taking software tools to create a set of electronic notes to support the research tasks. • There were no constraints as to which applications they could use, or how many. • Their notes had to be electronic and could not include printouts or handwritten notes. Each session took between 60 and 120 min per participant. All participants were video- and audio-taped as they worked through the given tasks on an individual basis. A portable 8 mm camcorder with built-in microphone was used. This camcorder was set up on a tripod in the normal work space of each participant, and its lens was focused on the computer screen. After each session concluded, the video-audio tape was rerecorded onto a VHS tape to facilitate later analysis. The observer remained in the same room as the participants to monitor time and the video cameras, but participants were reminded that once the session began, the ob-

416

Reimerand Douglas

server would not be able to answer any questions or to help them in any way until they finished both scenarios. Tasks The first research scenario given to the biologists was composed of five tasks, and it focused on having the users create a set of Web-based notes. Participants were asked to locate specific information in ZFIN and then to record and edit that information in their electronic notes. These tasks were chosen to encompass as many IA functions as realistically possible. For example, as illustrated in Table 2, Task 1 asks participants to locate the Amemiya lab page in ZFIN and then to record in their notes the lab name, address, members, and what the members look like. From an IA perspective, this task was designed to test how well participants could gather and save plain and formatted text, an image, lists, and tables from the Web in their notes. Task 2 tested how well participants could gather and save a subset of a list and active hyperlinks, and so on. The fourth column in Table 2 cross-references the primary IA goals for each task with the IA functional requirement numbers previously listed in Table 1 and shows that many of the original IA requirements are covered during the usability study. The second research scenario (labeled Tasks 6-10 in Table 2) focused on how users were able to track an ongoing Web work process. Participants were asked to locate four different ZFIN pages and then to record and save the following work process information about each page: the page title, URL, date and time of visit and an annotation.

Results As seen in Table 2, participants had mixed degrees of success depending on the task. Column 5 shows the success percentage per task (e.g., 9 out of 10 participants—or 90%—were successful with Substep 1 of Task 1, etc.). In general, the analysis favored success rather than failure, as noted with a qualified success symbol and explained by the comments column. All participants used relatively fast computers (ranging from 266-750 MHz) containing moderately current software applications (e.g., Netscape Version 4.5 and above, Microsoft Word 98 arid above, etc.), which indicates that task failure was not linked to outdated resources but is evidence of a more deeply rooted problem. In summary, the usability study results indicate that with current software, the following is true: • It is relatively easy for users to copy-paste plain text from the Web into e-notes. • It is difficult to capture-save images from the Web. • Hyperlinks get lost when copied and pasted from the Web. • It is difficult to create links to Web pages from the e-notes. • Maintaining work process flow notes is tedious and error prone.

01 £

&. 3 •

^

. -

^ 7 ^

bC 60 TO

^

E

•5

z

o

ra

>H in u

in •K _a> •

a!

-^

c

Z

a DC 0) 0)

s u (O •o

c

ra

C .£ g c S ^

m

p

'u

fX,

:s

c

u c

I-

>^

li,

. oj Q>

1^

^ c g ^ ^ ra ra

ra T3

a < N o

^

:= .2 ^ 3

T3

ra . Cu .-2

73

ra ra ra

ra

S 13 .r:

60

£ ,a. 5 c£ N t= x: S •^ S £ iS

4 E

•Si

"^

qj .-: T3

"P "S S -^ E

•

417

418

Reimer and Douglas

This analysis indicates that a number of critical IA tasks are currently difficult to perform with existing software and that users would benefit from new tools that help them with IA. The next section takes a look at how well other Web-based e-notebooks allow users to accomplish key IA tasks. 3.2. Heuristic Evaluation of Other Web Notebooks

A number of other Web-based systems that at least partially include functionality we deem necessary for an e-notebook have been developed in the past decade. The Hunter Gatherer (HG) interface (schraefel, Zhu, Modjeska, Wigdor, & Zhao, 2002) allows users to create collections of Web page components (e.g., paragraphs of text, images). The URL of the source page is automatically transferred along with the component, and users can edit their collections by sorting, adding, deleting, and renaming components. HG is undoubtably a useful tool for Web users, and it does appear to support IA in many critical ways. However, in HG the collected components remain dynamic, so that each time a page is loaded, the most current information from the source Web page is displayed. This suggests that users are unable to edit the content of the components they collect, which is a critical eiemen of the IA process. The HG developers acknowledge that saving "static" inforn:\ation is also an important feature, and they state that their second prototype does have a "save" option; however, it's not clear yet whether or not HG users will be able to edit that static information. The Nabbit prototype (Manber, 1997), whose primary purpose is to allow users to create new individual pages from collected Web information, does support the process of IA to some extent. Users can copy and paste text {both formatted and plain), images, lists, tables, and hyperlinks from other Web pages into their personal pages, and they can annotate any of their copied and pasted selections. Nabbit automatically inserts the source of the copied-pasted information and the date and time of the copy. However, it is unclear whether users can edit pages once they have been created, modify their own annotations, or add notes to an existing page without having to repeat the copy and past process. Although Nabbit seems useful as a tool for collectign Web information, it ultimately fails to support a user's need to assimilate that information by not allowing him or her to modify it or integrate it with other notes. Nabbit also does not appear to provide any special functionality that helps users track ongoing work processes. The main functionality of the Internet Scrapbook is to automatically refresh Web selections that users have copied and pasted into their own personal Web pages. Like Nabbit, the Internet Scrapbook supports IA in that users can gather and save text, images, lists, tables, and hyperlinks from the Web. However, the Internet Scrapbook lacks edit and annotation functionality, it does not help users track their ongoing work processes, and it stores Web information as individual pages rather than as an integrated notebook. WebBooks (Card, Robertson, & York, 1996) allow users to gather and organize multiple Web pages from different locations and to store these pages together as


419

one cohesive unit or book. Users can generate WebBooks in a variety of ways, including from bookmark lists, search results, relative URLs on a Web page, etc. Unlike Nabbit and the Internet Scrapbook, WebBooks are designed based on a book metaphor (i.e., users can "flip" through the pages, and the 3D graphic images closely resemble physical books). Despite its novel design and impressive graphical appearance, however, this application fails to support IA because users cannot further personalize or modify the information in a WebBook. Furthermore, WebBook pages are not saved locally on a user's system but instead are reloaded from the Internet each time they are accessed (making it n:\ore the case that the structure of WebBooks, rather than their actual content, can be saved), users cannot collect portions of existing Web pages (only entire pages), and there are no special tools for tracking ongoing work processes. The evidence presented in this section is crucial for motivating the need to develop tools that better support the tasks of I A. The scenario-based observational usability study indicates that a number of critical IA tasks are not well supported by current software. The heuristic evaluation of other Web-based applications is not meant to find fault with the overall design and implementation of these systems, but merely to point out that they were developed with different design goals in mind (other than to support IA). The overall result of these studies, though, is that we cannot expect users to conduct their critical Web-based IA tasks using a variety of piecemeal tools that fail to form a cohesive package, may not be readily accessible, work inconsistently, and are not designed for their specific note taking needs.

4. NETNOTES: A WEB-BASED E-NOTEBOOK THAT SUPPORTS IA

The studies presented in Sections 2 and 3, which provide strong evidence to indicate that the process of IAis critical to many Web users and that better tools are needed to support it, laid the groundwork for the development of NetNotes, a Web-based e-notebook designed specifically to support the process of IA. Because implen:\enting a fully functional general use Web-based e-notebook (i.e., one that can be used in conjunction with any Web site and for all Web users) poses significant technical challenges, the initial version of NetNotes is focused on solving a subset of IA-related issues. In particular, the NetNotes prototype does the following: • It works in connection with a handful of pages in a specific Web domain (ZFIN). • It provides for a subset of the highest priority IA requirements. • It deals with a limited number of static, dynamic, and linked Web components (no programmed elements). Although NetNotes represents only a limited solution, it successfully incorporates a number of key IA requirements, and it is robust enough to be used in future experiments. The remainder of this section describes NetNotes' user interface and functionality.

420

Reimer and Douglas

4.1. Functional Requirements

The functionality incorporated into NetNotes is based on the list of requirements previously identified and shown in Table 1 (see Section 2). A handful of functional requirements have been omitted from the initial version of NetNotes, including the users' ability to organize their e-notes (i.e., move formatted text, images, lists, tables, and hyperlinks around within a notes page) and to create cross-references within their e-notes. These particular requirements have not been implemented because they are least interesting in terms of technical and design challenges and because most of them are already commonly found in word processing software. However, these functions are still considered to be of high priority and should be included in any future robust software system designed to support lA. It should also be noted that images can only be saved dynamically in NetNotes (i.e., if the image is modified on the Web source page, it also changes in NetNotes), and notebook pages can be deleted only using system file management applications.

4.2. User interface and Functionaiity

The functionality implemented in NetNotes naturally fell into three different categories: note taking functionality, functionality to track an ongoing Web work process, and functionality to archive Web pages. These functionality subsets are reflected in the interface by three primary tabs: the Notes tab, the Work Process tab, and the Archive tab.

Notes Tab

When opened, the NetNotes program defaults to the Notes tab, the section of the notebook where users generate most of their Web-based notes. The functionality available from this section of the notebook, as well as the appearance of the user interface (see Figure 2), is quite similar to other standard word processing programs (e.g., Microsoft Word), although much simpler. This was done purposefully to capitalize on knowledge transfer and to make NetNotes easier to learn. The first pull-down menu in the Notes tab, labeled NotesPage, as well as the first three icons on the toolbar, allows users to create a Nezv page of notes, to Open an existing page of notes, and to Save a page of notes. The Edit pull-down menu contains the standard Cut, Copy, and Paste commands, which are also represented via toolbar buttons. When one is using these commands within the notebook only (i.e., not interacting with ZFIN), they work as expected with plain text, but not with other formatted elements. When users copy selections from the Web and then paste them into their notes page, formatted text, images, lists, tables, and active hyperlinks are transferred correctly. Because the most interesting challenge in developing NetNotes from both a technical and a design perspective—as well as one of the motivating factors behind its conception—has to do with the copying and


421

Eisen, Judith S.

F*3t IMII!" 6^M^

1S(L

•w nnir, imicm rd

•iiamiiii ml

nuBn

uuukul1

^

FIGURE 2 The notes section of NetNotes. pasting of information from ZFIN (i.e., the Web) into the notebook, more details of this interprocess communication follow. When users wish to copy and paste elements from ZFIN into NetNotes, they simply perform the following steps: (a) Use the mouse to select the elements in ZFIN they want to copy, (b) choose the Netscape copy command, and (c) select the NetNotes paste command. Figure 3 illustrates an example of this interaction. The upper screen shot shows a number of noncontiguous ZFIN selections as viewed in Netscape, and the lower screen shot shows how these selections appear in NetNotes after they have been copied and pasted (each selection is copied and pasted individually). NetNotes successfully handles the copying-pasting of text (plain and formatted), images, lists, tables, and active hyperlinks. The Format pull-down menu in the Notes tab contains commands that allow users to alter the Size of their notes (Regular or Heading), to change the Style of their notes {Italic, Bold, or Underline), and to change the Alignment of their notes (Left,

Center, or Right). Although the intention was not to create another word processing program complete with many of the same standard functions found in software such as Microsoft Word, it was important to offer users enough variety in NetNotes so that they could differentiate and highlight their notes as needed (this was an important finding from both the ethnographic study and from the literature review presented earlier). All formatting commands can be used on all notes regardless of whether the user created them manually (i.e., typed them in) or copied them from somewhere else (such as the Web). The Link pull-down menu contains a command called Insert Web Link that allows users to insert a hyperlink (i.e., cross-reference) to a Web page from the notes page. The Link to Web URL text box that appears in the subsequent dialog automatically contains the URL of the current Web (ZFIN) page. Users are also prompted to enter the Text of link name, which can later be edited directly in the notes. The Notes toolbar also contains two Page Mode radio buttons. Edit and View, which are used to deal with hyperlinks. When the page is in edit mode, users can edit the name of hyperlinks (either links copied from the Web or ones created directly in the notes). When the page is in view mode, users can click on hyperlinks.

422

Reimer and Douglas

N i b n p c I* ZFIN Copr »