What characteristics of an email message predict a users action on that message? ... help people manage their email, such as spam filters or automatic.
To reply or not to reply: Predicting action on an email message Laura Dabbish, Robert Kraut, Susan Fussell, and Sara Kiesler Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213
{dabbish, robert.kraut, susan.fussell, kiesler}@cs.cmu.edu ABSTRACT What characteristics of an email message predict a users action on that message? Participants in a survey of university faculty, staff, and students provided data on the characteristics of new email messages and their actions based on the messages. Statistical analyses of responses revealed several factors that were important in predicting the fate of a message. These were: importance of a message, number of recipients, sender characteristics, and the nature of the message content. Factors influencing user perception of message importance were also examined. Important messages were from high communication frequency work contacts requesting action, providing a status update, or scheduling a meeting.
Categories and Subject Descriptors H.5.3 [Information Interfaces And Presentation]: Group and Organization Interfaces – computer supported cooperative work, synchronous interaction
General Terms Management, Measurement, Design, Human Factors,
Keywords Electronic mail, Email, filtering, computer-mediated communication, messaging, intelligent agents
1. INTRODUCTION Ask any working professional if they feel like they get too much email, and the answer will almost always be yes. E-mail is by far the most common and popular form of computer-mediated communication. Furthermore, because email has been so widely adopted for communication within organizations, people are receiving an increasing amount of email on the job [7]. Peter Denning, started noticing the problem as early as 1982, when he talked about “the receiver’s plight” [5]. This paper addresses the
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference’04, Month 1–2, 2004, City, State, Country. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.
problem of insufficient time to reply to email messages. To inform the design of technology to alleviate this problem, the study described sought to understand decision rules and strategies people use to reply to email messages or to delete them. Previous research on email usage has repeatedly cited the information overload associated with the mass amount of email workers tend to receive [11,20,21,24]. In addition to growth in the volume of email that people receive, email is being used for an increasingly broad range of purposes. Workers use email to exchange information and social pleasantries, to assign responsibility and manage tasks, and as an information store [6,11,23]. In response to this increased email volume and the wider diversity in tasks for which email is used, substantial new technology has been developed to help people work with email. More efficient search engines [22], advanced interfaces for navigating contacts [14], and email interfaces designed around task management [3], are all aimed at helping people deal with the deluge of information and communication coming their way. In addition, researchers have attempted to characterize and develop tools to combat junk email containing advertisements (spam) [4,18]. Most previous work on email management describes at a general level the functions email serves and the problems associated with email overload. For example, several studies have focused on how people save their email, what purposes it serves for them, and its importance as a tool for coordination in the workplace [6,10,20, 21,22,24]. And researchers have even cited the problem of ‘organizational spam’ gratuitous messages sent to a large amount of individuals within a company [24]. In the current study, we build on this previous work by looking carefully at the decision rules people use in dealing with particular messages. Few previous studies have examined closely how people choose to reply to email messages or to save or delete them. Nor have researchers looked closely at the decision rules people use to decide to retain or delete a message. Analysis of email-related behavior as a function of message and user characteristics is essential for building theoretical models of email use. It is also important for the development of automated tools to help people manage their email, such as spam filters or automatic filing agents, that must attempt to approximate these decisions. The research described in this paper is designed to uncover the strategies, or deliberate choices people make in managing their email. Identifying decision rules people use could provide insight
for the design of email systems: intelligent agents could identify messages prime for deletion, or prioritize certain messages to receive attention first (as in [9]). In the next section we review the previous literature on email usage. We then present a model of the main functions of email in current organizational contexts, and how these functions relate to important characteristics of e-mail messages. In the remainder of the paper, we describe an email survey where participants submitted data on the characteristics of new email messages in their inbox, and their actions based on those messages. We provide a statistical analysis of user responses and the findings of this analysis as they relate to the questions posed. We conclude with the implications of this work for theories of email usage and the design of email applications. The data show that people use message characteristics in determining whether to reply to a message, but that message importance does not play a major role. They also show that people leave a large proportion of their email in their inbox, particularly message they need to respond to, but that importance of a message does not influence its retention.
2. PREVIOUS WORK Because email is one of the oldest uses of networked computers, and one of the most popular, social scientists have examined how people use it. Both qualitative studies and quantitative studies have examined how people use email systems and what purposes their email serves. Sproull and Kiesler [20] summed up much of the early work on the social and organizational aspects of email. Here we will focus on work about email and information management strategies, as well as research dedicated to alleviating the “email problem.”
2.1 Email as a task management tool As early as 1988, interview studies were looking at how email was being used in organizations. Sumner [21] interviewed and surveyed email users at an organization with an electronic email system in heavy use. She found that email was extensively displacing previous communication modalities, and warned that access to electronic mail systems might lead to information overload. Mackay [11] conducted a series of interviews examining the way that professional office workers used email to manage their daily work. She showed that electronic mail was far more than simply a communication system. Electronic mail also supported a variety of time management and task management activities. The interviews performed by Whittaker and Sidner [24] extended Mackay’s earlier work. They found that in addition to basic communication, email was “overloaded” in the sense of being used for a wide variety of tasks—communication, reminders, contact management, task management, and information storage. Ducheneaut and Bellotti [6] performed a study of email usage in three organizations and found as had previous studies, that email was being used for a wide variety of functions. In particular, they noted that people used emails as reminders for things they had to do and for task management more generally.
2.2 Individual differences in email handling Mackay also noted that the ways that people used email were highly diverse. She noted that people fell into one of two categories in handling their email: prioritizers or archivers.
Prioritizers were focused on managing messages as they came in, and kept a tight control of their inbox, whereas archivers focused on archiving information for later use, making sure they did not miss important messages. The interviews and observations by Whittaker and Sidner [24] examined how people handled the abundance of electronic mail they received. According to Whittaker and Sidner, people fell into one of three categories. People were either frequent filers who constantly cleaned their inbox, spring cleaners who cleaned their inbox once every few months, or no filers who didn’t clean up their inbox but used searching tools to manage it. Work towards identifying and comparing the different strategies for email management has continued. Extending Whittaker and Sidner [24], Bälter developed a mathematical model using keystroke-level analysis to examine the time necessary to use each organizational strategy [2].
2.3 Technologies to facilitate email handling More implementation-oriented research has attempted to design and implement e-mail systems that help people deal with their deluge of email and to better support the tasks it serves. Much of this work has focused on intelligently categorizing messages and determining what is important to the user. Neuwirth et al. [13] developed an electronic mail interface that was highly tasktailorable, allowing for increased visibility of the specific message elements a user deemed most important. Bellotti et al [3] focused on the task management function of email, and designed a system that supported the use of email messages as task reminders. TaskMaster [3] allows users to group their email specifically by its relationship to active tasks. Malone et al. [12] experimented with the ‘Information Lens’ an intelligent information sharing system centered around informational aspects of content, important to users. ContactMap [14] is focused on increasing the visibility of people and groups with relation to email. The problem now becomes knowing what aspect of email to make highly visible. Attempting to combine the elements of these kinds of interfaces only increases the overload problem. Machine learning and artificial intelligence techniques have also been applied to the email problem. Intelligent agent-based systems have been created to extract important content information from email, and create meaningful summaries for users [1,4]. As previously mentioned machine learning techniques have been aimed at reducing the junk-email problem, and intelligently filtering out “spam” [18]. Horvitz et al. have developed a system aimed at inferring the criticality of messages, and prioritizing email received [9]. The goal of such systems is to facilitate dealing with large numbers of emails in a short amount of time. But none of these systems have been entirely successful.
2.4 Message Importance Perceived importance of email messages may be influencing how people infer criticality and prioritization of particular messages in their email inbox. However, studies have not examined what aspects of a message factor into a user’s evaluation of its importance. Systems such as [9] that prioritize messages based on inferred importance assume that people want to see and reply to highly important messages instead of those of low importance. But these systems make assumptions as to what constitutes an important message. Currently there is no definitive association
between message characteristics and message importance as perceived by the user.
email messages will be related to scheduling and these messages will be more likely to receive a response.
Previous work and designs of systems such as Outlook or Eudora have assumed that by providing a priority field (which is often ignored [24]) users can more effectively manage their time using email by attending to important and critical messages first. The problem with this approach is that populating the priority field must be done manually, and thus is tedious to use . In addition, this field reflects only the priorities of the sender, providing even more reason for the recipient to disregard it. An intelligent system that could accurately infer message importance from email message attributes should help users immensely; they would be better able to efficiently direct their attention to critical messages.
-
To understand how a user evaluates importance of a message we looked at what message characteristics were associated with userperceived message importance. Knowing this association, an automated system could better assess the importance of a message and display that inferred importance to the user, or prioritize messages for viewing based on this importance similar to [9].
The purposes outlined above highlight some of the main reasons people exchange email messages in a work context, and suggest an appropriate length of time that a message should be retained, dependent on the life of the content within. These message functions may be shaped and defined by the characteristics of the message sender.
3. A MODEL OF EMAIL ACTIVITIES Based on our review of the literature, we developed a conceptual model of the main purposes that email serves in an organizational context. We identified four distinct purposes of email in organizational contexts: -
-
-
task delegation / project management / reminders Users frequently access their email inbox to read new messages. As Whittaker and Sidner [24] found, with repeated access the inbox acts as an external memory store with messages in view serving as reminders [3,6]. This purpose of email suggests the importance of the following message content types: action requests, status updates, and meeting and deadline reminders. Based on this previous work we predict that messages requesting an action, or providing a reminder will be left in the inbox rather than filed or deleted. information exchange, storage, and retrieval Often an email is sent in response to a question, request for a document or weblink, or a discussion. A message containing information, when deemed important by the recipient, can be stored for later retrieval. When such information is needed again the recipient must locate it in the archiving system they have set up. Previous work cites information archival as one of the major reasons users save messages [22,24]. This purpose of email suggests the following message content types: information requests, and information responses. We predict that information requests and responses will be less likely to be deleted by the user, and so more likely to be left in the inbox or filed. scheduling and planning As Sumner [21] noted, email is often used for scheduling or planning purposes. This could be to schedule a meeting, event, informal occasion, etc. Because email has largely taken the place of phone calls and memos [21] supporting scheduling and planning has become one of its main purposes. This purpose of email suggests the following message content types: meeting requests, and responses to meeting requests. Thus we predict that a high proportion of
informal communication As people access their email more frequently, users have come to expect instantaneous delivery, and rapid response to email communication. Though email communication is asynchronous, in many firms employees read and respond to their email throughout the day, and turnover is close to what might be expected for instant text messages. This purpose of suggests the importance of social content within a message. In addition, this type of rapid exchange suggests that informal messages, or messages with social content, may be more likely to receive a response.
By considering the different purposes that email serves in an organization we have identified 6 key message content elements. These are: action requests, status updates, reminders, information requests and responses, scheduling requests and responses, and social content. In addition we have made specific predictions about how these types of content should relate to the actions taken on a message. It is through the lens of these message types that we describe the data we collected on actual message handling behavior, specifically likelihood of response to a particular message.
4. METHOD To collect data on individuals’ actions with their email inbox we used a web-based survey completed over an internet browser.
4.1 Materials The survey itself was divided into several sections. We will focus on two of these sections. The first pertained to general email and communication behavior. These were questions about the number of email messages sent and received, number of messages in the email inbox, and general email habits. These questions were included to obtain a description of our response population with respect to email usage. The second section of the questionnaire asked each respondent to input detailed ratings of five new non-spam messages in their email inbox. For each of the five messages the respondents indicated the nature of the content (Fig. 1), the message importance, sender characteristics, and the action taken on the message (replied, plan to reply later, do not plan to reply), or what they did with the message (delete, file, or leave in Inbox) (Fig. 2). We will describe these measures in more detail below.
4.2 Measures 4.2.1 Message Importance We hypothesized that users evaluate message importance based on characteristics of the message (sender and content) and that this importance likely influences their action on the message. To assess the importance of a message to a user’s work we used a four item measure developed for this survey. The user was instructed to rate the following questions on a five point scale:
-
How important is this message for YOUR work?
-
How important is this message for the SENDER’s work?
-
How urgent are any deadlines associated with this message?
-
How much work does this message require of you?
These four items were combined to obtain a cumulative message importance score.
4.2.2 Sender Characteristics Because sender of the message was hypothesized to be important in determining the likelihood of response, for each message we obtained data on the characteristics of the message sender. To evaluate whether the sender was a work contact or not, we had the respondent select the sender’s role from a list including administrative assistant, co-worker, supervisor, friend etc. These responses were then coded into a binary variable with 1 for a work-related contact and 0 for non-work related contacts such as family, friends etc. Respondents also indicated communication frequency with the sender, and number of recipients on the message.
4.2.3 Message Content We also assessed the content of the message, because we had hypothesized certain message content from the purposes email serves in the organizational context. Particularly we were interested in task management and delegation, scheduling, information exchange, and informal communication. Table 1 lists the content items for each message. Content types were not mutually exclusive, for example a message could contain both a scheduling request and a piece of information. Finally we had respondents input their action on the message, because we wanted to predict reply action from message characteristics. For each message respondents had to indicate whether they deleted or filed a message, and whether they replied immediately or planned to later reply to a message. So, the survey respondents acted as classifiers, providing detailed categorization of messages in their inbox. The result was labeled message data. We used this data to examine the actions users take on specific messages as a function of message characteristics such as importance, urgency, sender relationship, content, etc. Table 1. Message content types. What was the content of this message, select yes for all that apply: -
Request for action.
-
Request for information (link, contact information, etc.) or a document (file, image, etc.)
-
Status update for an ongoing project or task
-
Request for a meeting or other communication with you, or response to a meeting request
-
Reminder for a meeting, event, or upcoming deadline
-
Social greeting or thank you
4.3 Participants The survey was sent via email to over 1300 individuals on a university department-wide distribution list. There was no special incentive offered for participation. We had roughtly a 10% response rate, with 121 individuals completing the survey in its
entirety. The survey took an average of 26.7 minutes to complete (standard deviation of 9.7 minutes). Out of the 124 respondents, 38 were professors or scientists (30.7%), 46 were students (37%), and 40 were other staff members, such as research programmers etc. (32.2%). Participants ranged in age from 20 to 57, with the average age being 30. A majority of the respondents were male (76%) reflecting the wider population demographic of the university department sampled.
5. RESULTS Our results indicate that sender and message content factors play a role in user’s perceptions of importance of a message. However, sender characteristics are key in predicting the probability of response to a message. We first present general statistics describing our response population with respect to email usage, and then proceed to discuss our models of importance and message response. We conclude by proposing a model of message response incorporating user’s perceptions of message importance and message characteristics..
5.1 Basic Email Statistics In order to obtain baseline statistics, we asked some basic questions about email usage. The results are summarized in Table 2 below. It is important to note that these data are from the entire sample, and that certain email usage data varied based on job role. Particularly, the professor/scientist portion of our sample reported reading significantly more messages per day than the students or other staff (F(2,118)=10.12, p
