Dealing With Task Interruptions in Complex Dynamic Environments: Are Two Heads Better Than One? Sébastien Tremblay, and François Vachon, Université Laval, Québec, Canada, Daniel Lafond, Defence R&D Canada—Valcartier, Québec, Canada, and Chelsea Kramer, Université Laval, Québec, Canada, and CAE Professional Services, Ottawa, Canada Objective: This study examined whether teaming up mitigates individual vulnerability to task interruptions in complex dynamic situations. Background: Omnipresent in everyday multitasking environments, task interruptions are usually detrimental to individual performance. This is particularly crucial in dynamic command and control (C2) safety-critical contexts because of the additional challenge imposed by the continually evolving situation during the interruption. Method: We employed a firefighting microworld to simulate C2 in the context of supervisory control to examine the relative impact of interruptions on participants working in a functional dyad versus operators working alone. Results: Although task interruption was detrimental to participants’ efficacy of monitoring resources, the negative impact of interruption was reduced for those working in teams. Teaming up translated into faster resumption time, but only if both teammates were interrupted simultaneously. Interrupting only one team member was associated with increased postinterruption communications and slower resumption time. Conclusion: These findings suggest that in complex dynamic situations working in a small team confers more resistance to task interruption than working alone by virtue of the reduced individual workload typical of teamwork. The benefit of collaborative work seems nevertheless mediated by the coordination and communication overhead associated with teamwork. Application: The present findings have practical implications for operators dealing with unexpected events such as task interruptions in C2 environments. Keywords: task interruption, teamwork, shared workload, complex dynamic situations, microworld, command and control Address correspondence to Sébastien Tremblay, École de psychologie, Université Laval, Pavillon Félix-AntoineSavard, 2325, rue des Bibliothèques, Québec, Québec, Canada, G1V 0A6; e-mail:
[email protected]. HUMAN FACTORS Vol. 54, No. 1, February 2012, pp. 70-83 DOI:10.1177/0018720811424896 Copyright © 2012, Human Factors and Ergonomics Society.
Introduction
Task interruptions that affect human behavior are omnipresent in many work domains. Indeed, interruptions are usually detrimental to complex or cognitively demanding tasks (Speier, Vessey, & Valacich, 2003). Given that individuals typically have to perform another task during the interruption, task interruptions are often said to require the management of multiple task goals that interfere and compete with each other for gaining control of action (Altmann & Trafton, 2002). Hitherto, most experimental research on interruption has dealt with static task environments in which no change occurred during the interruption. Dynamic environments, however, pose the additional challenge of dealing with situations that evolve during the interruption. Hence, in addition to recovering the state and goals of the interrupted task (Altmann & Trafton, 2002), the resumption of an interrupted dynamic task involves finding or inferring the changes that occurred during the interruption from the current state of the situation as well as understanding the potential consequences of those changes (St. John & Smallman, 2008). Such recovery requires a significant reassessment of the situation and a comparison with its last known state, which can lead to failure in situation awareness (SA) and impaired task performance (St. John, Smallman, & Manes, 2005). This is particularly crucial in safety-critical contexts such as command and control (C2) as the inability to deal effectively with interruptions in these complex dynamic domains can put operations and people at risk (McFarlane & Latorella, 2002; McGillis-Hall et al., 2009). In C2 domains, such as firefighting, intensive care units, crisis management, and military operations, cognitive work is complicated by severe constraints of time pressure, uncertainty, and
task interruptions 71
information overload (see Brehmer, 2007), which make these situations particularly susceptible to interruptions. The present study sought to explore the factors that modulate how C2 operators cope with task interruptions by examining whether teaming up renders an operator less vulnerable to interruptions in complex dynamic situations. In C2 situations, the expertise and cognitive resources required for the successful achievement of the mission generally extend beyond the capability of a single individual because of the inherently high levels of complexity, pressure, and risk (Wickens, Mavor, & McGee, 1997). Teaming up in such contexts is expected to facilitate individual work given that, among others, workload is shared across team members, therefore reducing the cognitive burden of each operator. In fact, groups as information processing systems are considered to have more total processing capacity than individuals (Van Merriënboer & Sweller, 2005). However, the mere act of bringing individuals together to accomplish interdependent tasks does not guarantee team effectiveness (e.g., Allen & Hecht, 2004; Steiner, 1972). Indeed, numerous factors can potentially influence the effectiveness of a team, such as task context (e.g., environmental stressors; Devine, 2002), the nature of the task (Stewart & Barrick, 2000), or the personal characteristics of the team members (Stevens & Campion, 1994). Team effectiveness also depends on team structure, that is, the organization of the roles, tasks, and resources for dealing with a situation (see Lafond, Jobidon, Aubé, & Tremblay, 2011). Different team configurations may impose various team requirements and affect individual workload. For instance, whereas in a divisional team structure, each member possesses the skills and resources required to autonomously achieve the mission goal, functional teams are composed of members occupying a specialized function so that teammates are highly interdependent and must collaborate to accomplish the task (e.g., Diedrich et al., 2002). The high task interdependency inherent to functional teams (see Salas, Cooke, & Rosen, 2008) inevitably comes at a cost associated with the need to distribute information, resources, and taskload
among team members (MacMillan, Entin, & Serfaty, 2004). Thus, coordination and communication are fundamental to functional teams (Lafond et al., 2011). In C2 situations, such a need for coordination and communication is particularly great, which may exacerbate teamwork requirements to the point of counteracting the benefits for an operator working in a team (e.g., process loss; see Steiner, 1972). Hence, the functional team structure is particularly interesting for the purpose of the present study. Interruptions to a safety-critical C2 task can modify the pace and demands of the situation (Huey & Wickens, 1993), testing the ability of individuals and teams to adapt during the execution of their tasks (LePine, 2005). This requires the coordination of fast decision making and actions, adding to individual cognitive burden. Yet the detrimental effects of interruptions can lead to poor coordination among team members and to execution errors (Talbot, 2004). Although there is a growing interest in interruption of teamwork in safety-critical domains (Grundgeiger, Sanderson, MacDougall, & Venkatesh, 2010; McGillis-Hall et al., 2009; Miller, 2004; Rukab, Johnson-Throop, Malin, & Zhang, 2004), it is still unclear whether the benefits (e.g., shared workload) and costs (e.g., the need for communication and coordination) of teamwork bestow more or less resistance to interruptions compared to individual work in C2 environments. This study examined whether an individual, when placed under complex dynamic contexts, can cope better with task interruptions when working in a functional team than when working alone. We employed a task environment mimicking the high-level aspects of real-world situations— a so-called microworld—to simulate C2 in a complex dynamic environment. Microworlds offer the significant advantages of experimental manipulation and control, without stripping away the complexity and the dynamic nature of the task (Brehmer & Dörner, 1993). The C3Fire microworld (Granlund, 1998) is a functional simulation of C2 operations in the firefighting domain involving time pressure and uncertainty and requiring dynamic decision making. In C3Fire, participants manage and coordinate various types of intervention units to control the
72
forest fire spread. C3Fire can produce flexible individual or team-based scenarios in which unexpected events can be embedded. C2 in C3Fire can be considered as a form of distributed supervisory control (Shattuck & Woods, 2000; Sheridan, 1992) where remote operators (human participants) supervise multiple local resources (intervention units) to control a dynamic process. Supervisory control requires multiple activities that are managed by one or more individuals whose orchestration is key to success: Preconditions (e.g., availability of intervention resources) need to be met, and priorities and constraints (e.g., temporal, causal) must be considered (Jones & Goyle, 1992). Hence, participants in C3Fire act as supervisory controllers of several resources by evaluating resources’ state, planning their actions, and executing the planned actions. In this study, the impact of interruptions on C2 effectiveness was compared between individuals acting alone and individuals working in functional dyads. We employed two types of resources (or intervention units) in the C3Fire operational environment—firefighters (FFs) and water tankers (WTs)—associated with two roles: extinguishing fires and water provisioning, respectively. This configuration requires the coordination of water refills between the FF and WT units. The number of units remained the same in every scenario regardless of whether the mission had to be accomplished by one or two individuals. In functional dyads, each team member fulfilled a specific role (firefighting or water supplying). Team members were highly interdependent and had to work together to accomplish the task. Although dyadic participants have the advantage of a reduced workload over individual participants, their high level of interdependency may nevertheless induce a coordination/communication overhead (MacMillan et al., 2004). Yet research on the influence of communication on performance is mixed. Although communication frequency has been positively correlated with team performance (Sexton & Helmreich, 1999), Cannon-Bowers, Salas, Blickensderfer, and Bowers (1998) instead observed a negative correlation, showing that well-performing teams reduce their communication flow under high
February 2012 - Human Factors
workload (also see Volpe, Cannon-Bowers, Salas, & Spector, 1996). To determine whether this interplay between benefits and costs of teamwork renders an individual more or less vulnerable to take interruption in complex dynamic situations, we contrasted a condition (individual) in which a single participant in charge of all intervention units was momentarily interrupted with two functional team conditions. Because interruptions in team contexts usually engage one individual at a time (see Rukab et al., 2004), we implemented a team condition (team/half-interrupted) in which only one team member was interrupted, either the participant in charge of the FFs or the one monitoring the WTs. However, this implied that half of the units were still actively supervised during the interruption. To avoid any team superiority effect to be interpreted as a mere artifact of the imbalance in the proportion of nonsupervised (i.e., interrupted) units, we added a second team condition (team/all-interrupted) in which both teammates were interrupted and, consequently, all units were unsupervised simultaneously. Method Participants
A total of 50 Université Laval students, who reported normal or corrected-to-normal vision and normal hearing, took part in the study. Participants were randomly assigned to an individual or team condition: 10 participants performed the experiment as single operators, whereas the others were paired to form 20 dyads, randomly allocated to one of the two team conditions. Materials
Participants performed computerized forest firefighting mission scenarios simulated by the C3Fire microworld. The C3Fire interface consists of a geospatial map, displayed on a 40 × 40 cell grid (see bottom of Figure 1), built up by a set of four interacting simulation layers. The fire layer defines four different states for each cell of the map, represented by a color code: on fire (red), extinguished (brown), burned out (brown), or clear (no color). The geographical objects layer corresponds to the different types
task interruptions 73
Figure 1. Illustration of the three conditions tested. The left panel illustrates the individual condition in which a single participant controls four firefighting (FF) units and four water tanker (WT) units. The two other panels depict the team conditions in which one participant is in charge of the FFs and another participant is in charge of the WTs. In the team/half-interrupted condition (middle panel), only one of the two participants is interrupted in each scenario, whereas in the team/all-interrupted condition (right panel), both participants are interrupted at the same time.
of physical entities displayed on the map (plain, lake, pine, birch, or house). The content of a cell directly influences its ignition time. The weather layer determines the strength and direction of the wind. The stronger the wind, the faster the fire spreads to neighboring cells. Finally, the unit layer refers to the intervention units (i.e., the resources) under the control of the participants. There were four FFs and four WTs, each represented by a numbered icon. Each type of unit has a specific role: FFs extinguish fire by moving to a burning cell while WTs supply water to FFs by moving to an adjacent cell (excluding diagonal cells). FFs could hold enough water to extinguish one fire cell. WTs could carry enough water to refill two FFs. Afterward, WTs had to refill at a lake. These settings made sure that there was a frequent need to coordinate the water-refill process between FFs and WTs throughout the task. To deploy a unit, participants have to left click on the unit and drag it to the desired destination.
Throughout each scenario, the Morae software (TechSmith) recorded every event that occurred in the microworld (e.g., keystrokes). Team members were located in distinct cubicles and communicated via TeamSpeak software (TeamSpeak Systems). Team members could speak to each other via headsets by holding down the appropriate key. Design and Procedure
The three conditions were contrasted in a between-subjects design (see Figure 1). In all conditions the goal was the same—that is, to fight forest fires and save as many houses as possible—and the resources available to achieve this were identical. In the individual condition, a single participant was monitoring the eight intervention units available, being responsible for both roles of firefighting and water supplying. The two team conditions (team/half-interrupted and team/all-interrupted) each consisted of 10 pairs of participants working in collaboration using a
74
February 2012 - Human Factors
Table 1: Characteristic of the Three Experimental Conditions of the Present Study
Condition Individual Team/half-interrupted Team/all-interrupted
No. of Participants
No. of Interrupted Units/Participant Participants
1 2 2
functional distribution of labor: One participant was in charge of the four FFs while the other member controlled the four WTs. The level of remaining water for each unit the participant was supervising was continually displayed to the left of the geospatial map. However, each team member was unaware of the resource status of his or her teammate, forcing team members to communicate and coordinate their actions to accomplish the task and solve problems, such as when an empty FF required a water supply or a WT was running out of water. Each dyad member was randomly assigned to either the FF station or the WT station located in separate cubicles, whereas the single participant performed both functions at the same station. Participants were told that their mission was to prevent houses from igniting, to limit the fire from spreading, and to extinguish burning houses. After the completion of a familiarization scenario guided by the experimenter, participants completed two training scenarios followed by four 10-min counterbalanced test scenarios designed to be equivalent. Although the starting location of the units and the fire was the same for all scenarios, the houses, forests, and lakes were displayed at preset starting positions that varied across scenarios. The strength and the direction of the wind were predetermined for each scenario and remained constant throughout the scenario. Every scenario was interrupted at some point for 20 s. The moment of the interruption was designed to be unpredictable by randomly varying the onset of the interruption across scenarios from 5:00 to 7:15. The scenario continued to evolve throughout the interruption. Although all units in the individual and team/all-interrupted conditions momentarily ceased to be actively
8 4 4
% of Unsupervised Units During the Interruption
1 1 2
100 50 100
supervised (because all supervisors were interrupted), only half the units were not supervised during the interruption in the team/half-interrupted condition (see Table 1). In the team/halfinterrupted condition, the interruption was alternated across scenarios between the team member in charge of the FFs and the one monitoring the WTs so that each member was interrupted two times during the test scenarios. The noninterrupted team member was not aware that his or her teammate was interrupted and continued to carry on the task during the interruption. During the interruption, the map disappeared, communications were disabled, and each interrupted participant was presented with two questions regarding the state of the situation at the time of the interruption. These queries simulated requests of information for an external authority. When the map reappeared, the interrupted participant had to not only recover the task state and goals of the C3Fire simulation but also assess the “new” situation—that is, detect and understand any changes that occurred during the interruption (e.g., fire spreading, wind direction change) and determine the state of all resources (e.g., location on the map, mobilized or not, water status)— define new priorities based on such assessment, and engage actions (e.g., unit mobilization) as a function of the new priorities. In the team conditions, this task-resumption process could be supported at all levels through communication with the teammate. Results
To examine how an individual working in a team or alone coped with interruptions in complex dynamic environments, we extracted indexes of supervisory control, performance, and the
task interruptions 75
(Total time - IdleFF+WT) Total Time
Figure 2. Proportion of time of effective monitoring as a function of time interval around the interruption for individual participants, team/half-interrupted noninterrupted participants, team/half-interrupted interrupted participants, team/all-interrupted interrupted participants in charge of the firefighters (FF), and team/ all-interrupted interrupted participants monitoring water tankers (WT). The negative sign refers to preinterruption intervals, whereas the positive sign refers to postinterruption intervals. Error bars represent 95% within-subjects confidence intervals calculated separately for each type of participant status.
resumption lag from test scenarios. We also examined communication frequency, content, and strategy after the interruption. Supervisory Control
Supervisory control was assessed through the concept of monitoring effectiveness (ME), which refers to the ability to identify idle units in a timely manner and give them a new movement order (see Lafond et al., 2011). We defined an idle unit as an inactive unit that could nevertheless work, that is, an inactive FF with a full water supply or an inactive WT with an empty water supply. In these two situations, the inactivity of a unit cannot be attributed to the wait that is often necessary to coordinate the refill process. The total time that units spent in an idle state was an indication of the lack of ME. Therefore, our ME measure was the proportion of time a unit spent in an effective monitoring (or nonidle) state,
,
(1)
where Total time is the duration of the time window of interest and IdleFF+WT is the mean idle time per unit averaged across FFs and WTs. Computing the level of ME per unit allows the comparison across conditions, regardless of the number of units for which each participant was responsible. ME was analyzed over time, anchored by interruption. More specifically, ME was assessed during three 30-s intervals around the interruption: 30 s before the interruption (–30 s), the first postinterruption 30 s (+30 s), and the following 30 s (+60 s). We contrasted ME across the five types of participant status in the study, yielding a 3 × 5 mixed design with time interval around the interruption (−30, +30, and +60 s) as the within-subjects factor and status (interrupted individual, interrupted in half-interrupted team, noninterrupted in halfinterrupted team, interrupted FF monitor in allinterrupted team, interrupted WT monitor in all-interrupted team) as the between-subjects factor. These data are plotted in Figure 2. A mixed ANOVA revealed significant main effects of status, F(4, 45) = 5.04, p = .002, and time interval, F(2, 90) = 26.17, p < .001. More important, a significant interaction arose, F(8, 90) = 4.67, p < .001, in part because the effect of time interval was significant for all statuses (Fs > 4.52, ps < .03) except for the noninterrupted participants (F < 1). Specifically, although no difference in ME was found across statuses in the −30-s interval (p = .353), interrupted participants, whether in a team or not, showed a decrease in ME within the +30-s interval compared to noninterrupted participants (ps < .01). However, this decrease was larger for individual participants than interrupted team participants (ps < .005). Moreover, individual participants spent significantly less time in an effective monitoring state in the +60-s interval than noninterrupted participants and team/allinterrupted participants (ps < .05), suggesting that individual participants did not recover from the interruption as well as the interrupted team participants. Interrupted team members showed the same pattern of ME in both team conditions.
76
February 2012 - Human Factors
Performance
Any performance measure should take into account the goals of house protection and fire suppression. Because the number of houses programmed to ignite before the interruption was not sufficient to extract any reliable metric about house protection, the performance index focused on the fire-suppression goal and was thus assessed through the number of extinguished cells. Performance reflected the combined effort of all resources at play, meaning the specific contribution of each participant was impossible to extract and so the performance index was examined at a global level. Given the 120 s for cells to burn out, it was impossible to assess performance within 30-s intervals as for ME. Moreover, analysis of performance during the postinterruption period was complicated by the fact that the spreading rate of the fire increased nonlinearly over the course of each scenario, and, consequently, postinterruption performance greatly depends on preinterruption performance. For instance, a poor performance (i.e., a small number of extinguished cells) before the interruption is likely to translate into an even worst performance after the interruption given that uncontrolled—that is, larger— fires tend to spread more rapidly than do more controlled—that is, smaller—fires. Because of such dependence, any analysis of postinterruption performance would provide at best incomplete information about the impact of the interruption, ignoring, for example, the state of the situation before the interruption occurred. Hence, we contrasted performance before any interruption to that for the whole scenario to provide a more complete picture of interruption effects on performance. Indeed, such an analysis provides a context in which analyzing postinterruption performance, that is, the difference between the whole-scenario performance and the preinterruption performance. We reasoned that if one condition was more sensitive to the interruption than another, the comparative pattern of results observed for the whole scenario would differ from that obtained before the interruption. The mean number of extinguished cells, plotted in Figure 3, was analyzed using a 2 × 3 mixed ANOVA with period (preinterruption vs. whole scenario) as the within-subjects factor and
Figure 3. Mean number of extinguished cells before the interruption and during the whole scenario for the three conditions. Error bars represent 95% withinsubjects confidence intervals calculated separately for each condition.
condition (individual, team/half-interrupted, team/ all-interrupted) as the between-subjects factor. The main effect of period was significant, F(1, 27) = 493.90, p < .001, but that of condition was not significant, F(2, 27) < 1, p = .406. More important, the interaction was significant, F(2, 27) = 4.70, p = .018, because the increase in extinguished cells from preinterruption to whole scenario was significantly smaller for individual participants than team participants (ps < .022), suggesting indirectly that team performance was less affected by the interruption than that of individual participants. The impact of interruption on performance was similar for both team conditions (p = .691). Resumption Lag
We computed the resumption lag as the interval from the offset of the interruption to the first moving order given by an interrupted participant to an intervention unit. (Although a wellaccepted measure in the interruption literature, the resumption lag does not provide a precise indication of the processes at play during the seconds preceding the first postinterruption order. One way to determine the extent to which the resumption lag reflect a real “lag” in task processing would be to compare it to the mean time between two orders. However, this average varies too much throughout each scenario to extract a meaningful index against which we could contrast the resumption lag.
task interruptions 77
Figure 4. Mean resumption lag (in s) in each condition. Error bars represent 95% between-subjects confidence intervals.
The rate at which orders are made by the participants highly depends on what is going on at that moment in the scenario. This is why we cannot take the time before the first action at the beginning of each scenario as a point of comparison; there is not much going on at the beginning of the scenario compared to the middle-end of it, that is, in the interruption time zone. Hence, the time to evaluate the situation and plan actions is not likely to be comparable at the beginning of the scenario versus the resumption of the task following an interruption.) A one-way ANOVA performed on the mean resumption lag shown in Figure 4 revealed an effect of condition that was significant using a more generous alpha level, F(2, 27) = 3.17, p = .058. Indeed, team/all-interrupted participants resumed the primary task faster than did interrupted participants in individual and team/ half-interrupted conditions (ps < .05). Communications
In an attempt to understand the difference in resumption time between the two team conditions, we examined communications around the interruption with a specific interest in frequency, content, and strategy. One team from the team/ half-interrupted condition was removed from all communication analyses because of the scarcity
of their communications. The same pattern of results was nevertheless observed when all teams were included in the analysis. Communication frequency within the 4 min around the interruption was analyzed over 30-s time intervals. This yielded four preinterruption intervals—from −120 to −90 s (−120), from −90 to –60 s (−90), from −60 to –30 s (−60), and from −30 s to the interruption (−30)—and four postinterruption intervals—from the interruption to +30 s (+30), from +30 to +60 s (+60), from +60 to +90 s (+90), and from +90 to +120 s (+120). The mean number of communications is plotted in Figure 5 as a function of time interval for both team conditions. A 2 × 8 repeatedmeasures ANOVA performed on these data revealed no main effect of condition or time interval (Fs < 1, ps > .60), but the interaction was significant, F(7, 119) = 3.12, p = .005. This interaction arose largely because there were significantly more communications in the +30-s interval than in all others but the +90-s interval (all ps < .05) in the team/half-interrupted condition, but no effect of time interval in the team/ all-interrupted condition, F(7, 63) = 1.13, p = .357. This increase in communications right after the interruption in the team/half-interrupted condition appeared to be related to the slower resumption time found in that condition relative to the
78
February 2012 - Human Factors
FF: OK. . . . I need water. Your Unit 4 is closer to my trucks. WT: My Unit 4, hum, it’s gonna be long. Hum. I’ll send my Unit 5 instead. But it’s far away. FF: The only one with water is my Unit 2. I just moved it.
Figure 5. Mean number of communications between teammates in the team/half-interrupted and team/ all-interrupted conditions as a function of the time interval (in s) around the interruption (t0, illustrated by the dotted central line). Each interval lasted 30 s. Negative time values refer to preinterruption intervals, and positive time values refer to posinterruption intervals. Error bars represent 95% within-subjects confidence intervals calculated separately for each condition.
team/all-interrupted condition. Indeed, we found a significant positive correlation between postinterruption communication frequency and resumption time in both team/half-interrupted (r = .39, p = .01) and team/all-interrupted (r = .42, p = .005) conditions. So as communications between teammates became more frequent following the interruption, participants required more time to resume the task. We turn now to examine communication content to understand this pattern of communication frequency. First, analysis of postinterruption communication content did not reveal any particular strategy teammates would have used to facilitate interruption recovery. The following is a representative example of the (translated from French) stream of communication between an interrupted participant (in charge of the WTs) and a noninterrupted participant (in charge of the FFs) following an interruption: WT: I’m coming now. I was answering some questions.
The only difference in communication content between the two team conditions arose when considering the first postinterruption communication. In the team/half-interrupted condition, 33% of first postinterruption communications were related to the interruption itself (e.g., “Sorry, I’m back from some questions”), whereas team/all-interrupted teammates never spoke to each other about the interruption. This type of communication may have ensued from the fact that the interrupted participant realized when coming back from the interruption that his or her noninterrupted partner was still working during this time (some units were active on the map) and that he or she may need to be informed about his or her new (no-more-interrupted) status. Hence, such interruption-related communications are likely to have contributed to the functional postinterruption coordination of the team. The difference in resumption lag between the two team conditions may also be the result of the communication strategy employed when resuming the task. In both conditions, resumption lag was longer and postinterruption communications were more frequent when the first communication occurred before rather than after an interrupted participant committed his or her first postinterruption action. In other words, communicating before acting was associated with longer recovery and more postinterruption communications. This early-communication strategy was adopted more often by teammates in the team/half-interrupted condition, probably because of the need for interrupted team member to inform his or her teammate about his or her new noninterrupted status. Discussion
The objective of this research was to assess the impact of task interruptions on individuals working either in a small team or alone in a
task interruptions 79
simulated firefighting C2 operation. Interruptionbased analyses of supervisory control and performance measures revealed a benefit of teamwork over individual work when dealing with interruptions in complex dynamic environments. Although interrupted participants had a temporarily reduced ability to manage multiple resources, those working alone appeared more vulnerable to the negative impact of the interruption. Here, the concept of two heads being better than one is not so surprising; nonetheless, our study highlights some factors that underpin team resilience and that may augment or reduce the beneficial effects of teamwork. The fact that a team superiority effect was found even when all operators were interrupted suggests that reducing individual workload through the distribution of responsibilities across team members can help an operator facing complex dynamic task interruptions. The factor that distinguished individual operators from operators working in dyads was that each of the former was in charge of twice the intervention resources than each of the latter. Moreover, individual participants had to deal with resources playing different roles, which was not the case for team participants. St. John and Smallman (2008) proposed that recovery from an interruption in complex dynamic work environments involves processes such as retrieving the primary task goal structure and the primary task state prior to the interruption (also see Altmann & Trafton, 2002) as well as detecting and understanding any changes that occurred during the interruption. It seems plausible that these processes, particularly postinterruption change detection and situation or resource state evaluation, may have been facilitated by the reduced workload associated with teamwork. The observed benefit of shared workload on interruption recovery also translated into a faster resumption time. More precisely, paired participants resumed the firefighting task more rapidly than individual participants—probably because the unit status information to assess and update following the interruption was cut by half—but only when both teammates were interrupted. This result is surprising as it
suggests that interrupting the whole dyad leads to faster recovery than interrupting only one of its members. However, a close examination of the communication pattern suggests that some aspects of interruption recovery, such as resumption time, may be mediated by the coordination and communication requirement associated with collaborative work (MacMillan et al., 2004; Rosen, Fiore, Salas, Letsky, & Warner, 2008). Indeed, the team condition that committed more postinterruption communications was the same that showed slower resumption times. The increased postinterruption communication frequency found only in the team/half-interrupted condition may have ensued, at least in part, from the need for the interrupted participant to update his or her status and align SA with the (noninterrupted) team member who was not necessarily aware that his or her teammate was temporarily occupied with other functions. In the team/all-interrupted condition, such an update was not necessary as both teammates shared the same knowledge about the interruption. Such a reduced need for communicating could be associated with a reduced workload for recovering SA (cf. MacMillan et al., 2004), which in turn could speed up the recovery process. In the task interruption literature, resumption time is often assumed to be an index of interruption recovery (Hodgetts & Jones, 2006; Monk, Boehm-Davis, & Trafton, 2004); that is, once participants have restarted the primary task they are assumed to have recovered from the interruption. However, the current results suggest that speed of recovery is no guarantee of quality of recovery; despite showing similar recovery of contextual information after the interruption, as reflected by ME, the two team conditions nevertheless differed in resumption time. Instead, our results provide evidence that interruption recovery may represent a longer process that extends beyond the first postinterruption action. Consistently, Altmann and Trafton (2007) demonstrated, in a static context, that interruption recovery typically unfolds over a series of postinterruption actions, with interaction time approaching baseline in a curvilinear fashion rather than instantaneously. Such a multiple-step recovery process is especially plausible in
80
dynamic contexts where the reconstruction of the preinterruption mental context required to perform the primary task is complicated by the fact that the situation has evolved during the interruption and that any significant changes must then be detected and their consequences understood to fully recover (St. John & Smallman, 2008). Thus, it would seem that in dynamic tasks, the resumption time may not be such a good indicator of interruption recovery. The present study constitutes a first step toward understanding how small teams in C2 environments respond to unexpected interruptions. Indeed, the generalization of the current findings would benefit from further work considering other levels of the factors that were fixed in the present context, that is, the nature and complexity of the primary task and the interrupting task, the access to information about the situation during the interruption, the number of teammates, or the length of the interruption studied herein. In fact, changing any of these may influence the way C2 teams deal with interruptions. Team-related factors should also be considered. Indeed, our results suggest that team vulnerability to interruptions is likely to be moderated by factors influencing the workload associated with the communication overhead, such as the number of team members (Arrow & McGrath, 1995) or team structure (Jobidon, Breton, Rousseau, & Tremblay, 2006; Lafond et al., 2011). For instance, interruption resilience could be increased through the implementation of a team configuration that minimizes the need for coordination and/or communication, such as a divisional team structure in which team members are distributed geographically and each operator possesses the skills and/or resources to accomplish the task autonomously (e.g., Jobidon et al., 2006). Yet the trade-offs among team organization, workload, and communication can be very complex and prone to counterintuitive effects (e.g., Naikar, Pearce, Drumm, & Sanderson, 2003). A noteworthy aspect of teamwork is that it may constitute in itself another potential source for interruptions (Chong & Siino, 2006; Rukab et al., 2004) and raises the question as to whether different communication structures may be more efficient than others in terms of the costs or benefits
February 2012 - Human Factors
of such interruptions. Systematic research in more complex, realistic settings, such as microworlds, may help identify how to reap the full benefits of teamwork in complex dynamic situations and how to minimize the impact of disruptive events such as task interruptions. Practical Implications
Based on the present findings, we believe that collaborative work may grant more resistance to interruptions because teams allow responsibilities to be distributed across team members, resulting in a parallel resumption of task work that must be achieved sequentially by individuals. Accordingly, we hypothesize that any effective means of lowering cognitive workload should help C2 operators deal with task interruptions more efficiently, at least up to a certain point. Another key finding suggests that if one team member is unaware that another team member’s primary function has been interrupted, then after the interruption the team members may need to update each other, which appears associated with increased communication overhead and slower recovery. To contain such an overhead, technological systems could be used to display the operational status of every team member (see, e.g., Fitzgerald et al., 2011), especially when operators work in remote locations. For example, Parush et al. (2011) examined team communication during open-heart surgeries, reporting that 49% of SA-related communication in the operating room was susceptible to information loss. The findings were used to derive requirements for an augmentative information display based on the team’s common information requirements to support SA. Accordingly, another potential implication of this research would be the development of methods to enhance teams’ resistance to interruptions through communication training. Team communication supports the knowledge building and information processing that lead to the construction of the team SA (Endsley & Jones, 2001; Shu & Furuta, 2005) which, when shared, may help team members resume task after the interruption. Training based on the maintenance of team SA would require a standardized articulation of each individual’s activity so that every team member
task interruptions 81
can synchronize his or her actions with those of the others to reach a collectively shared goal (Garbis & Artman, 2004). Such a solution, however, comes with a cost as it could reduce discretionary activity, which is also important for effective adaptation. Yet by integrating the known occurrence of team task interruptions to training-based communication strategies such as frequent status updates and predefined task resumption strategies, team members could be better prepared and able to recover from task interruptions. Acknowledgments Thanks are due to Rego Granlund for assistance in using C3Fire and to Jean-François Gagnon and Sébastien Walsh for running the experiment. This work was supported by the Natural Sciences and Engineering Research Council of Canada and by Defense R&D Canada—Valcartier.
Key Points •• Dealing with task interruptions in complex dynamic contexts, such as command and control, is particularly challenging because the situation continues to evolve during the interruption. •• Working in small teams seems to promote better interruption recovery relative to individual work by virtue of the reduced individual workload when responsibilities are distributed across team members. •• The benefit of collaborative work appears mediated by the coordination and communication requirements associated with teamwork. •• In complex dynamic tasks, resumption time is not necessarily a good indicator of quality of recovery.
References Allen, N. J., & Hecht, T. D. (2004). The “romance of teams”: Toward an understanding of its psychological underpinnings and implications. Journal of Occupational and Organizational Psychology, 77, 439–461. Altmann, E. M., & Trafton, J. G. (2002). Memory for goals: An activation-based model. Cognitive Science, 26, 39–83. Altmann, E. M., & Trafton, J. G. (2007). Timecourse of recovery from task interruption: Data and a model. Psychonomic Bulletin & Review, 14, 1079–1084. Arrow, H., & McGrath, J. E. (1995). Membership dynamics in groups at work: A theoretical framework. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 17, pp. 373–411). Greenwich, CT: JAI.
Brehmer, B. (2007). Understanding the functions of C2 is key to progress. International Journal of C2, 1, 211–232. Brehmer, B., & Dörner, D. (1993). Experiments with computersimulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9, 171–184. Cannon-Bowers, J. A., Salas, E., Blickensderfer, E., & Bowers, C. A. (1998). The impact of cross-training and workload on team functioning: A replication and extension of initial findings. Human Factors, 40, 92–101. Chong, J., & Siino, R. (2006). Interruptions on software teams: A comparison of paired and solo programmers. In Proceedings of the ACM conference on computer supported cooperative work (pp. 29–38). New York, NY: Association for Computing Machinery. Devine, D. J. (2002). A review and integration of classification systems relevant to teams in organizations. Group Dynamics: Theory, Research, and Practice, 6, 291–310. Diedrich, F. J., Hocevar, S. P., Entin, E. E., Hutchins, S. G., Kemple, W. G., & Kleinman, D. L. (2002, June). Adaptive architectures for command and control: Toward an empirical evaluation of organizational congruence and adaptation. Paper presented at the Command and Control Research and Technology Symposium, Monterey, CA. Endsley, M. R., & Jones, W. M. (2001). A model of inter- and intrateam situation awareness: Implications for design, training and measurement. In M. McNeese, E. Salas, & M. Endsley (Eds.), New trends in cooperative activities: Understanding system dynamics in complex environments (pp. 46–67). Santa Monica, CA: Human Factors and Ergonomics Society. Fitzgerald, M., Cameron, P., Mackenzie, C., Farrow, N., Scicluna, P., Gocentas, R., Bystrzycki, A., Lee, G., O’Reilly, G., Andrianopoulos, N., Dziukas, L., Cooper, D. J., Silvers, A., Mori, A., Murray, A., Smith, S., Xiao, Y., Stub, D., McDermott, F. T., & Rosenfeld, J. V. (2011). Trauma resuscitation errors and computer-assisted decision support. Archives of Surgery, 146, 218–225. Garbis C., & Artman, H. (2004). Team situation awareness as communicative practices. In S. Banbury & S. Tremblay (Eds.), A cognitive approach to situation awareness: Theory and application (pp. 275–296). Hampshire, UK: Ashgate. Granlund, R. (1998). The C3Fire microworld. In Y. Waern (Ed.), Co-operative process management (pp. 91–101). London, UK: Taylor & Francis. Grundgeiger, T., Sanderson, P., MacDougall, H. G., & Venkatesh, B. (2010). Interruption management in the intensive care unit: Predicting resumption times and assessing distributed support. Journal of Experimental Psychology: Applied, 16, 317–334. Hodgetts, H. M., & Jones, D. M. (2006). Interruption of the Tower of London task: Support for a goal-activation approach. Journal of Experimental Psychology: General, 135, 103–115. Huey, B. M., & Wickens, C. D. (Eds.). (1993). Workload transition. Washington, DC: National Academy Press. Jobidon, M.-E., Breton, R., Rousseau, R., & Tremblay, S. (2006). Team response to workload transition: The role of team structure. In Proceedings of the Human Factors and Ergonomics Society 50th annual meeting (pp. 1769–1773). Santa Monica, CA: Human Factors and Ergonomics Society. Jones, P. M., & Goyle, V. (1992). Cooperative support for distributed supervisory control: Issues in modeling cooperative work in complex systems. In Proceedings of the 1992 IEEE international conference on systems, man, and cybernetics (Vol. 2, pp. 1214–1217). New York, NY: Institute of Electrical and Electronics Engineers.
82
Lafond, D., Jobidon, M.-E., Aubé, C., & Tremblay, S. (2011). Evidence of structure-specific teamwork requirements and implications for team design. Small Group Research. 42,507–535. LePine, J. A. (2005). Adaptation of teams in response to unforeseen change: Effects of goal difficulty and team composition in terms of cognitive ability and goal orientation. Journal of Applied Psychology, 90, 1153–1167. MacMillan, J., Entin, E. E., & Serfaty, D. (2004). Communication overhead: The hidden cost of team cognition. In E. Salas & S. M. Fiore (Eds.), Team cognition: Process and performance at the inter- and intra-individual level (pp. 61–82). Washington, DC: APA Press. McFarlane, D. C., & Latorella, K. A. (2002). The scope and importance of human interruption in human-computer interaction design. Human-Computer Interaction, 17, 1–61. McGillis-Hall, L., Pedersen, C., Hubley, P., Ptack, E., Hemingway,A., Watson, C., & Keatings, M. (2009). Interruptions and pediatric patient safety. Journal of Pediatric Nursing, 25, 167–175. Miller, S. (2004). Interruption processing in a decision-making task: Successful integration of interruptions and task resumption (Unpublished doctoral thesis). George Mason University, Fairfax, VA. Monk, C. A., Boehm-Davis, D. A., & Trafton, J. G. (2004). Recovering from interruptions: Implications for driver distraction research. Human Factors, 46, 650–663. Naikar, N., Pearce, B., Drumm, D., & Sanderson, P. M. (2003). Designing teams for first-of-a-kind, complex systems using the initial phases of cognitive work analysis: Case study. Human Factors, 45, 202–217. Parush, A., Kramer, C., Foster-Hunt, T., Momtahan, K., Hunter, A., & Sohmer, B. (2011). Communication and team situation awareness in the OR: Implications for augmentative information display. Journal of Biomedical Informatics, 44, 477–485. Rosen, M. A., Fiore, S. M., Salas, E., Letsky, M., & Warner, N. (2008). Tightly coupling cognition: Understanding how communication and awareness drive coordination in teams. International C2 Journal, 2, 1–30. Rukab, J. A., Johnson-Throop, K. A., Malin, J., & Zhang, J. (2004). A framework of interruptions in distributed team environments. Studies in Health Technology and Informatics, 107, 1282–1286. Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50, 540–547. Sexton, J. B., & Helmreich, R. L. (1999). Analyzing cockpit communication: The links between language, performance, error, and workload. In Proceedings of the tenth international symposium on aviation psychology (pp. 689–695). Columbus: Ohio State University. Shattuck, L. G., & Woods, D. D. (2000). Communication of intent in military command and control systems. In C. McCann & R. Pigeau (Eds.), The human in command: Exploring the modern military experience (pp. 279–292). New York, NY: Kluwer/ Plenum. Sheridan, T. B. (1992). Telerobotics, automation, and human supervisory control. Cambridge, MA: MIT Press. Shu, Y., & Furuta, K. (2005). An inference method of team situation awareness based on mutual awareness. Cognition, Technology and Work, 7, 272–287. Speier, C., Vessey, I., & Valacich, J. S. (2003). The effects of interruptions, task complexity, and information presentation on computer-supported decision-making performance. Decision Sciences, 34, 771–797.
February 2012 - Human Factors
Steiner, I. D. (1972). Group process and productivity. New York, NY: Academic Press. Stevens, M. J., & Campion, M. A. (1994). The knowledge, skills, and ability requirements for teamwork: Implications for human resource management. Journal of Management, 20, 503–530. Stewart, G. L., & Barrick, M. R. (2000). Team structure and performance: Assessing the mediating role of intrateam process and the moderating role of task type. Academy of Management Journal, 43, 135–148. St. John, M., & Smallman, H. S. (2008). Staying up to speed: Four design principles for maintaining and recovering situation awareness. Journal of Cognitive Engineering and Decision Making, 2, 118–139. St. John, M., Smallman, H. S., & Manes, D. I. (2005). Recovery from interruptions to a dynamic monitoring task: The beguiling utility of instant replay. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting (pp. 473–477). Santa Monica, CA: Human Factors and Ergonomics Society. Talbot, D. (2004, November). How technology failed in Iraq. Technology Review. Retrieved from http://www.technologyreview .com/computing/13893 Van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17, 147–177. Volpe, C. E., Cannon-Bowers, J. A., Salas, E., & Spector, P. E. (1996). The impact of cross-training on team functioning: An empirical investigation. Human Factors, 38, 87–100. Wickens, C. D., Mavor, A. S., & McGee, J. P. (Eds.). (1997). Flight to the future: Human factors in air traffic control. Washington, DC: National Academy Press.
Sébastien Tremblay is currently a professor at the School of Psychology, Université Laval, Québec, Canada. He is also honorary research fellow of Cardiff University (UK) and director of the GR3C, a group of researchers interested in collaborative work and team cognition. He has expertise in a wide range of cognitive human factors issues. He holds a PhD in psychology (1999, Cardiff University). François Vachon is an assistant professor at the School of Psychology, Université Laval. His main research interests include the basic and applied cognitive psychology of attention and multitasking. He received his PhD in cognitive psychology from Université Laval in 2007. He then completed a postdoctoral fellowship in cognitive psychology at Cardiff University (2007), one in cognitive neuroscience at Université de Montréal, Canada (2008– 2009), and another in human factors at Université Laval (2009–2011). Daniel Lafond is a visiting fellow at Defence R&D Canada–Valcartier. He was previously a postdoctoral researcher in the applied cognition laboratory of Professor Tremblay (2008, Université Laval). His
task interruptions 83
research interests include cognitive engineering and decision making, teamwork, and mathematical modeling. He holds a PhD in psychology (2007, Université Laval).
Services, Ottawa. She received her MA in experimental psychology from Carleton University, Ottawa, Canada, in 2009. Her main research interests include team cognition in complex work domains.
Chelsea Kramer is a PhD student at Université Laval and a human factors consultant at CAE Professional
Date received: April 13, 2011 Date accepted: September 5, 2011
