Ergonomics Society of the Human Factors and Human Factors: The ...

Human Factors: The Journal of the Human Factors and Ergonomics Society http://hfs.sagepub.com/

Measuring Patterns in Team Interaction Sequences Using a Discrete Recurrence Approach Jamie C. Gorman, Nancy J. Cooke, Polemnia G. Amazeen and Shannon Fouse Human Factors: The Journal of the Human Factors and Ergonomics Society 2012 54: 503 originally published online 7 December 2011 DOI: 10.1177/0018720811426140 The online version of this article can be found at: http://hfs.sagepub.com/content/54/4/503

Published by: http://www.sagepublications.com

On behalf of:

Human Factors and Ergonomics Society

Additional services and information for Human Factors: The Journal of the Human Factors and Ergonomics Society can be found at: Email Alerts: http://hfs.sagepub.com/cgi/alerts Subscriptions: http://hfs.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

>> Version of Record - Jul 13, 2012 OnlineFirst Version of Record - Dec 7, 2011

Downloaded from hfs.sagepub.com at HFES-Human Factors and Ergonomics Society on October 10, 2012

What is This?

SPECIAL SECTION: Methods for the Analysis of Communication

Measuring Patterns in Team Interaction Sequences Using a Discrete Recurrence Approach Jamie C. Gorman, Texas Tech University, and Nancy J. Cooke, Polemnia G. Amazeen, and Shannon Fouse, Arizona State University

Objective: Recurrence-based measures of communication determinism and pattern information are described and validated using previously collected team interaction data. Background: Team coordination dynamics has revealed that “mixing” team membership can lead to flexible interaction processes, but keeping a team “intact” can lead to rigid interaction processes. We hypothesized that communication of intact teams would have greater determinism and higher pattern information compared to that of mixed teams. Method: Determinism and pattern information were measured from three-person Uninhabited Air Vehicle team communication sequences over a series of 40-minute missions. Because team members communicated using push-to-talk buttons, communication sequences were automatically generated during each mission. Results: The Composition × Mission determinism effect was significant. Intact teams’ determinism increased over missions, whereas mixed teams’ determinism did not change. Intact teams had significantly higher maximum pattern information than mixed teams. Conclusion: Results from these new communication analysis methods converge with content-based methods and support our hypotheses. Application: Because they are not content based, and because they are automatic and fast, these new methods may be amenable to real-time communication pattern analysis. Keywords: communication analysis, interaction analysis, pattern analysis, recurrence analysis, teamwork

Address correspondence to Jamie C. Gorman, Psychology Department, Texas Tech University, Lubbock, TX 79409; e-mail: [email protected]. HUMAN FACTORS Vol. 54, No. 4, August 2012, pp. 503-517 DOI:10.1177/0018720811426140 Copyright © 2012, Human Factors and Ergonomics Society.

Introduction Whether they are aware of it, people produce interaction patterns when they communicate to perform a team task (e.g., Bowers, Jentsch, Salas, & Braun, 1998; Fischer, McDonnell, & Orasanu, 2007; Kanki, Lozito, & Foushee, 1989; Miller, Scheinkestel, & Joseph, 2009; Pincus, Fox, Perez, Turner, & McGeehan, 2008; Sexton & Helmreich, 2000; Xiao, Seagull, Mackenzie, Klein, & Ziegert, 2008). If interaction patterns develop as people continue working together as a team (e.g., Achille, Schultz, & Schmidt-Nielson, 1995; Gorman, Foltz, Kiekel, Martin, & Cooke, 2003; Katz, 1982), then changes in team membership may be reflected in changes in the quantity and informational content of those patterns. Because team interaction patterns recur in time, often in discontinuous bursts and lulls of interactivity (Huberman & Glance, 1998), nonlinear dynamics may provide appropriate tools for quantifying pattern changes. With this in mind, we describe the measurement of communication determinism drawn from recurrence analysis for nonlinear time series (Webber & Zbilut, 1994) and a measure of communication pattern information drawn from information theory (Shannon & Weaver, 1949). To demonstrate these measures, we analyze communication data from an experiment in which members of three-person teams either changed or stayed the same following a retention interval (Gorman & Cooke, 2011; Gorman et al., 2006). Predictions based on prior results from that experiment, which used continuously scaled, content-coded measures of team coordination dynamics (Gorman, Amazeen, & Cooke, 2010), are made to evaluate the concurrent validity of the new measures. Furthermore, we propose that the discrete recurrence methods described later are amenable to the analysis of


504

August 2012 - Human Factors

Figure 1. A recurrence plot for 1,000 samples (100 Hz) of a well-known nonlinear (chaotic) dynamical system, the Lorenz mask: The recurrence plot of the underlying time series (left) after phase-space reconstruction (using 4,000 samples; right) according to “Dimension” and “Delay” parameter settings (shown above the recurrence plot). The minor diagonal of the recurrence plot is the reconstructed series plotted against itself (at i = j); moving up from the minor diagonal, dots corresponding to recurrent points are plotted at each x(i,j) where x(j) is within a “Threshold” of x(i). Diagonals of recurrent points correspond to time-shifted recurrent patterns. (These plots were created using the Cross Recurrence Plot Toolbox; Marwan et al., 2007.)

discrete team interaction sequences, whose data may be collected and analyzed in real time. Discrete Recurrence Measure of Communication Determinism

We measure communication determinism by applying recurrence analysis to discrete team interaction sequences, although it is typically applied to nonlinear time series (Marwan, Romano, Thiel, & Kurths, 2007). Whereas a time series consists of observations sampled at a regular time interval from a process continuous in both amplitude (variability) and time; by contrast, a discrete sequence is continuous in neither variability nor time. By discrete we mean (team interaction) states sampled from a nominal set of mutually exclusive codes (e.g., which team member is speaking), and by

sequence we mean the codes are ordered in time, but the exact onset or duration times of the individual codes are not used (Quera, 2008). (Later, we describe the analysis of a pattern, by which we mean a short array of ordered codes that may or may not be representative of an overall interaction sequence.) Whether a time series or discrete sequence is analyzed, the basis for recurrence analysis is a recurrence plot (RP). We start, then, by outlining the general concept of determinism in recurrence plots for time series, leading up to our calculations using discrete team interaction sequences. RPs for time series. For a time series x of length N, an RP is an N × N (symmetric) matrix where, if the value of x(j) is sufficiently close (within a threshold) to the value of x(i), then a


Measuring Patterns

in

Team Interaction 505

Figure 2. Simulated three-code sequences of length N = 50 and their recurrence plots for deterministic (left), random (middle), and mixed deterministic-random (right) sequences (DET = determinism).

dot is plotted at x(i,j) (Eckmann, Oliffson Kamphorst, & Ruelle, 1987; see Figure 1). The RP of a time series x, therefore, represents all pairwise combinations i,j that are sufficiently close, over all time scales, using a dot. Note that the minor diagonal in Figure 1 (from the bottomleft corner to the upper-right corner) is completely filled in because it is the one-to-one plot of the series against itself, at i = j. Because the plot is symmetrical, we analyze only the upper triangle. As we move toward the upper-left corner, away from the minor diagonal, the dots represent time-shifted points of recurrence in the time series. Recurrent points forming unbroken diagonals in the upper triangle are time-shifted, recurrent patterns of the time series. Recurrence analysis proceeds by quantifying recurrent patterns of the plot (Webber & Zbilut, 1994). Determinism (DET), one of the quantifications extracted from the plot, is the percentage of points forming diagonals to all recurrent points in the upper triangle: DET =

# recurrent points forming diagonals × 100 (1) total # of recurrent points

This measure ranges from zero to 100%, such that a time series that never repeats has

determinism zero and a time series that perfectly repeats has determinism 100. For example, the underlying time series for the RP in Figure 1, which was generated using a deterministic mathematical equation, has DET = 97.78. For noisy, real-world complex systems, however, we expect DET to lie between deterministic and random extremes. RPs and their quantifications have been applied to human interaction data that are continuous in both amplitude and time (e.g., childcaregiver interactions, Dale & Spivey, 2006; coupling between speaker-listener eye movements, Richardson & Dale, 2005; coordination of postural sway, Shockley, Santana, & Fowler, 2003). Those studies describe aspects of RPs, including phase-space reconstruction and threshold selection (see Figure 1), which are beyond the scope of the methods presented here. Because we are analyzing discrete team interaction sequences, research on recurrence analysis for ordinal (Bandt, 2005; Quera, 2008) and nominal (Dale, Warlaumont, & Richardson, in press; see also Pincus et al., 2008) data series provides the groundwork for the discrete recurrence approach presented here. Discrete RPs. Consider three discrete states corresponding to three team members, each of


506


Table 1: Mean Determinism (DET) Values Calculated Over 20 Simulations for Completely Deterministic, Completely Random, and Deterministic + Random Sequences as a Function of Number of Codes and Sequence Length Sequence Length Number of Codes

128

Deterministic sequence 3 100.00 (0) 4 100.00 (0) 5 100.00 (0) Random sequence 3 54.99 (1.33) 4 43.02 (1.44) 5 35.18 (2.21) Deterministic + random sequence 3 61.36 (2.59) 4 48.62 (1.95) 5 39.35 (1.42)

256

512

100.00 (0) 100.00 (0) 100.00 (0)

100.00 (0) 100.00 (0) 100.00 (0)

1024 100.00 (0) 100.00 (0) 100.00 (0)

55.25 (0.75)

55.42 (0.24)

55.49 (0.18)

43.30 (0.71) 35.64 (0.64)

43.70 (0.36) 35.91 (0.55)

43.67 (0.16) 35.96 (0.15)

61.80 (1.48) 48.86 (1.22) 39.43 (1.13)

62.02 (0.99) 49.22 (0.64) 40.29 (0.71)

62.05 (0.67) 49.34 (0.37) 40.14 (0.43)

Note. Standard deviations are shown in parentheses; bold entries correspond to number of codes and range of sequence lengths used in the current study.

whom is uniquely identified by a mutually exclusive number code (i.e., 1, 2, or 3), when they speak. (Later, we carry out these analyses on three-person teams; here, we demonstrate the approach with simulations using three discrete codes.) There is no metric of time, just a series of discrete speaker states represented by a sequence of codes. For a discrete sequence x of length N, the N × N recurrence matrix is formed by plotting a dot when the discrete code at x(j) exactly matches the discrete code at x(i). Figure 2 presents RPs for three simulations over discrete sequences, each of length N = 50 codes. DET is computed for each sequence over recurrent points and their diagonals, as in Equation 1, from the upper triangle of each RP. The left column of Figure 2 corresponds to a completely deterministic case. The same pattern of three codes (1-2-3) always repeats, such that x[i] = x[i + 3], resulting in a homogeneous RP with DET = 100. The same result would have been obtained, however, if the sequence had been generated by a different pattern of codes (e.g., 3-2-1 [period-3]; 2-1-1-3 [period-4]; 1-2-2-3-1 [period-5]; etc.) so long as the pattern

always repeats. This type of interaction sequence, and its RP, would be ideal for a rigidly structured team task, in which variations from a fixed interaction procedure (a repeating pattern) may be harmful. The middle column of Figure 2, conversely, corresponds to the completely random case. Compared to the deterministic case, the RP is relatively irregular, with DET = 50. Note that DET ≠ 0 because with only three codes there are bound to be chance-level pattern repetitions; that is, with only three codes, the chance of randomly drawing a repeating pattern is high. The results of a simulation study, shown in Table 1, illustrate that the expected values of DET for a random discrete sequence decay as the number of codes increases beyond three but, for three codes, DET is slightly greater than 55% even as sequence length increases. Unless the task environment were entirely unpredictable, it is hard to imagine a team task in which the random case would be ideal. The sequence in the right column of Figure 1 is a pattern that repeats, but not in exactly the same way. Namely, all three codes must be


Measuring Patterns

in


Table 2: Pattern Information for Four Test Patterns Relative to the Discrete Sequences Shown in Figure 2

Pattern

Det

Random

Det + Random

1-2-3 3-1-2 3-2-1 1-2-3-1

0.73 0.73 −∞ 0.73

0.01 0.01 −0.001 −0.01

0.07 0.09 0.08 0.01

Note. Det = Deterministic.

represented during each three-code cycle (e.g., 3 3 ∑ i = 1 x[i] = ∑ i = 1 x[i + 3]), but the order of codes varies probabilistically—in this case, randomly— from cycle to cycle. This type of sequence is not ideal; it is an abstraction. It represents small, adaptive adjustments (within each three-code cycle) in an otherwise deterministically structured team interaction process. Like adaptive processes studied in other domains (e.g., dynamics of cognitive processes, Van Orden, Holden, & Turvey, 2003; human heart rate dynamics, Goldberger et al., 2002), such sequences are neither random nor deterministic; to remain flexible, they are somewhere in between (Van Orden, Kloos, & Wallot, 2009). To remain flexible and adaptive, we argue, teams must maintain a balance between overly structured and completely random interaction patterns (viz. DET = 60; Figure 2). Pattern Information

In information-theoretic terms, the informational content of a sequence of communication codes can be measured by the number of decisions (usually in bits) required to represent the sequence (i.e., the sequence’s uncertainty; its “entropy”), but it does not tell us, in a literal sense, the meaning of the sequence. In this light, the information-theoretic content of a communication pattern (extracted from a longer sequence of codes) quantifies the average amount; our uncertainty about the overall sequence is reduced given our knowledge of the pattern. In other words, how much do observed patterns tell us about the overall sequence? Communication pattern information is quantified as the mutual information (Cover & Thomas, 2006; Gallager, 1968) of a short

pattern of codes relative to an overall sequence of codes:

Pattern Information = p(pattern)   p(pattern)  × log  # codes  ∏ p(code i )  i =1

(2)

If N = sequence length and L = pattern length, then p(pattern) = pattern frequency (3) N − L +1 and

p(code i ) =

code i frequency N

(4)

Theoretically, if a sequence consisted of only those codes in the pattern, and the pattern probability is identical to the probability of those codes occurring independently in the sequence, then pattern information with respect to the overall sequence will be zero (i.e., log[1] = 0). Therefore, a pattern that occurs randomly has pattern information equal to zero. If pattern probability is less than the probability of the codes occurring independently, then pattern information will be negative (i.e., if 0 < y = p[pattern] / Π p[code] < 1, then log[y] < 0). This so-called negative mutual information (Gallager, 1968) entails that the pattern occurs below random (chance) levels, such that the pattern may provide misleading information concerning the overall sequence (the logic underlying negative pattern information is detailed in the appendix). Alternatively, higher, positive information values indicate that when the codes do occur, they tend to occur together as a pattern, providing positive mutual information concerning the process that generated the overall interaction sequence. Sample communication patterns and their information values, with respect to N = 1,024 samples of the simulated sequences introduced in Figure 2, are provided in Table 2. Here, we demonstrate the pattern information calculations in Table 2 for the pattern 1-2-3 relative to the


508


overall deterministic sequence: The pattern frequency is 341 because 1-2-3 repeats 341 times in the N = 1,024 deterministic sequence. The possibility of occurrence of the 1-2-3 pattern within the overall sequence is given by N – L + 1 = 1,024 – 3 + 1 = 1,022, such that p(pattern) = 341 / 1,022 = .33 (i.e., if you randomly draw a contiguous, three-code pattern from the deterministic sequence, then you stand a 33% chance of observing the 1-2-3 pattern; the calculation of the possibility of pattern occurrence is detailed in the appendix). p(pattern) is then divided by the product of the simple probabilities of the codes in the pattern (i.e., [.33][.33][.33]), yielding Pattern Information = .33 × log(.33/.333) = .73. Although the counting of the pattern and code frequencies for the Random and Deterministic + Random cases requires more effort, their calculations are carried out in the same manner (Table 2). For illustrative purposes, Table 2 shows the results for variations of the same pattern (i.e., 1-2-3 and 3-1-2 are the same pattern shifted by one code), its mirror image (3-2-1), and a slightly lengthened version of the 1-2-3 pattern (1-2-3-1). Pattern length does not matter for the deterministic sequence. As long as the codes are properly ordered, nothing more can be learned by lengthening the pattern. Hence, 1-2-3, and its extension 1-2-3-1, are maximum information patterns for the deterministic sequence. Alternatively, if the pattern does not match the deterministic sequence (e.g., 3-2-1 never occurs in the sequence), then pattern information = –∞, indicating that the pattern is misleading with respect to the overall sequence with absolute certainty (Gallager, 1968). We expect that, like a deterministic sequence, high maximum information patterns are associated with rigidly structured team interaction processes. Pattern length also does not matter for the random sequence, nor does the order of codes in the pattern. In the random case, any randomly generated pattern should be just as informative with respect to the overall sequence as any well thought out, a priori specified pattern; namely, pattern information ≈ 0. The difficulty with the random case for real team interaction is that there would be no pattern (except the overall sequence itself) that would be informative

of how team members interact. Under those circumstances, pattern information would be useless. Again, however, it is hard to imagine a team task in which interaction processes are well suited by the random case. Recall from Figure 2 that we made the Deterministic + Random sequence by randomly ordering the set of codes (1, 2, or 3) for each repetition of a three-code cycle. Because they represent 3 of the possible 27 three-code combinations, the first three patterns in Table 2 are somewhat representative of the procedure by which the interaction sequence was produced. Accordingly, each of those patterns results in roughly equivalent, nonzero information concerning the overall process. Unlike the deterministic case, there is not a maximum information pattern; information is distributed over variants of the three-code pattern. Adding a fourth code (in the last pattern in Table 2), however, violates the deterministic rule of random pattern formation that patterns form three codes at a time, such that pattern information ≈ 0. The Deterministic + Random sequence has flexibility to change patterns, but only within deterministic constraints. We expect that teams that remain flexible and adaptive within the constraints of the task will distribute mutual information across interaction patterns, rather than exhibit high maximum pattern information. In the process of evaluating the informational content of the patterns in Table 2, we defined the patterns a priori, top-down, based on our knowledge of the overall sequences. Using that approach, we scanned sequences for known patterns to investigate their utility for explaining the process that generated the overall sequence. Another possibility would be to explore a sequence, bottom-up, for maximum information patterns. Following that approach, in the analyses described later we measure the informational content of recurrent patterns, extracted from discrete RPs, to analyze teams’ maximum pattern information. The Current Study

The DET and Pattern Information measures are applied to communication data collected in the context of a three-person Uninhabited Air Vehicle Synthetic Task Environment (UAV-STE;


Measuring Patterns

in


Cooke & Shope, 2005) for teams. The communication data come from an experiment in which team membership either changed or stayed the same following a retention interval (Gorman, Amazeen, & Cooke, 2010; Gorman & Cooke, 2011; Gorman et al., 2006). We have previously reported on team effectiveness (performance) data from that experiment, showing that “mixed” teams, whose members changed partway through the experiment, suffered a brief performance decrement following the change in membership. However, mixed teams quickly recovered to perform as well as teams that remained intact, and they had significantly higher team process ratings (Gorman & Cooke, 2011; Gorman et al., 2006). The reason for that difference was revealed in intact versus mixed team coordination dynamics: Intact teams exhibited rigid coordination dynamics, whereas mixed teams exhibited more flexible coordination dynamics (Gorman, Amazeen, et al., 2010). Mixed team coordination dynamics were also more stable, which was correlated with adapting to unpredictable changes in the task environment. Due to their increased flexibility, mixed teams were more stable with respect to the unpredictable dynamics of the task environment. Unlike those prior results, we want to demonstrate that the discrete recurrence approach presents valid measures for differentiating nonlinear dynamics using relatively simple (automatically generated) interaction sequences. Based on those prior results, however, we predicted that because intact teams had more rigid coordination dynamics, their interaction sequences would be more rigidly patterned, manifesting in relatively high determinism and maximum pattern information. On the other hand, because their coordination dynamics were more flexible, we predicted that mixed team interaction sequences would manifest in lower determinism and, relative to their overall interaction process, lower maximum pattern information. Method Participants

Forty-five three-person teams (135 total participants) participated across two experi men tal sessions, separated by a retention interval. Participants’ age ranged from 18 to 58 (M = 26),

and 96 were male. Participants did not know each other prior to the experiment. In the current study, we analyze communication data from the 39 teams who participated during the second session of the experiment. Participants were paid $70 during the second session with a $100 bonus for each member of the highest performing team. Procedure, Experimental Design, and Apparatus

The communication data came from a threeperson UAV-STE in which teams perform a series of 40-min missions (separated by 5-min breaks) during which they fly the UAV over ground targets to take reconnaissance photographs (11–12 targets per mission). The team members—pilot, photographer, and navigator—monitor role-specific workstations and coordinate control of the UAV over headsets using push-to-talk (PTT) buttons that may be directed to either or both of the other team members. The necessary and sufficient interactions to successfully photograph a target include the following: (1) the navigator provides target information to the photographer; (2) the pilot and photographer negotiate a final airspeed and altitude for that target; and (3) the photographer provides feedback on the status of the photograph. At any given time, however, teams are following this procedure for multiple targets. The result is multiple, nested communication “threads,” corresponding to different targets, operating simultaneously on different time scales. During the first session of the experiment, teams acquired task proficiency over five missions. Teams returned for the second session after a short (3–5 weeks) or long (10–13 weeks) retention interval. Teams returned for the second session with their same members (intact) or with different members (mixed); however, individuals maintained their same role on the team. Each team completed three missions during the second session (the postretention missions analyzed later). During each mission, the PTT system recorded who was talking, when, and for how long, such that the PTT system automatically codes for discrete (turn-taking) interaction sequences. PTT recordings were missing for 2


510


Table 3: Mean Number of Communication Events (= Sequence Length) by Postretention Mission and Experimental Condition Postretention Mission Mission 1 Mission 2 Mission 3

Short-Intact (n = 10)

Short-Mixed (n = 10)

Long-Intact (n = 8)

Long-Mixed (n = 9)

186.80 (86.77) 181.90 (99.45) 166.00 (85.60)

160.30 (47.50) 160.44 (54.48) 155.90 (55.82)

217.63 (49.88) 205.50 (52.38) 203.38 (71.35)

248.44 (87.58) 222.33 (84.03) 193.56 (76.52)

Note. Standard deviations are in parentheses.

of the 39 Session 2 teams; therefore, their sequences were excluded from the statistical analyses (i.e., 37 teams × 3 postretention missions = 111 interaction sequences). The lengths of the interaction sequences for each experimental condition are summarized in Table 3. Measures

Ordered sequences of codes (one for each UAV mission) are the input for the measures. The input consists of ordered sequences of mutually exclusive (nominal) codes for each UAV mission: 1 = Pilot Speaking; 2 = Photographer Speaking; and 3 = Navigator Speaking. Communication determinism. A discrete RP was constructed for each UAV mission. In Figure 3, the right panel shows the plot of a 210-element UAV sequence for one mission, in which black dots represent recurrent points. For example,

if communication code 80 was Pilot Speaking, then tracing up from 80 on the x-axis there is a black dot at y = 80 as well as each time Pilot Speaking recurred later in the sequence. In Figure 3, the left panel is an enlarged section of the plot showing recurrent diagonals in the upper triangle, wherein particular turn-taking patterns recurred later in the sequence (e.g., the pattern between communication numbers 23 and 33 recurred between communication numbers 100 and 110). DET scores were calculated from recurrent diagonals relative to all recurrent points, using Equation 1, for each UAV mission. Patterns of codes making up recurrent diagonals in the upper triangle of each RP were retained to calculate pattern information scores. Maximum pattern information. Using Equation 2, we calculated pattern information for each unique pattern extracted from each RP.

Figure 3. A discrete recurrence plot for an Uninhabited Air Vehicle (UAV) team interaction sequence (N = 210) is shown on the right; one section of the plot (showing recurrent diagonal points) is enlarged on the left. Downloaded from hfs.sagepub.com at HFES-Human Factors and Ergonomics Society on October 10, 2012

Measuring Patterns

in


Table 4: Mean Communication Determinism, Number of Unique Patterns Extracted, and Average Pattern Length by Postretention Mission and Experimental Condition Experimental Condition Variable


Communication determinism Mission 1 58.65 (2.96) Mission 2 59.23 (2.44) Mission 3 62.34 (2.78) Unique patterns extracted Mission 1 103.60 (46.32) Mission 2 93.50 (52.15) Mission 3 87.80 (47.90) Average pattern length Mission 1 2.26 (0.15) Mission 2 2.29 (0.14) Mission 3 2.34 (0.18)


Long-Intact (n = 8)

Long-Mixed (n = 9)

60.66 (4.29) 59.18 (3.98) 59.63 (2.18)

60.85 (2.58) 61.45 (2.90) 62.26 (3.20)

59.41 (3.52) 60.79 (4.43) 60.76 (4.12)

86.00 (27.98) 86.90 (29.36) 81.70 (33.90)

113.50 (33.36) 106.88 (26.58) 108.25 (44.88)

118.11 (27.83) 110.78 (38.35) 92.11 (30.43)

2.27 (0.10) 2.26 (0.11) 2.23 (0.15)

2.50 (0.16) 2.43 (0.16) 2.45 (0.18)

2.36 (0.17) 2.36 (0.21) 2.30 (0.18)


The maximum of those values for each RP is the measure of maximum pattern information. In this way, a measure of maximum pattern information was obtained for each UAV mission. Results Power analyses across previous UAV studies yielded low power with α = .05, although effect sizes were medium to large (see Gorman, Cooke, et al., 2010). Therefore, to reduce Type II error rate, we set α = .10. We report ω2 effect sizes to facilitate comparisons across studies and with conventional levels of practical significance: small effect ≈ .01; medium effect ≈ .06; large effect ≈ .15 (Cohen, 1988). For analyses comparing postretention DET and maximum pattern information to their preretention interval levels, Cohen’s d is used: small effect ≈ .20; medium effect ≈ .50; large effect ≈ .80 (Cohen, 1988). Communication Determinism

The determinism results are summarized in Table 4. DET was analyzed with a 2 (composition) × 2 (retention interval) × 3 (postretention mission) mixed analysis of variance (ANOVA).

The Composition × Post-Retention Mission interaction was significant, F(2, 32) = 2.75, p = .08, ω2 = .03, as was the main effect of PostRetention Mission, F(2, 32) = 3.84, p = .03, ω2 = .04. No other effects were significant. Figure 4 shows the interaction. The simple effect at the first mission following the retention interval was not significant, F(1, 33) = .06, p = .80, ω2 = –.03, nor was the simple effect at the second mission, F(1, 33) = .10, p = .76, ω2 = –.03. However, by the third mission, the intact teams’ DET scores (M = 62.06, SD = 3.00) were significantly greater than mixed teams’ scores (M = 60.18, SD = 3.12), F(1, 33) = 4.21, p = .05, ω2 = .08. As shown in Figure 4, intact and mixed teams’ DET scores did not differ immediately after the retention interval. However, the DET of the intact teams’ interaction patterns increased over the postretention missions, whereas mixed teams exhibited no increasing deterministic trend. To examine whether intact and mixed teams differed with respect to preretention interval DET, we compared postretention DET scores to their preretention levels. Mean DET at the fourth preretention interval mission (M = 61.67)


512


Table 5: Mean Maximum Pattern Information by Postretention Mission and Experimental Condition Experimental Condition Variable Mission 1 Mission 2 Mission 3



Long-Intact (n = 8)

Long-Mixed (n = 9)

.07 (.02) .07 (.02) .09 (.02)

.08 (.02) .07 (.03) .07 (.02)

.08 (.02) .07 (.02) .09 (.02)

.06 (.02) .06 (.02) .07 (.03)


was selected as the baseline because preretention team effectiveness was at its asymptote (Gorman et al., 2006, reports those team effectiveness results). One-sample t tests revealed that intact teams had significantly lower DET scores at Mission 1, t(17) = –2.94, p = .009, d = –.69, and at Mission 2, t(17) = –2.19, p = .042, d = –.52, but did not differ from the preretention level at Mission 3, t(17) = .93, p = .36, d = .22. Mixed teams, however, exhibited significantly lower DET at each postretention mission: Mission 1 t(18) = –1.80, p = .09, d = –.41; Mission 2 t(18) = –1.81, p = .09, d = –.42; and Mission 3 t(18) = –2.05, p = .06, d = –.47. Maximum Pattern Information

Maximum pattern information results are summarized in Table 5. Maximum pattern information was analyzed using a 2 (composition) × 2 (retention) × 3 (postretention mission)

Communication Determinism

65.00 Intact Mixed

65.50

60.00

57.50 1

2

3

Postretention Mission

Figure 4. Communication determinism by composition over postretention missions (the horizontal dashed line corresponds to mean preretention communication determinism; error bars are 90% confidence intervals).

mixed ANOVA. The main effect of composition on maximum pattern information was significant, F(1, 33) = 4.13, p = .05, ω2 = .08: Intact teams had greater maximum pattern information (M = .08, SD = .01) than did mixed teams (M = .07, SD = .02). The postretention mission main effect was also significant, F(2, 32) = 4.59, p = .02, ω2 = .08. Post hoc comparisons between missions revealed that maximum pattern information did not differ between Missions 1 and 2, F(1, 36) = .81, p = .37, ω2 = –.01, or between Missions 1 and 3, F(1, 36) = 2.16, p = .15, ω2 = .03; however, maximum pattern information at Mission 3 was greater than at Mission 2, F(1, 36) = 8.76, p = .005, ω2 = .17. Figure 5 illustrates the fluctuation in maximum pattern information, wherein intact teams ultimately exhibited greater maximum pattern information. Intact and mixed teams’ maximum pattern information scores were compared to mean preretention maximum pattern information (M = .074 at the fourth preretention mission) at each postretention mission. One-sample t tests revealed that maximum pattern information for mixed teams did not differ from the preretention interval level at either Mission 1, t(18) = –.86, p = .40, d = –.20, or Mission 3, t(18) = –1.27, p = .22, d = –.29, but were significantly lower at Mission 2, t(18) = –1.98, p = .06, d = –.45. Intact teams did not differ from the preretention interval level at either Mission 1, t(17) = –.34, p = .74, d = –.08, or Mission 2, t(17) = –.55, p = .59, d = –.13, but exhibited significantly greater maximum pattern information compared to the preretention level by the end of the experiment, at Mission 3, t(17) = 3.19, p = .005, d = .76.


Measuring Patterns

in


Maximum Pattern Infromation

0.10 Intact

0.09

Mixed

0.08 0.07 0.06 0.05 1

2 Postretention Mission

3

Figure 5. Maximum pattern information by composition over postretention missions (the horizontal dashed line corresponds to mean pre-retention maximum pattern information; error bars are 90% confidence intervals).

Surrogate Analysis

Surrogate analysis is a resampling method that tests the null hypothesis that the observed dynamics are due to the temporal (sequential) distribution of the data, rather than their marginal distribution properties (Theiler, Eubank, Longtin, Galdrikian, & Farmer, 1992). We conducted a surrogate analysis to ensure DET and, therefore, pattern analysis results were not artifacts of sequences consisting of only three codes or occurred randomly. Surrogate sequences were generated by randomly shuffling each observed postretention sequence such that the codes were randomly sequenced, but the observed and surrogate sequences retained the same marginal properties (i.e., the multinomial probability of codes is preserved after shuffling). DET values of surrogate sequences (M = 57.60, SD = 3.60) and observed sequences (M = 60.59, SD = 3.43) were compared using a sign test (see Van Orden et al., 2003). Of 111 comparisons, 93 yielded success (observed DET > surrogate DET; p < .0001), revealing that the observed dynamics were not due to marginal probabilities or paucity of codes. Discussion

We have presented methods that are simple applications of recurrence analysis and information theory that differentiate teams based on discrete interaction sequences. As we expected, the rigidity of intact teams was reflected in greater communication determinism and higher

maximum information patterns relative to mixed teams. We previously reported that intact teams initially outperform mixed teams, but that effect lasts only one UAV mission (i.e., the first postretention mission; Gorman & Cooke, 2011; Gorman et al., 2006). Mixed teams, however, exhibit higher team process ratings and more flexible coordination dynamics, which were correlated with adaptive behavior (Gorman, Amazeen, et al., 2010). Thus, mixed teams end up performing as well as intact teams, but they become more adaptive. In this report, new results indicate that those experimental effects may be further qualified by level of communication determinism and pattern information: As indexed by determinism, mixed team interaction is less rigid than intact team interaction, and, as indexed by maximum pattern information, single patterns do not explain as much of their interaction process. It has long been argued that, for teams and groups, familiarity can breed habituation of interaction patterns and rigidity (Gersick & Hackman, 1990; e.g., groupthink; Janis, 1972). Our current results provide further support for that account and provide evidence for the validity of the discrete recurrence measures. Whereas intact teams regained preretention levels of communication determinism, mixed teams did not. Relative to preretention levels of maximum pattern information, intact teams ultimately exhibited higher maximum pattern information; however, mixed teams were always at or below preretention levels. By these measures, it appears that mixing teams changes something fundamental in the learning and development of team interaction processes. It is important to note, however, that the developmental transition toward deterministic, high maximum information patterns for intact teams did not ultimately produce ideal deterministic results (cf. Tables 1 & 2). Although intact team interactions were significantly more deterministic than were mixed team interactions, both intact and mixed teams’ results were closer to a deterministic + random process than purely deterministic or random (Table 1). Does this begin to suggest something about the fundamental nature of team interaction in dynamic environments?


514


If team interaction is not deterministically structured (as it might be if teams rigidly follow an a priori script) and it is not random, then what is the nature of “in-between” forms of interaction—exemplified by the abstract deterministic + random process—such that teams may differentially acquire flexible yet stable interaction processes? We think the key lies in the dynamic interchange between team interaction and the task environment. That is to say that team members are compelled to adjust their interaction patterns (flexibility) in relation to changes in the task environment, such that the team maintains a stable trajectory toward meeting its goals. We have also found that changes in interaction constraints, which specify the nature of the team task, that perturb the stability of a team trajectory provide flexibility training for adaptive teams (Gorman, Cooke, et al., 2010). Hence, “local” interaction flexibility and “global” task stability may be reciprocally related. We believe discrete recurrence methods present not only methods for quantifying the effect of training and experimental interventions on flexibility of team interaction processes but also the dynamics that contribute to that process. The methods we have presented allow us to characterize interaction flexibility in terms of communication determinism and pattern information using relatively simple, discrete interaction sequences. In the next section, we consider the implications of this for automated data analysis, including real-time interaction analysis. Implications for Automated Data Analysis

The past research that led to our predictions in the current study used communication and video content to code (“log”) specific types of interactions to examine the dynamic flexibility and stability of team coordination. Content coding can be highly resource intensive and (most often) requires human expertise (e.g., Emmert & Barker, 1989; but see Foltz & Martin, 2009). By contrast, the discrete recurrence methods illustrated here may be automatically applied to automatically generated team interaction data. Although discrete sequential methods are not intended to capture deep-level interaction content, they may be useful for capturing some of the

same aspects of the nonlinear dynamics of team interaction. In this light, automated data analysis using discrete recurrence methods could be used like a filter to localize interesting data in need of more resource-intensive methods for deep-level content analysis (i.e., the “wedding cake” strategy for team communication analysis; Cooke & Gorman, 2010). Because they could be automated, discrete recurrence methods also have potential for realtime communication analysis. Although we analyzed our interaction data post hoc, those data were generated in real time. Therefore, discrete recurrence algorithms may be applied to team interaction sequences as they are collected. In the current study we used a PTT system to generate interaction data in real time. However, real-time interaction data can be generated by applying a low-pass filter to sound pressure level (SPL) at team-member microphones (Cooke & Gorman, 2010). When SPL exceeds team members’ thresholds, data about who is talking, when, and for how long can be automatically generated for real-time discrete recurrence analysis. Limitations and Future Directions

Several limitations warrant mentioning. First, we examined communication determinism and pattern information in a structured team task, but not a rigidly structured team task. For example, the UAV team members were not required to acknowledge the receipt of a piece of information. A more highly structured task, however, may introduce concurrent requirements for highly structured interaction. In that case, we would expect higher levels of communication determinism and pattern information, indicating that the team is interacting in a manner consistent with the requirements of the task. Second, we were concerned with interaction code sequences but not exact onset/offset (duration time) of codes. Therefore, our analyses are more similar to the analysis of eventcoded data than time-coded data (Bakeman & Gottman, 1997). Quera (2008), however, provided pattern analysis strategies for code sequences when code durations are used. Third, we looked only at mean results, not specific patterns. In future applications, these methods


Measuring Patterns

in


may be used to differentiate teams based on specific patterns or to identify patterns for detailed content analysis. Finally, a coding scheme need not correspond to speaker identity. So long as codes are mutually exclusive and exhaustive, any team interaction coding scheme may be used (see Bakeman & Gottman, 1997, for an overview of coding schemes). Several future directions also deserve mentioning. In our experiment, we manipulated team composition using just two levels of team membership—intact versus mixed—to examine the effect on team interaction processes. A difficulty with assessing interaction patterns in real teams, however, is that one or more key players may come and go at any time. Under those conditions, change in team membership is a variable that may take on many more values than the two levels in our experiment. In terms of nonlinear dynamics of team interaction, scaling of a (more) continuous team membership variable may result in discontinuous changes in the quality of interaction patterns. The challenge for the future, then, is to use discrete recurrence methods to detect qualitative changes (“phase transitions”) in the quantity and informational content of communication patterns with continuously scaled changes in team membership. Although we applied recurrence analysis to univariate discrete interaction sequences, recurrence analysis can be applied to bivariate nonlinear time series using Cross Recurrence Plots (CRPs; Marwan & Kurths, 2002). CRPs allow two dynamical systems to be compared to examine synchronization of states across systems (Shockley, Butwill, Zbilut, & Webber, 2002). A natural extension of the present research, then, is to examine discrete CRPs for synchronization between team interaction sequences. Using that approach, either two teams’ sequences could be plotted against each other or one team’s sequence could be plotted against an ideal referent. Conclusion

In this article we demonstrated recurrence methods for the analysis of discrete team interaction sequences. Recurrence analysis offers a powerful suite of methods for sequential data

analysis (Dale et al., in press). Our applications only scratch the surface, but given our results thus far, more research on its application to team communication analysis is warranted. Appendix Negative Pattern Information

For team communication, imagine a three-person (Persons 1, 2, and 3) task in which it makes no sense for Person 3 to follow Person 1 (i.e., the transition 1–3 is against the rules). Although the unitary events “1” and “3” may occur frequently in an observed sequence, we would expect any pattern containing the transition 1–3 to occur so rarely that its probability would be less than we would expect to occur randomly by chance. In fact, we might expect that any pattern containing the transition 1–3 should result in negative pattern information, because it is unlikely that any such pattern reflects the process that generated the overall sequence. Possibility of Pattern Occurrence Calculation

The calculation of the denominator in Equation 3 can be understood as follows: Consider a short sequence x of length N = 6 and a pattern of length L = 3. That pattern could occur at each of the following positions of x: [1,2,3]; [2,3,4]; [3,4,5]; and [4,5,6]. Hence, there would be a maximum of N – L + 1 = 6 – 3 + 1 = 4 locations where the pattern could occur within the sequence x. Note that if N = L, and if each code in the pattern matches the sequence, then p(pattern) must equal one; however, if the pattern does not occur in the sequence, then p(pattern) must equal zero. Acknowledgments This article uses unpublished data from a previously published experiment (Gorman, Amazeen, et al., 2010; Gorman & Cooke, 2011; Gorman et al., 2006) for the purpose of demonstrating a team communication analysis approach. None of the results reported here overlap with any prior publication. The original experiment was funded by Air Force Office of Scientific Research grant FA955004-1-0234 and Air Force Research Laboratory grant FA8650-04-6442.


516


Key points •• Team interaction patterns recur in time, often in discontinuous bursts and lulls of interaction; therefore, recurrence methods may be appropriate for quantifying pattern change. •• The recurrence-based communication determinism and pattern information measures were successful in detecting pattern differences in the interaction sequences of teams whose members changed partway through an experiment versus teams who remained intact. •• In support of predictions based on earlier analyses, intact team interaction sequences were more deterministic and had higher maximum pattern information than mixed team interaction patterns. •• The discrete recurrence method for measuring patterns in team interaction sequences is automatic and fast; the potential for real-time automation is high.

References Achille, L. B., Schultz, K. G., & Schmidt-Nielson, A. (1995). An analysis of communication and the use of military terms in Navy team training. Military Psychology, 7, 95–107. Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge, UK: Cambridge University Press. Bandt, C. (2005). Ordinal time series analysis. Ecological Modeling, 182, 229–238. Bowers, C. A., Jentsch, F., Salas, E., & Braun, C. C. (1998). Analyzing communication sequences for team training needs assessment. Human Factors, 40, 672–679. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Cooke, N. J., & Gorman, J. C. (2010). The pragmatics of communication-based methods for measuring macrocognition. In E. S. Patterson & J. E. Miller (Eds.), Macrocognition metrics and scenarios (pp. 161–178). Burlington, VT: Ashgate. Cooke, N. J., & Shope, S. M. (2005). Synthetic task environments for teams: CERTT’s UAV-STE. In Handbook of human factors and ergonomics methods (pp. 46-1–46-6). Boca Raton, FL: CLC Press. Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed.). Hoboken, NJ: John Wiley. Dale, R., & Spivey, M. J. (2006). Unraveling the dyad: Using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Language Learning, 56, 391–430. Dale, R., Warlaumont, A. S., & Richardson, D. C. (in press). Nominal cross recurrence as a generalized lag sequential analysis for behavioral streams. International Journal of Bifurcation and Chaos. Eckmann, J.–P., Oliffson Kamphorst, S., & Ruelle, D. (1987). Recurrence plots of dynamical systems. Europhysics Letters, 4, 973–977.

Emmert, P., & Barker, L. L. (1989). Measurement of communication behavior. White Plains, NY: Longman. Fischer, U., McDonnell, L., & Orasanu, J. (2007). Linguistic correlates of team performance: Toward a tool for monitoring team functioning during space missions. Aviation, Space, and Environmental Medicine, 78, B86–B95. Foltz, P. W., & Martin, M. J. (2009). Automated communication analysis of teams. In E. Salas, J. Goodwin, & C. S. Burke (Eds.), Team effectiveness in complex organizations: Crossdisciplinary perspectives and approaches (pp. 411–431; SIOP Organizational Frontiers Series). New York, NY: Routledge. Gallager, R. G. (1968). Information theory and reliable communication. New York, NY: John Wiley. Gersick, C.J.G., & Hackman, J. R. (1990). Habitual routines in task-performing groups. Organizational Behavior and Human Decision Processes, 47, 65–97. Goldberger, A. L., Amaral, L.A.N., Hausdorff, J. M., Ivanov, P. Ch., Peng, C.–K., & Stanley, H. E. (2002). Fractal dynamics in physiology: Alterations with disease and aging. Proceedings of the National Academy of Sciences of the United States of America, 99, 2466–2472. Gorman, J. C., Amazeen, P. G., & Cooke, N. J. (2010). Team coordination dynamics. Nonlinear Dynamics, Psychology, and Life Sciences, 14, 265–289. Gorman, J. C. & Cooke, N. J. (2011). Changes in team cognition after a retention interval: The benefits of mixing it up. Journal of Experimental Psychology: Applied. Advance online publication. doi: 10.1037/a0025149. Gorman, J. C., Cooke, N. J., & Amazeen, P. G. (2010). Training adaptive teams. Human Factors, 52, 295–307. Gorman, J. C., Cooke, N. J., Pedersen, H. K., Winner, J. L., Andrews, D., & Amazeen, P. G. (2006). Changes in team composition after a break: Building adaptive command-and-control teams. In Proceedings of the Human Factors and Ergonomics Society 50th annual meeting (pp. 487–492). Santa Monica, CA: Human Factors and Ergonomics Society. Gorman, J. C., Foltz, P. W., Kiekel, P. A., Martin, M. J., & Cooke, N. J. (2003). Evaluation of latent semantic analysis-based measures of team communication content. In Proceedings of the Human Factors and Ergonomics Society annual meeting (pp. 424–428). Santa Monica, CA: Human Factors and Ergonomics Society. Huberman, B. A., & Glance, N. S. (1998). Fluctuating efforts and sustainable cooperation. In M. J. Prietula, K. M. Carley, & L. Gasser (Eds.), Simulating organizations (pp. 89–103). Menlo Park, CA: American Association for Artificial Intelligence. Janis, I. L. (1972). Groupthink: Psychological studies of policy decisions and fiascoes. Boston, MA: Houghton Mifflin. Kanki, B. G., Lozito, S., & Foushee, C. H. (1989). Communication indices of crew coordination. Aviation, Space, and Environmental Medicine, 60, 56–60. Katz, R. L. (1982). The effects of group longevity on project communication and performance. Administrative Science Quarterly, 27, 81–104. Marwan, N., & Kurths, J. (2002). Nonlinear analysis of bivariate data with cross recurrence plots. Physics Letters A, 302, 299–307. Marwan, N., Romano, M. C., Thiel, M., & Kurths, J. (2007). Recurrence plots for the analysis of complex systems. Physics Reports, 5–6, 237–329. Miller, A., Scheinkestel, C., & Joseph, M. (2009). Coordination and continuity of intensive care unit patient care. Human Factors, 51, 354–367.


Measuring Patterns

in


Pincus, D., Fox, K. M., Perez, K. A., Turner, J. S., & McGeehan, A. R. (2008). Nonlinear dynamics of individual and interpersonal conflict in an experimental group. Small Group Research, 39, 150–178. Quera, V. (2008). RAP: A computer program for exploring similarities in behavior sequences using random projections. Behavior Research Methods, 40, 21–32. Richardson, D. C., & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29, 1045–1060. Sexton, J. B., & Helmreich, R. L. (2000). Analyzing cockpit communication: The links between language, performance, error, and workload. Human Performance in Extreme Environments, 5, 63–68. Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Urbana: University of Illinois Press. Shockley, K., Butwill, M., Zbilut, J. P., & Webber, C. L., Jr. (2002). Cross recurrence quantification of coupled oscillators. Physics Letters A, 305, 59–69. Shockley, K., Santana, M. V., & Fowler, C. A. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29, 326–332. Theiler, J., Eubank, S., Longtin, A., Galdrikian, B., & Farmer, J. D. (1992). Testing for nonlinearity in time series: The method of surrogate data. Physica D, 15, 77–94. Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Selforganization of cognitive performance. Journal of Experimental Psychology: General, 132, 331–350. Van Orden, G. C., Kloos, H., & Wallot, S. (2009). Living in the pink: Intentionality, wellbeing, and complexity. In C. Hooker (Ed.), Handbook of the philosophy of science Volume 10: Philosophy of complex systems (pp. 639–683). New York, NY: Elsevier. Webber, C. L., Jr., & Zbilut, J. P. (1994). Dynamical assessment of physiological systems and states using recurrence plot strategies. Journal of Applied Physiology, 76, 965–973. Xiao, Y., Seagull, F. J., Mackenzie, C. F., Klein, K. J., & Ziegert, J. (2008). Adaptation of team communication patterns: Exploring

the effects of leadership at a distance, task urgency, and shared team experience. In S. P. Weisband (Ed.), Leadership at a distance: Research in technology-supported work (pp. 71–93). Mahwah, NJ: Lawrence Erlbaum.

Jamie C. Gorman received his PhD in cognitive psychology from New Mexico State University in 2006 and is an assistant professor in psychology at Texas Tech University. Nancy J. Cooke received her PhD in cognitive psychology from New Mexico State University in 1987 and is a professor in cognitive science and engineering at Arizona State University and science director of the Cognitive Engineering Research Institute. Polemnia G. Amazeen received her PhD in experimental psychology from the University of Connecticut in 1996 and is an associate professor in psychology at Arizona State University and a faculty research associate at the Cognitive Engineering Research Institute. Shannon Fouse received her BA in cognitive science from the University of Pennsylvania in 2008 and is an MA student in applied psychology at Arizona State University and a graduate research assistant at the Cognitive Engineering Research Institute. Date received: January 14, 2011 Date accepted: September 7, 2011
