Modeling the neurodynamic complexity of submarine navigation teams
Ronald Stevens, Trysha Galloway, Peter Wang, Chris Berka, Veasna Tan, Thomas Wohlgemuth, Jerry Lamb & Robert Buckles Computational and Mathematical Organization Theory ISSN 1381-298X Comput Math Organ Theory DOI 10.1007/s10588-012-9135-9
1 23
Your article is protected by copyright and all rights are held exclusively by Springer Science+Business Media, LLC. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your work, please use the accepted author’s version for posting to your own website or your institution’s repository. You may further deposit the accepted author’s version on a funder’s repository at a funder’s request, provided it is not made publicly available until 12 months after publication.
1 23
Author's personal copy Comput Math Organ Theory DOI 10.1007/s10588-012-9135-9 SI: BRIMS 2011
Modeling the neurodynamic complexity of submarine navigation teams Ronald Stevens · Trysha Galloway · Peter Wang · Chris Berka · Veasna Tan · Thomas Wohlgemuth · Jerry Lamb · Robert Buckles
© Springer Science+Business Media, LLC 2012
Abstract Our objective was to apply ideas from complexity theory to derive neurophysiologic models of Submarine Piloting and Navigation showing how teams cognitively organize around changes in the task and how this organization is altered with experience. The cognitive metric highlighted was an electroencephalography (EEG)derived measure of engagement (termed NS_E) which was modeled into a collective R. Stevens () · T. Galloway · P. Wang UCLA IMMEX Project, 5601 W. Slauson Ave, #272, Culver City, CA 90230, USA e-mail:
[email protected] T. Galloway e-mail:
[email protected] P. Wang e-mail:
[email protected] C. Berka · V. Tan Advanced Brain Monitoring, 2237 Faraday Avenue, Suite 100, Carlsbad, CA 92008, USA C. Berka e-mail:
[email protected] V. Tan e-mail:
[email protected] T. Wohlgemuth · R. Buckles Submarine Learning Center, Groton, CT 06349, USA T. Wohlgemuth e-mail:
[email protected] R. Buckles e-mail:
[email protected] J. Lamb Naval Submarine Medical Research Laboratory, Groton, CT 06349, USA e-mail:
[email protected]
Author's personal copy R. Stevens et al.
team variable showing the engagement of each of 6 team members as well as the engagement of the team as a whole. We show that during a navigation task the NS_E data stream contains historical information about the cognitive organization of the team and that this organization can be quantified by fluctuations in the Shannon entropy of the data stream. The fluctuations in the NS_E entropy were complex, showing both rapid changes over a period of seconds and longer fluctuations that occurred over periods of minutes. The periods of low NS_E entropy represented moments when the team’s cognition had undergone significant re-organization, i.e. when fewer NS_E symbols were being expressed. Decreases in NS_E entropy were associated with periods of poorer team performance as indicated by delays/omissions in the regular determination of the submarine’s position; parallel communication data suggested that these were also periods of increased stress. Experienced submarine navigation teams performed better than Junior Officer teams, had higher overall levels of NS_E entropy and appeared more cognitively flexible as indicated by the use of a larger repertoire of available NS_E patterns. The quantitative information in the NS_E entropy may provide a framework for designing future adaptive team training systems as it can be modeled and reported in near real time. Keywords Team neurodynamics · Complexity · Entropy · Artificial neural networks · Electroencephalography · Nonlinear dynamics
1 Introduction The theory of social influence describes how the state of an individual depends on the state of other individuals resulting in models of social synchronization (Nowak and Vallacher 2007). This model views individuals in an interaction not as passive entities, but as separate systems capable of rich dynamics. The synchronization of individuals’ dynamics provides a higher order system(s) with its own dynamic properties as each individual attempts to achieve synchronization by adjusting his or her internal state or overt behavior in response to the evolving task and the state or behavior of the individuals with whom he or she is interacting. Speech analysis is often used to model teamwork as it contains information about knowledge, uncertainty, and awareness of the situation, stress and other cognitive states. Speech/dialog data streams have obvious structure identified by the content in relation to the context of the situation, and less obvious structures including flow (who is speaking or to whom they are speaking) and speech acts (are they questioning, answering, making a statement, etc.). While speech provides a rich and unobtrusive window into the cognition of the team there are limitations for its exclusive use in studying teamwork. First, there are times when nothing is being said. A lack of speech among team members does not mean that there is no communication as communication can still occur through gestures, motions, glances, etc. Other moments where nothing is said are deliberate
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
due to bias or affective concerns; in these situations the lack of speech may be highly relevant. Another challenge is the time needed to transcribe and code dialog. We believe that neurophysiologic methods can extend the use of speech for modeling team dynamics by providing “in the head” measures of team dynamics (Warner et al. 2005). As members of a team perform their duties, each would be expected to exhibit varying degrees of cognitive states such as attention, workload, or engagement and the levels of these components at any one time might reflect aspects of team cognition. Rather than focusing on neurophysiologic markers such as P300 or N400 which appear and disappear rapidly in response to a large variety of stimuli, we chose to use broader markers of cognition, such as high engagement or high workload, which would persist over longer periods of time during team activities (Berka et al. 2004). While speech, behavioral data and neurophysiologic measures can all be collected from teams, there remains the question as to what analytic framework to use to integrate data across these multiple subsystems. Nonlinear dynamics (NLD) is a general theoretical approach for understanding complex systems. When teamwork is viewed as a complex adaptive system, there are multiple concepts that can be applied including self-organization, attractors, phase shifts, instabilities, entropy perturbations, and intrinsic dynamics (Cooke et al. 2009; Gorman et al. 2010). The result is a view of teamwork where individuals are rich dynamic systems with the state of each member depending on the state of others. From these interpersonal interactions, patterns of activity qualitatively emerge that are characterized by fluctuations to and from stable states (Nowak and Vallacher 2007). The goal of this study was to apply these ideas to neurophysiologic models of Submarine Piloting and Navigation (SPAN) teams to analyze how the teams reorganize themselves in response to changes in the task, and to derive insights into the differences between novice and expert SPAN teams. The measures used, termed Neurophysiologic Synchronies (NS), are symbolic collective team variables derived from team members’ EEG data streams. They represent the relative levels of engagement (NS_E) of each person on the team as well as the team as a whole. Previously we have shown that NS are dynamic variables whose expression during SPAN teamwork is sensitive to task changes (Stevens et al. 2009, 2010a, 2010b, 2012). The hypotheses for this study were: • Multiple NS_E attractor basins (attractors) exist for SPAN teams; • State shifts occur in response to changes in the task and these shifts can be detected by entropy fluctuations in the NS_E data stream; • NS_E expression complements, rather than duplicates the information in speech streams.
2 Methods 2.1 Teams These studies were conducted with navigation training tasks that are integral components of the Submarine Officer Advanced Course (SOAC) at the US Navy Submarine
Author's personal copy R. Stevens et al.
School, Groton, CT. The task is a high fidelity Submarine Piloting and Navigation simulation that contains dynamically programmed situation events which are crafted to serve as the foundation of the adaptive team training. The data reported in this study was derived from twelve SPAN sessions selected from a total of twenty-one as: (1) persons in the same six crew positions were being monitored by EEG, (2) the same individuals repeated in the same positions across 2–5 training sessions over multiple days. The six members of the teams that were fitted with the EEG headsets were the Quartermaster on Watch (QMOW), Navigator (NAV), Officer on Deck (OOD), Assistant Navigator (ANAV), Contact Coordinator (CC), and Radar (RAD). Additional persons participating in the SPAN who were not fitted with the headsets were the Captain (CAPT), Fathometer reader (FATH), the Helm (HELM), and multiple Instructors/Observers (INST). There were three SOAC Junior Officer teams and three experienced submarine navigation teams that each performed two SPAN simulations. SOAC teams and sessions are designated with a ‘T’ for the team and ‘S’ for the session (i.e. T4S1); for expert teams, ‘E’ is substituted for the team designation (i.e. E1S1). 2.2 Tasks Events in the SPAN include encounters with approaching ship traffic, the need to avoid shoals, changing weather conditions and instrument failure. There are taskoriented cues to guide the mission, team-member cues that provide information on how other members of the team are performing/communicating, and adaptive behaviors that help the team adjust in cases where one or more members are under stress or are not familiar with aspects of the unfolding situation. A sample navigation task is diagramed in Fig. 1. In this simulation the submarine (black squares) was being steered southward (down) and its position is shown at different times (seconds or epochs) by the numbers. The submarine encounters a tug and a tow (dots around epoch 1150), an inbound merchant (dots around epoch 1916), and an inbound submarine (stars around epoch 1340) each requiring a change in course or speed to avoid collision. Towards the end of the simulation there was a Man Overboard (MOB) event. Each SPAN session contains three segments. It begins with a Briefing where the overall goals of the mission are presented along with information on position, contacts, weather, sea state, etc. The Scenario is an evolving task segment and is more dynamic than the Brief, containing easily identified processes of teamwork along with others which are less well defined. One of the obvious processes is the regular updating of the ship’s position termed ‘Rounds’. Here, three navigation points are chosen, usually visually, and the bearing of each from the boat is rapidly measured and plotted on a chart. Rounds are an important aspect of piloting and navigation that are repeated every 3 minutes. The team goes through a 5 step countdown each time Rounds is conducted and the regularity with which the 5 steps are performed can be considered a performance indicator (Fig. 2). The timing of these 5 steps is shown for 2 teams where each row represents one of the steps. The team designation beginning with an ‘E’ is an experienced submarine navigation team while that with a ‘T’ is a SOAC student team.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
Fig. 1 Sample navigation path for a SPAN session. The numbers on the tracks indicate the simulation epochs. The submarine’s track is shown by the squares
Fig. 2 The regularity of Rounds by an experienced and Junior Officer Navigation Teams. The periods highlighted in gray are those where the normal 5-step sequence of rounds was interrupted. The NS_E entropy averages for each performance were calculated as described in the Methods
The expert session, E1S1 shows nearly complete 5-step Rounds countdowns. The patterns were less regular for SOAC team where steps were omitted and occasionally fixes were dropped.
Author's personal copy R. Stevens et al.
While the temporal dynamics of the process are regular, the means of acquiring the position or ‘fix’ provides opportunities for teamwork and cognitive complexity. If the visibility is reduced by fog, then a new set of navigation aids has to be selected, perhaps identifiable by radar, and now the team responsible for taking Rounds has changed with the swapping of the radar operator for the periscope operator. Interleaved with these deterministic events are situations that arise due to new ship traffic, increased proximity to hazards, equipment malfunctions, reduced visibility or similar events. In contrast to the regular updating of the submarine’s position, these events can be regarded as perturbations to the regular functioning of the team and provide interesting points where the resilience of the team may be tested. Some events rapidly appear like a man overboard while others evolve over 5–10 minutes and are based, in part, on previous decisions. The Debrief section is in the style of Team Quality Management which is an open discussion of what worked, what other options were available, and long and short term lessons. The Debrief is the most structured part of the training with team members reporting in order often beginning with the Navigator. Within this reporting structure there are overlapping or underlying nested structures where specific events within the Scenario are discussed. 2.3 EEG The ABM, B-Alert® system contains an easily-applied wireless EEG system that includes intelligent software designed to identify and eliminate multiple sources of biological and environmental contamination and allow real-time classification of cognitive state changes even in challenging environments. The 9-channel wireless headset includes sensor site locations: F3, F4, C3, C4, P3, P4, Fz, Cz, POz in a monopolar configuration referenced to linked mastoids. ABM B-Alert® software acquires the data and quantifies alertness, engagement and mental workload in real-time using linear and quadratic discriminant function analyses with model-selected PSD variables in each of the 1-Hz bins from 1–40 Hz, ratios of power bins, event-related power (PERP) and/or wavelet transform calculations. The data processing begins with the eye-blink decontaminated EEG files containing second-by-second calculations of the probabilities of High EEG-Engagement (EEG-E), Low EEG-E, Distraction and High EEG-Workload (EEG-WL) (Levendowski et al. 2001; Berka et al. 2004). The studies in this report have used the High EEG-E and EEG-WL metrics. The two metrics have different functional properties in response to different tasks and the two data streams are poorly correlated with one another; when averaged over six members of one SPAN team the R was −0.19 ± 0.24 with an R 2 of 0.09 ± 0.05 (mean & SD). The neuropsychological tasks used to build the algorithm, and subsequently used to individualize the algorithm’s centroids were presented using proprietary acquisition software. The algorithm was trained using EEG data collected during the Osler maintenance of wakefulness task (OSLER) (Krieger and Ayappa 2004), eyes closed passive vigilance (EC), eyes open passive vigilance (EO), and 3-choice active vigilance (3CVT) tasks to define the classes of sleep onset (SO), distraction/relaxed wakefulness (DIS), low engagement (LE), and high engagement (HE), respectively.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation Fig. 3 Expression of a generic NS measure by members of a six-person team
Simple baseline tasks are used to fit the EEG classification algorithms to the individual so that the cognitive state models can then be applied to increasingly complex task environments, providing a highly sensitive and specific technique for identifying an individual’s neural signatures of cognition in both real-time and offline analysis. These methods have proven valid in EEG quantification of drowsiness-alertness during driving simulation, simple and complex cognitive tasks and in military, industrial and educational simulation environments.
3 Design and procedure 3.1 Modeling neurophysiologic synchronies In prior studies that analyzed the dynamics of problem solving with individuals, we used the raw EEG-E and EEG-WL data streams (Stevens et al. 2007). For studying team processes, we chose a symbolic approach for combining the data rather than directly using the six concurrent EEG data streams from the SPAN teams. This allowed the current status of the team as a whole to be represented by a single symbol. An example of a such a symbol is shown in Fig. 3 for a six person team. At the point in time represented, team members 3 and 5 had above average levels of this particular neurophysiologic measure and the other team members were below average. To generate these symbols we equated the absolute levels of EEG-E of each team member with his/her own average levels over the period of the particular task. This allowed the identification of whether an individual team member was experiencing above or below average levels of EEG-E and whether the team as a whole was experiencing above or below average levels. As described previously (Stevens et al. 2010a) in this normalization process the EEG-E levels were partitioned into the upper 33 %, the lower 33 % and the middle 33 %; these were assigned values of 3, −1, and 1 respectively, values were chosen to enhance visualizations (Fig. 4a).1 1 Alternative numbering schemes such as assigning values of 3, 2 and 1 to the partitions designated as upper, middle or lower third were also tested. These resulted in an R 2 of >0.95 when the entropy of the
NS_E data stream was compared with that generated from models where the vectors were assigned values of 3, 1 and −1. The resulting histogram displays were more difficult to understand and so the 3, 1 and −1 convention was retained.
Author's personal copy R. Stevens et al.
Fig. 4 Layered analytic models for detecting and describing Neurophysiologic Synchronies. The details are provided in the text
3.2 Artificial neural network clustering The next step combined these values at each epoch for each team member into a vector representing the state of EEG-E for the team as a whole; these vectors were used to train ANN to classify the state of the team at any point in time (Stevens et al. 2010a). In this process the second-by-second normalized values of team EEG-E for the entire episode were repeatedly (50–2000 times) presented to a 1 × 25 node unsupervised ANN (Fig. 4B). The result of this training was a series of 25 patterns that we call neurophysiologic synchrony patterns that show the relative levels of EEG-E for each team member on a second-by-second basis.2 The reason for choosing a self-organizing ANN over another form of clustering like k-means was that during the ANN training a linear topology developed where the EEG-E vectors most similar to each other become adjacent through short-range excitatory interconnections while the more disparate vectors are inhibited and colocate further away. For instance, in Fig. 4B patterns 1–5 showed mainly low levels of EEG-E for all members of the team while patterns 22–25 showed mainly high levels. In pilot studies the team members often changed positions across SPAN sessions so single-trial ANN models were created using the data from a single performance. While these modeling approaches were informative, there were challenges for their 2 Preliminary models were created with 400, 100, 25 and 16 nodes and 25 nodes provided the best balance
between sensitivity and speed.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
Fig. 5 Team NS_E profiles after ANN training. The order of team member roles associated with each bar in the 25 symbols is shown below the figure
practical application to training activities. As new models had to be created for each task and team it was difficult to compare across models/teams as the ANN designations changed due to the probabilistic assignment of vectors to specifically numbered nodes and states. Also, without standardized models it would be difficult to extend this analysis to rapid team modeling. We addressed these limitations by generating generic (i.e. cross-subject and/or subject independent) ANN models from 8 teams that were uniform with regard to the number of persons being monitored by EEG and the team positions represented. This resulted in 31,450 team training vectors (∼8 hours of teamwork), which were used as the training set. The validation of these models has been described by Stevens and Gorman (2011), and all studies reported in this work have used these generic models. The 25 histogram patterns for NS_E obtained following ANN training are shown in Fig. 5, and these constitute the state space of the system, i.e. the possible NS_E levels across the six members of the team. Some of the ANN patterns represented times when very few of the team members were engaged and other patterns represented times when overall team engagement was high. A more dynamic picture of NS_E expression was gained from transition matrices which plotted the NS Pattern being expressed at time t with that at t + 1 (Fig. 6). The starting hypothesis was that many of the second-by-second changes in team engagement would be local. With the linear architecture of the self-organizing ANN and the resulting topology of the NS_E symbols this would be reflected in transition matri-
Author's personal copy R. Stevens et al. Fig. 6 NS_E transition matrix. The transition matrix for NS_E plots the NS patterns being expressed at times t (From) and t + 1 (To) seconds
ces as movement around a diagonal. Larger state space shifts may reflect the teams’ response to the evolving teamwork or external changes to the task. Transitions that were expressed/repeated more frequently than others are referred to as the attractors for the system. The predominant attractors for NS_E were around NS_E patterns 10, 15 and 24. Referring to the histogram profiles for these patterns in Fig. 5, NS_E pattern 10 was where the NAV, OOD and CC had below average EEG-E levels; NS_E 24 was where all team members had above average EEG-E levels, etc. 3.3 Modeling the entropy of NS data streams While transition matrices helped identify transition points and preferred NS patterns, a more quantitative measure of the teams’ attractor stabilities/instabilities would be useful for linking with other metrics of teamwork. As the NS patterns are symbolic, one approach would be to calculate the Shannon entropy of the NS data stream (Shannon 1951). The idea of entropy is derived from information science and is a measure of the level of uncertainty or “amount of mix” in a symbol stream. Calculated entropy is expressed in terms of bits and the maximum entropy that we could expect from the 25 NS patterns would be log 2 (25) or 4.64. To develop an entropy profile over a SPAN session the NS_E Shannon entropy was calculated at each epoch using a sliding window of the values from the prior 100 seconds. The idea was that as teams entered an attractor state the entropy would decrease as fewer of the 25 NS_E patterns would be expressed.
4 Analysis 4.1 Capturing task-induced shifts in NS attractors Conceptually, one of the challenges in approaching a nonlinear description of a system is the definition of order parameters and control parameters. Order patterns are system properties that change, often discontinuously, as a result of an external condition. Control parameters are those that can be manipulated to induce instabilities causing the system to enter into different states.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
Fig. 7 Attractor basins for NS_E. The top figure shows the transition matrix for team T4S2, and the bottom figure shows the NS_E entropy from the same data stream
The first studies decomposed the NS_E transition matrix shown in Fig. 6 into periods representing the Briefing, Scenario and Debriefing segments (Fig. 7). The goal of this decomposition was to determine if NS_E patterns could serve as order parameters; i.e. would there be a change in the attractor basins in response to changes in the task (which could be considered a control parameter).
Author's personal copy R. Stevens et al.
In describing the status and functioning of the team through attractors, several features arose. First, the data streams for NS_E showed different and distinct attractor basins during the three parts of the simulation. This is suggestive of there being major cognitive reorganization events associated with changes in the task. Next, the attractors were not static within each segment of the simulation, but formed and dispersed multiple times within each segment as indicated by the transition matrices as well as the peaks and valleys in the entropy streams. Periods where the NS_E entropy was low were associated with a limited pattern of NS_E transitions; periods of higher entropy had less well organized transition matrices. Finally for NS_E there were few periods in any of the Scenario segments where all team members simultaneously and persistently had high levels of engagement (which, referring back to the Nodal Map would have been represented by NS_E patterns 14, 15, 21, 24). Instead the dominant attractor states were those where the majority of the team members had low E (NS_E patterns 10 & 11). 4.2 SOAC and experienced SUB navigation teams express different NS_E attractors We next examined the experience of the team as another possible control pattern. To focus on teamwork, these studies were conducted only with the Scenario segment from six SOAC teams and six experienced (SUB) SPAN sessions. The NS_E transition matrices with a 1 second lag are shown in Fig. 8. The diagonal line shows the persistence and local transitions of NS_E patterns with the more frequent transitions shown by the higher contours. The dominant attractor for SOAC teams was centered near NS_E patterns 10 & 11. From Fig. 5, this was where half of the team members had low EEG-E. The attractors for SUB teams clustered near NS_E patterns 22–25 with most of the team members showing high EEG-E. A second attractor was centered near NS_E pattern 15 where again most of the team showed above average EEG-E. Cross tabulation analysis showed the two groups of teams were significantly different from one another (χ2 = 298, df = 24, p < 0.001). The SUB teams also showed more minor transitions as evidenced by the darker background contours throughout the matrix. This increased use of minor transitions suggested that direct comparisons of the NS_E entropy streams may be a useful indicator of team experience. The NS_E patterns from 14 SOAC and 5 SUB team sessions were generated by testing the EEG-E data streams on the generic ANN model and then calculating the NS_E entropies. SUB teams had the highest levels of NS_E entropy while the lowest entropies were associated with the first two sessions of SOAC teams. The histograms in the top figure show the progressive increase in NS_E entropy as two of the teams gained experience (Fig. 9). 4.3 Linking NS_E entropy and transition matrices with the accuracy of Rounds SPAN simulations have the advantage of high ecological validity as they are realistic simulations and required components of the SOAC curriculum. A limitation of SPAN from a research perspective however, is that there are not detailed performance scoring criteria and most performance issues are raised and discussed during the Debriefing. A possible proxy for a performance score would be the regularity by which
Author's personal copy Modeling the neurodynamic complexity of submarine navigation Fig. 8 NS_E transition matrices for SOAC (top) and SUB (lower) teams during the Scenario. The samples included data from six SOAC (14,473 epochs) and six SUB (11,422 epochs) SPAN sessions
Rounds are conducted. This periodic updating of the submarines’ position is conducted every three minutes with a 5-step countdown during the last minute. The regularity of this countdown, along with possible deviations, can be obtained from the speech of the REC who is responsible for the countdown. Figure 10 shows the Rounds sequence for 5 SPAN sessions. All teams had periods where the rhythm of Rounds was broken. These periods are highlighted by gray boxes. These irregularities can be caused by making a turn, avoiding traffic or overloading of the team; they often indicate stressful conditions (Stevens and Gorman 2011). The SPAN sessions are listed in the order of decreasing overall NS_E entropy. Also shown in Fig. 10 are the transition matrices for each session as well as a secondby-second NS_E entropy profiles. The two SUB team sessions, E1S1 and E1S2 mostly showed regular and complete 5-step Rounds countdowns. These sessions also had the highest overall NS_E entropy which was evident in (1) the more patterned background of the transition matrices and, (2) the less jagged profile of the NS_E data stream. The Rounds sequence patterns were more irregular for SOAC teams T4S2 and T5S5 where steps, and occasionally a complete Rounds sequence were omitted. These teams showed more restricted transition matrices and more prominent attractor
Author's personal copy R. Stevens et al. Fig. 9 NS_E entropy levels from SOAC and SUB teams. The overall entropy of NS_E data streams during the Scenario portions of SPAN are shown for the first two and subsequent performances by SOAC teams and these are compared with 5 experienced submarine navigation teams (SUB)
basins than did the SUB teams. The NS_E entropy profiles also contained more peaks and troughs. Another expert team, E4S2, started the Scenario with four effective fixes and then began having difficulties conducting regular rounds; this example suggests there may be levels of expertise. 4.4 Linking NS expression with speech frequencies during SPAN We next examined the associations between speech frequency and NS_E expression during SPAN. The dialog of SOAC team T4S2 was transcribed and coded as to who was speaking and when. The frequency of speech of the 10 most verbal team members was then aggregated into 100 second bins (Fig. 11). The expression frequencies of the 25 NS_E patterns were similarly binned. There were parallels between overall speech frequencies and NS_E expression in that the distributions changed at the task junctions. During the Debriefing most of the talking was done by the CAPT while during the Scenario the CC, OOD, NAV and REC spoke most frequently. The frequency of speech by these persons gradually increased during the middle of the Scenario (epochs 1600–2600). Similarly, NS_E patterns 16–19 were most frequently expressed during the Debriefing, but were nearly absent during the Scenario. NS_E expression during the Scenario was, however, more dynamic than speech frequencies by different members of the team. This could be particularly seen during epochs 1800–2100 with the increased expression of NS_E patterns 8, 9, 11, 14 and 15.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
Fig. 10 Entropy, Rounds and transition matrices for 5 SPAN sessions. In this figure, the sequences of Rounds are ordered by decreasing levels of NS_E entropy. The periods highlighted in gray indicate periods where the Rounds sequence were irregular in the sense that specific steps were deleted or duplicated. Below each Rounds sequence is a histogram showing the second-by-second NS_E entropy fluctuations. To the right are the NS_E transition matrices for the Scenario segment of each performance
4.5 Linking the entropies for speech and NS_E with teamwork events To derive a detailed understanding of these relationships we compared the entropy of NS_E patterns, with the entropy of speech (Fig. 12), and then linked these to different events during the SPAN session. These linkages were facilitated by an evaluation
Author's personal copy R. Stevens et al.
Fig. 11 NS_E pattern expressions and team member speech frequencies during SPAN. The expression of the 25 NS_E patterns (top) and the frequency of speech by different team members (bottom) of SOAC team T4S2 were aggregated into 100 second bins during the Brief, Scenario and Debrief; the SPAN segments are also indicated by dotted lines
made by a SOAC instructor from the audio files of the simulation; his comments are shown in Fig. 13. The entropy for speech was calculated by first assigning a code to the 16 speakers in the SPAN. These symbols were substituted into the speech log and then the entropy was calculated as described for NS_E. The briefing began with the sailors chatting, accounting for the high speaker entropy. The entropy for NS_E was also high indicating a diversity of NS_E patten expression. The Speech entropy fluctuated as the NAV and CC took turns briefing the team and setting up for a static Round. The first dip in NS_E entropy began around epoch 650 and reached its lowest when a decision about how to pass a tugboat was made (epoch 820). There was then a period of quiet chatter and dropping Speaker entropy with mainly the OOD and CC speaking (from Fig. 11) until around epoch 1000 when the backup Visual Mapping System (VMS2) failed and all contacts were lost.
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
Fig. 12 Entropy profiles for NS_E and speech. The median values for each of the entropy profiles are shown by the horizontal line. The event numbers relate to those in the evaluation log (Fig. 13)
There was little cognitive reorganization of the team however and the NS_E entropy continued to rise; contacts continued to be tracked. The next drop in NS_E entropy occurred around epoch 1350 when the military GPS was lost; this was followed by an increase in Speaker entropy (many people speaking) which continued for the next 25 minutes. Around epoch 1800 the team began a significant cognitive reorganization, evidenced by the drop in NS_E entropy. The team was in a position of limited maneuverability and visibility with an approaching contact with a predicted close point of approach (CPA) of zero. The entropy for NS_E began to rise as the ship left the area of restricted maneuverability and the danger of collision passed. An area of particular interest was the period where there was a man overboard (MOB) event (epochs 2845–3358). There were slight dips in NS_E entropy but much less so than during the periods of restricted maneuverability and possible collision; Speaker entropy was also approaching median levels. The notes of the LT. reviewing the performance provide a possible perspective why.
5 Discussion Our goal for studying NS expression is to be able to rapidly determine the functional status of a team in order to assess the quality of a teams’ performance/decisions, and to adaptively rearrange the team or task components to better optimize the team. The results of the current studies make several contributions towards these goals. First, we have outlined a framework for inserting neurophysiologic measures into the study of complex teamwork. The task highlighted in this work, simulated submarine piloting and navigation, is a task required of all SOAC Junior Officers and
Author's personal copy R. Stevens et al.
Fig. 13 Evaluation of a SOAC team performance. An evaluation of the audio log for this performance was made by a SOAC instructor who summarized the progress of the simulation and his perception of the stress level of the team
teams will practice it multiple times over a nine-week period. Our results are more generally applicable, however, as similar results have been obtained from high school students performing online scientific problem solving (Stevens et al. 2009) and AntiSubmarine Warfare teams. This methodological framework for studying team neurodynamics was shaped by multiple factors. The metric we have studied is an EEG-derived measure of Engagement which is an approximation of how this term is represented in the literature. Cognitive Engagement for instance has been used to describe the amount of cognitive processing a learner applies to a subject (Howard 1996). It has been thought of as
Author's personal copy Modeling the neurodynamic complexity of submarine navigation
something that has to be broken during a task so that a learner can reflect on his/her actions, (Roberts and Young 2008). It shares similarities with alertness or attention and can be visual and/or auditory. Engagement can be elevated through a variety of activities such as inducing cognitive dissonance, posing argumentative questions requiring the development of a supportable position, and causing learners to generate a prediction and rationale during a lesson. It has been measured by surveys (Huber 2008), self-reports (Appleton et al. 2006), and for commercial products. Engagement can be thought of in terms of the brand, the brand message, or the context (Plummer et al. 2007). For the foreseeable future, therefore, we have to accept the premise that precise terms will be difficult to associate with different EEG-derived cognitive measures and that functional associations will need to be empirically derived. One functional property would be the resolving power, i.e. over what periods of time can associations be made between a neurophysiologic metric and aspects of team function. From Fig. 12 and previously reported studies (Stevens et al. 2010b) NS Patterns can rapidly change in response to changes in the task environment. To put these fluctuations into a broader context, one comparison that could be made is between the EEG measures and some other measure of the teams’ response to evolving events like eye blinks. Simultaneous eye blinking by some or all team members could also be a product of team synchronization to important events in a SPAN simulation. Embedded within the EEG data stream from each team member are eye blinks and the algorithms used by ABM for calculating EEG automatically detect and decontaminate the EEG streams by a process of interpolation. As the interpolation represents ∼5 % of the simulation time they are not likely to overly influence the calculation of NS_E, but the log files can be used to associate eye blink synchronization with SPAN events. Approximately 50.2 ± 4.5 % of the epochs contained an eyeblink.3 The eye blink data for each team member was calculated at the same frequency as NS expression (1 second) and then a series of inner joins were conducted on the data set across team members to derive the epochs where 2, 3, 4, 5 or all 6 team members simultaneously blinked. The distribution of these epochs is shown in Fig. 14 for the team performance T4S2 highlighted in the study. Whether the synchronization was examined across 3, 4, 5, or 6 team members, there were few obvious patterns either around the task junctions or significant events. The possible exceptions were around epochs 1000 and 3000. Comparing this figure, with the NS_E expression shows that NS expression has greater differential resolving power than eye blinks. Another methodological consideration was deciding to normalize the data over the period of the task, rather than an alternative like excursion from baseline. This decision was based on prior comparisons of individual’s EEG-Workload when they performed a task alone vs. when they performed it in a team (Stevens et al. 2009). In this study 15 high school students performed science simulations alone, or in a 3 Across the 6 members of one team the median decontamination period was 840 milliseconds and within
this window the first 160 ms and the last 320 ms were reference segments used for interpolation. The decontamination algorithm then examines each remaining data points (∼360 ms) and if 2 SD above the mean reference segment then they were interpolated. This resulted in ∼25 % of the segments being interpolated or ∼90 ms per second, or 45 ms/eye blink
Author's personal copy R. Stevens et al.
Fig. 14 Frequency of simultaneous eye blinks across team members
group of 3. The average EEG-WL was compared with a neutral baseline task which was the 3-choice vigilance task (3-CVT). When working in a team the same students had 20 % higher EEG-Workload levels than when working on the same task alone. This presumably represents the ‘process costs’ of teamwork which are the additional cognitive costs involved in teamwork resulting from the monitoring of others in the team (Cooke et al. 2008). Normalizing the data over the period of the task neutralizes the possible differences in these costs across team members. SPAN teams have cognitive attractors, or more appropriately attractor basins for NS_E that the team frequently visits. They did not appear to be on a common transition pathway between other attractors as they persisted for tens of seconds to minutes i.e. they were more stable behaviors. The localization of a team into an attractor basin could be detected by changes in the entropy of the NS Pattern data stream. Not all dips in entropy levels had the same attractors. For instance, during periods around epoch 2000 the attractor was centered on NS_E pattern 11; a prior decrease in entropy around epoch 800 was associated with an attractor near NS patterns 24 & 25. We would therefore consider entropy fluctuations as a first level of analysis indicating that the team is re-organizing itself into another state. It does not tell what state it is and so the identity of the attractors also has to be determined. A detailed movie showing the associations between NS_E, entropy, NS_ patterns and events from the performance log can be found at: the following URL,4 and a sample screen shot from this movie is shown in Fig. 15. This movie shows the track of the submarine and other contacts, the transition matrix, entropy profile, and entropy gauge showing the current NS_E entropy and an interactive map showing which NS symbols are represented by the different attractors. The idea of attractors could reasonably be expanded into the idea of nested attractors whose expression is directed first by the task segment and next by the immediate environment within each segment of the task. These ideas may have training significance as: (a) more experienced teams have higher overall NS_E entropy, and (b) the 4 http://www.teamneurodynamics.com.
Fig. 15 Online simulation of team neurodynamics. Details regarding the movie can be accessed at: http://www.teamneurodynamics.com/
Author's personal copy
Modeling the neurodynamic complexity of submarine navigation
Author's personal copy R. Stevens et al.
attractor basins for novice and experienced teams during the Scenario segment are different for novice and experienced teams (Fig. 8). The periods of reduced entropy were places where the team had cognitively reorganized itself as revealed by the transition matrices associated with the high and low entropy levels. From the event logs and expert evaluation these seemed to be periods of increased stress. The fine dynamics of more examples will need to be studied to determine whether they represent a teams’ response to increased stress, or whether this re-organization and perhaps lack of flexibility contributed to the stress. From the event logs these periods could also be viewed as ones where the team had to make important decisions and so perhaps tension would be a better term. While we have applied ideas from NLD, we have not yet applied other available NLD measures such as other forms of entropy such as Kolmogorov entropy or analyzed the expression of Hurst and Lyapunov exponents to quantitate the ‘ruggedness’ of entropy streams. The data are amenable to such analyses and will likely enhance our understanding of NS Pattern dynamics in teamwork. A key step for making these studies possible was the development and validation of cross-subject/subject independent models. Cross-subject models are those where the performance data from multiple teams are contained in the training set and then each team is re-tested with those models. Subject-independent models are where a team that was not in the training set is tested on the combined models. The team highlighted in this study T4S2 was not included in the training set. We have subsequently tested 6 other SPAN performances from novice/experienced navigation teams and compared the resulting data with that derived from their single-trial models. These results show strong concordances in NS_E entropy levels, particularly during the Scenario segments of SPAN suggesting that the development of subject-independent models may be possible. The continuing development, validation and application of such models of NS expression will allow direct comparisons across teams, levels of experience and effects of training. The refinement of these models will help accelerate the development of rapid modeling systems. As a first step, we have incorporated the generic models for NS_E into an online application that accepts raw EEG from each team member and rapidly outputs ANN Patterns and entropy levels. This will provide a working framework for near real-time formative and post-hoc assessment of team performance. Acknowledgements We extend a special thanks to Adrienne Behneman for the data collection and to Drs. Jamie Gorman and Polemnia Amazeen for helpful discussions. Approved for Public Release, Distribution Unlimited. “The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.” This is in accordance with DoDI 5230.29, January 8, 2009. This work was supported by NSF SBIR awards 1215327, 0822020, Office of Naval Research award N00014-11-M-0129, and an award from the Defense Advanced Research Projects Agency (DARPA) under contract numbers NBCHC070101, NBCHC090054 and W31P4Q12C0166.
References Appleton J, Christenson S, Kim D et al (2006) Measuring cognitive and psychological engagement: validation of the student engagement instrument. J Soc Psychol 44:427–445
Author's personal copy Modeling the neurodynamic complexity of submarine navigation Berka C, Levendowski DJ, Cvetinovic MM, Petrovic MM, Davis G et al (2004) Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int J HumComput Interact 17(2):151–170 Cooke NJ, Gorman JC, Kiekel PA (2008) Communication as team-level cognitive process. In: Letsky MP, Warner NW, Smith CAP (eds) Macrocognition in teams: theories and methodologies. Ashgate, Farnham, pp 51–64 Cooke NJ, Gorman JC, Rowe LJ (2009) An ecological perspective on team cognition. In: Salas E, Goodwin J, Burke CS (eds) Team effectiveness in complex organizations: cross-disciplinary perspectives and approaches. SIOP organizational frontiers series. Taylor & Francis, London, pp 157–182 Gorman JC, Cooke NJ, Amazeen PG (2010) Training adaptive teams. Hum Factors 52:295–307 Howard B (1996) Cognitive engagement in cooperative learning. In: Annual meeting of the eastern educational research association, Boston, MA, 21–24 February 1996. http://www.eric.ed.gov/ ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED404352 &ERICExtSearch_SearchType_0=no&accno=ED404352. Accessed 8 June 2011 Huber L (2008) Measuring cognitive engagement in middle school students. University of South Dakota thesis Krieger AC, Ayappa I (2004) Comparison of the maintenance of wakefulness test (MWT) to a modified behavioral test (OSLER) in the evaluation of daytime sleepiness. J Sleep Res 13(4):407–411 Levendowski DJ, Berka C, Olmstead RE, Konstantinovic ZR, Davis G, Lumicao MN, Westbrook P (2001) Electroencephalographic indices predict future vulnerability to fatigue induced by sleep deprivation. Sleep 24 (Abstract Supplement): A243–A244 Lindenberger U, Li S-C, Gruber W, Muller V (2009) Brains swinging in concert: cortical phase synchronization while playing guitar. BMC Neurosci 10:22–34 Nowak A, Vallacher R (2007) Dynamical social psychology. Guildord Press, New York Plummer J, Cook B, Diforio D, Schacter B, Sokolyanskaya I, Korde T (2007) Measuring cognitive engagement, vol. II. http://www.classmatandread.net/class/2.Measures~%20of~%20Engagement.pdf. Accessed 8 June 2011 Roberts E, Young RM (2008) Maintaining cognitive engagement in training scenarios using explicit cognitive models. In: Interservice/Industry service training and education conference (I/ITSEC), paper 8217, pp 1–9 Shannon CE (1951) Prediction and entropy of printed English. Bell Syst Tech J 30:50–64 Stevens RH, Galloway T, Berka C (2007) Allocation of time, workload, engagement and distraction as students acquire problem solving skills. In: Schmorrow D, Nicholson D, Drexler J, Reeves L (eds) Foundations of augmented cognition, 4th edn, pp 128–137 Stevens RH, Galloway T, Berka C, Sprang M (2009) Can neurophysiologic synchronies be detected during collaborative teamwork? In: Proceedings: HCI international 2009, San Diego, CA, 19–24 July 2009, pp 271–275 Stevens RH, Galloway T, Berka C, Behneman A (2010a) A neurophysiologic approach for studying team cognition. In: Interservice/Industry training simulation and education conference (I/ITSEC) 2010, Paper No 10135 Stevens RH, Galloway T, Berka C, Behneman A (2010b) Identification and application of neurophysiologic synchronies for studying team behavior. In: Proceedings of the 19th conference on behavior representation in modeling and simulation, pp 21–28 Stevens RH, Gorman G (2011) Mapping cognitive attractors onto the dynamic landscapes of teamwork. In: Proceedings of the 14th international conference on human-computer interaction, Orlando, FL, 9–14 July 2011 Stevens RH, Galloway T, Wang P, Berka C (2012) Cognitive neurophysiologic synchronies: what can they contribute to the study of teamwork? Hum Factors 54:489–502. 0018720811427296, first published on 12, 2011. doi:10.1177/0018720811427296 Warner N, Letsky M, Cowen M (2005) Cognitive model of team collaboration: macro-cognitive focus. In: Proceedings of the 49th human factors and ergonomics society annual meeting, Orlando, FL, 26–30 September 2005
Ronald Stevens is Professor and member of the Brain Research Institute at the UCLA School of Medicine and the CEO of The Learning Chameleon, Inc. For several years he has been directing the Team Neurodynamics Project which is developing neurophysiologic models of teamwork.
Author's personal copy R. Stevens et al. Trysha Galloway directs the EEG studies for The Learning Chameleon/IMMEX™ laboratory and is coauthor on multiple peer reviewed published studies. Peter Wang is the programming director for the IMMEX™ Project. During the past twenty years he has jointly developed hundreds of online scientific problem solving and data mining applications. Chris Berka CEO and Co-Founder of Advanced Brain Monitoring and has over 25 years of experience managing clinical research and developing and commercializing new technologies. She is co-inventor of seven patented and seven patent-pending technologies. Veasna Tan directs multiple EEG-related studies at Advanced Brain Monitoring, Inc. Thomas Wohlgemuth is Chief Technical Officer, Submarine Training Systems. Jerry Lamb is the Technical Director of the Naval Submarine Medical Research Laboratory. Robert Buckles is a SOAC Training Manager.
