Understanding coordination in emergency response Hedvig Aminoff1 Björn Johansson2 Jiri Trnka2 (1) Infocentret Ankaret Hagalundsgatan 33 169 66 Solna
[email protected] (2) Department for Computer and Information Science Linköpings Universitet SE-581 83 Linköping, Sweden E-mail:
[email protected],
[email protected]
Abstract This paper describes and discusses analysis of an emergency management exercise. In the exercise scenario, different emergency management organizations jointly try to cope with a forest fire and related incidents. The Extended Control Model is utilized for examination of the establishment of en emergent emergency response organization. Ambiguity in how functions are to be handled in a large event, indicating vulnerabilities in face of larger crises; functions moving across roles during the evolving event; and recognizable phases of a response are uncovered. This is assessed by utilizing episodic analysis of the communication between different functions and roles in the participating emergency management organizations. The results indicate requirements for future information and communication technologies, and occurrences that can be explored in future studies. Keywords: Extended Control Communication, Coordination
1
Model,
Emergency
management,
Introduction
In crises and emergency management (EM), exception is the norm: it is often difficult to know in advance which persons will assume which roles and what they will decide to do as immediate actions (Perrow, 1984; Turoff, 2002). This often results in an ad hoc ensemble of commanding officers from different emergency management organizations working together as a team on sometimes unpredictable and novel tasks (Johansson, 2005). In addition, these commanding officers often perform without spatial and temporal proximity (Artman, 1999). The lack of collocation is underscored by the nature of dynamic situations, which by definition are difficult to predict and plan for,
and complex to assess (Brehmer, 1992). Emergency response requires coordination between individuals, organizations and artifacts (Johansson, 2005; Johansson & Hollnagel, 2007). In order to understand such an organization, it is necessary to realize that is not an entity which has a given shape and function, rather it takes form as adaptations to the unfolding situation, according to constraints set by the organizational, social and technological structures and the operational missions of the commanding officers (Aminoff, 2007). It is difficult for any person to gain insight into involved control processes in an emerging situation, or even retrospectively find out what happened and what was done (Trnka, 2007). Emergency response operations are rarely reviewed, and it is difficult to document and catch organizational learning (Turoff, 2002). The consequences of the regulator paradox (Weinberg & Weinberg, 1988) augments this lack of insight in operations, as a work system can suffer a lack of feedback about its performance. Thereby it can operate without awareness of vulnerabilities or approaching breakdowns, until a critical incident occurs. In this paper, an emergency management exercise is analyzed from a Cognitive Systems Engineering (CSE) perspective, using the concept of the Extended Control Model (ECOM) and episodic analysis of communication.
2
Background – Joint Cognitive Systems and the ECOM
A cognitive system is defined as “a system that can modify its behavior on the basis of experience so as to achieve specific anti-entropic ends” (Hollnagel & Woods, 2005, p. 22). A Joint Cognitive System (JCS) is made up of two or more cognitive systems, of which at least one includes a human. In other words, humans interacting with other humans and machines are seen and studied as a JCS (Hollnagel & Woods, 2005). Humans using machines are thereby parts of a whole in co-agency with machines to maintain control (ibid.). A functional account of a JCS includes a description of the processes to maintain control, without necessarily focusing on who performs them: the conglomerate of humans, machines and the context holds a range of actions to meet demands. System boundaries are established relative to the context and the system's function as a whole. 2.1 The Extended Control Model In order to understand these processes, descriptions of patterns in performance are needed (Woods, 2003). The Extended Control Model (ECOM) (Hollnagel et al., 2003; Hollnagel & Woods, 2005) identifies patterns in control behavior, based on performance rather than information processing. The difference lies in part in the description of feedback processes, which underline the dynamics that exist between perceiving and acting. The model has its foundation in the basic cyclical model of human action (Hollnagel, 1993; 1998), which is based upon principles of Neisser’s (1976) perceptual cycle and extends from describing perception to describing action and control. The ECOM model provides means to identify patterns in activities that are performed on parallel, interacting levels in a JCS. The ECOM describes performance in relation to activities on four different, goal-related, levels (see figure 1.):
Targeting - the controller’s, or controlling system’s expectations of what will happen in the future derive plans and goals. Short and long term goals are set up and prioritized, which affects lower levels. Monitoring - the controller keeps track of the environment and its own actions, and produce plans based both on feedback from lower levels and expectations from the higher level. The plans generated are used by the regulating and tracking loops. Regulating - resources are managed in relation to goals and changes in the environment, often involving a number of tracking sub-loops. Tracking - feedback tasks controlled by higher levels’ goals and targets are performed.
Figure 1. The Extended Control Model (Hollnagel & Woods, 2005)
According to the ECOM, performance takes place simultaneously on concurrent loops of activity, i.e. performed at the same time and interconnected with regard to goals but within different time frames. Their coupling is represented in the way the higher levels' output serves as input for the subordinate level in the form of objectives. Thereby, a direct interdependence between objectives/plans and activities is portrayed. Different types of assessments are needed at each level, exemplified through activities in the JCS. A loss of control may thereby be directly linked to a disturbance in the interaction between the levels. A JCS must be in control of all loops at the same time: dependencies between the levels must have appropriate interfaces to each other otherwise the input from higher loop to lower in form of objectives not appropriately acted upon and vice versa. 2.2
The ECOM as a tool for understanding large scale coordination
In distributed coordinated activity, communication gives an indication of where there is interaction between levels of activity. The output from lower levels in the form of feedback, measurements and situation assessment thereby provides substance for upper
levels to refresh their plans or current understanding. Possible applications of the model are to provide means to analyze controller performance under given conditions. The ECOM does however not describe how the levels should be assessed or analyzed. Our suggestion is that, when studying large organizations that are form and re-form in dynamic situations, that both the levels as well as the interaction between them can be assessed by analyzing the communication between different actors, or “functions” in the participating organization.
3
The case – ALFA-05 exercise
The paper reports findings from an emergency management exercise, ALFA-05, (Trnka & Jenvald, 2006; Trnka, J., Johansson, B. & Granlund, R., 2006; Woltjer, Trnka, Lundberg & Johansson, 2006; Trnka, 2007), which took place in Linköping, Sweden, December 1, 2005. 3.1
Exercise settings and scenario
The exercise was designed as a real-time closed-room research simulation. The scenario was built on a single, self-contained event. This event was a middle-sized forest fire located at the border area of two municipalities, located in two different Swedish counties, during summer time. There were a number of additional incidents (disturbances) in the scenario. The additional incidents were: (1) traffic disturbances on surrounding roads, (2) threat from the fire to a neighbouring zoo with several thousands of visitors, (3) search and rescue of a group with small children on a picnic, (4) a life threatening allergic reaction of one of the responding fire-fighters, (5) a traffic accident on the nearby highway, (6) a fire engine failure, and (7) pressure from media requiring incident information. The simulation of the scenario took place on Saturday, July 2, 2005, starting at 10:30 and continuing for approximately two hours (not to be confused with the actual time of the execution of the exercise itself, see above). The scenario was planned as a continuous real-time scenario, where scenario time was equal to real-time. The participants were eight commanding officers from the concerned EM organizations in the scenario location: • County 112/911 emergency operator (2x) • Municipal fire & rescue on-site incident commander (2x) • Municipal fire & rescue dispatch officer (1x) • County police on-site incident commander (1x) • County police dispatch officer (1x) Due to its complexity and dynamics, the exercise is controlled by several umpires (game masters). The exercise staff is a team of researchers and exercise managers, which are supported with various computer-assisted tools, such as visualisation tools, geographical information systems, simulations, surveillance equipment, etc. Figure 2. shows the organisation of the exercise.
Figure 2. The organisation of the observed Exercise (Trnka, Johansson & Granlund, 2007) Communication between the participants was by text messages only. The exercise was followed by an after-action review. During the exercise and the after-action review a wide range of data was collected, including (a) recording of all text message communication, (b) video recording of the participants’ workplaces, (c) video recording of the after action review, (d) direct observations collected by three observers during, and (e) all materials used in the exercise and after-action. In this paper, the analysis is based on the text messages exchanged between the participants.
4
Data analysis
The communication between the different participants in the exercise was examined according to two complementary lines of inquiry. To follow an exchange of messages is in itself difficult due to the fragmentation into parallel activities, yet simply extracting exchanges between two commanding officers risks belying the complexity of communications in a coordinated response, therefore effort was taken to enhance the understanding of what was taking place, from several different perspectives. In initial readings of the communication logs, all exchanges from each individual participant were sorted. This provided an image of each participant’s communications and access to information at each point in time. In addition, exchanges between dyads were sorted. This provided an image of the specific functions, which certain roles provide to each other. 4.1 Episodic analysis The communication analysis is based on dialogism, described by Linell (1998), using episodic analysis (Linell & Korolija, 1997). This method has earlier been suggested as having potential as a tool for analyzing human interaction in dynamic environments (Johansson & Waern, 2000). Our ambition is to trace the development of events through the communication recorded during the ALFA-05 exercise, to combine the knowledge from the scenario used in the exercise with the recorded development from a multitude of sources.
The aim of analyzing the communication was not to define interaction qualities; rather, the focus was on the behaviors the communication relates to. The coconstruction of meaning in the activities undertaken by the various actors was thus central. This was attempted by looking at the function of the interactions, and how an understanding of the situation evolves through exchanges of information, directives and feedback. With this aim in mind, frequency counts of types of speech acts would not be sufficient (Stroomer & van Oostendorp, 2003). Instead, the focus was on content and the behavior it appeared to be related to (ibid.). Topics that can be followed in an entire discourse or in parts of it are referred to as episodes in Linell & Korolija (1997). An episode is “a unit of natural social interaction, manifest at a structurally intermediate, or a global level of conversation”. In this study, the topic is what guides the delineation of an episode. However, no strict method of coding episodes (e.g. Topical Episode Analysis, Linell & Korolija 1997) has been used. Rather, coherence, which is related to what is said, how and when, has been used as boundary (Downing, 2000). 4.2 Applying the ECOM The final step was to interpret ECOM for the domain. An analysis of a JCS demands a definition of the boundaries between internal components and between the system and its environment (Hollnagel & Woods, 2005). According to Hollnagel & Woods (2005), drawing system boundaries demands attention to whether or not the potential system components are possible for the JCS to control, and whether or not they are important to system function. When analyzing the ALFA-05 exercise, the entire group of participants is included in the JCS. The control loops in ECOM provided a structure for identifying components according to function, by identifying control loops in concurrent activities.
5 Result - Making Sense of Inter-organizational Emergency Management The analysis aims at looking at the involved EM participants as one JCS, which in this case is a complex and changing set of organizations trying to make sense of, and gain control over, a dynamic event (the forest fire and other incidents). There are in many cases no clear picture of how coordination between municipalities and counties is to proceed. Instead, the development evolves in according with the changing conditions, and the JCS is constantly being re-configured in respect to the ECOM levels. The following episodes were identified in the material: 1. Identification and localization of the reported fire 2. Establishing a staging area 3. Search for the lost small children 4. Protection of a valuable collection of antique cars 5. The hand-over of tasks from the fire & rescue services dispatch officer to the fire & rescue on-sire incident commander.. It is apparent that the EM organizations work reactively and proactively - a consequence of the larger incident is that the responders anticipate sub events. As these
accumulate, they finally reach a critical mass, at which point higher command resources are called in. 5.1
Using the ECOM to understand the dynamic nature of EM
The exact constellation of an entire group of actors in an emergency operation as the one described above is partly unknown ahead, even if smaller units may be relatively constant and the deployment of units is standardized according to certain parameters, such as common operational procedures. It was, in this case, possible to analyse the behaviour according to the patterns described in the ECOM: targeting, monitoring, tracking and regulating. These functional levels all have their own specific characteristics in relation to the ongoing activities. These were possible to identify given domain knowledge of the scenario and the involved participants roles in the exercise. Below follows excerpts from various situations in the exercise, presenting examples of all four levels of the ECOM. 5.1.1 Targeting Targeting is the highest level in the ECOM and includes synthesizing information/feedback and general assessments with reference to present context, preevent plans and mission statements. In normal operations, targeting objectives can be seen as present in the definition of each organization’s mission and in the allocation of roles and resources. It is also present in pre-event activities such as risk analyses and planning. When an incident exceeds a certain complexity or degree of novelty, these templates need to be complemented by human agents which can support sublevels in the need for a long term, synthesizing perspective. The frequency of assessments may be connected to factors such as experience, precedence, seriousness, risks, and consequences. Assessing changes relative to the long-term goal is founded on a wide situation perspective. This demands revising the criteria for adjacent loops, by receiving and synthesizing feedback from performed actions and constant updating of understanding to see if new developments have occurred, or whether hypothesis need adjusting or not. There are repeated examples of this occurring through cycles of updating and reassessments in the exercise scenario logs, throughout the different JCS constellations (see table 1). Table 1. Example of targeting. County Police To : Operative on-scene Dispatch E-county commander E-county Time : 11:51:21 Operative on-scene To : County Police commander E-county Dispatch E-county Time : 11:51:52
har vi läget under kontroll på plats? Is the situation under control on location? Nej jag vet ännu inte omfattningen av branden men då det kan vara liv i fara måste vi ha mer folk No I don’t know how large the fire is but since there may be a risk for lives we need more people
In the example the county dispatch center for the police wishes to receive a general assessment of the scenario. The county police on-site incident commander replies that
the situation is not well understood but that additional resources are necessary in order to avoid the threat of injury. This exemplifies the repeated cycles of updating and reassessment that take place. The high-level targeting goal of protecting lives sets the path for the chain of activities that follow. Cyclical reassessments lead to chains of events e.g. new plans/objectives such as new priorities for actions (monitoring level) and deployment or redistribution of personnel and vehicles (regulating level). An example is when the news of a lost pre-school class in the fire area surfaces. The incident leads to revision of criteria for the other loops, such as regulating, as vehicles must be redistributed and priorities changed. A high priority such as a group of children lost in an area close to a forest fire affects all levels of activities. 5.1.2 Monitoring Monitoring concerns the state of the JCS in relation to the goal; how the JCS should be manoeuvred with relation to the goal in the situation at hand. It involves setting objectives and choosing plans for action, considering the activities relative to the environment and developing incident, i.e. a higher-level goal than in regulating. This requires a constant awareness of the state of the operators and equipment, and the status and location of vehicles. Information for this is provided as feedback from the regulating level. This forms a basis for whether or not resources should be relocated. Monitoring events become traceable in the dialogs by information passed to the targeting level actors, and by plans/objectives communicated to those performing regulating activities. Examples: ● Activities which include making sure that sufficient resources are available to cover other areas of the jurisdiction ●
Judging and setting priority for coinciding tasks.
These decisions shape regulating activities and provide vital information for assessments on a targeting level. Table 2. Examples of monitoring and tracking. Operative Commander To : County Police Kan du uppdatera mig om var alla bilar är nu E-County Time : 12:19:29 Dispatch E-County (MONITORING) Time Can you update me on where all vehicles are now? (MONITORING) County Police Dispatch To: Operative Vi har 5950 och 5924 vid djurparken. 5830 E-County Time : 12:25:06 Commander E-County vid strandvägen. 5920 i norrköping. 5914 vid norsholm. vi har 3754 vid stavsjö och 3934 på väg genom krokek till djurparken. 5854 är vid djurparken (TRACKING) We have 5950 and 5924 by the zoo. 5830 by the beach road. 5920 at stavsjö and 3934 on their way through krokek to the zoo. 5854 are by the zoo. (TRACKING) Operative Commander To : County Police Tack för lägesrapporten, den stämde med E-County Time : 12:27:30 Dispatch E-County min karta, vi måste ta in mer resurser till FCP som kan hjälpa till med eftersöket av dagisbarnen ta fram 5920 (MONITORING)
Thanks for the situation report, it was in accordance with my map, we need more resources to FCP to help with the search for the pre-school class get 5920 (MONITORING)
In table 2 above, we can see how the county police on-site incident commander in Ecounty communicates with the county police dispatch officer in order to asses the situation and assure that everything is developing in accordance with the overall plan. The county police dispatch officer in E-county replies with detailed information, working on the tracking level. In this case we can see how two different levels of the ECOM interact, keeping the JCS up to date with the current situation. 5.1.3 Regulating Regulating activities are those that work toward short-term goals, and are partially a result from plans and objectives from the monitoring and targeting levels. The regulating level gives input to tracking either by specifying actions as tracking subloops, or goals and criteria for the tracking level. In emergency response, the roles of the County police on-site incident commander and the County police dispatch officer involve handling the position of components relative to each other (tracking sub-loops), coordinating resources, vehicles, personnel and their allocation. In the communication log-files, regulating activities are traceable by the situation assessments fed to personnel performing monitoring activities, and in the actions target values given to those that perform or give directives to those that act on the tracking level. Table 3. Example of regulating. Incident Commander To : County Police E-County Time : 12:05:49 Dispatch E-County
Ja gör det. 5854 kan söka i området med hund om de kan ta sig in från Nordost. Dvs från Orrbergen, men de skall vara försiktiga Yes do that. 5854 can search the area with a dog if they can enter from north-east. That is from Orrbergen, but they should be careful.
Table 4. Example of regulating. Incident Commander To : County Police E-county Time : 12:29:22 Dispatch E-County
Se till att pojken stannar där han är så kan du beordra en av poliserna i 5924 att gå dit Make sure the boy stays there so you can order one of the police in 5924 to go there.
Table 5. Example of regulating. County Police Dispatch To : County Police D-County Time : 12:14:25 Dispatch E-County County Police Dispatch To : Incident E-county Time : 12:14:28 Commander E-county Time
t 3754 och 7 3934 på väg. på plats om 20-30 min t 3754 and 7 3934 are on the way. There in 20-30 minutes. Södermanland meddelar att de skickar två bilar från nyköping. Var vill du ha dem? Södermanland report that they are sending tow vehicles from nyköping. Where do you want them?
In tables 3, 4 & 5 we can see how the different roles regulate each other’s activities. As we can see in the examples, information is propagated through the system. The regulating actions do not necessarily take the form of direct orders, but rather as directives to other actors whom in turn will transform them into action. 5.1.4 Tracking Activities at the level of tracking are steered by input in the form of goals/criteria from the regulating level, which prescribe actions, or targets and criteria for actions. The emergency response is actuated on this level, in actions performed by police patrol, units, ambulance operators and fire fighters in order to minimize damages and perform rescues such as evacuation or restricting access, extinguishing fires etc. In high-risk dynamic environments the activities necessarily require attention, yet can largely be described as closed-loop control. In the dialogs, tracking events are traceable by the feedback/measurements reported to the personnel that perform regulating activities. Table 6. Example of tracking. County Police Dispatch E-län Time : 12:27:58
To : Incident Pojken är vid parkeringen närmast ingången. Commander E- The boy is at the parking closest to the entrance County
Table 7. Example of tracking. Rtj 25xxx Time : 12:28:35
To : Rescue Vi ser inte vad som händer där uppe eftersom vi Services arbetar på den södra fronten. Bilarna är väl redan Norrköping city begagnade? We can’t see what’s happening up there since we are working on the south front. The cars are used, aren’t they?
As we can see in the excerpts in tables 6 & 7, the information exchanged in the messages has a very high level of detail. The messages concerns specific events within a very short time-horizon.
6
Discussion – analyzing large scale operations using the ECOM
There are a number of configurations of the JCS during the emergency response in the exercise. As functions are moved across roles, assessments, which have been made in a certain context by a participant with a specific competence, are moved and in certain cases transformed. This means that transient and implicit information can be lost in the process, just as in a “whisper game”. The source and context of information has importance to how it is interpreted. Initially, the County 112/911 emergency call center fills functions that are allocated to each organization. How they are moved to the Police and to the Fire & rescue service differs. Once the Fire & rescue service has received and confirmed the alert, and has forces on the scene, they are responsible for the full range of ECOM activities. The Police however, take full responsibility once they have received the alert. In each organization, targeting and monitoring is not allocated to a separate role until the incident complexity escalates. It is thus clear from the analysis
that the unit of analysis move from individuals to groups or even organizations as the scenario escalates. The usefulness of focusing on communication and using episodic analysis is that it is possible to identify both the functional levels of the ECOM as well as the transitions that take place when functions are moved. 6.1
Methodological insights
The ECOM is an approach to modeling a JCS with functions in focus. However, using models such as the ECOM has consequences when working towards design solutions; a model does not prescribe a final format. Instead, models are useful in ranging the design problem, in part by identifying variables that are relevant to focus on (Sutcliffe, 2003). Analyzing tasks and mapping out the social interactions in work is a first step towards understanding a design space (ibid.). In this sense, the ECOM has been valuable, as it has helped to identify interactions and functional levels. This application of the model must however be seen as explorative, as the model is abstract and rarely if ever applied in the same manner as in this text. Operating with the aim of understanding work, one goal is to extract patterns in performance and adaptation. However, this requires comparable and converging studies across the domain (Woods, 2003). In addition, a combination of methods and perspectives is desirable if research is to avoid fallacies arising from adjusting a study to support a hypothesis (ibid). Contrasting patterns identified in this text with other studies from other domains or from a different perspective would be a way of validating conclusions drawn here. 6.2
Implications for system design
The analysis has shown that it is possible to identify the ECOM levels in the collected data, but also that it is possible to identify transitions of functions between roles and organizations. An important issue is how such transitions can be supported? Transitions in responsibility and control always make a system vulnerable since it is both resource and time demanding to perform them. Disturbances during the transition may have large consequences since the function under transition probably suffers from degraded performance. To develop methods and aids for the support of transitions, as well as developing monitoring tools that can detect an upcoming transition is thus essential. In many cases, such preparations exist, but when concerned with inter-organizational work, preparations often lack. 6.3
Future research
It has been seen in the ALFA-05 exercise that different personnel from different organizations operate at the same level of activity, and the same role can perform different functions in different phases of a response. The system itself is not a stable entity, which lies waiting for an incident, although the organizational structures are latently present. In the analysis performed for this paper, focus has been on how to identify the different levels of the ECOM. However, it is also possible to look for breakdowns in the functional levels, something that has not been done in this material.
7
References
Aminoff, H. (Unpublished Manuscript): An Analysis of Coordination in Emergency Management from a Joint Cognitive Systems Perspective. Masters Thesis, Linköping University, Linköping. Artman, H. (1999): Fördelade kunskapsprocesser i ledningscentraler vid nödsituationer / Distributed Cognition in Control Rooms for Emergency Situations. Thesis no. 186. Dept. of Communication, Linköping University, Linköping. Brehmer, B. (1992): Dynamic Decision Making: Human Control of Complex Systems. Acta Psychologica, 81, 211-241. Downing, A. (2000): Talking Topically. Universidad Computense: Departamento de Filología Española III. Hollnagel, E. (1993): Models of cognition: Procedural prototypes and contextual control. Le Travail humain, 56(1), 27-51. Hollnagel, E. (1998): Context, Cognition and Control. In (Ed.) Y. Waern, Co-operation in process management – Cognition and information technology. Taylor & Francis, London. Hollnagel, E., Nåbo, A. and Lau, I.V. (2003): A systemic model of driver-in-control. In Proceedings of the second international driving symposium on human factors in driver assessment, training and vehicle control., Park City, Utah, 86-91. Hollnagel, E. and Woods, D.D. (2005): Joint Cognitive Systems. Foundations of Cognitive Systems Engineering. Taylor & Francis, Boca Raton, FL. Johansson, B. and Hollnagel, E. (2007): Pre-requisites for Large Scale Coordination. Cognition, Technology & Work, 9, 1, 5-13. Johansson, B. and Waern, Y. (2000): Understanding Communication in a Dynamic Situation - Argumentation in Situation assessment and planning. In Proceedings to COOP 2000, Sophia Antipolis, France. Johansson, B. (2005): Joint Control in Dynamic Situations. Linköping Studies in Science and Technology. Thesis No: 972, Linköping. Linell, P. (1998): Approaching Dialouge. John Benjamin Publishing, Amsterdam, Holland. Linell, P. and Korolija, N. (1997): Coherence in multi-party conversation. Linguistics 34, 44, 799-831. Neisser, U. (1976): Cognition and reality: Principles and implications of cognitive psychology. W. H Freeman, San Fransisco. Perrow, C. (1984) Normal Accidents: Living with High-Risk Technologies. Basic Books, New York. Sutcliffe, A.G. (2003): Mapping the design space for socio-cognitive task design. In (Ed.) E. Hollnagel, Handbook of cognitive task design. Lawrence Erlbaum Associates, Mahwah NJ, 549-575. Stroomer, S. and Van Oostendorp, H. (2003): Analyzing Communication in Team Tasks. In (Ed.) H. van Oostendorp, Cognition in a Digital World. Lawrence Erlbaum Associates, Mahwah NJ, 175-204. Trnka, J. (2007): Perquisites for Data Sharing in Emergency Management: Joint Analysis Using a Real-Time Role-Playing Exercise Approach. Linköping Studies in Science and Technology, Thesis no. 1293, Linköping University.
Trnka, J., Johansson, B. and Granlund, R. (2006): Information Support in Collaborative Command and Control Work – An Empirical Research Using a Role-Playing Exercise Approach. In Proceedings of The 11th ICCRTS – Int. Command and Control Research and Technology Symposium, Cambridge, UK. Trnka, J. and Jenvald, J. (2006): Role-Playing Exercise – A Real-Time Approach to Study Collaborative Command and Control. Int. J. Intelligent Control and Systems, 11, 4, 218-228. Turoff, M. (2002): Past and Future Emergency Response Information Systems. Communications of the ACM, 2002, 45, 4, 29-33. Weinberg, G. M. and Weinberg, D. (1988): General Principles of Systems Design. Dorset House Publishing, New York. Woltjer, R., Trnka, J., Lundberg, J. and Johansson, B. (2006): Role-Playing Exercises to Strengthen the Resilience of Command and Control Systems In (Eds.) G. Grote, H. Günter and A. Totter, Proc. of the 13th European Conference on Cognitive Ergonomics (ECCE) – Trust and Control in Complex Socio-Technical Systems, Zurich, Switzerland, 71-78. Woods, D.D. (2003): Discovering How Distributed Cognitive Systems Work. In (Ed.) E. Hollnagel, Handbook of Cognitive Task Design. Erlbaum, 37-54.
