Boundary, Purpose, and Values in Work-Domain Models - IEEE Xplore

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A : SYSTEMS AND HUMANS, VOL. 35, NO. 5, SEPTEMBER 2005

603

Boundary, Purpose, and Values in Work-Domain Models: Models of Naval Command and Control Catherine M. Burns, David J. Bryant, and Bruce A. Chalmers

Abstract—This paper presents an application of work-domain analysis (WDA) to the domain of the command and control of a multipurpose naval frigate—the Canadian Halifax Class frigate. This represents an application of this approach to a real system and, to the authors’ knowledge, is the most extensive WDA of a naval work domain. In particular, and in contrast to other applications of cognitive work analysis, the authors extended the basic WDA framework to handle a multipurpose, loosely bound work domain. In addition, the naval domain is value driven, and this affects naval decision making. Values were incorporated as a social organizational analysis into the work-domain model and were represented as a type of soft constraint. A total of 38 submodels of the work domain were developed, whose primary models are discussed in this paper. From these models, 132 information requirements were extracted, substantiating that WDA is a worthwhile technique for supporting interface design. This paper makes a theoretical contribution by extending the WDA framework and a practical contribution by demonstrating the usefulness of the framework in a real design context. This paper concentrates on presenting WDA as a process, not as a finished product, showing intermediate levels of models and the design requirements that can be extracted from the early stages of the WDA. Index Terms—Cognitive work analysis (CWA), ecological interface design (EID), naval command and control, work-domain models.

I. INTRODUCTION

N

AVAL command and control is a domain that has been evolving with the emergence of new technology [15]. From a relatively centralized control structure, new naval command and control systems are becoming increasingly decentralized and distributed [6]. Access to new information and advanced decision support systems are increasing the range of decisions that an individual ship commander can make and making local distributed control more necessary [11]. The nature of naval operations themselves has changed. Manuscript received September 7, 2001; revised September 6, 2004. This work was supported under contract by the Defence R&D Canada—Valcartier (formerly Defence Research Establishment Valcartier), with Dr. B. Chalmers as the Scientific Authority. This paper was recommended by Associate Editor N. H. Narayanan. C. M. Burns is with the Advanced Interface Design Laboratory, Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail: [email protected]). D. J. Bryant was with Humansystems, Inc., Guelph, ON N1H 3N4, Canada. He is now with the Command Effectiveness and Behaviour Section, Defence R&D Canada—Toronto, Toronto, ON M3M 3B9, Canada. B. A. Chalmers is with the Maritime Information and Combat Systems Section, Defence R&D Canada—Atlantic (formerly Defence Research Establishment Atlantic), Dartmouth, NS B2Y 3Z7, Canada. Digital Object Identifier 10.1109/TSMCA.2005.851153

Naval crews must participate in a variety of mission types, often facing less organized and less predictable operations other than war and working within multi-national task groups with different policies, skill sets, and equipment capabilities. Now, more than ever, decision support systems are needed that manage the increasing amount of information in a way that will create flexible, knowledge-based problem solvers from naval command teams. The Canadian Navy is no exception to this situation. If anything, Canada’s small-scale but technologically advanced fleet is frequently a member of multinational efforts. Its Halifax Class frigate was designed primarily for antisubmarine warfare but, in reality, must take a role in a variety of missions—from surveillance, to escort, to information gathering, threat intervention, and the delivery of humanitarian aid. The Halifax Class frigate is equipped with a wide variety of above-water sensors, extensive sonar equipment, and while not intended to be an offensive ship, enough weapon and decoy capacity to meet self-defense needs and to participate in a variety of missions. Its operations room (ops room) team works in a highly dynamic environment where multiple operators and controllers manage a large variety of incoming information. In the current ops room, this information filters and integrates upward through several levels of the team to the members of the command team, who manage the execution of the mission as a whole. In previous literature, the domain of the Canadian Navy’s Halifax Class ops room has been analyzed with cognitive task analysis [28]. In a U.S. environment, Hutchins [12] provides an example of the richness and complexity of the naval environment [12]. While these approaches have been successful in the past, given the dynamic information-dense nature of the modern military environment, we chose to apply cognitive work analysis (CWA), and in particular work-domain analysis (WDA), in order to see whether this approach could generate some new information requirements for operators. WDA forms the basis of an approach to decision support system design that has evolved from the domain of nuclear power plant control. WDA reveals domain constraints and relationships between variables and generates requirements not typically obtained from a task analysis [14]. WDA is complementary to these previous types of analysis and ideally should be used as part of a complete analytical approach. WDA has been applied before to model both naval command and control and naval vessels. Chin et al. looked at how workdomain models can be used at different levels of a generic naval command organization [8], from ships to higher level command

1083-4427/$20.00 © 2005 IEEE

604


TABLE I COMPARISON OF SYSTEM CHARACTERISTICS OF NUCLEAR POWER PLANT CONTROL AND NAVAL COMMAND AND CONTROL

structures. Petersen and Neilsen performed a WDA that was focused on the maneuvering of a naval vessel [19]. These particular studies are relatively general and do not examine a specific vessel or look at the entire naval work domain. Bisantz and colleagues performed a more detailed WDA at the ship level than Chin and a broader model than Petersen and Neilsen. Bisantz modeled a U.S. destroyer [2], and a high-level comparison of our models and the destroyer project has been published [1]. This paper, however, discusses scope and boundary issues in performing a WDA and the need for a three-part domain model for the naval work domain. The three-part model is a novel application of the WDA framework that has implications for eventual systems design. The author also demonstrates how values or “soft constraints” can be integrated in the work-domain analysis as another layer of analysis. Finally, the utility of the modeling work is demonstrated by giving some examples of information requirements that can be derived directly from the WDA models. II. METHOD RATIONALE Following Three Mile Island, WDA, and its implementation—ecological interface design (EID)—arose as approaches with promise for designing decision support systems that could support problem solving in unanticipated situations in the nuclear industry [22]. The approach grew out of field studies of expert troubleshooters. Studies of these troubleshooters provided insights into what kind of information support would be necessary to support human operators in other work environments so that they would act as experts in their own domain [20], [21]. Rasmussen [22] argued that, in order to handle unanticipated situations, operators needed to be able to reason effectively about the goals they wanted to achieve and how the resources and equipment they had available could be used to achieve those goals. He developed a structure for mapping out the purposes and physical constraints of a system called the abstraction hierarchy, which is the structure that results from a WDA. In this paper, we will use the term work-domain model as synonymous with abstraction hierarchy. Since then, WDA and EID have been shown experimentally to lead to improved problem diagnosis and more effective control strategies in a process control environment (cf. [18]). WDA is the first analysis

in the emerging package of analytical techniques called CWA [26]. A complete CWA begins with a WDA, followed by the analysis of control tasks, strategies, social organization, and operator competencies [26]. WDA and EID have been used to develop industrial research-level power plant displays [3], [29], and work-domain models have been explored for aviation [e.g., [17]], petrochemical process control [13], and biomedical applications [10] to name a few. These results should be relevant to other naval command and control platforms, and to command and control environments in general, and complementary to other analyses of command and control. Decision making in naval command and control has several similarities to the domain of nuclear power plant control (Table I). Like nuclear power plant control, many of the decisions made in naval situations are potentially critical and life threatening, to the ship itself, and to society. As well as in nuclear power, there are underlying physical constraints of the situation that the team must operate within to be effective. For example, weapons have limited ranges and the relations between range and the degree of accuracy that they can achieve. Ships have certain speeds that they can achieve and a fixed maximum maneuverability. In both nuclear power plant control and naval command and control, human operators are faced with the inevitability that they will eventually encounter unanticipated situations that will require flexible knowledge-based reasoning to manage effectively. For these reasons, we decided to see whether or not the WDA framework could be applied to naval command and control successfully and whether or not the use of this framework could result in design guidance that would be relevant to the designers of decision support systems [7]. There are several aspects of naval command and control that differ from nuclear power plant control that made the application of this framework challenging, requiring that the framework itself be developed further in order to meet the design challenges of this new application domain. Unlike a power plant that can be modeled with a very defined system boundary, a naval system has boundaries that are very difficult, if not impossible to define. A naval frigate can be expected to undertake a variety of missions, varying widely in their types, whereas a power plant ultimately has a very clear purpose—the generation of electrical energy. A naval frigate, being used for command and control,

BURNS et al.: NAVAL C2 WORK DOMAIN MODELS

is an information-gathering vessel, almost a sensor itself. This work makes a significant contribution to the existing work on EID and WDA by providing an extensive analysis of a loosely bounded system, and ultimately, creating a different form of work-domain model to handle this type of system. This extension of work-domain models will enlarge the applicability of the framework, providing guidance on how to use this approach in other similar loosely bounded domains such as ground transportation, aviation, or shipping. In addition, these work-domain models demonstrate an application of the work-domain framework to a problem of real-life scale, moving the approach from the laboratory into a usable informant to design. III. WORK-DOMAIN ANALYSIS A WDA models the physical and functional constraints of a system from the purposes of the system down to the physical equipment and form [22]. The goal of a WDA is to create a set of models that describe how the system works, and then use these models to inform interface design. The objective behind this approach is to provide the operator with a correct model of the system so that the operator, or team of operators, may develop the correct mental model and therefore control the system and handle problems with the system more effectively. A typical work-domain model consists of five levels of work-domain constraint [22]: functional purpose, abstract function, generalized function, physical function, and physical form. Functional purpose describes the purposes of the system, i.e., why the system was designed. Abstract function describes the principles by which the system operates, most commonly in terms of flows of mass or energy. Generalized function describes the processes used in the system. Physical function describes the capabilities of the equipment in the system, and physical form describes the physical features of the components in the system. The links between each of the five levels are “means-end” links [22] and indicate how the higher level element is achieved, or conversely, why the lower level element is available. Beyond the basics of WDA, there are several further decisions that the analyst must make. The analyst must clearly determine the system of analysis, outlining what is considered to be the work domain in the given situation. The analyst must determine the purposes for which the domain exists and whether or not the domain is also constrained by values or social organizational factors. Most work-domain analyses to date have concentrated on primarily physical or causal systems that are governed by the laws of nature. A system such as command and control, while certainly constrained by the physical laws affecting movement, weapon, and sensor usage, are also constrained by social values. Rasmussen, Pejtersen, and Goodstein [24] discuss these different kinds of systems in terms of the role that human intention plays in the system. No system is entirely causal or intentional, but rather influenced by these constraints to different degrees. A power plant, though largely governed by natural laws, has intentionality embedded in its objectives (functional purpose) and in its automated control systems [24]. In contrast, social laws (such as national objectives and international law) also govern a naval command and control system. We found it useful in con-

605

ducting this WDA to explicitly define a system boundary and to explicitly identify and differentiate causal and intentional constraints. These steps are not clearly identified in other recorded uses of the WDA approach. Therefore, the discussion of these stages makes a significant contribution to WDA methodology that should be of benefit to practitioners and researchers in this area. A. Work-Domain Boundary A WDA is a systems analysis, and therefore the unit of analysis will affect the type of model that is developed. Determining the unit of analysis requires defining a system boundary for the purposes of modeling. A system boundary is an artificial division and yet, for modeling purposes, it is needed to keep the model within a useful scope. In work-domain models to date, a power plant has typically been modeled as a tightly bounded system (cf. [3]); that is, all the components of the system are contained within the walls of the plant and are not intended to interact beyond the bounds of the plant. While it is untrue that a power plant is a completely bounded system, for modeling purposes it can be treated as such, and useful models of the work domain have been constructed in this way [3], [24]. In effect, the system boundary is established at the plant walls and at the connection with the electrical grid. However, this boundary is clearly artificial when the domain changes scale. In modeling the electrical system for a city, the power plant would become a component in this larger system, affected by the loads and energy sources of the city electrical system. In contrast, we considered command and control to be a loosely bounded system. By a loosely bounded system, we mean that components of the ship (e.g., mobile resources like helicopters and allied ships) enter and leave at various times, the ship interacts with various numbers and types of entities, and the ship has varying levels of control over its own and other external entities. In modeling command and control for a ship, it does not make sense to draw the system boundary at the hull of the ship. Command and control is about moving and controlling entities outside of the ship using the ship, its on-board or off-board organic resources, and other platforms and their associated resources also under its control. In addition, a ship may operate in single-ship or task group operations, depending on the mission and its role in that mission. For this reason, we chose to model command and control as a loosely bounded system. While the resources under command are available for control, a significant part of the command and control problem involves interacting with parties outside of the sphere of direct command. A work-domain model of this domain, therefore, must accommodate this open nature of the command and control problem. Work-domain models of loosely bounded systems have been developed before in the areas of nuclear emergency management [16], [23] and ambulance dispatch management [9]. Both of these systems are loosely bounded in the sense that, given the system boundary used for the model, the operator has control over a collection of resources but does not have direct control over the entire work domain. That is, incidents or accidents,

606


TABLE II WORK-DOMAIN MODEL OF AMBULANCE DISPATCH MANAGEMENT [9]

such as a radiation release or an injured person, can occur within the work domain. The operator has resources available to mitigate this situation but does not have control over the elements that caused the situation. This lack of complete control is an inevitable characteristic of loosely bounded systems. From a work-domain modeling perspective, however, the models must strive to capture this aspect of the work domain in order to represent the domain faithfully. These types of systems have been modeled using a two-domain model. As shown in the example of the ambulance dispatch in Table II, the top level of the model—functional purpose—is shared across the two domains [23]. However, in sharing the purpose at the top level, it is assumed that the ambulance dispatcher’s purpose of maximizing the emergency health care response corresponds with the injured person’s purpose under most circumstances. B. Shared or Different Purposes Existing models, such as the nuclear emergency management and ambulance dispatch models discussed previously, provided an influential beginning for our models of naval command and control. Certainly, as in these domains, naval teams must exert control over situations that they did not create. We needed to model naval command and control as including the frigate and other entities within the system boundary, where the specifics of these entities are not predetermined. In the two previous situations, the other entities (the nuclear emergency and the injured person) could be presumed, under most circumstances, to have shared purposes with the mitigator. In naval command and control, an external entity could be in conflict with the ship and have entirely different purposes. In previous nuclear emergency and ambulance dispatch management WDA models, it was assumed that external entities would not actively exert control over the mitigating system. In a naval conflict, on the other hand, a contact could quite reasonably attempt to control the ship by modifying its functional capacity (i.e., by attacking it with weapons). We needed to develop a model of command and control that revealed the purpose of the ship and contact in its most basic design purpose but also captured its naval purpose.

TABLE III CHARACTERISTICS OF SYSTEMS AND THEIR WORK-DOMAIN MODELS

Another characteristic of naval environments is that the crew must interact constantly with the weather and the natural environment. The natural environment can impose limitations on movement and constrain the use of sensors or weapons. As an example, sea temperature and salinity can greatly impact the range and accuracy of sonar readings. Any model of this domain that did not include these factors would be ignoring a critical aspect of naval operations. Unlike the ambulance dispatch system modeled in Table II, listing weather as an element would not model the role of the environment in this work domain adequately. To model the environment usefully, we modeled it within the system boundary as a third “entity.” To summarize, naval command and control is more than just a loosely bounded system, it is a system with multiple potentially conflicting entities and a very powerful natural environment. The proposed models for command and control can be considered an extension of the WDA framework to handle systems of this type. Like the system models performed before, we modeled more than one domain. We chose to model three domains: the frigate itself, a contact, and the natural environment. We selected these as the basic interacting units of this system. We did not specify that a contact be friendly, neutral, or hostile but kept the contact model generic. As a generic model, it would also allow the existence of multiple contacts, which is the most likely situation. We modeled the natural environment as an interacting entity on its own. Unlike previous WDA models, we did not model functional purpose as being shared across domains, since this would not represent the domain correctly (Table III). We modeled the frigate as having its purposes and the contact as having its own possibly friendly, neutral, or hostile purposes. We did not model a purpose level for the natural environment. In this way, we proposed three separate hierarchies, with separate purposes and separate domains of control. These hierarchies combine to complete the work domain and interact with each other at the level of physical form. C. Incorporating Values or Soft Constraints As well as managing the physical constraints of the work domain, operators in military domains work within the value structure of their organization. While values and priorities, sometimes called intentional constraints, have been included in workdomain models before ([9], [21], [24], and [25] are examples), the treatment of values has not been detailed. The understanding


Fig. 1.

Different regions of physical and social organizational constraint.

of the role of values in the work domain has not been explicit, and the role of values in relation to physical constraints has not been clearly described. Values and priorities are social organizational factors that constrain action; however, they are different from physical constraints. Whereas physical constraints cannot be violated and are as such, “hard constraints,” social organizational constraints can be broken but probably will not be or should not be. They are in this sense “soft constraints.” Operators can take action outside of their social organizational constraints, although it is most likely that such action will be socially unacceptable or subject to discipline. A “rule of engagement” (ROE) is an example of a social organizational constraint. A ROE consists of directions and orders regarding the use of force in a domestic or international operation in peace time, periods of tension, or in an armed conflict. ROEs are designed to remove any legal or semantic ambiguity that could lead a military commander to violate policy by inadvertently underreacting or overreacting to an action in a situation. In this way, we defined two different kinds of constraints— physical and social organizational constraints—which constrain the envelope of possible actions as shown in Fig. 1. We discovered that, in this domain, the influence of social organizational constraints on decision making varies with the level of aggregation of the system. At the national strategic levels, values and priorities are very influential in decision making. At the level of the individual frigate, values and priorities have been incorporated into the definition of mission or purpose. As such, frigate commanders cannot deviate from the socially acceptable region and still accomplish their mission. Values and priorities, therefore, were not a part of the work domain over which frigate commanders had very much control. Adherence to actions permitted by rules of engagement, however, was one area of control at the frigate level. To handle social organizational constraints, we developed our physical constraint hierarchy first. We then defined social organizational purposes and outlined social organizational constraints at the five levels of Rasmussen’s abstraction hierarchy. We integrated these with the physical work-domain constraints to create a work-domain model with physical and social organizational constraints, with the social organizational constraints clearly identified as soft constraints. As can be expected with a

607

constraint hierarchy, adding more constraint reduces the envelope of action, reflecting the smaller space shown in Fig. 1. Interestingly, the fourth stage of a CWA is a social organizational analysis [26]. There are very few such analyses whose results are available in the literature. We propose that the approach followed here, of defining social organizational constraints using an abstraction hierarchy, fully integrated with the physical constraints of the domain is a useful approach for contributing to this fourth stage of CWA. The abstraction hierarchy delimits and classifies the social organizational constraints, as well as provides a useful way of integrating these constraints with physical constraints. In the following section, we discuss the information-gathering activities that were used to inform the work-domain models and the general structure of the overall model of the work domain. IV. METHOD Over the course of one year, we performed several information-gathering exercises and performed several levels of modeling iterations. These are discussed in this section, with the view of illustrating the process that we followed. A. Information Gathering We reviewed naval tactical documents, training publications, and documents on the design of the frigate. We observed three training exercises in a full-scope simulator, went on a three-day observational sea study and interviewed command team officers on several occasions. The following table (Table IV) shows how these information-gathering activities were used to contribute to our work-domain models, by mapping out the information sources by work-domain model level. The sources for social organizational constraints are shown in the bottom row, shaded in gray. B. Model Iterations With a domain this complex, it was not possible to create a set of work-domain models in a single modeling exercise. We chose instead to go through several iterations, starting first with a skeletal framework and increasing in model complexity. At the skeletal level, the main objective was to isolate the major parts of the model and to determine a boundary for our system of analysis. The second iteration provided more details of the physical work domain. Our third iteration incorporated soft constraints into the model. We completed the models in further detail [4] but that degree of detail cannot be shown here. Table V shows the range of the modeling effort and the various detail levels. In the next section, our results will be presented in this progressive fashion, first discussing our skeletal level model, then the first level expansion, and then the addition of social organizational constraints to the models. It is important to note that the number of models generated, in this case 38, is a function of the complexity of the system, the objectives of the project, and the choices of the analyst. The number of models in this kind of exercise is not rigidly determined but instead reflects the varying amount of detail and presentation chosen by the analyst in each modeling exercise.

608


TABLE IV USE OF INFORMATION SOURCES IN CREATING THE WORK-DOMAIN MODELS

TABLE V RANGE OF THE MODELING EFFORT

V. RESULTS We began from a very basic “skeletal” model of the three parts of the work domain—the Halifax, the natural environment, and a contact (Fig. 2). These three parties interact with each other physically through the level of physical form, that is, they sense each other, beginning with their physical form, and they can physically contact each other at this level. These three elements are all at a single level of decomposition, the frigate at a decomposition of “ship,” the environment considered as a whole, and the contact considered to be a whole ship, whole plane, or possibly whole missile as examples. The basics of the work-domain model are here—basically, to understand the work domain, we must understand the purposes of the Halifax ship, the contact’s purposes,

Fig. 2.

Skeletal framework of the work domain.

and how they both use their capabilities, in the context of the environment, to achieve those purposes. From this skeletal model, we expanded each of the three major units, ultimately into 38


Fig. 3.

First-level expansion of the Halifax work domain.

submodels. Table V shows the full range of models, with several levels of expansion of the Halifax. Only the most significant of these expansions will be shown here, and all of these expansions are at the levels of decomposition discussed previously. A. Expansion of the Halifax The main portion of our modeling effort focused on models of the Halifax Class frigate, its purposes, and capabilities. Fig. 3 shows the entire Halifax side of the model for the physical and causal constraints. Each level is discussed in the subsequent sections. 1) Functional Purpose: The Halifax Class ship can take on many different kinds of missions. Our objective in modeling this level was to abstract from these different mission types those elements that were common, and thereby understand, more globally, the purpose of the frigate. We modeled the frigate as having four main purposes: 1) 2) 3) 4)

609

maintaining its own survival; moving from one location to the next; exerting sea control; maximizing information gathering.

We determined these purposes by working from a list of frigate missions and examining whether or not the purposes were involved to some extent in each of the different missions (Table VI). For example, in an escorting mission, the ship must move with

the protected unit and control a sector, removing or diverting possible threats along this path. In a very different mission type, a search and rescue, similar purposes exist. The ship must travel in its sector but in this case gather information on the search target, as opposed to a threat. We defined sea control as purposes outside of relocation, for the purpose of controlling a designated area. Sea control captures the need to gain information on the control area and, if needed, to exert influence over other parties in that area. Note that, as portrayed in this figure, links between the Halifax, the environment, and the contact cannot be shown, although they exist in the complete models. Sea control is an example of a model element with many connections to the contact side of the model, since sea control is focused on understanding the capabilities and intention of the contact, and if needed, modifying those capabilities and intentions. The contact model, presented later, determines the information that would need to be gathered by an ops room team, in order to have a complete understanding of the work domain. Information gathering, therefore, is represented in two ways. The first representation is through the contact model that shows the information from the world that must be gathered. The second representation is through the information gathering constraints on the Halifax model. These constraints show the limits on the information that can be sensed and processed by the ship. 2) Abstract Function: At the abstract function level, we modeled three main principles: the flow and balance of energy, the flow and balance of mass, and the flow and balance

610


TABLE VI MAPPING OF FUNCTIONAL PURPOSE ON TO TYPICAL HALIFAX CLASS FRIGATE MISSIONS

of information. Maintaining energy is important to maintain movement, and balancing mass reflects movement, flotation, and resource expenditures. We also included the flow and balance of resources in particular as a specific type of mass flow. To understand how this level acts as constraints on the actions of the frigate, the frigate cannot expend more energy than it has, cannot use more resources than it has, and must manage its mass balance to maintain buoyancy. 3) Generalized Function: The generalized function level describes the physical processes of the frigate. We modeled six primary processes: the process of maintaining “watertightness” or allowing or restricting water flow to maintain the mass balance needed for buoyancy, the processes of moving, launching resources, generating signals, and receiving and transforming signals. From these basic processes, we expanded each process group in more detail. 4) Physical Function: The physical function level describes the capabilities of the equipment and personnel. We included personnel at this level since the WDA is being performed from the perspective of a ship commander. From this view, personnel are also components of the ship, with various capabilities. 5) Physical Form: The physical form level describes the physical characteristics of the equipment. Physical characteristics include the location, size, shape, color, and identifying features of the equipment. At this level, we also included the composition materials of the equipment and the hull of the ship, the volume of the ship, and the condition of the equipment and the personnel.

B. Expansion of the Environment In our model of the environment, we did not model the level of functional purpose, but the four lower levels of abstract function, generalized function, physical function and physical form (Fig. 4). 1) Abstract Function: At abstract function, we modeled the conservation and flow of energy and mass, similar to other workdomain models. We also included the creation of entropy, since most natural processes dissipate energy and tend toward disorder. 2) Generalized Function: We modeled major environmental processes and distributed them into classes based on the medium involved (air, water, land, or electromagnetic radiation) and the time dynamics of the process (slow or fast). This resulted in the categories of processes shown in Fig. 4. Acoustic propagation and decay would be examples of “water processes.” 3) Physical Function: Not surprisingly, our components divided into the same four categories of air, water, land, and electromagnetic radiation. At this level, however, we are discussing the capabilities of these media in terms of dimensions relevant to naval operations, i.e., the capability to interfere with movement, to transmit, scatter, or interfere with electromagnetic radiation and the capability to transmit sound. 4) Physical Form: The physical form level contains many elements that navigators or radar or sonar operators would currently collect. At the physical form level, we included water temperature, salinity, the location of clouds and cloud type, and


611

Fig. 4. First-level expansion of the environment.

the location of land in terms of shore and in terms of waterway depth. These are just examples of potentially useful information at the level of physical form, and there are many others. C. Expansion of the Contact The contact model is very similar in structure to the Halifax model (Fig. 5). Our objective in creating the contact model was not to model a specific contact but to leave a framework where future different contact types could be modeled. For those reasons, the model is quite generic, and new categories may be added or removed, depending on the specific contact being modeled. 1) Functional Purpose: Like the Halifax, the contact also has intentions or a mission to accomplish and, we assumed, had the goal of maintaining its survival until that mission was accomplished. If the contact were friendly, that purpose would be complementary to the purpose of the Halifax and, if not friendly, in conflict with the Halifax. We did not explicitly model a movement purpose in case our contact was stationary but, as in the Halifax model, that could be added. 2) Abstract Function: We modeled the abstract function level as having the same components of the Halifax, primarily some form of mass and energy flow, and potentially a resource flow. 3) Generalized Function: As processes, we modeled a movement process in the case that the contact was able to change position, although this could be removed for a stationary contact. We modeled a process of launching resources, generating signals, and maintaining some kind of physical

integrity. We chose physical integrity over watertightness to allow models of air or land-based contacts. 4) Physical Function: The physical function level is an example of one place where the model would need to be tailored specifically to the contact being modeled. For now, we modeled our contact as having weapons, decoys, signal generators, and engine capabilities. These capabilities would be added or deleted, depending on the specific contact. 5) Physical Form: At the level of physical form, we left placeholders for physical characteristics such as size, shape, color, material, condition, and location. D. Social Organizational Constraints As discussed previously, we chose to model value-based properties of the work domain as social organizational constraints. We chose to model them using the abstraction hierarchy structure and five levels of description. However, we differentiated between these constraints and physical constraints, in order to show the boundaries drawn in Fig. 1. In our view, this was an important distinction to make when modeling to support decision making. We integrated these constraints with the physical constraints to show a more complete model of the work domain (Fig. 6). In Fig. 6, only the links related to the added social organizational constraints have been shown. As in the other sections, these constraints are discussed by level of abstraction. 1) Functional Purpose: To include value-based constraints, we needed to include some kind of value purpose to the work domain. We described this generically as meeting naval values

612


Fig. 5. First-level expansion of the contact.

yet used it as a placeholder for several different goals. We considered value-based purposes to include reaching a certain effectiveness level, operating to reach certain economic goals, attaining a certain balance of force in operations and a certain degree of adherence to international law. All of these factors act as soft constraints on the work domain. It is better for the decision maker to work within them, but it is not physically infeasible for a decision maker to decide to take action that is overly costly, excessive in force deployed, counter to international law, or ineffective. 2) Abstract Function: We modeled the work domain as having four major balances. Naval command teams have to balance authority between the individual ship and the rest of the task group or Navy. They must also balance the degree of force that they apply in a given situation. Economic expenditure flows within the ship, the task group, and the Navy, tied closely but not completely to resource use. Finally, naval effectiveness is determined by how the team balances success and failure and how they determine what is an acceptable level of risk. 3) Generalized Function: Generalized function is a description of process. We modeled confirming and operating within legal limits as a process and also the concentration of force, ei-

ther individual weapons or several ships. The higher level balances also connect to many of the physical processes. 4) Physical Function: We modeled or ROEs as instantiations of social organizational constraints that provided capabilities. ROEs are rules that constrain action given the context of the situation—determining when a ship may fire, use certain kinds of radar, etc. 5) Physical Form: At the physical form level, we included the cost of equipment and its use. This was the only social organizational element that we modeled at this level. By argument, social organizational constraints are a generally higher level and therefore influence the upper regions of the abstraction hierarchy greater than the lower regions. VI. DESIGN IMPLICATIONS WDA is a framework to generate information requirements for design, both in terms of data that need to be collected and in terms of information that needs to be derived through processing of this data. For example, the WDA for the DURESS system [27] resulted in the addition of new kinds of information to a display. Studies showed that this WDA guided display re-


613

Fig. 6. Work-domain model of the Halifax with social organizational “soft” constraints.

Fig. 7.

Between-domain requirements.

sulted in improved fault diagnosis performance [27] and control strategies [18]. The WDA approach is highly analytical, however, and a “complete” WDA for a system as complex as a frigate could take a very long time. It is important, therefore, to prove that benefits can be extracted from the process at intermediate stages, with incremental benefit being gained as the project progresses. To show this, we derived information requirements from the models presented previously. These information requirements will be discussed in two sections: “between domain” requirements and “within domain” requirements.

A. Between-Domain Requirements The advantage of the first-level “skeletal” model (Fig. 2) is that it shows relations between the three parts of the work domain. From these, we can extract between-domain requirements. These requirements form the basis of interdomain coordination and allow the crew of the Halifax Class frigate to understand their capabilities in the larger context of the entire work domain (Fig. 7). These requirements can be discussed in terms of domain pairs. 1) Halifax—Contact Requirements: The ops room team must understand the contact’s intent, constraints, and capabilities. If those intentions are compatible, then each group must

614


TABLE VII INFORMATION REQUIREMENTS DERIVED FROM THE ENVIRONMENT MODEL

TABLE IX INFORMATION REQUIREMENTS DERIVED FROM THE FRIGATE MODEL

TABLE VIII INFORMATION REQUIREMENTS DERIVED FROM THE CONTACT MODEL

C. Generating New Information From the Work Domain

coordinate and collaborate with each other. If not, the groups may engage in a conflict. For a conflicting contact, the team may need to deliberately search or use intelligence sources to understand the intention and capabilities of the contact. The team must also remain aware of how they are communicating their own intents, processes, and capabilities. To aid collaboration, they may want to communicate their information clearly; to thwart hostilities, they may want to disguise or to limit the enemy’s access to information from the ship (e.g., deception, distraction, and emission control). 2) Halifax—Ops Room: The team must understand the capabilities of the environment and, more specifically, how the environment can aid or hinder their actions. Similarly, they will want to understand how the environment will aid or hinder the contact, positively or negatively. This information, at the highest level, is most likely provided by weather forecasters and weather modeling. The lowest levels will be directly observable by the team or could be sensed with the ship’s sensors. In general, environmental data to support this type of analysis can come from historical databases, synoptic updates, or in situ measurements. B. Within-Domain Requirements Within-domain requirements can be extracted in more detail from the models presented in Figs. 3–5. Three tables (Tables VII–IX) give examples of some of the information requirements, which can be derived from the models. The information shown in italics is largely available right now from frigate sensor systems, though rarely brought together in a single display. The other information, however, is not provided and could be obtained from models or data fusion approaches. A complete ecological interface in this domain would try to show as much of all these information requirements as possible. In particular, information from the contact domain is critical to understanding the work domain properly.

Many of the information requirements indicated in Tables VII–IX are not available in the current C2 system. In fact, a rough count, just using the tables in this paper, suggests that at least 54% of the requirements here are new (this count is a rough guide only, since some of the requirements refer to multiple pieces of equipment and are only counted once here). This is consistent with the findings of Miller and Vicente [14]. The analysis identifies worthwhile information to collect or to calculate algorithmically by data fusion algorithms. An obvious choice would be contact intent, but a more novel choice might be contact energy reserves. For example, contact energy reserves could reveal the potential mission and potential actions of the contact. This work begins to identify these elements, although more work is required to develop these fully. VII. REVIEW OF THE MODEL We examined the content of our model in two ways. First, we used a scenario from a prior CTA [28] in a scenario mapping exercise. We had operations room officers step through the scenario and generate work-domain elements at each stage. Operations room officers are part of the command team of a Halifax Class frigate. They are largely responsible for handling tactical operations under the command of the commanding officer who has ultimate charge of the ship. These work-domain elements were compared with the original WDA model. The comparison revealed a small number of missing model elements that were later added to the analysis. These results have been published elsewhere [5]. We also had operations room officers review a list of work-domain elements and verify whether or not these elements were part of the work domain. Operations room officers confirmed the appropriateness of the various model elements. This particular review, however, was not as useful in soliciting missing model elements as was the scenario mapping exercise. VIII. CONCLUSION Work-domain analysis (WDA) and cognitive work analysis (CWA) have been widely anticipated to be very effective approaches for improving the design of interfaces for complex sys-


tems. In the literature, however, there are few well-worked examples of their application to large-scale real domains. We have applied WDA to the command and control of a frigate in the Canadian Navy to demonstrate the use of the approach, the development of the models, and how the models can be used to extract design implications. As a key aspect of this, we have shown how design implications can be extracted midprocess with some success. This extraction is critical to establishing the value of this kind of analytic process for real-world design. In working with a system of this scale and nature, we had to apply WDA in some novel ways to usefully model this domain. The scale of the domain meant that we had to carefully define the system of interest for our analysis. We chose to describe an interacting unit of analysis, consisting of the frigate, the environment, and a contact as a basic unit. This type of unitization of the work domain has not been applied previously. We were also modeling a domain that is highly affected by the values of the organization. While these values do not physically constrain decision making, ship command team members were well aware of the importance of taking action within these boundaries. We modeled these as soft constraints, adding an interior constraint boundary within our model. With a technique as relatively young as WDA, it is predictable that new attempts to apply the approach will result in the growth and development of the approach itself. Further examples of this approach’s application to other real-world domains, and models of those domains’ characteristics, promise an exciting direction for the growth of this approach. ACKNOWLEDGMENT The authors would like to thank the members of the Canadian Navy who assisted in this work by providing generous access to their simulation and training facilities, both on shore and at sea, and for answering questions on the work domain of the Halifax Class frigate. The authors would also like to thank the reviewers of this paper for their thoughtful comments, which have greatly improved the paper. REFERENCES [1] A. M. Bisantz, C. M. Burns, and E. M. Roth, “Validating methods in cognitive engineering: A comparison of two work domain models,” in Proc. 46th Annu. Meeting Human Factors Ergonomics, Baltimore, MD, Sep. 28–Oct. 4, 2002, pp. 521–525. [2] A. M. Bisantz, E. M. Roth, B. Brickman, L. Lin, L. Hettinger, and J. McKinney, “Integrating cognitive analyzes in a large scale system design process,” Int. J. Hum. Comput. Studies, vol. 58, pp. 177–206, 2003. [3] C. M. Burns, “Putting it all together: Improving display integration,” Hum. Factors, vol. 42, pp. 226–241, 2000. [4] C. M. Burns and D. J. Bryant. (2000) Feasibility study of cognitive work analysis for command and control work environment of a Halifax Class ship: Work domain models. Humansystems, Inc., Defence Research Establishment Valcartier, Valcartier, QC, Canada. [Online]. Available: http://candid.crad.dnd.ca [5] C. M. Burns, D. J. Bryant, and B. A. Chalmers, “Scenario mapping with work domain analysis,” in Proc. 45th Annu. Meeting Human Factors Ergonomics Soc., Minneapolis, MN, Oct. 8–12, 2001, pp. 424–428. [6] A. K. Cebrowski and J. J. Garstka. “Network-centric warfare: Its origin and future” [Online]. Available: http://www.usni.org/Proceedings/Articles98/PROcebrowski.htm [7] B. A. Chalmers, “On the design of a decision support system for data fusion and resource management in a modern frigate,” in Proc. Sensor Data Fusion Integration HUMAN, NATO System Concepts Integration Panel Symp., 1998, pp. 2-1–2-13.

615

[8] M. Chin, P. Sanderson, and M. Watson, “Cognitive work analysis of the command and control work domain,” in Proc. 1999 Command Control Research Technology Symp., vol. 1, Newport, RI, 1999, pp. 233–248. [9] J. R. Hajdukiewicz, C. M. Burns, K. J. Vicente, and W. Eggleston, “Work domain analysis for intentional systems,” in Proc. Human Factors Ergonomics Soc. 43rd Annu. Meeting, 1999, pp. 333–337. [10] J. Hajdukiewicz, D. J. Doyle, P. Milgram, K. J. Vicente, and C. M. Burns, “A work domain analysis of patient monitoring in the operating room,” in Proc. Human Factors Ergonomics Soc. 42nd Annu. Meeting, 1998, pp. 1034–1042. [11] J. R. Hajdukiewicz, K. J. Vicente, and R. Eggleston, “Development of an analytical framework and measurement tools to assess adaptive performance of individuals and teams in military work domains,” Univ. of Toronto, Toronto, ON, Canada, WPAFB Final Rep. CEL 99–01, 1999. [12] S. G. Hutchins, “Decision-making errors demonstrated by experienced naval officers in a littoral environment,” in Naturalistic Decision Making, C. E. Zsambok and G. Klein, Eds. Mahwah, NJ: Lawrence Erlbaum Assoc., 1997, pp. 207–215. [13] G. A. Jamieson and K. J. Vicente, “Modeling techniques to support abnormal situation management in the petrochemical processing industry,” in Proc. Canadian Soc. Mechanical Engineering Symp. Industrial Engineering Management, vol. 3, 1998, pp. 249–256. [14] C. Miller and K. J. Vicente, “Comparison of display requirements generated via hierarchical task and abstraction–decomposition space analysis techniques,” Int. J. Cognitive Ergonomics, vol. 5, pp. 335–356, 2001. [15] New World Vistas: Air and Space Power for the 21st Century—Information Applications Volume, USAF Advisory Board, Washington, DC, 1995. [16] N. Moray, P. M. Sanderson, and K. J. Vicente, “Cognitive task analysis of a complex work domain: A case study,” Reliab. Eng. Syst. Saf., vol. 36, pp. 207–216, 1992. [17] N. Naikar, P. M. Sanderson, and G. Lintern, “Work domain analysis for identification of training needs and training-system design,” in Proc. Human Factors Ergonomics Soc. 43rd Annu. Meeting, 1999, pp. 1128–1132. [18] W. S. Pawlak and K. J. Vicente, “Inducing effective operator control through ecological interface design,” Int. J. Human–Comput. Studies, vol. 44, pp. 653–688, 1996. [19] J. Petersen and M. Neilsen, “Analyzing maritime work domains,” presented at the Conf. Cognitive Science Approaches Process Control, Universität der Bundeswehr München, Neubiberg, Germany, Sep. 24–26, 2001. [20] J. Rasmussen, “Outlines of a hybrid model of the process plant operator,” in Monitoring Behavior and Supervisory Control, G. T. J. Sheridan, Ed. New York: Plenum, 1976, pp. 371–383. , “Skills, rules, knowledge; signals, signs, and symbols, and other [21] distinctions in human performance models,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, no. 3, pp. 257–266, 1983. [22] , “The role of hierarchical knowledge representation in decision making and system management,” IEEE Trans. Syst., Man Cybern., vol. SMC-15, no. 2, pp. 234–243, 1985. [23] J. Rasmussen, O. M. Pedersen, and C. D. Grønberg, “Evaluation of the use of advanced information technology (expert systems) for data base system development and emergency management in non-nuclear industries,” Risø National Laboratory, Roskilde, Denmark, Rev. Rep. JRC-ISPRA RISØ-M-2639, 1985. [24] J. Rasmussen, A. M. Pejtersen, and L. P. Goodstein, Cognitive Systems Engineering. New York: Wiley, 1994. [25] D. V. C. Reising, “The abstraction hierarchy and its extension beyond process control,” in Proc. Int. Ergonomics Assn./Human Factors Ergonomics Soc. (IEA 2000/HFES 2000) Congr., 2000, pp. 194–198. [26] K. J. Vicente, Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. Mahwah, NJ: Lawrence Erlbaum Assoc., 1999. [27] K. J. Vicente and J. Rasmussen, “The ecology of human–machine systems II: Mediating “direct perception” in complex work domains,” Ecol. Psych., vol. 2, pp. 207–249, 1990. [28] M. L. Matthews, R. D. G. Webb, and D. J. Bryant, “Cognitive Task Analysis of the Halifax Class Operations Room Officer,” Humansystems, Inc., Defence and Civil Institute of Environmental Medicine, Downsview, ON, Canada, 1999. [29] Y. Yamaguchi and F. Tanabe, “Creation of interface system for nuclear reactor operation—Practical implication of implementing EID concept on large complex system,” in Proc. 14th Triennial Congr. Int. Ergonomics Assn./44th Annu. Meeting Human Factors Ergonomics Soc., vol. 3, 2000, pp. 571–574.

616


Catherine M. Burns is an Associate Professor in Systems Design Engineering at the University of Waterloo, Waterloo, ON, Canada, where she directs the Advanced Interface Design Laboratory. Her research examines user interface design, visualization, and cognitive work analysis. Her work has been applied to military systems, healthcare, power plant control, and oil and gas refining. She is the author of more than 98 publications and recently coauthored a book titled Ecological Interface Design (Boca Raton, FL: CRC, 2004).

David J. Bryant received the Ph.D. degree in psychology from Stanford University, Stanford, CA, in 1991. He has conducted independent research on human spatial cognition, human factors of aviation security, and, over the past five years, decision making as it is related to military command and control. He is currently a Defense Scientist with Defence R&D Canada—Toronto, Toronto, ON, Canada, where he is pursuing research on operational planning, inferential processes involved in situation assessment, and tactical picture compilation.

Bruce A. Chalmers received the B.Sc. (first-class hons.) degree in mathematics from the University of Manchester, Manchester, U.K., in 1972, the M.A. degree in mathematics from the University of Western Ontario, London, ON, Canada, in 1974, the Ph.D. degree in mathematics from the University of Toronto, ON, Canada, in 1984, and the M.Sc. degree in computing and information science from Queen’s University, Kingston, ON, Canada, in 1989. After an undergraduate teaching career of several years at Mount Allison University, Sackville, NB, Canada, in 1990 he joined the Defence Research Establishment Valcartier (DREV), Valcartier, QC, Canada, where he worked on the design and analysis of parallel algorithms, system architectures, and high-performance parallel architecture simulations for naval resource management systems, and the integration of these systems with automated data fusion. He also played a key role in defining research and development activities to develop concepts for improved decision support for command teams on a Canadian Navy frigate. He led an investigation of the Cognitive Work Analysis framework as part of those activities. In 2000, he was transferred to Defence R&D Canada—Atlantic (formerly Defence Research Establishment Atlantic), Dartmouth, NS, Canada, where he continues to investigate from a human-centered perspective Cognitive Systems Engineering frameworks and methodologies for developing computer-based tools to support shipboard operators. A particular research focus is the development of representational aids for tactical picture compilation and response management in both single-platform and naval task group settings. He also has been an Adjunct Professor in computer science at the Université Laval, QC, Canada. He has published more than 25 papers in conference proceedings and academic journals. Dr. Chalmers is a Member of the Human Factors and Ergonomics Society.