analysis. The objective of the study conducted in support of the US Army's Battle Command Battle. Lab, was to determine the information elements required to ...
Developing Models of Decision Making Walter L Perry, RAND Exchange Scientist and James Moffat, Senior Consultant High Level Studies Department Centre For Defence Analysis Ministry of Defence Broad&, Parvis Road West Byfleet, Surrey KT14 6LY United Kingdom
courses of action, [16]; and (3) reducing the Abstract Studies that focus on the decision making process can be classified in (at least) two ways: by psychological content and by contextual content. Thefirst describes the degree to which the analysis focuses on the mental processes associated with decision making and the second deals with the degree to which the analysis is dependent upon the study’s context. Seveml studies are examined in terms of where they fall in this two-dimensional taxonomy. Two analyses of maritime command and control conducted by the authors are examined more fully within this taronomy. Both are context full, but are at opposite ends of the psychological content spectrum. These categorisations are useful in focusing future research aimed at developing models of decision making.1
Introduction The dif€iculty with modelling the command and control process for any human activity system, is understanding how decision makers decide. The tools needed to model decision support systems consisting of communications, information processing, operational procedures and operational relationships are fairly well known. The ultimate aim of decision support systems is to provide the decision maker with high quality and timely information that he can use to make his decision. What is missing, is an understanding of how he uses the information: how he decides.
Several studies have been conducted aimed at some combination of three analytic methods designed to either understand the process or to finesse it. They are: (1) understanding the psychological component of decision making, [4] and [9]; (2) representing decisions as empirical frequency distributions over a set of alternative 0 British Crown Copyright 199S/MOD. Published with the permission of the Controller of Her Britannic Majesty’s Stationery Offce.
decision making process to production rules2. In this report, we focus on the first two. We develop a taxonomy of decision making analyses and use it to classtfy several existmg studies. In particular, we dwell on two aspects of ow own work at CDA with maritime command and control. Our particular focus is the usefulness of the results of these analyses to modelling the decision making process in combat models. The question is: is it possible to use these results alone or in combination to develop acceptable models of decision making?3
Taxonomy of Decision Making Analyses It is possible to define (at least) two attributes which serve to discriminate among Werent types of analysis of the decision-making process. The first is the psychological content of the analysis. This is a measure of the extent to which the analysis focuses on understanding the mental processes involved in decision making (internal - external). Examples include Klein’s work on Recognition-Primed Decisions, [9] and [IO], the work of Eden and others [4] on Cognitive Mapping, and our work on Influence Connections [16]. At the other extreme are analyses that focus entirely on the ansequences or impacts of decisions in the real world. These analyses generally disregard the psychological aspects of how decisions are made and examine only the decisions themselves. Examples include the US Army’s Critical Information Requirements study (Ingram [7]) and our work on Decision Frequency Distributions [161. Two combat models cwently in use in the US that employ production rules to model command and control are the Vector in Commander (VIC) model developed by the US Army’s Training and Doctrine Analysis Command (TRAC) and the Joint Integrated Contingency Model (J’ICM) developed by RAND, Santa Monica California. VIC is a theatre level, land-air campaign model and the JICM models global conflict. For a lucid discussion on the applicability of models, see Hodges excellent paper, Six (or So) Things You Can Do with a Bad Model [ 6 ] .
somc Recent Stlldia
The second attribute we refer to as the contextual content. It mcasu~esthe extent to which the decision-maker, in the analysis. operates within a context similar to that in which he makes decisions ('context-full') or operates within a situation which bears M particular relation to his operational situation ('contextfree'). An example of a context-full analysis would be the use of an aircraft simulator and an example of a context-free analysis would be a simple one-on-one inmview away from the wo*k. The work of Men [4] and Ingram [7l are examples of studies that are relatively context free. The situations presented to the shdy Nbjeas were more vignettes than full contextual scenarios. In contrast,our own work on maritime "mand and conlrol, [IZ], [15], and [161. and that of Feher [SI are examples of full contextual d e s . In both cases, w t s were e m to make decisions in the context of fully developed scenarios.
The following is a brief catalogue of -t decision making studies in which we discuss methodologies and O U ~ C O ~inC tams ~ of their psychobgical and comcxtuel Contem. our work on ~commandaadcontrolissu"arised h u e a n d i n m o r e d c t a i l l a t a . Theworkof Bowcn. [U,wen 141 and Ingram 17l arc also summarised to present a contrast. Finally. semal studieswehaveexvruncd . are plotted in a "content-space" diagram in Fijpt. 2. wc have carried The majorityof the respa to "t command and
out with
control 1161 can be -c ' asmntext-~. The adysis of In&lmce dons is more psychologically iatcraal and the Dccision Frequency Dis&ibution wodr is purely exterd. In both cases. we pmvided tbe participaots an environment within which we emulated, in certain ways, the e " c n t within which a high level naval c o " d c T might find himalfin time ofwar. In tbeadyxkof i d u u m , we were concerned with how the information they received, mupled with their experience and judgement, lead to the decisions they todr In developing frequency distributions we focused on the information prwidca to the commander, and the mnsequercs of his decisions, rather than considering the mental model he c o " i of the situation.
Combining the two attributes in a twodimensional view and allowing for a continuum on each axis, we are able to classify analysea of decision making as depicted in Figure I. Those analysts fauing in the darker region on the diagram are relatively contexr-fie and internal d e s . Thcsc focus more on an academic understanding of how decisions are made. Although their direct U& in models of decision making is Iimiw they contribute to producing more general analytic tools. Analyses falling in the lighter region are more contexr-fill and external. Here the emphasis is more on d t s within a specific context than on the general applicability of the methodology used. In some instances, the tools developed in those studies in which understanding the decision making process are applied to spenfic problems (cfEden 141).
Bowen's [I] recent work in the area of cooperative decision making call be classified as context-full and internal. The obJeaive of his work with a private school was to help management develop a stratcgy for making building and space a l l d o n decisions. It was conten-fdl in the sense that the qu&ions put to the participants dealt with the issues associated with the decision to be taken,i.e., fund-raising, architectural considerations, pmperty Comminee workings, etc. It was internal in that the participants were asked to focus on the issues that should govem the decision making rather than actually offer solutions. The result of the study was a consem decision making framwork expressed as a shared mental model: not an actual space allocation decision.
t
c r m h-
CONTEXT
The work of Eden [4] and others in developing the concept of cognitive mapping, is another example of context-fdl and psychologdly internal research In this case however, a general me.thodology has emerged that facilitates the development of spxiiic cognitive models. As with Bowen [I]. the output of this
C-ly
Fig. 1 -Two-Dimensional Depiction of Decision Analysis Taxonomy
2
type of analysis is a model, albeit more structured, that is designed to guide the decision maker through the decision making process. The decision maker is confronted with his own mental model which hopefully will provide him with important insights. Some first principles also emerge from this work that can be used in other contexts: cognitive models must capture complexity; peculiarity and idiosyncrasy, generalities; particularities; and they must be ~omprehensible.~
Fig. 2 --Classifying Recent Studies Figure 2 represents a notional placement of the studies summarised above and others in twodimensional content space. The numbers in the ' [I" are the study reference numbers. Although this is not a complete list by any means, it does illustrate the not-so-surprising fact that most studies are context-full whether external or intemal. A question arises from all of this concerning the decision maker's mental model: does it vary with context or is there an adjustable, underlying model? We explore this somewhat in our discussion of influence factors. In essence, we point to experience, training and judgement as the foundation of an underlying model.
The work reported by Ingram [7] is an example of a relatively context-free external analysis. The objective of the study conducted in support of the US Army's Battle Command Battle Lab, was to determine the information elements required to support operational commanders. The methodology consisted of gathering data from several sources: 0
0
0
Maritime Decision Making Work recently completed at CDA W g h Level S t ~ d i e s )will ~ serve as more detailed examples of context-full external and context-fill internal types of analysis (Pew et al. [ 161).
Questionnaires: These were administered to students at the US Army's Command and General Staff College. They were entirely context-free. Students were simply asked to rank order a set of information elements.
A study was recently conducted by the authors that aimed at uncovering (1) what information has value to naval commanders when taking decisions during combat operations, and (2) whether different commanders take the Same decision when confronted with the Same information. Maritime combat scenarios were used to provide context for the purpose of eliciting the expert opinions of the study participants. The scenarios were similar in that they all required that the participants assume command of a naval task force formed to deliver an amphibious force to a landing site.
Observation of Exercises: These ranged from rather context-full simulations of combat to seminars in which the candidate information elements were discussed in a context-free environment. Historical Analysis: This consisted of a survey of existing Army doctrine and the results of previous studies attempting to achreve the Same objective. This process was entirely context-free.
The analysis was psychologically external. Emphasis was placed on the decisions made (the ranking of information elements by importance), and not the mental process used to arrive at the decision.
Although not the main purpose of the study, we were able to provide data which is of use in developing decision making models. The series of decisions taken by the participants allowed us to develop a statistical distribution across a small set of decision categories thus providing a frequency representation of decision making that might be used in models of similar situations. In addition, the examination of the deliberation process, preceding the selection of a course of action, allowed us to develop a set of influences associated with each of these decision categories. Thus, in the same study, we were able to support an exfernal statistical model and one which exploits more fully the psychological effects of
t
Formerly the Defence Operational Analysis Centre (DOAC).
Eden, et a1 [4] page 53.
3
I
judgement and experience setting.
- both in a context-full
Ten experimental sessions were held in which expert participants (either former Royal Navy Captains or Admirals) assumed the mle of Task Force Commander and Deputy Task Force Commander in the context of a combat d o . site.
To simpliry the process and, more importantly, to avoid the pitfalls of combinatorial explosions due to branching of possible options at each of the decision pints, the analysis focused on a single decision point as illustrated in Figure 3. The stream of prior decisions, (contextual considerations in the diagram) and currmtly available information were fransformed into a next decision in Figure 3a (dark region in Figure 3). the Task Force Commander has made several decisions represented by the set of "prior" decisions. Faad with a critical situation, he obtains the additional information quircd to make a decision from his intelligence brid. In Figure3b, we model this processby collapsingthe prior decisions into a single contexhd briefing. In addition, the commander is given a brief on the status of friendly and enemy f o r m and he is allowed to request additional information as required. Judgement and experience are intangibles equally applicable to the model and to the reality ofthe decision making process. If we ignore the judgement and experience component of the model, a context-MI extemal model of the decision making process ensues. Considering judgement and experience through the deliberation process can produce. a more psychologidly intemal analpis resultiog in expressions of likely causality bmad corresponding to a representation of the decision maker's mental model.
An additional tluoz sessions wcn conduned using cumntly serving Royal Navy Capains and one session was conducted at the Maritime Tactical School at HMS Dryad. In this latter session, the Director assumed the mle of Commander and several of the MTS statf assumed the mles of combat statfpnonnel.
The participants were briefed on the political and military events leading to the outbmk of hostilities, the sequenaofdecisions made and the outcomes obsaved up to the cumnt time. with knowledge of the campaign objectiw, the participants were thenasked to cboose a course of action that would best whim a desirable outcome. Information available h m intelligence and other sou~ceswas pnsmtcd in the form of a situation and intelligence briefing. Additional information was pmvidmi only upon requa. The participants were expected to make the decision based on the situation pnsentcd to them, and the information they requsted They were told that they had no mle in the initial planning of the operation they wcn now about to lead, nor had they any mle in the conduct of the campaign up to t h i s p i n t Thywuethen"inscltcd"intothe scenario at the critical decision point
The impact of the decision made on the outcome of the campaign was adjudicated by a computer simulation. The model used was the Interactive MaIitime campaign Program (IMCP) (Perry and White [131). The IMCP is a stochastic, event driven simulation model of maritime combat at the Task Force level. It can be intermpted by the player to probe for information, or to alter the simulation by inserting orders to the force. The sequence of decisions and outcomes presented to the participants were generated by chaxting a reasonable coum of events based on earlier decisions made and IMCP adjudications. These events led to a single critical decision that was then presented to the participants.
Fig 3Single Decision Point Model
Analytic Methods
Structured interview techniques have been
Experimental Design
used before to elicit expert evidence in a way that focuses on a single issue. These are normally
The methodology involves a man-machine simulation of naval combat set in three scenarios, each dealing with a requirement lo escort an amphibious landing force to a designated landing
referred IO as modified Delphi techniques Linestone [ 1 I ] and Shephard [ 17). The Delphi method is generally used to elicit expert testimony
A
I made by the test group during the experimental process. However, not all questions asked were at the same level of aggregation. For example, one question may ask about the weapon system suite on board an enemy submarine whereas another may request intelligence on the likely intent of the enemy fleet. This led us to establish a taxonomy of information categories. A complete listing of the information elements for each of the 5 levels compiled is included in the Appendix. The decimal numbering provides a convenient way to identify the parentchild relationships among the information elements.
on issues that, because of their complexity and abstraction, do not lend themselves to traditional analytic processes such as simulation and modelling. The term "modified" is generally applied in those cases where scientific principles are used in the analysis such as in this study. In either case, the experts are expected to select alternatives to solving a problem. The information they employ and the judgements they make in the decision process are generally ignored. The emphasis is usually placed on achieving a consensus through discussion and feedback. In this study, it was important that the relationship between the information elements requested by the participants and the decisions they made were highlighted for each of the participants.
Referring to the Appendix, we observe that the first question can be categorised as information about enemy submarines (1.1.3.1.10). At a higher level of aggregation, it can be categorised as information about weapon systems (1.1.3.1). At an even higher level, it can be categorised as information about relative combat effectiveness (1.1.3) and at a higher level yet, about combatant forces (1.1). The second question starts at a higher level of aggregation in that it can be categorised as information about the current tasking (1.1.2.1). At the next level, it can be categorised as information concerning plans and operations (1.1.2) and then, information about combatant forces (1.1). Thus, the two questions merge at this level of aggregation. All questions converge at the highest level: situation assessment (I).'
Several methods of analysis were used to summarise the results. These range from simple descriptive statistical representations to formal correlation analysis. In this work, we examined 3 methods of analysis that relate information available and requested to decisions made and (1) their impact on campaign outcomes: probability distributions on decisions conditioned on information requested; (2) influence diagrams drawn from statistical output and anecdotal evidence; and (3) fuzzy membership functions to characterise the information requested in support of decisions taken and to assess group consensus. The methods pertinent to t h s discussion are (1) and (2).
The hierarchical categorisation of information suggests that analysis can be conducted at varying levels of resolution. The set A, = {a,,,a,,,...,a,} is the set of basic information elements at level I, and therefore F, = {F;,,F,,;..,F,} is the set of characteristic information sets for each of the m participant pairs at information level I with n I m as before. The next step was to develop a criterion to establish which information elements, a,,, should be included in a participant pair's characterising information set, Fd , at each level. The objective was to separate casual questions from those of genuine concern. Several criteria were explored, but one based on the frequency of requests for information in a category was selected to illustrate the concept:8
The Information
We begin by identlfying the set of fundamental information elements, A = {a,,a,,~~-,a,}, made up of the information requested by the participants to support a decision to be taken. Of interest are the subsets of A representing information requested by the individual participant pairs. We denote these by where F; c A . The set F] characterises the information requirements of the participant pair i . Therefore, the set F = {4,&,...,F,},6 (nsm), characterises the entire test group of m participants. Note that without membership restrictions on the sets F ] , { l&, F,) = A . We expand on this more h l l y below.
~~
The elements of the set A are taken to be mutually exclusive and collectively exhaustive. They are derived from the information requests
In this research, we were able to categorise 705 requests from 28 experiments with 14 participant pairs at the 5 infomation levels. A frequency criterion is not always the best measure of the genuineness of a participant pair's information concerns although it does have the merit of
The subscripts here do not necessarily correspond to participant pair.
5
requests are included in m01c than one category and at more than one lcvel. There. WQC I4 sus of participants in tbc s M y , each pmbcing decisions in the conttxt of two d o s . W o r e , a total of 28 tabla of this form WQC
Definition: For each participant pair, r. the informauon sd Fu which best characterises their information requirements at each information level, 1, consists of information elements aj E A, , which were constituent parts of more than 10% of the puestioas they asked during the experiment
compiled.
Althoughweprcsent the results here at levcl 3only,the~ampattcrnholdsatall1cvels. Thot is, information derivative of combatam forces information at I d 2 dominatar the infor" quested Figure 4 dcscriks tbe I d 3 dishibution We have that 71% of the 694 information demmts pmvidcd in mpons~to paaicipant RQutsts WQZe chssilied as information concuning ordns of battle (l.l.l), plans and operations (1.12). and combat e&ctivcmss (1.1.3). At level 2, the results an even more dramatic in that 81% of the 705 mprsta were cstegorisedas~tforces(1.1).
From the foregoing we are able to identify the following propenies of pr : Property 1: Each information set is a subseI of A,. Formally, for every sd, F,EF,,wehavethat & C A , atevery level, 1. A h , bckluse of the membershp restrictions on 5 , we have that
{ ~ " , , F Y } E A , .m t i s , upper bound
on Charactensl . .ng sets. {
A, isonlythe the union of
U=,4).
Property 2: The elements of F, are not necesgarily mutnally exclusive. Thus, the s e t s m a y " o v e r ~ i n t h a ti f i $ . F , ~ F , then it is not necessaty that
1IB
F,nF,=0.
Fig 4 -Incormation Distribution at Level 3
Property% Themaximumcardinality (set size) of i$ is k, the cardinality of A, and the maximum cardinality of F, is 2'. the cardinality of the power set of A,? Table 1 illustrates a partial classification for one participan~pair, p . for one scenario. The qUeSIions asked are listed first followed by the response given by the military briefer. The last two columns class@ the information r a p e d at all pertinent levels (a,,). Note also that some beiig purely quantitative. For example, it may be that several questions wncuning combat effectiveness may be asked but only one question concerning rules of olgsgemmt. In the first wse, because of the level of sggregation used, several questions about weapon systems at lower levels are counted. However, the fact that only one question about NI= of engagement was asked docs not u d l y imply that the pair considered it less important. We account for this later througj~the w , o f lnnuence analysis. In reality, the effective upper bound on F, is the number of participant pairs for all but levels in which the n u m b of information categories is small.
6
1
response to idle questions and was not used in the
Once the classification process for the
28
samples was complete, the task was to C h a r a a e n’ s e the information requested by each of the participant pairs within each scenario. If the sample s t had been larger, then it would have been possible to simply characterise the information as the set of unique information elements at each of the levels. For example, at level 3, a complete ’ tion of the participants’ requests based on the partial r e p d o n in Table I is the set F,, = {LU,LU, U.6, 13.1). H then are 12 information elements at this level, a f i ,and so then are 4096 possible sets of this type and thus it is extremely likely that there would be unique sets for the 28 samples.lo Aside from the fact that establishing a relationship would be futile using this Charactensatl . ‘on, it is not clear that this is really an a-te rdection of the information requirements since no acwunt is taken of the fresuency with which information is providd in
deliberation pmass.ll There were three ways to pmaed. first, a charactaisaton based solely on the frequency with which SUeJtions werc asked; second, a characterisation based solely on the use made of the infomation; and finallv. a combination of the two. We satled on the first method because it was the simplest to apply and because it was the most objective approach in an already heavily subjeaive amlySis.12 The fnrluency -C ‘ ‘on records the number of tinus questions werc asked conom@ each of the information elements and at each level of aggregation. For example, Table 2 records the fresuencics for the unnplete set of information elements for the sample participant pair. Although there were only 16 questions asked, there were 20 information elements provided in response to these questions.
Applying the 10% rule (see page 6) to Table
TABLE 1: Information Categories Worksheet for Participant Pair,p: partial Set) a single information element category. Nor is there any account taken of the fact that some of the information may have been provided in
2, we get the following charactaistic sets:
‘1
The number 4096 is the cardinal~tyof the power set for level three information elements. That is 2’* = 4096(lhis includes the extreme cases where all information is requested and where none is requested beyond the initial intelligmce and situation briefmgs).
lo
his o c c ~ m don accasion concaning questions about the weather. Participants also felt obliged to ask questions on occasion to “maintain momentum”. l 2 We have not dismissed the other WO methods, merely postponed their application.
I
(1.1.1, 1.1.2, 1.1.3, 1.2.3, 1.3.1)
2
(1.1.2, 1.1.3) (1.1.1, 1.1.2, 1.1.3)
3
Characterisation
F,,
i
0rdersofBattle.PlansandOpemtions.Combor Efech'veness, Exclusion zones, Political Support Orders of Battle, Plans and Opemtions Orders of Battle, Plans and Opemtions, Combat
{1.1.2, 1.1.3, 1.3.1)
TABLE 3: Characteristic Information Set at Le1 3: F3= {E3 I i = 12,-..,10},
Level 2:
F,, = { 1.1, 12)
Level 3:
Fp3= (1-1.1 1.12, LL3)
Level 4:
Fp4 = {1.11.3, L12.1, 1.13.1).
Clearly, the fact that no distinction is made between an element that has an 80% frequency and one that has a 10% frequency is slightly problematic. In subsequent work, we have accounted for this "strength of membership" problem by considering a consensus metric. That is, we examine the degree to which the participants agreed with each other concerning the information elements deemed necessary to support maritime decisions such as the ones posed in this study.13
At level 3, for example, the level 2 root, combatant forces (1. l), dominated the information set. Information consisted of orders of battle (1.1.1). plans and operations (1.1.2) and combat effectiveness (1.1.3). When applied to all 28 samples, the complete set of characteristic information sets at level 3 is reduced to the set depicted in Table 3. The information elements in bold type are derivative of the dominant level 2 information element: combatant forces. In each of the sets there is at least one information element that has 1.1 as its ultimate "parent" thus illustrating the major influence of information dealing with the combatant forces across all the participants. This is further illustrated below when we examine the distributions of information required conditioned on the decision taken. The "Characterisation" column also reflects the dominance of the combatant forces stem in that the italicised descriptions correspond to the bold element numbers.
l 3 We define the membership function, U , = ( u,(F;)I i = l , 2 , - - - , n ) , for each of the m participant pairs. The quantity u,(F;) measures the degree to which the participant pair, j . supports the consensus that the set 4 characterises its information requirements. We restrict u,(F;) to be between 0 and 1, and to sum to 1 across F. In symbols we have that, uj(F;) 2 0
and
iu,(e) =1.
Consequently, U, is a
1-1
membership function defined on the set F for participant pair j and u,(F;) is a chumcterisric function defined on the fuzzy subset, E;; [SI and [IS].
In determining the values for u,(F;), we again resort to frequency of the requested information sets. Note that we drop the subscript denoting the information category level for simplification. It is understood that the discussion applies to all category levels.
8
. a re-configured force" were classified as reconfigurations. The reason for this is that we considered the operational implications of the tasks associated with reconfiguring the force far outweighed those associated with rerouteing the force.
The Decision In all three scenarios, the mission of the force was to ensure that an amphibious force is escorted to a designated landing site with minimum casualties to both the amphibious force and its escort. Prior to the start of the
experimental phase, several courses of action were postulated by the CDA Naval Staff. These were subsequently added to those selected by the participants. Each course of action selected was then placed in one of the four decision categories listed below. Assignments were made based on the study team's understanding of the primary intent of the decision. For example, if the major intent of a decision was to alter the route of the Task Force while maintaining its current structure, the course of action was considered to be a re-route -- regardless of the myriad other features the decision might possess.
0 Withdraw-Threat Reduction: This category was suggested by the participants and was added as the experiments were being conducted. The concept is to withdraw the amphibious force and its escorts to a safe haven and then use organic and land-based air assets along with forward deployed submarines to reduce the threat to the Task Force before proceeding to the landing site. The concept is the naval equivalent to ground warfare's "shaping the battlefield", i.e., engaging the enemy from a considerable stand-off distance.
0 Planned Route: In tlus category are courses of action that consist of continuing
The participants were asked to articulate their decision once they completed their
TABLE 2: Information Frequencies deliberations. This was taped and recorded in writing. Figure 5 depicts the distribution of the 28 decisions they took by decision category for each of the scenarios. This might be used as a frequency distribution for a results oriented depiction of command decisions for models of littoral maritime operations similar to those depicted in the scenarios. To support the validity of this depiction, we must show that the dominant decision categories (re-route and withdraw-threat reduction) are the best decisions in some sense and that their selection is independent of the information requested. l4 We explore both of these conditions next.
with the current plan or that are variations that have the force continuing essentially along the planned route with a force whose composition is essentially unaltered.
Re-Route: The re-route category consists of all courses of action that are primarily focused on altering the planned route of advance significantly but that also maintain an integral force. Note that the same criteria applies here as with the planned route option, that is, the emphasis is on the intent of the decision and not in the details.
0
0 Re-Configure: Here the emphasis is on reconfiguring the force. Generally this consists of splitting the existing force to provide a decoy or to separate the amphibious group from the escort force to maintain separation between the landing force and the enemy. The categories: "re-route a re-configured force" and "continue along planned route with
l4 The information that was considered valuable is discussed later.
9
succusful in destroying or remcnring tkm as a threat to the Amphhious Group.
Rclativc Effcctivmar dAir Thislartcomponeatwasdesigned to "re the degree to which the air campsip was sucEes6111. S u c h m e a w g a s t b e d e g r r e t o which the friendly force had air superiority and the abiity ofthe gmund and carria-bared airaaft to attack the "y surface ships and conduct MPA altacks WQC applicable. A Forcc Loss Exchange Ratio (pLw) was 0,
R d u -
---
Fig 5 - D c r i n Distribution
developed that messupcd the dative capabiityofthe enemy and friudly air form in tenns of tbeir e k t k n e s indestmying high d u e targes.1'
The Ootcoma Several outcome m a s u m were used in a multi-athibutc evaluation of how well the decision taken arromplisbed the stated mission. Thcse arc listed below. k u s e the missions were similar in each scenario, a set of component outcome m e m r e s applicable to all scenarios was posslble.
Each of the outcome components, o,, was taLen to be a disxetc random variable. Therefore, we were interested in the pmbability that each component outaune would OCNT conditioned on each of the decision catepri~. Because the IMCP is a monte-carlo simulation, it gcnuates fresuency distributions based on 400 replicatioas of the d o events for tach ofthe hrst 5 component out" variables. The FLER is derived from the IMCP f r q l m c y distributions as illustraed in the Appeodix. The sample spaa for each is a simple pattition of the its relevant domain as illusnsrcd in Table 4.
AmphibiW FOW SarViv&iility:. The domestic and international political impact of casualties weighed heavily on the d a i o n to continue operations. This was oompounded by the fact that the embarkad fora was requiredto conduct a critical gmund operation when delivered to the landing site. 0,
The discrete sample spacc arsociated with the component vm'ables delined above are listed in Table 4. Together they gcaaate 216 possible unique outcome sets consisting of one outcome from each of the six component sample spaces. Of these, it is easy to select the "best" set of outcomes and the "worst" set of outcomes. Establishing an ordinal ranldng of the remaining 214 is a bit problematic. ConsequMltly. we fd our attention on these two extnmes, that is, the outcome sets 0(oii.0a29~31.04r. omoa) and
0, Survivability of the Enemy Surface Force: The presmce of an enemy surface force threatening the Task Force constihrtcd a serious threat to the success of the mission SucCesF in dealing with this threat was meanued in terms of the ability of the enemy fllrface force to continue to be an e&aivc fighting force.
Escort Force Survivability: The rationale for this component was, in part, the same as for the amphibious force. Added was comm wer the abiity of the force to extricate itself from the landing site and to return to its base. 0 3
0- = ( 0 1 3 , 4 1 *
03,. 041 3%.
0,
).
Outcomes Conditioned on Information
0, Carrier Survivability: Carriers were treated separately because of their central role in maritime operations. There was a strong Iielihood that the loss of a carrier would abort the mission.
What we would like to evaluate are the probabilities, P(0-( Dt) and
conditional
P(O,IDk), where Dt. (k=I,23,4) are the decision categories defined above. As tempting as it is to assume independence among the outcome component variables, this is clearly not the case
o5 Enemy Submariues: Measures of the effectiveness of the application of ASW included the ability to locate the submarines and the degree to which the Task Force was
Is Appendix.
IO
The derivation of this metric is in the
and
thus
n
a
simple
product
is
correct.
and operations for both forces and the combat effectiveness of both forces. If we reverse the conditioning, and examine the probability that each of the level 3 characteristic information sets was selected given that the participants courses of action were classified in each,of the decision categories, we can see again from Figure 6, that only two characteristic information sets, F,, = {lU, 1.L3) and 4, = {l.ll, 1.12, 1.1.3) depict any sigruficant dependency. Figure 7 depicts the number of times sets &,, &,, FZ3UF,, and all others characterised the decision makers conditioned on the 4 decision categories. Stated in terms of probabilities, Figure 7 depicts the conditional probability distributions: P(FJ D)
6
P ( o , ~ D ~=)
P(o,~Q)
not
,=I
Consequently, we resorted to a multi-attribute assessment of the two conditional probabilities. The results indicated that the outcomes associated with re-routeing decisions and the withdrawthreat reduction decisions were optimal. l6 The conditional distributions allowed us to examine the support for the hypothesis that the information requested was in some way a predictor of the decision to be taken and the converse; that the decision taken implied the type of information requested. The study results do not support either of these hypotheses. This can be seen by the overwhelming amount of information concerning the combatant forces requested by all participants (Figure 4) regardless of the decisions taken. The following analysis draws the same conclusion in a more structured way.
for i = 2,3 along with the union P(F,, U &,ID) and the distribution of all other characteristic sets, P(F, - [F,, U F,,] 1 D). The two characteristic sets, &,, and & 3 , account for 20 of the 28 decisions taken. Only one participant pair requested information characterised by each of the other characteristic information sets in F3 and these were predominantly associated with the same two decision categories. The conclusion therefore is that regardless of the information requested, participants were likely to decide on a re-route or a withdraw-threat reduction course of action.
Again, we report on only 3rd level characteristic information sets (although the same results were obtained at all levels). We are interested therefore in the conditional probability P(D,IF3), where the set distributions, l7
D = { D , , D 2D3, , D4}is the set of decision categories described above. Figure 6 depicts the number of times decisions in these categories were taken for each of the characteristic sets listed in Table 3. From ths it is clear that regardless of the decision taken, the primary information requirements centre on characteristic information sets F,, = {L12, L1.3) and F,, = {l.Ll, L12, 1.1.3) (the level subscript is omitted on the diagram). We conclude therefore that regardless of the decision taken the participants in this study requested information concerning the orders of battle for both enemy and friendly forces, plans l 6 The methodology used was to examine the statistical probability that the “best” and “worst” cases occurred for each decision category in each scenario. Next we examined the outcome pairs and discovered that re-route or withdraw-threat reduction decisions dominated all other pairs for all outcome components. ‘7 It is important to note that the representations in Figures 6 and 7 are not probabilities, but rather frequencies. Because the number of decisions (28) was rather small, it was not possible to depict the results in terms of probabilities. However, the focus on probabilities in the accompanying discussion is justified in that it provides a useful tool for conceptualising the process for cases in which the number of decisions is acceptably large.
11
Fig 6 -Characteristic Set Frequencies Conditional on -on
Categnnes
Component Virisble
Outcome Sunplc Spice
0 , Amphbious Force Sumvability
o,, Amphblous Group lossCS M lrrs than IW.. o,y Amphbious Group losses M morc than 10% but less than 20% o,, Amphbious Group losses M more than 20% 02, Enemy surface force greata than 3W. effcctlvc 0, Enemy surface forces less UW 30% cffcd~ve
o2 Smvability of the Enany Surface Face 0,
Escort Forte Smvability (less
C”) 0,. Carrier Survivability.
Losses are less than IW. more than 10% but I*u than 20% 0,. Losses exceed 20%. O,,
1
03yLosses M
I o,, carrier is incapacitated. ,
I 0.. . carria is not i n d t a t e d . 0,. Enemy s u b ” .
0,. Relative ENectiveness of&
Assets.
0,, .50% or more of the emmy submarines are hapcitated.
o,, . Between 20% and 50% of the enemy submminu M incapacitated. q3.Fewer than 20% of the enemy submarine8 are incapacitated. O,, . FLER is greater than 1 (favours Task Forte). OSy.FLER is less than 1 (favours enemy).
TABLE 4: Component Outcomes not satisfy all modelling requkamts. However, the method is valid and might k applied to more detailed courses of a d o n with a larger sample W.
In this example, we have resorted to examining the output from the decision making proass (the decisions), and developed an empirical frequency distribution that could be used to model similar decisions in similar scenarios. We obsewed that (I), the decisions taken most frwluently did pmduce “good outcomes” in some statistical sense, and (2). the decisions were not dependent upon the information requested, i.e.. that all 4 decision categories were likely in accordance with the derived frequency distribution depicted in Figure 5. From a practical point of view, the highlv aggregated nature of the decision categories may
That commanders tend to make “good“ decisions was comforting, but the absence of a strong link betwan the information requested and the decision taken was a bit unsatling. We therefore re-examined the data focusing on the participants’ decision making procss as opposed to the more objeftivc o b s e d o n of the decisions they took. That is, we conducted an internal, context-full analysis to complement our external context-full results.
12
0
TbMt Pcreeption: There WMe three main
Aaalysu of Influence
threats to the friendly Task Force in each of the scenarios: organic and land-bssed air,
We did this by extending the work to include the more subjective pof analysing the !kctors that influenced the participants in taking the decisions they did. The objective with nSpea to modelling the decision making proass was to establish an approximate causal relation behwen
enemy surfacc action grow (SAG); and enemy submarines. The influence of these threats on the murse of action taken by the participants aepended upon their pcrccption of how serious the threat was to the accomplishment of their mission.
scenario muditions ( i n f o d o n and intelligence) and the commimdn's elrperience and judgement. Thus, we moved away from a purely external analysis to one more p s y f h o l o g i ~internal while at the same time,mntinuing with a mntext111dimeosion
oThnc: Thetimeallowedtoaccomplishthe mission varied among the scenarios and the innuence of time constraints on the decision takenvaried accordingly.
Rirk Aversion: The participants' attitudes towards risk in milimy operations was a considerable idin the decision making proass. In general, participants chose to avoid risk because of the vulnnability of the amphiim force they were charged with escorting to the landing site. In addition to the natural impulse to prscrn the force, risk aversion is also caught up with the h i r e to maintain political support. The possibility of high casualties generally emdes support for military operations.
Another argument for expanding the study is that mtidcal Summaries such as those discwed above, can be problematic, and for several reasons: the sample size, the nature of the sample (not random), the variability of the experiments (despite best &or& to eliminate bias), and the inevitable ef€ects of group dynamics. Couseqmntly, a more subjective anatysi based on an examination of innuenccS of information on decisions takm is further justified. We used the anecdotal data caphned during the participant deliberation process descn'bed above to construct graphical intlumce connections. We then focused on how the content of the information lqlmted mupM with decision makers' attitudes towards risk and their tactical and strategic mncem,
political Factors: There were few questions about political issues and M o r e political factors had l i Mumce on the decision making pmcss. This was mainly due to the fad that the participants were inserted in an already ongoing mntlict where d e s of engagement had been set previously.
affectthedecisiontaken.
We observed five basic positive and negatrve influence mmponents. These are defined below. Table 5 records the detailed summaries of the influences within each of the decision categories across all three scenarios for all participants.
s Environment: Here we focus on the in the area of physical e d " e n t operations. This includes such things as the
-
U_
Fig 7 -Decision Categories Conditioned on Characteristic Set Frequencies
13
. weather, the navigability of the waters in the area, the status of the landing site, etc. Althongh several questions were asked about the environment, environmental considerations intluenced only a few participants and only when considering reroutdng the force.
Strategy Aims: The participants wed in the study were considerably experienced and most had formulated strategies based on their interpretation of some basic war-fighting principles. The factoxsthat they considend in articulating their strategy aims were: reduction of the W;securing sea control; deceiving the en-, protecting mission essential nnits (MEW; and conantrating the force. Thsc aims constimed the single greatest i n i l m on the decisions taken. In all three scenarios and for dl decision categories mnsidetw.i, participants ani&ted several factors that i n i l d their decision In general withdraw-threat reduction options were aimed at gaining sea mntml as well as reducing the enemy threat. Re-muteing options were f a v n u e d where deception and the possibility of increasing the enemy's radius of action were considered important. Consideration was given to reanfiguration options to concentrate the f o r a Figure 8 depicts how isfor", judgement and experhce innucna the selection of courses of action within each of the decision categories for the three ScenaTiOs nsed. The number, n, in the box on an an: means that the component in the
Fig. 8 -Infhmca on Decishm
The summaries in Table 5 end in Figtire 8 illustrate the major intlucnca on the dccipions taken. Notethatacrcwsallthreesmarioqthe~ sum of i " on T o c o decisions ~ was +20; on rc-routc decisions, +25; on plaMed decisions, +4; and on withdraw-thrcat Icduction decisions, +29. Notc also that the gnatest influences on decisions (R~IUC 8) were the strategy aims of the participgnts and their perceptionsofthcthrrat. Tbonainnmncescons induce an ordering on the dsisions taken which is condstent with the decision distribution depicted in Figun 5 except that hen, there were more net intlnentxs on the withdraw-thrrat
TABLE 5: Influences on the Decisions Taken
14
reduction decisions than there were for re-route decisions. Nevertheless, Table 5 helps to explain the distributions in decisions and it illustrates how the dominant, level 3 information sets FI3 and F,, depicted in Figures 6 and 7 influence the decisions taken in each scenario given that they articulate the threat.
external. At the extremes, internal context-free analyses focus more on understanding the decision making process and developing tools for its representation. At the other end of the spectrum are those studies that are external and context-full. These are almost exclusively application oriented. Their main focus is problem solving.
Recall that the information elements in F,, and F,, deal with orders of battle, plans and operations, and the combat effectiveness of both friendly and enemy forces. We observed that regardless of the decision taken, these two information sets dominated the questions asked by the participants and therefore we were not able to establish a relation between the information requested and the decision taken. Examining the influences associated with the information content, we can now examine how the information presented in response to participants' queries influenced their decisions. The information contained in the two dominant information sets helped to establish a force balance which the participants translated into a net threat assessment. This is evident from the dominant influence of threat perception on the decisions taken. Other information influenced the decision to a much lesser degree. The dominant role of strategy aims illustrates the existence of internal influences not captured in the information sets.
The implication of this categorisation for the development of models of decision making appear to be in recognising the degree to which a representation of the mental process is required as part of the simulation. Models of human mental processes with respect to decision making draw on the results of internal analyses whereas models for which decision malung is subsumed in a command and control module would benefit more from the results of external studies. The contextual dimension can impact on the credibility of the reported results provided the context is considered to be representational at whatever degree of resolution.
Summary
ACKNOWLEDGEMENT
In this paper, we have suggested a twodimensional taxonomy of decision making analyses that might prove helpful in the development of models of human decision making. Several of the studies examined focused more on the psychological aspects of how decisions are made (internal) whereas others are concerned only with the decisions taken (external).
The authors wish to thank Professor Ken Bowen, University of London, for taking time to review this paper. He provided an invaluable service in catching those glaring errors that escape all authors, and more importantly, his insightful comments drawn from years of experience helped sharpen the final product.
Analyses of decision making processes can also be categorised in terms of the degree to which context is employed in the study process. The participants may be placed in the context of a realistic scenario (context-full) or they may be asked to simply respond to questions devoid of any specific context (context-free).
Information Elements The information elements derived from the maritime command and control study are presented below. Only those elements requested by at least one of the participants are listed. At the highest level, all questions were aimed at clanfling the battlefield situation.
In reality, most studies fall somewhere between the two extremes in both dimensions. Most of those we examined included some context but were relatively evenly distributed between being predominantly internal and predominantly
1 Situation Assessment
Our work in the area of maritime command and control serves as an example of both internal and external, context-full analyses. Analysis of decision influences is an attempt to examine the internal mental processes in decision making whereas the statistical representation of decisions is clearly an external process. In both cases,the use of detailed scenarios provided full context.
APPENDIX
1.1 Combatant Forces 1.2 Operating Environment 1.3 Political Considerations
1 . 1 Combatant Forces
15
1.1.1 Orders of Battle 1,1,2 Plans and Operations 1.1.3 Combat Effectiveness
1.1.2.1 Current T a s k q and Plans 1.1.2.1.1 Planned Advance Route of Friendly Force 1.1.2.1.2 Perception of Enemy Actions 1.1.2.1.3 Task Force Formation 1.1.2.1.4 Role of Friendly Organic and LandBdAir 1.1.2.1.S Role of Enemy Organic and LandBdAir 1.1.2.1.6 Role of Friendly ASW and ASuW
1.2 Operating Environment 1.2.1 Weather 1.2.2 GeographidOceanographic 1.2.3 Exclusion Zones 1.2.4 Status of Non-Combatants 1.2.5 Rules of Engagement 1.2.6 Weapons of Mass Destruction
Assets 1.1.2.1.7 RoleofEnemyASW and ASuW
1.3 Political Considerations 1.3. I Political Support 1.3.2 Enemy Alliances 1 .3.3 Neutral Nations
Assets 1.1.2.1.8 Timing Issues 1 . I .3.1 Weapon Systems Performance 1.1.3.1.1 Friendly Land-Based Air I . 1.3.I .2 Enemy Land-Based Air 1.1.3.1.3 Friendly Organic Air I . 1.3.1.4 Enemy Organic Air 1.1.3.1.S Friendly Surveillance Systems 1.1.3.1.6 Enemy Surveillance Systems 1.1.3.1.7 Friendly Surface Ships
1.1.1 Orders of Battle 1.1.1.1 Organisation 1.1.1.2 Forces 1.1.1.3 Surveillance 1.1.2 Plans and Operations 1.1.2.1 Current Tasking and Plans 1.1.2.2 Location of Forces
1.1.3.1.8EnemySurfaceShips 1.1.3.1.9 Friendly Submarines 1.1.3.1.10 Enemy Submarines 1.1.3.1.1 1 Friendly Air Deface 1.1.3.1.12 Enemy Air Defence 1.1.2.1.13 Friendly Ground Forces 1.1.2.1.14 Enemy Ground Forces 1.1.2.1.15 Friendly EW Systems 1.1.2.1. I6 Enemy EW Systems
1.1.3 Combat Effectiveness 1.1.3.1 Weapon Systems Performance 1.1.3.2 Damage Assessment 1 . I .3.3 Repair and Re-supply 1.1.3.4 Training and Morale 1.1.3.5 C31 1.3.1 Political Support 1.3.1.1 Domestic Support 1.3.1.2 International Support
1.1.3.2 DamageAssessment 1.1.3.2.1 Friendly Land-Based Air 1.1.3.2.2 Enemy Land-Based Air I . I .3.2.3 Friendly Organic Air I . 1.3.2.4 Enemy Organic Air 1.1.3.2.5 Friendly Surveillance Systems 1.1.3.2.6 Enemy Surveillance Systems 1.1.3.2.7 Friendly Surface Ships 1.1.3.2.8 Enemy Surfaceships 1.1.3.2.9 Friendly Submarines 1.1.3.2.10 Enemy Submarines 1.1.3.2.11 Friendly Air Defence 1.1.3.2.12 Enemy Air Defence
1.1.1.1 Organisation 1.1.1.1.1 Friendly Command Levels
1.1.1.1.2 Enemy Command Levels 1.1.1.1 .3 Friendly Command Relationships 1.1.1.1.4 Enemy Command Relationships 1.1.1.2 Forces 1.1.1.2.1 Friendly Land-Based Air 1. I . 1.2.2 Enemy Land-Based Air 1.1.1.2.3 Friendly Organic Air 1.1.1.2.4 Enemy Organic Air 1.1.1.2.5 Friendly Air Defence 1.1.1.2.6 Enemy Air Defence 1.1.1.2.7 Friendly Surface Ships 1.1.1.2.8 Enemy Surface Ships 1.1.1.2.9 Friendly Submarines 1. I . 1.2.10 Enemy Submarines
1 .1.3.3 Repair and Re-Supply
1.1.3.3.1 Friendly Air to Air Refuelling 1.1.3.3.2 EnemyAirto Air Refuelling 1.1.3.3.3 Friendly Ammunition 1 . I .3.3.4 Enemy Ammunition 1.1.3.3.5 Friendly On-Board Repair Capacity 1.1.3.3.6 Enemy &-Board Repair Capacity
1.1.1.3 Surveillance I , I . 1.3.1 Friendly National and International 1. I , 1.3.2 Enemy National and International 1. I , 1.3.3 Friendly Organic Assets 1.1.1.3.4 Enemy Organic Assets 1 . I . I .3.5 Friendly Land-Based Surveillance I . 1.1.3.6Enemy Land-Based Surveillance
1.1.3.5 Command Control Communications and Intelligence 1.1.3.5.1 Friendly Command Relationships 1.1.3.5.2 Enemy Command Relationships 1 . I .3.5.3 Friendly Operational Procedures I . 1.3.5.4 Enemy Operational Procedures 1.1.3.5.5 Quality of Friendly Intelligence 1.1.3.5.6Quality of Enemy Intelligence
16
,
9
‘i
1.1.3.5.7 Friendly Communications . . COMeCtiVity 1.1.3.5.8 Enemy Communications Connectivity
Helsinki July 1-3, 1992, Helsinki University of Technology, Systems Analysis Laboratory, Research Report A56, December 1994. Conference
Force Loss Exchange Ratio (FLER) The Force Loss Exchange Ratio for aircraf? is a relative measure of the effectiveness of friendly and enemy air operations in each scenario. The numerator consists of the s u m of weighted fractions of enemy losses due to friendly air operations and the denominator consists of the sum of weighted friendly losses due to enemy air operations. Consequently, a ratio greater than 1 indicates that fnendly air is superior and a ratio between 0 and 1 favours the enemy. It is defined as: FLER =
c:=,
4.
Eden, C., et. al., Thinhng in Organizations, MacMillan Press, London, 1979.
5.
Feher, B. Methoak for Assessing Tactical Decisionmaking in Battle, Pub XB-ONT,
(Wl
where r, , 6, is the fraction of enemy, friendly destroyers-frigates killed; r z , bz is the fraction of enemy, friendly submarines killed; r3, b, is the fraction of enemy, fnendly Amphibious craft killed; r4, b4is the fraction of enemy, friendly aircraf? killed; and the w,are weighting factors for each of the fractions. l8
6.
Hodges, J.S., “Six (or So) Things You Can Do with a Bad Model”, Operations Research, Vol. 39, No. 3, May-June 1991.
7.
Ingram, M.C., Defining the Relevant Common Picture, US Army Training and Doctrine Command Study Paper, TRADOC Study Analysis Center, August 1994.
8.
The losses reported by the MCP are average values over several runs and therefore, the FLER is considered as an average of some kind. To be consistent with the other outcome measures, we assess the probability that the true FLER value is greater than 1 or between 0 and 1 based on the average value calculated. An empirical distribution was used for this purpose based on the frequencies associated with the FLERs component elements.
Kaufmann, A., Introduction to the Theory of Fuzzy Sets: Volume I , Fundamental Theoretical Elements, Academic Press, New York, 1975.
9.
Klein, G.A., “Recognition-Primed Decisions”, in Advances in Man-Machine Systems Research, Vol. 5., Rouse, W.R., ed.
10. Klein, G.A., and Thordsen, M., “Recognitional Decision Making in C2 Organizations”, Proceedings 1989 Symposium on Command and Control Research, National Defense University,
REFERENCES
Washington D.C., June 27-29, 1989.
Bowen, K., “System-based Interviewing: its role in co-operative groupdecision”,
11. Linstone, H.A., et al, eds., Delphi Method, Addison-Wesley, 1975.
Proceedings of the Post - Conference on M/S Sally Albatross. E U R O X I I f l I M m . Joint International Conference Helsinki July 1-3, 1992, Helsinki University of
12, Perry, W., and Moffat J “Measuring Consensus on Information Needed by Maritime Task Force Commanders”, Proceedings I995 Symposium on Command
Technology, Systems Analysis Laboratory, Research Report A56, December 1994. 2.
Daniel, D. “What Influences a Decision? Some Results from a Highly Controlled Defence Game”, OMEGA, The Int. JI of Mgmt Sci., Vol. 8, No. 4.
NCCOSC, RDT and E Div., San Diego CA, 1993.
C:E,bIWl ’
1.
3.
and Control Research and Technology,
Bowen, K., “Understanding and Structuring New Problems”, Proceedings of the Post -
National Defense University, Washington, D.C.. June 19 - 23, 1995.
Conference on MIS Sally Albatross. EURO.YII/TIMSKKYI Joint International
13. Perry, W.L. and White, S.J. The IMCP: A Toot for the Analysis of Maritime C3I,
Unpublished MOD Report.
l 8 These weights sum to 1 and are based on the relative importance of each of the high value assets.
17
'I ?'
b
6
14. Perry, W.L., et al, "The Value of Information on the Outcome of Maritime Operations: The Use of Experts in Analysis", Proceedings Eleventh International Symposium on Military Operational Research, The Royal Military
College of Science, shrivenham, 'Wiltshire, UK,September 5th - 9th 1994. 15. Perry, W., and Moffat J., "Assessing the Value of Information: The Use of Experts in Analysis", Military Science and Modeling, RAM)Corporation, Vol. 6, No 3, November 1994. 16. Perry, W., Moffat, J., and White, S., The Impact of C31 on Maritime Operations: The Value of Information, CDA unpublished
report. 17. Shephard, R.W., Some Experiences in the Conduct of Expert Panels, BDM International, McLean VA, November 1992. 18. Yager, R., et al, ed. Fuzzy Sets and Applications: Selected Papers by L. A. Zadeh, John Wiley, New York, 1987.
18
