Abstract. We analyze the case of protecting an airport, in which there is concern with terrorist threats against the Air Traffic Control (ATC) Tower. To deter terrorist ...
Security Economics: A Multiobjective Adversarial Risk Analysis Approach to Airport Protection Javier Cano1, David R´ıos Insua2 , Alessandra Tedeschi3, U˜gur Turhan4 1 Department
of Statistics and Operations Research, Rey Juan Carlos University 2 Royal
Academy of Sciences, Spain 3 Deep
4 Anadolu
Blue Srl, Italy University, Turkey
Abstract We analyze the case of protecting an airport, in which there is concern with terrorist threats against the Air Traffic Control (ATC) Tower. To deter terrorist actions, airport authorities rely on various protective measures. They have considerable costs, but by deploying them, airport authorities expect to minimize the risks of terrorist actions. We aim at giving advice to the airport authorities by devising a security resource allocation plan. We use the framework of adversarial risk analysis to deal with the problem. Keywords: Adversarial Risk Analysis, Intelligent Attacker, Critical Infrastructure, Multiattribute Expected Utility, Airport Installation, Airport/ATM Security.
1
Introduction
As cogently illustrated in Lomborg (2008), many of the world’s biggest problems, like proliferation of weapons, armed conflicts, corruption, terrorism, drug trafficking or money laundering, are related with security, which is indeed one of the most menacing global issues. 1
By now, there is a large tradition within economics and modeling of security, including the pioneering work of Becker (1968), who largely initiated the field with his economic theory of delict, or Cornish and Clarke (1986), who emphasized operational aspects, like those of situational crime prevention and the reasoning criminal, stressing the relevance of rational choice theory within criminology. Large scale terrorist attacks, like 9/11 or the Madrid train bombings, see Haberfeld and von Hassell (2009), have entailed a renewed interest in the economics of security, given the significant aftermath investments in preventive and reactive measures, which have been questioned by public opinion. Merrick and Parnell (2011) provide a review of recent approaches in counterterrorism modeling, favoring adversarial risk analysis (ARA). In ARA, see R´ıos Insua et al. (2009), the aim is to support one of the participants (the Defender) who will use a decision analytic approach to solve her decision making problem. For this, she needs to forecast the actions of the other participants and, consequently, the outcomes which she and her opponents will receive. In this paper, we describe how ARA may be used to find the optimal security resource allocation to protect a single site of interest to a Defender, which might be a Government or a private body responsible of security, from the attacks of a terrorist organization, which we designate the Attacker. From the Defender’s point of view, these problems often have a multiobjective nature. Although security of people and installations is always a main concern for authorities, there will typically be other relevant issues that need to be taken into account as e.g. the investments in preventive measures or the economic, social and/or political implications of the potential damages caused by terrorists. Interestingly enough, note that the Attacker might have also multiple objectives in his decision making. Thus, ARA aims at providing one-sided prescriptive support to one of the opponents, the Defender, based on a subjective expected (multiattribute) utility model, treating the adversary’s decisions as uncertainties. In order to predict the adversary’s actions, we model his decision problem and try to assess his probabilities and utilities. Assuming that the adversary, the Attacker, is also a expected (multiattribute) utility maximizer, we can predict his actions by finding 2
the action that maximizes his expected multiattribute utility. Our uncertainty about the Attacker’s probabilities and utilities is propagated over to the Attacker’s optimal decision and incorporated in our forecasting model. We focus on an airport security problems. Millions of people pass through airports every day, not only passengers and their companions, but also ground and airline staff, and other personnel. Such gathering of people, together with the strategic value of their installations make airports a potential and prime target for terrorists. Furthermore, the possibility of hijacking an airplane and using it as a lethal weapon against people and/or infrastructures, as in S-11, adds an extra motivation on the terrorists’ will to attack airport installations. Airport authorities worldwide are deeply concerned with this type of threats. To this aim, they invest annually large amounts of money in preventive measures, trying to minimize the chances of any dangerous situation happening within airport sensitive areas. As such, airport security serves several purposes in this context: (1) to protect the airport installations from the terrorist threat; (2) to guarantee the safe functioning of a vital transportation means; and (3) to protect a nation and its people. Thus, we analyze how to support the authorities of an airport who are concerned with terrorist threats against airport installations and operations. Specifically, we focus on a particularly critical scenario: the unlawful access to the ATC Tower, aimed at taking hold of Air Traffic Control Officers (ATCOs) before or during flight control operations. Consequences of these acts have a multiattribute nature, and could be severe, including: (1) a crisis on air traffic operations in the airfield and airspace; (2) flight safety very negatively affected; (3) air traffic canceled or diverted to other ATC units or airfields, with important economic, social (in extreme cases, even in terms of human lives) and image consequences. As a way to minimize the probability of occurrence of such an attack, as well as to reduce the severity of its impact, airport authorities may consider incrementing current security levels by investing more in already existing human and technical resources. These preventive measures include police and security guards, as well as screening and detecting devices, with non-negligible costs. Besides, in the event of a successful attack within the specific country we consider, a 3
Special Police Force (SPF), linked with the Government, will be immediately called on to take control of the situation, aiming at recovering from the attack as soon as possible, trying, at the same time, to minimize its consequences. We assume that airport authorities do not have any alternative for this decision: they will call the SPF for sure. The intervention of the SPF might have drastic consequences, especially for the terrorists, whose lives will be at risk, although collateral damages on persons and goods could also be suffered on the Defender’s side. Negotiation with the terrorists is not considered, in principle, as an acceptable option. However, the presence of hostages could postpone the deployment of the SPF for a certain period of time, while airport authorities try to convince the terrorists to surrender without blood spilling. Nevertheless, should the terrorists not drop their attitude, threatening other people and fixtures on the Defender’s side, the SPF will finally intervene. The multiobjective nature of the problem becomes a more delicate issue as human lives and severe consequences are concerned. We model the problem as a particular case of a Sequential Defend-Attack model within the Adversarial Risk Analysis framework, see R´ıos and R´ıos Insua (2012), in which the airport authorities would first deploy a portfolio of preventive measures to deter or mitigate the actions of the terrorists. Then, the terrorists, having observed such decision, would follow a strategy to perform their terrorist attacks. Should the attack be successful, an SPF would be immediately deployed. We assume that the consequences for the Defender will depend on the effort in implementing their preventive actions, the impacts of the attack and the eventual result of the SPF action. Similarly, the consequences for the Attacker will depend on the costs of deploying their terrorist actions, the impact of their attack, and the final outcome of the crisis after the intervention of the SPF, which will possibly entail some casualties on their side. Both the Defender and the Attacker are regarded as expected utility maximizers. The structure of the paper is as follows. In Section 2, we provide a detailed description of the selected scenario: the unlawful access to the ATC Tower. Section 3 describes the ARA model we have devised to solve the problem. We apply it to a specific case study in Section 4
4. We end up with some discussion.
2
Description of the Scenario
In the case of concern, the airport ATC Tower has its only access gate within the terminal main hall. One can only reach this gate after passing the security checks situated at the entrance of the terminal building, which are performed by the private security personnel. Access to the ATC Tower is controlled by the ATCOs with the aid of a camera installed over the access gate. When an authorized person needs to enter into the ATC Tower, she has to ring a bell and, upon approval by the ATCOs, the door will be remotely opened. An Attacker among the passengers or airport workers can plan to enter the ATC Tower and take hold of the ATCOs, before or during flight control operations. We consider that the Attacker is a group of between one to five terrorists. We assume that the more members in the terrorist cell, the more chances they will have to be successful in their attack. After passing by the first security checks, the Attackers could create an opportunity to enter into the ATC Tower gate, capture the ATCOs and use telecommunications to interfere with air traffic operations.
2.1
Multiattribute consequences of a terrorist act against the ATC Tower
The impacts of a successful terrorist act against the ATC Tower can be potentially catastrophic, causing a crisis over air traffic operations, in the airfield and the airspace. As a consequence, the safety of all flights involved would be negatively affected, and it could be necessary to divert air traffic to an alternate ATC unit or airfield. Besides, during the initial crisis phase, pilots and/or other affected stakeholders might not be able to understand the seriousness and implications of the situation, preventing them from making the most appropriate decisions, something which could eventually worsen the situation. Under all circumstances, pilots and ATCOs should be able to manage their flights and operations in 5
the safest manner. Other collateral consequences after a successful attempt to access the ATC Tower occurs could be: • Besides safety and security impacts, cancelation consequences can be considerable, as connected national and international flights and/or airports and airspaces could be affected. • Media will inform people immediately about the case. This could cause yet another crisis around airport facilities, because of people trying to access/escape from it. • Negative perception of security by airport users. As a result, people’s image of aviation industry could be affected negatively and they could opt for alternative transport means, affecting the balance sheet of the operator.
2.2
Countermeasures deployed by airport authorities
Airport authorities are considering investing in technical resources to increase security by checking passengers (through biometrics) and their baggages. They also check airport and airline staff or personnel of entities providing services to the airport (construction, maintenance, catering, etc). Besides, there are several different bodies in charge of airport security, depending on the regulations of the specific country and on the size and features of the airport under concern. However, we shall encompass them into two broad groups: airport police and private security personnel. We outline below the main features of all these countermeasures. • Cameras. They are used for biometric control, identifying people through their characteristics or traits. They are used as a form of identification and access control, and also as a way to identify individuals in groups under surveillance. In general, the more cameras, the more chances to alert from suspect people entering the airport. • Metal detectors. In general, the more scanning units and more operations per unit, the more customers and baggages will be checked, reducing the probabilities of a terrorist 6
attack. • X-ray devices, have a preventive role, similar to metal detectors. • Airport police. In general, the more police, the less chances that an unauthorized person will gain access to the ATC Tower. • Airport private security. In general, the more personnel, the more customers and baggages will be checked, reducing the chances that an attacker could enter and/or introduce dangerous material within the airport. The SPF, who will intervene in case an attack succeeds, will be considered as a recovery measure. They are an elite police team, specifically trained for this kind of events, and linked with the Government. As such, their hypothetical deployment would entail no financial consequences for the airport authorities. They will be called on in case of emergency, commanding the situation. Their major concern is to recover control of the ATC Tower and capture the attackers with no life cost. However, in our incumbent case, should the attackers put up a bloody resistance, airport authorities would have little concern about terrorists’ lives. On the other hand, the presence of hostages on the Defender’s side would have little influence on the forcefulness of the SPF action in case the terrorists refuse to drop their attitude. Furthermore, there is no quandary about their intervention: they will be deployed mandatorily whenever a successful attack occurs (and, in consequence, there is actually no decision associated with this event). Nevertheless, we shall include them in the model since their intercession will affect substantially the final result of the attack.
3
The Model
We shall consider a sequential Defend-Attack model to structure the problem, see e.g. R´ıos and R´ıos Insua (2012). In it, airport authorities first deploy a set of preventive measures to protect, among other targets, the access to the ATC Tower. The Attacker, who observes such
7
measures, will decide on whether or not to launch an attack. The Attacker may consider different severity options for the attack, which will be modeled through the number of terrorists taking part in the attack. Finally, should an attack be successful, airport authorities will try to recover from it and minimize its consequences by deploying additional measures, which in our case will imply calling the SPF. As mentioned before, given the peculiarities of this case study, in which human lives are involved and enormous economic, social and political consequences are in play, airport authorities will deploy all the available recovery resources after a successful attack, regardless of any other consideration. Therefore, we will not consider their deployment as a second defensive decision step, typical of sequential defend-attack-defend models. A biagent influence diagram for the problem is shown in Figure 1, see Pearl (2005), with white nodes belonging to the Defender, dark grey nodes belonging to the Attacker and light grey nodes shared by both of them. Node “Prev. measures” corresponds to the Defender’s portfolio of preventive measures, x ∈ D1 . Node “Countermeasures” is a deterministic node related with the deployment of recovery measures. There is actually no decision associated to it but, rather, an automatic response: in case of a successful attack, the SPF will be immediately called on. No additional resources will be summoned if the attack fails, since we assume, in that case, that the terrorists have been killed or detained by ordinary police and/or private security personnel or, eventually, some of them managed to escape. The node “Attacker decision” represents the decision undertaken by the terrorists, once they have observed the defensive measures deployed by the Defender. The set of all possible attacks a is denoted by A. The only initially relevant uncertainties for this problem are: (1) the preliminary result of the attack, s1 , represented by the node “Result”, which depends probabilistically on (x, a) ∈ D1 × A; and (2) its final outcome after the intervention of the SPF, s2 , represented by the node “Final Result”. We denote by S1 and S2 the sets of all possible outcomes for the corresponding events. Regarding the multiple attributes of the problem, detailed later on, we assume that the consequences for the Defender, as represented by the chance 8
Prev.
Attacker
measures
decision
Result
Countermeasures
Final Result Cons.
Cons.
airport
attacker
uD
uA
Figure 1: Influence diagram for the airport case study. node “Cons. airport”, will depend on (x, s1 , s2 ), i.e., the effort spent in implementing the protective measures, the initial impact of the attack and its final result after the intervention of the SPF. Then, she will get her utility uD . Similarly, the multiple consequences for the Attacker, summarized on the chance node “Cons. attacker”, will depend on (a, s1 , s2 ), i.e., the effort spent in launching the attack, and the related initial and final outcomes. He will then get his utility uA . We describe now in detail the different elements in D1 , A, S1 and S2 . • D1 . We consider all feasible portfolios (x1 , x2 , x3 , x4 , x5 ) of measures which, respectively, represent the number of additional cameras, metal detector units, X-ray devices, police and private security members deployed. They have associated per unit costs for the incumbent planning period, cj , j = 1, . . . , 5, respectively. Then, if we denote by
9
B the available budget, the feasible portfolios of preventive measures would satisfy
c1 x1 + c2 x2 + c3 x3 + c4 x4 + c5 x5 ≤ B, x1 , x2 , x3 , x4 , x5 ≥ 0, x1 , x2 , x3 , x4 , x5 integer.
• A. We model the Attacker as an organized group composed of between one and five terrorists. Therefore, we shall consider the number of terrorists actually performing the attack against the ATC Tower as the decision variable for the Attacker, i.e. a = {0, 1, 2, 3, 4, 5}, including the possibility of no attack (a = 0). • S1 . The possible values for s1 are {0, 1, . . . , a}, representing how many terrorists managed to gain access into the ATC Tower. The rest of them would have been killed or detained during the attack, or might have avoided being captured, as specified later on. If s1 = 0, the attack fails and the problem is over. Irrespective of the result of the attack, some casualties could also occur among the defenders. We will discuss this issue later on, when analyzing the Defender’s problem. • Regarding S2 , we shall assume that the ATC Tower will be always recovered by the SPF, since this has been historically the case in similar episodes in the past: security forces take hold back of the situation sooner or later. However, we are interested in how many terrorists will be killed or detained. We then define the possible values in S2 as s2 ∈ {0, 1, . . . , s1 }, representing the number of terrorists killed. The remaining terrorists, s1 −s2 , will be detained. At this stage, we explicitly rule out the possibility of some terrorists getting away from the ATC Tower. Besides, additional casualties could also occur among the defenders. We will give details about the entailed consequences of the recovery actions for both adversaries later on.
10
3.1
The Defender’s problem
We sketch the Defender’s problem in Figure 2. As we can observe, the Attacker’s decision node, is perceived by the Defender as a chance node.
Prev.
Attacker
measures
decision
x
a|x
Result
s1 , z1 |x, a
Counter-
Final
Cons.
measures
Result
airport
d2 |s1
s2 |s1 , d2
cD |d1 , s1 , s2
uD
uD (cD , d1 , d2 )
Figure 2: Influence diagram for the Defender’s problem. The Defender’s dynamics involve the following stages: 1. She invests (x1 , x2 , x3 , x4 , x5 ), incurring in a cost c1 x1 + c2 x2 + c3 x3 + c4 x4 + c5 x5 . Note that (x1 , x2 , x3 ) serve to reduce the likelihood of any prohibited item or artefact being introduced in the airside or in security restricted areas. They also increase the probability of detecting suspect people trying to enter the ATC Tower with criminal intentions. On the other hand, (x4 , x5 ) serve to deter the actions of potential attackers. 2. She observes whether an attack is launched by the terrorists. If such attack is successful, she calls on the SPF, so as to hold back control of the ATC Tower. 3. She faces the multiple consequences in relation with the possibility of a terrorist attack against the ATC Tower, and the recovery from it after the intervention of the SPF. We specify such consequences below. 4. She attains her (multiobjective) utility. We provide now a detailed description of the relevant consequences for the Defender in case of a successful attack: • Lives and injuries. When the first strike of the terrorists takes place, everybody present in the scenario could be in injury and/or life danger, including the possibility of being 11
taken as hostages by the terrorists. This affects passengers and staff inside or near the facilities, as well as ATCOs in the ATC Tower. Additional casualties among SPF members, other involved security personnel or civilians could also happen during recovery actions. We take all this into account by defining a variable y ∈ {0, 1, . . . , m}, representing the number of casualties or severely injured people on the Defender’s side. Here, m is a maximum theoretical number of possible victims. Furthermore, we assume a fixed cost of a life on the Defender’s side clife, irrespective of their affiliation. For simplicity, we assume the same consequences for a killed or badly injured person and, on the other hand, we disregard the associated consequences of mildly injured people. • Flight diversion and cancelation. In principle, this is one of the main targets for the terrorists: disrupting air traffic as much as possible. Usually, the airline operator is responsible for passengers’ related costs in the event of a flight diversion or cancelation. However, once the incident reports and investigations have been completed, airlines can ask their insurance companies for partial or total compensation of entailed costs. The airport could be eventually found liable to refund part of these costs to the airlines. These include alternative transportation (paying the new ticket and/or refunding the original one, or any other compensation mode), extra catering and/or accommodation costs. We explicitly distinguish the consequences for flight diversions and cancelations. When cabin crew informs about an emergency or extraordinary situation, they can divert to an alternate aerodrome or fly back to the departing airport. The decision is made by the cabin crew and the company, in collaboration with air traffic management. Regarding flight cancelation, it can create a chain reaction, affecting connected flights, resulting in additional consequences for passengers, airlines and airports. Given the difficulties in assessing such consequences, see e.g. Cook et al. (2012), we shall aggregate them into a single quantity, f . It seems reasonable to assume that the consequences (in terms of costs) for airport authorities will be similar, regardless of the number of terrorists actually succeeding in their attempt to access the ATC Tower, except for the case in which only one attacker succeeds (s1 = 1), for which we will assume a lesser 12
impact. We should also take into account the inherent uncertainty on the value of f , expressed through a probability distribution pD (f |s1 ). • Image consequences. If a security incident occurs, this might essentially happen because there was a security breach. This will yield an immediate deterioration on the image of the airport as perceived by customers, even if no life or injury damage occurs. News of the crisis will be spread and amplified by the media, magnifying its impact at national or even international level. This “panic effect” is a main objective for the Attacker. We can think of different image impacts: (1) Airport security image; (2) Aircraft security and safety image; and (3) National image consequences. We shall subsume all previous effects into a single variable, g. We will use a probability distribution pD (g|a) to express our uncertainty about it, whose expected value will increase with the number a of terrorists (not only of those actually succeeding in their attempt). We summarize all relevant impacts for the Defender in Table 1. Table 1: Relevant impacts for the Defender Concept Impact Investment costs Cost of a life Flight diversion/cancelation Image
c1 x1 + c2 x2 + c3 x3 + c4 x4 + c5 x5 clife f g
We use the measurable multiattribute value function concept together with the relative risk aversion concept in Dyer and Sarin (1979, 1982) to come out with the Defender’s utility function. First, the multiattribute value function for the Defender will be described through:
cD (x, y, f, g) =
(c1 x1 + c2 x2 + c3 x3 + c4 x4 + c5 x5 ) + clife · y + f + g, c1 x1 + c2 x2 + c3 x3 + c4 x4 + c5 x5 ,
if a ≥ 1, if a = 0,
effectively monetizing consequences. We then consider that the Defender is constant risk averse with respect to cD . Thus, her utility function is (strategically equivalent to) uD (cD ) = 13
− exp(kD · cD ), with kD > 0. D Then, once she has assessed all the involved uncertainties, pD (f |s1), pD (g|a), pD 0,x , pa,x
and pD s1 ,a,x , she has to compute the expected utility of each alternative, proceeding through the following steps: 1. Compute the random expected utility ψD (x, a, s1 , y) when a ≥ 1 ψD (x, a, s1 , y) =
ZZ
uD
5 X j=1
!
cj xj + clife · y + f + g pD (f |s1 ) pD (g|a) df dg.
2. Compute the random expected utility ψD (x, a, s1 ) when a ≥ 1 ψD (x, a, s1 ) =
m X
pD y,a ψD (x, a, s1 , y),
y=0
where pD y,a = Pr(y casualties on Defender’s side|a terrorists attacking). 3. Compute the random expected utility ψD (x, a) when a ≥ 1 ψD (x, a) =
a X
pD s1 ,a,x ψD (x, a, s1 ),
s1 =0
where pD s1 ,a,x models the Defender’s beliefs about the number of those terrorists actually succeeding when the investment is x, and the number of terrorists is a. 4. Compute the random expected utility ψ¯D (x) when a ≥ 1 ψ¯D (x) =
5 X
pD a,x ψD (x, a),
a=1
where pD a,x models her beliefs about the number of terrorists performing the attack when the investment is x.
14
5. Compute the random expected utility ψD (x)
ψD (x) = pD 0,x uD
5 X
cj xj
j=1
!
+ ψ¯D (x).
All the previous steps can be aggregated in a single expression
ψD (x) =
pD 0,x uD
5 X
cj xj
j=1
×
" m X y=0
pD y,a
!
ZZ
+
5 X
pD a,x
a=1
uD
5 X j=1
×
(
a X
pD s1 ,a,x
s1 =0
!
cj xj + clife · y + f + g pD (f |s1 ) pD (g|a) df dg
#)
, (1)
where pD y,a is the probability of having y casualties on Defender’s side, given that there are a terrorists attacking; pD s1 ,a,x models the Defender’s beliefs about the number of terrorists actually succeeding when the investment is x, and the number of terrorists is a; and pD a,x models her beliefs about the number of terrorists performing the attack when the investment is x. She must then find the maximum expected utility countermeasure portfolio
max x∈D1
ψD (x).
Provided that the number of portfolios is not too large, this can be accomplished by evaluating all portfolios x ∈ D1 . Should the number of portfolios be large, we would typically proceed by simulating ψD at a few x values, fitting a regression metamodel ψˆD (x), see e.g. Kleijnen and Sargent (2000), and solving for
max x∈D1
ψˆD (x).
Note that we shall need Monte Carlo simulation to evaluate the integral in (1). Of all the elements that need to be assessed by the Defender, she will only find structural difficulties in modeling pD a,x . This will requires strategic thinking, as we describe below.
15
3.2
The Attacker’s problem
In order to come out with pD a,x , we describe now the Attacker’s problem, whose influence diagram is shown in Figure 3, together with the involved random variables and their dependencies.
Prev.
Attacker
measures
decision
x
a|x
Result
s1 , z1 |x, a
Counter-
Final
Cons.
measures
Result
attacker
d2 |s1
s2 |s1 , d2
cA |a, s1 , z1 , s2
uA
uA (cA , a)
Figure 3: Influence diagram for the Attacker’s problem. Regarding the dynamics of the Attacker, we have that: 1. The Attacker sees the security measures deployed by the Defender (x1 , x2 , x3 , x4 , x5 ). 2. The Attacker decides his attack a ∈ A. If he decides not to attack, the consequences are negligible for him. 3. In case of attacking, he observes the result of the attack, s1 , and faces his operational consequences. We model this by defining two variables, z1 and w1 , representing the number of terrorists killed and detained in the attack, with {z1 , w1 } ∈ {0, 1, . . . , a−s1 } and z1 +w1 ≤ a−s1 . The remaining terrorists, a−s1 −z1 −w1 , will manage to get away. Note that we explicitly distinguish between being killed or imprisoned (only suicide terrorists will be most likely indifferent between both possibilities, but they do not seem to be a real threat in the political context of the incumbent country. Therefore, we shall not consider here such possibility). 4. In case of a successful attack, he faces the recovery measures deployed by the Defender and its consequences. We assume that, after the intervention of the SPF, s2 terrorists will be killed, with s2 ∈ {0, 1, . . . , s1 }, and the rest, s1 − s2 , are detained. Here, d2 16
is a deterministic variable, which depends only on the outcome of the initial attack performed by the terrorists, s1 . If s1 = 0 (failed attack), the SPF is not called on, i.e., d2 = 0. Otherwise, d2 = 1, meaning that the SPF will intervene. 5. He gets the corresponding utility. The Defender considers that the relevant multiple consequences for the Attacker are: • Preparation costs. As for many other types of terrorist attacks, preparation costs are not particularly high, especially when compared with the consequences they aim at inflicting to the Defender. We shall assume a fixed cost cp for the whole operation per each involved terrorist. • Whether they are able or not to take control over air traffic operations. This is the main target for terrorists. To simplify matters, we assume that the terrorists give the same value as the Defender to image and operational consequences, i.e., f and g. We shall use also probability distributions pA (f |s1 ) and pA (g|a) to describe the Attacker’s uncertainty about such consequences, and his belief that they will depend on the number of successful and total attackers, respectively. • The number of terrorists killed or detained. We shall assume that the terrorists put a value c′life to their lives and a value cd to the fact of being detained. As before, we consider the same approach to model the Attacker’s utility function. First, we aggregate the attributes in a multiattribute value function
c=
f + g − cp − c′life · (z1 + s2 ) − cd · (w1 + s1 − s2 ), −cp ,
if a ≥ 1,
(2)
if a = 0,
and, then, assume that the Attacker is risk prone in benefits. Therefore, his utility function is strategically equivalent to
uA (c) = exp(kA · c), kA > 0. 17
The elements involved in solving the Attacker’s problem are: • (s1 , z1 , w1 , a − s1 − z1 − w1 ), is the initial result of the attack, in terms of the number of terrorists succeeding, killed, detained or escaping. We shall consider a distribution pA s1 ,z1 ,a,x = pA (s1 , z1 , w1 |a, x) to model the Attacker’s beliefs about the probabilities over the four incumbent events, given the investments x and the number of terrorists a. We assume that the more defensive resources, the less terrorists will be likely to succeed in the attack or escape after it in case they fail. Regarding technological measures (cameras, metal detectors and X-ray devices), they will only have influence on the detention of terrorists, whereas human resources (police and private security) could also cause some casualties among the attackers. • s2 , is the final result of the attack, after the intervention of the SPF. pA (s2 |s1 ) models the Attacker’s beliefs about it. The steps needed to solve the Attacker’s problem are: 1. We integrate out the uncertainty over c. We get the expected utility
ψA (a, s1 , z1 , w1 , s2 ) =
ZZ
uA (c) pA (f |s1 ) pA (g|a) df dg,
where c is defined in (2), and pA (f |s1 ) and pA (g|a) are the densities over f and g, which, in turn, induce the distribution pA (c|a, s1 , z1 , w1, s2 ). 2. We reduce the uncertainty over s2 . We get the expected utility ψA (a, s1 , z1 , w1) as
ψA (a, s1 , z1 , w1 ) =
X
s2 ∈S2
pA (s2 |s1 ) ψA (a, s1 , z1 , w1 , s2 ).
3. We eliminate the uncertainty over s1 . We get the expected utility ψA (a, x) as
ψA (a, x) =
X
s1 ,z1 ,w1 ∈S1
pA (s1 , z1 , w1|a, x) ψA (a, s1 , z1 , w1 ). 18
4. We find the optimal strategy for the Attacker by solving
ψA (x) = max ψA (a, x). a∈A
This provides a(x) = arg maxa ψA (a, x), the optimal attack level when the security investment is x. Note, however, that we have uncertainty about uA (·), pA (f |·), pA (g|·), pA (s2 |·) and pA (s1 , z1 , w1 |·), which we model through the random utilities and probabilities UA (·), PA (f |·), PA (g|·), PA (s2 |·) and PA (s1 , z1 , w1 |·). Then, we propagate such uncertainty as follows, for each x: 1. Compute the random expected utility
ΨA (a, s1 , z1 , w1, s2 ) =
ZZ
UA (c) PA (f |s1 ) PA (g|a) df dg.
2. Compute the random expected utility
ΨA (a, s1 , z1 , w1 ) =
X
s2 ∈S2
PA (s2 |s1 )ΨA (a, s1 , z1 , w1 , s2 ).
3. Compute the random expected utility
ΨA (a, x) =
X
s1 ,z1 ,w1 ∈S1
PA (s1 , z1 , w1 |a, x)ΨA (a, s1 , z1 , w1 ).
4. Compute the random optimal alternative
A(x) = arg max ΨA (a, x). a∈A
D Then, we would have that the desired distribution pD a,x in (1) would be pa,x = Pr(A(x) = a).
In order to estimate it, we may proceed by simulation as follows, where K is the Monte Carlo sample size: 19
Algorithm 1: Simulating the optimal attack level For each x For i = 1 to K Sample UiA (·), PiA (f|·), PiA (g|·), PiA (s2 |·), PiA(s1 , z1 , w1 |·). Compute ΨiA (a, s1 , z1 , w1 , s2 )
=
ZZ
UiA (c) PiA (f|s1 ) PiA (g|a) df dg.
Compute ΨiA (a, s1 , z1 , w1 ) =
X
s2 ∈S2
PiA (s2 |s1 )ΨiA (a, s1 , z1 , w1 , s2 ).
Compute ΨiA (a, x) =
X
s1 ,z1 ,w1 ∈S1
PiA (s1 , z1 , w1 |a, x)ΨiA (a, s1 , z1 , w1 ).
Compute the random optimal alternative
Ai = arg maxa ΨiA (a, x).
Finally, we approximate Pr(A(x) = a) ≈ #{1 ≤ i ≤ K : Ai = a}/K.
4
A Case Study
We consider the case of a small-size international airport. In what follows, we shall express all the involved costs in euros, as it is the relevant currency in the incumbent case. It has an average annual budget of 3 million euros, with around 5% of the total budget, 150,000 euros, to be invested in new security measures on top of the current ones. We first discuss issues related with the Defender’s problem. Then, we give details related to the point of view of the Attacker. The assessments were made with the aid of the experts of the incumbent airports, later validated in an airport security expert workshop and checked for robustness through sensitivity analysis.
20
4.1
Defender’s assessments
According to the airport authorities, they are considering the maximum planned investments in security resources summarized in Table 2. We have also included the qualitative deterrent and detection rates of these measures, as assessed by the airport authorities.
Measure
Table 2: Maximum planned investments in security measures Max Annual cost (e)/unit Deterrence Detection
Cameras Metal detectors X-ray devices Police Private security
4 1 1 5 9
650 6,500 90,000 19,200 15,600
Moderate-high Moderate Moderate High High
Moderate (persons) High (material) High (material) High (persons) Moderate (persons)
We assessed that the number of casualties on the Defender’s side follows a binomial distribution y ∼ Bin(m, pd ) with a small probability pd (indiscriminate killing does not seem to be a target for terrorist in our scenario and, therefore, few casualties are to be expected on the Defender’s side), which will depend, in turn, on the number of terrorists a. We use expert judgement to elicit the value of pd . We start with pd = 0.005 · a. As an illustration, if we set the number of persons on the Defender’s side present at the moment of the attack to be m = 100, the expected number of casualties during an attack would vary between 0.5 (when only one terrorist performs the attack) and 2.5 (when the cell is composed of five terrorists). As far as quantifying the value of a human life, we shall use the statistical value of life (adapted to the country in our scenario), estimating it in 2 million euros for the Defender, see Viscusi and Aldy (2003) for a review on the topic. We specify now pD (f |s1 ). We use a truncated normal distribution with a mean value µf dependent on the number of attackers succeeding to access the Tower. If s1 = 1, we assume a smaller impact than when s1 ≥ 2. This seems reasonable, since if there is more than one terrorist taking hold of the ATC Tower, it will be more likely that they will be able to affect, for longer and with more severe consequences, air traffic operations than if only one of them is able to get into the ATC Tower. Besides, we also take into account different scenarios
21
for the potential damages caused by terrorists, depending on the air traffic complexity and density during the attack period. Upon discussion with experts, we considered three possible scenarios, which are representative of the usual activity at the incumbent airport: • Low traffic level (L): one international flight and four training local flights simultaneously. • Medium traffic level (M): two international flights, one domestic flight and six training local flights simultaneously. • High traffic level (H): four international flights, two domestic flights and eight training local flights simultaneously. With all these elements in mind, we assume that the flight diversion and consequences costs follow a truncated normal distribution
f ∼ T N µf |s1 , σf2 , with different expected values depending on s1 and on the traffic level, which have been assessed through expert elicitation as shown in Table 3. We have also indicated, in parentheses, the corresponding standard deviation σf , as assessed by our experts. Table 3: Expected cancelation/diversion costs (s1 ≥ 1) L M H s1 = 1 5 · 104 (8 · 104 ) 105 (105 ) s1 ≥ 2 105 (105 ) 2 · 105 (2 · 105 )
2 · 105 (2 · 105 ) 4 · 105 (3 · 105 )
A similar reasoning can be applied to the model for image costs, which are assumed to follow a truncated normal distribution
g ∼ T N µg , σg2 = 108 . However, in this case, the entailed consequences depend on the total number of terrorists 22
initially launching the attack, a, because, even if none of them succeeds in the attack, their attempt will still have some impact on the airport image. The expected costs, which have been again elicited with the aid of experts, are shown in Table 4. As we can observe, the influence of each additional successful terrorist is mitigated. The variance σg2 has been considered equal for the three scenarios of concern. Table 4: Expected image costs (a ≥ 1) L M H √ √ √ µg |a 105 · a 1.5 · 105 · a 2 · 105 · a Regarding the number of terrorists succeeding, killed, detained or escaping on the first stage of the attack, we shall consider a multinomial distribution, (s1 , z1 , w1 , a−s1 −z1 −w1 ) ∼ P M(a; δ1′ , δ2′ , δ3′ , δ4′ ), with δi′ = δi /δs , i = 1, 2, 3, 4, being δs = 4i=1 δi . Using expert opinion, we adjusted the following expressions for δ1 , δ2 , δ3 , δ4 :
δ1 = δ2 = δ3 = δ4 =
P 5 γ1,r · exp − j=1 γ1,j xj , h i γ2,r 1 − exp −γ2,4 x4 − γ2,5x5 , P 1 − exp − 5j=1 γ3,j xj , P exp − 5j=1 γ4,j xj ,
which account for the fact that each additional unit of (x1 , x2 , x3 , x4 , x5 ) is expected to reduce the number of successful and escaped terrorists, and increase the number of terrorists killed or detained. We have assessed, with the aid of experts, the following values for the incumbent parameters, based on the qualitative values shown in the last two columns of Table 2. They reflect the expected impact of each additional resource on the outcome of the attack, and have been chosen so that the δ’s are always positive: • γ1,1 = 0.1, γ1,2 = 0.15, γ1,3 = 0.25, γ1,4 = 0.4, γ1,5 = 0.2; γ1,r = 0.5. • γ2,4 = 0.3, γ2,5 = 0.1; γ2,r = 0.3. 23
• γ3,1 = 0.1, γ3,2 = 0.2, γ3,3 = 0.25, γ3,4 = 0.4, γ3,5 = 0.25. • γ4,1 = 0.15, γ4,2 = 0.2, γ4,3 = 0.4, γ4,4 = 0.45, γ4,5 = 0.3. Specifically, we asked the experts about the expected deterrent effect of each countermeasure when considered separately, fitting the expression for δi , i = 1, . . . , 4 and obtaining the corresponding γ’s. As an example, let us consider the expression for δ3 , assuming that the only countermeasure available is private security, x5 . δ3 is the parameter of the multinomial distribution related with the number of detained terrorists after the first attack. When x5 = 0, i.e., if there is no investment, we cannot expect more arrests (δ3 = 0). On the other hand, the experts considered that hiring one security guard is expected to imply the detention of one terrorist (for a terrorist cell of five members)1. With this value, we fitted γ35 = 0.25. We checked for consistency of the assessment, asking the experts about the expected increase in the detention rate if more than one security guard were to be hired, obtaining consistent results. Similar reasonings were used to assess the rest of the involved parameters.
The
value of γ1,r expresses the operator’s belief that the number of successful terrorists is expected to be approximately half the number of those being able to escape when no resources are deployed. A similar interpretation holds for γ2,r , in that the number of killed terrorists is expected to be approximately one third of the number of those being detained if all the available resources were deployed. To further illustrate these values, let us consider the number of police members, x4 , as if they were the only available preventive measure for the Defender. Using the values of the γ·,4’s, γ1,r and γ2,r stated above, and assuming that a = 5, the expected number of succeeding, killed, detained or escaping terrorists for the different values of x4 are shown in Table 5. As we can observe, the contribution of each additional police member is mitigated. Finally, we have assessed, with the aid of experts, a value kD = 0.02 for the risk aversion parameter. We have performed a sensitivity analysis for kD , suggesting robustness. 1
Recall that the expected values of a multinomial distribution (n1 , . . . , nk ) ∼ M(N ; δ1 , . . . , δk ) are E[ni ] = N · δi , i = 1, . . . , k.
24
Table 5: Influence of the number of police in the expected outcome of the attack x4 0 1 2 3 4 5 Successful Killed Detained Escaped
4.2
1.67 0.00 0.00 3.33
1.21 0.28 1.19 2.31
0.85 0.51 2.09 1.54
0.59 0.69 2.72 1.01
0.40 0.82 3.13 0.65
0.27 0.92 3.40 0.41
Attacker’s assessments
Regarding the probability distribution that the Attacker shall use to describe his uncertainty about f |s1 , we also use a truncated normal distribution with the same expected value than the Defender, although with a variance ten times higher. Similarly, we assume that the probability distribution that the Attacker uses to describe his uncertainty about g|a is a truncated normal distribution with the same expected value as the Defender, and a variance ten times higher. We shall also assume a binomial distribution for the number of terrorists killed after the intervention of the SPF, which will depend on the number of successful terrorists on the first attack, s1 , provided that s1 ≥ 1, s2 ∼ Bin(s1 , pt ). Since the deployment of the SPF is mandatory, its influence on the number of terrorists killed at this second stage will be similar regardless of any other consideration (defensive measures initially deployed by the Defender, number of terrorists attacking, etc). According to experts’ opinion, there is a probability of 10% that at least one terrorist will be killed during the recovery action, irrespective of the actual number of them occupying the ATC Tower. This corresponds to an approximate value of pt ≃ 0.05. This estimation is subject to uncertainty, although we shall neglect it, since it will not affect significantly the final results. Regarding the value of a terrorist life, we estimate at about 200,000 euros, whereas we use a value of 100,000 euros if the terrorist is imprisoned. These values seem reasonable since terrorists usually assess their lives at a much less value than those people on the Defender’s side, see e.g. Viscusi (2009). However, unless they are suicide terrorists (which is not the 25
case in our problem), they will still prefer to be imprisoned rather than killed in case of a failed attack. Nevertheless, the expectation of a long prison term in case they are captured implies little differences between the two values. Regarding the preparation costs for the Attacker, we assume a fixed cost of 20,000 euros per involved terrorist, which may account for the need of being armed with weapons and/or trained as ATCOs, as well as for the time spent on gathering the necessary intelligence in order to launch a successful attack. Note that we do not take into account uncertainty over such costs. Finally, we assume a random utility model for the Attacker
UA (c) = exp(kA · c), kA ∼ U(0, KA ). The Defender thinks that the parameter kA that determines the Attacker’s utility function, takes a maximum value KA = 5.
4.3
Results
Based on Table 2, we have 495 feasible portfolios. We use K = 10000 in Algorithm 1, for each of these portfolios. We have first chosen the scenario in which the traffic level is low. As discussed, for each portfolio x, we have obtained as a result an empirical distribution of the probability that the Attacker would choose an attack a ∈ A. Once with the estimation of pD a,x for each x, we have solved the Defender’s problem, finding the optimal portfolio of preventive measures which maximizes the Defender’s expected utility. Figure 4 shows the Defender’s estimated expected utility for all possible portfolios x. From left to right, the portfolios on the horizontal axis begin with x = (0, 0, 0, 0, 0), x = (0, 0, 0, 0, 1) and so on, sequentially increasing the values in x5 , x4 , x3 , x2 and x1 , and finishing with x = (4, 1, 1, 2, 0). The optimal portfolio (4, 1, 0, 5, 1) is highlighted with a vertical dashed line. We show in Table 6 the estimated probabilities pD a,x for some representative portfolios, together with their corresponding investments. In the first row, we have included the optimal 26
−1.09
Expected utility
−1.092 −1.094 −1.096 −1.098 −1.1 −1.102 −1.104
0
50
100
150
200
250 300 Portfolio
350
400
450
500
Figure 4: Estimated expected utility for the Defender. portfolio, for which the Defender attains her maximum expected utility, as we comment below. Besides, we have also included those portfolios in which just one of the preventive measures attains its maximum allowable value, with no investment in the other measures. For instance, the second row of Table 6 corresponds to a portfolio in which airport authorities would have only invested in cameras, acquiring the maximum allowable number of them (4). Finally, to complete our analysis, we have also considered the five portfolios which entailed the highest investments. With this, we aim at investigating whether the highest investments are necessarily the most effective ones, in terms of the Defender’s multiattribute value function. The optimal portfolio for the Defender corresponds to four cameras, one metal detector, five policemen and one security member, with an associated investment of 120,700 euros. The estimated probabilities pD a,x for this portfolio were (0.61, 0.25, 0.07, 0.04, 0.02, 0.01), which have the following interpretation. Should the operator choose this optimal portfolio, it would be highly likely that the terrorists decide not to attack (61%), or that they just launch a low-profile attack, with only one terrorist (25%). Attacking with two or more terrorists is not regarded as such a worthy option for the Attacker. Note that the optimal portfolio does not exhaust the available budget. In this sense, we should mention that the optimal portfolio implies maximum investments in three of the five resources: cameras, metal detector and
27
Table 6: Estimated probabilities for some representative portfolios pa,x x Investment 0 1 2 3 4 5 (4, 1, 0, 5, 1) (4, 0, 0, 0, 0) (0, 1, 0, 0, 0) (0, 0, 1, 0, 0) (0, 0, 0, 5, 0) (0, 0, 0, 0, 9) (4, 1, 1, 1, 2) (4, 1, 0, 0, 9) (3, 0, 1, 3, 0) (3, 0, 0, 2, 7) (1, 1, 0, 5, 3)
120700 2600 6500 90000 96000 140400 149500 149500 149550 149550 149950
0.61 0 0.01 0.01 0.45 0.48 0.30 0.41 0.35 0.54 0.48
0.25 0.13 0.06 0.13 0.27 0.29 0.36 0.36 0.30 0.31 0.36
0.07 0.26 0.20 0.17 0.11 0.08 0.13 0.11 0.16 0.08 0.07
0.04 0.18 0.21 0.21 0.10 0.09 0.09 0.07 0.12 0.03 0.07
0.02 0.31 0.21 0.21 0.05 0.05 0.08 0.03 0.06 0.03 0.01
0.01 0.12 0.31 0.27 0.02 0.01 0.04 0.02 0.01 0.01 0.01
police. On the contrary, there would be no investment in the most expensive resource, the X-ray device. This seems reasonable, since its higher efficiency, relative to the other related resources (e.g. metal detectors; compare the values of γ·,2 ’s and γ·,3’s), is beaten by its comparatively much higher costs. A similar reasoning holds for the investment in private security: only one of the maximum nine allowable units would be hired. Although their salaries are lower than those of a police member, this is not compensated by the fact that they are less efficient than the police (compare the values of γ·,4’s and γ·,5’s). With regards to the other portfolios displayed in Table 6, it is interesting to note that those including high numbers of police and/or private security members imply higher probabilities of not being attacked by the terrorists. On the other hand, investing mainly in technological resources does not seem to bring such good results. The most extreme cases are portfolios (4, 0, 0, 0, 0), (0, 1, 0, 0, 0) and (0, 0, 1, 0, 0), for which terrorists would be prone to attack with several members. This is especially symptomatic in the case when the operator would only invest in the X-ray device: in spite of the high costs, the deterrent effect seems quite limited. The above results are sensitive to changes in the estimation of the entailed consequences to the airport in case of a successful attack. In this regard, we have repeated our calculations
28
considering the two other scenarios regarding the traffic level. For the high traffic level scenario, the optimal portfolio was (4, 1, 0, 4, 4), corresponding to four cameras, one metal detector, four policemen and four security members. It has an associated investment of 148,300 euros, and an estimated probability pD a,x = (0.09, 0.47, 0.17, 0.13, 0.09, 0.05) for a = , 0, 1, . . . , 5. As we can observe, under this new scenario of higher expected losses, the operator would opt for a more expensive portfolio, exhausting almost entirely the available budget. Again, it becomes clear that it is worthier for the operator to invest in human resources than in expensive technological measures. Finally, we briefly comment the case of medium traffic level. The optimal portfolio was (4, 1, 0, 5, 2), corresponding to four cameras, one metal detector, five policemen and two security members. It has an associated investment of 136,300 euros, and an estimated probability pD a,x = (0.48, 0.36, 0.08, 0.02, 0.04, 0.01) for a =, 0, 1, . . . , 5. However, our aim is to provide airport authorities with a unique security plan, which should be adequate for all day periods. Analyzing the three proposed scenarios and their associates optimal portfolios, we conclude that, in order to protect the airport in the best manner throughout the whole day, the optimal alternative would be to implement the optimal portfolio for the medium traffic level scenario, i.e., x = (4, 1, 0, 5, 2), which entails a moderate investment. Summarizing, we have observed the following trends in the Attacker’s behavior after performing our analysis. Under the scenario of an airport which will incur in big losses if a terrorist attack occurs, the terrorists would behave in the following manner: (1) They tend to be cautious when they see that the defensive measures are too intense, typically choosing attacking with, at most, only one terrorist; (2) Otherwise, if they feel that the ATC Tower is vulnerable, they would launch the most powerful attack they can; and (3) Only in case of doubt, when they do not perceive with clarity any of the situations mentioned above, they would opt for an intermediate strategy, sending between two to four attackers. However, should the terrorists feel that the damages inflicted to the airport will not be so considerable, their strategy would change radically. Although they are considered as risk seekers, they also 29
put a certain value to their lives and, therefore, they will not put themselves in unnecessary risk if the chances of causing spread and costly damages to airport authorities are reduced. In the light of our analysis, it is clear that not always the most expensive measures are the most appropriate ones for a given situation. Furthermore, it is also important to remark that experts’ opinion should be always taken into account when devising security plans.
5
Discussion
We have analyzed the case study of an airport, threatened by the possibility of a terrorist attack aimed at taking control over the ATC Tower. The operator has a set of feasible portfolios of preventive measures she could invest in. The Attacker would decide to attack or not with a given power depending on the preventive measures deployed by the operator. In case of a successful attack, the operator would call on a Special Police Force, who will be in charge of the situation, trying to recover the ATC Tower as soon as possible. In devising a suitable model for this problem, we have taken into account all relevant consequences for the operator, in terms of lives lost and operations and image consequences, incorporating also the inherent uncertainty. In a similar way, we have assessed the consequences for the Attacker, in terms of preparation costs, lives lost or possibility of being imprisoned, and the revenue they aim at obtaining: disrupting air traffic and bringing about, as much as possible, economical and political damage to airport authorities and governments. We have also taken into account uncertainty on these quantities. We have addressed the problem as a particular case of a Sequential Defend-Attack model within the Adversarial Risk Analysis framework. The final aim of the model was to give advise to the Defender for devising a security plan. In this regard, we have highlighted the need of assessing in a precise way the efficiency of all available resources before making a decision on where to invest. The proposed methodology may be used for other cases related to airport security, as e.g. unlawful interference with apron, airside and/or security checks, cyber-attacks to the ATC or bioterrorism, among others. It could be also adapted to deal with similar security
30
problems involving critical infrastructures, or alternative transportation means. We have only considered a single site to protect, but the extension to multiple site protection is relevant in applications. In this case, additional constraints should be placed on the available portfolios of countermeasures, since some of them might have to be be shared among the different sites.
Acknowledgements This project has received funding from the European Union’s Seventh Framework Programme for Research, Technological Development and Demonstration under grant agreement no 285223. Work has been also supported by the Spanish Ministry of Economy and Innovation programs MTM2011-28983-C03-01 and IPT-2012-0912-430000, the Government of Madrid RIESGOS-CM program S2009/ESP-1685 and the AXA-ICMAT Chair on Adversarial Risk Analysis. We are grateful to airport experts and stakeholders for fruitful discussions about modeling issues. This work was completed while the first author was visiting Uppsala University, supported by a grant from URJC’s postdoctoral program
References G. S. Becker. Crime and punishment: An economic approach. Journal of Political Economy, 76(2):169–217, 1968. A. Cook, G. Tanner, and A. Lawes. The hidden cost of airline unpunctuality. Journal of Transport Economics and Policy, 46(2):157–173, 2012. D. B. Cornish and R. V. Clarke. The Reasoning Criminal: Rational Choice Perspectives on Offending. Reprint 2011. Research in Criminology. Springer, London, 1986. J. S. Dyer and R. K. Sarin. Measurable multiattribute value functions. Operations Research, 27(4):810–822, 1979.
31
J. S. Dyer and R. K. Sarin. Relative risk aversion. Management Science, 28(8):875–886, 1982. M. R. Haberfeld and A. von Hassell. A New Understanding of Terrorism: Case Studies, Trajectories and Lessons Learned. Humanities, Social Sciences and Law. Springer, 2009. J. P. C. Kleijnen and R. G. Sargent. A methodology for fitting and validating metamodels in simulation. European Journal of Operational Research, 120(1):14–29, 2000. B. Lomborg. Solutions for the World’s Biggest Problems. Costs and Benefits. Cambridge University Press, 2008. J. Merrick and G. S. Parnell. A comparative analysis of PRA and intelligent adversary methods for counterterrorism risk management. Risk Analysis, 31(9):1488–1510, 2011. J. Pearl. Influence diagrams—Historical and personal perspectives. Decision Analysis, 2(4): 232–234, 2005. J. R´ıos and D. R´ıos Insua. Adversarial risk analysis for counterterrorism modeling. Risk Analysis, 32(5):894–915, 2012. D. R´ıos Insua, J. R´ıos, and D. Banks. Adversarial risk analysis. Journal of the American Statistical Association, 104(486):841–854, 2009. W. K. Viscusi. Valuing risks of death from terrorism and natural disasters. Journal of Risk and Uncertainty, 38(3):191–213, 2009. W. K. Viscusi and J. E. Aldy. The value of a statistical life: a critical review of market estimates throughout the world. Journal of Risk and Uncertainty, 27(1):5–76, 2003.
32
