Quantifying security threats and their potential ... - Semantic Scholar

Innovations Syst Softw Eng DOI 10.1007/s11334-010-0123-2

ORIGINAL PAPER

Quantifying security threats and their potential impacts: a case study Anis Ben Aissa · Robert K. Abercrombie · Frederick T. Sheldon · Ali Mili

Received: 6 June 2009 / Accepted: 17 February 2010 © Springer-Verlag London Limited 2010

Abstract In earlier works we presented a computational infrastructure that allows an analyst to estimate the security of a system in terms of the loss that each stakeholder stands to sustain as a result of security breakdowns. In this paper we illustrate this infrastructure by means of an e-commerce application. Keywords Cyber security metrics · Risk management · Information security

1 Introduction Abercrombie et al. [1,2] present an infrastructure that allows an analyst to estimate the security of a system in terms of the loss that each stakeholder stands to sustain as a result of security breakdowns. The infrastructure in question reflects the values that stakeholders have in each security requirement, the dependency of security requirements on the operation of architectural components, the impact that security threats A. Ben Aissa (B) Faculty of Sciences of Tunisia, University of Tunis, Tunis, Tunisia e-mail: [email protected] R. K. Abercrombie · F. T. Sheldon Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA e-mail: [email protected] F. T. Sheldon e-mail: [email protected] A. Mili College of Computing Sciences, New Jersey Institute of Technology, Newark, NJ 07102-1982, USA e-mail: [email protected]

have on the proper operation of security components, as well as the threat configuration that looms on the operation of the system. This latter feature is the security equivalent of a fault model in reliability modeling. We illustrate the proposed quantitative model using a simple example, consisting of an e-commerce application. We illustrate each stakeholder can determine their stake in the secure operation of the system and ensure their individual mission fulfillment. We consider the following criteria: • The threat configuration under which the application operates, which is represented by a vector of probabilities of occurrence of security breakdowns for a given duration of operation. • The impact that security breakdowns have on the proper operation of individual components of the architecture (depending on which part of the system each threat targets). • The dependency that exists between architectural components and security requirements; this relation reflects how/to what extent each component of the system contributes to meeting each security requirement. • The stakes that each stakeholder has in meeting each clause of the security requirements specification. In addition to analyzing the e-commerce scenario to quantify security threats and their impact, we discuss how and by whom each aspect of this quantitative model is derived. In this way we can assess the cost effectiveness of security measures, and judiciously allocate the cost of implementing said security measures on the appropriate benefactor. We build upon the quantitative model of Abercrombie et al. [1,2], and provide a rationale for it (using probability theory) as compared to other methods [1,3] Sect. 2. In Sects. 3, 4, and 5, we discuss in turn the stakes matrix, the dependency matrix, and the impact matrix; Sect. 6 is an

123

A. Ben Aissa et al.

analysis and a discussion of the threat configuration. Section 7 is an application of MFC and we finish in Sect. 8 by a redistribution balanced of the ROI.

2 A cascade of linear models

• Em+1: No component is affected. Given a set of complementary events E1, E2, E3, …, Eh, Eh + 1, we know that the probability of an event F can be written in terms of conditional probabilities as:

P(F) =

1≤ j≤n

If we let MFC be the column-vector of size k that represents mean failure costs, let ST be the k × n matrix that represents stakes, and let PR be the column-vector of size n that represents probabilities of failing security requirements, then this can be written using the matrix product (◦): MFC = ST ◦ PR. The stakes matrix is filled, row by row, by the corresponding stakeholders. As for PR, we discuss below how to generate it. 2.2 The dependency matrix We consider the architecture of system S, and let C1, C2, C3, . . .,Ch, be the components of system S. Whether a particular security requirement is met or not may conceivably depend on which component of the system architecture is operational. If we assume that no more than one component of the architecture may fail at any time, and define the following events: • Ei, 1 ≤ i ≤ h, is the event: the operation of component Ci is affected due to a security breakdown.

123

P(F|E k ) × P(E k ).

k=1

2.1 The stakes matrix We consider a system S and we let H1, H2, H3, … Hk, be stakeholders of the system, i.e., parties that have a stake in its operation. We let R1, R2, R3, … Rn, be security requirements that we wish to impose on the system, and we let STi,j, for 1 ≤ i ≤ k and 1 ≤ j ≤ n be the stake that stakeholder Hi has in meeting security requirement Rj. We let PRj, for 1 ≤ j ≤ n, be the probability that the system fails to meet security requirement Rj, and we let MFCi (mean failure cost), for 1 ≤ i ≤ k, be the random variable that represents the cost to stakeholder Hi that may result from a security failure. We quantify this random variable in terms of financial loss per unit of operation time (e.g. $/h); it represents the loss of service that the stakeholder may experience as a result of a security failure. Under some assumptions of statistical independence, we find that the mean failure cost for stakeholder Ti can be written as: STi, j × PR j . MFCi =

h+1

We instantiate this formula with F being the event: the system fails with respect to some security requirement. To this effect, we let Fj denote the event that the system fails with respect to requirement Rj and we write (given that the probability of failure with respect to Rj is denoted by PRj):

PR j =

m+1

P(F j |E k ) × P(E k ).

k=1

If • we introduce the Dependency (DP) matrix, which has n rows and h + 1 columns, and where the entry at row j and column k is the probability that the system fails with respect to security requirement j given that component k has failed (or, for k = h + 1, that no component has failed), • we introduce vector PE of size h + 1, such that PEk is the probability of event Ek, then we can write PR = DP ◦ PE. Matrix DP can be derived by the system’s architect, in light of the role that each component of the architecture plays to achieve each security goal. As for deriving vector PE, we discuss this matter in the next section. 2.3 The impact matrix Components of the architecture may fail to operate properly as a result of security breakdowns brought about by malicious activity. In order to continue the analysis, we must specify the catalog of threats that we are dealing with, in the same way that analysts of a system’s reliability define a fault model. To this effect, we catalog the set of security threats that we are facing, and we let T1, T2, T3, …, Tp, represent the event that a catalogued threat has materialized, and we let T p + 1, be the event that no threat has materialized. Also, we let PT be the vector of size p + 1 such that: • PTq, for 1 ≤ q ≤ p, is the probability that threat Tq has materialized during a unitary period of operation (say, 1 h).

Quantifying security threats

• PTp+1 is the probability that no threat has materialized during a unitary period of operation time.

P(E k |Tq ) × PTq .

q=1

…Rj…

Rn

H1

R1

Stake that stakeholders Hi has in meeting requirement Rj

…Hi…

PEk =

p+1

Stakeholders

Then, by virtue of the probabilistic identity cited above, we can write:

Requirements

ST

Hm

If

R1

Ch+1

Prob of failing requirement Ri. once component Ck has failed

Threats

IM

…Tq…

Tp+1

C1

T1

Prob that Component Ck fails once threat Tq has materialized

Ch+1

Components

Given the stakes matrix ST, the dependability matrix DP, the impact matrix IM and the threat vector PT, we can derive the vector of mean failure costs (one entry per stakeholder) by the following formula:

…Ck…

Rn

Matrix IM can be derived by analyzing which threats affect which components, and assessing the likelihood of success of each threat, in light of perpetrator behavior and possible counter-measures. Vector PT can be derived from known perpetrator behavior, perpetrator models, known system vulnerabilities, etc. We refer to this vector as the Threat Configuration Vector or simply as the Threat Vector. 2.4 Summary

C1

…Ri…

PE = IM ◦ PT

Components

DP

Requirements

• We introduce the Impact (IM) matrix, which has h + 1 rows and p + 1 columns, and where the entry at row k and column q is the probability that component Ck fails given that threat q has materialized (or, for q = p + 1, that no threat has materialized), • We introduce vector PT of size p + 1, such that PTq is the probability of event Tq, then we can write

PT

…Tq….

Tq Prob that threat materializes during unitary period of operation

Tp+1

Where matrix ST is derived collectively by the stakeholders, matrix DP is derived by the system’s architect, matrix IM is derived by the security analyst from architectural information, and vector PT is derived by the security analyst from perpetrator models. The figure below illustrates these matrices and their attributes (size, content, indexing, etc).

Threats

T1

MFC = ST ◦ DP ◦ IM ◦ PT

123

A. Ben Aissa et al. Table 1 Stakes (ST) matrix: cost of failing a security requirement stakes in $/h

Security requirements Confidentiality

Integrity

Availability

Non-repudiation

Authenticity

Privacy

Stake holders Customer

10

5

3

4

6

Merchant

120

70

140

110

105

6

Tech Int

20

20

40

20

30

20

Fin Int

20

60

50

40

40

60

3 Illustration: an e-commerce application We illustrate the use of our Cyber Security Econometrics Model [1,2] on a practical application, namely an e-commerce system. We derive in turn the three matrices of interest, starting with the stakes matrix. To this effect, we first identify the security requirements [4], then the stakeholders and their stakes in meeting these requirements. 3.1 Security requirements We identify the following security requirements for this application [5,6]: • Confidentiality. Need to ensure that data are accessible only to those who are authorized to view them [7]. • Integrity. Need to ensure that information that is displayed or transmitted had not been altered by unauthorized parties. • Availability. Need to ensure that the e-commerce application is operational whenever a customer needs to use it. • Non-repudiation. Need to ensure that no party in an operation can deny participating in the operation. • Authenticity. Need to ensure that all actors in a system are properly authenticated, and their privileges and responsibilities defined accordingly. • Privacy. Need to ensure that information pertaining to system users is not improperly divulged. 3.2 Stakes and stakeholders: the stakes matrix We recognize four stakeholders in this application, namely, the customer, the merchant, the technical intermediary, and the financial intermediary. We briefly review the stakes that they have in meeting the security requirements, as these determine the corresponding values in the stakes matrix. • The customer. The stakes that the customer has in the secure operation of the system include: the loss of confidential information, which the customer may provide during the e-commerce transaction; transaction failure; and identity theft.

123

12

• The merchant. The stakes that the merchant has in the secure operation of the system include: the loss of business that may result from failing the availability requirement; the loss of customer loyalty that may result from failing the availability requirement; the loss of customer loyalty that may result from failing the confidentiality or the privacy requirements; the loss of business that may result from failing the integrity requirement, etc. • The technical intermediary. The stakes that the technical intermediary has in the secure operation of the system include: the loss of business from the merchant; the loss of reputation for good service, which may result in lost corporate value. • The financial intermediary. The stakes that the financial intermediary has in the secure operation of the system include: financial losses that result from malicious activities by customers; the loss of business from the merchant; and the loss of reputation for good service, which may result in lost corporate value. Based on a quantification of these stakes in terms of dollars per hours of operation (under the hypothesis that the system fails to meet each security requirement), we produce the following stakes matrix as shown in Table 1.

4 The dependability matrix The dependability matrix represents how (to what extent) security requirements are dependent on the proper operation of system components. In order to derive this matrix, we must first look at the architecture of the application. We adopt the following tiered architecture, borrowed from [8,9]. We review in turn several of the tiered components of this architecture: Web browser, proxy servers, Web servers, application servers, and database servers. 4.1 Tiered components of dependability matrix The following sections (3.1.1–3.1.5) follow the logic developed by Ahmed for the eBay e-commerce scalability scenario [9].


4.1.1 Web browser

4.2 Generation of the dependency matrix

The end user typically interacts with the website through a Web browser. Web browsers support user interface modifiability in a wide variety of ways, as the user interface that the browser supports is not hardwired but it is specified via HTML.

The question we address in this section is how to estimate the probability that a particular security requirement is violated in the course of operating the system for some period of time [1,2,11]. The idea that we pursue here is to link the probability of failing a particular requirement with the probability of failure of a component of the system. The elucidation of this probabilistic link involves an analysis of the system’s architecture, to determine which component contributes to meeting which requirement. Assuming that separate commercial components of the same type play interchangeable roles, we do not need to represent individual components in the dependability matrix; it suffices to represent general families of components. Hence we must consider the following (families of) components:

4.1.2 Proxy servers Request from individual browsers may first arrive at a proxy server, which exists to improve the performance of the Web-based system. The proxy server cache frequently accessed Web pages so that users may retrieve them without having to access the main website [9]. However, if a user chooses a particular item, with the intention of bidding or selling, then he must be shown real-time data. These proxy servers are typically located close to the users, often on the same intranetwork, thus saving a tremendous amount of communication and computation resources. 4.1.3 Web servers The HTTP or HTTPS request reaches the Web server. The Web servers are multithreaded, utilizing a pool of threads, each of which can be dispatched to handle an incoming request. A multithreaded server is less susceptible to bottlenecks (and hence long latency) when a number of longrunning HTTP or HTTPS requests (such as credit card validation) arrive because other threads in the pool are still available to serve incoming requests [9,10]. This introduces concurrency at the Web server level. Upon analyzing the request, the Web server sends it to an application server that responds using the service of a database. 4.1.4 Application servers From the Web server the request is forwarded to an application server. These application servers run in the middle business rules and application architecture as illustrated in the figure above. These servers implement business logic and connectivity, which dictate how clients and servers interact. This allows the databases to concentrate on the storage, retrieval, and analysis of data without worrying about precisely how that data will be used. 4.1.5 Database servers Finally, the request for service arrives at the database, where it is converted into an instruction to add, modify, or retrieve information. The relation database management system (RDBMS) must be able to support all incoming requests from the application servers.

• • • • • • •

Browser, Proxy server, Router/Firewall, Load balancer, Web server, Application server, and Database server.

Assuming no more than one component fails at a time, and considering the additional event that no component has failed, the dependability matrix has (7 + 1 =) 8 columns and 6 rows (one for each security requirement), for a total of 48 entries. We cannot comment on all 48 entries, but will give below a sample of the reasoning that goes into filling the dependability matrix; those values on which we comment are represented in Table 2. • If no component fails, then (presumably) all security requirements are satisfied. • If one of the database components fails, then this does not affect the availability of the system (since according to our hypothesis, the other database server is necessarily operational); loss of a database server may affect response time, but not necessarily availability. • Assuming confidential information is stored in only one database (for enhanced protection), then failure of a database server causes a failure with respect to confidentiality, authentication and privacy with probability 0.5. • If a browser fails then availability is not satisfied. • If a Proxy server fails, then availability is not satisfied. • If the Router/Firewall fails, then no dimension of security is satisfied. • If a Web server fails then all the dimensions of security have probability 0.33 to fail (all the queries that are routed to that server lead to unpredictable outcomes). • If the router is assumed to check when a Web server fails, then these probabilities would be 0.0.

123

A. Ben Aissa et al. Table 2 Dependency (DP) matrix: links requirements with components

Components Browser

Proxy server

Router/ firewall

Load balancer

Web server

Appl. server

Database server

No failure

0.2

1.0

1.0

0.333

0.333

0.5

0.0

Security requirements Conf

0.2

Int

0.2

0.2

1.0

1.0

0.333

0.333

0.0

0.0

Avail

1.0

1.0

1.0

1.0

0.333

0.333

0.0

0.0

NR

0.2

0.2

1.0

1.0

0.333

0.333

0.0

0.0

Auth

0.2

0.2

1.0

1.0

0.333

0.333

0.5

0.0

Priv

0.2

0.2

1.0

1.0

0.333

0.333

0.5

0.0

5 The impact matrix

5.3 Threats on the information

The impact matrix relates component failures to security threats; specifically, it represents the probability of failure of components given that some security threat (from a precatalogued set) has materialized. The first step in deriving the impact matrix is, of course, the derivation of the set of threats that we wish to consider; this is akin to defining a fault model (including a set of possible faults) in the analysis of the reliability of a system.

This last threat type can be used to exploit brand recognition for profit. Another example would deface (i.e., introduce false information) a site to maliciously affect the brand image of a company. In general, attack forms include:

5.1 Threats on communication protocols This threat category exploits the weaknesses in the basic Internet protocols such as TCP/IP, HTTP, and FTP. The main attack modes include: • Attacks to disable Web services (e.g., DoS, Denial of Information [DoI]), • Eavesdropping (man-in-the-middle) communications attacks, • The replacement and the manipulation of data, and • Covert channel, data exfiltration and diversion of protocols. 5.2 Threats on systems This category includes the attacks which exploit the weaknesses at the level of the standard applications of the server. This problem is supported by the standardization of operating systems (UNIX, NT, XP, or Vista) and standard applications of communication (SMTP e-mailer, browser using HTTP or still use of SQL for databases). The different possibilities of attacks included in this category are: • Attacks on unused network services and not or weakly protected; • Attacks on the availability of the service by use of the bugs in applications; and • Attacks aiming at accessing the computer systems of the company.

123

• Attacks against a site by manipulation of publicly available Internet information resulting in disinformation, defrauding customers, compromising the integrity of the company, • Attacks resulting in attaining information illegally from a site via secretive means (e.g., infiltration via social engineering techniques of phishing, spear-phishing and whaling) and • The modifications of contents via transactions resulting in data exfiltration (e.g., credit card numbers and other personally identifiable information [PII], and identity theft).

5.4 The passive listening The current mechanism used by attackers is monitor (listen to) communication traffic on the network to try to obtain information concerning authentication such as the login and the password of a user. Once this is obtained, the attacker uses this information to connect to the server and impersonate the real user or install a sniffing backdoor tool (non-promiscuous or promiscuous) to repeatedly access the machine. 5.5 Virus The infection of the server by a virus can provoke total or partial unavailability, but more serious still is the fact that the server can propagate the virus to system users. For example, the Asprox virus infection weaves a complex chain of dependencies involving bots that perform SQL injection on vulnerable Web servers and visitors whose machines get compromised simply by visiting infected websites [12].

Quantifying security threats Table 3 Impact (IM) matrix: links components to threats

Threats Comm

Sys

Info

List

Virus

Troj

DoS

DB

NoT

Brws

0.0

0.1

0.1

0.1

0.3

0.4

0.2

0.0

0.0

Prox

0.5

0.1

0.1

0.3

0.3

0.4

0.2

0.0

0.0

R/FW

0.5

0.1

0.1

0.3

0.3

0.4

0.6

0.0

0.0

LB

0.0

0.1

0.1

0.1

0.3

0.4

0.6

0.0

0.0

WS

0.0

0.6

0.6

0.2

0.3

0.4

0.2

0.0

0.0

AS

0.0

0.1

0.1

0.1

0.3

0.4

0.2

0.0

0.0

DBS

0.0

0.1

0.1

0.0

0.5

0.6

0.3

0.8

0.0

NoF

0.4

0.3

0.1

0.1

0.05

0.05

0.1

0.2

1.0

Components

5.6 Trojan The Trojan horse, also known as Trojan, in the context of computing and software, describes a class of computer threats that appears to perform a desirable function but in fact performs undisclosed malicious functions that allow unauthorized access to the host machine, giving them the ability to save their files on the user’s computer or even watch the user’s screen and control the computer. Trojan horse payloads are almost always designed to cause harm, but can also be harmless. They are classified based on how they breach and damage systems. The main types of Trojan horse payloads are: • • • •

Remote access, Data destruction, Downloader/dropper, Server Trojan (Proxy, FTP, IRC, Email, HTTP/HTTPS, etc.), and • Disable security software.

5.7 Denial-of-service (DoS)/Distributed DoS A denial-of-service attack (DoS attack) or distributed denialof-service attack (DDoS attack) is an attempt to make a computer resource unavailable to its intended users. A DoS attack can be perpetrated in a number of ways. The five basic types of attack are: • Consumption of computational resources, such as bandwidth, disk space, or processor time, • Disruption of configuration information, such as routing information, • Disruption of state information, such as unsolicited resetting of TCP sessions, • Disruption of physical network components, and

• Obstructing the communication media between the intended users and the victim so that they can no longer communicate adequately.

5.8 Threats on the database One of the possible attacks, in principle, modifies, indirectly the SQL statement sent to the server, by including special character strings instead of the parameters that are expected by the application software. If a Web application fails to properly check user-supplied input, it is possible for an attacker to alter the construction of the a SQL statement. This technique allows getting back confidential information of the database or simply meta given. We can make, for example, a normal call in the address bar of the browser: http://server/prog? User=name_user. We can then make a call falsified by the type: http://server/prog?User=other_user. Either we obtain directly information concerning the other user, or an error which can give us indications that allow learning, for example, as to names of users. Worse yet, if the attacker is able to modify a SQL statement, the process could run with the same permissions as the original statement resulting in the attacker possibly gaining control of the database.

5.9 Generating the impact matrix Given that we have catalogued eight security threats, the impact matrix will have nine columns, one for each threat plus one for the absence of threats. On the other hand, it has eight rows, one for each component plus one for the event that no component has failed during the unitary time period. This gives a total of 72 entries; we will comment on some of them, which we will represent in Table 3. The absence of threats does not cause the failure of any component, and leads to event NoF (no failure) with probability 1.0.

123

A. Ben Aissa et al.

• We estimate that threats to the database cause a failure of the database with probability 0.8, say, to make provisions for the case where an attack fails to achieve its goal; they may cause event NoF (no failure) with probability 0.2. We assume that because the DB component is the only target of this threat, the probability that it causes a failure of any other component is 0.0. • Generally, the row labeled NoF represents the probability of failure of each threat, i.e., the probability that it does not cause any component to fail. • The threat on communication protocol (Comm) targets the proxy servers and the routers; we assume that the probability that it causes a failure of the other components is 0.0. • A virus has some likelihood of affecting any component of the system, through propagation. • A Trojan horse has some likelihood of targeting any component of the system, through propagation. • The threat passive listening (list) targets primarily the components that are involved with communication. • The denial-of-service attacks (DoS) may target the bottlenecks of the architecture, for maximal effect.

Stakeholders

MFC ($/h)

Customer

8.11

Merchant

112.97

Technical intermediary

31.17

Financial intermediary

54.24

Using this data, we now compute the vector of mean failure costs [2,13,14], using the formula MFC = ST ◦ DP ◦ IM ◦ PT. Substituting each matrix by its value, in Table 5, we find: While our focus so far has been on measuring mean failure cost, we discuss in the next section how to control mean failure cost. We do so by analyzing the cost of various measures that one can deploy to improve security, and match these costs against the benefits that result from these measures in terms of reduced mean failure costs.

7 Application of MFC: security measures and counter-measures

6 Threat configuration Vector PT (Table 4) characterizes the threat situation by assigning to each threat category the probability that this threat will materialize over a unitary period of operation (say, an hour). We assume that no more than one threat can materialize within a unitary period of time, and we make provisions for the case where no threat materializes. Hence this vector contains a probability distribution of complementary events. We assume that, in light of “log” data, known vulnerabilities, and known perpetrator behavior, that we can determine that the prevailing threats have the probabilities indicated in Table 4.

Table 4 Vector PT providing threat probability Probability Threats Comm

0.01

Sys

0.02

Info

0.01

List

0.01

Virus

0.03

Troj

0.06

DoS

0.03

DB

0.02

NoT

0.81

123

Table 5 Stakeholder mean failure cost (MFC)

As an example of application of the mean failure cost, we perform a cost/benefit analysis on a number of security measures that one can deploy. Because the mean failure cost is calculated as the product of many factors (the stakes matrix, the dependability matrix, the impact matrix, and the threat vector), we can control mean failure costs by controlling any one of these factors. For the sake of argument, we classify security measures according to which factor they involve. We briefly discuss this classification, below: • Mitigation measures: controlling the stakes matrix. This family designates measures which we take to reduce the impact of failures on costs incurred by users. • Failure tolerance measures: controlling the dependability matrix. This family designates measures which minimize the impact of component failures on system failures by enhancing the failure tolerance of the system (using Redundancy, for example). • Fault tolerance measures: controlling the impact matrix. This family designates measures which minimize the incidence of component failures by eliminating or mitigating component vulnerabilities. • Evasive measures: controlling the threat vector. This family designates measures which aim to conceal component vulnerabilities, or otherwise making it harder to exploit them.


• In the remainder of this section we review some examples of security measures, and assess their cost effectiveness by matching their implementation cost against their benefits, measured in terms of reduction in mean failure cost. We quantify the cost/benefit analysis by means of return on investment (ROI) formulas, and compare alternative measures by comparing their respective ROIs. 7.1 Deploying an anti-virus The deployment of an anti-virus reduces the mean failure cost by reducing the threat vector PT. However, the impact of this measure wanes with time, as perpetrators find ways to go around it. The question we raise is then: how do we estimate the effectiveness of an anti-virus, and how do we estimate its rate of decline over time? A study conducted by AV Comparatives [15] (www.av-comparatives.org) gives us concrete data to answer both questions. Specifically: • A 1-week field study of 15 of the most commonly used anti-virus products reveals a rate of effectiveness (ratio of active viruses that are detected and eliminated by the antivirus) that varies between 18 and 71%, with an average of 44.2%. • A 4-week field study of the same anti-virus products reveals a decline in performance that varies between 8 and 67%, with an average decline of 40.13%. Assuming a constant rate of decline, this figure allows us to estimate the weekly decline as the fourth root of 0.5987 (1 − 0.4013), which is 0.8796. We view the purchase and deployment of the anti-virus as an investment decision, where the investment cost is the cost of acquiring and deploying the anti-virus. In order to assess the cost effectiveness of this investment decision, we must match its cost against the benefits gained from deploying the antivirus. We quantify these benefits by means of reduced mean failure costs for the various stakeholders. To match costs and benefits, we use a return on investment formula, which has the form: ROI = NPV =

NPV , C(0) W B(w) − C(w) , (1 + d)w

w=0

• C(w) is the cost incurred on week w. • We resolve to compute the ROI of each stakeholder, to quantify the stake that each stakeholder has in acquiring and deploying the anti-virus. To this effect, we take the following options: • The discount rate is computed on the basis of the annual rate of 0.15. For our purposes, we let it be 0.15/52 = 0.0029 (as there are approximately 52 weeks/year). • The duration of the investment cycle is estimated at 50 weeks, by virtue of a forthcoming discussion (to the effect that, under the adopted hypotheses, the anti-virus has no impact after about 50 weeks). • For the sake of simplicity, we assume that C(w) = 0 for all w > 1, and for all stakeholders. • For the sake of simplicity, we assume B(0) = 0 for all stakeholders. • For the sake of argument, we assume that the cost of acquiring and deploying the anti-virus is 2,500 $. • For the sake of fairness, we assume that this investment cost is charged on the stakeholders in proportion to their stake in the system, as measured by their respective MFCs. We further assume that for the customer, this cost is charged indirectly though higher purchase prices. These assumptions enable us to compute the cost charged on the stakeholders at week 0, for the acquisition and deployment of the anti-virus system, as follows in Table 6: To estimate B(w) for each stakeholder, we must analyze the reduction in mean failure cost that each stakeholder will experience as a result of installing the anti-virus. For each week, starting from the week the anti-virus is installed, we compute the success rate of the anti-virus, from which we infer the probability of detecting viruses in the threat vector; assuming the anti-virus affects no other threat but the virus threat, this allows us to estimate the probability of the “No Threat” event. We find the following matrix (Table 7), in which we represent, for each week, the rate of decline (which, in this example, we assume constant) of the effectiveness of the anti-virus; whence we derive the (declining) success rate of the anti-virus, then the failure rate, from which we infer the (increasing) probability of occurrence of a virus threat, and the (decreasing) probability of no threat. For the sake of illustration, we show relevant values of these parameters for weeks 0, 5, 10, 15, … until 50. By week 50, the effect

where • ROI is the return on investment, • NPV is the net present value of the investment, • W is the duration of the investment cycle (in weeks, in this case), • d is the (weekly) discount rate, obtained (by approximation) by dividing the yearly discount rate by 52. • B(w) is the benefit gained from the investment at week w.

Table 6 Initial cost of investment Stakeholders

MFC ($/h)

C(0) ($)

Customer

8.11

98.19

Merchant

112.97

1367.74


31.17

377.38


54.24

656.69

123

A. Ben Aissa et al. Table 7 Evolving threat probabilities Week 0

Week 5

Week 10

Week 15

Week 20

Week 25

Week 30

Week 35

Week 40

Week 45

Week 50

Decl

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

0.8796

Succ

0.442

0.2327

0.1225

0.0645

0.0339

0.0179

0.0094

0.0049

0.0026

0.0014

0.0007

Fail

0.558

0.7673

0.8775

0.9355

0.9661

0.9821

0.9906

0.9951

0.9974

0.9986

0.9993

Virus

0.03

0.0230

0.0263

0.0281

0.0290

0.0295

0.0297

0.0298

0.0299

0.0299

0.03

NoT

0.81

0.8169

0.8136

0.8119

0.8110

0.8105

0.8103

0.8101

0.8101

0.8100

0.8100

Table 8 An infrastructure for computing weekly benefits, B(w) Week 0

Week 5

Week 10

Week 15

Week 20

Week 25

Week 30

Week 35

Week 40

Week 45

Week 50

Virus

0.03

0.0230

0.0263

0.0281

0.0290

0.0295

0.0297

0.0298

0.0299

0.0299

0.03

NoT

0.81

0.8169

0.8136

0.8119

0.8110

0.8105

0.8103

0.8101

0.8101

0.8100

0.8100 8.11

MFC w. anti-virus Cust

8.11

7.95

7.94

8.02

8.06

8.08

8.09

8.10

8.10

8.10

Merch

112.97

108.54

110.63

111.77

112.33

112.65

112.78

112.84

112.90

112.90

112.97

Tech

31.17

29.95

30.52

30.84

31.00

31.08

31.12

31.14

31.15

31.15

31.17

Finan

54.24

52.12

53.12

53.66

53.94

54.09

54.15

54.18

54.21

54.21

54.24

Cust

0

0.16

0.17

0.09

0.05

0.03

0.02

0.01

0.01

0.01

0

Merch

0

4.43

2.34

1.2

0.64

0.32

0.19

0.13

0.07

0.07

0

MFC gain

Tech

0

1.22

0.65

0.33

0.17

0.09

0.05

0.03

0.02

0.02

0

Finan

0

2.12

1.12

0.58

0.3

0.15

0.09

0.06

0.03

0.03

0

Hours of operation Cust

2

2

2

2

2

2

2

2

2

2

2

Merch

168

168

168

168

168

168

168

168

168

168

168

Tech

168

168

168

168

168

168

168

168

168

168

168

Finan

168

168

168

168

168

168

168

168

168

168

168 0

Benefit, B(w) Cust

0

0.32

0.34

0.18

0.1

0.06

0.04

0.02

0.02

0.02

Merch

0

744.24

393.12

201.6

107.52

53.76

31.92

21.84

11.76

11.76

0

Tech

0

204.96

109.2

55.44

28.56

15.12

8.4

5.04

3.36

3.36

0

Finan

0

356.16

188.16

97.44

50.4

25.2

15.12

10.08

5.04

5.04

0

of the virus is virtually gone, and the probabilities of events Virus and NoT in the threat vector revert to what they were on week 0. For each week w of the investment cycle, we compute the threat vector PT(w) by replacing the Virus and NoT entries in vector PT (Sect. 6) by the values given in Table 7 for that week. From PT(w) we infer the MFC vector (as discussed in Sect. 6). The difference between MFC with the anti-virus and MFC without gives us the benefit vector for the given week. Because this is an hourly rate, we convert it to a weekly rate by taking into account the number of hours of usage per week. For customers, we assume 2 h/week; for the remaining stakeholders, we take 168 h (assuming perfect availability of the system). This gives us the matrix of benefits of all the stakeholders for all the weeks of the investment cycle. See Table 8, where we show a sampling of the weeks of interest.

123

Table 9 Stakeholder mean failure cost (MFC) Stakeholders

ROI

Customer

−0.98

Merchant

0.118


0.114


0.111

Using this data, we derive the ROI of the anti-virus for all four stakeholders; the results are given in Table 9. Under the current circumstances, the ROI of the user is negative, hence the proposed investment cannot be deemed worthwhile for all the stakeholders. In order to remedy this situation, we resolve to revisit the formula we had used to determine initial investment costs. Instead of charging


stakeholders according to their pre-investment MFC, we will instead charge them in such a way as to make all ROIs identical. This resolution produces a system of four equations in four unknowns: • The unknowns are the initial investments of all the stakeholders, i.e. C(0) for the customer, the merchant, the technical, and the financial intermediary. • The four equations we are referring to include three equations that say the four initial investments are equal, and an equation that says the sum of all initial investments equals the total investment. We are going to generalize the resolution of this problem and then specify it to our case. For simplicity we assume that: • For all stakeholders Ki , Bi (0) = 0 and Ci (w) = 0 for all week between 1 and Wi • The return on investment of stakeholder Ki is i Bi (w) 1 × Ci (0) (1 + d)w

W

ROIi = −1 +

w=1

If we suppose that the ROIi are equal for all stakeholders Ki , 1≤i ≤n To search the initial cost Ci (0) for all stakeholders we need to resolve the linear system composed by n equations which

Table 10 Redistribution balanced of the ROI in deploying an anti-virus Stakeholders

Ci (0) $

ROIi

Customer

0.98680858

0.07256744

Merchant

1426.35436

0.07256744


391.982004

0.07256744


680.676826 4 i=1 Ci (0) = 2, 500$

0.07256744

Table 11 Dependency (DP ) matrix for the redundant architecture

we can resumed in ROI = · · · = ROIi = · · · = ROIn , i ∈ [1, n] n 1 i=1 Ci (0) = C(0) The solution of this system is: Wi C(0) × w=1 Ci (0) = n W i i=1

Bi (w) (1+d)w Bi (w) w=1 (1+d)w

In our case we have four stakeholders if we replace i by (Customer, Merchant, Technic_inter, Finan_inter) and W by 50 (number of week) we find in Table 10. As we expect, the ROI is the same for all stakeholders; also, they are all positive, though rather small. Nevertheless, we assume that this value is considered sufficient for the purposes of the stakeholders to justify the investment. 7.2 Deploying redundancy As an alternative to the anti-virus, or in addition to it, we propose to implement a redundancy mechanism that duplicates some component of the system. Given the current architecture of the system, a prime candidate for duplication is the load balancer. We assume that we now have two load balancers, along with a mechanism that manages their deployments: One load balancer is used as a default, and another as spare. When the monitor determines that the default has failed, it switches to the spare. In the original architecture, a failure of the load balancer leads to a failure of the system with respect to all requirements; whence a column of 1.0 in the dependability matrix for the load balancer component. In the new architecture, assuming no more than one component fails at a time, the column for the load balancer goes from 1.0 to 0.0. In practice, if we want to take into account the possibility that the monitor of the load balancers may fail to recognize that the default has failed, or may fail to switch from the default to the spare, this probability may be greater than 0.0, but for the sake of argument we keep it at 0.0. This yields the matrix given in Table 11.

Components Browser

Proxy server

Router/ firewall

Load balancer

Web server

Appl. server

Database server

No failure

0.0

Security requirements Conf

0.2

0.2

1.0

0.0

0.333

0.333

0.5

Int

0.2

0.2

1.0

0.0

0.333

0.333

0.0

0.0

Avail

1.0

1.0

1.0

0.0

0.333

0.333

0.0

0.0

NR

0.2

0.2

1.0

0.0

0.333

0.333

0.0

0.0

Auth

0.2

0.2

1.0

0.0

0.333

0.333

0.5

0.0

Priv

0.2

0.2

1.0

0.0

0.333

0.333

0.5

0.0

123

A. Ben Aissa et al. Table 15 Redistribution balanced of the ROI in deploying redundancy

Table 12 Stakeholder mean failure cost (MFC) MFC

Stakeholders

$/h

Stakeholders

Ci (0) $

ROIi 4.215960952

Customer

5.91

Customer

39.21009482

Merchant

82.66

Merchant

45377.48628

4.215960952


22.92


12351.17987

4.215960952


39.39


22232.12376 4 i=1 Ci (0) = 80, 000$

4.215960952

Table 13 Initial cost of investment Stakeholders

C(0) $

Customer Merchant

43,768


12,076


21,014

Using this new dependability matrix, we can now compute the resulting vector of mean failure costs using the formula: MFC = ST ◦ DP ◦ IM ◦ PT. The results are recorded in Table 12: Using this new value of the MFC vector, we can estimate gains in MFC for each stakeholder, and use this data to estimate weekly benefits for each stakeholder by considering the number of hours of usage every week for each stakeholder. As for the other parameters of the return on investment, we resolve to take the following options: • For the sake of comparison, we use the same periodicity (a week), same cycle length (50 weeks), same discount rate (yearly discount rate of 0.15, prorated to a week), as the example above. • We assume that the generation and deployment of the spare load balancer, as well as the logic that monitors balancer performance and switches to the spare, costs a total of 80,000$ • For the sake of argument, we distribute this cost on the four stakeholders in proportion to their stake in the system, as measured by their current MFC. This yields the following table for C(0). • We assume that customers are charged their share of stakeholder costs through price hikes. • We assume that customers use the system an average of 2 h a week, and the other stakeholders use it all week Table 14 Infrastructure for computing ROIs for the load balancer redundancy

Stakeholder Cust

MFC before

MFC after

Details of the calculation of the ROI of the proposed investment for all stakeholders are given in Table 14. The ROIs of the stakeholders are shown in Table 14. As we can see, the ROI of the customer is once again negative. We make the same resolution as above, namely, to distribute the investment costs among stakeholders in such a way as to make all ROIs identical. The result is shown in Table 15. It is abundantly clear, from this table, that the investment is worthwhile for all stakeholders, and that it is much more so than the anti-virus. Given that all these ROIs are equal and positive and deploying an anti-virus or Redundancy deemed worthwhile for all stakeholders.

8 Conclusions In this paper we have illustrated the application of the CSES/MFC model for estimating system security by means of a concrete example. The quantification of security attributes by means of costs to stakeholders opens a wide range of possibilities for further economics based analysis, and provides a valuable resource for rational decision making; our future plans call for exploring such opportunities. The examples we have discussed in Sect. 6 of the applications of MFC are only a sample of the type of quantitative reasoning and decision making that is possible thanks to this measure.

MFC gain

Hours of oper 2

B(w)

4.4

C(0)

W

d

ROI −0.934

8.11

5.91

2.2

3 142

50

0.0029

112.97

82.66

30.31

168

5092.08

43 768

50

0.0029

4.4077

Tech

31.17

22.92

8.25

168

1386

12 076

50

0.0029

4.3348

Finan

54.24

39.39

14.85

168

2494.8

21 014

50

0.0029

4.5183

Merc

123

(perfect availability), to a total of 168 h. This hypothesis allows us to convert the benefits of the investment (resulting from reduced mean failure cost) from an hourly figure to a weekly figure. Unlike the previous example, benefits are assumed to be the same for all weeks of the investment cycle, for each stakeholder. This yields a simpler calculation.

3,142


References 1. Abercrombie RK, Sheldon FT, Mili A (2008) Synopsis of evaluating security controls based on key performance indicators and stakeholder mission value. In: 11th IEEE high assurance systems engineering symposium (HASE ‘08), Nanjing, China, pp 479–482 2. Sheldon FT, Abercrombie RK, Mili A (2009) Methodology for evaluating security controls based on key performance indicators and stakeholder mission. In: Proceedings of 42nd annual Hawaii international conference on system sciences (HICSS-42), Waikoloa, HI, p 10 3. Abercrombie RK, Sheldon FT, Mili A (2009) Managing complex IT security process with valued based measures. In: 2009 IEEE symposium on computational intelligence in cyber security (CICS 2009), Nashville, TN, p 7 4. Firesmith D (2004) Specifying reusable security requirements. J Object Technol 3(1):61–75 5. Rocha SVd, Abdelouahab Z, Freire E (2005) Requirement elicitation based on goals with security and privacy policies in electronic commerce. In: Anais do WER05—workshop em Engenharia de Requisitos, Porto, Portugal, pp 63–74 6. Sekaran KC (2007) Requirements driven multiple view paradigm for developing security architecture. In: PWASET, Proceedings of World Academy of Science, Engineering and Technology, pp 156–159 7. Sawma VD, Probert RL (2003) E-Commerce authentication: an effective countermeasures design model. In: ICEIS 2003, Proceedings of the 5th international conference on enterprise information systems, Angers, France, pp 447–455

8. Pritchett D (2007) The eBay architecture: striking a balance between stability, feature velocity, performance, and cost. In: Colorado Software Summit 2007, Keystone, CO, pp 1–39 9. Ahmed MU “eBay—eCommerce platform, a case study in scalability. McGill University, pp 1–13 10. Goswami R, De SK, Datta B (2005) E-business adoption in select Indian firms and segments: a stakeholders’ approach through select Indian portals analysis. In: CISTM 2005, conference of information science, technology and management, pp 1–22 11. Sheldon FT, Abercrombie RK, Mili A (2008) Evaluating security controls based on key performance indicators and stakeholder mission. In: Proceedings of the 4th annual cyber security and information intelligence research workshop, Oak Ridge, TN 12. Shin Y, Myers S, Gupta M (2009) A case study on asprox infection dynamics. In: Sixth conference on detection of intrusions and malware & vulnerability assessment (DIMVA 2009), Milan, Italy. Springer, Heidelberg, pp 1–20 13. Mili A, Sheldon FT (2007) Measuring reliability as a mean failure cost. In: Proceedings of the 10th IEEE high assurance systems engineering symposium, Dallas, TX, pp 403–404 14. Mili A, Sheldon FT (2009) Challenging the mean time to failure: measuring dependability as a mean failure cost. In: Proceedings of 42nd Hawaii international conference on system sciences (HICSS-42), Waikoloa, HI, p 10 15. Anti virus comparative test No 20: http://www.av-comparatives. org, November 2008

123