Stochastic Game Nets and Applications in Network Security Chuang Lin, Yuanzhuo Wang, Yang Wang, Haiyi Zhu Department of Computer Science and Technology Tsinghua University, Beijing 100084, P.R. China. Email: {clin, yzwang,ywang}@csnet1.cs.tsinghua.edu.cn,
[email protected] Quan-Lin Li Department of Industrial Engineering, Tsinghua University Beijing 100084, P.R. China. Email:
[email protected] No Institute Given
Abstract. In this paper, we propose a novel modeling method, Stochastic Game Nets(SGN). SGN is an good method to model and deal with the game issues, which takes advantages of both stochastic game theory and Stochastic Petri Nets. The SGN could inherit the efficient and flexible modeling approach of Stochastic Petri Nets, and also make well use of the game-theoretical framework from game theory. Meanwhile, we apply the SGN method to model and analyze the network attacks, compute the Nash equilibrium and best-response strategies to defend the attacks. We believe that the SGN opens a new avenue to deal with the game issues in computer networks. Key words: Stochastic Petri Nets, Game Theory, Network Security, Nash equilibrium
1
Introduction
Network systems are becoming more complex and larger, and specifically network attacks and security destroy is also increasingly popular and multiple. Network security is being confronted with many significant challenges. Network security is one of the most important scientific issues in the world and has been an important factor which effects on social improvement and national develop strategy. Most of the early work in security emphasizes the prevention of attacks in system. Later most work focuses on system-level security mechanisms so that the system can perform its intended function through detecting and preventing malicious attacks. More recently, the notion of intrusion tolerance has been advocated to allow the system to continue performing its intended function despite partially successful attacks [1]. Most attempts to validate security mechanisms and strategies, however, have been qualitative by showing that the process employed to construct a system is secure. But it can not be practically feasible to construct perfectly secure mechanisms and strategies, in face of complex and
2
C. Lin et al.
various attack behaviors. Security specialists are interested in knowing what an intruder can do to a computer network, and what can be done to prevent or counteract attacks much more. Game theory now has been invited to the field of network security and computer security. In literature [2], a game theoretic method for analyzing the security of computer networks is presented. The interactions between an attacker and the administrator are modeled as a two-player stochastic game for which best-response strategies (Nash Equilibriums) are computed. Literature [3] proposes a new protocol for preventing malicious bandwidth consumption, and demonstrates how game-based formal methods can be successfully used to verify availability-related security properties of network protocols. Literature [4] presents a general incentive-based method to model attacker intent, objectives, and strategies (AIOS) and a game-theoretic approach to inferring AIOS. The authors developed a game theoretic AIOS formalization which can capture the inherent interdependency between AIOS and defender objectives and strategies in such a way that AIOS can be automatically inferred. Literature [5] and [6] propose the puzzle auction mechanism to defend the DoS and DDoS attacks based on game theory. Where the puzzle auction protocol of [5] allowed clients to bid for service by computing puzzles with difficulty levels of their own choices. The ones bidding for the most difficulty puzzles would achieve the highest priorities of being served. Boldizsar, Istvan and Levente [6] used static game theory to analyze the optimal strategy for the server and the DoS attacker when client puzzle was utilized. In literature [7], Xu and Lee use game-theoretical framework to analyze the performance of their proposed DDoS defense system and to guide its design and performance tuning accordingly. Browne [8] describes how static games can be used to analyze attacks involving complicated and heterogeneous military networks. In his example, a defense team has to defend a network of three hosts against an attacking team’s worms. In the previous work, the interaction between the attacker and administrator are modeled as some game relations. A purely competitive (zero-sum) stochastic game would make us always find a Nash equilibrium. And a general-sum stochastic game would allow us to find potentially, multiple Nash equilibrium. A Nash equilibrium gives the administrator an idea of the attacker’s strategy and a plan for what to do in each state in the event of an attack. According to the Nash equilibrium, we could know about the attacker’s best attack strategies. By using a stochastic game model, we are also able to capture the probabilistic nature of the state transitions, which is more realistic. However, some essential limitations affect applications of the game theory. Firstly, for the complex network structure, game theory has not enough modeling ability to describe interaction relations. The comprehensive and exact models are hard to upbuild. Secondly, the complicated mathematics descriptions and the state transition graphs make people hard to understand the actual signification of the models. And they are difficult to update when conditions change. Thirdly, the general game model is that the full state space can be extremely large. We are however, interested in only a small subset of states that are in attack
Stochastic Game Nets and Applications in Network Security
3
scenarios. Solutions for stochastic models are hard to compute. In addition, for reality, it may be difficult to quantify the costs for some actions and transition probabilities may not be easily available. Aiming at the limitations of game theory, the objective of this paper work is to research a new method, Stochastic Game Net (SGN), to model and analyze the expected attacker behavior based on Stochastic Petri Nets (SPN). Stochastic Petri Nets [9] are a modeling formalism that can be conveniently used for the modeling and analyzing complex systems, such as performance analysis and reliability evaluation. The automatic construction of the probabilistic models that underlie the dynamic behaviors of these nets relies on a set of results which are derived from the theory of Petri Nets. SPN models have good expansibility. The new components can be easily appended. On the other hand, Stochastic Reward Nets (SRN) [10], SPN models added reward function, are more suitable to describe the rewards of players in an attack game. The contributions of this paper are two-folded. First, a new modeling method, Stochastic Game Net, is presented. It unites the advantages of stochastic game theory and Stochastic Petri Nets, and supplies the limitations of game theory to model network attack. Second, using this method, we successfully model the network attacks, compute the Nash equilibrium and best-response strategies for the players (attacker and administrator). We then explain why the strategies are realistic and how administrators can use these results to enhance the security of their network. We hope that our approach will lead to further investigation of modeling and analyzing theories in dealing with game issues in computer networks. The rest of the paper is organized as follows. Section 2 introduces stochastic game and stochastic Petri net. Section 3 presents the definition and properties of Stochastic Game Net. In Section 4, SGN is applied to model and analyze network attacks, where system performance and security are estimated from the model solution. Section 5 concludes the paper.
2 2.1
Stochastic Game and Stochastic Petri Net Stochastic Game
Definition 1. (Stochastic Game) A two-player stochastic game model is described by a 5-tuple vector as GM = (S, Ak , Q, Rk , β), where: (1) S = {s1 , . . . , sn } is a finite set of states. (2) Ak = {ak1 , . . . , akMk }, (k = 1, 2, Mk = |Ak |) is the action set of player k. The action set for player k at state s is a subset of Ak , i.e., Aks ⊆ Ak and SN k k i=1 Aεi = A . (3) Q : S × A1 × A2 × S → [0, 1] is the state transition function. (4) Rk ; S × A1 × A2 → R, k = 1, 2, is the reward function of player k. (5) δ, 0 < δ < 1 is a discount factor for discounting future rewards, i.e., at the current state, a state transition has a reward worth its full value, but the reward for the transition from the next state is worth δ times its value at the current state.
4
C. Lin et al.
The game is played as follows: at a discrete time instant t, the game is in state st ∈ S. Player 1 chooses an action a1t from A1 and player 2 chooses an action a2t from A2 . Player 1 then receive a reward rt1 = R1 (st , a1t , a2t ), and Player 2 gets a reward rt2 = R2 (st , a1t , a2t ). The game then moves to a new state st+1 with conditional probability P (st+1 |st , a1t , a2t ) according to Q(st , a1t , a2t , st+1 ). We are interested in determining a course of action for a player in this environment. Specifically we want to learn a stationary though possibly stochastic strategy that maps states to a probability distribution over its actions. The goal is to find such a strategy that maximizes the player’s discounted future reward. Definition 2. (Strategy) A strategy π k for player k is a vector. π k (s) can be described by [π k (s, a1 ), π k (s, a2 ), . . . , π k (s, aM k )]T , where π k (s, a), is the probability that player k should use to take action a while it is in state s. A strategy is also called a mixed or randomized strategy, which means that the player chooses an action in random manner. The set of mixed strategies includes the pure strategies, when the player chooses the actions in a deterministic way. A pure strategy is a special case of mixed strategy such that probability 1 is assigned to one action and 0 to all other actions. A stationary strategy π k is a strategy that is independent of time and history. A mixed or randomized stationary strategy is one where π k (s, a) ≥ 0, s ∈ S and a ∈ Ak and a pure strategy is one where π k (s, ai ) = 1 for some ai ∈ Ak . Stochastic Games can be usefully classified according to the structure of their payoff functions. Two common classes of games are purely collaborative and purely competitive games. Purely collaborative games are ones where all the players have the same reward function. Purely competitive, or zero-sum, games are two-player games where one player’s reward is always the negative of the other’s. Like matrix games, zero-sum stochastic games have a unique Nash equilibrium, although finding this equilibrium is not so easy. A non-trivial result, proven by [11] for zero-sum games and by [12] for general-sum games, is that there exist equilibrium solutions for stochastic games. 2.2
Nash Equilibrium
A Nash equilibrium, named after John Nash, is a set of strategies, one for each player, such that no player has incentive to unilaterally change her action. Players are in equilibrium if a change in strategies by any one of them would lead that player to earn less than if she remained with her current strategy. For games in which players randomize (mixed strategies), the expected or average cost must be at least as large as that obtainable by any other strategy. Definition 3. (Nash equilibrium) Given a normal form game, a Nash equilibrium (NE) is vector of mixed strategies, one for each player, such that no player has an incentive to deviate from its mixed strategy given that the others do not deviate. That is, for any player k and any alternative mixed strategy π i , that ν k (π 1∗ , π 2∗ , . . . , π i∗ , . . . , π n∗ ) ≥ ν k (π 1∗ , π 2∗ , . . . , π i , . . . , π n∗ ), where ν k (π 1∗ , π 2∗ , . . . , π i∗ , . . . , π n∗ ) is the expected return vector of the game for player k when all players play their stationary strategies π 1∗ , π 2∗ , . . . , π n∗ respectively.
Stochastic Game Nets and Applications in Network Security
5
At this equilibrium, there is no mutual incentive for anyone of the players to deviate from their equilibrium strategies π 1∗ , π 2∗ , . . . , π n∗ . A deviation will mean that some of them will have lower expected returns, i.e., ν k (π 1 , π 2 , . . . , π n ). A vector of Nash equilibrium strategies is also known as best responses. The objective of each player is to maximize some expected return. Let st be the state at time t and rtk be the reward received by player k at time t. We define an expected return payoff function as the column vector: ν k (π) = [ν k (π, s1 ), ν k (π, s2 ), . . . , ν k (π, sN )]T .
(1)
Let δ be a fixed number in (0, 1), and define the expected payoff function for player k as: ν k (π, s) = E[rk (st ) + δrk (st+1 ) + δ 2 rk (st+2 ) + · · · + δ N rk (st+N )] N X = E[ δ n rk (st+i )], i=0
for each multistrategy π and each initial state s. It represents the expected present value of the rewards of player k under the multistrategy π. The number δ is called a ”discount factor”. The expectation operator E[] is used to mean that player k plays π k , i.e., player k chooses an action using the probability k distribution π k (st+n ) and receives an reward rt+n . For a two-player stochastic game, a Nash equilibrium in stationary strategies (π 1∗ , π 2∗ ) is one which satisfies: ν 1 (π 1∗ , π 2∗ ) ≥ ν 1 (π 1 , π 2∗ ); ν 1 (π 1∗ , π 2∗ ) ≥ ν 1 (π 1 , π 2∗ ). Reward is written as: k rt+n = π 1 (st+n )T Rk (st+n )π 2 (st+n ),
(2)
where Rk (s) = [Rk (s, a1 , a2 )], (a1 ∈ A1 , a1 ∈ A1 ) is the reward matrix in state s of player k. If multiple Nash equilibrium are found, that means multiple pairs of Nash strategies are found. In each pair, a strategy for one player is a best-response to the strategy for the other player and vice versa. 2.3
Stochastic Petri Net
Stochastic Petri Nets are Petri nets augmented with the set of average transition rates for the exponentially distributed transition-firing times. A transition represents a class of possible changes of markings. Such a change, also called transition firing, consists of removing tokens from the input places of the transition and adding tokens to the output places of the transition according to the expressions labeled on the arcs. A transition may be associated with an enabling
6
C. Lin et al.
predicate which can be expressed in terms of the place marking expressions. If the predicate of a transition evaluates to be false, the transition is disabled. In SPN models, transitions can be categorized into two classes: transitions of Class One are used to represent logical relations or determine if some conditions are satisfied [13].This class of transitions is called immediate transition with zero firing time. Transitions of Class Two are used to represent the operations on the tasks or information processing. This class of transitions is called timed transition with exponential distributed firing time. A marking in a SPN model represents a distribution of tokens in the model. The state space of a model consists of the set of all markings reachable from the initial marking through the occurrence of transition firing. A SPN is homomorphism to a continuous time Markov Chain (MC), and there is a one-to-one relationship between markings of the SPN and states of the MC [13] and [14]. Definition 4. (Stochastic Petri Nets) Stochastic Petri Net is a quadruple (P, T, F, λ), where (1) P is a finite set of places; (2) T is a finite set of transitions (P ∩ T 6= ∅); (3) F ⊆ (P × T ) ∪ (T × P ) is a set of arcs; (4) λ = (λ1 , λ2 , . . . , λn ) is a set of firing rates of transitions in transition set. As an extension of Stochastic Petri Nets (SPN), Stochastic Reward Net (SRN) is a powerful graphical and mathematical tool, which not only is able to model concurrent, asynchronous, stochastic and nondeterministic events, but also provide transition enabling function and firing probability that can be used to model various algorithms and strategies. The SRN differ from the SPN in several key aspects. From a structural point of view, both formalisms are equivalent to Turing machines. But the SRN provide enabling functions, marking dependent arc cardinalities, a more general approach to the specification of priorities, and the ability to decide in a marking-dependent fashion whether the firing time of a transition is exponentially distributed or null, often resulting in more compact nets. Perhaps more important, though, are the differences from a stochastic modeling point of view. The SRN formalism considers the measure specification as an integral part of the model. Underlying an SRN is an independent semiMarkov reward process with reward rates associated to the states and reward impulses associated to the transitions between states [15].
3
Stochastic Game Nets
The aim of this section is introduce the Stochastic Game Nets(SGN). The SGN structure will represent all possible strategies existing within the game. Definition 5. (Stochastic Game Nets) A Stochastic Net is the 9-tuple SGN = (N, P, T, π, F, R, λ, U, M0 ), where (1) N = 1, 2, . . . , n denotes a finite set of players; (2) P is a finite set of places;
Stochastic Game Nets and Applications in Network Security
7
(3) T = T 1 ∪ T 2 ∪ · · · ∪ T n is a finite set of transitions, where T k is the set of transitions with respect to player k, for k ∈ N ; (4) π : T → [0, 1] is a routing policy representing probability of choosing a particular transition; (5) F ⊆ I ∪ O is a set of arcs where I ⊆ (P × T ) and O ⊆ (T × P ), such that P ∩ T = ∅ and P ∪ T 6= ∅; (6) R : T → (R(1) , R(2) , . . . , R(n) ) is a reward function for the player taking each action; (7) λ = (λ1 , λ2 , . . . , λk ) is a set of firing rates of transitions in transition set, where k is the number of transitions; (8) U (pki ) is the utility function, when player k in the condition pi . Accordingly, the player can choose the best transition; (9) M0 is the initial marking. Firing Rule: The firing rule of a SGN = (N, P, T, π, F, R, λ, U, M0 ) is given as follows. A marking m represents a distribution of the tokens in SGN. Each token s is related with a reward vector h(s) = (h1 (s), h2 (s), . . . , hn (s)) as its properties. Each element of T represents a class of possible changes of markings. Such a change of t, also called transition firing, consists of removing tokens from a subset of places and adding them to other subsets according to the expressions labeling the arcs. A transition t is enabled under a marking M whenever, for all p ∈ P and(p, t) ∈ F , M (p) 6= φ. Each player gets the reward R(t) through the transition, and the reward is recorded in the reward vector h of each token. Now, we present some notations with respect to a 9-tuple Stochastic Game Net SGN = (N, P, T, π, F, R, λ, U, M0 ). Definition 6. An action t ∈ T with respect a SGN = (N, P, T, π, F, R, λ, U, M0 ) is a optimum decision, while it is optimum according to the utility function for player k. Definition 7. A strategy with respect to a SGN = (N, P, T, π, F, R, λ, U, M0 ) is identified by δ and consists of the transition sequence represented in the SGN graph model. Definition 8. An optimum strategy with respect to a SGN = (N, P, T, π, F, R, λ, U, M0 ) is identified by δ, and consists of the transition sequence represented in the SGN graph model where a Nash equilibrium is reached for all the palyers. Theorem 1. If a SGN = (N, P, T, π, F, R, λ, U, M0 ) has a finite set of places and transitions, there always exits a Nash equilibrium. Proof. Theorem 2. A SGN = (N, P, T, π, F, R, λ, U, M0 ) is sufficient to describe a game problem. Proof.
8
C. Lin et al.
Theorem 1 presents the feasibility of the SGN tool. Theorem 2 proposes the maturity of the SGN tool. Moreover, we could determine the scope of this approach in a given area, such as the problems with respect to network security. Now, we present the two steps to solve the SGN to find the Nash equilibrium. The Nash equilibrium corresponds to the optimized strategy of each player. We first construct the reachability tree according to the SGN, and then find out the Nash equilibrium. Algorithm-1: Construct the Reachability Tree from SGN A reachability is consist of nodes, which are denoted by all the reachable markings of the SGN, and the arcs among the nodes. From a SGN with a starting marking M0 , we can construct a reachability tree. The algorithm has three steps. (1) Make M0 the root r of the tree. (2) Node x marked by M is a leaf if and only if there isn’t a transition t ∈ T which is enabling under M ,or there is a node y 6= x along the road from r to x, which has a similar mark M 0 as M . We define two marks M1 and M2 similar as follows: M1 = φ if and only if M2 = φ for all p ∈ P . (3) If a node x marked by M is not a leaf, fire a transition t, (p, t) ∈ F to construct a new node in the reachability tree marked as M 0 . Following the above three steps, we can construct the reachability tree from the SGN. The algorithm is similar with that in Stochastic Petri Nets. Algorithm-2: Find out the Nash Equilibrium The algorithm is to find the Nash Equilibrium of an action sequence with π ∗ for all the players. For every leaf node xi marked by Mi in the reachability tree and a token s such that there is a state p, Mi (p) = si , 1 ≤ i ≤ n in the reachability tree. Generally, there are multiple paths from the initial state to a leaf node. Assume xi is a leaf node, and there are wi separate paths from the root to xi . Let (i,w) (i,w) (i,w) t1 , t2 , .., tk(i,w) be the wth path from root node to leaf node xi . We define a leaf probability for the leaf node xi of the wth path as (i,w)
f (w) (xi ) = π(t1
(i,w)
) · π(t2
(i,w)
) · ... · π(tk(i,w) ).
(3)
Then the final utility vector for the system is (U1 , U2 , . . . , Un ) =
wi m X X [ f (a) (xi ) ∗ (h(a) (si ))],
(4)
i=1 a=1
where m is the number of leaves in the reachability tree. Note that h(a) (si ) of size n × 1 is the reward vector of the token in leaf node xi on the ath path, and n is the number of players as in the definition of SGN. According to the state of Nash equilibrium, every player has achieved his best when others don’t change their strategies. Thus, the problem is to find such
Stochastic Game Nets and Applications in Network Security
9
π that (U1 , U2 , . . . , Un ) is a Nash equilibrium for each player, which could be given as: max U = (U1 , U2 , . . . , Un ). (5) π
Note that, the above equation is a multi-objective optimization, which can be solved using the mathematical programming methods. Remark 1. The algorithm can be implement based on that of Stochastic Petri Nets, where the Nash equilibrium equation can be automatically achieved.
4
Applications in Network Security
In this section, we will apply the Stochastic Game Nets to model the attack and defence actions, and investigate the security properties based on the Nash equilibrium, and propose the optimum strategy for the computers at each stage to minimum the loss during computer attacks. We apply the SGN to three typical cases including the basic attack-defend case, the multi-round case and the multiplayer attack case. Three cases demonstrates the three fundamental structures, basic attack-defend case shows the sequence structure, multi-round case shows the structure of loop and, the multi-player attack case presents the modeling of the multiple tokes. First, we conclude the steps to apply the Stochastic Game Nets doing the security analysis, as the following six steps. Step 1: Determine the players in the game N ; Step 2: Present the targets of each player k, and construct each player’s action set T k ; Step 3: Define the reward function R for each transition; Step 4: Construct the SGN model; Step 5: Find the Nash equilibrium with respect to the SGN model, and propose optimum strategy accordingly; Step 6: Simplify the SGN model and, compute the stationary probability distribution and security and performance measures according to the transition firing rate λ. 4.1
Basic Attack-defend Case
The attack-defend system is the most general form among all the network attacks. In a basic attack-defence cast, there are two players, the defender and the attacker. For easy to illustrate, we choose a simple attack case in this subsection. In this case, a attacker will try to intrude a computer system, and the computer takes actions to defend. Assume attacker as Player 1 and the defender Player 2. The transition set of the Player 1, the attacker, is given in the following table. t1 t2 t5 t6 t7 http attack ftp attack web attack continue attack webserver sniffer
10
C. Lin et al.
The transition set of the Player 2, the defender, is also given in the following table. t3 t4 t8 t9 defend of http attack tolerant defend of web attack tolerant Thus, the SGN model of this attack-defend case is presented in Fig. 1. There the double circle state denotes the ending place, which means the ending of the attack. Remark 2. There the node can also be remove in other models, according to the system properties.
p1
: Attacker’s action t2
t1 p2 t3
: Defender’s action p3
t4 t7
p5
p4 t5
t6 p7
p6
t8 p8
t9 p9
Fig. 1. The SGN model for the attack-defend example
The initial marking M0 of SGN model is (1, 0, 0, 0, 0, 0, 0, 0, 0), which means there is only one token s in place p1 , and h(s) = (0, 0) as the reward vector. The reward function R of each action is presented as follows, while assume h(s) = (x, y). The reward functions are estimated from the statistic data and experiences. Rt1 (s(x, y)) = (x + 2, y − 3); Rt2 (s(x, y)) = (x + 3, y − 1); Rt3 (s(x, y)) = (x, y + 5π(t1 )); Rt4 (s(x, y)) = (x, y); Rt5 (s(x, y)) = (2, −2); Rt6 (s(x, y)) = (x + 2, y); Rt7 (s(x, y)) = (x + 1, y);
Stochastic Game Nets and Applications in Network Security
11
Rt8 (s(x, y)) = (−2, −10π(t8 ) + 6); Rt9 (s(x, y)) = (x + 1, y − 1). (6) Now, we can construct the reachability tree for the SGN model as shown in Fig. 2. There are 9 states in the reachability tree, including 4 leaf nodes. The marking of each states are given in the following table.
s1 t1 t2 s2 t4
t3 s4
s3 s5
t5
t6
s6
t7 s7
t8
t9
s8
s9
Fig. 2. Reachability tree for attack-defend example
s1 s2 s3 s4 s5 s6 s7 s8 s9
(1, 0, 0, 0, 0, 0, 0, 0, 0) (0, 1, 0, 0, 0, 0, 0, 0, 0) (0, 0, 1, 0, 0, 0, 0, 0, 0) (0, 0, 0, 1, 0, 0, 0, 0, 0) (0, 0, 0, 0, 1, 0, 0, 0, 0) (0, 0, 0, 0, 0, 1, 0, 0, 0) (0, 0, 0, 0, 0, 0, 1, 0, 0) (0, 0, 0, 0, 0, 0, 0, 1, 0) (0, 0, 0, 0, 0, 0, 0, 0, 1)
Now we can present the Nash equilibrium equation for this model. Let m = 4, and x1 = s4 , x2 = s4 , x3 = s8 , x4 = s9 . max U = (U1 , U2 , . . . , Un ) π
=
wi m X X [ f (a) (xi ) ∗ (h(a) (si ))], i=1 a=1
(7)
12
C. Lin et al.
where, h(x1 ) = (2, 5π1 − 3); h(x2 ) = (2, −2); (1) h (x3 ) = h(2) (x3 ) = (−2, −10π8 + 6); h(1) (x4 ) = h(2) (x4 ) = (5, −3); (8) Obviously, the constrains are π1 + π2 π3 + π4 π5 + π6 π7 π8 + π9
= 1; = 1; = 1; = 1; = 1. (9)
Now we are to find out the π = (π1 , π2 , .., π9 ). To solve the Nash equilibrium equation by using the programming techniques, we can get the optimum strategy as: π = (0.2, 0.8, 0.0325, 0.9674, 1, 0, 0.45, 0.55). The following table shows the detailed strategy for attacker and the defender in Nash equilibrium. 4.2
Multi-round Case
In this subsection, we apply SGN to a multi-round case, a typical bargain game which consists of multiple round of stages. This multi-round case is according to a typical repeated game in a common defence system. Nowadays, more and more systems and agents are trying to cooperate to get a more powerful system. However, each participator in the combined system requires an individual security strategy to satisfy its own target, and these strategies are actually inconsistent at most of times. Therefore, the participators would comes to a bargain game about the security strategy. The bargain game consists of multiple round of bargain until it gets to a acceptable solution for all the participators. We choose a two-participator system as an example in this subsection. In the bargain of security strategy, one player take turns to propose a solution, one at each round. The game would end when the two players agree on a strategy. In the kth (k = 1, 3, 5, . . . , 2m + 1, . . .), Player 1 shall propose a security strategy strk . If Player 2 agrees with strk , then Player 1 gets a utility of x·δ1k−1 and, Player 2 gets (1 − x) · δ2k−1 , otherwise, the game continues. δ1 , δ2 ∈ [0, 1] are discount factors for Player 1 and Player 2 respectively, which measures the bargaining cost on time scale, as defined in the game theory. In the kth (k = 2, 4, 6, . . . , 2m, . . .), it is Player 2’s turn to propose his security strategy strk . And, if Player 1 agrees with strk , then Player 2 gets a utility of y · δ1k−1 and, Player 2 gets (1 − y) · δ2k−1 , or the game continues.
Stochastic Game Nets and Applications in Network Security
13
We present the SGN model of this multi-round game in Fig. 3. Each player has two actions: accept the strategy and refuse. The initial marking M0 = (1, 0, 0), and there the reward vector of the initial token is written as h(s1 ) = (0, 0).
p3: ending place
t1: player 1 accepts
t3: Player 2 accepts
p1: Player 1’s turn
p2: Player 2’s turn
t2: Player 1 refuses
t4: Player 2 refuses
Fig. 3. The SGN model for the multi-round case
Then, the reachability tree can be given in Fig. 4, where s1 = (1, 0, 0), s2 = (0, 0, 1), s3 = (0, 1, 0).
s1 t1 s2
t2 t3
t4 s3
Fig. 4. The reachability tree for the multi-round case
According to a subgame perfect equilibrium, Player 1 would always propose 1−δ2 1−δ1 x = 1−δ , and Player 2 would propose y = 1−δ . 1 δ2 1 δ2 Now, at the kth round of Player 1’s turn, the utility function of the two players is: U (k,1) = (
(1 − δ2 )δ1k−1 (1 − δ1 )δ2 δ2k−1 , ). 1 − δ1 δ2 1 − δ1 δ2
(10)
14
C. Lin et al.
In the same way, at the kth round of Player 2’s turn, the utility function of the two players is written as: U (k,2) = (
(1 − δ2 )δ1 δ1k−1 (1 − δ1 )δ2k−1 , ). 1 − δ1 δ2 1 − δ1 δ2
(11)
There is only one leaf node in the reachability tree. And, we have the constrains: π1 + π2 = 1 π3 + π4 = 1 Thus, we have the following utility function to find the Nash equilibrium: max U = (U1 , U2 ) π
(1 − δ2 )δ1 δ1 (1 − δ1 )δ2 1 − δ2 (1 − δ1 )δ2 , ) ∗ π1 + ( , ) ∗ π3 ∗ π2 1 − δ1 δ2 1 − δ1 δ2 1 − δ1 δ2 1 − δ1 δ2 X (1 − δ2 )δ1k−1 (1 − δ1 )δ2 δ2k−1 + [( , ) ∗ π1 ∗ (π2 π4 )k−1 1 − δ1 δ2 1 − δ1 δ2
=(
k=3,5,7,...,2m+1,...
+(
(1 − δ2 )δ1 δ1k (1 − δ1 )δ2k , ) ∗ π3 ∗ (π2 π4 )k ]. 1 − δ1 δ2 1 − δ1 δ2 (12)
Therefore, we can easily find out that π1 = 1 is the Nash equilibrium. In contrast, π3 = 1 would be the Nash equilibrium while Player 2 gets the first chance to propose the security strategy. Therefore, we can see that accepting the counterpart’s price at the first round is a Nash Equilibrium in the multiround attack case. Meanwhile, the model and analysis approach of SGN in this subsection can be extended to other scenarios with repeated games. 4.3
Multi-player Attack Case
We focus on a multi-player attack case, where several players join the attack game in computer networks. We apply the SGN to describe and analyze the multi-player case. This example can be easily extended to other situations with more participators. The multi-player attack in this subsection is another attack-defend system which has three kinds of participators, including normal users, malicious users and the defender. In a real system, malicious users have choices of attacking or not attacking; and the normal users may appear to be attacking. Since the defender is hard to distinguish the normal user’s ”seemly attacking” and the malicious user’s real attacking. Thus it is a game among the three players. It is to be noted that this is a common case for IDS in defending various kinds attacks, such as DoS attack and DDoS attacks, and so on.
Stochastic Game Nets and Applications in Network Security
p1
p2 t2
t1 p3 t6
p6
t4
t3
p5
p4
t5 p7
15
t7
p8
t9
t8 t8 p9 p9
p10
: Users’ action : Defender’s action Fig. 5. The SGN model for the multi-player attack case
The SGN model of the multi-player attack case is shown in Fig. 5. There are three players in the model, the normal user as Player 1, the malicious user Player 2 and, the defender as Player 3. The transition set and reward functions are given in the following tables. users: normal user normal user malicious user malicious user transitions: t1 t2 t3 t4 actions: seemly ”attack” normal behavior normal behavior attack R(s(x, y, z)): (x, y, z) (x, y, z) (x, y, z) (x, y + 3, z) users: defender defender defender defender defender transitions: t5 t6 t7 t8 t9 actions: miss defend miss miss defend R(s(x, y, z)): (x, y, z) (x − 2, y, z − 3) (x, y, z) (x, y, z − 2) (x, y − 6, z + 2) Then, we can generate the reachability tree of this SGN model. While only two users are in the system, one normal user and one malicious user, there will be 36 states in the reachability tree. Usually, a system may have many users simultaneous. Thus, the reachability tree would be huge. Because of the space limitation, we omit the figure here. We would assume five normal users and one malicious user exists in the system in this example. Finally, we write the following utility function to find the Nash equilibrium, which can be generated by the algorithm-2 automatically. The final can be deduced as: max U = π1 π6 (2, 0, 3) + 5π4 π8 (0, 3, −2) + 5π4 π9 (0, −3, 2). π
(13)
The defender can not distinguish between the normal user and the malicious user, thus, we have same probability for the defender to choose a transition
16
C. Lin et al.
while it face an attack, which is π5 = π8 , and π6 = π9 . Meanwhile, since we can estimate the probability that a normal user seems to be attack from statistical data, the probability π1 and π2 can be predetermined. We assume π1 = 0.9 and π2 = 0.1 in this example. Obviously, we have π7 = 1. Solving the Nash equilibrium equation by using the programming techniques, we can get the optimum strategy as: π ∗ = (0.1, 0.9, 0.1875, 0.8125, 0.5, 0.5, 1, 0.5, 0.5), which means that it is a best choice for the defender to defend with the probability 0.8125, while it finds an attack regardless of its facticity. The multi-player attack example demonstrates the applications of SGN in multi-player games. It present a detailed approach to deal with the multi-players issues. This method can also applied to other scenarios with respect to multiplayer cases.
5
Conclusions and Future work
In this paper, we propose a novel modeling method, Stochastic Game Nets(SGN). SGN introduces the game theory into the Stochastic Petri Nets, and thus takes advantages of both stochastic game theory and Stochastic Petri Nets. SGN is an good method to model and deal with the game issues, which are more and more prevalent in various fields, especially network security issues. This new modeling tool could inherit the efficient and flexible modeling approach of Stochastic Petri Nets, and also make well use of the game-theoretical framework from game theory. The SGN method can be successfully used to model and analyze the network attacks, compute the Nash equilibrium and best-response strategies to defend the attacks. Meanwhile, the SGN model can also be applied to other areas, such as the channel allocation in wireless networks and so on. We believe that the proposed SGN opens a new avenue to deal with the game issues in computer networks. We in this paper present a definition of SGN, and apply it model and analyze three typical problems of network security. These applications demonstrate the soundness and efficiency of the SNG tool. However, our design is just a beginning. In this following, we discuss the future work along this approach. we will first study the the essential properties to consummate the SGN tool. One key factor in this approach is to compute the Nash equilibrium of the game issue. Since SGN proposes a more flexible formulation for the game issue, there may be more than one Nash equilibrium in the solution. Thus, we need to specify the multiple solutions of Nash equilibrium, or try to propose a bound for the multiple Nash equilibriums. At the same time, although we have validated some typical models of SGN with respect to network security, we would like to present a precise applicability scope of SGN, especially in a given field, like network security. Moreover, in terms of the modeling and analyzing approach, some simplification and approximation methods of SPN could be well conduced in SGN, we believe which would be a promising in handle the complex game issues. We will focus on how to split or combine the tokens and the according properties. Meanwhile, because the SGN model describe the transition process
Stochastic Game Nets and Applications in Network Security
17
of the whole system, we can do the performance and security analysis based on the SGN model and the predicted Nash equilibrium. Especially, the transition rate λ as defined in SGN = (N, P, T, π, F, R, λ, U, M0 ) is a characterization of each transition, which can describe the properties of the action, such as the execute capability. Thus, the λ can be used to estimate the performance measure of the system while the best-response strategies are chosen, including availability, survivability, performability, measures related to security, and so on.
References 1. D.M. Nicol, W.H. Sanders, and K.S. Trivedi. Model-basedevaluation: From dependability to security. IEEE Transactions on Dependability and Secure Computing, 1(1), 2004. 2. K. Lye and J.M. Wing. Game strategies in network security. In Proceedings of the 15th IEEE Computer Security Foundations Workshop, 2002. 3. A. Mahimkar and V. Shmatikov. On the advantage of network coding for improving network throughput. In Proceedings of 18th IEEE Computer Security Foundations Workshop, 2005. 4. P. Liu, W. Zang, and M. Yu. Incentive-based modeling and inference of attacker intent, objectives, and strategies. ACM Transactions on Information and System Security, 8(1):1–41. 5. X. Wang and M. Reiter. Defending against denial-of-service attacks with puzzle auctions. In Proceedings of IEEE Security and Privacy, 2003. 6. B. Bencsth, L. Buttyn, and I. Vajda. A game based analysis of the client puzzle approach to defend against dos attacks. In Proceedings of IEEE Conference on Software, Telecommunications and Computer Networks, 2003. 7. J. Xu and W. Lee. IEEE Transactions on Computers, (4):195–208. 8. R. Browne. C4i defensive infrastructure for survivability against multi-mode attacks. In Proceedings of 21st Century Military Communication-Architectures and Technologies for Information Superiority, 2000. 9. M. K. Molloy. IEEE Transactions on Computers, (9):913–917. 10. G. Ciardo, A. Blakemore, P.F. Chimento, and K.S. Trivedi. Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets, volume 48, chapter Linear Algebra, Markov Chains, and Queueing Models, IMA Volumes in Math. and its Applications, pages 145–191. Springer-Verlag, 1993. 11. L.S. Shapley. Stochastic games. In Proceedings of the National Academy of Sciences, pages 1095–1100. 12. J. Filar and K. Vrieze. Competitive Markov Decision Processes. Springer Verlag, New York, 1977. 13. G. Ciardo and K. S. Trivedi. Performance Evaluation, pages 37–59. 14. Z. Shan, C. Lin, F. Ren, and Y. Wei. Modeling and performance analysis of a multiserver multiqueue system on the grid. In Proceedings of the 9th International Workshop on Future Trends of Distributed Computing Systems, pages 337–343, May. 15. R.A. Howard. Dynamic Probabilistic Systems, Volume II: Semi-Markov and Decision Processes. John Wiley and Sons, New York, 1971.
