Hierarchical Network Formation Games in the Uplink of Multi-Hop Wireless Networks Walid Saad1 , Quanyan Zhu2 , Tamer Bas¸ar2 , Zhu Han3 , and Are Hjørungnes1 1
UNIK - University Graduate Center, University of Oslo, Kjeller, Norway, email: {saad,arehj}@unik.no 2 Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, USA, email: {zhu31,basar1}@illinois.edu 3 Electrical and Computer Engineering Department, University of Houston, Houston, USA, email:
[email protected] Abstract— In this paper, we propose a game theoretic approach to tackle the problem of the distributed formation of the hierarchical network architecture that connects the nodes in the uplink of a wireless multi-hop network. Unlike existing literature which focused on the performance assessment of hierarchical multi-hop networks given an existing topology, this paper investigates the problem of the formation of this topology among a number of nodes that seek to send data in the uplink to a central base station through multihop. We model the problem as a hierarchical network formation game and we divide the network into different hierarchy levels, whereby the nodes belonging to the same level engage in a noncooperative Nash game for selecting their next hop. As a solution to the game, we propose a novel equilibrium concept, the hierarchical Nash equilibrium, for a sequence of multi-stage Nash games, which can be found by backward induction analytically. For finding this equilibrium, we propose a distributed myopic dynamics algorithm, based on fictitious play, in which each node computes the mixed strategies that maximize its utility which represents the probability of successful transmission over the multi-hop communication path in the presence of interference. Simulation results show that the proposed algorithm presents significant gains in terms of average achieved expected utility per user up to 125.6% relative to a nearest neighbor algorithm.
I. I NTRODUCTION In recent years, hierarchical multi-hop network architectures have emerged as an essential aspect of emerging communication networks. For instance, while cellular-based communication has been the leading architecture in the past decade, recent advances in wireless networking, such as the need for distributed multihop communication imposed a hierarchical architecture on many next generation wireless networks. In fact, hierarchical structures have become ubiquitous in many networks such as broadband networks [1], cognitive radio networks [2], wireless local area networks (WLANs) [3], cellular networks [4], or sensor networks. In this regard, several IEEE workgroups have included hierarchical architectures in recent standards. For example, the IEEE 802.16j mobile multi-hop relay (MMR) task group introduced the hierarchical tree architecture as the base architecture in next generation IEEE 802.16 WirelessMAN (WiMAX) family of broadband networks [1]. Moreover, IEEE 802.11s standardized tree-based routing in mesh-based WLANs [3]. Communicating under hierarchical architectures faces several challenges, and existing literature has tackled many important aspects. In [5], given a wireless tree network, the authors propose a low complexity cooperative protocol that improves the average throughput of multi-hop upstream transmissions. The authors in [6] study the optimal deployment (that maximizes the throughput) of a single relay station for two-hop transmission in a hierarchical IEEE 802.16j network. In [7], the performance of This work was done during the stay of Walid Saad at the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, and was supported by the Research Council of Norway through projects 183311/S10, and 18778/V11.
multi-hop relaying is studied when dual relaying is performed. Further, the authors in [8] study the resource allocation problem for a multi-hop hierarchical cognitive network in the presence of an existing hierarchical topology. Other aspects of hierarchical wireless networks, such as optimal deployment of nodes, or routing are discussed in [4], [9–11]. In summary, existing literature mainly focused on the performance assessment, or resource allocation in multi-hop networks in various disciplines given an existing hierarchical topology. However, as the wireless nodes are generally selfish, there is a need for developing distributed algorithms that can model the interactions among the nodes for forming the hierarchical topology. In a nutshell, while existing literature thoroughly assessed the performance of multihop networks, one key question that remains unanswered is how to form the hierarchical topology in a wireless network, in a distributed manner. The main contribution of this paper is to propose a novel distributed network formation algorithm suitable for forming the network required for uplink communication in hierarchical multihop wireless networks. In this regard, given a network of wireless nodes that need to communicate to a central base station through multi-hop, we propose to divide the network into different hierarchy levels, whereby the nodes belonging to the same level engage in a non-cooperative Nash game with their strategy spaces being the nodes of the next level with whom they want to connect. The nodes at the same level interact by selecting the mixed strategies that maximize their individual utilities, in terms of probability of successful multi-hop transmission, while taking into account the interference at every node in the communication path. To characterize the solution to the game, we propose a novel equilibrium concept, the hierarchical Nash equilibrium, which is the equilibrium solution of a series of multi-stage Nash games, using backward induction among the stages (each Nash game corresponds to a particular hierarchy level). For finding this equilibrium, we devise a myopic dynamics algorithm based on fictitious play. In this algorithm, using empirical probabilities, the nodes update their mixed strategies given the previous network states, until convergence to a hierarchical Nash network in mixed strategies. We study the properties and characteristics of the proposed algorithm, as well as the resulting hierarchical structures. Simulation results show that the proposed algorithm presents significant gains in terms of average achieved expected utility per user reaching up to 125.6% compared to a nearest neighbor algorithm. The rest of this paper is organized as follows: Section II presents the system model and the game formulation. In Section III, we present the solution to the hierarchical network formation game, and we propose an algorithm for network
formation. Simulation results are presented and analyzed in Section IV. Finally, conclusions are drawn in Section V. II. S YSTEM M ODEL AND G AME F ORMULATION Consider a network of N users (or nodes) that are transmitting data to a central base station (BS) in the uplink. Let N denote the set of nodes in the network. For improving their performance, the nodes can transmit through other nodes, i.e., multi-hop transmission. Hence, the final architecture governing the network is a hierarchical architecture, whereby each node i ∈ N is connected to one or more nodes in N . For this purpose, we separate the nodes into l hierarchical levels (or stages1 ) according to some criterion, for example, by geographical distances to the BS (other division criteria may also be used along with our proposed algorithm). For multiple access at every hop, we consider a CDMA-based transmission2 . Let L = {1, 2, · · · , l} denote the set of levels in the network. The levels are ordered in such a way that the nodes at level h1 are considered farther away from the BS than nodes at level h2 when h1 < h2 , ∀h1 , h2 ∈ L. The nodes belonging to a hierarchy level h ∈ L form a set Nh ⊂ N . The different Nh , h ∈ L are mutually exclusive sets, i.e., Ni ∩ Nj = ∅, for i 6= j, i, j ∈ L and ∪lk=1 Nk = N . We also denote nh = |Nh | as the number of nodes at level h. The main objective of this paper is to devise an algorithm that allows the nodes at each level to select their next hops, in a distributed manner. For modeling these interactions among the nodes seeking to form the uplink tree structure, network formation games provide a suitable framework [12–14]. In such games, several independent decision makers (players) interact in order to form a network graph G = (N , E), where N is the set of nodes or vertices and E is the set of directed edges or arrows. The essence of network formation is to find the best set of directed edges E among all the possible configurations. Instead of an exhaustive search, we aim at a network formation scheme that allows distributed decision-making at each node. Depending on the goals of each node, a final network graph forms as a result of individual nodes’ decisions. Thus, we model the uplink tree formation problem as a network formation game for finding the directed uplink edges through which the nodes can transmit to the BS. For each level h ∈ L, a strategic game in normal form is defined by Ξh = hNh , (Ai,h )i∈Nh , (Ui,h )i∈Nh i, where Nh is the set of players (nodes) of the game, Ai,h is the set of actions of a player i ∈ Nh , and Ui,h is the utility function of a player i ∈ Nh . In the proposed uplink network formation game, the action space Ai,h of a node i ∈ Nh is given by Ai,h = Nh+1 , and, thus, |Ai,h | = |Nh+1 | = nh+1 . At the last stage h = l, Al is simply the set of singleton, comprising of the BS. We denote (ai , m) ∈ Am as the node at level m + 1 chosen by i ∈ Nh at level h. If i ∈ Nh and m = h, we call the choice (ai , m) made by i direct. If m > h, then, we call the choice (ai , m) indirect as the choice of node i at a higher level is made via some other intermediate nodes. For example, when m = h + 1, the choice (ai , m) is indirectly made by (ai , h), which is directly chosen 1 We
use the terms “level” and “stage” interchangeably. multiple access techniques can be used without loss of generality in the analysis done in the rest of this paper. 2 Other
by i at level h. For m < h, we assume that (ai , m) = i, the node itself. Given this notation, the average SINR of any node k ∈ Nm , denoted by (ai , m − 1) (node k is any node selected by node i ∈ Nh at level m), received at node (ai , m), the direct choice of node i, is given by [15], [16] (a ,m) P(ai ,m−1) · g(aii,m−1) (a ,m) Γ(aii ,m−1) = S · , (1) (a ,m) I(aii,m−1) where P(ai ,m−1) is the transmit power of node (ai , m − 1), S is ³ ´−µ (a ,m) (a ,m) is the the spreading factor, and g(aii,m−1) = κ · D(aii,m−1) channel gain between the node at level m and its selected node (a ,m) at level m + 1 with D(aii,m−1) the distance between the nodes (ai , m−1) and (ai , m), µ the path loss exponent, and κ the path (a ,m) loss constant. The term I(aii,m−1) is the intra-level interference (and noise) perceived by (ai , m − 1) at (ai , m) from nodes at level m + 1 that are connected to (ai , m) and is given by X (a ,m) (a ,m) 2 I(aii,m−1) = σm + Pk · gk i , (2) (a ,m)
k∈Nm i
−{(ai ,m−1)}
2 where σm P
is the Gaussian noise variance at level m, while the (a ,m) term P · gk i represents the inter(a ,m) k∈Nm i −{(ai ,m−1)} k ference from the other nodes at level m that are connected to (a ,m) node (ai , m) (Nm i is the set of players at level m connected to (ai , m)) with P being k ³ ´ the transmit power of node k and (a ,m)
(a ,m)
−µ
gk i = κ · Dk i being the channel gain between k and (ai , m). Moreover, we define the following utility function Ui,h of a node i ∈ Nh from the starting stage h to final stage l. l Y (a ,m) Ui,h = Pr(aii ,m−1) (3) (a ,m)
m=h
where Pr(aii ,m−1) is the probability of having the average received SINR at node (ai , m) from node (ai , m − 1) larger than a target ν(ai ,m−1) (desired by (ai , m − 1)), i.e., the probability of successful transmission for a single hop, and is given by the following approximation (in a Rayleigh fading channel) [17] (a ,m) Pr(aii ,m−1)
−
=e
ν(a ,m−1) i (ai ,m) (ai ,m−1)
Γ
,
(4)
where ν(ai ,m−1) is the target received SINR at (ai , m), and (a ,m) Γ(aii ,m−1) is given by (1). Hence, the utility function Ui,h : ´ ³Q Ql nm A → [0, 1] of player i ∈ Nh corresponds to m j=1 m=h the multi-hop probability of successful transmission from the starting stage h to the final stage l. Note that the utility of a player i depends on its own strategies as well as the strategies of its parent nodes in the tree. In a nutshell, the action of any node at level m is to choose a node at level m + 1 and the corresponding utility is dependent on the actions of the nodes at level m + 1. Moreover, we consider that each node requires a certain amount of money to be paid for offering its services, i.e., we consider a pricing scheme in the network. Let c(ai ,m) ∈ R++ be the cost that node i ∈ Nh has to pay per unit of traffic if i transmits through (ai , m) ∈ Am . If m = h, the cost is direct; otherwise, it is said to be indirect. We consider only direct costs, i.e., c(ai ,m) = 0 if m 6= h. When m = h, a cost is incurred if a connection is made. Hence, the nodes pay only to the nodes at
the next level. Thus, the utility function U i,h to transmit from h to l for unit traffic becomes (a ,m) l −ν /Γ i Y e (ai ,m−1) (ai ,m−1) U i,h = , (5) c(ai ,h) m=h which corresponds to the success rate per unit money. By taking the logarithm of both sides in (5), we get l l X X ν(ai ,m−1) ei,h = − ui,m −˜ c(ai ,h) , (6) U −˜ c = (a ,h) i (ai ,m) m=h Γ(ai ,m−1) m=h ν i ,m−1) where u ˜i,m is the stage utility defined by u ˜i,m := − (a , (ai ,m) Γ(a
i ,m−1)
and c˜(ai ,h) = ln c(ai ,h) . Therefore, in the transformed game e h = hNh , (Ai,h )i∈N , (U ei,h )i∈N i, each node i at the level Ξ h h h attempts to optimize ei,h . (7) max U (ai ,h)∈Ah
Since all nodes in N attempt to choose an action to optimize their payoffs at the level they belong, the utility of i ∈ Nh at ei,h , is dependent on the utility obtained by j ∈ Nh+1 , level h, U e Uj,h+1 , at the next level h + 1, which is an outcome of the game Ξh+1 , where each player j at level h + 1 solves ej,h+1 . max U (8) (aj ,h+1)∈Ah+1
This dependence on the next level is not present at the last stage e l , where each node k ∈ Nl optimizes Ξ ek,l = max u ˜k,l − c˜(ak ,l) . (9) max U (ak ,l)∈Al
Fig. 1. An illustrative example of the proposed hierarchical network model.
The action profile (a, h)∗ is said to be a stage-h Nash equilibe ∗ ((a, h)∗ ) is called rium, and the payoff at the equilibrium U i,h the stage-h optimal payoff to player i. Note that in (10), we shorthand the dependence of the payoff ei,h into the actions at its own level, as the the choices of of U actions made by nodes at its own level directly determines the utility. To find the Nash equilibrium defined in (10), the game at a particular stage l0 can be decomposed as follows. ∗ ei,l U 0
=
max
(ai ,l0 )∈Al0
(ak ,l)∈Al
The proposed system model and the corresponding notations are illustrated in Fig. 1. In this figure, a network of 9 users divided into 3 levels is considered, hence, N1 = {1, 2, 3}, N2 = {4, 5, 6}, N3 = {7, 8, 9}, and A1 = {4, 5, 6}, A2 = {7, 8, 9}, and A3 = {BS}. From the chosen connections, we observe that (a1 , 1) = 4, (a2 , 1) = 6, (a3 , 1) = 6 and (a4 , 2) = 7, (a5 , 2) = 8, (a6 , 2) = 8 and (a7 , 3) = (a8 , 3) = (a9 , 3) = BS. The utility e1,1 = u of node 1 at level 1 is given by U ˜1,1 + u ˜1,2 + u ˜1,3 − c˜4 = e e e e7,3 = u u ˜1,1 + U4,2 − c˜4 , U4,2 = u ˜4,2 + U7,3 − c˜7 , and U ˜7,3 − c˜BS . Finally, note that the analysis and algorithm derived in the rest of this paper are not limited to the case of a single BS at the last stage, as shown in Fig. 1, but are also applicable to the case of multiple receiving nodes at the last stage. III. H IERARCHICAL N ETWORK F ORMATION G AME : S OLUTION AND DYNAMIC L EARNING A LGORITHM A. Hierarchical Network Formation Game Solution For solving the game presented in Section II, the proposed network formation game is defined by a sequence of none h }h=1,··· ,l . We denote by (ai , h)∗ cooperative Nash games {Ξ the optimal action Qnhchosen at level h by a player i ∈ Nh directly. Let (a, h) ∈ k=1 Nh be an action profile at level h, i.e., (a, h) = [(ai , h)]i∈Nh . For convenience, we denote (a−i , h) = [(aj , h)]j6=i,j∈Nh and hence (a, h) = [(ai , h), (a−i , h)]. Given a level l0 between the initial level h and the final level l, we define Nash equilibrium of the hierarchical network formation e h }h=1,··· ,l as follows. game {Ξ e h }h=1,··· ,l be a hierarchical network Definition 1: Let {Ξ formation game. An action profile (ai , h)∗ ∈ Ah+1 , for i ∈ Nh , and h ∈ L is a hierarchical Nash equilibrium if ei,h ((ai , h)∗ , (a−i , h)∗ ) ≥ U ei,h ((ai , h), (a−i , h)∗ ), U (10) ∀i ∈ Nh , (ai , h) ∈ Ah+1 , h ∈ L.
= =
max 0
(ai ,l )∈Al0
˜i,l0 U l X m=l0 +1
u ˜∗(ai ,l0 ),m +
ν(ai ,l0 −1) (a ,l0 )
Γ(aii ,l0 −1)
− c˜(ai ,l0 ) ,
0 e ∗ 0 0 + ν(ai ,l 0−1) − c˜(a ,l0 ) , U (ai ,l ),l +1 i (a ,l ) (ai ,l )∈Al0 Γ(aii ,l0 −1)
max 0
(11)
0 e∗ 0 0 for all i ∈ Nl0 , where U (ai ,l ),l +1 is the stage-l + 1 optimal 0 payoff to (ai , l ). The payoff of the game at the last stage is νi,l−1 ∗ ei,l U = (a ,l) − c˜(ai ,l)∗ , (12) Γ(aii ,l−1) where {(ai , l)∗ , i ∈ Nl } is a Nash equilibrium at level l. Note (a ,l) that Γ(aii ,l−1) depends (through the interference term) on the actions of all the nodes at level l at the Nash equilibrium, i.e., ((a1 , l)∗ , (a2 , l)∗ , · · · , (anl , l)∗ ). Hence, by (11), we can obtain the Nash equilibrium of the game by starting with the final stage l0 = l and solve the game iteratively by backward induction till stage l0 = h. Such decomposition of the game into stages is possible because the players at each level are different from each others and the game at a level l0 is independent of the games at level k < l0 (in contrast the game at l0 is dependent on higher levels k > l0 through the utility). In a nutshell, we can easily proceed as follows for solving the hierarchical network formation game. Proposition 1: Consider a network formation game e h }h=1,··· ,l . An action profile (ai , h)∗ ∈ Ah+1 , for i ∈ Nh , {Ξ h ∈ L is a hierarchical Nash equilibrium if and only if it solves (11) recursively from stage h = l backwards to stage h = l0 . Thus, Proposition 1 provides an analytical way to solve for the hierarchical Nash equilibrium using backward induction. As the Nash equilibrium in pure strategies may not exist, we generalize the above concepts to the case where the nodes at every level use mixed strategies, since the mixed Nash
0
0
equilibrium always exists [18]. Let pi,l ∈ P i,l be the mixed strategies of i-th player at stage l0 ,¯ where ¯ X 0 0 0 P i,l = {pi,l ∈ Rnl0 ¯¯ pi,l (ai ,l0 ) = 1, (ai ,l0 )∈Al0 0
0 0 0 pi,l (ai ,l0 ) ≥ 0, ∀(ai , l ) ∈ Al }, i ∈ Al .
e h }h=1,··· ,l be a hierarchical network Definition 2: Let {Ξ i,h formation game. A mixed-strategy profile pi,h∗ , for (ai ,h) ∈ P i ∈ Nh , and h ∈ L is a hierarchical Nash equilibrium in mixed strategies if the following nh conditions are satisfied X X ei,h ··· p1,h∗ p2,h∗ · · · pnh ,h∗ U (a1 ,h) (a2 ,h)
(a1 ,h)∈Ah
≥
X
(anh ,h)∈Ah
(a1 ,h)∈Ah
(a1 ,h)∈Ah
X
···
(a1 ,h)∈Ah
X
2,h∗ nh ,h∗ e p1,h (a1 ,h) p(a2 ,h) · · · p(an ,h) Ui,h , h
(anh ,h)∈Ah
X
≥
X
···
······························ ei,h p1,h∗ p2,h∗ · · · pnh ,h∗ U (a1 ,h) (a2 ,h)
(anh ,h)∈Ah
X
···
(anh ,h)
(anh ,h)
2,h∗ nh ,h e p1,h∗ (a1 ,h) p(a2 ,h) · · · p(an ,h) Ui,h , h
(anh ,h)∈Ah
∀i ∈ Nh , h ∈ L. i,h∗
The mixed-strategy profile p is said to be a mixed stage-h Nash equilibrium, and the payoff at the equilibrium X X 2,h∗ nh ,h∗ e ∗ bi,h U = ··· p1,h∗ (a1 ,h) p(a2 ,h) · · · p(an ,h) Ui,h (a1 ,h)∈Ah
(anh ,h)∈Ah
h
is called the stage-h optimal payoff to player i. We can use a similar decomposition as in (11) to show that the mixed Nash equilibrium can also be found by backward induction from the final stage game up till the initial stage h. B. Hierarchical Network Formation Algorithm For finding the hierarchical Nash equilibrium, we propose a dynamics algorithm that allows a distributed formation of the hierarchical network structure. The proposed dynamics assume that each node is myopic, in the sense that the nodes aim at improving their payoffs considering only the current, and previous states of the network. The proposed network formation algorithm consists of three phases: Hierarchy formation, hierarchical fictitious play, and data transmission. In the first phase, the hierarchy in the network is formed. In this paper, we consider a distance-based hierarchy whereby each hierarchy level corresponds to the area between two circles centered at the BS and with constant radii with the upper most level corresponding to the area within a circle centered at the BS with a specified radius. This hierarchy division can be performed by the BS, and is assumed as fixed throughout the network operation. Once the hierarchy is formed, Phase I continues by allowing each player belonging to a certain level l selects the nearest neighbor in level l + 1. Hence, the initial network consists of a nearest neighbor algorithm with predetermined hierarchy levels. Once the network is initiated, the hierarchical fictitious play phase of the algorithm begins. In this phase, the actual network formation process start. Thus, for forming the network in Phase II, we use fictitious play [19] at each stage
l0 ∈ L0 to find the mixed Nash equilibrium at that level. Let pi,l (ai ,l0 ) (k) be the empirical probability that a player i ∈ Nl0 , i = 1, 2, · · · , nl0 at a certain level l0 chooses an action (ai , l0 ) ∈ Al0 at the k-th iteration of the algorithm. Denote 0 n0l ,l0 0 pil0 (k) = [p1,l (a1 ,l0 ) (k), · · · , p(a 0 ,l0 ) (k)] a nl -dimensional vector n l
of i-player’s empirical mixed strategy at time k. At each iteration, the players update their strategy pil0 (k) as follows: ¢ 1 ¡ i pil0 (k + 1) = pil0 (k) + vl0 (k) − pil0 (k) , (13) k+1 i where vli0 (k) = [v(a is an nl0 -dimensional 0 (k)](ai ,l0 )∈Al0 i ,l ) vector with vai ,l0 (k) = 1 if at time k, i-th player chooses the action (ai , l0 ) and vai ,l0 (k) = 0, otherwise. Since a player chooses only one action at every step, vli0 (k) is a vector with the entry that corresponds to the chosen action (ai , l0 ) being 1 while the remaining terms equal 0. In the hierarchical fictitious play phase, the action (ai , l0 ) of the i-th node at time k is the best response to the observed empirj ical strategies of the opponents. Let p−i l0 (k) = [pl0 (k)]j∈Nl0 ,j6=i and qi,l0 (k) denote the action taken by i−th node at time k given by qi,l0 (k) = arg max(ai ,l0 )∈Al0 gi ((ai , l0 ), p−i ) and ql0 (k) = [qi,l0 (k)]i∈Nh , where e gi ((ai , l0 ), p−i l0 ) = Ep−i (U(ai ,l0 ),l0 ) =
X (a1 ,l0 )∈Al0 0 p1,l a1 ,l0 , · · ·
···
X
X
(ai−1 ,l0 )∈Al0 (ai+1 ,l0 )∈Al0 0 i+1,l0 , pi−1,l ai−1 ,l0 , pai+1 ,l0 , · · ·
X
···
(an0 ,l0 )∈Al0 l
n ,l0 e , panl0 ,l0 U (ai ,l0 ),l0 (k) 0
(14)
l
and U(ai ,l0 ),l0 (k) is the payoff at step k, which is dependent on the payoff matrix U(ai ,l0 ),l0 +1 (k) at step k, i.e., νi,l0 −1 e(a ,l0 ),l0 (k) = U e(a ,l0 ),l0 +1 (k) + U − c˜(qi,l0 (k)) , i i (ai ,l0 ) Γ(ai ,l0 −1) (ql0 (k)) and at the terminal stage νi,l ei,l (k) = U − c˜(qi,l (k)) . (15) (ai ,l) Γ(ai ,l−1) (ql (k)) The optimal action qi,l0 (k) taken at iteration k determines the vector vli0 at the following iteration and, hence, allows to update the empirical frequency. This iterative process continues until the empirical frequencies converge to the hierarchical Nash equilibrium. Note that it is well-known that whenever fictitious play converges, it converges to a Nash equilibrium [19], [20]. Hence, in our model, by using fictitious play at every level, we ensure that our algorithm reaches the mixed Nash equilibria at every level when it reaches steady-state (consequently, the network converges to a hierarchical Nash equilibrium). In general, fictitious play algorithms have been proven to converge in almost all cases, and many modification schemes have also been proposed to ensure convergence [19], [20]. Upon convergence of Phase II to a hierarchical Nash equilibrium, the nodes are ready to start their transmission, in the last phase of the algorithm. In this phase, the nodes have already computed their mixed strategies, and hence, they choose their next hop based on the probabilities that resulted from Phase II. Note that although the nodes mix between different actions with different probabilities, at any given time, the network is
structured into a tree architecture rooted at the BS. The proposed algorithm is summarized in Table I. The proposed network formation algorithm in Table I can be implemented in a distributed manner. For hierarchy formation, the BS can broadcast this information to all the nodes at the beginning of all time, and hence, the remaining phases of the algorithm can be performed with no further reliance on the BS (since the hierarchy is fixed, and based on distance). For instance, for the hierarchical fictitious play algorithm, at every time k, each node i at a level l0 needs only to know the payoffs of its parent nodes, i.e., the nodes at level l00 > l0 that link node i to the BS, from the previous time instant k − 1, as well as the empirical probabilities of the nodes competing with i at the same level l0 . This information can be easily gathered by the nodes by observing the past actions of their opponents, as well as the payoffs of the players at the next level, in a distributed manner, without relying on the BS. The last phase of the algorithm is simply a transmission phase where each node sends its data to the BS through multi-hop, and using the mixed strategies resulting from hierarchical fictitious play. IV. S IMULATION R ESULTS AND A NALYSIS For simulations, the following network is set up: the BS is placed at the origin of a 2 km ×2 km square with the nodes randomly deployed in the area around the BS. For simulations, we consider three levels of hierarchy as follows: The first level consists of nodes randomly deployed in the area between two circles centered at the BS and with radii 0.6 km and 1 km, the second level consists of nodes randomly deployed in the area between two circles centered at the BS and with radii 0.3 km and 0.6 km, and the third level consists of nodes randomly deployed within the area of a circle centered at the BS with radius 0.3 km. The nodes’ transmit power is set to Pi = 10 mW, ∀i ∈ N , the target SINR is set to νi = 10 dB ∀i ∈ N , the noise level is set to σ 2 = −90 dBm for all levels, the path loss constant is set to κ = 1, while the path loss exponent is set to µ = 3. The spreading factor is set to S = 64 which is typical for data services in the uplink of CDMA networks [16]. The pricing parameter is set to ci = 1, ∀i ∈ N . In Fig. 2, we randomly deploy N = 10 nodes within the BS area, with n1 = 4 nodes in the first hierarchy level, and n2 = n3 = 3 nodes in the second and third hierarchy levels.
0.4
2
8
0.2 0.7565 0 Position in y (km)
TABLE I P ROPOSED HIERARCHICAL NETWORK FORMATION ALGORITHM Phase I - Hierarchy Formation a) The network is divided into l hierarchy levels, e.g., based on the distance to the BS. b) Each node at a level l0 selects the nearest neighbor in level l0 + 1 (initial network state). Phase II - Hierarchical Fictitious Play The nodes engage in a hierarchical network formation game. repeat (Iteration k) a) Each node i ∈ N selects its best response based on the payoffs and empirical probabilities of the opponents in iteration k − 1 as per (14). b) Node i updates its mixed strategies based on (13). until convergence to the hierarchical Nash equilibrium. Phase III - Data Transmission The hierarchical Nash architecture is formed and each node at level l0 transmits to the BS using the nodes (strategies) at level l0 + 1 with different probabilities over time (mixed strategies).
0.2435
9
1
0.8348 10
3
0.5565 0.2783
1 0.1652
−0.8 −1
1
1
7
0.8348
−0.4
−0.6
6 0.1652
−0.2
5 0.1652
4
3 Hierarchy Levels: Level 3: nodes 8, 9, and 10 Level 2: nodes 5, 6, and 7 Level 1: nodes 1, 2, 3, and 4 −0.8
−0.6
−0.4
−0.2 0 0.2 Position in x (km)
0.4
0.6
0.8
1
Fig. 2. Snapshot of a tree topology formed using the proposed algorithm with 10 randomly deployed nodes in three hierarchy levels. For each node, the solid arrows show the strategies with largest probabilities, while the dashed arrows indicates the remaining strategies.
The network formation game starts with the nearest neighbor tree whereby, within the same level, each node is connected to the nearest node in the next level with a probability of 1. Fig. 2 shows the hierarchical Nash network resulting from our algorithm with the solid arrows indicating the strategies of each node that have the highest probability of selection, and the dashed arrows indicating the remaining strategies with non-zero probability. On one hand, as a result of the network formation game, Nodes 3, 5, and 6 choose to connect to their nearest neighbor in the next hierarchy level (respectively, Nodes 6, 10 and 9) with a probability of 1. On the other hand, Nodes 1, 2 and 7, choose between two different strategies with different probabilities, while Node 4 mixes between three strategies. For instance, Nodes 1 and 2 choose their mixed strategies in such a way that a high probability is given for connecting to the nearest node, i.e., Node 7, while a small probability is given to a connection with Node 5. Further, although Node 4 is at an almost equal distance from both Nodes 5 and 6 (Node 6 is slightly closer to Node 4 than Node 5), it chooses to connect to Node 6 with a probability of 0.5565, which is much higher than the probabilities of 0.1658 and 0.2783 with which Node 4 selects Nodes 5 and 7, respectively. This choice by Node 4 is justified by the fact that this node attempts to minimize the interference with Nodes 1 and 2 that may occur at Nodes 5, and 7. In summary, Fig. 2 summarizes how the different nodes in a wireless network select mixed strategies and self-organize into a hierarchical Nash network. Fig. 3 provides an insight on the convergence time of the algorithm. In this figure, we show the probabilities of all three strategies of Node 4 from Fig. 2 as the number of iterations increases. Node 4 starts by being connected with its nearest neighbor, Node 6, with a probability of 1. As the network formation game evolves, Node 4 adapts its mixed strategies by decreasing the probability on Node 6 and increasing the probabilities on selecting Nodes 5 and 7. The mixed strategies finally converge towards probabilities of 0.1658, 0.5565, and 0.2783 for
1
1
0.9 Average expected payoff per user
Probability (mixed strategy)
0.8 0.7 0.6 0.5 0.4 0.3
0.8 0.7 0.6 0.5 0.4 0.3
0.2
0.2
0.1
0.1
0 0
Proposed hierarchical network formation algorithm Nearest neighbor algorithm Star topology (direct node − BS connection)
0.9
Probability that Node 4 connects to Node 5 Probability that Node 4 connects to Node 6 Probability that Node 4 connects to Node 7
20
40
60 80 Number of iterations
100
120
Fig. 3. Convergence of the mixed strategies of Node 4 from Fig. 2.
connecting to Nodes 5, 6, and 7, respectively. The convergence is ensured at around 25 iterations, hence, the convergence time of the proposed algorithm is quite reasonable. Similar convergence results can be seen for all nodes of Fig. 2, but are omitted due to space limitation. In Fig. 4, we show the average achieved expected utility per node as the number of nodes, N , in the network increases (nodes are equally divided between the 3 hierarchy levels). The results are averaged over random positions of the nodes. We compare the performance of the proposed algorithm against the nearest neighbor algorithm whereby each node is connected to the nearest node belonging to the next level, as well as the star topology whereby each node is directly connected to the BS (no hierarchy). Fig. 4 clearly shows that as the number of nodes N increases, the performance of all three algorithms decreases due to the increased interference. However, the proposed hierarchical network formation algorithm presents a slower slope of decrease compared to the other two algorithms. In addition, the proposed algorithm presents a significant performance advantage over the other two algorithms, increasing with the number of the nodes, and reaching up to 125.6% relative to the nearest neighbor algorithm (at N = 21, i.e., 7 nodes per level). V. C ONCLUSIONS In this paper, we proposed a distributed hierarchical network formation game approach for the construction of the uplink network topology in a wireless multi-hop network. For this purpose, we proposed to divide the network into a number of hierarchy levels, whereby the nodes belonging to the same level play a non-cooperative Nash game. For solving the game and finding the network topology, we proposed a novel equilibrium concept, the hierarchical Nash equilibrium, which is the solution to a series of multi-stage Nash games using backward induction among the stages. For reaching this equilibrium, a myopic fictitious play based algorithm is proposed for allowing the nodes to compute their mixed strategies, in a distributed manner. Through the proposed algorithm, the nodes can self-organize into a hierarchical multi-hop architecture, while improving their utility, in terms of probability of successful transmission, in the presence of interference. Finally, the proposed hierarchical
0 3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 Number of nodes (N)
Fig. 4. Performance assessment of the proposed distributed hierarchical network formation algorithm through the average expected payoff per user vs. number of users N in the network (average over random positions of the nodes).
network formation game can easily extend to other applications such as multi-hop cognitive radio, sensor networks, and others. R EFERENCES [1] The Relay Task Group of IEEE 802.16, “The p802.16j baseline document for draft standard for local and metropolitan area networks,” 802.16j06/026r4, Tech. Rep., Jun. 2007. [2] D. Niyato, E. Hossein, and Z. Han, Dynamic spectrum access in cognitive radio networks. Cambridge, UK: Cambridge University Press, 2009. [3] IEEE 802.11, “Draft amendment to standard for information technology IEEE 802.11:ESS mesh networking. ieee,” P802.11s/D1.08, Tech. Rep., Jan. 2008. [4] Y. D. Lin and Y. C. Hsu, “Multihop cellular: a new architecture for wireless communications,” in Proc. IEEE Conf. on Comp. Comm. (INFOCOM), Tel aviv, Israel, Mar. 2000. [5] A. de Baynast, O. Gurewitz, and E. W. Knightly, “Cooperative strategies and optimal scheduling for tree networks,” in Proc. IEEE Conf. on Comp. Comm. (INFOCOM), Alaska, USA, May 2007. [6] B. Lin, P. Ho, L. Xie, and X. Shen, “Optimal relay station placement in IEEE 802.16j networks,” in Proc. International Conference on Communications and Mobile Computing, Hawaii, USA, Aug. 2007. [7] ——, “Relay station placement in IEEE 802.16j dual-relay MMR networks,” in Proc. Int. Conf. on Communications, Beijing, China, May 2008. [8] H. Shiang and M. van der Schaar, “Distributed resource management in multi-hop cognitive radio networks for delay sensitive transmission,” IEEE Trans. on Vehicular Technology, vol. 58, no. 2, pp. 941–953, Feb. 2009. [9] Y. Yu, S. Murphy, and L. Murphy, “A clustering approach to planning base station and relay station locations in IEEE 802.16j multi-hop relay networks,” in Proc. Int. Conf. on Communications, Beijing, China, May 2008. [10] D. Niyato, E. Hossein, D. Kim, and Z. Han, “Joint optimization of placement and bandwidth reservation for relays in IEEE 802.16j mobile multihop networks,” in Proc. Int. Conf. on Communications, Dresden, Germany, Jun. 2009. [11] W. Qiu, E. Skafidas, and P. Hao, “Enhanced tree routing for wireless sensor networks,” Ad hoc networks, vol. 7, no. 3, pp. 638–650, To appear, May 2009. [12] M. O. Jackson, “A survey of models of network formation: stability and efficiency,” Working Paper 1161, California Institute of Technology, Division of the Humanities and Social Sciences, Nov. 2003. [13] G. Demange and M. Wooders, Group formation in economics: networks, clubs and coalitions. Cambridge, UK: Cambridge University Press, Mar. 2005. [14] J. Derks, J. Kuipers, M. Tennekes, and F. Thuijsman, “Local dynamics in network formation,” in Proc. Third World Congress of The Game Theory Society, Illinois, USA, Jul. 2008. [15] R. D. Yates, “A framework for uplink power control in cellular radio systems,” IEEE J. Select. Areas Commun., vol. 13, no. 7, pp. 1341–1347, Sep. 1995. [16] A. Toskala and H. Holma, HSDPA/HSUPA for UMTS. New Jersey, USA: Wiley, John & Sons Incorporated, Jun. 2006. [17] J. Proakis, Digital Communications. McGraw-Hill, 4th edition, Aug. 2000. [18] T. Bas¸ar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed. Society for Industrial Mathematics, 1995. [19] D. Fudenberg and D. Levine, The Theory of Learning in Games. Cambridge, MA, USA: MIT Press, 1998. [20] J. Shamma and G. Arslan, “Unified convergence proofs of continuous-time fictitious play,” IEEE Trans. Automatic Control, vol. 49, no. 7, pp. 1137– 1142, Jul. 2004.
