Resource Allocation in Software Defined Fog ... - ACM Digital Library

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

Resource Allocation in Software Defined Fog Vehicular Networks Yaomin Zhang

Haijun Zhang

Keping Long

Beijing University of Chemical Technology Beijing, China [email protected]

University of Science and Technology Beijing Beijing, China [email protected]

University of Science and Technology Beijing Beijing, China [email protected]

Xiaoming Xie

Victor C.M. Leung

Beijing University of Chemical Technology Beijing, China [email protected]

University of British Columbia Vancouver, Canada [email protected]

ABSTRACT

ACM Reference format: Yaomin Zhang, Haijun Zhang, Keping Long, Xiaoming Xie, and Victor C.M. Leung. 2017. Resource Allocation in Software Defined Fog Vehicular Networks. In Proceedings of DIVANet’17, November 21–25, 2017, Miami, FL, USA, , 6 pages. DOI: http://dx.doi.org/10.1145/3132340.3132357

Vehicular network is an important application scenario of the fifth generation (5G) mobile communications. Due to the increasing number of vehicles and the users’ various requirements, resource allocation problem in vehicular networks becomes more serious and has attracted researchers’ attention. In this paper, we investigate the resource allocation in software defined fog vehicular networks where we formulate the problem as a mean-field game (MFG). We present a state space function by considering both interference factors and energy availability. Then the problem can be modeled as a cost minimization with the constraint of state space. Through the mean-field approximation method, we derive the corresponding Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations. A joint finite difference algorithm is proposed to solve the coupled HJB and FPK equations. The numerical results are presented to show the effectiveness of the proposed algorithm.

1

INTRODUCTION

The future 5G communication is a more flexible, intelligent, efficient and open network system based on software defined network (SDN) and fog computing technology. Benefitted from these features, the Internet of things will be the main driver of the communication network. As an important extending of the Internet of things, vehicular network has envisioned to improve the transportation efficiency, guarantee the traffic safety, reduce incidents, and mitigate of traffic congestion [1]. There are two main types of services in the vehicular networks: vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) [2][3]. In the V2I mode, vehicles can access network through roadside units (RSUs) which provide safety and efficient transmit with minimal latency. V2V communication is based on Device-to-Device (D2D) technology, in which user data can be transmitted directly between terminals, thereby improving network performance and user experience. SDN is a promising technique for the 5G mobile communications, which decouples the control layer from data layer to facilitate network pressure, realize unified management and integration of network resources. While fog radio access network (F-RAN) integrates fog computing into cloud-RAN, which makes full use of the edge of networks and reduces the fronthaul link overhead. Vehicular network is regarded as a highly delay-sensitive communication scenario, which needs more flexible network architecture and lower network latency. Exactly, SDN and F-RAN can satisfy many high requirements about vehicular networks. In the vehicular networks, designing efficient resource allocation method with different objectives and technologies is among one of the major challenges. In [4], authors solved the coexistence problem between a vehicular and an 802.22

CCS CONCEPTS •Networks →Wireless access points, base stations and infrastructure; •Computer systems organization →Cellular architectures; Heterogeneous (hybrid) systems;

KEYWORDS Fog computing, mean-field game, resource allocation, SDN, vehicular networks

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. DIVANet’17, November 21–25, 2017, Miami, FL, USA © 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM. ISBN 978-1-4503-4918-5/17/11. . . $15.00. DOI: http://dx.doi.org/10.1145/3132340.3132357

71

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

network via proposed resource allocation algorithms. The energy efficiency maximization problem with adaptive power allocation to the vehicular secondary users was investigated in [5]. The authors in [6] proposed a resource allocation scheme for D2D-enabled vehicular communication, where spectrum sharing and power allocation are performed based only on slowly varying large-scale fading information of wireless channels. In spite of being under intensive investigation in vehicular networks, many resource allocation methods remain unexplored, such as mean-field game. Mean-field game is a systematic differential theory to deal with large-scale vehicular users communication networks. Different from traditional game theory, which models the interaction of transmitters at successive transmission intervals [6][7][8], MFG analyzes the individual player’ s interaction with the mean-field. The theory has been successfully applied in wireless access network. In hyperdense heterogeneous networks, using a mean-field game-theoretic approach for interference problem was considered in [9], where the problem was formulated as an overlay problem at the macro base station (MBS) level and an underly problem at the small base station (SBS) level. Based on the theory of meanfield multi-armed bandit games, authors in [10] presented a distributed uplink user association for energy harvesting devices in an ultra-dense small cell networks. In this paper, we consider a two-tier heterogeneous vehicular network integrating the concept of software defined and fog computing technology. All the SBSs and vehicular users are equipped with caching mechanism. We formulate the downlink resource allocation problem as a mean-field game. In our MFG network, vehicular users will choose the optimal serving point to achieve the maximum benefit, i.e., serving point’s minimum cost. We construct the state space function, which is consisted of energy availability and interference constraint. The power control MFG in vehicular networks is a differential game, for which we can divide MFG into two equations, HJB and FPK. Through resolving the two equations, the proposed mean-field equilibrium can be solved. To the best of our knowledge, using a MFG theoretic method for power control in vehicular networks has not been considered in existing works. The remainder of this paper is organized as follows. Section II describes the system model and assumptions. Then, we formulate the differential game model in Section III. In Section IV, the power control problem is addressed as a MFG. Section V gives the solution of the MFG using proposed finite difference method. Meanwhile numerical and simulation results are discussed in Section VI before the conclusion of the paper.

&RUH1HWZRUN &ORXG/D\HU %DFNKDXO

)RJOD\HU

)98

)98

)6%6 )6%6 )98

0%6

)98

)6%6 )6%6

098

)98

)98

)6%6

)6%6

)98

)98

Figure 1: Software Defined Fog Vehicular Network Topology.

with data cache, collaboration radio signal and cooperative radio resource management functions [11]. F-VUs are priority served by F-SBSs that storage the content they want. We assume that all the F-SBSs have the same storage capacity and storage content. Combined with the concept of software defined network, F-SBSs as low power nodes account for high data rate transmission, while MBS as the high power node provides the control layer functions. We consider the heterogeneous OFDMA vehicular network downlink scenario with bandwidth of B. A macro-cell area consists of one MBS and K co-channel F-SBSs. We think K is very large, that is, K → +∞. M macro-VUs (M-VUs) are uniformly distributed in the macro-cell. Similar to the traditional work, each M-VU can be performed by using fair power allocation and round-robin scheduling scheme [12]. We assume that each F-SBS serves only one F-VU at a particular time slot. The channels between all the transmitters and all the receivers experience path loss and Rayleigh fading. F −S M F −S Note that gk,k and go,k are the channel gains of F-VU k in the correspond small-cell k and macro-cell, respectively, F −SM where k ∈ {1, 2, ..., K}; let gk,w be the channel gain from M-VU w (w ∈ {1, 2, ..., M }) in the macro-cell to F-SBS k; pF let pF k and ok as the F-VU kąŕ s transmit power serving by F-SBS and MBS, respectively. The signal-to-interference-plus-noise ratio (SINR) in the kth F-SBS for serving F-VU k at time t is given by: pF g F −S k k,k

γk =

2

)6%6

)6%6

M F −S pF + ok g o,k

SYSTEM MODEL

A software defined fog vehicular network topology is shown in Fig. 1, where the whole network is divided into Cloud layer and Fog layer. This paper concentrates on the resource allocation problem for Fog layer. In the Fog layer, the FogSBSs (F-SBSs) and Fog-vehicle users (F-VUs) are equipped

M F −S and Where pF ok g o,k

K ∑ i=1,i̸=k

K ∑

(1) F −S

pF k g i,k

F −S

pF i g i,k

+ σ2

denote the interfer-

i=1,i̸=k

ence caused by macro-cell and other small-cells. And σ 2 is the additive white Gaussian noise (AWGN) power.

72

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

Based on Shannon theorem, the downlink capacity of FVU serving by F-SBS k is shown as: cF k = log2 (1 + γk )

3

where ck (T ) represents the cost value at terminal time T . We define the value function υk (t) as follows: ∫ T υk (t, statek (t)) = min E[ ck (t)dt + υk (T, statek (T ))] (9)

(2)

pF k

DIFFERENTIAL GAME MODEL

where υk (T, statek (T )) is the value of state statek (T ) at final time T . The optimal power allocation p∗k is a Nash equilibrium game of the Gs if and only if [8]: ∫T p∗k (t) = arg min E[ 0 ck (pk (t), p∗−k )dt + ck (T )]

According to the SINR equation above, we model the power control problem as a differential game Gs with the interference dynamics and the energy dynamics [7]. Here, we first consider the state space function.

3.1

pF k

s.t. C1 : statek (t) = [ek (t), βkinterf (t)] ek (t) = − pF C2 : d dt k (t)

State Space Function

We take into account the energy state function ek (t) of FSBS k on serving the correspond F-VU k at time slot t as the amount of available energy. ek (t) is potentially range of [0, ek (0)]. ek (0) denotes the initial state of energy. The power control problem has total power constraint: pF k ∈ F F [0, Pmax ], where Pmax denotes the maximal transmit power of each F-SBS. And we define the energy usage state function by the following equation: d ek (t) = − pF k (t) dt

interf

F −SM where fkgain = gk,w +

=

pF k

K ∑ u=1,u̸=k

fkgain

(3)

4

(4)

3.2

4.1

d pF k (t) dt

+ pF k (t)

gain

dfk

(t)

dt

MEAN-FIELD GAME FOR POWER CONTROL

HJB Equation

−∂t υk (t, statek (t)) = min [ck (t, statek (t), pF k (t)) + ∂t statek (t) ∗ ∇υk (t, statek (t))] pF (t−T ) k

(11) where ∂t υk is the differential function of t, ∇υk is the gradient of the function υ with state. And we define the Hamiltonian: H(pF k (t), statek (t), ∇υk (t, statek (t))) = min [ck (t, statek (t), pF k (t)) + ∂t statek (t) ∗ ∇υk (t, statek (t))]

(6)

Cost Function

Here, we define the cost function: ck (t) = ω pF k (t) − ln(1 + γk ) − caching

= fkgain (t)

We have known that υk (t) is the value function of power pF k (t). We have the HJB function as follows:

dβkinterf (t) dfkgain (t) d pF (t) = fkgain (t) k + pF (5) k (t) dt dt dt Based on the energy state and interference state functions, we have the state space function for player k: ∈ {1, 2, ..., K}

(t)

Firstly, we analyze the power control problem as a MFG. And then we derive the corresponding HJB and FPK equations. The MFG power control problem can be modelled as the combination of couple HJB and FPK equations [8]. The FPK equation evolves the forward respect to time, and solving the HJB equation to obtain final value.

tor. From (4), the interference function at time t depends gain on the transmit power pF . So, the k and gain factor fk interference state function can be defined by:

statek (t) =

dt

(10)

where p∗−k is the transmit power vector of the F-SBSs except F-SBS k. By resolving the Nash equilibrium game Gs , all the players can obtain the minimum cost. Essentially, there exists at least one Nash equilibrium for the differential game Gs [7][8]. In the following, we formulate the Nash equilibrium game Gs to a MFG.

F −S gk,u denotes the gain fac-

[ek (t), βkinterf (t)], k

dβk

C3 :

We also consider the interference state function βkinterf that describes the interference caused by F-SBS Ik to the nearest M-VU and other F-VUs at time t: βkinterf (t)

t

(7)

pF (t−T ) k

where ω is the pricing factor, and caching = rk gcache represents the reward of caching, where rk is the request rate of the content from F-VU k and gcache is the gotten gain by caching the content. The cost function ck (t) is convex with respect to pF k (t).

(12)

4.2

Mean-Field and Mean-Field Approximation

We define the mean-field expression:

3.3

Power Control Policy

p∗k (t)

K→∞

pF k

ck (t)dt + ck (T )]

(13)

k=1

where [[1]] denotes the returned value as 1 if the condition is satisfied and zero, otherwise. At a certain time, the meanfield is the state probability distribution of the set of players.

T

= arg min E[

1∑ [[1]]{statek (t)=state} K K

m(t, state) = lim

Based on the state space function and cost function, the power control problem p∗k (t) from time 0 to T can be written as: ∫ (8)

0

73

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

5.2

In our system, all F-SBS players are coupled with each other by interference effect. Therefore, we can formulate the mean-field equation via mean-field approximation. For example, the interference function caused by other smallcells can be simplified as: K ∑

F −S pF ≈ (K − 1) pˆF ˆkF −S k gi,k k g

(14)

pF k

i=1,i̸=k

s.t. : C1 : ∂t m(t, state) + ∇e m(t, state)ek (t) +∇β ∫m(t, state)βkinterf (t) = 0 C2 : state m(state)d(state) = 1, ∀t ∈ [0, T ]

pF where pˆF k =E[ k (t)] denotes the average transmit power across F −S g F −S all interferers. Let gˆkF −S be the average of g1,k , 2,k , ..., F −S F −S F −S g(k−1),k , g(k+1),k , ..., gK,k . In the proposed MFG, each F-SBS as player plays game with the other, and each FSBS pre-measure the cross-layer interference caused by MBS. Based on that, we consider combining the cross-layer interM F −S + σ 2 . Substiference and AWGN power as σo2 = pF ok g o,k tuting (14) into (1), we have the mean-field approximation SINR, pF g F −S k k γˆk = (15) F F −S (K − 1) pˆk gˆk +σo2 cˆk (t) = ω pF ˆk ) − caching k (t) − ln(1 + γ

(16)

FPK Equation

Similarly, the FPK equation of game Gs can be given by: ∂t m(t, state) + ∇(∂t statek (t) ∗ m(t, state)) = 0

(17)

j,k

discretized grid. By solving the function

where state is the state space function. The FPK equation describes the defined mean-field game with respect to time and state space function.

5

= 0, we obtain

j,k

Lagrange multiplier λi−1 j,k in (23). We still remains to a complete algorithm to obtain the execution structure and converging point as Algorithm 1.

Our target is to find the solution of the proposed mean-field game, i.e., mean-field equilibrium. Here, we use a joint finite difference scheme based on the method in [7][8]. Through solving the couple equations iteratively, we can finally lead to the mean-field equilibrium. In our system model, the time interval [0, T ], energy state space [0, emax ], and interference state space [0, βmax ] are discretized into [X, Y, Z] three-dimensional space. We define the iteration steps of three directions:

5.1

∂Ld ∂pij,k

the values of the transmit power in (22) at the top of next page. Similarly, for arbitrary point (i, j, k) in the discretized ∂Ld grid, we can solve the equation ∂m = 0 to update the i

SOLUTION TO THE MEAN-FIELD GAME

∆t =

(19)

The C2 constraint condition is to guarantee that mean-field gives the probability density function of the state distribution over F-SBSs at each time instant [8]. Accordingly, the Lagrangian function in (19) can be written as (20) at the top of next page, where λ(s, t) is the Lagrange multiplier. Here we assume that terminal cost ck (T ) = 0. Similar to the solution of FPK equation, we use a joint finite difference scheme to solve the Lagrangian function (20). We give the discretized Lagrangian function in (21), where mij,k , pij,k , cij,k and λij,k are the values of the mean-field, power, cost function and Lagrange multiplier, respectively, at time t instant i with the energy level j and interference state k in the discretized grid. Let (P ∗ , M ∗ , λ∗ ) denotes the optimal decision variables. Based on the above analysis, we derive the optimal power ∂Ld = 0 for arbitrary point (i, j, k) in the control when ∂p i

Then, the mean-field cost function can be re-defined as

4.3

Solution to the HJB Equation

Because of the Hamiltonian, the finite difference method cannot be used to solve the HJB equation directly. Therefore, we re-express the HJB equation as its corresponding optimal power control problem with the FPK equation as a constraint condition. Based on that, we have ∫T min E[ 0 cˆk (t)dt + cˆk (T )]

Algorithm 1 Solution of mean-field equilibrium 1: Initialize mean-field value m and Lagarangian variable vector

λ, set i = X + 1 2: Initialize power p using equal power allocation 3: For each F-SBS, do 4: for i = 1 to X do 5: for j = 1 to Y do 6: for k = 1 to Z do 7: calculate mi+1 j,k using equation (18)

T emax βmax ,∆e = ,∆β = X Y Z

Solution to the FPK Equation

8:

if pi+1 j,k = 0 then

9:

12: 13:

i mi+1 j+1,k+1 = mj,k else mi+1 j+1,k+1 = 0 end if calculate pij,k using equation (22)

14:

calculate λi−1 j,k using equation (23)

10: 11:

We use Lax-Friedrichs scheme to guarantee the positive value of the mean-field. By applying the Lax-Friedrichs scheme to the FPK Equation, we have (18) at the top of next page, i where mij,k , pij,k , and fj,k−1 denote, respectively, the values of the mean-field, power and interference gain factor at time t instant with the energy level j and interference state k in the discretized grid.

15: end for 16: end for 17: end for

74

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

i i i i 1 mi+1 j,k = 2 (mj+1,k + mj−1,k + mj,k+1 + mj,k−1 ) i i i i i i ∆t ∆t − 2∆e [pj+1,k mj+1,k − pj−1,k mj−1,k ] − 2∆I [pij,k+1 fj,k+1 mij,k+1 − pij,k−1 fj,k−1 mij,k−1 ] (18)

L(m(t.state), p(t, state), λ(t, state)) ∫T ∫T emax ∫ βmax ∫ = E[ 0 cˆk (t)dt + cˆk (T )] + λ(t, state)(∂t m(t, state) + ∇e m(t, state)ek (t) + ∇β m(t, state)βkinterf (t))dβdedt =

∫T

emax ∫ βmax ∫

t=0 e=0 β=0

[c(t, state)m(t, state) + λ(t, state)(∂t m(t, state) + ∇e m(t, state)ek (t) + ∇β m(t, state)βkinterf (t))]dβdedt

t=0 e=0 β=0

(20)

Ld = ∆t∆e∆β

X+1 +1 Z+1 ∑ Y∑ ∑

[mij,k cˆij,k + λij,k

i=1 j=1 k=1 i i i i i pj+1,k mj+1,k −pj−1,k mj−1,k +λj,k 2∆e

+

mi+1 −0.5(mij+1,k +mij−1,k +mij,k+1 +mij,k−1 ) j,k ∆t

fi mi −pi fi mi pi λij,k j,k+1 j,k+1 j,k+12∆βj,k−1 j,k−1 j,k−1 ]

(21)

pij,k =

(K − 1) pˆF ˆkF −S +σo2 2∆e∆βω k g − gkF −S ∆β(λij+1,k − λij−1,k ) + ∆e(λij,k+1 − λij,k−1 ) (22)

It can be proved that provided Algorithm is the convergence point of the mean-field equilibrium of game Gs with the cost function ck (t) [8]. 0.08

NUMERICAL RESULTS AND DISCUSSION

0.06

mean-field

6

In this section, we give the numerical results of the mean-field distribution and optimal power control policy. In the simulation, we take into account OFDMA downlink transmission with carrier frequency of 2GHz and system bandwidth of B = 10M Hz. The AWGN power spectral density is −170dBm/Hz. Without loss of generality, we assume that channel fading is composed of shadowing fading, path loss and frequency selective Rayleigh fading [12]. We assume that all the small-cells are equal. The distribution of network elements is as follows. A MBS is located in the center point of macro-cell where 50 MVUs are uniformed distributed. The number of F-VU is 100, and each F-VU only be served by one F-SBS. We assume all the F-VUs are uniformly distributed in the macro-cell coverage area with the radius of 300m. And F-SBSs are equipped at the top of the F-VUs. We set the mean-field parameters as the time T = 1s, energy state space emax = 5J, and the interference state space Imax = 5.8 ∗ 10−8 W . And we consider a uniform distribution of initial energy. The time, energy and interference are discretized into 50*50*50 grids. Because the mean-field is a four-dimensional vector, we fix a variable to observe the relationship between other three. Through the distributed iterative algorithm, we obtain the mean-field distribution with respect to varying time and energy but fixed interference in Fig. 2, varying time and interference but fixed energy in Fig. 3, and varying energy and interference but fixed time in Fig. 4, respectively. We can see that mean-field distributions is random with the energy space and interference space. According to the mean-field distribution, the varying power control policy can always achieve the target SINR after convergence as shown in Fig. 5.

0.04

0.02

0 6 1

4

0.8 0.6

2

energy, J

0.4 0

0.2 0

time, sec

Figure 2: Mean-field Distribution vs Time and Energy.

7

CONCLUSION

In this paper, we investigated the power control in software defined fog vehicular networks. Firstly, we presented the state space function by both considering energy constraint and interference state. According to the state function, each F-SBS will resolve the power control policy to minimize the system cost. Based on the state space function and cost function above, we analyze the power control problem as a MFG. Then the mean-field game power control problem can be modelled as the combination of HJB and FPK equations. We provided a joint finite difference method to solve the mean-field equilibrium based on Lax-Friedrichs scheme and Lagrange relaxation. Numerical results have shown the meanfield distributions and power control policy.

75

Session: Resource Management

DIVANet’17, November 21–25, 2017, Miami, FL, USA

i i i i λi−1 j,k = 0.5 ∗ (λj−1,k +λj+1,k +λj,k−1 +λj,k+1 ) +

i ∆t i ∆t i p (λij+1,k −λij−1,k )+ 2∆β pj,k fj,k (λij+1,k −λij−1,k )−∆tˆ cij,k 2∆e j,k

(23)

ACKNOWLEDGMENT

0.08

This work is supported by the National Natural Science Foundation of China (61471025, 61771044), the Young Elite Scientist Sponsorship Program by CAST (2016QNRC001), and the Fundamental Research Funds for the Central Universities. The corresponding author of this paper is Haijun Zhang.

mean-field

0.06

0.04

0.02

REFERENCES

0 6.96e-08

[1] Q. Zheng, K. Zheng, H. Zhang and V. C. M. Leung, “DelayOptimal Virtualized Radio Resource Scheduling in SoftwareDefined Vehicular Networks via Stochastic Learning," IEEE Trans. Veh. Technol., vol. 65, no. 10, pp. 7857-7867, Oct. 2016. [2] L. Sun, H. Shan, A. Huang, L. Cai and H. He, “Channel Allocation for Adaptive Video Streaming in Vehicular Networks," IEEE Trans. Veh. Technol., vol. 66, no. 1, pp. 734-747, Jan. 2017. [3] M. Li, L. Zhao and H. Liang, “An SMDP-based Prioritized Channel Allocation Scheme in Cognitive Enabled Vehicular Ad Hoc Networks," IEEE Trans. Veh. Technol., vol. PP, no. 99, pp. 1-1, 2017. [4] Y. Han, E. Ekici, H. Kremo and O. Altintas, “Resource Allocation Algorithms Supporting Coexistence of Cognitive Vehicular and IEEE 802.22 Networks," IEEE Trans. Wireless Commun., vol. 16, no. 2, pp. 1066-1079, Feb. 2017. [5] D. Das, and S. Das, “Adaptive Resource Allocation Scheme for Cognitive Radio Vehicular Ad-hoc Network in the Presence of Primary User Emulation Attack," IET Networks, vol. 6, no. 1, pp. 5-13, Feb. 2017. [6] T. K. Thuc, E. Hossain and H. Tabassum, “Downlink Power Control in Two-Tier Cellular Networks with Energy-Harvesting Small Cells as Stochastic Games," IEEE Trans. Commun., vol. 63, no. 12, pp. 5267-5282, Dec. 2015. [7] C. Yang, J. Li, P. Semasinghe, E. Hossain, S. M. Perlaza and Z. Han, “Distributed Interference and Energy-Aware Power Control for Ultra-Dense D2D Networks: A Mean Field Game," IEEE Trans. Wireless Commun., vol. 16, no. 2, pp. 1205-1217, Feb. 2017. [8] P. Semasinghe, and E. Hossain, “Downlink Power Control in SelfOrganizing Dense Small Cells Underlaying Macrocells: A Mean Field Game," IEEE Trans. Mobile Comp., vol. 15, no. 2, pp. 350-363, Feb. 2016. [9] A. Y. Al-Zahrani, F. R. Yu and M. Huang, “A Joint CrossLayer and Colayer Interference Management Scheme in Hyperdense Heterogeneous Networks Using Mean-Field Game Theory," IEEE Trans. Veh. Technol., vol. 65, no. 3, pp. 1522-1535, March 2016. [10] S. Maghsudi, and E. Hossain, “Distributed User Association in Energy Harvesting Dense Small Cell Networks: A Mean-Field Multi-Armed Bandit Approach," IEEE Access, vol. 5, pp. 35133523, 2017. [11] M. Peng, and K. Zhang, “Recent Advances in Fog Radio Access Networks: Performance Analysis and Radio Resource Allocation," IEEE Access, vol. 4, pp. 5003-5009, 2016. [12] H. Zhang, C. Jiang, N. C. Beaulieu, X. Chu, X. Wen and M. Tao, "Resource Allocation in Spectrum-Sharing OFDMA Femtocells with Heterogeneous Services," IEEE Trans. Commun., vol. 62, no. 7, pp. 2366-2377, July 2014.

1

4.64e-08

0.8 0.6

2.32e-08

0.4 0.2

interference, J

0

0

time, sec

Figure 3: Mean-field Distribution vs Time and Interference.

1

mean-field

0.8 0.6 0.4 0.2 0 6.96e-08 5

4.64e-08

4 3

2.32e-08

interference, J

2 1

0

0

energy, J

Figure 4: Mean-field Distribution vs Energy and Interference. 1

mean-field SINR

0.8

0.6

0.4

0.2

0 0

5

10

15

20

25

30

35

40

45

50

time, sec

Figure 5: Mean-field Distribution and Power Control Policy after Convergence.

76