A Dynamic Programming Approach for Base Station Sleeping in

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

551

PAPER

A Dynamic Programming Approach for Base Station Sleeping in Cellular Networks∗ Jie GONG†a) , Sheng ZHOU†b) , Student Members, and Zhisheng NIU†c) , Fellow

SUMMARY The energy consumption of the information and communication technology (ICT) industry, which has become a serious problem, is mostly due to the network infrastructure rather than the mobile terminals. In this paper, we focus on reducing the energy consumption of base stations (BSs) by adjusting their working modes (active or sleep). Specifically, the objective is to minimize the energy consumption while satisfying quality of service (QoS, e.g., blocking probability) requirement and, at the same time, avoiding frequent mode switching to reduce signaling and delay overhead. The problem is modeled as a dynamic programming (DP) problem, which is NP-hard in general. Based on cooperation among neighboring BSs, a low-complexity algorithm is proposed to reduce the size of state space as well as that of action space. Simulations demonstrate that, with the proposed algorithm, the active BS pattern well meets the time variation and the non-uniform spatial distribution of system traffic. Moreover, the tradeoff between the energy saving from BS sleeping and the cost of switching is well balanced by the proposed scheme. key words: base station (BS) sleeping, blocking probability, dynamic programming (DP), neighboring BS cooperation

1.

Introduction

The continuously growing demand for ubiquitous information access leads to the rapid development of the information and communication technology (ICT) industry, which has become one of the leading consumers of energy and is expected to grow continuously in the future. As a result, energy saving is urgently required by both governments and network venders. For instance, GreenTouch has promised to improve energy efficiency in wired/wireless networks by a factor of 1,000 by 2015 compared with 2010 [1]. The energy consumption in ICT industry comes mainly from data centers, backhaul routers and cellular access networks. In cellular networks, the energy consumption of base stations (BSs) is 60% to 80% of that of the whole network [2], and will increase as network structure migrating from macrocell to microcell to meet the increasing demand of radio resources. As a result, the energy consumption of BSs becomes a major portion of the whole network energy consumption. Since the energy consumption of a BS mainly comes from baseband signal processor, controller, air-conditioner and etc., rather than transmit power which consumes only 3.1% [3], turning Manuscript received July 12, 2011. Manuscript revised October 27, 2011. † The authors are with the Tsinghua National Laboratory for Information Science and Technology, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. ∗ Part of this paper has been presented at IEEE IWQoS’10. a) E-mail: [email protected] b) E-mail: [email protected] c) E-mail: [email protected] DOI: 10.1587/transcom.E95.B.551

BSs into sleep mode whenever possible is considered as a promising technique to reduce the energy consumption. In fact, due to the variation in time domain and the dynamic distribution among cells in space domain [4], there are opportunities for some BSs to turn to sleep mode when the traffic load in their coverage is low. However, when BSs turn to sleep mode, radio coverage and quality of service (QoS, e.g., blocking probability) must still be guaranteed. Thanks to the concept of cell zooming [5], the users in the sleeping cells can be served by the neighboring active BSs by transmit power adjusting, antenna re-configuration, wireless relay and BS cooperation technologies. As a consequence, BS sleeping is a feasible approach for energy saving in cellular networks. To design efficient BS sleeping schemes, following issues must be carefully studied. - On the one hand, BS mode switching decision cannot be made by each BS individually. Not only the load condition of a BS itself, but also the load of its neighbors needs to be considered. For instance, a BS may not turn to sleep while its neighboring BSs are over loaded, even if its own traffic load is low. For this reason, each BS should make its mode switching decision via BS cooperation. - On the other hand, although cooperation among all the BSs can achieve the optimal sleep policy, it is not applicable in real system due to the high complexity. Suboptimal solution obtained by local cooperation among neighboring BSs is preferable. - Finally, taking signaling overhead, device lifetime and switching energy consumption into account, frequent BS mode switching should be avoided. That is, BSs should try to minimize the number of switching actions, or in other words, maximize the BS mode holding time, which is defined as the holding duration between two successive switching actions. In this paper, we exploit the traffic variation feature to design an energy-efficient BS sleeping algorithm, which is then formulated as a dynamic programming (DP) problem with a combined cost function of energy consumption, switching cost and blocking probability penalty. To reduce the dimension of state space and that of action space, percell Q-factor based on the cooperation among neighboring BSs is introduced, and a low-complexity algorithm is proposed to find the suboptimal policy. In addition, to match the system with BS sleeping behavior, user association and

c 2012 The Institute of Electronics, Information and Communication Engineers Copyright 

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

552

handover algorithms are re-designed. The rest of the paper is organized as follows. Related work is summarized in Sect. 2. Section 3 introduces the system model. The DP problem formulation is presented in Sect. 4. In Sect. 5, the proposed cooperative BS sleeping algorithm is described. The simulation results are presented to evaluate the proposed framework in Sect. 6. Finally, Sect. 7 concludes the paper. 2.

Related Work

BS sleeping is drawing more and more attention in recent years. Reference [6] suggests to switch off half of the cells at night while keeping the blocking probability below a given target. Reference [2] gives a static BS sleep pattern according to a deterministic traffic variation pattern over time. However, neither the randomness nor the spatial variation of the traffic is considered. Our work focuses on the scenario where traffic intensity varies in both time domain and space domain. In our previous studies [7], preliminary attempt of algorithm design is presented. Centralized greedy and distributed heuristic algorithms are proposed to dynamically turn on and turn off BSs. Similar work [8] proposes a BS awake selection algorithm. Nevertheless, switching cost was not considered. In [9], the authors consider the tradeoff between delay and energy consumption and formulate the problem as a Markov decision process problem and find a two-threshold optimal sleeping policy structure. This work and references therein provide inspiring examples about how to model the switching cost. However, how the BS switching actions influence with each other in multicell scenario remains open, which is the focus of our work. Reference [10] proposes a resource on-demand (RoD) strategy for high-density centralized WLANs. “Greencluster” is formed by a set of access points (APs) which are close to each other. A cluster-head AP is sufficient to provide coverage to users in the cluster, so that other APs in the cluster can be switched off when the traffic load is low. However, the channel model of WLANs is different from that of cellular networks where path-loss effect is dominating. Different from this work where cluster is fixed and non-overlapped, we implicitly adopt the concept of dynamic cluster by cooperation among neighboring BSs. BS energy saving problem can be modeled as a dynamic cell planning problem. However, the existing cell planning algorithms (see [11] and references therein) are too complicated to be used for dynamic adjustment. For dynamic resource control with respect to the spatial and time traffic variation, one could refer to load balancing schemes (see [12] and [13] for example). Indeed, properly utilizing load balancing is also helpful to BS energy saving, since it can reduce the blocking probability effectively. We will demonstrate how load balancing could improve the BS energy saving performance through simulations.

3.

System Model

Consider a downlink cellular network consisting of M BSs with universal frequency reuse. Let M = {1, . . . , M} denote the set of BSs. The maximum coverage of BS m is the area where BS m can provide the required data rate, and the cell m is defined as the area that is nearest to BS m compared with other BSs. As depicted in Fig. 1, the cell radius is Rc and the BS maximum coverage radius is Rb , which indicates that each BS is able to cover its neighbor cells. In the traditional cellular networks where all BSs are active, each cell is taken care of by its own BS. When some BSs turn to sleep, the actual BS coverage extends from their own cells to the neighbors with sleeping BSs. This is reasonable in urban scenarios, where BSs are densely deployed. The neighbors of BS m are denoted as m(1), . . . , m(B), where B is the number of neighbors (B = 6 in hexagonal cellular system). Denote Bm = {m, m(1), . . . , m(B)} as the set of BSs which can provide service to the users in cell m. 3.1 Traffic Model and Channel Model The traffic arrives in cell m at time t as a Poisson process with intensity λm (t). The traffic is assumed uniformly distributed in each cell, but asymmetric among different cells. Assume that the system have the statistic traffic information, M , which is a pei.e., the average arrival rate λ(t) = {λm (t)}m=1 riodic function with period T (for example, 24 hours). Each user has a minimum rate requirement r0 . All users arrive randomly and then remain stationary until the transmission is finished. The transmission duration of each user follows exponential distribution with mean 1/μ. Assume that each active BS m has limited radio resource, i.e., the maximum bandwidth Wmmax . Notice that the bandwidth here is the generalization of wireless resources

Fig. 1 Cellular network architecture. The cell radius is Rc and the maximum coverage radius is Rb , which indicates the overlapped network structure. When some BSs turn to sleep, the active BSs extend their actual coverage from their own cells to the neighbors with sleeping BSs.

GONG et al.: A DYNAMIC PROGRAMMING APPROACH FOR BASE STATION SLEEPING IN CELLULAR NETWORKS

553

that the BS can allocate, i.e., sub-carriers or time-slots, etc. If user k is associated to BS m, the corresponding bandwidth demand is r0 , (1) wmk (t) = Cmk (t) where Cmk (t) is the spectrum efficiency (instantaneous peak rate per unit bandwidth). It varies over time due to shadowing, multi-path fading and etc. Nevertheless, as the long time-scale performance is considered here, we ignore the fast fading effects, i.e., we assume that the spectrum efficiency is constant during the transmission, which is determined by the large-scale path-loss. The expressions are then simplified to Cmk and wmk without time index. The intercell interference is assumed already being taken care of by certain reuse or interference management schemes. Interference power is averaged over all possible user positions assuming all the BSs are in active mode. It is a conservative estimation since interference is reduced when some BSs go to sleep mode. Based on above assumptions, Cmk depends only on the distance lmk from BS m to user k   ⎧ P βl−α ⎪ ⎪ ⎨ log2 1 + tN0mk , 0 < lmk ≤ Rb , C(lmk ) = ⎪ (2) ⎪ ⎩ 0, l >R , mk

b

where Pt is the transmit power; β is the path-loss constant and α is the path-loss exponent; N0 is the noise-plusinterference power.

of BS m. At each time spot t(i) , i = 0, . . . , N, the BSs take the M action u(i) = {u(i) m }m=1 . The action space is U = U1 × U2 × . . . × U M ,

(5)

where Um = {0, 1}, m = 1, . . . , M is the set of actions of BS (i) = 1 as the action that BS m switches its workm. Denote um (i) = 0 otherwise, as that BS m maintains its ing mode and um working mode. The overview of the system operation is as follows. The BSs work in the constant state s(i) during the time interval τ(i) , i = 1, . . . , N and the users are served by the currently active BSs. At each time spot t(i) , i = 0, 1, . . . , N − 1, the BSs decide weather to switch their working modes or not according to the action u(i) . If a BS switches from active mode to sleep mode, the associated users are shifted to the active neighbors. After the BS mode switching process is finished, the system goes into the next time interval τ(i+1) with updated state s(i+1) . Generally speaking, the object of BS sleeping algorithm is to determine the action u(i) , i = 0, . . . , N − 1 to minimize the system energy consumption given the initial state s(0) and statistic traffic information λ(t), while at the same time maintain a predefined blocking probability and avoid frequent mode switching. The problem formulation is detailed in the next section. 4.

Problem Formulation

3.2 BS State and Control Action The time period T is divided into N + 1 time intervals (each with index i) as shown in Fig. 2. Note that to be able to track the system traffic variation, the length of time interval τ(i) varies with λ(t) so that on average, constant number of users, Kτ , arrive during τ(i) . Then we have  Kτ =

M t(i) +τ(i)

t(i)

λm (t)dt,

(3)

m=1

where τ(i) can be calculated by some numerical method. Assume that each BS m ∈ M can work in two modes: (i) = 1) and sleep mode (denoted active mode (denoted as sm (i) as sm = 0). In each time interval τ(i) , i = 1, . . . , N, the (i) M }m=1 . The state system works in the fixed state s(i) = {sm space is S = S 1 × S 2 × . . . × S M,

(4)

where S m = {0, 1}, m = 1, . . . , M is the set of working modes

In the literature, some algorithms have been proposed to minimize energy consumption for a given QoS requirement [2], [6]–[8]. However, these algorithms can not solve the problem of avoiding frequent mode switching since it introduces time correlation, i.e., the actions taken in current time will influence those taken in the future. DP algorithm [14] is an effective solution for such a complex time-correlated problem by optimizing the actions jointly over all the time slots. Consequently, we formulate the problem as a DP problem as follows. A standard DP problem contains the following elements: state, action, state transition and per-stage cost [14]. The states and the actions are already described in Sect. 3.2. Note that the system state is actually (s(i) , λ(i) ), where λ(i) =

t(i+1) (i) M (i) }m=1 , λm = 1/τ(i+1) t(i) λm (t)dt is the average arrival {λm rate during t(i) and t(i+1) . We denote s(i) as the system state for notation convenience, since λ(i) is system-determined parameter and do not change with any action. Given the current system state s(i) and the control action u(i) , the state transition is determined by (i) (i) M s(i+1) = f (s(i) , u(i) ) = {|sm − um |}m=1 .

(6)

The per-stage cost function g(i) (s(i) , u(i) ) is composed of three parts. The first part is the energy consumption of BS operation, which is calculated as Fig. 2 System operation over time. The network keeps a constant state s(i) in each time interval τ(i) , and operates action u(i) at each time spot t(i) .

ge(i) (s(i) , u(i) )

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

554

=

M 

(i+1) (i+1) , s(i+1) m Pmax + (1 − sm )Pmin τ

(7)

m=1

where Pmax is the power consumption in active mode including signal processing, air-conditioning, power amplifier and etc, Pmin is the minimum power consumption in sleep mode to be able to wake up. The second part is the BS mode switching cost (i) (i) g(i) s (s , u ) =

M

(i) E s um ,

(8)

covered by the BSs in Bm , the area blocking probability of Am can be approximated as Pa (Am , s) ≈ P˜ a (Am , s˜ m ), where P˜ a (Am , s˜ m ) = ⎛ ⎞ ⎛ ⎞ ⎜⎜⎜ ⎟⎟⎟  ⎜⎜⎜ ⎟⎟  ⎟⎟ ⎜⎜⎜⎜ max ⎟ ⎜ ⎟ xnk wnk > Wn ⎟⎠ |k ∈ Am ⎟⎟⎟⎟⎟ Pr ⎜⎜⎜ ⎜⎝wnk + ⎜⎝n∈B : s =1, ⎟⎠ k m n wnk Wmmax ⎟⎟⎟⎠⎟⎟⎟⎟⎟ , Psys (s) = Pr ⎜⎜⎜⎜ ⎜⎝ ⎟⎠ m∈M: sm =1, wmk Pthr to keep the blocking probability below the given threshold. Otherwise when the blocking probability requirement is satisfied, i.e., P˜ a (Am , s˜ m ) ≤ Pthr , the penalty is zero. As a result, the penalty function can be written as h(P˜ a (Am , s˜ m ), Pthr ) =  Eb P˜ a (Am , s˜ m ), if P˜ a (Am , s˜ m ) > Pthr , 0, else,

(20)

where Eb is a very large number. In summary, the per-stage cost is given by (i) (i) g(i) (s(i) , u(i) ) = ge(i) (s(i) , u(i) ) + g(i) s (s , u )

+gb(i) (s(i) , u(i) ), i = 0, 1, . . . , N − 1, g

(N)

(s

(N)

) = 0.

{u ,...,u

N−1

(22)

}

g(i) (s(i) , u(i) ),

(23)

i=0

and find an optimal control policy ν = {ν(0) , ν(1) , . . . , ν(N−1) } that satisfies ν = arg

min

{u(0) ,...,u(N−1) }

N−1

g (s , u ). (i)

(i)

(i)

(24)

i=0

In next section, we first present the standard DP algorithm. Then by reducing the state size using per-cell Qfactor, a low-complexity algorithm is proposed. Also, BS sleeping related user association and handover control is discussed, as well as implementation issues. 5.

j∈Bn ∩Bm

λ j In Bλn ,s =0 j



Rj Rc

⎞−1 ⎤ ldl ⎟⎟⎟⎟⎟ ⎥⎥⎥⎥⎥ ⎟ ⎥, C(l) ⎟⎠ ⎥⎥⎥⎥⎥

(16)

spot i with initial state s(i) . Proceeding backward induction of Eq. (26) from N − 1 to 0, the optimal cost is equal to J (0) (s(0) ) for the given s(0) . Furthermore, if ν(i) = u(i) (s(i) ) minimizes the right side of Eq. (26) for each s(i) and i, the policy ν = {ν(0) , ν(1) , . . . , ν(N−1) } is optimal. Note that the cardinalities of the state space S and the action space U are both 2 M , which increase exponentially with the number of BSs M. Due to the curse of dimensionality (as termed in [14]), the computational requirement to obtain the optimal control policy is overwhelming if the network size is large. As a consequence, it is very difficult to implement the standard DP algorithm in practical systems. In the following, we introduce per-cell Q-factor estimation to reduce the size of state space and propose a low-complexity algorithm to simplify the decision process.

(21)

In this paper, given the traffic variation function λ(t) and the initial state s(0) , we seek to minimize the total cost of all stages min(N−1) (0)



Dynamic Programming Algorithm

5.2

Q-factor and Space Reduction

Define the Q-factor [14] as follows: Q(i) (s(i) , u(i) ) = g(i) (s(i) , u(i) ) + J (i+1) ( f (s(i) , u(i) )),

(27)

where i = 0, 1, . . . , N − 1. It represents the cost of applying the action u(i) at the current state s(i) plus the minimal cost of the tail subproblem that starts at time spot i + 1 with initial state s(i+1) = f (s(i) , u(i) ). According to (26) and (27), we have Q(i) (s(i) , u(i) ), J (i) (s(i) ) = min (i)

(28)

Q(i) (s(i) , u ) = g (s(i) , u(i) ) + min Q(i+1) ( f (s(i) , u(i) ), u(i+1) ).

(29)

u ∈U (i) (i)

u(i+1) ∈U

To reduce the size of state space, we approximate the Q-factor as a sum of per-cell Q-factors, i.e., Q(i) (s(i) , u(i) ) ≈

M

(i) (i) (i) Q˜ m (˜sm , u˜ m ),

(30)

m=1

where the per-cell Q-factor is 5.1 General Solution DP solves a complex problem by breaking it down into simpler subproblems and tackling them recursively. The cost minimization problem (23) can be solved by the standard DP algorithm taking the form [14] (25) J (N) (s(N) ) = 0,  (i) (i) (i) (i) (i) (i+1) (i) (i) ( f (s , u )) , (26) J (s ) = min g (s , u )+ J u(i) ∈U

where i = 0, 1, . . . , N − 1, and the functions J (i) (s(i) ) denote the minimal cost for the tail subproblem that starts at time

(i) (i) (i) (i) (i) (i) (˜sm , u˜ m ) = g˜ m (˜sm , u˜ m ) Q˜ m (i) (i) sm , u˜ m ), u˜ (i+1) + min Q˜ (i+1) m ( f (˜ m ), u˜ (i+1) m

(31)

which, similar to Q(i) (s(i) , u(i) ), is the local cost of applying (i) (i) at the current local state s˜ m plus the the local action u˜ m minimal local cost of the tail subproblem assuming the BSs and users in all the other cells are moved out of the sytem. The per-cell per-stage cost is 1  (i+1) (i) (i) (i) (˜sm , u˜ m ) = g˜ m sn Pmax B + 1 n∈B m

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

556

 +(1 − s(i+1) )Pmin τ(i+1) + E s un(i) n (i) (i) +h(P˜ a (Am , f (˜sm , u˜ m )), Pthr ),

(32)

which includes the energy consumption and the switching cost of the BSs in Bm , and the area blocking probability of (i) (i) (i) (˜sm , u˜ m ) only needs the cell m. The per-cell Q-factor Q˜ m information of the BSs in Bm , which indicates the limited cooperation among neighboring BSs. It can be recursively calculated for each BS m ∈ M. The suboptimal control policy is then given by ν(i) (s(i) ) = arg min

u(i) ∈U

M

(i) (i) (i) Q˜ m (˜sm , u˜ m ).

(33)

Algorithm 1 Action Iteration 1: for i = 0 to N − 1 do 2: Set u∗ = 0, Qˆ = ∞, Qmin = Eb . 3: while Qmin < Qˆ do 4: Set Qˆ = Qmin . 5: for m = 1 to M do 6: Find the optimal local action ν˜ ∗m of the problem ν˜ m

7: 8: 9: 10: 11: 12:

m

Recall that during the time interval τ(i) , i = 1, . . . , N, the users are served by the currently active BSs. At each time spot t(i) , i = 0, 1, . . . , N − 1, if a BS switches from active mode to sleep mode, the associated users are shifted to the active neighbors. As the accessible BSs provide different signal strength and are of various load conditions, user association and handover should be carefully designed to optimize resource allocation. 5.3.1 User Association

m=1

Remark 1 (State Space Reduction): The size of state space in each stage is substantially reduced from 2 M (exponential growth w.r.t the number of BSs M) to M2B+1 (linear growth w.r.t. M). Although the size of state space is reduced by introducing per-cell Q-factor, the minimization progress (33), which requires exhaustive search over the action space U, is still of high complexity. Based on the per-cell Q-factor, we propose an iterative decision making algorithm. The basic idea is to iteratively find the optimal local action ν˜ ∗m = {ν∗m , ν∗m(1) , . . . , ν∗m(B) } of each BS to minimize the sum of percell Q-factors, and update the local elements of the global action accordingly. Until the sum of per-cell Q-factors does not decrease, the iteration terminates and the action is determined simultaneously. The detailed description of the algorithm is summarized in Algorithm 1.

min

5.3 Systematic Design

Q˜ n(i) (˜sn(i) , u˜ n ), ν˜ m ∈ {0, 1}B+1 ,

n=1

where u˜ n is determined as: ul = u∗l , if l  Bm ; ul = |u∗l − νl |, if l ∈ Bm . Update u∗ : if l ∈ Bm , u∗l = |u∗l − ν∗l |; else, u∗l = u∗l . end for  M ˜ (i) (i) ∗ Qn (˜sn , u˜ n ). Update Qmin = n=1 end while Set ν(i) = u∗ , s(i+1) = f (s(i) , ν(i) ). end for

Remark 2 (Action Space Reduction): In the greedy search step 6, the size of decision space is 2B+1 . Obviously, the iteration (from step 3 to step 10) converges in a finite number of iterations. Simulations show that the number of iterations is no more than 4 mostly. As a result, the action search complexity in each stage is reduced from O(2 M ) to O(M2B+1 ).

Load balancing scheme is implemented for the user association to reduce system blocking probability. In the literature, load balancing has been extensively studied and some efficient scheduling methods have been proposed. We make use of the load-aware cell-site selection scheme [15]. If user k arrives in cell m, its candidate serving BS set is Ck = {n ∈ Bm |sn = 1}. The user selects the serving BS from Ck with higher channel quality C(lnk ) and lower traffic load Wn as well, where Wn is the allocated bandwidth of BS n. Notice that there are several candidate serving BSs in Ck and their remaining bandwidth maybe not enough to accept user k, that is, Wnmax − Wn < wnk . Consequently, the user should try the BSs in Ck one by one to reduce the blocking probability. In the proposed algorithm, this is carried out by setting up a BS list according to both the channel quality C(lnk ) and the traffic load Wn , and searching from the top of the list. The algorithm terminates as long as the user can be accepted. If none of the BSs can provide enough bandwidth, the user is blocked. The algorithm is detailed in Algorithm 2. Note that | · | is the cardinality of a set. Algorithm 2 User Association 1: Set up a BS list Lk = {n1 , n2 , . . . |n j ∈ Ck } with cn1 ≥ cn2 ≥ . . ., where cn = C(lnk )Wnmax /Wn . 2: while |Ck | > 0 do 3: Take BS n from the top of Ck . 4: if Wnmax − Wn ≥ wnk then 5: xnk = 1, Wn = Wnmax − wnk . The algorithm terminates. 6: else 7: Remove n from Ck . 8: end if 9: end while 10: if |Ck | = 0 then 11: User k is blocked. 12: end if

5.3.2 User Handover At each time spot t(i) , i = 0, 1, . . . , N − 1, the users which associate with the BSs turning from active mode to sleep mode, should change their association to the neighboring active BSs. Denote

GONG et al.: A DYNAMIC PROGRAMMING APPROACH FOR BASE STATION SLEEPING IN CELLULAR NETWORKS

557 (i) H (i) = {k|xmk = 1, sm = 1, s(i+1) = 0, m ∈ M} m

(34)

as the set of users to be handed over. To minimize the number of droppings, the handover algorithm also takes the idea of load balancing. The process is similar with Algorithm 2 except for the list setting up. As there are multiple users to be simultaneously considered, we build up a BS-user pair list instead of the BS list. Once a user handover is success, the pairs which contain the user are removed from the list immediately. The algorithm does not terminate until the list becomes empty. The users remained in H (i) are dropped. The handover algorithm is presented in Algorithm 3. Algorithm 3 User Handover 1: Set up a BS-user pair list L(i) = {(m1 , k1 ), (m2 , k2 ), . . . |s(i+1) m j = 1, k j ∈ H (i) , C(lm j k j ) > 0} with c(m1 ,k1 ) ≥ c(m2 ,k2 ) ≥ . . ., where c(m,k) = C(lmk )Wmmax /Wm . 2: while |L(i) | > 0 do 3: Take the BS-user pair (m, k) from the top of L(i) . 4: if Wmmax − Wm ≥ wmk and k ∈ H (i) then 5: xmk = 1, Wm = Wmmax − wmk , H (i) = H (i) \ {k}. Remove all the pairs (·, k) from L(i) . 6: else 7: Remove the pair (m, k) from L(i) . 8: end if 9: end while 10: if H (i)  ∅ then 11: Users k ∈ H (i) are dropped. 12: end if

5.4 Implementation Issue Since the proposed algorithm offers an off-line solution, different policies are implemented for different traffic variation pattern. A typical example is that policies for workday and weekend should be distinguished. In real networks, the statistic features of traffic distribution and variation may change. For instance, the increase of the total number of subscribers enhances the average traffic intensity; a newly opened business center will become a new hotspot in the daytime, which changes the traffic distribution in space domain. To be able to track the long-term variation of traffic, the system should establish a dataset and record the number of calls in each cell. Depending on the statistic information obtained from the dataset, the system can operate the proposed algorithm to update the BS sleep pattern whenever necessary. 6.

Simulation Study

The simulation layout is 10 by 10 hexagon cells with wrap up to avoid boundary effect, which is shown in Fig. 3. The link parameters are set according to ITU micro-cell test environment [16]. The cell radius is Rc = 200 m. According to the coverage assumption, the BS’s maximum coverage is set Rb = 520 m. We set Pmax = 1 kW which is a general BS power level, and ignore the power consumption in sleep

Fig. 3 Simulation layout and a traffic distribution example. Nh = 3 hotspots are formed and move along the red-dashed line anticlockwise every 24 hours a cycle. The highest load is λh (t), and the others are αl λh (t), l = 1, 2, 3, 0 ≤ α3 ≤ α2 ≤ α1 ≤ 1, respectively.

mode, i.e., Pmin = 0. Actually, we do not rely on the real value since the results are shown in terms of number of active BSs. Other than the bandwidth for interference management, the available bandwidth is Wmmax = 5 MHz. User rate requirement is r0 = 122 kbps. Transmission duration parameter is μ = 1/180 s−1 . The transmit power is Pt = 41 dBm. The noise-plus-interference power N0 is calculated by setting the reference SNR at distance 200 m to be 0 dB. Pathloss model is PL dB (lmk ) = 33.05+36.7 log10 (lmk ). The number of user arrivals in each time interval is Kτ = 1×104 . The blocking probability penalty is Eb = 1×108 J and the threshold is Pthr = 1%. The time-varying and asymmetric traffic distribution is configured according to [17] and detailed as follows: - Average arrival rate  M (or traffic intensity) of the whole λm (t) varies along time domain network λ(t) = m=1 with period of T = 24h. - Nh hotspots are generated and move along some directions randomly every 24 hours a cycle. Assume each hotspot covers 2-tiers of the hotspot center cell. - Set the arrival rates of the hotspot center cells as λm (t) = λh (t). Then the arrival rates of the lst-tier of hotspot center cells are λm (t) = αl λh (t), l = 1, 2, and the others are λm (t) = α3 λh (t), where 0 ≤ α3 ≤ α2 ≤ α1 ≤ 1. Figure 3 shows a traffic distribution example with Nh = 3 hotspots. The simulation is performed as follows. We first calculate the sleeping policy with respect to the statistic traffic information λ(t) and the given initial state s(0) . Then the random user arrival is generated in accordance with λ(t) to test the performance of the policy obtained by the proposed algorithm. The initial state s(0) is set by activating half of the BSs in the network uniformly and then opening two more BSs in each hotspot. The mobility is considered in two ways. Firstly, the handover caused by fast mobility is implicitly modeled by user departure in one cell and arrival

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

558

Fig. 4 Number of active BSs compared with average traffic intensity in time space with different traffic configurations.

Fig. 5

Traffic distribution in spatial domain (λ1 , Nh = 3).

in another. Secondly, the slow mobility is represented by the hotspots movement. 6.1 BS Sleeping Pattern In this part of simulation, parameter settings are: E s = 2.5 × 105 J/switch, α1 = 0.88, α2 = 0.63, α3 = 0.50. BS mode switching behavior along with the average traffic intensity is presented in Fig. 4. It is shown that our proposed DP algorithm tracks the variation of the average traffic intensity well in time domain with different simulation configurations. Note that for a given traffic intensity λ1 , the curves of number of active BSs for Nh = 1 and 3 are very close. The reason is that for a given average traffic intensity in each time period, the number of active BSs required is almost the same to guarantee required wireless resources. To illustrate the spatial consistency between the traffic distribution and the number of active BSs, we take the stage i = 20 with the traffic intensity λ1 and the number of hotspots  Nh(i) = 3 as an example. We calculate (i) + sm( j) /2 to imply the number of active sm = sm

Fig. 6

BS state distribution in spatial domain (λ1 , Nh = 3).

m( j), j=1,...,B

BSs around each cell. Comparing Fig. 5 with Fig. 6, we can see that more BSs are active in the highly loaded area, while less BSs are active in the area with low load. Still, in the low load area, there generally are some active BSs in order to guarantee the network coverage. As a result, the active BS distribution well meets the spatial distribution of traffic intensity. In addition, the blocking probability in each time interval is maintained below the target (1%) almost all the time (see Fig. 7) for different traffic configurations. The average blocking probability over 24 hours is around 0.3% for traffic intensity λ1 , and is getting lower for higher traffic intensity λ2 , which shows that the area blocking probability estimation is conservative, especially for high traffic scenario. More elaborate area blocking probability analysis can be performed to further improve the energy saving performance.

Fig. 7 System blocking probability variation versus time with different traffic configurations.

6.2 Comparing with Uniform BS Sleeping We compare the proposed DP algorithm with the uniform BS sleeping approach proposed in [2], where active BSs are uniformly located in the network with traffic intensity λ1 (see Fig. 4). In addition, since the sleep pattern is not restricted as long as the coverage is guaranteed in our settings, the uniform BS sleeping algorithm is modified from binary patterns

GONG et al.: A DYNAMIC PROGRAMMING APPROACH FOR BASE STATION SLEEPING IN CELLULAR NETWORKS

559

Fig. 8 Comparison of proposed DP BS sleeping algorithm and Uniform BS sleeping algorithm.

to multiple patterns to track the time variation more flexibly. For comparison fairness, Algorithm 2 and Algorithm 3 are applied for user association and user handover respectively. The number of active BSs are determined according to ! 5(λ(t) − λmin ) Non (t) = 55 + 5 × , (35) λmax − λmin which is a function of average traffic intensity λ(t), λmin < λ(t) < λmax . As a result, Non (t) ∈ {55, 60, · · · , 75}. Here · rounds the real number to the nearest integer no larger than it. We set Nh = 3 and E s = 2.5 × 105 J/switch for the simulations. In Fig. 8, the average number of active BSs and the average blocking probability are compared. Firstly, we set α1 = α2 = α3 = 1, which indicates the uniform traffic distribution over the whole network. This is the same scenario studied in [2]. The proposed DP algorithm shows little energy cost and blocking probability decrease. This is because that the optimal solution for uniform traffic distribution should be turning off BSs uniformly. So intuitively, the uniform BS sleeping algorithm, which even though uses a pre-defined sleep pattern, performs very close to the optimal solution. As a result, the proposed solution can not improve very much. Then we decrease the value of α1 , α2 and α3 to enhance the degree of asymmetric traffic distribution. As the traffic distribution becomes more and more asymmetric, the performance gaps become larger. Specifically, the improvement is small in number of active BSs, but is significant in blocking probability. The reason for small improvement in number of active BSs is stated in Sect. 6.1. While the proposed algorithm greatly reduces the blocking probability since it optimizes the resource allocation among space domain according to the traffic distribution. The figure also shows the following result, which is a little bit surprising: both the number of active BSs and the blocking probability of the proposed DP algorithm decrease as the parameters α1 , α2 and α3 decrease. As a matter of fact, the blocking events mainly comes from two aspects: 1) blocking caused by over loading in the hotspot cells; 2) blocking caused by the high bandwidth requirement of the users in sleeping cells, which are denoted as

Fig. 9 System blocking probability and dropping probability versus switching cost E s . LB: proposed load balancing based BS selection and user handover algorithm; SS: strongest signal based algorithm.

coverage edge users. In current simulation settings, the systems is in low traffic scenario. As a consequence, the blocking events caused by coverage edge users outweigh those caused by over loaded hotspot. As the traffic distribution becomes more and more asymmetric, more and more users are taken care of in the hotspots, and the number of coverage edge users becomes less. Hence, the blocking probability becomes lower and more BSs can sleep. Note that if the system traffic is extremely high on the contrary, the blocking events happened in hotspots outweigh those caused by coverage edge users, the blocking probability might increase. 6.3 Switching Cost Extensive simulations are run to test the influence of We set the switching cost E s on the performance. λ1 , Nh = 3, α1 = 0.88, α2 = 0.63, α3 = 0.50 for the simulations. With different switching cost E s = 0, 2.5 × 105 , . . . , 1 × 106 (J/switch), the average numbers of active BSs in the period of 24 hours are 62.9, 63.9, 65.2, 66.0 and 67.2 respectively. The blocking probability and the dropping probability are depicted in Fig. 9. The proposed load balancing based algorithms for user association and handover are compared with the strongest signal based ones, where the selection criterion is simply C(lmk ) and the selection process is similar with Algorithm 2 and 3. The result shows that by effectively utilizing the wireless resources, load balancing is helpful for reducing the number of blocking and dropping events. It also illustrates that with the increase of E s , the energy consumption increases, while both the blocking probability and the dropping probability decrease. It can be concluded that there is a tradeoff between the energy saved from turning BSs into sleep mode and the energy cost of BSs’ mode switching. Because high switching cost prevents the BS switching from active to sleep, blocking probability is reduced. Figure 10 shows the cumulative distribution function (CDF) of BS mode holding time. Without the switching penalty, more than 70% of BSs’ measured mode holding

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

560

adopted in this paper, would be a valuable extension of the present paper. Acknowledgments The research work is partially sponsored by the National Basic Research Program of China (973 Program: No. 2012CB316001); by the Nature Science Foundation of China (No.61021001, No.60925002); and by Hitachi R&D Headquarter. References Fig. 10 Cumulative distribution function of mode holding time with different switching cost E s (J/switch).

Table 1

Handover performance with different switching cost.

E s (J/switch) switch/stage handover/stage dropping/stage 0 2.5 × 105 5 × 105 7.5 × 105 1 × 106

32.6 10.0 4.06 1.85 1.63

385.7 108.5 41.74 17.73 16.03

19.6 4.16 0.72 0.20 0.17

time is less than 1 hour. Such frequent mode switching maybe not acceptable for BS equipments in the real system. It not only consumes additional energy, but also brings large amount of handover, which causes exploding signaling overhead and user QoS degradation (shown in Table 1). This result explains the necessity of integrating switching cost into the total cost. As the value of E s increase, the BS mode holding time increase accordingly, which shows that our algorithm well balances the tradeoff between energy saving from sleep and cost from switching. 7.

Conclusion

In this paper, a low-complexity BS sleeping algorithm has been proposed to find the suboptimal BS sleeping policy via DP. With our strategy, network energy consumption is greatly reduced while the required call blocking probability is guaranteed. The number of BSs in active mode well matches the variation of the network traffic in both time and spatial domain. As a result, our algorithm outperforms the existing uniform sleeping algorithm. By adjusting the switching cost parameter E s , the tradeoff between the energy saved from BS sleeping and the energy cost of BS mode switching is well balanced. The limitation is that our algorithm relies on the traffic feature. If the traffic is uniformly distributed, we can get little performance gain. Also if the traffic is always heavy, our algorithm cannot work. Future work may include integrating relaying and BS cooperation into the energy saving framework to enhance coverage [18], and building a testbed to evaluate the performance in the real system. In addition, more general energy consumption model with multiple modes, other than the binary model

[1] http://www.greentouch.org [2] M.A. Marsan, L. Chiaraviglio, D. Ciullo, and M. Meo, “Optimal energy savings in cellular access networks,” Proc. IEEE ICC’09 Workshop, GreenComm., June 2009. [3] H. Karl, “An overview of energy-efficiency techniques for mobile communication systems,” Technical Report, Telecommunication networks Group, Technical University Berlin, Sept. 2003. [4] D. Willkomm, S. Machiraju, J. Bolot, and A. Wolisz, “Primary user behavior in cellular networks and implications for dynamic spectrum access,” IEEE Commun. Mag., vol.47, no.3, pp.88–95, March 2009. [5] Z. Niu, Y. Wu, J. Gong, and Z. Yang, “Cell zooming for costefficient green cellular networks,” IEEE Commun. Mag., vol.48, no.11, pp.74–79, Nov. 2010. [6] L. Chiaraviglio, D. Ciullo, M. Meo, and M.A. Marsan, “Energyaware UMTS access networks,” Proc. 11th Int. Symp. Wireless Personal Multimedia Commun., Sept. 2008. [7] S. Zhou, J. Gong, Z. Yang, Z. Niu, and P. Yang, “Green mobile access network with dynamic base station energy saving,” MobiCom’09 poster, Sept. 2009. [8] K. Samdanis, D. Kutscher, and M. Brunner, “Dynamic energy-aware network re-configuration for cellular urban infrastructures,” Proc. IEEE Globecom Workshop GreenComm., Dec. 2010. [9] I. Kamitsos, L. Andrew, H. Kim, and M. Chiang, “Optimal sleep patterns for serving delay-tolerant jobs,” Proc. ACM e-Energy, April 2010. [10] A.P. Jardosh, K. Papagiannaki, E.M. Belding, K.C. Almeroth, G. Iannaccone, and B. Vinnakota, “Green WLANs: On-demand WLAN infrastructures,” Mobile Networks and Applications, vol.14, no.6, pp.798–814, Dec. 2009. [11] R. Mathar and M. Schmeink, “Optimal base station positioning and channel assignment for 3G mobile networks by integer programming,” Annals of Operation Research, vol.107, pp.225–236, Oct. 2001. [12] A. Sang, X. Wang, M. Madihian, and R.D. Gitlin, “Coordiniated load balancing, handoff/cell-site selection and scheduling in multicell packet data systems,” Proc. MobiCom’04, pp.302–314, Sept. 2004. [13] Y. Bejerano and S.J. Han, “Cell breathing techniques for load balancing in wireless LANs,” IEEE Trans. Mobile Comput., vol.8, no.6, pp.735–749, June 2009. [14] D.P. Bertsekas, Dynamic Programming and Optimal Control, 3nd ed., Athena Scientific, Massachusetts, 2005. [15] A. Sang, X. Wang, M. Madihian, and R. Gitlin, “A load-aware handoff and cell-site selection scheme in multi-cell packet data systems,” Proc. Globecom’04, pp.3931–3936, Dec. 2004. [16] R1-091320, “Radio characteristics of the ITU test environments and deployment scenarios,” Ericsson, 3GPP TSG RAN WG1 Meeting #56bis, March 2009. [17] L. Du, J. Bigham, and L. Cuthhen, “A bubble oscillation algorithm for distributed geographic load balancing in mobile networks,” Proc. IEEE Infocom’04, 2004. [18] D. Cao, S. Zhou, C. Zhang, and Z. Niu, “Energy saving performance

GONG et al.: A DYNAMIC PROGRAMMING APPROACH FOR BASE STATION SLEEPING IN CELLULAR NETWORKS

561

comparison of coordinated multi-point transmission and wireless relaying,” Proc. IEEE Globecom’10, Dec. 2010. [19] T. Bonald and A. Proutiere, “Wireless downlink data channel: User performance and cell dimensioning,” Proc. MobiCom’03, pp.339– 352, Sept. 2003. [20] L. Kleinrock, Queueing Systems, John Wiley & Sons, 1976.

Appendix:

Derivation of Area Blocking Probability

In this section, we ignore the stage index i for simplicity. According to the fact that a newly arrived user in cell m is blocked if all the active BSs in Bm are out of bandwidth, we have ⎞ ⎛ ⎟⎟⎟ ⎜⎜⎜  P˜ a (Am , s˜ m ) = Pr ⎜⎜⎜⎝ Wn ≥ Wnmax ⎟⎟⎟⎠ n∈Bm ,sn =1  Pr(Wn ≥ Wnmax ), (A· 1) = n∈Bm ,sn =1

where the second equality holds because the bandwidth utilization of BSs are independent with each other. If sn = 1, n ∈ Bm , the users in cell n can be served by BS n. The density of users in cell n is Kn /πR2c , where Kn is the number of users in cell n. Under the assumption that the user distribution in each cell is uniform, the bandwidth utilization of BS n for the users in cell n is calculated as  Rc Kn Wnn = wnk 2 2πldl πR 0 c  2r0 Rc ldl = Kn 2 . (A· 2) Rc 0 C(l) If s j = 0, j ∈ Bm , due to the nature of load balancing technique, the area of cell j is evenly assigned to its neighbor active BSs. Assume that the shifted traffic are uniformly distributed in the 1/B annulus sector with larger radius R j and smaller radius Rc . The bandwidth requirement of the shifted traffic is  Rj K j 2π W jn = wnk 2 ldl πR Rc c B  Rj 2r0 ldl , (A· 3) = Kj 2 BRc Rc C(l) where R j is set to evenly assign the cell area to its neighbor active BSs, i.e., " B + 1, (A· 4) R j = Rc I j M on j where M on j =

 j ∈B j ∩Bm

s j is the number of active neighbor

BSs of sleeping BS j, I j = 1 if j = m and I j = 2 if j = m(b), b = 1, . . . , B. Note that local information sets s˜ m and λ˜ m do not contain the full local information of BS m(b), b = 1, . . . , B. Therefore, we introduce the parameter I j assuming that the local information of BS m(b) is symmetric, i.e., the state and the arrival rate of BS j ∈ Bm(b) \ Bm are the same

as these of BS j ∈ Bm(b) ∩ Bm . In summary, the bandwidth utilization of active BS n is W jn In Wn = Wnn + j∈Bn ∩Bm ,s j =0

=

Kn γn ,

where Kn = Kn +

(A· 5)  j∈Bn ∩Bm ,s j =0

In K j /(I j M on j ) is the total num-

ber of user served by BS n, and 

 Rc ldl λ I Rj 2r0 + j∈Bn ∩Bm ,s j =0 Bλj nn R R2c 0 C(l) c γn =  λ j In 1 + j∈Bn ∩Bm ,s j =0 I j M on λn

ldl C(l)

 , (A· 6)

j

where we make use of the fact that Kn /λn = K j /λ j . At the same time, the traffic load in cell n is evenly shifted to its neighbor active BSs. Similarly, we assume that half of the traffic of BS m(b), b = 1, . . . , B is shifted to accessible active BSs in Bm . As a result, the traffic load of BS n(sn = 1) is

λn = λn +

j∈Bn ∩Bm ,s j =0

λj . I j M on j

(A· 7)

As the radio resource is shared by active users, the number of users Kn associated with BS n evolves like the number of customers in a processor-sharing queue with Poisson arrivals and i.i.d. service times [19]. The key property of the processor-sharing queue is that the stationary distribution of the number of customers is insensitive to the distribution of service times. Hence the stationary distribution of the number of active users is given by Pr(Kn = k) = (ρn )k (1 − ρn ) with mean E[Kn ] = ρn /(1 − ρn ), where ρn is the average traffic load of BS n. Applying Little’s law [20], we get E[Kn ] = λn /μ, which results in ρn = λn /(λn + μ). Finally, the area blocking probability is expressed as  Pr(Kn ≥ Wnmax /γn ), P˜ a (Am , s˜ m ) = n∈Bm ,sn =1

=



Wnmax /γn 

ρn

.

(A· 8)

n∈Bm ,sn =1

Summarizing the equations derived above, we obtain the expression of the approximated area blocking probability as stated in Sect. 4.

Jie Gong was born in Hunan in 1986. He received his Bachelor’s degree in Department of Electronic Engineering in Tsinghua University, Beijing, China, in 2008 and is currently a Ph.D. degree student there. His research interest is in cooperative communication and green communication.

IEICE TRANS. COMMUN., VOL.E95–B, NO.2 FEBRUARY 2012

562

Sheng Zhou was born in Shanghai in 1983. He received his B.S. degree from Tsinghua University, Beijing, China, in 2005. Now he is a Ph.D. candidate in Dept. Electronic Engineering, Tsinghua University. His research interest includes resource allocation and transmission protocol design for wireless MIMO systems.

Zhisheng Niu graduated from Northern Jiaotong University, Beijing, China, in 1985, and got his M.E. and D.E. degrees from Toyohashi University of Technology, Toyohashi, Japan, in 1989 and 1992, respectively. In 1994, he joined with Tsinghua University, Beijing, China, where he is now a full professor at the Department of Electronic Engineering and the deputy dean of the School of Information Science and Technology in charge of research activities and international collaboration. He is also an adjunction professor of Beijing Jiaotong University. From April 1992 to March 1994, he was with Fujitsu Laboratories Ltd., Kawasaki, Japan. From October 1995 to February 1996, he was a visiting research fellow of the Communications Research Laboratory of the Ministry of Posts and Telecommunications of Japan. From February 1997 to February 1998, he was a visiting senior researcher of Central Research Laboratory, Hitachi Ltd. He also visited Saga University, Japan, and Polytechnic University, USA, in 2001 and 2002, respectively. Prof. Niu’s current research interests include teletraffic theory and queueing theory, performance evaluation of broadband multimedia networks, radio resource management, mobile Internet, wireless ad hoc networks and wireless sensor networks, and Stratospheric Communication Systems. He received the PAACS Friendship Award from the IEICE of Japan in 1991 and the Best Paper Award from the 13th Asia-Pacific Conference on Communications (APCC2007). Dr. Niu is a senior member of the IEEE and the Chinese Institute of Electronics (CIE) and a fellow of the IEICE, Japan. He has been the TPC Chair of APCC2004 and the TPC Co-chair of IEEE ICC2008. He is now the director of IEEE Communication Society Asia-Pacific Board.