Utility Based Semi-Adaptive Call Admission Control Xu Yang
John Bigham
MPI-QMUL Information Systems Research Centre Macao Polytechnic Institute Macao SAR, P.R.China
[email protected]
Department of Electronic Queen Mary University of London London, UK
[email protected]
Abstract—This paper proposes a novel approach to call admission control (CAC) in adaptive wireless networks, called Semi-Adaptive CAC (SA-CAC). When a new or handoff call comes, the bandwidth allocated to the call can be adjusted according to the load of the network cell. Once the call is admitted, its allocated bandwidth is never adjusted throughout its lifetime. Therefore it will not lead to delay and overload of message transition and processing caused by degradation and upgrades, however it still maintains the advantages of flexible bandwidth allocation. We adopt one form of NeuroEvolution algorithm to learn good SA-CAC policies. The numerical results demonstrate that the proposed scheme is capable of maintaining the upper bounds of call blocking rate (CBR) and handoff failure rate (CDR), as well as increasing the network utility compares with traditional threshold based CAC scheme. Keywords-call admission contro, QoS constraint, Adaptive multimedia application, Utility, NeuroEvolution.
I. INTRODUCTION With the increasing number of wireless personal devices such as laptops, PDAs, and mobile phones, there has been explosive demand for wireless communications to provide high speed and multi-class of traffic. Call Admission Control (CAC) has become vital to guarantee the QoS for the multiple services and utilize the network resources under the limited capacity. The most significant QoS parameters in the existing wireless mobile multimedia networks are the call blocking probability (CBR) and the handoff failure rate (CDR). Generally dropping a handoff call due to handoff failure is more unbearable to users than blocking a new call, therefore prior reservation of a set of channels for future handoffs can minimize the CDR. [1, 2] However, the CBR may increase as a result of such bandwidth reservation. Reduction of CBR and CDR are in fact conflicting requirements, and optimization of both can be extremely complex, especially in multi-class services with diverse characteristics context. Recently there is a growing interest in adaptive multimedia systems, where the bandwidth of an ongoing multimedia call can be dynamically adjusted to adapt to the various communication environments, especially in overloaded situations. Compared to existing non-adaptive networks, adaptive multimedia frameworks require bandwidth adaptation algorithms (BAA) and call admission control (CAC) that consider the bandwidth degradation of ongoing calls to perform the QoS provisioning. The BAA determines which calls are to be adapted and the degree of bandwidth adaptation to be performed. Adaptation will be triggered whenever there are
arrival acceptance events and call departure events. [3] Several BAA algorithms have been proposed [4-7], and their results shows the BAA can enhance network utility, and also decrease the handoff failure rate. However BAA can cause several new QoS problems: 1) the acceptance of a new or handoff call can be delayed due to degradation of ongoing calls when network is overloaded; 2) the message/signalling overhead to ensure optimal bandwidth adaptation is inherently high; 3) the bandwidth of a call can be adjusted too many times in its lifetime; 4) the degree of degradation also need to be considered. In this paper we study a different approach to applying CAC in an adaptive multimedia network, which is called semiadaptive call admission control (SA-CAC). BAA and CAC are combined in one procedure. When a new or handoff call comes, the bandwidth allocated to the call can be adjusted according to the load of the network cell. Once the call is admitted, its allocated bandwidth is never adjusted throughout its lifetime. Therefore it will not lead to delay and overload of messaging and processing caused by degrades and up-grades, while still maintaining advantages of flexible bandwidth allocation. In conclusion, there is a tradeoff between having less QoS (bandwidth) allocated and reducing the new and handoff calls failure. The objective of the proposed SA-CAC is to guarantee the upper bounds of CBR and CDR, while maximize the network utility. This is a constrained reinforcement learning problem [8]. Comparing with other reinforcement learning algorithms, we adopt a form of NeuroEvolution (NE) algorithm called NeuroEvolution of Augmenting Topologies (NEAT) [9] to learn a good CAC policy. The proposed semi-adaptive CAC scheme can perform the call admission control in multimedia cellular networks with multiple classes of traffic and different QoS requirements. The performance of the algorithms have been evaluated using three QoS metrics: total utility accruing from the accepted calls (total rewards), CBR and CDR for each class of traffic. The learned best policy is compared with two basic CAC algorithms. The rest of this paper is organized as follows. Section 2 defines the utility based traffic model; Section 3 gives a brief introduction to NeuroEvolution of Augmenting Topologies (NEAT), and how to apply NEAT to the SA-CAC application; section 4 formulates the fitness function, while section 5 compares the performance with two traditional CAC schemes; Finally, section 6 gives a conclusion.
1-4244-1312-5/07/$25.00 © 2007 IEEE
795
II. UTILITY-BASED TRAFFIC DEFINITION Utility was originally used in economics and has been brought into networking research in recent years. It represents the “level of satisfaction” of a user or the performance of an application. [4] A utility function here is a curve mapping bandwidth received by applications to their performance as perceived by the user. The network can obtain a certain amount of utility per logical second by carrying each individual call.The problem of semi-adaptive CAC becomes challenging when these utility functions are nonlinear. In this paper, two classes of adaptive traffic have been used to demonstrate our scheme. Each multimedia application is presented to the network in the form of a hierarchy of scalable streams. Depending on the resource availability, a subset of these streams is selected and transported by the network. Figure 1 shows the fitness functions of two different classes of traffic. The possible allocated bandwidth levels for C1 and C2 are 11. The maximum requesting bandwidth for C1 is 1 and for C2 is 2. C1
Util ity Function
C2
1.2
1
There are a variety of artificial intelligence techniques have been applied to the CAC schemes, such as Reinforcement Learning (RL)[8]. However traditional RL algorithms (e.g. QLearning) are difficult to scale up to larger domains due to the exponential growth of state variables. In complex real world situations, the learning time for these algorithms is very long [8]. Additionally RL tries all possible actions to obtain the optimal policy in a dynamic environment. This may lead to unpredictable and unacceptable situations and the system could have low performance over a long running time. NE is a combination of neural networks and genetic algorithms where neural networks are the phenotype being evaluated. The genotype is a compact representation that can be translated into an artificial neural network [9]; it has been shown to work very efficiently in complex RL problems with scarce reinforcements.
0.8
Utility
bandwidth levels to requesting calls; hence a completed action space includes the total possible bandwidth levels for each application, which are 11. Additionally, there are two classes of traffic; each class of traffic has two different requests: setup calls and handoff calls. The network does not know which class the next request call is. To make a CAC decision, the network needs to identify the request call and the load of the cell ( sum of all bandwidth consumed by each class of traffic). We assume the total capacity of the cell is 20, and the bandwidth level increment is 0.1 (C1) and 0.2 (C2), therefore the state space is 4 × 200 × 100 at most. Some of these states will not occur (e.g. ones where the load of cell is all consumed by C1). Nonetheless, with 11 actions per state, it is clearly infeasible to represent the value function in a table. Hence, success in this domain requires function approximation.
0.6
0.4
0.2
0 0
0.2
0.4
0.6
0.8
1
1.2
1. 4
1.6
1. 8
2
Possible allocated BWU for C1 and C2 Figure 1. Utility Function for C1 and C2
We also assume that service requests of each class arrive according to Poisson distribution. The holding time for each service class is exponentially distributed. All the arrival distributions and call holding distributions are independent of each other. The arrival events include new call arrival events and handoff arrival events. Since the call departures do not affect the CAC decisions, we only consider the arrival events in the CAC state space. We denote λi, s and λi, k as the arrival rate −1 −1 of new setup and handover requests in class i , μi ,s and μi, h as the average holding time of setup and handover calls in class i. Additionally, we only consider one cell with fixed capacity. The fixed total number of bandwidth (channels) is C = 20 . III. WHY AND HOW TO APPLY NEAT IN SA-CAC A primary obstacle of applying reinforcement learning methods to this domain is the size of the state and action space. In our experiments, the scheduler’s actions consist of allocating
In [10] there is a comparison of Evolutionary and temporal differece methods in a reinforcement learning . It concluded that NEAT (one form of NE algorithm) learns faster when the domain has a deterministic fitness function. In this paper we adopt NEAT to learn good CAC policies. The NEAT method for evolving artificial neural networks is designed to take advantage of neural network structure as a way of minimizing the dimensionality of the search space. The evolution starts with a randomly generated small set of neural networks with simple topologies. Each of these neural networks is assigned a fitness value depending on how well it suits the solution. For example, given a function to maximize, the value of the function itself can serve as a fitness factor. Once all the members of the population have been assigned fitness values, a selection process is carried out where better individuals (with high fitness values) stand a greater chance to be selected for the next operation. Selected individuals undergo the recombination and mutation process and result in new individuals. Individuals with low fitness values are discarded from the population and better ones are kept. Structural mutations add new connections and nodes to neural networks, leading to incremental growth. The whole process is repeated for the new population until some termination criteria are met. The average fitness of the population is expected to increase over generations. [9] NEAT can be seen as a black box that provides a neural network for receiving inputs and generating outputs. To utilise
1-4244-1312-5/07/$25.00 © 2007 IEEE
796
NEAT, it is necessary to: a) define the input nodes and the outputs nodes, b) formulate a fitness function to evaluate the networks created, c) define how to stop the learning process ,and d) extract the learned neural network with the highest fitness score. The issue of preventing the generated outputs from damaging the system also needs to be considered in the context of the CAC.
total number of request calls (includes setup and handoff calls), a is the total number of accepted calls. For large N,
During the learning period, many neural networks are randomly generated and some of them can lead to very bad performance. Therefore, an Internal Supervisor is needed to filter out the obvious bad ones. In CAC, neural networks that always reject all calls are frequently generated and evaluated. However these neural networks with always-reject-policy are obvious bad ones and the evaluation is pointless. The Internal Supervisor then gives them fitness score of “0” directly.
Here m denotes the total number of classes of traffic, i denotes the class of traffic for each call; n denotes the total number of bandwidth levels assigned for each class of traffic; ui , j denotes the utility of a call in class i with bandwidth level j ; ki, j, s or ki, j, h denotes the number of the accepted new setup calls or handoff calls in class i with bandwidth level j .The relationship below also holds.
Additionally, the outputs generated by NEAT are not always feasible actions. For example, an evaluating neural network may try to accept a request call when the system is full, which is physically unrealizable. Therefore an External Supervisor is added and uses predefined knowledge to filter impossible actions according to the system constraints. Input 0
Selected policy
Input 1 Capacity consumed by C2
Internal Supervisor
External Supervisor
Valid Output
Input 2 The new request call
F1 =
N
population
μ −1i , s + ki , j , h μ −1i ,h )
(1)
N
λi , s m
¦ (λi, s + λi , j )
pi , j , s and ki , j , h = N
i= 0
λi ,s m
¦ (λi, s + λi, j )
pi , j , h
(2)
i =0
Where pi , j , s and pi , j , h denotes the acceptance rate of new setup request calls and handover calls in class i with assigned bandwidth level j .
λi , s
m
¦ (λ
i,s
i =0
+ λi ,h )
Cell
μ −1i , s and α i, h =
Figure 2 illustrates the process of setting up a connection using our scheme. There are three inputs and one output. Input 0 and 1 are real numbers between 0 to 20, input 2 identifies the type of the new request call. The output value is a real number between 0 and 1; it is divided into 11 different levels corresponding to the allocated bandwidth levels from 0 to 10. If the output is infeasible, the neural network will be replaced by reject action. FITNESS FUNCTION
i,s
A. First Goal: maximize utility- F1 If the goal is simply to maximize utility the fitness can be assessed by F1 , which can be calculated from the average utility obtained per request call as equation (1). Let N be the
+ λi, h )
(4)
n
i =0 j =1
(5)
It can be seen that the total utility is determined by both of the service demand parameters α and the learned CAC performance ( pi , j , s and pi , j , h ), which can be calculated after evaluation of each individual policy.
B. Maintain the QoS constraints-- F2 When the observed CBR or CDR exceeds their predefined upper limit bound ( Tcdr and Tcbr ), the evaluated policy needs to assign a negative fitness score f to the neuron network, which is large enough to affect the total fitness F2 . m n m F = u (α p + α p ) − ( f + f ) (6) 2
The goal of NEAT is to evolve neural networks with higher fitness score, so the fitness function should be directly related to the goal of the SA-CAC scheme.
¦(λ
μ −1i , h
F1 = ¦¦ ui , j (α i , s pi , j , s + α i , h pi , j , h )
Therefore
Figure 2. To apply NEAT in SA-CAC
λi ,h
m
i =0
m
User Request
(3)
To simplify the above formula, define the service demand parameter α as α i, s =
IV.
i, j,s
i = 0 j =1
≅
l =1
ki , j , s = N
n
i, j
l
accepted setup callsin class i with bandwidth level j ° pi , j , s = request setup callsin class i ° ® accepted handoff callsin class i with bandwidth level j °p = °¯ i , j , h request handoff callsin class j
NEAT
Capacity consumed by C1
m
¦¦ u ( k
a
¦U
¦¦ i =0 j =1
i, j
i, s
i, j,s
i ,h
i, j , h
¦
i,s
i ,h
i =0
V. SIMULATION EXPERIMENT In the simulation, we define two classes of traffic labeled C1 and C2 , their utility function is shown in equation 6. There are two kinds of requests: setup and handoff. Additionally a single cell with limited number of channels ( C = 20 ) is considered. The traffic parameters are shown in Table 1.
1-4244-1312-5/07/$25.00 © 2007 IEEE
797
Table. 1 Parameters of Traffic Model Traffic Model Parameters
Table Head copy
λ1,s
λ1,h
λ2,s
λ2,h
μ
0.2
0.1
0.1
0.05
20
The SA-CAC is compared with two basic CAC policies: 1) greedy and complete share based threshold CAC policy (GCST-CAC), in which the network prior reserve a set of bandwidth for the handoff calls so as to decrease CDR, and each accepted call is assigned by its maximum bandwidth requirement; 2) a greedy and complete share based CAC policy (GCS-CAC), this CAC policy is similar to the previous one except that it doesn’t reserve any bandwidth for the handoff calls. 25 simulation runs were performed in which NEAT attempt to discover good policies using the setup described in Section 3 and 4. Results shown below are the averaged outcome of 25 runs. In these simulations, the population size p was 100, the number of generations g was 100. Each policy was evaluated with 20000 events. See Table 2 for more details on the NEAT parameters used in our experiments. Table. 2 Result Comparison of three CAC policies Experiment Result Comparison
Table Head
CDR
CBR
U
GCS-CAC
3.24%
3.15%
1
GCST-CAC
20.8%
0.08%
0.88
0ˊ005%
0.005%
0.987
SA-CAC
Percentage of allocated BW levels
0.8
C1-setup C2-setup C1-handoff C2-handoff
0.4
0.2
0.0 1
2
3
4
5
6
7
8
9
CONSLUSION
We provide a different approach to perform CAC in adaptive multimedia networks, called SA-CAC. Each request call is assigned a certain amount of bandwidth according to the network load, which will not be adjusted during its lifetime. It can be seen as a tradeoff between having less QoS (bandwidth) allocated and reducing new and handoff call failures. This task is a constrained reinforcement learning problem and, a form of NeuroEvolution algorithm called NEAT is adopted to learn good CAC policies. The numerical results demonstrate that the proposed scheme is capable of maintaining the upper bounds of call blocking rate (CBR) and handoff failure rate (CDR), as well as obtaining much higher network utility than traditional threshold based CAC scheme. ACKNOWLEDGMENT Thanks Kenneth O. Stanley and Ugo Vieruccifor very much for providing free software of NEAT at http://www.cs.utexas.edu/~nn/index.php. REFERENCES
Figure 3 The percentages of allocated bandwdith levels
0
VI.
[1]
1.0
0.6
Figure 3 demonstrates the percentage of the allocated bandwidth levels for each kind of traffic. It can be seen that C1 has higher percentage to be allocated with maximum bandwidth level compares to C2, this is because C1 requires less bandwidth unit than C2 if the same utility acquired, and therefore the network can obtain high utility.
10
Bandwidth Level
Figure 3 Percentages of allocated bandwidth levels
Table 2 compares three different CAC schemes. It can be seen that SA-CAC almost eliminates the handoff failures and call blockings, and its utility is just 0.013 less than the GCSCAC policy. For GCST-CAC, although the handoff failure rate is small, the call blocking rate is too high to accepted, furthermore its utility is almost 10 percent less than SA-CAC. It is obvious that the SA-CAC is better than the traditional threshold based CAC policies.
Wee-Seng Soh and Hyong S. Kim, “Dynamic guard bandwidth scheme for wireless broadband networks”. IEEE INFCOM vol.1, pages 572581,2001. 0-7803-7016-3/01 [2] Sunghyun Choi, and Kang G. Shin.” Adaptive bandwidth reservation and admission control in QoS-sensitive cellular networks”. In IEEE Transactions on parallel and distributed systems, Vol. 13, September 2002 [3] V. Bharghavan, K. Lee, S. Lu, S. Ha, J. Li and D. Dwyer, “The TIMELY adaptive resource management architecture”, IEEE Personal Communications Magazine 5(8) (August 1998) [4] TaekYong Kwon, ,YangHee Choi, Chatschik Bisdikian, and Mahmoud Naghshineh “QoS Provisioning in Wireless/Mobile Multimedia Networks Using an Adaptive Framework,”wireless networks 9, 51-59, 2003. [5] Ning Lu, John Bigham, “Utility-Maximization Bandwidth Adaptation for Multi-Class Traffic QoS Provisioning in Wireless Networks,”1st ACM international workshop on Quality of service & security in wireless and mobile networks, 2005 [6] Chun-Ting Chou and Kang G. Shin, “Analysis of Combined Adaptive Bandwidth Allocation and Admission Control in Wireless Networks,” INFOCOM 2002 [7] Yang Xiao and C.L.Philip Chen “Improving Degradation and Fairness for Mobile Adaptive Multimedia Wireless Networks,”IEEE ICCN 2001 [8] Richard S. Sutton, Andrew G. Barto. “Reinforcement Learning: An Introduction.” MIT Press, Cambridge, MA, 1998. [9] Kenneth O. Stanley, “Efficient Evolution of Neural Networks through Complexification.” 2004, PhD thesis, Artificial Intelligence Laboratory, The University of Texas at Austin, TX 78712 [10] Matthew E. Taylor, Shimon Whiteson, and Peter Stone, “ Comparing Evolutionary and Temporal difference Methods in a Reinforcement Learning Domain.” In Proceeding of the Genetic and Evolutionary Computation Conference (GECC0-2006). Seattle, WA, July 2006.
1-4244-1312-5/07/$25.00 © 2007 IEEE
798
