Adaptive QoS Resource Management by Using Hierarchical Distributed Classification for Future Generation Networks Simon Fong Faculty of Science and Technology University of Macau, Macau SAR
[email protected]
Abstract. With the arrivals of 3G/4G mobile networks, a diverse and new range of applications will proliferate, including video-on-demand, mobile-commerce and ubiquitous computing. It is expected a sizable proportion of these traffics move along the networks. Resources in the networks will have to be divided between voice support and data support. For the data support, multiple classes of services from the new mobile applications that have different requirements have to be monitored and managed efficiently. Traditionally Quality-of-Service (QoS) resource management was done by manual estimation of resources to be allocated in traffic profiles in GSM/GPRS environment. The resource allocations parameters are adjusted only after some period of time. In this paper, we propose a QoS resource allocation model that dynamically monitors every aspect of the network environment according to a hierarchy of QoS requirements. The model can derive knowledge of the network operation, and may even pinpoint the cause, should any anomaly occurs or malfunctions in the network. This is enabled by a hierarchy of classifiers or decision-trees, built stream-mining technology. The knowledge from the classifiers is inferred by using reasoning-of-evidence theory, and it is used for subsequent resource allocation. By this way, the resources in the network will be more dynamically and accurately adjusted, and responsive to the fluctuating traffic demands. Keywords: QoS, Resource Management, Hierarchical Classifiers, Streammining.
1
Introduction
In the advent of 3G deployments and convergence of 4G mobile and wireless networks, a new breed of mobile applications emerged and their popularity increased at exponential rate. By the time of September 2010, Apple’s App Store has reached 260,000 apps for iPhone (source: kenyarmosh.com). This diversity of mobile applications demands on underlying resources from the networks. The demands shift dramatically both in amount, and in distributed manner in spatial-temporal contexts. Such demands are often tied with the term Quality-of-service (QoS) which is a generic term referring to a range of parameters and mechanisms targeted at provisioning network users with a satisfactory level or guarantee of consistency in the
delivery services they receive from the network. Supporting QoS for mobile users has always been an important area of research in mobile networks. In the past, most of the research efforts were focused on the connection level of resource allocations; areas of traditional QoS support methods, for example, admission control [1], scheduled resource allocation methods [2] and use of mobility and user characteristics [3] have been studied. Issues of QoS with respect to 3G networks and optimum resource allocation for multimedia traffic have been discussed by [4] and [5] respectively. In particular, the authors [6] stressed that specific QoS metrics are needed to represent adequately the ability of wireless networks to support mobile commerce transactions. The need could be generalized to other resource-demanding applications that generate and receive packet data in the network other than voice services. In response to the challenging issue of managing QoS, we in our previous work [7] has proposed a data-mining model which optimized the traffic profiles by analyzing the collected traffic statistics from the network. Radio planners consult the data-mining results for doing the optimization over the traffic profiles, such as how the proportion between channels to be allocated to voice and data traffic usage should be, and Erlang calculation for forecasting the anticipated loads etc. In GSM UMT, resource allocation was programmed; adjustment was not on the fly. Manual adjustment may be suitable for medium- or long- term network resource management. Upon the arrival of 3G and 4G services, we desire to have the resources be adjusted dynamically and more flexibly, in response to the actual situations of the network environment and the ever-changing users’ workloads. Resources in UMT are represented in specific parameters. The values of the parameters, although being collected and monitored, may not have a clear connection to the final perceived output of the users. In other words, a high level performance parameter may be a result of composite variables which span across various levels of technical communication structures. For example, a delay resulted from the acknowledgement of an m-commerce transaction may due to a net of reasons, and sometimes the reasons are implicit. Similarly a dropped call in the connection level may be due to a composite set of reasons in the underlying network. It is therefore desirable to string the QoS across the underlying layers of the protocols. So that more specific reasons and causes could be known, that would be useful for troubleshooting. Four QoS classes, conversational, streaming, interactive and background have been defined in the Third Generation Partnership Project (3GPP) in order to meet the new requirements. Another standard called ANWIRE grouping maps the user requirements into system concepts, grouped together in the categories of 3G systems, in a structural manner from user end-device to network and services [8]. Currently, the QoS terms and standards made available up-to-date are represented by a static structure. For example of ANWIRE has a static mapping of user requirements. User requirements are grouped according to different perspectives and metrics – user interface, access, security, contents, mobility and billing. The structure is layered and hierarchical in nature. The layers are logically placed from high to low level, similar to classical OSI protocol stack – requirements of user applications are on top, systems and network-oriented requirements are in the middle, and those of the underlying communication media are at the bottom. Those so-called QoS standards serve however largely as references for implementers who want to design a QoS manager over the network operations as well as the applications running on top.
In this paper, we propose a dynamic QoS management model that adapts to the current situations of the network. The model is based on real-time parameters currently being reflected from the network, which is an advantage over the other models. The parameters are measured and collected from a net of sensors distributed in various parts of the whole communication infrastructure. As a result negotiation of QoS between layers can be more precise. We know also precisely which components and where the network in the context of QoS structure deviate from the normal, should anything was perceived inadequate in the top level applications. Our model is composed of a hierarchy of classifiers, namely cascading classifiers, for governing the QoS parameters in the entire communication network architecture. The classifiers are constructed pertaining to each layer of the QoS communication architecture. The most bottom layer of the classifiers intake the most specific technical parameters in the lowest layer of the network. The individual classifiers from all layers form a global classifier that can be viewed as a flat classifier. However hierarchical classifiers have the advantages that different parts of the classifiers can be operated independently, covering almost all the measurable metrics of the communication network and services. In situations where a high level QoS negotiation takes place that requires a set of underlying components to agree on the service requirements the individual classifiers would serve as performance indicator for decision supports. The paper is organized in this way: The overall hierarchical QoS model is shown in Section 2, depicting which network communication parameters would be grouped together. Section 3 discusses about a framework for hierarchical classifiers as a general model upon which actual network communication QoS can be mapped. Some evaluation experiments were done under this the framework and the results are presented in Section 4. A conclusion is drawn at the final section.
2
2.1
Hierarchical QoS Model
Resource Management in Mobile Network
Mobile networks rely on trunking to accommodate a large number of users in a limited radio spectrum. The channels or circuits uses trunking theories to determine the number of phone circuits that need to be allocated for fixed locations (office buildings) with certain amount of telephones, and the same principle is used for designing mobile network. In order to design an efficient trucked radio system that can handle a specific capacity at a specific ‘grade-of-service’ (GOS) trunking theory and queuing theories are used, and the performance would have to be constantly monitored. GOS is a measure of the ability of user to access a trunked system during the busiest hour. Radio planners need to forecast an upper limit of capacity and to allocate certain number of channels so to meet the GOS. Traditionally GOS is defined as the probability that a call would be blocked, or the chance of a call experiencing a delay longer than the normal queuing time. Also radio frequency planner only uses a constant GOS value such as 2% or 5% in planning the capacity of the radio network.
To embrace a new generation of network applications and their demanding stringent resource usages, GOS that was a prime measure on the grade of service perceived from a mobile network may no longer be sufficient. A more sophisticated set of QoS such as ANWIRE as recently proposed would be needed to map over the physical network. In the later section, a hierarchical QoS model is introduced that covers through several layers of functions and parameters for referencing the current performance of a mobile application service (including the supporting servers and networks) as a whole. In our previous paper [7], data mining techniques were used to find out what the best design GOS value would be. The so-called best value of GOS for planning the radio network capacity was calculated from a range of network performance variables whose values were mined from the historical operational dataset. Using data mining it can find a best fitted GOS, and the related TCH value which can maximize the radio capacity and utilization while keeping the network in a healthy condition. For using data mining as an optimization technique on performance estimation, an additional server called Resource Management and Prediction Server (RRMPS) was added. The goal of RRMPS is to optimize the system traffic loading for different radio channel configurations in GSM cell sites. 2.2
The Concept of Layered Model
In our previous design, RRMPS was mainly responsible for taking care of the resources in the context of the networking infrastructure. It helps answering questions in the decision of system implementation, such as: How many channels or how much resources are available for each site/system by the governance of RRMPS. We expand this model further by incorporating the concept of hierarchy and a wide coverage of performance parameters, for a complete QoS model (which is not only centered at traffic channel or resource allocation). Conceptually the QoS model can be viewed in three layers of performance indicators which are closely inter-related with each other. For instance, the performance of a mobile commerce application is largely supported and hence affected by the performance of the related m-commerce servers as well as the network on which it operates.
Fig. 1. Three layered QoS model
Fig. 2. Mapping of the QoS groups on a network architecture As shown in Figure 1, three groups of QoS parameters namely, QoSAL, QoSSL, and QoSNIL are conceptually defined; each group of QoS concerns about the performance metrics pertaining to the layer of functions. The boundary of the QoS groups are drawn over the respective hardware and parts of network design (including the RRMPS and mobile-payment server [9]) for illustrative purpose in Figure 2. QoSNIL concerns the operations of the network as a whole infrastructure. QoSSL are measures mainly based on the corresponding computer servers that includes both the hardware, software, the operating systems and running processes) for supporting the user-level applications whose QoS are defined in QoSAL. With this layered hierarchy in place, the measurements of the corresponding functions as QoS metrics can be grouped and shown in Figure 3. The list in Figure 3 by no means is thoroughly complete as it serves as a demonstrative guide rather than a full reference, on how a hierarchical QoS model would look like.
3
Cascading Classifiers
For practical realization of the hierarchical QoS model which has the QoS metrics divided in multiple layers, a variant of data mining technique called Cascading Classifiers are needed. A classifier basically is a decision support mechanism that classifies a set of testing data into one of the predefined classes. The predefined classes are some nominal levels of quality that take values, e.g. Acceptable or Unacceptable; Excellent, Average or Poor in terms of service performance. Depending on the core classification algorithm to be chosen for implementing the classifier (whether it is a decision-tree or neural network, e.g.), the output could be either be in discrete nominal classes or a numeric continuous number. In essence, the
classifiers are first trained with historical data, and the trained model could be subsequently updated, such that it remembers certain rules and tendency for making a future prediction when new testing data are submitted upon. So when the classifiers have been trained and readily put into operation in QoS monitoring, the classifiers will be able to answer in real-time on whether a request from an upper level could be fulfilled and will meet the service requirements, given the current conditions of the network and servers.
Fig. 3. List of performance metrics displayed in a hierarchical QoS model.
Fig. 4. A comparison of hierarchical classifier and a classifier with a flat hierarchy Fundamentally, with all the performance metrics and the respective measurements available, a global classifier can be easily built, which is similar to constructing a single classifier in a flat hierarchy. By doing so, however, a single answer would be outputted by the global classifier with the inputs of the all the measurements entered. The single answer would be the prediction value of the net and final QoS, usually at the most top level. This violates the concept of the hierarchical QoS model which was defined in Section 2. It is desirable to know, instead of a singular final output, the QoS measures and estimated results in each layer, and in each part of the hierarchical network. Therefore, distributed QoS negotiation and management would be possible in different places, and that could possibly be automated by using Agents technology. Consequently this hierarchical QoS design and the cascading classifiers would enable the blue-prints of the distributed QoS management by Agents and Middleware as published in [10, 11, 12]. As shown in Figure 4, the shaded boxes represent the leaves of a decision tree classifier that is a measured value to be inputted to the classifier. In our case, the leaves are the individual measurement from the network, servers and applications. The leaves are the nodes at the bottom level in a decision tree. In the flat hierarchy, each of these leaves is used as inputs to the classifier. However, for hierarchical classifier there are some intermediate nodes in white color as shown in Figure 4. Some inductive reasoning technique is required to derive the values as prediction output for these nodes given their immediate subordinate nodes and/or leaves.
4
Experiment
In order to overcome this technical challenge, techniques of abstraction and aggregation in decision trees, as describe in [13] are used. A set of hierarchical classifier sample data downloaded from UCL repository that is popular among researchers for evaluating machine learning algorithms, is used in our experiment. We constructed a small representative of hierarchical decision tree which is not meant to be exhaustive but it elegantly embraces a hierarchy of certain depth with three typical
types of intermediate nodes (IN): IN nearest to the root and with direct leaves, IN below another IN and with direct leaves, and IN between root and another IN. The IN’s are labeled at N1, N2 and N3 as shown in Figure 5. Figure 5 shows an example of a hierarchical classifier that has a typical layered structure, and the nodes of the decision tree are taken from the QoS list for mobile network. This hierarchical classifier though simple, can be heuristically constructed up into a large spanning cascaded hierarchical classifier. In a cascaded hierarchical classifier, any hierarchical classifier can consist of smaller hierarchical classifiers, and it can also be a part of a larger classifier. In other words, it is scalable and it can potentially grow to infinite number of layers should the requirements of QoS evolve when future applications and technologies develop and become an integral part of the whole network. Readers who are interested in the theoretical formulation and algorithms of building cascading decision trees are referred to [14] for more details.
Fig. 5. A hierarchical classifier of mobile network QoS’s.
In our example as above, the ultimate predicted output is QoSNIL. QoSNIL is a composite metric that spans across the QOS hierarchy. It is composed of two major QoS metrics that have been simplified for the sake of experimentation: Service Integrity and Service Accessibility. These two are the service oriented metrics that in turn being composed for further metrics, such as quality-of-call related measurements and network accessibility related measurements respectively. By the Dempster-Shafer theory which is popular in evidence reasoning, multiple factors can be pieced together with certain inputs of evidence, contributing to the certainty of a particular outcome [15]. For the root class as well as other IN’s in this example, the predicted values are a range of ordinal data, namely Very good, Good, Acceptable, and Unacceptable. These abstract classes generally imitate those QoS classes of mobile network communication. When the “factors” which are the values of the leaves here are being combined, their relative importance commonly called weights are needed to be estimated. By the nature of decision tree, the weights of factors towards the IN’s intuitively should be in consistency with those weights of factors towards the overall output class – the root. The data distribution that ultimately biased towards one of the predicted classes for the root should be the same
for the IN’s. Based on this assumption, we calculate the relative weights for the factors for the whole decision tree as well as for the IN’s by applying popular feature selection techniques such as Chi-squared, Information Gain, Gain Ratio feature, ReliefF Ranking and Symmetrical Uncertainty Ranking. In general all these techniques are for calculating the correlations between the attributes (the factors) and the predicted class (the roots). The relative importances in terms of weights are computed by the Dempster-Shafer theory for the IN’s along the direction of abstraction in the spanning tree. The results are shown in the following Table. Table. 1. Results of weights at different level of nodes calculated by various methods.
The results from Table 1 show that Call setup time/delay in general carries the heaviest weight, followed by call blocking rate, call clarity and call interference. The results are quite consistent across various correlation computation methods, with deviation less than 15% in all cases. Chi-square technique is chosen for the subsequent calculation for the belief that Chi-square is based on Pearson correlation and is deemed suitable for handling nominal data which is the case in this experiment. A recent research work [16] has successfully used Dempster-Shafer theory in a cascading hierarchical classifier that was constructed by a spanning tree of neural network. In our experiment, we geared towards a flexible QoS model hence several classical machine learning algorithms for building the classifiers was attempted. The algorithms are: RepTree, M5P, Multilayer Perception and Linear Regression. These algorithms can be found from an open-source data mining package called Weka. RepTree is a quick decision tree learner. It builds a decision tree using information gain and prunes the resultant tree using back-fitting. In our experiment, RepTree achieved the lowest prediction accuracy (hence the best accuracy performance) among all. M5P, on the other hand, adds linear regression functions to a conventional decision tree at the nodes. Our experiment results show that M5P can achieve a more compact decision tree compared to RepTree.
Multilayer Perception is essentially a neural network algorithm with multiple layers of neurons for learning non-linear data patterns. In our experiment setting, all nodes including the root possess of linear class range (from very acceptable to unacceptable), plus the data distribution might also be linear. It suffered a high prediction error rate. Inclusion of Multilayer Perception is for the reason that to demonstrate it is possible to replicate the model by [16] for mobile network QoS. A lot of efforts however would be expected in fine-tuning the neural network settings for better accuracy. Linear Regression is the worst performer in the comparison because of the potentially high data dimensionality in the QoS framework, i.e., many attributes might be used collectively to predict a higher QoS result. Of all the algorithms, except Linear Regression, we can observe that generally a high degree of correlation between the attributes and the prediction target was yielded (ranging from 0.8969 to 0.9763). That means the distribution of the relative importance among the attributes or factors is quite well. As the results shown in Table 2, we repeated the same prediction performance evaluation experiment for abstract layers, n-1 and n-2. At level n which is the bottom level, all the leaves were used as attributes in the classifier and the classifier would perform as if it was a global classifier with a flat hierarchy. In level n-1, the attribute Call setup time, plus IN#1 Service Integrity and IN#2 Network accessibility, are used in the classification test. In level n-2, the most abstract level in this case, IN#1 and IN#3 Service accessibility are used in the classification test. All tests are pivoted at predicting the target class at the root which is the overall QoS of the network layer. RepTree outperformed the others for the classification tests at the abstract levels. Again, RepTree showed that the correlation net values at the abstract level are 0.997 and 0.993 respectively. That means the statistical relations between the attributes to the root target class are very stable even in the case of using cascading classifiers. Table. 2. Classification tests by using different algorithms at different abstract levels.
Lastly we want to test the performance of the individual classifiers in the QoS hierarchy. Specifically, the individual classifiers are taken independently with the IN#1, IN#2 and IN#3 as the root of the respective classifiers. For instance, we have a separate classifier with the Service integrity as the target class with only the two attributes Call clarity and Call interference. Likewise is for IN#2 and IN#3 where only the intermediate sub-ordinate attributes or nodes are used as inputs. The purpose of testing classifiers individually is to verify the hypothesis that the embedding classifier within a hierarchical classifier still works well and can achieve high classification accuracy.
For this round, the classification tests are repeated for individual decision trees headed by IN#1, IN#2, and IN#3. The results are shown in Table 3. The accuracies are relatively very high for the individual classifiers. In particular, Linear Regression succeeded with error-free because the individual classifiers are small and have no intermediate step. This set of tests shows embedding classifiers can possibly function independently and individually. At the same time, it is possible too to apply them operate collectively as building blocks for a larger hierarchical classifier. This is important because in distributed QoS management environment, it may be desirable to deploy local monitors at dispersed locations or spots of the network. Local QoS monitors can function independently for checking their specific aspects of the QoS, at the same time they can be jointly work together as a hierarchical classifier. Table. 3. Classification tests by using different algorithms at individual local classifiers.
5
Conclusion
For the future generation of network applications, resource management becomes more complex. The recent QoS specification by ANWIRE that maps a variety of users requirement to technical operational parameters naturally possess a hierarchy of functions. Hence a new breed of QoS monitoring and management framework would preferably take the shape of hierarchy like a spanning tree that covers all the functions in conceptual layers, from applications to underlying network communication. Such that available resources and the conditions of the network would be known in realtime, and QoS decisions could be made adaptively to the fluctuating demands of the users. In this paper, a novel QoS model was proposed that was built on cascading classifiers in a hierarchical tree. An example of QoS mapping was given as a reference guide, also the performance of the classifiers both global and local individuals were tested. The results show that it is computationally feasible to use this hierarchical QoS classification model. To summarize, this paper has two main contributions. First, it presents a model of hierarchical classification, discussing the main approaches and different techniques developed for solving the relevant challenges. Second, it discusses how this model of hierarchical classification could be applied to a very challenging and important problem in supporting mobile commerce and 3G/4G new applications, namely the dynamic resource allocations via using QoS classifiers for decision making.
References 1. Levine, D. A., I. F. Akyildiz, and M. Naghshineh. (1997). A Resource Estimation and Call Admission Algorithm for Wireless Multimedia Networks Using the Shadow Cluster Concept. IEEE/ACM Transactions on Networking, (5) 2. M. El-Kadi, S. O. and H. Abdel-Wahab. (2002). A Rate-based Borrowing Scheme for QoS Provisioning in Multimedia Wireless Networks. IEEE Transactions of Parallel and Distributed Systems, (13) 3. Ye, J., J. Hou, and S. Papavassiliou. (2002). Comprehensive Resource Management Framework for Next Generation Wireless Networks. IEEE Transactions on Mobile Computing, (1)4, pp.249–264 4. Maniatis, S., E. Nikolouzou, and I. Venieris. (2002). QoS Issues in the Converged 3G Wireless and Wired Networks. IEEE Communications Magazine, (40)8, pp.44–53 5. Chen, H., Zeng, Q.-A., and D. P. Agrawal, (2002). A Novel Analytical Model for Optimal Channel Partitioning in the Next Generation integrated Wireless and Mobile Networks. Proceedings of the 5th ACM International Workshop on Modeling analysis and Simulation of Wireless and Mobile Systems, pp. 120–127. 6. Ahluwalia, P. and U. Varshney (2003). A Link and Network Layer Approach To Support Mobile Commerce Transactions. In IEEE 58th Vehicular Technology Conference, 2003, (5), pages 3410–3414 7. Edison Lai, Simon Fong, Yang Hang, Supporting Mobile Payment QOS by Data Mining GSM Network Traffic, The 10th International Conference on Information Integration and Web-based Applications & Services (iiWAS 2008), Linz, Austria, 24-26 November 2008, ACM, ISBN:978-1-60558-349-5, pp.279-285 8. User requirements for next generation networks, D1.1.1, IST-2001-38835 ANWIRE, November 2002 9. Simon Fong, Edison Lai, Mobile Mini-payment Scheme Using SMS-Credit, The International conference on computational science and its applications (ICCSA 2005), Singapore, 9-12 May 2005, Lecture notes in computer science, ISSN 0302-9743, Springer, Berlin, pp.1106-1116 10.Kosuga, M., Yamazaki, T., Ogino, N., Matsuda, J., "Adaptive QoS management using layered multi-agent system for distributed multimedia applications", International Conference on Parallel Processing, 1999, Japan, pp.388-394 11.Denise J. Ecklund, Vera Goebel, Thomas Plagemann and Earl F. Ecklund Jr., "Dynamic end-to-end QoS management middleware for distributed multimedia systems", Special issue on Multimedia Middleware, Multimedia Systems, Springer, Volumen 8, No.5, pp.431-442 12.Xuan Thang Nguyen, "Agent-Based QoS Management for Web Service Compositions", PhD Thesis, Swinburne University of Technology, Australia, June 2008 13.Simon Fong, Alice Tang, A Taxonomy-based Classification Model by Using Abtraction and Aggregation, The 2nd International Conference on Data Mining and Intelligent Information Technology Applications (ICMIA 2010), 30 November-2 December 2010, Seoul, Korea, pp.448-454 14.Milton García-Borroto, José Fco. Martínez-Trinidad and Jesús Ariel Carrasco-Ochoa, "Cascading an Emerging Pattern Based Classifier", Advances in Pattern Recognition, LNCS 2010, Volume 6256/2010, pp.240-249 15.Kari Sentz, Scott Ferson, "Combination of Evidence in Dempster-Shafer Theory", SAND 2002-0835, April 2002, pp.3-96 16.Rebecca Fay, Friedhelm Schwenker, Christian Thiel, and G¨unther Palm, "Hierarchical Neural Networks Utilising Dempster-Shafer Evidence Theory", ANNPR 2006, LNAI 4087, Springer, pp.198–209
