Globecom 2012 - Communications QoS, Reliability and Modelling Symposium
ANFIS-based Quality Prediction Models for AMR Telephony in Public 2G/3G Mobile Networks Charalampos N. Pitas*1, Dimitris E. Charilas*2, Athanasios D. Panagopoulos*3, Periklis Chatzimisios†, Philip Constantinou*4 * Mobile Radiocommunications Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, Greece Email: {1chpitas, 2dcharilas, 3thpanag, 4fkonst}@mobile.ntua.gr † CSSN Research Lab, Department of Informatics, Alexander TEI of Thessaloniki, Greece Email: †
[email protected] Abstract—The scope of this research paper is the proposal of voice quality prediction models, based on Adaptive Neuro-Fuzzy Inference Systems (ANFIS), for Adaptive Multi Rate (AMR) telephony service provided by GSM and UMTS rollout public land mobile networks. For the scope of our research we performed an experimental drive-test measurement campaign in order to evaluate objectively the End-to-End Service QoS (ESQoS) as well as radio System QoS (SQoS) parameters. Subsequently, the collected measurement data are used to train ANFIS empirical models. We then assess the prediction performance by depicting 2D/3D FIS surfaces and the impacts of SQoS to ESQoS. The prediction methodology can be successfully applied in Quality of Experience centric radio network planning, fine tuning and optimization processes by mobile operators. Keywords- Adaptive Network-based Fuzzy Inference Systems (ANFIS), Fuzzy Logic, Quality of Service, Speech Quality, Mean Opinion Score (MOS)
I.
INTRODUCTION
Nowadays, both Global System for Mobile Communications (GSM), and Universal Mobile Telecommunication System (UMTS) have been developed all over the world and the most used provided service is the classic mobile telephony. While Radio Access Technologies (RATs) are being spread, the basic Radio Access Networks (RANs), GSM/EDGE (Enhanced Data Rates for GSM Evolution) RAN (GERAN) and UMTS RAN (UTRAN), deliver reliably Circuit Switched (CS) voice telephony. Voice quality is the most significant key performance indicator that characterizes the Quality of Experience (QoE) of the end user. For this scope, mobile operators conduct extensively and continuously drive-test measurements campaigns and network monitoring for performance evaluation and service/system optimization. Endto-End Service QoS (ESQoS) and radio System QoS (SQoS) performance parameters are described in [1]. Radio coverage is of crucial importance in mobile networks. On the one hand, the most important radio parameters for GSM mobile networks are clearly denoted in [2]: RxLev (dBm) (received signal level) is based on field strength of serving cell and RxQual (received signal quality) is based on the BER (bit error rate) rating from 0 (excellent) to 7 (bad). SinceRxLev and RxQual are generic terms, RxLevFull and RxLevSub as well as RxQualFull and RxQualSub can be measured. On the other hand, UMTS coverage is defined
sufficiently [3] by the Common Pilot Channel (CPICH) and the most important and useful parameters are CPICH Received Signal Code Power (RSCP) and CPICH Ec/I0. RSCP (dBm) is the field strength measured and Ec/I0 (dB) is the ratio of the received energy per chip for the CPICH to the total received power spectral density at the antenna connector of the user equipment. Voice compressor-decompressor (codec) systems used in mobile communications can be configured to demand bandwidth, but there is a trade-off between voice quality and bandwidth use. The best codecs require the greatest bandwidth reservation while producing the least degradation of voice quality. AMR codec [4] is used for speech coding in GSM and UMTS networks. AMR codec use a set of modes in Full Rate (FR) channel as well as in (Half Rate (HR) channel. Enhanced Full Rate (EFR) [5] has also been adopted as an industry standard codec for GSM. AMR WB (AMR Wideband) [6] and AMR-WB+ (Extended AMR WB) codecs [7] are used mainly in UMTS and Packet-Switched Streaming service (PSS), MMS (Multimedia Messaging Service) and MBMS (Multimedia Broadcast and Multicast Service). Perceptual assessment of speech quality is the crucial metric for end-to-end QoS telephony service, since speech transmission over public mobile networks is subject to quality degradation. A well-known algorithm that estimates and compares the original speech signal with the received one, which is degraded due to mobile radio channel, is the perceptual evaluation of speech quality (PESQ); the latter is recommended in ITU-T Rec. P862 [8]. Furthermore, PESQLQ (Listening Quality) modified the score to improve correlation with subjective test results at the high and low ends of the scale where the raw PESQ score was found to be less accurate. A mapping described in ITU-T Rec. P.862.1 [9] was been released that further modified the raw score and correlated better to subjective testing. Next generation voice quality testing algorithm is POLQA (Perceptual Objective Listening Quality Analysis) or ITU-T Rec. P863 [10] that provides significantly improved accuracy. A contemporary research area in QoE is quality prediction which can be performed by computational models [11]. In this paper, we build prediction models using the fuzzy logic approach that has been proposed in the literature for intelligent network selection. In particular, we employ ANFIS in order to predict objective ESQoS from SQoS. The paper is structured
1746
Figure 1.Quality prediction methodology.
as follows: In Section II, we present our quality prediction methodology which is based on measurement campaign data and ANFIS platform. Section III, is devoted to ANFIS configuration details. Numerical results and 2D/3D surfaces are provided in Section IV. Finally, the paper closes with Section V where a general discussion on conclusions takes place. II.
QUALITY PREDICTION METHODOLOGY
We propose a quality prediction methodology based on drivetest campaign for measurements data acquisition and fuzzy inference systems for quality estimation, as depicted in Fig. 1. A. Measurement Campaign SwissQual Diversity Benchmarker [12], which is integrated end-to-end QoS measurement equipment, was used for the experiment campaign and it consists of a test mobile station and a fixed-side voice server. The mobile measurement station is a set of test-mobile phones that are connected to the central processing unit and controlled by a laptop. A GPS receiver is used for geo/time reference and a GSM/UMTS fast frequency scanner is used for passive radio coverage measurements. The modular architecture of the system supports more than one test devices in order to conduct performance benchmarking for all mobile operators and all available radio access technologies (2G/3G). The mobile measurement station was installed in a car for drive-test measurements campaign. The fixed-voice server was connected to six dedicated ISDN-lines. Both mobile and fixed measurement systems are synchronized via initializing calls before the experiment starts. The service under test was CS-voice telephony in GSM and UMTS networks. Thanks to a measurement script, voice samples were exchanged between the mobile and fixed sides of the measurement system (uni-directional) during test voice calls that were set up along the campaign. Test Mobile Originating and Mobile Terminated Calls (MOC/MTC) were sequentially performed during the campaign time. An experimental measurement campaign was conducted in the metropolitan area of Athens (Greece), where three mobile operators have developed 3G and 2G licensed RANs. Thus, the measurement equipment was configured with three test handsets locked in GSM and another three in UMTS. QoS aspects for popular services in GSM and 3G networks are specified by ETSI in [13]. B. Measurements Data At the end of the campaign measurement files were collected both from mobile station and fixed-server side and they were
introduced in geo-temporal data base using NetQual NQDI for post processing and Microsoft SQL Server for data base management. Voice samples were evaluated with PESQ algorithm and objective MOS values were computed with SwissQual Squad algorithm. For the scope of our work, we extracted and classified data with SQL according the following criteria: a) RAN: GSM or UMTS b) Voice codecs: AMR HR, AMR FR and EFR for GSM, and AMR-WB for UMTS. c) Direction of voice sample: uplink (AB), is the direction from the handset to the voice server, and downlink (BA), is the direction from the voice server to the test handset. In the case of GSM test voice calls, the useful information was test voice call id, call type (MOC/MTC), mobile operator, voice sample id, direction of voice sample (A2B, B2A), voice codec (AMR FR/HR, EFR), average values of RxLevSub (in dBm) and RxQualSub ([0,7]) during the voice sample, voice quality performance (PESQ, PESQ-LQ, obj. voice MOS). In the case of UMTS test voice calls, the useful information was test voice call id, call type (MOC/MTC), mobile operator, voice sample id, direction of voice sample (A2B, B2A), voice codec (AMR WB), average values of CPICH RSCP and Ec/I0 during the voice sample, voice quality performance. C. FIS Construction These measurement data sets collected from drive-test campaign are ideal for the next step of our methodology, which is to build empirical voice quality prediction models based on ANFIS. For this reason, we used MATLAB Fuzzy Logic Toolbox [14]. We then constructed 8 fuzzy inference systems (FIS) for uplink and downlink; 2 for GSM AMR-FR codec, 2 for GSM AMR-HR codec,2 for GSM and EFR codec, and 2 for UMTS AMR-WB codec. Each FIS has two inputs and one output: the output is always the objective MOS for voice, while the input parameters are RxLevSub, RxQualSub for GSM and CPICH RSCP, Ec/N0 for UMTS. A critical point of the analysis is the configuration and optimization of ANFIS. The systems are then trained with the previously acquired data sets, as it is explained in the following section of the paper. III.
ANFIS CONFIGURATION
ANFIS is a FIS whose membership function parameters are adjusted using either a back propagation algorithm alone or in combination with a least squares type of method. A fuzzy
1747
system of this kind that is able to learn from the data is modeling, since it has a structure similar to that of a neural network. The parameters associated with the membership functions change through the learning process. For ANFIS implementations, we utilized the Sugeno method of fuzzy inference [14].
It can be easily observed from all the sub-figures that no linearity exists in the majority of cases; this is explained by the innate variance of the training data sets or could also be the result of few faulty measurements. As a general remark, the prediction model is conservative and strict compared to the actual measurements; and in most cases the predicted values are slightly lower than the measured ones. V.
Figure 2. ANFIS configuration for training of UMTS FIS (A2B)
In order to train the FIS, the ANFIS configuration screen must be invoked (see Fig. 2 for the case of UMTS uplink). The generation of the FIS can be performed by either Grid Partitioning or Subtractive Clustering, using various configurations. For the purposes of this study we have used Subtractive Clustering, with the following configuration: Range of Influence varying from 0.2 to 0.5, Squash Factor 1.25, Accept Ratio varying from 0.7 to 0.9 and Reject Ratio 0.15. We have experimented with a large number of configurations to find the most suitable ones according to the structure and variance of our specific data sets. The configuration of ANFIS depends on the statistical parameters of the training data. After the training phase, additional datasheets are deployed to test and check the generated FIS. Due to the adaptive and dynamic nature of ANFIS the FIS is altered so as to incorporate the newly-received data and thus optimize its performance.
In this paper, we have proposed an ANFIS approach for the voice quality prediction in mobile communication systems. In particular, we took advantage of the drive-test measurement campaign of commercial mobile networks and we used the measurements data in order to train and optimize a large number of configurable FIS. We then presented indicative results from the prediction models. The prediction methodology can be easily applied in QoE-centric mobile network planning as well as for quality performance benchmarking purposes. Future research could cover packet switched or Voice over IP (VoIP) mobile telephony in Beyond 3G (B3G), like evolved High Speed Packet Access (HSPA) and 4G (Long Term Evolution) mobile networks. ACKNOWLEDGEMENTS Measurement equipment was acquired by MRC-Lab NTUA during “05ΑΚΜΩΝ-95” project funded by the General Secretariat of Research and Development, Greece. The research is co-funded by the European Social Fund and National Resources ESPA 2007-2013, EDULLL “Archimedes III”.
REFERENCES [1] [2]
[3] [4] [5]
IV.
NUMERICAL RESULTS
Fig. 3 collects the 3D representation of the constructed prediction systems. Each surface demonstrates how the output (objective MOS) is affected by the two input variables. In particular, 3D FIS surface views are depicted for GSM Telephony with AMR FR (Fig.3a), AMR HR (Fig.3b) and EFR (Fig.3c) both for uplink and downlink directions. Objective voice quality is estimated by RxLevSub and RxQualSub. It is obvious that RxQualSub is a crucial indicator of quality performance. Moreover, voice quality is predicted for UMTS telephony with AMR codec based on CPICH RSCP and Ec/N0 in Fig.3d. Quality degrades dramatically as Ec/N0 is decreased. Finally, in Fig. 4 we consolidate the impacts of key SQoS parameters, RxLevSub for GSM (Figs 4a, 4b, 4c) and Ec/N0 (Fig.4d), to objective voice ESQoS. In case of GSM, eight RxQualSub (for all values {0, 1 , 2,…, 7}) dependent curves are presented for each GSM codec as well as for each transmission direction. Fig.4d likewise shows five Ec/N0 (for the discrete values {-3,-5,-7,-9,-11} dB) dependent curves.
CONCLUSIONS
[6]
[7]
[8] [9] [10] [11]
[12] [13]
[14]
1748
ETSI TR 126 944, v10.0.0. “End-to-end multimedia services performace metrics”. ETSI, Technical Report, Sophia Antipolis, Apr 2011. ECC Rep.118, “Monitoring Methodology to Assess the Performance of GSM Networks,” Electronic Communications Commitee CEPT, Athens, Feb. 2008. ECC Rep. 103, “UMTS Coverage Measurements,” Electronic Communications Commitee-CEPT, Nice, May 2007. ETSI TS 126 071 v.10.0.0. AMR speech Codec; General description. ETSI, Technical Specification, Sophia Antipolis, Apr 2011. ETSI EN 300 726 v.5.2.1. Enhanced Full Rate (EFR) speech transcoding. ETSI Standard, Sophia Antipolis, Jul 1999. ETSI TS 126 190 v.10.0.0. Adaptive Multi-Rate - Wideband (AMRWB) speech codec; Transcoding functions. ETSI Specification, Sophia Antipolis, Apr 2011. ETSI TS 126 290 v.10.0.0. Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Transcoding functions. ETSI Specification, Sophia Antipolis, Apr. 2011. ITU-T P.862. Perceptual evaluation of speech quality (PESQ). ITU-T Recommendation, Geneva, Feb 2001. ITU-T P.862.1. Mapping function for transforming P.862 raw result scores to MOS-LQO. ITU-T, Recommendation, Geneva, Nov 2003. ITU-T P.863. Perceptual Objective Listening Quality Assessment (POLQA). ITU-T, Recommendation, Geneva, Jan 2011. A. Khan, L. Sun and E. Ifeachor, “QoE Prediction Model and its Application in Video Quality Adaptation over UMTS Networks”, IEEE Transactions on Multimedia,, vol. 14, issue 2, pp. 431-442, April 2012. SwissQual AG, “Diversity Benchmarker”; http://www.swissqual.com/. ETSI TS 102 250, Parts 1-7. Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks. ETSI, Technical Specifications, Sophia Antipolis. Mathworks, “Fuzzy Inference Systems:: Tutorial (Fuzzy Logic Toolbox)”; http://www.mathworks.com/.
a. Surface Views for GSM Telephony with AMR FR (left: uplink, right: downlink)
3.8
3.5
3.4
3 obj.MOS
obj.MOS
3.6
3.2 3
2.5 2
2.8
1.5 6
-50 4
1 7
-60
-70
-50 -60 -70 6
5
-80
2 -100
0
RxQualSub
RxLevSub(dBm)
-80
4
-90
3
-90
2
1
0
RxQualSub
-100
RxLevSub(dBm)
b. Surface Views for GSM Telephony with AMR HR (left: uplink, right: downlink)
4.5 3.6 3.4
4 obj.MOS
obj.MOS
3.2 3 2.8
3.5
3
2.6 2.4
2.5 6 -60
4
6
-100
0
RxQualSub
-60
4
-80
2
-80
2
RxLevSub(dBm)
0
RxQualSub
-100 RxLevSub(dBm)
c. Surface Views for GSM Telephony with EFR (left: uplink, right: downlink)
4 4 3.5 obj.MOS
obj.MOS
2
3
0 -2
2.5 -4 6
6 4 2 0
RxQualSub
-100
-90
-80
-70
-60
4
-50
-60
-80
2 0
RxQualSub
RxLevSub(dBm)
-100
RxLevSub(dBm)
d. Surface Views for UMTS Telephony with AMR (left: uplink, right: downlink)
4
4
obj.MOS
obj.MOS
3.98 3.96 3.94 3.92
3.9
3.8
3.7 -4
-60 -6
-8
-4
-80 -10
Ec/No(dB)
-12
-14
-100 RSCP(dBm)
-60 -6
-8
-80 -10
Ec/No(dB)
-12
-14
-100 RSCP(dBm)
Figure 3.3D FIS surface views: GSM telephony with (a) AMR FR, (b) AMR HR and (c) EFR, and UMTS telephony with (d) AMR-WB.
1749
a. Impact of RxLevSub to obj. MOS for GSM Telephony with AMR FR (left: uplink, right: downlink) 4
4 3.9
3.5
3.8 3.7
3 obj.MOS
obj.MOS
3.6 3.5
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub
3.4 3.3 3.2 3.1 3 -95
-90
-85
-80 -75 -70 RxLevSub (dBm)
-65
-60
=0 =1 =2 =3 =4 =5 =6 =7
2.5
2
1.5
1 -95
-55
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub -90
-85
=0 =1 =2 =3 =4 =5 =6 =7 -80
-75
-70
-65
-60
-55
RxLevSub (dBm)
b. Impact of RxLevSub to obj. MOS for GSM Telephony with AMR HR (left: uplink, right: downlink) 3.8
5
3.6
4.5
3.4
4
obj.MOS
obj.MOS
3.2 3 2.8 2.6 2.4 2.2 -95
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub -90
-85
=0 =1 =2 =3 =4 =5 =6 =7
3.5
3
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub
2.5
2
-80 -75 -70 RxLevSub (dBm)
-65
-60
1.5 -95
-55
-90
-85
-80 -75 -70 RxLevSub (dBm)
-65
-60
= = = = = = = =
0 1 2 3 4 5 6 7
-55
c. Impact of RxLevSub to obj. MOS for GSM Telephony with EFR (left: uplink, right: downlink) 4
4.5
3.8
4
3.6
3.5 3
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub
3.2 3 2.8 2.6
=0 =1 =2 =3 =4 =5 =6 =7
obj.MOS
obj.MOS
3.4
2
RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub RxQualSub
1.5 1
2.4 2.2 -95
2.5
0.5 -90
-85
-80 -75 -70 RxLevSub (dBm)
-65
-60
0 -95
-55
-90
-85
-80 -75 -70 RxLevSub (dBm)
=0 =1 =2 =3 =4 =5 =6 =7
-65
-60
-55
d. Impact of RSCP to obj. MOS for UMTS Telephony with AMR (left: uplink, right: downlink) 4.01
3.99
obj.MOS
3.98
4.05
Ec/No = -3 dB Ec/No = -5 dB Ec/No = -7 dB Ec/No = -9 dB Ec/No = -11 dB
4
3.95 obj.MOS
4
3.97
3.9
3.96
3.85 3.95 3.94 3.93 -100
Ec/No = -3 dB Ec/No = -5 dB Ec/No = -7 dB Ec/No = -9 dB Ec/No = -11 dB
3.8
-95
-90 -85 RSCP (dBm)
-80
3.75 -100
-75
-95
-90 -85 RSCP (dBm)
-80
-75
Figure 4.Impact of RxLevSub and CPICH RSCP to objective MOS. For GSM and UMTS various values of RxQualSub and CPICH Ec/N0 are considered respectively.
1750
