Page 1 of 6
1
Unified Simulation Evaluation for Mobile Broadband Technologies Yuehong Gao, Xin Zhang, Dacheng Yang and Yuming Jiang Abstract—System level simulation has been widely used to evaluate system performances. Simulation methodologies have also been comprehensively discussed in different organizations or institutions. Most of the simulation methodologies proposed however mainly focus on one area of specific technologies. How to evaluate the performance of multiple air-interface systems (such as cdma2000, WCDMA, WiMAX and their evolutions) in a fair and comprehensive manner has not been addressed extensively. This article presents a unified simulation methodology, including fading channel models, system configurations, and how to consider technology-dependent algorithms, such as scheduling, overhead modeling, interference margin definition, and resource allocation based on system loading. This article uses this framework to compare three major existing radio technologies: cdma2000 1x EV-DO Rev.A, WCDMA HSPA and Mobile WiMAX based on 802.16e. Key simulation results based on our suggested system models and settings are presented and analyzed. It is shown that under our unified framework, the two CDMA systems exhibit higher spectrum efficiency than Mobile WiMAX, especially on the downlink, while Mobile WiMAX provides higher peak rate. Keyword—performance evaluation, simulation methodology, wireless system
I. INTRODUCTION
while rooted from 3GPP (3rd Generation Partnership Project), 3GPP2, as well as IEEE (Institute of Electrical and Electronics Engineers). There are two 3G CDMA (Code Division Multiple Access) variants widely commercialized or in the process of being commercialized today, which are cdma2000 and WCDMA (Wideband CDMA). They are FDD (Frequency Division Duplex) solutions based on 1.25 MHz and 5 MHz bandwidth, respectively. Another broadband technology, mobile WiMAX (World Interoperability for Microwave Access), which is based on OFDMA (Orthogonal Frequency Division Multiple Access) technology with a relatively wide band, typically 10 MHz, and TDD (Time Division Duplex), is also deployed in some part of the world. In order to improve the capacity, 3GPP, 3GPP2, WiMAX Forum and other organizations have developed corresponding evolutionary systems, such as the AIE (Air Interface Evolution) for cdma2000 and LTE of WCDMA, and 802.16m in IEEE. To evaluate the expected performance of these mobile systems, multiple simulation methodologies have been proposed. Each organization has published the simulation methodology for the system it standardizes. For example 3GPP2 published the methodology used for cdma2000 1x EV-DO/EV-DV (Evolution-Data and Voice) evaluations in [1] and 3GPP issued [2, 3] for WCDMA. These methodologies also evolve with the standard progressions. Each of them, however, focuses only on one specific system. How to evaluate the performance of these systems in a fair manner under unified settings has not been addressed extensively. WiMAX Forum has proposed simulation methodologies that compare against the three systems and provided some results such as [4], but there are still some issues regarding fairness, some configurations and methodology that need to be clarified, as for example, advanced receivers for CDMA systems have not been considered, and overhead assumptions have been simplistic. Other questions are also open as what fading channel model should be used, how to evaluate the key technologies involved and network performance, and how to make use of simulation for standardization process. It is our major task of this article to evaluate the performances of the systems mentioned above in a unified manner, and to clarify the issues between different simulation methodologies developed by various standardization bodies so as to obtain reasonable results as much as possible. We give an evaluation of each system performance based on comprehensive consideration of system overhead, receiver setting, fairness in resource allocation, and unified channel
iew
ev
rR
Fo
Manuscript received June, 2008. This work was supported in part by QUALCOMM Inc.. Yuehong Gao is a Ph.D candidate of the Wireless Theories and Technologies (WT&T) Lab in Beijing University of Posts and Telecommunications (BUPT), Beijing, China, and she is also a joint-PhD student in Centre for Quantifiable Quality of Service in Communication Systems (Q2S), Norwegian University of Science and Technology (NTNU), Trondheim, Norway. Xin Zhang., is an associate professor of Wireless Theories and Technologies (WT&T) Lab in Beijing University of Posts and Telecommunications (BUPT), Postbox 93, Beijing, 100876, China (phone and fax:86-6228-3232, e-mail:
[email protected]). Dacheng Yang is a professor of Wireless Theories and Technologies (WT&T) Lab in Beijing University of Posts and Telecommunications (BUPT), Beijing, China. Yuming Jiang is a professor of the Centre for Quantifiable Quality of Service in Communication Systems (Q2S), Norwegian University of Science and Technology (NTNU), Trondheim, Norway.
ly
Wireless communication continues to evolve rapidly since the 1990s. 3G (3rd Generation) technologies such as cdma2000 EV-DO (Evolution-Data Optimized) and WCDMA HSPA (High Speed Packet Access) are undergoing commercialization today, while new technologies such as LTE (Long Term Evolution) are being adopted as B3G (Beyond 3rd Generation) systems. Most of the commercial mobile systems today are evolved from IMT-2000 (International Mobile Telecommunications - 2000), defined by a set of ITU (International Telecommunication Union) recommendations,
On
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Page 2 of 6
2 models. Our results are obtained via computer simulation under an appropriately simplified deployment scenario designed for theoretical analysis. Comparison between these technologies under the unified framework provides insight to the related key air interface techniques and leads to better understanding of the potential of commercial wireless communication networks. In addition to establishing a comprehensive simulation methodology for comparing the performances of cdma2000 1x EV-DO Rev.A, WCDMA HSDPA/HSUPA (High Speed Downlink/Uplink Packet Access) and Mobile WiMAX, this article also highlights the features of air interfaces and points out the key design advantages of different systems while making comparisons between them. The rest of this article is organized as follows. First, existing simulation methodologies and conditions are briefly reviewed in Section II. A unified methodology is proposed in Section III, which includes the evaluation workflow, introduction to simulation platform, the channel model, key technologies, overall system parameters and platform calibration methods. Further discussions are presented in Section IV with conclusions in Section V and acknowledgement in Section VI.
rR
Fo
II. SIMULATION METHODOLOGIES: A REVIEW
ev
System level simulation has been widely used to evaluate system performances with incorporation of link level simulation curves. Figure 1 shows the typical simulation methodology model. System level simulation concentrates on evaluation of the performance when multiple BSs (Base Stations) and MSs (Mobile Stations) operate together, while link level simulation generally focuses on the single transmitter-to-receiver link performances. The results from the link level simulation are usually adopted as inputs of the system level simulation. Simulation methodologies have also been comprehensively discussed in different organizations or institutions. International organizations and companies have put much effort to develop the simulation methodology of cdma2000 systems and its evolutions. Simulation methodology used for cdma2000 has been explicitly presented, i.e. 3GPP2 C.R1002 [1]. Basic system level simulation models for both FL (Forward Link, or DL/downlink) and RL (reverse link, or UL/uplink) are defined. Some simulation results are given in [1] as well. The report [1] has been updated to the 7th version, in which ITU channel models, SCM (Spatial Channel Model), C/I modeling for simulation, simulation flow model used for wrap-around method, universal service models (such as VoIP, Voice over IP) are added. 3GPP2 published the technical report of C.R1008-0 v1.0 in 2007, which provides the evaluation methodology of multimedia services and presents part of the results. 3GPP2 also developed C.R1009-0 v1.0 and C.R1010-0 v1.0. The former one is a software tool to implement multimedia simulation while the latter one is to implement H.263 video codec in simulation.
Traffic Model System Level Simulation Radio Resource Management (SLS) (Scheduling Algorithms, Power Control and etc.)
Interface between SLS & LLS
Link Level Simulation (LLS)
Physical Layer Wireless Channel
Figure 1. Simulation Methodology Model
IEEE Publications embody a great deal of papers with regard to cdma2000, among which majority involve simulation methodology and results, including radio resource assignment and scheduling models, handoff algorithms, channel estimation and other newly proposed methods. 3GPP also puts much effort to simulation method for WCDMA and its evolutions, and many contributions have been submitted and discussed. These documents involve cell layout model, antenna pattern, channel model, typical simulation parameters and other key factors when evaluating WCDMA and its enhanced systems such as HSPA and LTE. In addition, a part of link level curves and system level simulation results are also been included. In [5], link level curves and system performance are presented. Ref. [2], however, mainly focuses on evaluating HSUPA with different system configurations and typical simulation scenarios included. Suggestions on improved methodology after introducing OFDM technology are discussed in [6], where several paramount aspects were mentioned, including link level assumptions, mapping of SINR (Signal to Interference plus Noise Ratio) when using Rake receiver, calculation of downlink C/I (Carrier power to Interference power), feedback mode of uplink CQI (Channel Quality Indicator), different scheduling algorithms, implementation of HARQ (Hybrid Automatic Repeat reQuest） technology and etc. As for system level, simulation methods of different service models, such as Full Queue and Burst Traffic, were also presented. In a word, 3GPP and its members have tightly tracked the improvements of system simulation methodology along its evolutionary road, and its methodology has been commonly adopted when evaluating system performances. WiMAX has gained a lot of attention since it participated into the evaluation process as a 3G candidate system in ITU-R. Some organizations proposed simulation methodologies to evaluate WiMAX system performances; for example, WiMAX forum published a series of white papers on the technical overview as well as performance evaluations and comparisons [4, 7]. Evaluation methodologies proposed by WiMAX forum
iew
ly
On
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Page 3 of 6
3 cover every aspect of simulation. The outcome includes link level and system level results. System modeling contains: cell layout, interference calculation model, mobility model, channel model, traffic model, implementation of key technologies (such as power control, HARQ and handover), interface between system level and link level, various output metrics and so on. Typical WiMAX system configurations and parameters as well as corresponding results with SIMO (Single Input Multiple Output) and MIMO antennas are listed in [7]. Ref. [4] mainly focuses on comparisons between WiMAX and other 3G/B3G systems. WiMAX capacity of VoIP traffic, performances of handover and data traffic under specific simulation parameters are presented in [8]. In a word, many organizations, among which WiMAX Forum is the leading one, involve in developing simulation methodologies for WiMAX, and have established an architecture including the descriptions of every aspect.
Fo
III. UNIFIED METHODOLOGY: CHANNEL MODEL, KEY TECHNOLOGIES, SYSTEM CONFIGURATIONS, CALIBRATION AND OUTPUT MATRICES
rR
Through the review of existing simulation configurations and method published by various organizations or researchers, we found that each proposed methodology aimed at one specific system, and not quite applicable to other systems. If different simulation methods or configurations are used to evaluate systems performance, the comparison-fairness baseline cannot be guaranteed because of inconsistent criteria. Hence, we propose the unified simulation configurations in this article after carefully analyzing each paramount factor, such as channel model, scheduling algorithms, fairness rule and primary system parameters. In this section, the unified approach is introduced first, and then a unified set of channel models is proposed. The simulation methods of key technologies, such as scheduling, are described. Primary system parameters are also summarized in this section. In addition, we study the calibration of fairness and stability for various systems. At last, we give the output matrices of various systems under our simulation.
methodology model is as plotted in Figure 1. The typical simulation operates in a top-down-first and bottom-up-later manner, which can be divided into 4 layers as Figure 1 shows. The link level simulation is abstracted into results curves and the statistic data are mapped through the interface to the system level to finish one simulation circle. As far as the number of simulation runs is large enough, the simulated system will reach a converged state, when simulation outcomes are collected. We use full buffer as the traffic model, which is not practical in real systems but it is helpful when comparing the upper bound of throughput. In the radio resource management layer, scheduling algorithms are optimized for each system for the sake of comparing the throughput bound. The physical layer parameters used in deployment or operating scenarios are adopted in this article, which are generally hardware limited and therefore some of them are not the same for different systems, and this part forms the essential proportion of unified evaluation. As for the wireless channel, the resolutions for multi-path are different due to various bandwidths. Therefore, the common set of different models is used for evaluation. C. Channel Model Currently widely used channel models are modified from the multi-path channel model in ITU-R recommendation of M.1225, because different systems have distinct bandwidth, which leads to different multi-path resolution and prevents from using M.1225 model directly. M.1225 model defines several scenarios: Indoor A/B, Pedestrian A/B and Vehicular A/B, each with 6 sub-paths except Pedestrian A. The relative time delay and average power for each sub-path are specified. 3GPP, 3GPP2 and ITU have recommended the channel models for WCDMA, 1x EV-DO and WiMAX with SIMO (typically 1 Tx and 2 Rx) antenna configuration respectively. When MIMO technology is adopted, the SCM (Spatial Channel Model) and SCM-e (SCM-enhancement) should be used to generate multi-path fadings.
iew
ev
B. Simulation Platforms The platforms used are developed by the WT&T (Wireless Theories and Technologies) Lab at BUPT (Beijing University of Posts and Telecommunications). The simulation
Calibration of Each Platform EV-DO WiMAX HSxPA Rev.A
ly
A. Flowchart of Unified Evaluation Figure 2 shows the flowchart of our work. First we develop the simulation platform (described in Subsection B) for each system involved, and then calibrate each of them by adopting commonly used simulation configurations and comparing the outcomes with published results. Then we apply the unified settings (shown in Subsection C, D and E), which are proposed based on existing documents of different organizations, to each platform. Calibration between platforms (described in Subsection F and G) is implemented in order to meet the comparison baseline.
On
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Unified Settings
Settings used by Different Organizations
Channel Model Key Algorithms Overall Configurations
3GPP
3GPP2
ITU
etc.
Calibration between Platforms Fairness Calibration Stability Calibration
Results Figure 2. Unified Evaluation Workflow
Page 4 of 6
4 Channel model proposed by 3GPP2 is summarized in Table 2.2.2-1 and Table 2.2.2-2 in [1]; 3GPP’s channel model is in Table A-3 through Table A-6 in [2] and ITU’s model for WiMAX is in TABLE 16 and TABLE 17 in [10]. These modified models are originally designed for single system evaluation, not for the purpose of comparison between several systems. Therefore, only part of each modified model can match with each other. We carefully analyze and compare these models and propose the following mixed model as in TABLE I. Although it is a specific configuration, it simulates a unified mobile environment and can be equally applicable to many systems. It is shown that 3 types are included, i.e. Pedestrian B with speed of 3 km/h, Vehicular A with 30 km/h and 120 km/h. Different multi-path parameters are proposed for corresponding systems because of distinct bandwidths leading to diverse resolutions for multi-path.
Fo
D. Key Algorithms Some technologies are widely used by many systems, for example, power control on the reverse link (or uplink), scheduling both on forward link (or downlink) and reverse link, HARQ and AMC (Adaptive Modulation and Coding). Since each system has its own characteristics, the algorithms should be adjusted accordingly. Several methods have been standardized or widely used [1, 2, 3] while some are for further study. Compared with EV-DO Rev.A and WCDMA HSPA, standardization of Mobile WiMAX is more open than the other two systems. Here, we won’t describe the well-accepted algorithms in detail for the sake of simplicity. We would like to illustrate our schemes for scheduling algorithm and dynamic overhead model in Mobile WiMAX, because they are of great importance for the final outcomes. 1) Scheduling Algorithm in Mobile WiMAX Scheduling process can be divided into several steps: determination of packet transmission ordering over air interface, resource allocation and creation of DL/UL maps. In our simulation, one basic resource unit in frequency domain is one sub-channel. On each sub-channel, PF (proportional fair) scheduling algorithm is used. 2) Dynamic Overhead Model in Mobile WiMAX The physical overhead is a critical factor that may affect the overall performance significantly. Models used to calculate the overhead in EV-DO Rev.A and HSPA have been well established by removing the power of signaling channels from the total available power. But the impact of overhead on system performance in WiMAX has not gained enough attention. The overhead is usually set to a fixed value in the major published documents [4, 8]. Several researches were conducted considering the MAC overhead. Specific overhead size consumed on the physical layer using 802.16e WiMAX as backhaul is calculated and analyzed in [9]. But [9] assumed fixed overhead size, which means that the overhead size is predefined before the simulation and will not be changed according to different user numbers, scheduling results and
TABLE I MIXED CHANNEL MODEL Model I Multi-Path Model Speed (km/h) Assignment Probability Organization
Parameters
Pedestrian B 3 60% 3GPP Power (dB) 0 -0.9 -4.9 -8.0 -7.8 -23.9
Multi-Path Model Speed (km/h) Assignment Probability Organization
Parameters
WiMAX Forum Delay Power Delay (chip) (dB) (ns) 0 -3.92 0 1.23 -4.82 200 2.83 -8.82 800 -11.92 1200 -11.72 2300 -27.82 3700 Model III
3GPP2
Delay Power (ns) (dB) 0 -1.64 200 -7.8 800 -11.7 1200 2300 3700 Model II Vehicular A
Vehicular A
30
120
30%
10%
3GPP Power (dB) 0 -1.0 -9.0 -10.0 -15.0 -20.0
Delay (ns) 0 310 710 1090 1730 2510
3GPP2 Power (dB) -0.9 -10.3
Delay (chip) 0 1.23
WiMAX Forum Power Delay (dB) (ns) -3.14 0 -4.14 310 -12.14 710 -13.14 1090 -18.14 1730 -23.14 2510
iew
ev
rR
other factors. In addition, the downlink overhead includes the FCH (Frame Control Header), DL_MAP, UL_MAP and DL_ACK (ACKnowledgement), while the uplink overhead consists of three parts: FF (Fast Feedback), Raging and UL_ACK in [9]. But we deem this modeling of DL/UL overhead is not complete to evaluate system performances. We propose a more accurate model to dynamically calculate the overhead size during simulation so as to guarantee the validity of outcomes as follows in [10]:
ly
On
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
OH DL = P + FCH + DL _ MAP + UL _ MAP + DCD + UCD + ACK DL + PCDL OH UL = ACKUL + Ranging + CQIUL
(1) (2)
3) Link Level to System Level Mapping In order to reduce the simulation complexity, the effects of link level technologies are abstracted into several curves, which will be used in system level. This method is widely used in CDMA systems. As for OFDMA based system, such as Mobile WiMAX, the effective SINR in one sub-channel is needed. But one sub-channel has several subcarriers, which should be considered when calculating effective SINR. EESM (Exponential Effective SINR Mapping) model is a typical solution. But when the bandwidth is large, such as 10 MHz in which the FFT (Fast Fourier Transfer) size is 1024, the amount of calculation would be quite large and it will greatly reduce the
Page 5 of 6
5 simulation efficiency. Through our simulation verification and validation, it is found that if the effective SINR is calculated by using the value on every 4 sub-carrier instead of every sub-carrier, the simulation results under the channel models mentioned above will not be affected but it will significantly reduce the simulation complexity. E. Overall System Configurations As we mentioned, different system operates with various parameters; some are configurable while others are hardware limited. Each organization recommends a set of parameters for simulations, which cannot match with the others completely. Therefore, in order to run simulations on a unified baseline, we compare related documents [1, 2, and 8] and summarize a unified configuration as shown in TABLE II. The results shown later are all based on these configurations.
Fo
F. Fairness Calibration In order to provide a fair comparison between results, all of the systems are adjusted and optimized so that the following two
rR
TABLE II OVERALL SYSTEM CONFIGURATIONS Simulation Parameters Unit Unified Values System Configurations Operating Frequency GHz 2 FDD for EV-DO and HSxPA Duplex TDD for WiMAX 1.25 for EV-DO on FL or RL Bandwidth MHz 4.5 for HSDPA or HSUPA 10 for WiMAX DL and UL Log-normal Std. Deviation dB 8.9 Thermal Noise Density dBm/Hz -174 Frequency Reuse Factor 1 BS-to-BS Distance km 2 Minimum Distance m 35 between BS and MS Maximum RL Total Path dB 140 Loss (no fading) Antenna Configuration FL: 1*1 RL: 1*2 Propagation Model Propagation Model 128.1+37.6lgd d in kms Antenna Height of BS m 30 Antenna Height of MS m 1.5 BTS Configurations 43 per carrier in EV-DO BTS Tx Power dBm 43 per carrier in HSDPA 43 per antenna in WiMAX BTS Tx/Rx Antenna Gain dBi 15 with Cable Loss Other Loss dB 10 BTS Noise Figure dB 5 Antenna Horizontal 70 deg(-3 dB) with 20 dB dB Pattern front-to-back ratio Correlation between BS 0.5 MS Configurations 23 per carrier in EV-DO MS Tx Power dBm 21 per carrier in HSUPA 23 in WiMAX MS Tx/Rx Antenna Gain dBi -1 MS Noise Figure dB 9 Other Losses dB 10
criteria are fulfilled: 1) Their fairness curves (defined in section 2.1.6 in [1]) are similar, which will ensure the consistence in the comparison; 2) Their fairness curves are as close to the fairness criterion as possible, which indicates that the throughput is maximized. The fairness curves for EV-DO, HSUPA and WiMAX systems are demonstrated in Figure 3. In addition, the percentage of users in soft/softer handoff should also be adjusted. As Figure 3 shows, the percentage for EV-DO is about 30%, (the step on the curve is due to soft/softer handoff) which equals to that of HSUPA. Note that hard handover is used in Mobile WiMAX. G. System Stability Calibration On the RL/UL, system stability must be guaranteed. The percentage of time when RoT (Rise over Thermal, or IoT, Interference over Thermal, in WiMAX) is above the given bound is used to measure the stability of the RL system. The allocation of the total system resource is carried out while the reverse link load does not exceed the pre-defined maximum threshold to keep the system stable. At the same time, RoT (or IoT in WiMAX) outage criterion must be met, i.e., the percentage of time when RoT (or IoT in WiMAX) is above the upper bound shall not exceed a limit. To keep the system stable enough, the overall throughput can be tradeoff in some cases. Since this criterion will great affect the performances, it must be calibrated across different systems in order to guarantee the fairness. Figure 4 is demonstrated as an example. It is shown that when the RoT outage criterion is met in EV-DO, the average RoT is about 6.2dB (the corresponding RoT when the value on y-axis is 0.5). But in WiMAX system, when the IoT outage rate is met, the average IoT is about 5 dB. If the average IoT is adjusted to 6.2 dB, the outage criterion will be violated. Hence, it is indicated that the interference management mechanism in WiMAX can be further improved.
iew
ev
On
H. Output Matrices Different traffic has various QoS (Quality of Service) requirements. For example, the latency requirement of VoIP is much stricter than that of FTP service. Hence, the simulation output matrices depend on the characteristics of traffic. For the packet data traffic, the sector/cell throughput is widely used to evaluate system performances. But it is correlated with system bandwidth and typically increases when more bandwidths are occupied. Hence, it can only reflect the absolute outcomes and conceals the efficiency. Spectrum efficiency is defined as the throughput divided by the effective bandwidth with the unit of bits per second per Hertz. Higher efficiency means that more data can be transmitted over one time-frequency unit. Table III shows the throughputs and spectrum efficiencies of different systems with 10 Full Buffer FTP users per sector. Take the FL/DL as an example, it is obvious that WiMAX has the biggest throughput, but the spectrum efficiency seems not. More results can be found in [11].
ly
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Page 6 of 6
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6 CCDF
CDF
6
0.5 0.4 0.3
0.5
0.3 Fairness Criterion WiMAX EV-DO HSUPA
0.2 0.1 0
WiMAX UL: Average IoT=6.2dB WiMAX UL: P(RoT>7dB)7dB)
