Radio Base Stations in the Cloud - Wiley Online Library

◆

Radio Base Stations in the Cloud

Bernd Haberland, Fariborz Derakhshan, Heidrun Grob-Lipski, Ralf Klotsche, Werner Rehm, Peter Schefczik, and Michael Soellner During the past few years, architectures for cellular radio networks like 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE) have evolved towards flatter topologies with fewer centralized nodes, shifting more and more radio-related processing towards the base station or even further into so-called remote radio heads near the antennas. With Internet Protocol (IP) traffic growth, user traffic lost its predictability, which resulted in the need for the over-dimensioning of processing and forwarding resources in the radio path. This is especially true in upcoming picocell and femtocell configurations in heterogeneous cellular architectures. To hold down costs resulting from these emerging architectures and deployments, we developed a concept of a cloud base station. Developed as a candidate for a lightRadioTM evolution, this cloud base station leverages virtualization techniques such as dynamic and elastic allocation of baseband processing resources between small and large cell sites, as well as load balancing between temporarily overloaded sites and their less utilized neighbors. A cloud base station in a virtualized radio access network (RAN) can serve to ease a smooth standards migration towards LTE-Advanced with the same deployed equipment, and in the future, support the transformation of dedicated baseband hardware into general purpose processing platforms. In this paper, we discuss the tight networking requirements of a cloud base station in a virtual radio access network, architecture challenges, and our cloud management solutions. We also demonstrate research results on the concept, provide validation based on simulations, and discuss business case related implications. © 2013 Alcatel-Lucent.

Introduction Over the years, the radio system infrastructure for second generation (2G) GSM/EDGE/CDMA2000, third generation (3G) UMTS/HSPA/3G1X/EV-DO, and fourth generation (4G) LTE/LTE-Advanced wireless platforms

has been specified and deployed based on evolving architectural and technological ideas. Current advances in technologies such as software-defined radio, cloud computing, and virtualization enable us to create flexible systems based on common radio platforms that can

Bell Labs Technical Journal 18(1), 129–152 (2013) © 2013 Alcatel-Lucent. Published by Wiley Periodicals, Inc. Published online in Wiley Online Library (wileyonlinelibrary.com) • DOI: 10.1002/bltj.21596

Panel 1. Abbreviations, Acronyms, and Terms 2G—Second generation 3G—Third generation 3GPP—3rd Generation Partnership Project 4G—Fourth generation ARQ—Automatic repeat request BBU—Baseband unit BSC—Base station controller CAPEX—Capital expenditure CDF—Cumulative distribution function CDMA—Code Division Multiple Access CoMP—Cooperative multi-point processing CPI—Cyclic prefix insertion CPR—Cyclic prefix removal CPRI—Common Public Radio Interface C-RAN—Centralized RAN DCC—Decentralized cloud controller DCH—Dedicated channel DEMUX—Demultiplexer DL—Downlink DSCH—Downlink shared channel E-DCH—Enhanced DCH EDGE—Enhanced Data Rates for GSM Evolution eNB—Evolved NodeB (LTE base station) ETH—Ethernet EV-DO—Evolution Data Optimized FACH—Forward access channel FEC—Forward error correction FFT—Fast Fourier transform FP—Frame Protocol FTP—File Transfer Protocol GPRS—General Packet Radio Service GTP—GPRS Tunneling Protocol GSM—Global System for Mobile Communications HS-DSCH—High speed DSCH HSPA—High Speed Packet Access HW—Hardware IFFT—Inverse fast Fourier transform Inter-RM—Inter MSS-BBU resource manager Intra-RM—Intra MSS-BBU resource manager IP—Internet Protocol be used for existing standards as well as for recent evolutionary steps, such as LTE-Advanced, [6]. In this paper, we focus on the access system between the air interface and the gateway to the respective mobile core network. We will not discuss mobile terminal issues. We are presenting the first results of a feasibility study analyzing a novel cloud base station concept,

130

Bell Labs Technical Journal

DOI: 10.1002/bltj

IQ—In-phase and quadrature LRM—Local resource manager LTE—Long Term Evolution MAC—Media Access Control MCS—Modulation and coding scheme MIMO—Multiple input multiple output MME—Mobility management entity MSS-BBU—Multi-site/multi-standard BBU MUX—Multiplexer MVNO—Multiple Virtual Network Operator NGMN—Next Generation Mobile Networks OPEX—Operating expenditure PA—Power amplifier PCH—Paging channel PDCP—Packet Data Convergence Protocol PHY—Physical layer PRB—Physical resource block QoS—Quality of service RACH—Random access channel RAN—Radio access network RBC—Radio bearer control RLC—Radio Link Control RM—Resource manager RNC—Radio network controller RRC—Radio Resource Control RRH—Remote radio head Rx—Receiver S1AP—S1 Application Protocol SCTP—Stream Control Transmission Protocol SGW—Signaling gateway SINR—Signal-to-interference-plus-noise ratio SW—Software TEID—Tunnel endpoint identifier TTI—Transmission time interval Tx—Transmitter UDP—User Datagram Protocol UL—Uplink UMTS—Universal Mobile Telecommunications System UP—User processing

where we put digital processing resources into a processing cluster. This concept is applicable in future wireless networks and also in Alcatel-Lucent’s lightRadioTM approach [8]. Our paper is structured as follows: We begin with a brief review of the requirements, challenges, and opportunities for a future multi-standard evolution

of a cloud radio base station. Then we dig deeper into aspects of virtualization and cloud principles for the radio access network (RAN). After that we discuss the overall architecture, processing split, cloud control architecture, and resource pooling management. Next, we discuss issues for backhaul and fronthaul. We analyze opportunities for a case study to evaluate statistical multiplexing gains and processor pooling gains. Finally, we offer conclusions and provide a vision for the future.

Evolution Challenges and Concepts of a Future Mobile RAN Mobile networks are being challenged by exponential increases in traffic load. Innovative solutions above and beyond traditional performance upgrades must be sought both because of the overall user acceptance of smartphones and because of continuous increases in smartphone traffic. Both new and improved terminal applications are contributing to the exploding demand. The number of terminals connected via mobile networks is expected to grow to several billion worldwide by 2020. If machine-to-machine devices are included as well, terminal quantities may exceed the world’s population and have been forecasted to grow from the current 5.8 billion to 20 billion in 2020 [18] or even 50 billion [13]. This growth puts pressure on network resource management. Load expectations over the coming years have been widely published, and [10] predicts a 78 percent growth rate in annual network traffic from 2011 to 2016. Obviously capacity but also service quality parameters are of great concern for upcoming installations and they need to be ensured by network technology upgrades. User traffic will become more and more non-uniform with higher user data rates in 3rd Generation Partnership Project (3GPP) networks [1] and with machine-to-machine applications in the future that will also impact traffic characteristics and processing needs. The expected traffic explosion will place intense pressure on operators’ future investments, network technologies, and operations. Coping with this demand while fully maintaining and even improving

economic constraints is one of the primary objectives of the concept presented in this paper. We cover system architectures and installation concepts, as well as prospects for reducing energy consumption, since all of these dimensions can substantially impact capital (CAPEX) and operational (OPEX) expenditures. In current mobile access network deployments, antenna, power amplifier, and signal processing are linked 1:1, including a fixed allocation of the signal processing function to the hardware. Usually, these elements are located in close proximity. However, a number of concepts to concentrate the antenna and power amplifier in remote radio heads (RRHs) and separate them from the baseband processing have been circulating for some years [12]. With optical fibers as the transmission medium between them, distances up to some ten kilometers [21] are achievable. Limits today are mainly due to the timing and synchronization constraints inherent in current radio standards. Co-locating base stations serving multiple RRHs already provides gains with respect to infrastructurerelated cost elements, such as civil works, room rental, and maintenance cost. This concept is known as “base station hoteling,” and its advantages are based on shared infrastructure and supply systems (including buildings, air conditioning, and power supply). However, in order to increase the benefit of this hoteling concept, further architectural changes are required such as pooling baseband processing resources and managing them in the cloud. The work on cloud radio networks is a topic of intense study today. Recent proposals from the Next Generation Mobile Networks (NGMN) alliance [17] target an evolution of the radio access network (RAN) towards a centralized radio access network (C-RAN). The vision behind moving the base station portion of the C-RAN architecture into the cloud, is to introduce a centralized baseband processing pool using real-time cloud computing resources together with separated radio units. A recent description of related concepts and initial approaches, along with an overview of NGMN’s C-RAN activity, is presented in [9]. The NGMN paper discusses an approach that virtualizes the entire baseband processing of one cell

DOI: 10.1002/bltj


131

in a common virtualization instance. In contrast, the approach presented in our paper overcomes this constraint and opens the path to a flexible pooling of resources on a user (bearer) level rather than at the sector or cell site level.

Technical and Customer Requirements Typical cellular deployments in urban and suburban environments consist of cells of various sizes (macrocells, metrocells, picocells) with heterogeneous traffic characteristics in use between residential, business, and other hotspot areas. Field observations show that such cells are sufficiently decorrelated with each other with regard to their traffic load over time. This means that in a cluster of cells, different cells do not necessarily have peak busy hours at the same time. This is particularly valid if a cell cluster includes different traffic scenarios in outdoor and indoor environments. As a consequence, statistical multiplexing gains can be expected within such a cell cluster, and this can translate to effective pooling gains for processing resources. Therefore, the pooling of relatively expensive baseband processing equipment inside the cloud promises lower operating costs. With sophisticated load sharing for installed processing resources accounting for different load profiles over time, we can achieve a significant reduction in the number of sites and hardware required. This offers further potential for OPEX and CAPEX reductions for the operators. In addition to balancing the processing loads, it should also be possible to allocate processing resources from low traffic areas to high traffic areas in a general context of load management. A configuration with a baseband processing pool connected to a cluster of radio heads can also support forthcoming LTE-Advanced features [2, 5] such as network multiple input multiple output (MIMO) (also known as cooperative multi-point processing (CoMP)), due to the central location of the packet schedulers of the cells and available data branching and combining points. Instead of exploiting hardware reduction gains, the processing resource pool can also master the increased demand for processing

132


DOI: 10.1002/bltj

required by future complex features without a need for additional infrastructure deployment. Another positive effect of resource pooling is that a multi-standard configuration can be effectively adapted to the varying traffic demands of users migrating over time from one standard (e.g., 3G) to another (e.g., 4G), even if single processing elements are software-defined but semi-statically configured, and are only capable of operating one radio standard at a given time. This provides the operator with more flexibility in evolving to a multi-standard system without a need to change any baseband hardware.

Virtualization Approach Virtualization is characterized by the ability either to aggregate multiple physical resources in order to achieve required levels of performance, or to share resources in order to increase utilization and efficiency in cases of lower traffic demand. In any case, the external interfaces to the surrounding non-virtualized parts of the system (independent of traffic) shall not be affected and mapped to their internal resource equivalents. Virtualization also provides the isolation between the virtualized parts to limit the impact of load dependencies, system faults, and other vulnerabilities. Our architectural concept is based on the virtualization of the cell concept, where the different user processing components are distributed over several hardware (HW) platforms. Cloud computing typically comes with virtual resource pooling and rapid elasticity to quickly scale out and scale in [16], i.e., to adapt the allocated resources to the demand by combining or sharing physical resources. The capabilities available for provisioning a cloud application often appear to be unlimited and usable in any quantity at any time. Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported and this provides

transparency for both the provider and the consumer of the utilized service. In the case of the “cloud base station,” the radio access infrastructure is provided “as a service” for the mobile operator in form of a private cloud. In principle, this concept is also applicable to a multi-tenant system that shares its resources amongst several operators (“operator sharing”) and provides strict isolation for processing and networking resources. This is in contrast to the existing multiple virtual network operator (MVNO) concept, where the MVNO uses the access infrastructure of its host network to resell spare (radio bandwidth) capacity and diverts at the mobile services level (e.g., billing).

Architecture Solutions and Challenges In this section, we will introduce the multi-site/ multi-standard baseband unit (MSS-BBU) as one of the central elements for a future cloud-based RAN implementing a “radio base station in the cloud” with flexible 2G, 3G, and 4G capabilities. Future RAN Architecture As shown in Figure 1, a multi-standard system architecture includes multiple remote radio heads (RRHs) that provide the transmitter (Tx) and receiver (Rx) antenna functions and power amplifiers (PAs) at the cell and sector sites. These RRHs are connected by high-speed optical links to the associated but locally separated baseband processing pool (multi-site/ standard baseband units: MSS-BBU). This pool can be optimally positioned within a cluster of several RRHs serving macrocells, picocells, and femtocells with different output power levels in indoor and outdoor environments. Different MSS-BBUs in different locations are interconnected with each other (e.g., in a ring) by high-speed optical links either as an enhancement of the existing X2-interface [4], called “eX2,” or as a proprietary interface. One set of MSS-BBUs and a cluster of RRHs represent a new multi-standard cloud base station, connected to an LTE signaling gateway (SGW)/mobility management entity (MME) via an S1 interface, to a

UMTS radio network controller (RNC) via an Iub interface, and to a GSM base station controller (BSC) via an Abis interface. The number of RRHs clustered and served by an MSS-BBU should be large enough to ensure enough potential pooling gain taking benefit of statistical multiplexing gains. However, the coverage area of such an RRH cluster is limited by the maximum radio interface response times of LTE and UMTS, which can be translated into latency constraints in the overall transport system between an MSS-BBU and the RRH cluster and between the MSS-BBUs. For LTE this leaves around 100 μs to 150 μs available for transmission between remote processing entities. This constraint is derived from the hybrid automatic repeat request (ARQ) latency requirements over the air interface, the processing within the user equipment (UE), and the eNodeB. As an additional functional element, the decentralized cloud controller (DCC) is available once in every MSS-BBU to act as a pool manager to realize the load balancing within a cloud base station and interact with DCCs from adjacent MSS-BBUs. The DCC serves as the resource manager (RM) at the intra- and inter-MSS-BBU level, and controls the functions for redirecting (by address dispatching) the relocated data streams via the involved internal routers and switches and the involved schedulers. This also enables load balancing between cloud base stations and provides processing capacity to follow traffic hotspots or hot zones. Therewith, load profiles between different areas (e.g., indoor/outdoor, business/ residential) can be leveled off resulting in reduced physical resource demands. By accessing all relevant internal processing resources, the DCC can manage the intra-MSS-BBU level entirely on its own. The inter-MSS-BBU level is realized by cooperation between DCCs and the DCC internal functions. This concept potentially replaces the fixed 1:1 association of BBUs and RRHs at the radio site in the conventional solution today, where the BBUs have to be dimensioned according to peak traffic load in a single cell.

DOI: 10.1002/bltj


133

• Remote radio head (RRH) cluster: macro, HetNet, indoor or any mix • RRH to MSS-BBU connectivity: 50-100

Multi-standard cloud base station

RRH

RRH

RRH

RRH

RRH

RRH

…

RRH

…

RRH RRH

RRH

RRH

RRH MSSBBU

MSSBBU

DCC

eX2 MSSBBU

MSSBBU

DCC

DCC MSSBBU

…

DCC

S1

DCC

LTE (via MME/SGW)

Iub Abis GSM (via BSC)

BBU—Baseband unit BSC—Base station controller DCC—Decentralized cloud controller eX2—Enhanced X2 GSM—Global System for Mobile Communications HetNet—Heterogeneous network MME—Mobility management entity

UMTS (via RNC)

MSS-BBU—Multi-site/multi-standard BBU RAN—Radio access network RNC—Radio network controller RRH—Remote radio head SGW—Signaling gateway UMTS—Universal Mobile Telecommunications System

Figure 1. Future RAN architecture.

MSS-BBU Functional Architecture We now describe the functional architecture of a single MSS-BBU element. The architecture is pictured in Figure 2. The baseband processing part of a base station can be subdivided into a radio control part, a user processing part (UP), and a cell processing part (PHYcell). The control and user functions for GSM, UMTS, and LTE are connected to the respective standard interfaces Abis, Iub, and S1. For the LTE and UMTS radio interface, the dedicated user processing (UP) required per user or per user radio bearer in uplink and downlink (UL/DL) is the object that will be virtualized and possibly relocated. It covers layer3, layer2, and part of the physical (PHY) layer functionalities. However, the GSM portion of

134


DOI: 10.1002/bltj

the MSS-BBU is not considered to be virtualized at all, since the GSM baseband functions are already integrated in the typical RRHs and additional flexibility is not expected in the future. In detail, the LTE virtual UP functional entity includes [2]: • S1-MME termination stack. Ethernet (ETH), Internet Protocol (IP), Stream Control Transport Protocol (SCTP), S1APded. (dedicated control plane). • S1-U termination stack. ETH, IP, User Datagram Protocol (UDP), GPRS Tunnel Protocol (GTP)-U (user plane). • Uu control plane stack. Radio Resource Control (RRC), Packet Data Convergence Protocol (PDCP), Radio Link Control (RLC), Media Access

S1MME

eNodeB control functions

Scheduler

UP 1

RRH1 RRH2 RRH3

UP22 UP

MUX/* DEMUX

S1-U

RRH4 RRH5

UP 3 PHYcell UP 4 Iub

NodeB control functions

Spreader/ despreader

UP 5

RRHm

PHYcell UP i-1 Abis

BTS control functions

UP i UP j

UMTS functions

eX2 To other MSSBBUs

DCC

MSS-BBU LTE functions

MAC(e)hs/e scheduler

UP j-1

Cell signals

GSM functions

*Framer/deframer/IFFT/FFT are assumed to be in the RRH BBU—Baseband unit BTS—Base transceiver station DCC—Decentralized cloud controller DEMUX—Demultiplexing eX2—Enhanced X2 FFT—Fast Fourier transform RRH—Remote radio head RNC—Radio network controller SGW—Signaling gateway

GSM—Global System for Mobile Communications IFFT—Inverse fast Fourier transform LTE—Long Term Evolution MAC—Medium access control MSS-BBU—Multi-site/multi-standard BBU MUX—Multiplexer UMTS—Universal Mobile Telecommunications System UP—User processing stack

Figure 2. MSS-BBU functional architecture.

•

•

•

Control (MAC), PHYuser (dedicated radio control plane). Uu user plane stack. PDCP, RLC, MAC, PHYuser (radio user plane). and for UMTS [7]: Iub termination user plane stack for HSPA. ETH, IP, UDP, Frame Protocol (FP), high speed downlink shared channel (HS-DSCH), enhanced dedicated channel (e-DCH) (user plane). Uu termination stack for HSPA. MAC (HSPA related), PHYuser.

The PHYuser functions for UMTS and LTE are mainly the functions of forward error correction (FEC), modulation/demodulation, channel estimation, and multi-antenna processing. As shown in Figure 2, the overall number of LTE user instances (UPi) and UMTS user instances (UPj) can be different depending on the bandwidth allocated to the respective systems. The processes for all other common control channels in UMTS and LTE such as the DCH, random access channel (RACH), forward access channel (FACH), and paging channel

DOI: 10.1002/bltj


135

(PCH) will not to be virtualized since little gain is expected for load balancing. Besides the PHYuser functions, there is a PHYcell functional entity for LTE and UMTS in the MSS-BBU, which includes: • LTE. Dynamic multiplexing/demultiplexing (MUX/ DEMUX) of PHYuser signals for one cell, where the bandwidth depends on the cell load. • UMTS. Spreader/despreader, where functions are available according to the number of UMTS cells times the number of antennas per cell. Other LTE PHYcell functions like framer/inverse fast Fourier transform (IFFT), fast Fourier transform (FFT), cyclic prefix insertion (CPI), and cyclic prefix removal (CPR) are integrated in the RRH. From the 3GPP specifications, e.g., [2], the interface between PHYuser and PHYcell as described above can be identified in the protocol stack. When splitting the functionality between the BBU and RRH at this internal interface, the resulting bandwidth on the connecting Common Public Radio Interface (CPRI*), which is used as protocol wrapper, can be derived. This bandwidth is load dependent and reduced by a factor of 5.5 (at 100 percent cell load) up to a factor of 12 (for 30 percent cell load) compared with full uncompressed in-phase and quadrature (IQ) transmission via CPRI, thus driving substantial cost savings for the transport system. This function is available according to the number of LTE cells times the number of antennas per cell. The multiplexing of user to cell functions in each standard is managed by a packet scheduler. There is one packet scheduler per cell for LTE and HSPA. Any UP of one standard can be connected to any PHYcell of the same standard. This is directly controlled by the responsible cell scheduler. The DCC handles overall management for allocating the virtual processing entity UP to a PHYcell. The user plane parts of the UPs are connected to the S1-U or Iub interface. The dedicated control plane parts of the UPs are connected via the enhanced NodeB (eNB) and NodeB control functions to the S1-MME and Iub interface. RRH1 to RRHm denote the external interfaces of the MSS-BBU to the RRH sets connected to the common PHYcell functions.

136


DOI: 10.1002/bltj

MSS-BBU HW Architecture Figure 3 shows the HW architecture of the MSSBBU. The BBU boards accommodate a variable number of UP processes (as explained in a previous section) depending on bandwidth per user up to a maximum bandwidth and the processing of cell functions for one RRH set per radio site (PHYcell, cell control functions, and radio resource management functions). The PHYcell signals as described can either be inserted in the CPRI protocol or in a proprietary interface to the RRHs. The BBU board can act as either an LTE or UMTS HW module and can be configured to each radio standard by software replacement and without any hardware modifications. In addition, the BBU hosts the packet scheduler for LTE or HSPA, which is needed on a per cell basis. BBU boards may be available i-times for UMTS and k-times for LTE. The ratio between i and k can be modified. This means the multi-standard mix between UMTS and LTE within the cloud base station can be adapted according to need by SW replacement. This is not possible with conventional solutions today. The MUXarray HW module accommodates additional PHYcell functions and connectivity to RRHs as described above for LTE and for UMTS. This MUXarray module is only needed to serve the RRHs that are actually connected if the load balancing gains as a result of the cloud configuration are used to reduce the number of BBUs. In this case, full connectivity is still provided to all existing RRHs in the cluster. The related UPs and cell control functions are then processed in the remaining BBUs. The BBUs and the MUXarray HW boards within one MSS-BBU are interconnected via an appropriate high speed technology (such as Gigabit Ethernet or Serial RapidIO* (SRIO) [19]. Either an enhanced X2 (eX2) interface or a proprietary interface may be used to interconnect BBUs or MUXarrays with BBUs from other MSS-BBUs. We considered using a high speed Ethernet connection or another medium range technology such as InfiniBand* [15] for this interface. In the other case, when the number of BBUs is not reduced below the number of connected RRHs, the MUXarray module is not needed and the pooling

RRH1

RRH2

RRH3

RRH m

O/E

O/E

O/E

O/E

Switching fabric BBU i MUXarray

BBU 1

BBU 3

BBU 2

BBU k

Cell signal (from other MSSBBU) eX2 IF

User level

eX2 to other MSS-BBUs

Ethernet switch

Router/address dispatcher

DCC

LTE+GSM UMTS + GSM

Abis

Iub

BBU—Baseband unit DCC—Decentralized cloud controller eX2—Enhanced X2 GSM—Global System for Mobile Communications HW—Hardware LTE—Long Term Evolution

S1-U

S1-MME

MME—Mobile management entity MSS-BBU—Multi-site/multi-standard BBU MUX—Multiplexer O/E—Optical/electrical RRH—Remote radio head UMTS—Universal Mobile Telecommunications System

Figure 3. MSS-BBU HW architecture.

gains are used to create processing capacity headroom to implement future enhancements. The DCC is responsible for BBU pool resource management, which includes load detection and the load balancing decisions on an intra- and interMSS-BBU level, and intercommunication with the BBUs, the router, and the switch fabric. This is described in more detail in the section on “Control Plane Architecture” below. The router/address dispatcher and the Ethernet switch are needed to route the S1-U, S1-MME (optionally), and Iub packets in a direct way to the allocated BBU decided by the DCC. This minimizes the transfer of signals between BBUs and is described in more detail in the section on “Address Dispatcher.” The Abis packets do not need to be

re-routed, since we do not foresee a cloud solution for GSM. The switch fabric is an optional entity, which might be needed in two cases: • Recovery for a failed BBU in order not to lose a connected RRH, and • Insertion of complete cell signals coming from remote MSS-BBUs via eX2. For the recovery case, the MUXarray module has an n + 1 redundancy of its PHYcell functions, which can be used for this purpose. In this case, the switch fabric performs an electrical switching and reallocation of PHYcell signals which now originate from one MUXarray instead of a failed BBU, or from a BBU in a remote MSS-BBU via eX2.

DOI: 10.1002/bltj


137

Address Dispatcher Distributing the resources of a base station, or more precisely the baseband unit, as a set of components in the cloud introduces more flexibility but it requires its own control plane. Furthermore, this flexibility interferes with the LTE network architecture, especially if it is used dynamically on individual active data flows. The LTE view of the radio access network sees all components as singular physical entities, and therefore reaches a base station directly via a GPRS Tunnel Protocol (GTP) tunnel. In our approach, an MSS-BBU is a pool of base station resources and thus the boundary for “a base” station blurs. However, data and control traffic need a precise routing to the resources allocated for their processing. The solution presented below is transparent to the standards and uses information easily accessible in the protocols of the LTE S1 interface. User data between the base station (eNB) and the SGW is protected by encryption of the GTP tunnel. The gateway directly addresses the base station the user is booked in, on the IP layer. Since all tunnels reaching one base station use the same IP address pair, we cannot use these addresses to redirect individual flows to distributed MSS-BBU resources. The router/address dispatcher entity is at the edge of our cloud and performs a specialized kind of routing: Within the GTP header the tunnel endpoint identifier (TEID) can be mapped 1:1 to a mobile user. The allocation of a particular mobile user inside the MSSBBU is controlled by the DCC. Deep packet inspection is used to reroute this information to reach the UP instance, thereby offering a processing pipeline for the single user identified by the TEID. The user plane for HSPA and the dedicated control plane for LTE are items for further study. Virtualization Concept Figure 4 shows our multi-standard virtualization concept in more detail. The RNC, the MME, and the SGW are connected via a router/address dispatcher. The routing of the LTE and UMTS packets to the BBUs for processing is transparent to the RNC, MME, and SGW. Details of this solution are covered in the previous section.

138


DOI: 10.1002/bltj

As an example, BBU1 and BBU2 are assumed to be in the same MSS-BBU of one cloud base station, while BBU3 is part of another MSS-BBU in another cloud base station. BBU1 is assumed to provide all the cell control functions and the PHYcell for a particular cell of interest. The DCC resource manager decides which UPs for one cell are processed by which BBU within one MSS-BBU (in this example BBU1 and BBU2) and communicates with the DCC of another MSS-BBU in case the cell needs additional processing for UPs from or external BBU (in this example the BBU3). All UPs for one cell (in this example from BBU1, BBU2, and BBU3) are then connected in a duplex direction to the PHYcell of BBU1. Some useful definitions for a particular user in this context: • For users 1 . . . x (UP1 . . . UPx), BBU1 is the home and the serving BBU. • For users x + 1 . . . n (UPx + 1 . . . UPn), BBU1 is the home BBU and the BBU2 the remote BBU on an intra-MSS-BBU level. • For users n + 1 . . . n + y (UPn + 1 . . . UPn + y) BBU1 is the home BBU and BBU3 the remote BBU on an inter-MSS-BBU level. The PHYcell function of BBU1 is then connected to the RRH, which represents the radio cell of interest. Cloud Control Architecture In this section we describe an approach for the control architecture that supports the entire resource management system of the mobile cloud. The architectural components have been designed to facilitate the assignment of resources which represent the intraMSS-BBU and the inter-MSS-BBU pool on two hierarchical levels. An intra-MSS-BBU resource pool includes the processing resources for a multitude of BBU modules. They are interconnected with each other via an internal high-speed intra-MSS-BBU interface. Each BBU is accessible via an IP and MAC address. The inter-MSS-BBU resource pool comprises the processing resources of several neighbor MSS-BBUs. Figure 5 depicts the MSS-BBU control architecture. It shows a home MSS-BBU and several remote MSS-BBUs, because MSS-BBUs adopt different roles according to their current logical function on the

MME/SGW S1-MME

RNC Iub

S1-U

DCC

Router

Cell control functions

Inter-MSSBBU via eX2 IF

BBU2

BBU1

UPx+1… UPn

MSS-BBU internal connection

UP1…UPx

DCC

Scheduler

BBU3

PHY cell

Inter-MSS-BBU via eX2 IF

MSS-BBU1

UPn+1… UPn+y

MSS-BBU2

RRH BBU—Baseband unit DCC—Decentralized cloud controller DEMUX—Demultiplexing eX2—Enhanced X2

MME—Mobility management entity MSS-BBU—Multi-site/multi-standard BBU RNC—Radio network controller

RRH—Remote radio head SGW—Signaling gateway UP—User processing stack

Figure 4. Multi-standard virtualization concept.

user level. The home MSS-BBU is a logical function per user and includes the BBU with its control and cell functions connected to the RRH carrying the cell where the user is located. The remote MSS-BBU is a logical function per user and includes the BBU with the remote user functions (UP) put into the cloud. Independent of its current role, each MSS-BBU contains a DCC, which again possesses an intra- and an inter-MSS-BBU resource manager (intra-RM/ inter-RM). The intra-RM is responsible for allocating resources to the BBU modules in the intra-MSS-BBU pool. It is responsible for the assignment of resource requests and it selects appropriate BBUs. The intraRM has an interface to each BBU’s local resource manager (LRM), the entity which manages the detailed processing allocation of a UP entity within each BBU and reports on available processing resources (e.g., available processing bandwidth).

This enables the intra-RM to decide whether to allocate processing resources within the home MSSBBU or to a remote MSS-BBU depending on the load situation, also taking into account the available link bandwidth. The control architecture includes a radio bearer control (RBC) function on each BBU which has a direct interface to the intra-RM. The RBC plays an important role at the home MSS-BBU during the radio bearer setup procedure since it sends respective requests for the user’s quality of service (QoS) requirements. Even though the RBC exists in each BBU of each MSS-BBU, it is not shown for the remote MSS-BBU because it doesn’t play an active role there. The inter-MSS-BBU resource manager (interRM) manages the resources of the inter-MSS-BBU resource pool. Figure 5 also depicts the router function (including the address dispatcher) connected

DOI: 10.1002/bltj


139

Intra-RM

Inter-RM

Intra-RM

Inter-RM

DCC

DCC

RBC RBC LRM .....

BBUn

…… RBC BBU2

LRM

R O U T E R

eX2

LRM .....

R O U T E R

BBUn

…… RBC LRM

BBU2

RBC

RBC

LRM

BBU1

LRM

BBU1

Remote MSS-BBU

Home MSS-BBU

S1

S1

MSS-BBU internal control interface BBU—Baseband unit DCC—Decentralized cloud controller eX2—Enhanced X2 Inter-RM—Inter-MSS-BBU resource manager

Intra-RM—Intra-MSS-BBU resource manager LRM—Local resource manager MSS-BBU—Multi-site/multi-standard BBU RBC—Radio bearer control

Figure 5. Control architecture of MSS-BBU.

to the intra-RM on each MSS-BBU. This is described in the section on “MSS-BBU HW Architecture.”

Processing Pool Management In this section we describe the mechanisms necessary for intra- and inter-MSS-BBU pooling management. These mechanisms, which encompass information retrieval and decision and connection management including the necessary signaling exchange, are designed for the Radio Bearer set-up phase. Dynamic processing reallocations, which are necessary due to the significant variance of processing needs, are a subject of further study. Intra-RM and Inter-RM Resource Information Retrieval The LRM inside each BBU continuously monitors the BBU’s resource status and reports it to the intra-RM if it detects significant changes. The frequency of these reports needs to be traded off against

140


DOI: 10.1002/bltj

the accuracy of the available status information. The intra-RM determines the resource status of the BBU with the largest available processing resource. It then builds an envelope of the maximum value on the basis of past experiences (variances of the maximum value) and writes it into the memory it shares with the inter-RM. The inter-RM retrieves the resource state of its own MSS-BBU from this shared memory when a resource demand arrives from the inter-RM of another MSS-BBU, and it exchanges it on demand with its peer inter-RMs on remote MSS-BBUs. Intra-MSS-BBU Processing Resource Allocation If inside a BBU a radio bearer request (with a capacity description derived from the quality of service parameters) arrives at the intra-RM from the RBC, the intra-RM decides on the BBU allocation within the BBU pool (Figure 6) by a filling

DOI: 10.1002/bltj


141

Home MSS-BBU

ACK

Figure 6. Intra-MSS-BBU processing resource allocation.

……

Inter-RM

Intra-RM—Intra-MSS-BBU resource manager LRM—Local resource manager MSS-BBU—Multi-site/multi-standard BBU

Routing information of remote/home BBU

BBU—Baseband unit eX2—Enhanced X2 Inter-RM—Inter-MSS-BBU resource manager

Radio bearer request ACK

ACK

BBU allocation

Decision on BBU

Resource pooling management case: processing resource available at home MSS-BBU Router at Router Shared LRM per Radio S1/eX2 Intra-RM Inter-RM at S1/eX2 memory BBU bearer control per BBU Resource status Resource state . . . . . Radio bearer request . (resource description) LRM per BBU per MSS-BBU

BBU Allocation

. . .

Remote MSS-BBUs

. . .

Resource status

Intra-RM

Resource state

Shared memory

algorithm, minimizing the inter-BBU link bandwidth when allocating the UP to a remote BBU. After an ACK by the LRM, the intra-RM sends the IP address of the allocated BBU to the router. After an ACK from the router, the intra-RM sends an ACK to the RBC and the new radio bearer can be set up. Inter-MSS-BBU Processing Resource Allocation If inside a BBU (home BBU) a radio bearer request with a capacity description derived from the quality of service parameters arrives at the LRM from the RBC (radio bearer control), and if the available resources of the intra-MSS-BBU pool are insufficient to meet the resource request, the intra-RM sends the resource demand with the capacity description to its inter-RM for a remote resource allocation (Figure 7). In more detail, the intra-RM continually scans the BBUs of the very same MSS-BBU and collects load status data on all relevant processors, including the current data link loads to those BBUs. With this information, it compiles a sorted list of BBU candidates beginning with the least loaded one, provided that the respective link capacity is sufficient. On receipt of a radio bearer request, the assignment process in the intra-RM selects the BBU where the request will be executed, i.e., the BBU on which the UP will be located. It first checks the load status of the BBU, where the PHYcell processing is located. If the resources required for the request exceed the remaining processing capacity of this BBU, the intraRM checks the first BBU in its sorted list to balance the BBU loads. If even this BBU cannot afford to process the request, the intra-RM triggers the inter-RM to administer the remote assignment of the request to another MSS-BBU by sending the resource demand with the capacity description to its inter-RM. The inter-RM of the home MSS-BBU now triggers an actual delay measurement to its neighbor list on a per user basis. This is a result of the delay initialization procedure, where delays from the home MSS-BBU to its RRH cluster as well as the delay between the home MSS-BBU and the potential remote MSS-BBU are measured. When the delay from the home MSS-BBU to a RRH which carries the cell of a particular user is known, the allowed delay

142


DOI: 10.1002/bltj

to a remote MSS-BBU can be determined, and a related-user neighbor list can be created. The actual delay measurement is needed to measure the dynamic part of the delay caused by queuing effects due to bandwidth variations at the eX2 interface. After a response from potential remote MSS-BBUs, the inter-RM updates the neighbor list for a particular user after a new comparison with the allowed delay at eX2. Now the inter-RM forwards a resource demand message from its intra-RM to all the inter-RMs of the potential remote MSS-BBUs according to the updated neighbor list. Each inter-RM which receives such a resource demand reads the resource state from the shared memory. If the BBU with the highest free capacity meets the requested capacity, the inter-RM of the potential remote MSS-BBU answers with an ACK (including the resource status). Otherwise, the inter-RM responds with a NACK. After this, the inter-RM of the home MSS-BBU determines the target MSS-BBU for remote processing allocation. It either selects the MSS-BBU with the first ACK response, or it chooses the MSS-BBU with the largest capacity margin indicated in its resource state response. After this decision, the inter-RM of the home MSS-BBU sends a remote allocation of MSS-BBU message including the IP/Ethernet MAC address of the home BBU and a user ID to the inter-RM of the selected remote MSS-BBU. The inter-RM of the remote MSS-BBU now sends a resource request message to its intra-RM, which includes the capacity description received with the resource demand message, the address, and the user ID. The intra-RM now decides on a BBU to process the job using a procedure similar to that described in the previous section (but without first selecting the BBU hosting the PHYcell function). The intra-RM sends a BBU allocation message to the LRM of the selected BBU. After receiving an ACK from this LRM, the intraRM sends routing information for the home and remote BBU to the router. The additional handshake to exchange the necessary routing information between the home MSS-BBU and the remote MSS-BBU can be seen in the message sequence chart in Figure 7.

DOI: 10.1002/bltj


143

LRM per BBU Inter-RM

Figure 7. Inter-MSS-BBU processing resource allocation.

……

ACK

MAC—Medium access control MSS-BBU—Multi-site/multi-standard BBU

Remote MSS -BBUs

(IP/Ethernet MAC address of remote BBU)

ACK

ACK

BBU allocation

Decision on remote BBU

(Ethernet MAC address home/remote BBU) ACK (IP/Ethernet MAC address of remote BBU)

. . .

Resource demand (capacity description and Ethernet MAC address home BBU)

. . .

LRM per BBU per MSS-BBU

Resource status

Intra-RM

Resource state

Shared memory

Read resource state

Inter-RM

Routing information home/remote BBU

Remote allocation of MSS-BBU (Ethernet MAC address home BBU)

(resource state)

ACK

Resource demand (capacity description)

Actual delay response

Intra-RM—Intra-MSS-BBU resource manager IP—Internet Protocol LRM—Local resource manager

ACK Home MSS-BBU

Router at S1/eX2

Actual delay measurement (access neighbor list)

Router at S1/eX2

Routing info home/remote BBU (Ethernet MAC address home/remote BBU)

ACK (IP/Ethernet MAC address of remote BBU)

Decision remote allocation

Update neighbor list

Retrieve neighbor list

Remote resource demand (capacity description)

BBU—Baseband unit eX2—Enhanced X2 Inter-RM—Inter-MSS-BBU resource manager

Radio bearer request ACK

Shared memory

Resource state

Intra-RM Resource status

Radio bearer request (resource description)

Radio bearer control per BBU

Resource pooling management case: no processing resource available at home MSS-BBU

After receiving an ACK from the router, the intra-RM sends a radio bearer request ACK message to the radio bearer control included in the home BBU, where the cell and control functions are allocated.

Backhaul/Fronthaul Solutions The centralized processing concept for mobile access systems modifies design requirements for both backhauling (link to and from the core network) and fronthauling (between the processing location and RRH). In addition, resource pooling among individual clusters puts another focus on the backhaul and results in a modified set of requirements. Modifications in the backhaul result from the fact that many BBUs are clustered within the same system in one single location. Consequently, the resulting supply data rates need to be adapted accordingly to values up to 10 Gbit/s and beyond for large clusters that are heavily loaded. Balancing the load among all cells of the cluster is advantageous so the accumulated link capacity no longer needs to be designed to meet the layout maximum of any individual site, but instead with respect to a resulting average value. Thus, the capacity is upgraded by employing a proportionality factor considerably lower than those compared to conventional distributed installations. Several approaches might be feasible for the fronthaul link from the MSS-BBU to the RRH (ref. Figure 8), ranging from ring structures over tree layouts to point-to-point links. Key selection parameters are determined by the process distribution among these elements, i.e. the data rate and latency requirements. The CPRI interface is an established standard [11] and is continuously updated. However, using system parameters of LTE very likely for future installations, data rates easily are in excess of 1 Gbit/s or even 10 Gbit/s to a single site. So a future LTE site with, e.g., three sectors with eight antennas each and a bandwidth of 40 MHz would require an uncompressed IQ data CPRI rate of 59 Gbit/s (per site). Therefore, sophisticated optical fiber transmission needs to be implemented, and the prospect of

144


DOI: 10.1002/bltj

using multiplexing and link sharing becomes rather limited. A modified split of PHYcell processing between the MSS-BBU and the RRH considerably reduces link data rate requirements and in turn allows multiplexing concepts. With the parameters mentioned, we would then end up with a site rate of around 5 Gbit/s, roughly a tenth of the uncompressed CPRI value, compatible with low-cost optical technologies. Due to very stringent latency requirements for radio access systems, fronthaul links are limited in length. Acceptable latency values should be lower than ~150 μs, corresponding to a physical link length at a maximum of 15 km. This needs to be considered when selecting routes and architectures. Similar limits apply for links between neighbored MSS-BBUs for efficient resource pooling. Therefore, cluster sizes of RRHs involving neighbored MSSBBUs are limited to about 10 km in radius.

Analysis and Opportunities So far, we have described the functional architecture for a cloud base station along with its technical challenges. In the following paragraphs we present an analysis quantifying the benefits of such a solution based on real world data and system simulations. It is obvious that offering increased user data rates in the evolving standards can only be achieved by more and more sophisticated and computationally complex algorithms in the physical radio layer. Base station equipment for a single cell must be dimensioned to carry the maximum busy hour traffic at the radio site, even if this occurs, for example, only 20 percent of the time. Moreover, peak traffic loads are often different depending on the characteristics of the area served, i.e., typical residential areas may be populated at times other than typical business areas. These variations in time and area can be utilized by a cloud approach. The estimated average long-term “potential cloud gain” is given by the following formula: maxj=1,. . . T rq(∑i=1, . . . ,C tr(i, j)+Δ″) Potential_ = 1− cloud_gain ∑i=1, . . . ,C rq(maxj=1,. . . , T)(tr(i, j)+Δ′)

MSSBBU

DCC RRH RRH RRH RRH

MSSDCC BBU

MSSDCC BBU

RRH RRH

RRH

RRH RRH MSSDCC BBU RRH

RRH

RRH MSSDCC BBU

MSSBBU

DCC

RRH

RRH RRH

MSSBBU

DCC

BBU—Baseband unit DCC—Decentralized cloud controller MSS-BBU—Multi-site/multi-standard BBU RRH—Remote radio head

Figure 8. Schematics of fronthauling options among MSS-BBU, RRH, and links among cooperating MSS-BBUs.

where tr(i,j) stands for the average traffic demand for the cells c(i), i = 1, … ,C at timeslots t(j), j = 1, … ,T (granularity in the range of 15 to 60 minutes). Resource quantization rq(tr) maps the traffic demand to the required discrete (processing) resources (e.g., the number of required CPUs, memory, or bandwidth). The overload margins Δ are added to cope with statistically short-term variations, when a certain level of grade of service shall not be exceeded. The margins may depend on the traffic distribution for a given service level of more than, e.g., 99 percent or

99.9 percent availability. Typically we can expect some multiplexing gains, which means that in the case of a larger traffic bundle (as in the cloud) the required margins are relatively lower than for single servers. A more detailed analysis of short-term effects is discussed in the section on “Cloud Simulation Studies.” In brief, the dimensioning rule for cloud resources is oriented to the maximum total resources required over time (sum over cells), whereas conventional cell planning estimates the sum of resources over the cells used at their individual peak traffic time.

DOI: 10.1002/bltj


145

Cloud Case Study In order to verify the potential cloud gain in realistic scenarios, samples of field traffic measurements have been used to retrieve daily and weekly user traffic profiles in selected service areas. From this data, the effect of cloud resource pooling has been studied to take different key factors into account, e.g., variation of traffic in time and location, as well as the cluster size for the resource pool (number of cells served by a fictitious cloud base station). In Ratti’s case study [20], location-based services were used to analyze mobile user activities in an urban region in the Milan, Italy metropolitan area. A geographical mapping of cellphone usage at different times of the day was recorded and evaluated. Though the study focused on voice traffic, the observed evolution of mobile user activity through space and time in that study proves that our assumptions around cloud gains based on traffic variation hold true. As an example, the paper shows the spatial distribution of mobile user traffic activity in the Milan area. The shift of traffic peak areas over time becomes evident. We reused this data in order to

build a fictitious network of radio base stations over the whole domain (240 three-sectorized cell sites in an area of 250 km2), aggregating traffic according the given activity values, and then clustering them into potential base station clouds of various size. As a performance measure, we assumed that “user activity in %” translates into a fraction of available aggregated throughput per cell, which was fixed at 180 Mb/s for the downlink of a three-sector LTE site with 20 MHz bandwidth each. The resource utilization of a cell (or accordingly, the cloud) is then given by the throughput used in relation to the maximum available throughput. Obviously, the potential cloud gain is not a “constant,” and depends heavily on the variation of traffic figures in the area considered. Nevertheless we observed typical gain estimations between 30 percent and 50 percent in terms of total capacities to be installed. This also varied in a systematic way depending on the cluster size (number of cells in the cloud), even as our clustering only took into account the uncorrelated neighbor relationship of cells without a dedicated planning process. The example depicted in Figure 9 shows the

(Mean) cloud gain per cluster

0.5

0.4

Utilization = 25% Effective throughput per cell = 45 Mb/s

0.3

0.2

0.1

0

0

50

100 150 Cells per cluster

Figure 9. Expected (mean) cloud gains in dependence of cluster size.

146


DOI: 10.1002/bltj

200

250

cloud gain for a cluster distribution of a certain size as the complement of the ratio of dimensioned cluster throughput to respective conventional throughput (as in formula above), and its mean value over cluster size at a system utilization of 25 percent. It can be deduced that with increasing cloud sizes, the expected (mean) cloud gain also increases rapidly, reaching sort of a saturation plateau for large cluster sizes, depending on the cell configuration. A numerical analysis of this effect and how it can be predicted in order to optimize the “cloud planning” (based on measured field data) is a subject for further study. Cloud Simulation Studies In order to estimate the statistical short-term effects of traffic variation with impact on cloud gains, we set up a system simulation, which models a cellular area with traffic created by randomly positioned mobile users. The current simulation study is based on a wireless LTE scenario consisting of 19 RRH sites with

a distance of 500 m. Each site provides three sectors. Altogether, the playground covers an area of 4.1 km². Our initial simulation results were gained from simplified and straightforward models. Table I shows the basic simulation parameters and their values. The traffic is generated according to a scalable model that implements the increased Internet traffic produced from today’s wireless terminals, which exceeds the capabilities of the 3GPP File Transfer Protocol (FTP) traffic model [3]. The user requests are created on the (IP-based) application layer according to a Poisson arrival process since we assume discrete arrivals of an infinite and independent number of sources. Web server responses with varying object sizes were modeled according to a mix of three Pareto log-normal distributions [14]. As soon as a request is generated, the position of the associated user is chosen randomly according to a scaled distribution between a uniform and a Gaussian distribution with its peak value in the

Table I. Simulation parameter values. Parameter

Value

Scenario

LTE, 10 MHz uplink, 10 MHz downlink FDD

Playground

4.1 km2, 19 sites, 500 m distance, 3 sectors per site

Mobiles

Unlimited number, not moving

Request messages

Created on application layer according to a Poisson inter-arrival process; constant message size of 1 kB

Load distribution

Randomly according to a configurable mix of uniform and Gaussian, mean in central site; non-full buffer model

Response messages

One per request; varying object sizes according to a mix of 3 log-normals (Hernandez-Campos)

Channel model

Path loss and static shadowing

Handover

Based on SINR

Outage

Requests from mobiles dropped if SINR below –6 dB

Scheduling

Frequency-diverse allocation cell reuse one scheme, resource-fair round-robin bandwidth assignment

Link adaptation

Capacity calculated according to Shannon formula, no losses, no HARQ/ARQ

Processing resource model

Processing resources scale linearly with the share of allocated radio resources

ARQ—Automatic repeat request FDD—Frequency division duplexing HARQ—Hybrid automatic repeat request LTE—Long Term Evolution SINR—Signal-to-interference-plus-noise ratio

DOI: 10.1002/bltj


147

100% 99.9%tile 99%tile 95%tile 90%tile

90% 80%

Used resources

70% 60% 50% 40% 30% 20% 10% 0% 0

0.2

0.4

0.6

0.8

1

Fraction of uniformly placed users

Figure 10. Quantiles of the resource usage for different load scenarios.

center of the playground. By varying the respective factor (1 meaning completely uniform, 0 standing for Gauss only) it is possible to easily adapt the share of hotspot users and the corresponding spatially nonuniform load distributions and their spatial variance. More realistic models for the spatially distributed load are also under consideration. The channel model considers path loss effects and static shadowing. The available air interface bit rate is currently derived from the estimated signal-tointerference-plus-noise ratio (SINR) and the channel bandwidth according to the Shannon channel capacity formula. The wireless users are then scheduled according to the frequency-diverse allocation cell reuse one scheme, which scatters user resources over the available bandwidth and operates on the same frequency channel. In a first step, a resource-fair round-robin bandwidth assignment is assumed to ensure that each user will have an equal share of

148


DOI: 10.1002/bltj

resources. This approach will be further enhanced by a detailed modeling of the scheduling function, taking the modulation and coding scheme (MCS) selection into account for the physical resource block (PRB) allocation. To generate the results shown in Figure 10, the network traffic load was varied by modifying the inter-arrival time of the Poisson process in order to determine the maximum system capacity. We also altered the share of hotspot users by varying the portion of spatially non-uniform distributed load in order to increase the variance of user density over the cluster area considered. As an output, the fraction of (overall) processing resources used was measured (at comparable quality and outage levels), as this is an indication of the potential resource savings in a cloud scenario. In a first step, we assumed that the portion of required processing resources scales linearly with the share of allocated radio resources.

The impact of the actual MCS distribution on processing resources will be studied later on. The simulations confirmed that the maximum acceptable traffic load depends on the fraction of uniformly placed users. In contrast to scenarios with a proportionally high share of uniformly distributed users (i.e., low spatial variation of traffic distribution), configurations with many hotspot users quickly overload resources available only at the center of the playground. The initial simulation series was performed to determine the cumulative distribution function (CDF) of overall resource usage per transmission time interval (TTI) depending on the fraction of uniformly placed users. Figure 10 shows the resulting quantiles for 90 percent, 95 percent, 99 percent, and 99.9 percent of the time the system is running at its capacity limit. If we have only hotspot users (exclusively Gaussian distributed users), the system reaches a resource utilization level of up to 20 percent 90 percent of the time, and up to 38 percent 99.9 percent of the time. In this case, pooling gains of 62 percent to 80 percent result from the non-uniform distribution of the mobile devices and the user traffic model. With only uniformly placed users, up to 65 percent of the resources are in use over 90 percent of the time, and up to 91 percent of resources over 99.9 percent of the time with the system at its capacity limit. In this case, the pooling gains of 9 percent to 35 percent are only caused by the user traffic model. The uniform case is the reference for the comparison to handle the non-uniform traffic by a centralized processing cloud. From these results, we expect that a cloud approach can drive improvement in resource utilization between the extreme user distribution cases (non-uniform to uniform) up to a factor of 3. These results are the first indications of the potential benefits of cloud concepts in baseband processing that also include short-term statistics of the user traffic model. However, the initial findings need further confirmation by broadening the scope of the covered scenarios (realistic cell and traffic distributions) and including more relevant influence factors into system simulation such as scheduling algorithms and modulation and coding schemes with their processing complexity.

Conclusions and Outlook We have presented a novel research concept for future centralized radio base stations that revolutionizes conventional baseband processing in radio access networks by integrating elements of cloud computing. A key driver for this concept is reducing the number of processing resources required without compromising performance or—taking another perspective—increasing capacity-consuming unmodified resources. In turn, this potentially creates considerable cost savings for the operator. The actual amount depends very much on the operators’ scenario, cluster sizes, availability of transport connections, and the equipment already deployed (RRHs and antennas). Therefore this can hardly be estimated in general, but needs detailed considerations. Moreover, the ability to provide further flexibility and processing headroom for future functional standard extensions in LTE-Advanced (e.g., CoMP and other concepts for interference mitigation, hence radio capacity enhancements) without major changes in deployed infrastructure might be attractive to operators as well. By starting from conventional architecture concepts, our proposal overcomes the limitations and restrictions of the conventional direct mapping between the processing elements and antenna. In order to achieve and profit from user-related resource pooling rather than an intermediate solution limited to cell-based load assignment, we performed a careful analysis of process flows and associated technical requirements. This included requirements for simultaneous multi-standard processing. Study of the presented functional and control architecture revealed its capability to simultaneously integrate state-ofthe-art and advanced versions of the 3GPP standard family within the approach. In order to quantify key elements of the concept, we presented initial results of several modeling activities currently underway that proved the basic assumption of substantial resource and cost savings. However, in order to transfer our research into a real product approach, all system elements still need careful evaluation.

DOI: 10.1002/bltj


149

Other features such as operator sharing can be envisaged with such a concept, where the different baseband pools are operator-specific and the radio sites are shared among different operators. Last but not least, additional machine-to-machine traffic scenarios will be considered as this will cause an even higher variance of data rates in different cells, where RAN cloud computing will provide a higher flexibility of processing allocation. Another object for further study might be the flexible use of processing resources based on an increasing share of general purpose processors, not excluding appropriate hardware accelerators for layer1 functions. This may pave the way for less specialized hardware configurations, thus enabling shorter time to market. Another advantage lies in the integration of application and access cloud computing on a common platform. User-related applications might profit considerably from expected low latency responses and the heavy processing load associated with these applications can be partly shifted from the Internet to the access network. Applications envisaged for such an approach may be driven by augmented reality or high volume video sharing. Acknowledgements The authors would like to thank Dipl.-Ing. Thomas Werthmann from the Institute of Communication Networks and Computer Engineering in Stuttgart, Germany, who contributed results from the simulation study in the context of a collaboration. In addition, the authors gratefully acknowledge valuable contributions and comments from our Bell Labs team, and in particular from Horst Roessler. *Trademarks CPRI is a trademark of Nokia Siemens Networks. InfiniBand is a registered trademark of the InfiniBand Trade Association. RapidIO is a trademark of RapidIO Trade Association.

References [1] 3rd Generation Partnership Project, . [2] 3rd Generation Partnership Project, “Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Overall Description,

150


DOI: 10.1002/bltj

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

Stage 2,” 3GPP TS 36.300, . 3rd Generation Partnership Project, “Evolved Universal Terrestrial Radio Access (E-UTRA), Further Advancements for E-UTRA Physical Layer Aspects,” 3GPP TR 36.814, . 3rd Generation Partnership Project, “Evolved Universal Terrestrial Radio Access Network (E-UTRAN), X2 Application Protocol (X2AP) (Release 10),” 3GPP TS 36.423, Mar. 2011, . 3rd Generation Partnership Project, “Feasibility Study for Further Advancements for E-UTRA (LTE-Advanced),” 3GPP TR 36.912, . 3rd Generation Partnership Project, “LTE-Advanced,” . 3rd Generation Partnership Project, “Radio Interface Protocol Architecture (Release 10),” 3GPP TS 25.301, v10.0.0, Mar. 2011, . Alcatel-Lucent, “lightRadio Portfolio: White Paper 1 – Technical Overview,” Technology White Paper, 2011. China Mobile Research Institute, “C-RAN: The Road Towards Green RAN,” White Paper, Version 2.5, Oct. 2011, . Cisco, “Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2011– 2016,” White Paper, Feb. 14, 2012, . Common Public Radio Interface, “CPRI Specification, Version 5.0,” . D. M. Dicke and P. Cameirao, “lightRadio™ Baseband Processing and Backhauling,” Alcatel-Lucent TechZine, Sept. 22, 2011, . Ericsson, “Vestberg Foresees Industry Shift,” Press Release, Feb. 15, 2010, .

[14]

[15]

[16]

[17]

[18]

F. Hernández-Campos, J. S. Marron, G. Samorodnitsky, and F. D. Smith, “Variable Heavy Tails in Internet Traffic,” Perform. Eval., 58:2–3 (2004), 261–284. InfiniBand Trade Association (IBTA), “InfiniBand® Roadmap,” . P. Mell and T. Grance, “The NIST Definition of Cloud Computing,” U.S. Dept. of Commerce, National Institute of Standards and Technology, Information Technology Laboratory, Computer Security Division, NIST Special Pub. 800–145, Sept. 2011. NGMN Alliance, “Projects,” . F. Pujol, “Mobile Traffic Forecasts 2010– 2020 & Offloading Solutions,” IDATE Consulting and Research, May 15, 2011, . RapidIO Trade Association, “RapidIO Technology Overview,” . C. Ratti, R. M. Pulselli, S. Williams, and D. Frenchman, “Mobile Landscapes: Using Location Data from Cell Phones for Urban Analysis,” Environ. Planning B, 33:5 (2006), 727–748. M. Strasser, “FTTA Fiber-to-the-Antenna – Technology Change in Mobile Communications,” Huber + Suhner White Paper, .

various generations of mobile radio networks. Mr. Haberland has published numerous research papers in journals and scientific conferences and holds many related patents. His current focus is on mobile networking and cloud technologies.

FARIBORZ DERAKHSHAN is a research engineer in the Network Technologies Research domain at Bell Labs in Stuttgart, Germany. He graduated with a degree in electrical engineering at the University of Karlsruhe in Germany. Mr. Derakhshan worked in the Numeric Research Group for Supercomputing, as well as the Networking Group at the University of Karlsruhe before joining Bell Labs 12 years ago. His current research interests include resource management and network optimization algorithms for cloud environments.

(Manuscript approved December 2012)

HEIDRUN GROB-LIPSKI is a research engineer at AlcatelLucent Bell Labs in Stuttgart, Germany. She received her diploma in mathematics from the Julius-Maximilians University of Würzburg. She has substantial research experience in innovative technologies, architectures, and protocols for fixed and wireless communication systems. Her current focus lies in cloud networking technologies and resource management for future virtualized networks. Ms. Grob-Lipski is a member of the Alcatel-Lucent Technical Academy (ALTA) and holds several international patents predominantly in the area of resource management, signaling, and LTE handover strategies. In these fields she (co-)authored numerous technical proposals and research papers.

BERND HABERLAND is a department head at AlcatelLucent Bell Labs in Stuttgart, Germany. He received a Dipl. Ing. degree in communication technology, digital signal processing, and computer science from the University of Erlangen/Nuremberg, Germany. After joining Alcatel-Lucent (at that time SEL) he took on responsibilities in research, development, integration, and customer validation for mobile communications and has led a number of multinational teams. He became an Alcatel Fellow in 2004 and a Bell Labs Fellow in 2010 in recognition of his extensive contributions and breakthroughs in

RALF KLOTSCHE is a system design engineer at Bell Labs in Stuttgart, Germany. He received a diploma in electrical engineering from the Fachhochschule Bielefeld, Germany, and then joined the Research Centre at Standard Elektrik Lorenz (Alcatel-SEL) after consulting on various industry projects. He has contributed to several German national and international European Union (EU)-funded research projects on terminals, network devices, services, and system design for broadband switching and broadband

[19]

[20]

[21]

DOI: 10.1002/bltj


151

access networks, leading projects in the domain of service deployment and reachability management. His latest activities include work on solutions for the Future Internet, on human factors, and on future communication associations within Bell Labs’ Grand Challenge Project. Mr. Klotsche is currently focused on concepts and implementations for a media cloud and related resource distribution and allocation.

WERNER REHM is a senior research engineer at AlcatelLucent Bell Labs in Stuttgart, Germany. He holds a diploma degree and a Ph.D. in semiconductor physics, both from Marburg University in Germany. Prior to his current position at Bell Labs, he spent a few years as postdoc at the Max Planck Institute for Solid State Research in Stuttgart, Germany. He has gathered broad experience with work on optical semiconductors, fiber communication, and components assembly and combined that with research processes and project management. Dr. Rehm has published numerous research papers in journals and scientific conferences. His current focus is on networking and cloud technologies.

PETER SCHEFCZIK is a member of technical staff at Bell Labs in Stuttgart, Germany. He received a Ph.D. degree in applied mathematics from the University of Erlangen/Nuremberg in Germany. In his career of over 20 years, he has been involved in the design and development of new network architectures and advanced protocols for both wireless and wireline communication systems. In addition to his work in fourth generation wireless network architectures, heterogeneous access networks, and network convergence, he has played a key role in the design and creation of the base station router (BSR), which today is offered within the Alcatel-Lucent small cell solution. His current research interests concentrate on laying the foundations for the next-generation Internet and include femtocell environments for the home, sensor networks, and the Internet of Things. He is an advocate for cloud networking technologies and cloud solutions for future virtualized radio access networks within collaborative European research projects. Dr. Schefczik received the Central Bell Labs Team Award in 2004 and the Bell Labs President’s Award in 2005. He was awarded membership in the

152


DOI: 10.1002/bltj

Alcatel-Lucent Technical Academy (ALTA) in 2008 and 2011. He holds several international patents in the area of data transmission and mobile networks. He has also authored or co-authored numerous articles and research papers in this field.

MICHAEL SOELLNER is a technical manager at Bell Labs in Stuttgart, Germany. He received a diploma in mathematics from the University of Erlangen/Nuremberg, and a Ph.D. degree in applied mathematics from the University of Bochum, both in Germany. Since then, he has held various positions in communication technology where he focused on systems engineering and research for advanced technologies, protocols and architectures for mobile systems beyond third generation (3G) and future wireline/wireless converged networks. At present, Dr. Soellner is responsible for a collaborative crossindustry research project in the European FP7 Research Programme dealing with innovative approaches to a Future Internet regarding connectivity and cloud networking. Current focuses of his research work are advanced technologies for cloud-based mobility and resource management in the Future Internet. ◆