A Seamless Integration of Computationally-Enhanced Base Stations into Mobile Networks towards 5G Miguel A. Puente1, Zdenek Becvar2, Matej Rohlik2, Felicia Lobillo1, Emilio Calvanese Strinati3 1
ATOS, Spain (
[email protected]); Czech Technical University in Prague, Czech Rep; 3 Commissariat à l’Energie Atomique et aux Energies Alternatives, France. 2
Abstract—Following Mobile Cloud Computing, Mobile Edge Computing and Network Functions Virtualization tendencies, we envisage the utilization of computationally-enhanced base stations as computing nodes in which Virtual Machines can be deployed to perform computing tasks, leveraging the closeness of computing resources to end-users. This paper presents a seamless approach for the deployment of computationally-enhanced Small-Cells, also applicable to macro base stations, with no impact on the LTE-A architecture. To that end, the conventional mobile traffic and the traffic generated and consumed by the new computing resources are segregated and handled independently at the access point, with the latter being transmitted through the radio channel making use of the pre-established Data Radio Bearers. Assuming a general-purpose hardware configuration for the Small-Cells, we describe the functionality of the different physical and logical components along with the new protocol stacks and interfaces. Finally, we evaluate the additional delay and amount of signaling overhead introduced by the system to benchmark the proposed solution. Index Terms— mobile cloud computing, mobile computing, LTE-A, small cells, small cell cloud, 5G.
O
edge
I. INTRODUCTION
ne of the key enablers to satisfy the tremendous growth of network capacity demand forecasted for the next decade is the (ultra) dense deployment of heterogeneous radio access points, where macro-cells coexist with small-cells over the same geographical area [1], providing significant increase of spatial reuse of spectral resources. At the same time there is a strong tendency towards Mobile Cloud Computing as a way to extend mobile devices’ capabilities and battery lifetime; and Mobile Edge Computing, which enables the improvement of network performance due to the proximity of computing resources to end-users. The work in [2] introduces the “Small-Cell Cloud” (SCC) concept describing its benefits towards users and operators. SCC refers to a cluster of computing-enhanced Small-Cells (SCeNBce) interconnected for mobile computing purposes. The SCC locates computing resources by means of Virtual Machines (VM) running at the base stations, bringing the computing power closer to the end-user.
The SCC differs from traditional clouds in the proximity of computing resources to users and in the dynamicity of the scenario. The users’ activity and requested resources may fluctuate frequently. Moreover, SCeNBces can be switched on and off at any time at users’ will (residential deployments) or for energy saving purposes, since energy efficiency is a major constraint and one of the solutions is to switch off the access point when not needed [3]. This brings management challenges such as dynamic resource allocation and reallocation (e.g. when a node is switched off, its allocated computing resources might be migrated to other nodes). To that end the SCC is orchestrated by the Small-Cell Cloud Manager (SCM), which is a logically centralized component that dynamically manages the SCC resources. The SCM is aware of the overall cluster context (both radio and cloud wise) and optimizes the overall system’s operation. The SCM and the SCeNBces are new elements to be integrated with the LTE-A architecture. The SCM can be deployed in various places in the network. Different architectural options for the SCM location are described in [2]. In this paper, we follow the “standalone SCM” architecture (Figure 1), in which the SCM is not part of any other component of the LTE-A network architecture [4]. This is the most generic architectural option and the others can be understood as derivations of this one, since the solutions used in the standalone architecture can be used in the other architectures without major modifications regardless of the SCM location. This paper presents a detailed approach for SCC deployment, architecture and protocols (also applicable to macro eNBs), which has no architectural or functionality impact on the existing LTE-A specification [4]. The proposed approach allows the deployment of SCCs with no modifications in the MME, S-GW, P-GW and their interfaces. The LTE-A stack and interface with the UE remains the same as well, but in this case the UE covers application level enhancements to manage and leverage the new distributed cloud resources.
978-1-4799-8088-8/15/$31.00 ©2015 IEEE
Uu Z‐UE
SCeNBce
UE
S1‐U S1‐MME X2 Z
Z
SCM
Uu
UE
Uu
SCeNB UE
P‐GW S1‐U S1‐MME X2
S1‐U
S‐GW
S1‐MME
Uu Z‐UE
UE SCeNBce
S1‐U S1‐MME X2 Z
MME
HSS
Fig. 1. SCC architecture. Fig. 2. SCeNBce hardware configuration.
II. ARCHITECTURE Figure 1 depicts the overall view of the architecture, showing the LTE-A network enhanced with SCC components and interfaces [2]. The Radio Access Network (RAN) includes (i) SCeNBce, the computationally-enhanced SC and (ii) SCeNB, a regular Small Cell eNB. The Core Network includes (iii) the Service GW (SGW), a mobility anchor which transfers the user traffic from the old serving base station to the new one upon radio handover; (iv) the P-GW, a gateway to external Packet Data Networks (i.e. the Internet); and (v) the MME, Mobility Management Entity which commands the SGW. Finally (vi) the SCM and (vii) UE: conventional 4G smartphones connected through the LTE-Uu interface and future 5G-like smartphones, connected also through the Z-UE interface and thus making use of the SCC capabilities. The SCeNB and SCeNBce communicate among them using the X2 interface and with the MME over the S1-MME control interface using SCTP. Both are connected to the S-GW over the S1-U interface, on which the user data is encapsulated over GTP. The SCM and SCeNBces exchange control messages over the new proposed Z interface. The UE communicates with the base station through the radio channel using the legacy Uu interface and the proposed Z-UE interfaces for the conventional LTE and the SCC signaling respectively. Communications going to and coming from the Virtual Machines running in the SCeNBces are not tunneled over GTP, being transmitted using the typical IP stack. III. SCENBCE HARDWARE CONFIGURATION To enable the SCC concept, a shift from regular SCeNBs towards computing-enhanced SCeNBces is required. The HW configuration of a legacy Small Cell (SCeNB) [5] and the HW configuration considered for the SCeNBce are depicted in Figure 2. In the legacy SCeNB, the Baseband Processor (BP) [6], which is in charge of the radio signal processing, carries out all the signal and traffic processing, connecting to the backhaul through an Ethernet interface and to the antenna through a modulator/demodulator. The BP hosts its own memory and runs a real-time operating system to cope with the radio signal processing requirements.
In the SCeNBce, a single system bus architecture interconnects all the components: (i) the BP, placed as a peripheral; (ii) the CPU, which is a general-purpose processor entity that executes the computing tasks along with (iii) the main memory; (iv) persistent storage is used to store the needed non-volatile data; (v) the Network Interface Card (NIC) [7]; and (vi) the Direct Memory Access (DMA) controller, which allows direct read/write access to the main memory without occupying CPU cycles. Using DMA, the CPU initiates the data transfer from the BP to the memory, carries out other tasks while the transfer is in progress, and receives an interrupt from the DMA controller when the transfer is done. This operation is equivalent to the data transfer to and from the NIC. SCeNBces allow the CPU to be in charge of traffic processing tasks that are conventionally part of the BP functionality in regular base stations. As we show later, we leverage this feature for the seamless SCC-LTE integration. Since processing traffic in the CPU involves overhead, the possibility of processing the entire traffic load in the BP (or even by means of an additional network processor placed as a peripheral) can be considered. However, it is preferable to execute network functionalities directly at the CPU since: (i) we assume that a HW architecture dedicated to that purpose will have its operation adapted to minimize the overhead, i.e. attending BP and NIC interrupts with maximum priority (thus handling the traffic as soon as it arrives at the SCeNBce) and using DMA whenever possible (thus minimizing computing resources dedicated to traffic aspects); (ii) additional processing hardware increases cost and complexity; (iii) recent developments of “network softwarization” widely recognized by network operators, such as NFV [8] and SDN [9], call for the usage of general purpose hardware for deploying network functionality which is typically carried out by means of physical middle-boxes (NAT, firewalls, etc.), which has clear benefits on scalability, programmability, customizability, costs, time to market, etc.; (iv) supporting the previous point, companies such as Intel are already thinking of this new scenario and preparing new general-purpose HW processors that have the performance to process packets and run multiple applications simultaneously [9]; (v) Business
IP
UDP GTP‐U
S‐GW
IP
Payload
IP
UDP GTP‐U
IP
Payload
S5 TEID DL S5 TEID (DL) S1 TEID (DL) S1 TEID (DL) IP
UDP GTP‐U
Src IP = S‐GW Dst IP = BS
IP
S1 TEID (DL)
S1 bearer
Payload Src IP = ext Dst IP = UE
S1 TEID (DL)
IP Table: UE IP ‐> TEID SCC IPs
DRB ID (DL)
UDP GTP‐U
Src IP = S‐GW Dst IP = BS
S1 bearer SCeNBce
IP
IP
S1 TEID (DL)
Payload Src IP = UE Dst IP = ext
No SCC IP S1 TEID (UL) DRB ID (UL)
Uu DRB ID (DL)
IP
Uu
Src IP = ext Dst IP = UE DRB ID (UL)
DRB UE
DRB
Payload
IP
Payload
Src IP = UE Dst IP = ext
Payload DRB ID (UL) Application
UL TFT
Fig. 3. S1 bearer/DRB mapping (mobile traffic).
Systems Support functions (such as charging) may need to see the traffic being transmitted, which will force the traffic (or a part of it) to be redirected to where those functions reside, i.e. the system’s CPU, or will involve traffic monitoring and therefore additional overhead; and (vi) additional processing hardware does not eliminate the overhead, and some performance degradation of the conventional mobile traffic when compared to regular SCeNBs will occur anyway. Therefore, in our vision, it is fundamental to locate the novel SCC network functionalities as software entities executed in the CPU of the SCeNBce rather than in the BP or additional network processors. The BP is a separated component due to the tight real-time requirements of the radio signal processing, also limiting the problems related to the potential security risk related to denying conventional mobile traffic by overloading the computing resources if located at the BP. IV. SCC-LTE SEAMLESS INTEGRATION A seamless SCC-LTE integration requires differentiating two classes of traffic: (i) the conventional mobile traffic traversing the mobile network (e.g. voice calls) and (ii) the cloud traffic going to and coming from the cloud entities (Virtual Machines and SCM). In the following we describe how the integration is achieved for both communication directions. A. Mobile/cloud traffic segregation The UE can communicate with an entity located outside the operator’s network by means of communications through the P-GW, which implies a significant latency. Instead, by offloading the cloud traffic directly to the Internet the latency is kept to a minimum, while reducing the core network’s load. For the cloud traffic to be offloaded to the Internet without traversing the core network, a traffic breakout mechanism to
Fig. 4. UE IP address/DRB (cloud traffic).
identify and segregate mobile and cloud traffic is needed at the SCeNBces. LTE-A technologies addressing traffic offload are for example LIPA and SIPTO [10]. They offload traffic to a local network (LAN) and to the Internet respectively. Both are based on the collocation of a Local Gateway (L-GW) either on the eNB or separated from it. The inclusion of the L-GW involves additional functionality that must be added to the EPC components [11] and thus impacting the existing LTE-A architecture. Although SIPTO functionality fits our purposes, we have not relied on it due to its architectural impact. However, we intend to further study how to be in line with these 3GPP specifications. The segregation functionality consists on the following main steps: (1) The BP processes the radio signal and decapsulates the physical and link layers; (2) the data is inspected to identify whether it is conventional mobile or cloud traffic; (3) the data is encapsulated accordingly for each kind of traffic and (4) handed over to the NIC to be delivered through the backhaul. Traffic processing tasks (2) and (3) can be carried out either by the BP or the CPU, while (4) can only be launched by the CPU, which handles the connections with the NIC. The overall goal is to encapsulate and transmit the conventional mobile traffic over the S1-U (data plane) or S1MME (control plane) interfaces, thus being transmitted to the EPC (S-GW and MME respectively); and encapsulate the cloud traffic over the typical IP stack, thus being offloaded to the Internet without traversing the core network. B. Delivery of cloud traffic to the UE The segregation functionality takes place for communications originating at the UE. In the same way, for communications terminating at the UE, the cloud and conventional mobile traffic (arriving at the base station from different sources and with different protocol stacks) shall be delivered to the corresponding UE appropriately.
The cloud traffic requires special treatment compared to the conventional mobile traffic. The IP packets arriving at the SCeNBce shall be delivered to a specific UE; however, the only information that these packets contain is the UE IP address. Having only the IP address makes it impossible for a regular base station to identify the destination UE, since these do not see nor handle the IP layer. Therefore, a mechanism to identify and deliver the cloud traffic to the UE is needed at the SCeNBce. In LTE-A the traffic is transmitted though pre-established bearers [12] between the UE and the P-GW. Each UE has at least one default bearer established during the UE Attach procedure [13] which has nominal QoS requirements. Additionally, the UE may establish further default bearers (on a PDN connection basis) and dedicated bearers (with different QoS requirements). These bearers are not IP based, i.e. the routing of the traffic along the P-GW – S-GW – eNB – UE path is not based on the UE IP address, but rather on GTP tunnels from P-GW to eNB (S1/S5/S8 bearers) and explicit radio resource allocation from eNB to UE (radio bearers and logical channels). The mapping between S1 bearers (GTP tunnels) and Data Radio Bearers (DRB) is done based on the TEID (Tunnel Endpoint ID) and DRB ID (Data Radio Bearer ID). The TEID is transmitted in the GTP-U header allowing de-multiplexing traffic incoming from remote tunnel endpoints and also allowing multiplexing of different users (using their associated DRB IDs) [14]. Figure 3 depicts the S1-DRB bearer mapping carried out in LTE-A, which is also present in the SCC. The solution we propose to identify the UE is to map UE IP addresses to radio bearers/logical channels (i.e. to TEID) at the SCeNBce, so that the TEID can be used to send the data through a Data Radio Bearer. All the established bearers (default and dedicated) have their associate TEID at the base station and any of them could be picked up to send the user cloud traffic. Dedicated bearers transport QoS sensitive traffic (e.g. voice calls), so these bearers cannot be used for the cloud traffic. On the other hand, default bearers have nominal QoS constraints and can accommodate the additional cloud traffic without disrupting the established dedicated bearers. Consequently, using the initial default bearer, which is established upon UE attachment, is the most desirable option since: (i) it is always present as long as the UE is connected; (ii) the necessary data to carry out the mapping (i.e. UE IP and TEID) can be obtained upon the UE Attach procedure and (iii) it has the minimal QoS requirements and the lowest priority [12], which does not affect the traffic of the QoS constrained mobile services. This leaves the cloud traffic to be transported with an effective throughput that depends on the conventional mobile traffic load. Figure 4 depicts the UE IP address-DRB mapping used to deliver the cloud traffic to the UE, using the UE IP address-TEID mappings stored in the IP Table located at the SCeNBce. V. COMPONENTS AND INTERFACES Figure 5 depicts the design overview of the hardware and software components that form the SCC along with their
SCeNBce Guest Operating System
VM VM VM
UE IP
SCM
Host Operating System
Z TCP IP L2 L1
SCM‐BS
TEID
SCC‐GW Z
IP Table UE IP
BP
MME IP
TEID SCC IPs
NAS S1‐AP Mod/ Demod.
App TCP/UDP IP GTP‐U
Antenna
Uu
App TCP/UDP IP PDCP RLC MAC PHY
NAS RRC PDCP RLC MAC PHY
Z TCP IP PDCP RLC MAC PHY
Z TCP IP GTP‐U
NIC S1‐U
Z‐UE
X2‐AP SCTP IP L2 L1
NAS S1‐AP SCTP IP L2 L1
MME
S1‐MME
App TCP/UDP IP GTP‐U UDP IP L2 L1
P‐GW
S‐GW
App TCP/UDP IP L2 L1
SCeNBce
UE SCM‐UE
X2‐U UDP IP L2 L1
VM VM
Fig. 5. SCC modules, interfaces and protocol stacks.
interfaces and protocol stacks. The physical nodes, highlighted with light grey background, are the UE, SCeNBce and the EPC components MME, S-GW and P-GW. Highlighted with dark grey background are the individual components relevant for the SCC operation. The hardware part is composed of the BP, modulator/demodulator, antenna and the NIC; the logical part is composed of all the other components. Although presented so far as a standalone component, the SCM can be understood as a logical entity that can be placed in either a dedicated physical device or a shared one. The SCC-GW (Small-Cell Cloud Gateway) is the component in charge of executing the traffic segregation functionality. It holds an IP table storing the UE IP-TEID mapping along with the MME IP and the set of IP addresses belonging to the cloud entities. The SCC-GW and BP exchange S1-AP packets for the LTE-A control plane and GTP-U packets for the LTE-A user plane and cloud traffic. In turn, the BP communicates the UE IP-TEID mappings to the SCC-GW upon the initial default bearer establishment. The SCC management is carried out by three entities: the SCM, the SCM-BS (SCM Base Station) and the SCM-UE (SCM User Equipment). Their role is to perform the SCM functionality at each component, allowing the placement of distributed management functionality at the SCeNBce and UE to improve efficiency and scalability. The most relevant management action for the SCC-LTE integration is the update of the IP table with the IP addresses of the cloud entities. To that end, upon a new SCeNBce connection the central SCM distributes the IP address to the SCM-BS in other SCeNBces, which in turn update their IP table locally. The SCM and SCM-BS communicate through the Z
interface while the SCM-BS and SCM-UE communicate through the Z-UE interface. The Z and Z-UE interfaces use the Z Protocol, which is an application protocol that defines the exchange of control messages. Actions described by Z Protocol messages are for example deployment/destruction of VMs, monitoring messages, updates of the IP tables and all the other management actions needed for the SCC operation.
existing mobile services, since the delay and traffic overhead introduced by the SCC can be considered almost negligible. In future work we will address the complete SCC design and implementation addressing topics such as security, mobility and dynamic clustering; validating as well that the solution does not impact LTE-A. We also intend to show detailed system performance studies over a real proof-of-concept, fully demonstrating the feasibility of the solution.
VI. EVALUATION In this section we evaluate the delay introduced by the proposed solution from a theoretical point of view. Additionally, we assess the signaling overhead introduced by the management system by means of simulations. For end-to-end measurements, most tools and research work assume that local processing and queueing delays are negligible [9] and interpret their results without considering local delays. Therefore, in this evaluation we consider negligible the additional latency due to the traffic processing tasks in the CPU, interrupts, context switching and so on. Besides, we have described hardware requirements to minimize this overhead in section IV. However, virtualization involves the inclusion of additional software layers whose latency impact is not negligible. The work in [9] analyzes the effects of virtualization on packet delays. For the worst case, they obtain a one-way delay overhead of 5 ms when the workload is placed in the host operating system, which is where the SCC-GW is located (Figure 5). The delay due to virtualization would be the only one included by our solution due to computing and software aspects. ITU-T G.114 [16] recommends a maximum of a 150 ms one-way latency for mobile services (e.g. voice calls). Therefore, we can conclude that the delay introduced by the SCC does not affect the existing services in a significant way. As for traffic load overhead, we have analyzed the signaling overhead introduced by the SCC management system. We have carried out simulations considering real-world IPv4 residential scenarios for the case in which users execute an application that offloads mobile computation to the SCC. In this case we have obtained that the signaling overhead introduced by the Z protocol when there are 50 users per base station is roughly 53 kB/minute, i.e., 0.018 kB/s per user. Therefore, we can conclude that amount of overhead generated due to management of the proposed solution is negligible even for very low quality backhauls such as ADSL.
ACKNOWLEDGMENT This work has been performed in the framework of the FP7 project TROPIC IST-318784 STP [18], which is funded by the European Community. The Authors would like to acknowledge the contributions of their colleagues from TROPIC Consortium (http://www.ict-tropic.eu). REFERENCES [1] [2] [3] [4] [5] [6] [7]
[8] [9] [10] [11] [12] [13] [14]
VII. CONCLUSION AND FUTURE WORK
[15]
In this paper we have presented a solution for the seamless deployment of computationally-enhanced base stations within existing 4G networks. To that end, conventional mobile traffic and computing related traffic are segregated and handled individually at the base stations. This solution relies on Network Function Virtualization and Software Defined Networking concepts, proposing the inclusion of a softwarebased gateway (that can be virtualized) which executes the traffic segregation functionality. We have evaluated that the delay introduced by the solution does not pose a threat to the
[16] [17]
[18]
J. G. Andrews, H. Claussen, M. Dohler, S. Rangan, M. C. Reed, "Femtocells: Past, Present, and Future", IEEEJournal on Selected Areas in Communications, Vol. 30, No. 3, April 2012. F. Lobillo, Z. Becvar, M.A. Puente et al, “An architecture for mobile computation offloading on cloud-enabled LTE small cells”, IEEE WCNC, 2014. A De Domenico, E Calvanese Strinati, A Capone, “Enabling green cellular networks: A survey and outlook”, Computer Communications 37, 5-24, January 2014. 3GPP TS 36.300, “Evolved Universal Terrestrial Radio Access (EUTRA) and Evolved Universal Terrestrial Radio Access Network (EUTRAN); Overall description; Stage 2”. Texas Instruments, Mobile Base Transceiver Station Block Diagram. Available: http://www.ti.com/solution/metro_micro_base_station Texas Instruments, TCI6636K2H Multicore DSP+ARM KeyStone II System-on-Chip (SoC), revised October 2013. Available: http://www.ti.com/lit/ds/symlink/tci6636k2h.pdf Brien M., "Networking Basics: Part 1 - Networking Hardware". Windowsnetworking.com. TechGenix Ltd. Retrieved, 2006. Available: http://www.windowsnetworking.com/articlestutorials/netgeneral/Networking-Basics-Part1.html. “Network Functions Virtualisation. An Introduction, Benefits, Enablers, Challenges & Call for Action”, SDN and OpenFlow World Congress, October 2012. Open Networking Foundation, "Software-Defined Networking: The New Norm for Networks", White paper, April 2012. 3GPP TR 23.829, “Local IP Acess and Selected IP Traffic Offload (LIPA-SIPTO)”. Gupta Rajiv, Rastogi Nupur, “LTE-Advanced LIPA and SIPTO”, Aricent, 2012. 3GPP TS 24.301, “Non-Access-Stratum (NAS) protocol for Evolved Packet System (EPS); Stage 3”, version 12.6.0, September 2014. 3GPP TS 23.060, “General Packet Radio Service (GPRS); Service description; Stage 2”, release 13, September 2014, 3GPP TS 29.281, “General Packet Radio System (GPRS) Tunnelling Protocol User Plane (GTPv1-U)”, version 12.0.0, September 2014. Whiteaker J. et al, “Explaining Packet Delays under Virtualization”, ACM SIGCOMM, 2011. ITU-T G.114, “One-way transmission time”, 2003. M. Laner et al, “A comparison between one-way delays in operating HSPA and LTE networks”, 10th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt), 2012. FP7 project TROPIC: "Distributed computing, storage and radio resource allocation over cooperative femtocells," ICT-318784, www.icttropic.eu.
