MPLS-Based Satellite Constellation Networks

438

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

MPLS-Based Satellite Constellation Networks Anton Donner, Matteo Berioli, Student Member, IEEE, and Markus Werner, Senior Member, IEEE

Abstract—Nongeostationary satellite constellations with intersatellite links are a challenge for networking due to their continuously changing topology. In order to make maximal use of the network’s capacities, special attention has to be paid to routing and traffic engineering. Multiprotocol label switching (MPLS) as underlying protocol is an interesting candidate for this task since it offers many possibilities to exert influence on traffic flows and supports today’s dominating Internet protocol traffic very well. This paper describes a general MPLS-based networking concept for satellite networks and discusses different scenarios considering the particularities and constraints of the dynamic topology. Functional elements of MPLS like ingress, egress, or core routers have to be mapped onto the physical entities of the network and prerequisites for traffic engineering are discussed. Routing and rerouting of paths is of key interest since this affects route computation effort and routing performance. Thus, an analytical estimation of routing effort is deduced and numerical and simulation results are presented. Index Terms—Intersatellite link (ISL), multiprotocol label switching (MPLS), nongeostationary orbit (NGSO) satellite constellation network, routing effort, traffic engineering.

I. INTRODUCTION

G

EOSTATIONARY satellite systems are very well suited for broadcast services, but the ever increasing need for bandwidth of point-to-point and (multi)point-to-multipoint applications have again raised particular interest in broadband nongeostationary orbit (NGSO) satellite constellations, which offer smaller latency, lower free space loss, true global coverage, and better reuse of available ground-space communication frequencies. Earlier research work on constellations with intersatellite links (ISLs) has clearly shown that the asynchronous transfer mode (ATM) would be an interesting candidate for dynamic satellite network topologies due to its virtual path/virtual channel concept and, moreover, as different levels of quality-of-service (QoS) guarantees are provided. Based on the connection-orientation paradigm and on a well-specified framework of services and traffic classes (including their QoS parameters) supported, reliable and powerful traffic engineering methods can be applied in network design [1], [2]. Unfortunately, the more and more dominating connectionless Internet protocol (IP) does not provide mechanisms for traffic engineering like ATM does, and this imposes to think about a connection-oriented solution for future IP-based core networks. Multiprotocol label switching (MPLS), an upcoming technique

Manuscript received December 15, 2002; revised July 1, 2003 and November 10, 2003. This work was presented in part at the 18th International Teletraffic Congress (ITC-18), Berlin, Germany, September 2003. The authors are with the Institute of Communications and Navigation, German Aerospace Center (DLR), Weßling D-82234, Germany (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/JSAC.2004.823406

designed for Internet backbone networks, seems to be an interesting candidate to bridge this gap: It allows to adopt new paradigms for conventional IP traffic by decoupling packet forwarding from the information carried in the IP header. This is achieved by distributing routing information to the core routers of the network and assigning short fixed-length ATM-like labels to IP packets at the ingress point. MPLS has already been proposed for geostationary earth orbit (GEO) systems with multiple spot beams, mainly as a means to simplify this complexity of on-board switching [3]. However, particularly challenging technology issues arise in such an environment where huge traffic volumes have to be switched between several tens or hundreds of input/output ports serving the spot beams. In contrast to that, the situation is different for low earth orbit (LEO) satellites interconnected by ISLs: A potentially high traffic load has to be switched between a small number of ports, since the number of ISLs per satellite is strictly limited (typically 4 to 6). This makes constellation networks with ISLs more similar to terrestrial backbones and, thus, allows to focus on traffic engineering issues related to the dynamic of these networks. This paper is organized as follows. Section II first recalls relevant basics of MPLS and provides a brief introduction to dynamic satellite constellation networks. A general MPLS-based networking concept for such dynamic satellite constellations is then presented in Section III, and three implementation scenarios are discussed in more detail, focusing on their respective advantages and drawbacks. As routing performance turns out to be of key significance in the considered dynamic network topologies, a generic analytical estimation of (re)routing effort in this particular environment is deduced in Section IV, and some numerical and simulation results are presented under reference assumptions. A short summary and an outlook on future research work concludes the paper. II. FUNDAMENTALS OF MPLS AND SATELLITE CONSTELLATION NETWORKS A. MPLS Key Concepts The basic idea of MPLS is quite simple: At the ingress point of an MPLS network, the ingress label edge router (LER), packets are classified according to information carried in the packet (e.g., source/destination address, service class, etc.) or according to network related information (e.g., ingress port), or combinations of both. A group of packets treated in the same way is called forwarding equivalence class (FEC). It is also possible to bundle a set of FECs and use one single label for this union. This procedure is known as aggregation. Then, a unique locally significant fixed-length label is chosen for packets belonging to a certain FEC and attached to each

0733-8716/04$20.00 © 2004 IEEE

DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS

packet. Subsequent label switching routers (LSRs) examine the packets’ labels, replace them with already specified new labels, and forward the packets according to information stored in a table to the next LSR, until the egress point (egress LER) of the network is reached. The unidirectional path along which a packet traverses the MPLS domain is called label switched path (LSP). The forwarding procedure (forwarding plane) is completely decoupled from the MPLS control plane, which gives service providers a lot of possibilities to influence the network’s behavior. The control plane itself can be divided into two parts. One part, the label distribution protocol, is responsible for distributing labels to all LSRs along an LSP. Currently, two different protocols are defined by the Internet Engineering Task Force (IETF): 1) RSVP-TE [4] is the traffic engineering extension for MPLS to the resource reservation protocol (RSVP) [5] and 2) CR-LDP [6] is the constraint-based routing label distribution protocol, an extension to label distribution protocol (LDP) [7], which has been newly developed for MPLS. The main difference between these two label distribution protocols is the way in which they exchange control information within the MPLS domain. RSVP-TE uses a soft state approach with refresh messages, whereas CR-LDP follows a hard state approach with transmission control protocol (TCP) as underlying protocol. An examination of their suitability for a dynamic network topology would go beyond the scope of this paper and, furthermore, in February 2003, the IETF decided to discontinue work on CR-LDP [8]. It has to be noted that in a lot of publications label distribution protocols are called signaling protocols, but they do not have much in common with classical signaling protocols like the ones used in ISDN. Throughout this paper we will make no difference between the two label distribution protocols as both cover almost the same functionality. Although basic traffic engineering can be achieved by manually defining LSPs (explicit routing), one central goal in networks with fixed nodes is to automatically choose an adequate LSP through the MPLS domain in order to achieve maximal utilization of the network and to meet service class requirements like delay or guaranteed bandwidth. Therefore, the second part of the control plane consists of mechanisms which gather network state information and compute routes for LSPs. An example is OSPF-TE [9], the traffic engineering extension to open shortest-path first (OSPF) for use with MPLS. Later, in Section III-C, we will see that this approach contains several drawbacks if used in a satellite constellation and that precalculated explicit routes will be more suitable. MPLS has been defined to support various data link layers and network layers (cf. Fig. 1). ATM and frame relay can be directly used as the MPLS header is mapped onto the existing header elements; other data link layer protocols need a shim header to carry the MPLS label information. Integrated services (IntServ) and differentiated services (DiffServ) are the two philosophies in IP networks to support QoS requirements. MPLS has been developed to support both: IntServ uses RSVP for making end-to-end reservations, and the enhancement of RSVP for MPLS, namely RSVP-TE, can be directly used for setting up LSPs in the MPLS domain [4].

439

Fig. 1.

MPLS between data link layer and network layer [10].

In a DiffServ domain, all IP packets on a link requiring the same DiffServ behavior constitute a behavior aggregate (BA), which is represented by DiffServ code point (DSCP) attached to each packet. Mapping of DiffServ behavior aggregates (BAs) entering an MPLS domain onto LSPs can be easily defined; the interested reader may refer to [11] for further information. B. Basics of Dynamic Satellite Constellation Networks In developing an MPLS-based networking concept for satellite constellation networks, it is first necessary to recall their relevant features setting the framework and particular challenges; among these, the most obvious is the twofold dynamics of the network: first, variations within the ISL topology, and second, dynamics of the whole ISL network with respect to served ground users. Earlier studies on the routing in ISL networks of polar or near-polar star pattern [12] constellations (like Iridium [13]) have clearly identified the seam between counter-rotating orbits and the ON/OFF-switching of its interorbit ISLs as two fundamental drawbacks for the connection-oriented operation [14]. This has stimulated the investigation of moderately inclined delta pattern [12] constellations employing ISLs [15]. It was shown that such constellations generally provide the possibility to set up a number of interorbit ISLs that can be maintained permanently with acceptable pointing, acquisition and tracking (PAT) requirements. In particular, pointing investigations have shown that laser ISL terminals are capable of meeting the azimuth and elevation swivel requirements encountered in such constellations with acceptable precision at the necessary speed [16]. Link permanence is especially important for optical ISLs as a time-consuming acquisition of a new neighbor satellite or periodic reacquisition for a switched-off link can be completely avoided, at least under nominal operating conditions. Moreover, permanence of physical ISLs is in general a highly desirable feature in the light of real-time connection-oriented services, as path switching can be completely reduced to such cases where it is inevitable due to handover of the satellites serving the ground users of considered end-to-end connections. Celestri [17] was one of the first commercial system proposals aiming at the promising combination of a delta pattern constellation and optical ISLs. The Celestri constellation consists of 63 satellites evenly distributed in 7 orbit planes, as illustrated in Fig. 2. Table I lists all relevant constellation and ISL parameters at a glance. Using Celestri as an example, we now briefly summarize the major findings and results from a systematic ISL topology design process [2], referring to Fig. 3.

440

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

(a)

Fig. 2.

(b)

Celestri constellation [17].

TABLE I CELESTRI CONSTELLATION AND ISL PARAMETERS [16], [17]

(c)

(d)

Fig. 3(a) shows the generic candidate ISLs for a reference satellite in the constellation that may be implemented on principle, and Fig. 3(c) displays the corresponding link length (and thus delay) variation over one constellation period (roughly 114 min) for the candidate interorbit ISLs. Of these envisaged generic ISLs, the two candidates to the after next orbit, 0–26 and 0–25, are not feasible for the particular Celestri constellation parameters, due to earth shadowing as indicated by the dash-dot line representing the respective upper bound for allowed link length.1 As the curve for the pair 0–9 clearly reveals, also this generic ISL would not be permanently available for the same reason. Consequently, choosing only the generic interorbit ISL 0–17 to be implemented, we end up with four physical bidirectional ISLs for each satellite as shown in

1Earth shadowing is here defined as a shortest distance of 150 km between the (optical) link and the earth’s surface, taking into account optical free-space communication conditions.

Fig. 3. ISL topology illustration for the example system Celestri. (a) Generic candidate ISLs for a reference satellite. (b) Four established ISLs for one satellite to form a regularly meshed ISL network. (c) Link length variation of interorbit ISLs over one constellation period; the dash–dot line represents the bound with respect to earth shadowing. (d) Considered reference ISL topology based on the Celestri constellation: solid lines, intraorbit ISLs, dashed lines, and interorbit ISLs.

Fig. 3(b). A snapshot of the whole resulting ISL topology for this case is displayed in Fig. 3(d).2 Note that, due to the link permanence, the shown regular mesh topology does not change during the course of time; only the pointing angles and lengths of interorbit ISLs exhibit periodic changes with the orbit period. Thus, the interorbit link length is the only remaining parameter of potential relevance for routing options within the ISL subnetwork. However, the major challenge for any connection-oriented networking approach remains the inevitable relative movement between the satellite network and served ground users, which accounts for permanent forced satellite handovers.

2Note that the generic ISL topology derived in this way is not the original one proposed for Celestri, which foresees links to six neighbors.

DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS

C. Rerouting in Dynamic Topologies In satellite constellations and in every MPLS network with a changing topology, rerouting of existing traffic paths is one of the key issues. The ability to reroute paths and to build recovery paths does not depend on the signaling protocol, since both label distribution protocols already provide mechanisms for this purpose: • fast rerouting; • make-before-break. Fast rerouting is a technique developed for setting up backup paths for failure protection of links, nodes, or complete network segments. It is very expensive in terms of resource reservation and is normally only used in case of very strict reliability requirements. The idea behind make-before-break is to set up a new LSP in parallel to the still existing old LSP due to topology or traffic variations. The protocols avoid unnecessary double reservations of bandwidth on commonly used links, as usually the two links are used one after the other and not simultaneously. In case of an explicitly routed LSP, it is of course up to the ingress LER to set up the alternative LSP. Throughout this paper, the make-before-break option is assumed for rerouting in the satellite constellation. Independent of the particular mechanism, however, one key challenge with rerouting in a connection-oriented environment (with packet sequence typically being preserved) remains the proper alignment of the respective packet streams flowing on the old and new LSPs, respectively. Such an alignment is essential for nondisruptive operation of (mainly real-time) end user connections served via the LSP. Related solutions have been studied in detail for ATM networks. In [18], an alignment strategy for terrestrial ATM networks is proposed that requires duplication of the cell streams during the alignment phase and thereby guarantees nearly optimal and error-free operation. Similar approaches could certainly be used in an MPLS environment; this remains an issue for separate research and is not further considered in this paper. III. MPLS NETWORKING CONCEPT FOR NGSO SATELLITE CONSTELLATIONS In this section, we bring together the two worlds of 1) MPLS networking and 2) satellite constellations employing a dynamic ISL topology forming the space-based core network. We first define the appropriate boundaries of the satellite MPLS network, then move on to describing the overall networking concept discussing its main functionalities, and finally, present three possible implementation scenarios comparing their relative advantages and drawbacks. A major distinction between the scenarios is made up by the different locations in the network, where routing functionality (or intelligence) is physically established.

441

Section II-B, we only consider permanent topologies avoiding the most unhandy switched ISLs, there are still inherent and frequent handovers between ground users/stations and serving satellites which have to be taken into account. Let us consider such ground users or stations simply as MPLS routers from a networking perspective. The fact that regular and frequent handovers appear between network nodes (routers) is strongly related to the question where to set the boundaries of the MPLS network properly. As a first option, those satellites serving ground (i.e., due to the motion of the satellites effectively all satellites at least at a certain time) could be considered as LERs; this would restrict the MPLS backbone network completely to the ISL space network, which has a permanent topology and could thus, on principle, be operated without stringent LSP rerouting requirements. However, this solution has some serious drawbacks on the other hand. LERs, as already explained in Section II-A, constitute in a way the intelligence of the network, they manage the label distribution and, in some cases, perform route computations. So placing the LERs “in the sky” means notable on-board processing; we will see in more detail in Section IV how large this computational effort can be. Leaving the satellites as simple LSRs to avoid such on-board processing, we must place the LERs (and the network boundaries) on ground. This causes the earth-satellite link to fall inside the MPLS network. This first link becomes very critical for the operation of any LSP using it, since the link is involved in frequent handovers and this implies continuous rerouting decisions and computations for the LSP. It should be noted, however, that rerouting has to be performed in both cases—either if the LERs are placed on board or if they are placed on ground, respectively—as end-to-end traffic always flows from ground to ground, but there is a conceptual difference: In the former case of satellite LERs, the ground station may trigger a systematic handover from an old to a new serving satellite (LER) by asking to tear down the active LSP (from the old satellite) and sending a new LSP setup request to the new satellite; consequently, a new phase of LSP activation begins with its related QoS negotiation and its admission control. In the latter case, when the LERs are placed on ground, approaching a handover they can ask for a rerouting computation, but there is no QoS negotiation nor an admission decision: the LSP was already active and its attributes are already known, so, if possible, it has to be maintained with the same characteristics. In conclusion, there are no worthwhile advantages to implement LER functionalities on board. Rather two advantages of having LERs placed on ground are dominating: first, there is no need to restart a QoS negotiation or admission control for rerouting of an LSP due to satellite handover; second, expensive and complex on-board processing for advanced routing functionality is avoided.

A. Network Boundaries From the previous section, it has become clear that the particular challenge for employing MPLS networking in satellite networks is their inherent dynamics. Although, as motivated in

B. Overall Concept and Decomposition of Functionalities Developing a tailored MPLS networking concept for satellite constellations implies also to consider issues related to various

442

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

Fig. 4. Functional blocks and their interrelation for MPLS-based networking in dynamic satellite constellations.

protocols involved in MPLS. To this end it is helpful to separate and break down the functionalities starting from top level as illustrated in Fig. 4. All relevant MPLS functionalities in dynamic satellite networks can be summarized in four main functional blocks: 1) Network State: including in particular monitoring and state information distribution; 2) User Behavior/QoS Requirements: making up the demand; 3) Route Computation: as the key block comprising “intelligence” in terms of traffic engineering, adaptiveness, and/or optimization; 4) LSP Rerouting/Creation/Release: establishing the result of “route computation” in the network. The distribution of network state information is usually performed through link state routing protocols such as OSPF or intermediate system to intermediate system (IS-IS), that can spread various link attributes. In the ISL network scenario, relevant state information comprises link availability, capacity, available bandwidth after resource allocation, and delay (if not calculated on demand, or precalculated and kept in memory, as the topology is periodically deterministic). This information must be provided to all nodes that perform route computation. Another important information to be spread is the state of active LSPs to perform proper rerouting in case a traversed link fails or when a handover has to be performed because of the regular motion of the constellation. “Route computation” deals with QoS negotiation, admission control, and traffic engineering issues in reaction to incoming

LSP requests, constituting quite a complex set of procedures. An LSP request is formulated including particular constraints that have to be respected by the computed path. This module evaluates all incoming requests, checks whether the satellite network is able to host the requested LSPs, and gives the authorization to set up the paths. For the scope of this work, we can imagine that the whole satellite network belongs to only one service provider, which can manage this QoS negotiation as preferred. So we can assume, for example, that requests are formulated in terms of source and destination LERs, maximum tolerated delay, and minimum required bandwidth along the path. Once an LSP request is accepted, its negotiated QoS attributes are typically passed to a constraint-based routing module, which takes care of calculating the path accordingly. When a path is released, there is no need for any computation, but this information is simply spread and the relative labels are set free. Constraint-based routing is usually performed as optimization with respect to a scalar metric, and may be implemented using one of several proven algorithms where the choice depends on the particular set of optimization metric and constraints. In a traffic adaptive scheme, routes are typically chosen in such a way that traffic load is evenly distributed in the satellite network, LSPs are routed around network link failures and bottlenecks, and congestion is avoided as far as possible. There are several algorithms that perform such computations, paying attention to different aspects reflected in respective metrics. Some algorithms try to find always the shortest path for each LSP (like the commonly used Dijkstra shortest-path algorithm) in terms of either number of hops or generalized link cost. Other algorithms try to maximize the probability to accept future routing demands; one candidate of the latter, minimum interference routing algorithm (MIRA) [19], has already been tested for MPLS. Finally, earlier work on ATM-based satellite constellation networks has focused on algorithms that try to exploit the periodicity of the ISL topology in order to minimize the number of path reroutings within the ISL subnetwork [14], [20]. An important aspect that should be discussed in this context is the frequency of route recomputations (see Section IV). One option is that a route is only calculated once after a setup request and each LSP would then stick to this route until a handover between ground and serving satellite causes a recomputation. Alternatively, routes could be recomputed periodically, which may prove useful in the considered dynamic scenario with continuously changing network state. Better routes may become available for already existing paths, resulting in a network performance gain. However, such an approach would also imply to reroute paths not only because of satellite handover or link failures, but also to improve utilization or adapt to priorities. So any QoS or performance gain must be thoroughly traded off versus an increase in both computational effort and signaling load due to label distribution. In particular, it should be evaluated whether computationally heavy periodic routing updates are worthwhile, and whether optimizing already existing LSPs does not cause too many rerouting procedures on the other hand. In the remainder of this section, we now focus on the question where routing functionality should be physically implemented

DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS

Fig. 5.

MPLS networking concept: candidate implementation scenarios.

in the network, presenting three respective candidate scenarios for implementation. Fig. 5 provides an illustrative and simplified view on their key characteristics at a glance.

443

In an IP network with fixed nodes and no traffic engineering requirements, OSPF version 2 [21] is used as protocol to distribute network information and to synchronize the link state advertisements (LSAs) in the LSDBs of the nodes. This basic OSPF version judges links between nodes according to a single dimensionless metric and each router constructs a tree of shortest paths with itself as root according to this metric. LSDBs are updated periodically and due to changes in the network topology, but it is assumed that new network information is distributed very quickly in order to avoid routing errors or loops. It is clear that this solution is not suitable for MPLS traffic engineering and, therefore, enhancements are under study in the IETF. OSPF-TE uses opaque LSAs [22] and distributes much more information than standard OSPF can do. Besides a traffic engineering metric the routers know also about maximum (reservable/unreserved) bandwidth and can include these constraints in their path computation. In order to avoid an excessive flooding of the network with LSAs, a link attribute (e.g., bandwidth, which can vary quickly) may have to cross a certain threshold before an update is sent, but all other link attributes are distributed as well. Ongoing further developments try to avoid this unnecessary traffic by distributing only incremental updates of single attributes without sending the complete LSA, but again the whole network has to be flooded. In spite of the improvements of OSPF for use with MPLS, a direct adaption of the terrestrial scenario to the satellite network is abundantly questionable. The nodes are not fixed and there is a continuously changing delay of the interorbit links which has to be monitored and announced to all LSDBs. Moreover, the synchronization of the LSDBs suffers from this changing interorbit satellite distance and the high signal propagation delay caused by the earth enclosing size of the topology, too. It would be possible to avoid OSPF traffic related to a change in link delay by precalculation in every node as the constellation moves in a deterministic manner, but for reasonable traffic engineering it is still necessary to distribute link state information. The second drawback of this scenario is its inability to sensibly modify existing traffic flows. Although MPLS offers the possibility that traffic with high priority may suppress traffic with less priority, this behavior is not desirable as the ingress node of the traffic with low priority has to be notified about this “link interruption” via a link failure message or an update of its LSDB and it has to set up a complete new path through the network on less utilized links again.

C. Scenario 1: Distributed Routing and LSP Management

D. Scenario 2: Centralized Routing—Distributed LSP Management

A direct adaption of the terrestrial situation to the satellite network is shown in Fig. 5(a). All LERs on ground and the LSRs on-board the satellites maintain identical link state databases (LSDBs) representing the entire network’s state, and routes are computed at the ingress points of the network. It is up to the ingress LER on ground to initiate a setup of and switch over to an alternative LSP according to the information stored in its LSDB and, therefore, it is crucial to receive continuous updates for the LSDB.

A more efficient traffic engineering can be performed only with a global view of the network. For this, it is necessary that each ingress LER trying to access the network sends the traffic parameters (traffic category, bandwidth, etc.) to a central database, which has total knowledge of the network state and computes an optimum route for the incoming request. Then, the central database answers with a message containing the explicit route (ER) for an LSP which the ingress LER has to create itself. In parallel, the central LSDB may send new ERs for existing

444

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

LSPs to other ingress LERs in order to trigger handover events or to reroute traffic flows for a better network utilization. This approach [see Fig. 5(b)] has two major advantages: first, satellites with their typical constraints in computing power and memory do not have to maintain LSDBs and perform route calculations, and second, the network utilization is increased drastically [23]. The drawbacks are the necessity of a new protocol exchanging information between ingress LER and central LSDB and a more or less accurate knowledge of the ground stations’ coordinates. In scenario 1, the ground stations only need information about visibility and distance of satellites for determining alternative LSPs and the time to switch to the new path. Scenario 2, however, offers two possibilities regarding the time of rerouting: either the central LSDB offers one or several alternatives for ERs and the decision when to start the rerouting is completely up to the ground station (like in scenario 1), which then of course has to inform the central LSDB immediately about any action, or the ground station has to take the new route directly after reception of the ER from the central LSDB. The former option does not require detailed position information and is suitable for satellite constellations with several visible satellites at the same time (satellite diversity), out of which the best one is chosen, and the latter option is more appropriate for a small number of visible satellites, but then the LSDB needs very accurate information about the ground stations’ locations to avoid routing errors.

E. Scenario 3: Centralized Routing and LSP Management Further development of the centrally triggered LSP rerouting leads to the absolute control scenario shown in Fig. 5(c). Here, the ingress points of the network do not set up LSPs any more; all nodes of the network get their tables for label swapping directly from the central database via logical links. Any decision about traffic engineering driven rerouting or handover events is up to the LSDB, but this approach has only little remaining commonalities with terrestrial use of MPLS, including for instance the label swapping mechanism. One advantage of this approach is the faster installation of LSPs. The central LSDB distributes label swapping tables among the satellites directly after the reception of a setup request from one of the LERs, and of course due to handover or rerouting events. Then, it sends an acknowledgment back to the origin of the connection request and this may immediately start to use the already existing LSP without having to set up one itself. In case of handover or rerouting events, the procedure has to be carefully considered with respect to synchronization requirements: all updated tables must start to be used at the same time. A possible solution is to update all tables along the new LSP with the ingress LER being the last node updated. A major drawback of this scenario is the design of a new signaling protocol to distribute labels among the LSRs. Both centralized scenarios (2 and 3) present scalability problems with respect to the number of LSPs: the central station may only be able to handle up to a limited number of them. This is a

crucial point and is, therefore, (implicitly) addressed in the following when we estimate the routing/rerouting effort required to maintain a certain number of LSPs. IV. ESTIMATION OF ROUTE COMPUTATION EFFORT The previous section has clearly indicated the crucial significance of (re)routing performance in the considered dynamic scenario. Consequently, the effort which is spent in route computations for both LSP setup and rerouting is a key performance parameter for the operation of a satellite MPLS network. Route computation effort may be measured in several ways and at different levels (e.g., related amount of signaling, calls of the basic routing algorithm, elementary processor operations, etc.). Here, we are interested in a simple, yet meaningful, general metric. For the scope of this paper, this is the sheer number of LSP setup and rerouting occurrences, and more specifically, just its (long-term) time average value for the whole network. This selection is mainly motivated by two facts: first, for any more sophisticated metric this number would certainly remain an important multiplicative factor; second, one single average value is a quick and intuitive way to perform qualitative case studies—as needed at this initial stage of our research—and to “compare at a glance” complete global traffic scenarios and/or different constellations. In the following, we first develop a generic analytical estimation approach. Subsequently, a simple numerical example is discussed to derive qualitative, as well as first quantitative conclusions that are indicative for the ongoing research work. Finally, a simulation framework for discrete-event simulation and performance evaluation of the MPLS networking and LSP (re)routing scenario is presented, and simulation results are compared with the analytically derived figures. A. Analytical Model Before setting up and running extensive simulations of the satellite MPLS network, we are keen on developing an analytical model that allows to estimate the route computation effort under some simplifying assumptions, yet capturing the “live’” network conditions in a sufficiently realistic way. A well-proven approach for such cases has often been a queueing system modeling, where the key model parameters can be adapted to any numerical real-world situation in a tractable way. As concluded in Section III-A, we assume that LERs are located on ground and that all satellites exclusively act as LSRs. Taking a queueing modeling approach, we further assume that each ground LER receives requests to create LSPs according to a Poisson arrival process with a mean arrival rate . This means equivalently that the system has the property to be memoryless and that the interarrival times of the requests are negative exponentially distributed [24]. In such a system, if a ground LER remains inside the footprint of one particular satellite (see , the probability that in this time interval Fig. 6) for a time LSP requests are generated from the LER is (1) It has now to be considered that inside one satellite footprint there can be several ground LERs at given instants of time. Fig. 7

DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS

445

, establishing equiin the considered queueing system is librium operation. In our case, we can, thus, state the average number of active LSPs in one satellite as (2)

Fig. 6.

LERs inside the footprints of moving satellites.

This is an important first result for our satellite network model. So far, we have not taken into account that in the considered scenario satellites typically serve ground LERs at all only during a certain share of time , namely when they are over populated areas, whereas they are “idle” over deserted regions and oceans. Or equivalently, the average number of satellites serving ground stations is (3) Now, in the whole constellation there are unordered origin-destination (OD) satellite pairs. As we are looking at unidirectional LSPs established between pairs of ordered OD satellites, we would start to consider pairs with potentially established LSPs. Following our earlier terminology and considerations, the average number of ordered and, thus, “active” OD pairs would amount to . In other words, in the average, each of the active satellite shares its active paths with partner satellites. Thus, the average number of active LSPs for a given OD pair is (4)

Fig. 7.

Satellite footprint moves across several LERs on ground.

illustrates the simplified geometry of an example situation and the respective time-varying number of LERs served by the considered satellite. Each of the LERs would in reality set up and would obviously tear down its own LSPs, and in particular be strictly related to the concrete position of the LER with respect to the edge of the footprint and direction of satellite movement, so in essence a deterministic value in each single case. However, with our stochastic modeling approach, we are aiming at an average performance value over time and over the whole network anyway. For the sake of notation simplicity, we further consider as the effective average arrival rate and as the effective average remaining time of the LSPs served by one satellite, respectively, no matter if they actually originate from one or more LERs. One could alternatively assume that all LSPs in a satellite always come from one fictive ground LER only. We now assume that the duration of an LSP itself (until it expires, not affected by handover/rerouting) is negative expo. nentially distributed with an average LSP duration of Under the further assumption that all LSP requests can be accepted, the above characteristics hold for active or established LSPs in the satellite MPLS network. Thus, our system model is queueing system: the service time (LSP duration) an does not depend on the number of users (active LSPs) inside the system. The rate at which LSPs are torn down in one satellite, on the contrary, directly depends on the number of active LSPs, and it can be shown [24] that the average number of users

being the average handover rate—for both LERs With and LSPs according to our earlier assumptions—and assuming completely uncorrelated handover occurrence in the corresponding source and destination satellites, the handover rate for one single LSP is simply the sum of the handover rates of its source and destination satellites (5) and with (4) the LSP handover rate for each OD pair becomes (6) Multiplication of this value with the average number of ordered “active” OD pairs gives us the average LSP handover rate for the whole network

(7) Finally, the transition back to route computation effort in the network is straightforward due to the chosen simple metric for the latter, namely, average number of (re)routing occurrences per time unit. The rerouting effort per time unit is simply identical with the average LSP handover rate in the network as given in (7), and the routing effort related to setup of new

446

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

LSPs per time unit, is just the product of their arrival rate . per satellite with the average number of active satellites Thus, the total network route computation effort becomes

TABLE II NETWORK ROUTE COMPUTATION EFFORT R FOR SELECTED COMBINATIONS OF  AND . THE NUMBER OF ACTIVE LSPS PER SATELLITE = = = 600 IS CONSTANT P

(8) B. Qualitative and Numerical Discussion and as fixed values reIn the following, we assume flecting the dynamic constellation considered. and are the increases with both two parameters of interest. First of all, and , i.e., routing effort increases when LSP requests come more often and when requested LSP durations get longer. More specifically, the routing effort related to setup of new , of course, depends only on their arrival rate , LSPs, , on the other hand, depends on whereas the rerouting effort the average number of active LSPs , because they remain in the constellation during handovers. Let us also state the following two indicative results from (8), concerning the relative contributions from LSP setup and LSP rerouting: • the rerouting effort is smaller than the setup effort if the average LSP duration remains below half the average time till handover (9) • the rerouting effort becomes completely dominating (and the setup effort negligible) if the average LSP duration is much larger than half the average time till handover (10) Now, consider that realistic values of in satellite constellations are in the order of tens of minutes, whereas LSPs in MPLS networks certainly last much longer if applied in typical fashion. Thus, in satellite constellation networks employing MPLS, the rerouting effort can be expected to be dominating—unlike in terrestrial MPLS networks. This fact raises the importance of research into highly efficient LSP rerouting mechanisms for the satellite scenario. Regarding the scalability problem of a centralized approach, from (10), one can see that the rerouting effort increases linearly with the number of active LSPs, and a similar behavior can be expected for the amount of signaling. Let us finally adopt some reference values to perform a small numerical case study. We fix the constellation related values and , and let and vary as free parameters. For a given constellation, we want to see what happens if incoming traffic changes its characteristics, namely the average rate of incoming . LSP requests , and the average requested LSP duration We set the following: • (for the Celestri example constellation); (roughly reflecting a reasonable share of land• masses with respect to the earth’s surface). The parameter can be estimated in the following way. The duration in which a ground station is passed by a satellite foot, which is 16 print is determined by the minimal elevation for the Celestri example, the orbit altitude , and of course the

length of the track of the ground station in the satellite footprint (a small circle on the sphere), which can range from the footprint’s diameter to very small values next to the edge of the footprint. Using the (one-sided) earth central angle , which is determined by (11) with the mean equatorial earth radius km, the mean duration a ground station stays inside a satellite footprint is given by (12) with the orbit period . Assuming that the terminal on ground always chooses the nearest satellite (i.e., the satellite with maximal elevation) and uses it until it falls below the horizon (i.e., ), the mean time a ground station remains attached below . A nuto the same satellite, can be estimated to merical integration of (12) with the Celestri parameters gives s. The numerical results are summarized in Table II and in Fig. 8. They confirm the above qualitative considerations. In particular, Fig. 8(b) illustrates how the route computation effort asymptotically approaches the value set in (10) when rerouting becomes dominating. C. Simulation Approach and Results In order to validate the presented results, a simulation environment has been set up based on the network simulator ns-2.3 Values for the Celestri constellation were chosen according to Table I, and eight ground stations were placed randomly next to big cities all over the world. LSP requests were generated at was different Poisson rates and the average LSP duration set accordingly in order to achieve a mean total number of ac. tive LSPs in the network equal to In the simulation environment LSP requests are not generated in every ground station, nor in every satellite, but in the overall system. Thus, we have that in the entire constellation LSPs in our per second are generated, and so corresponds to theoretical discussion. This choice avoids the estimation of which is a very system-dependent value and which has no big importance for our scope. Thus, using (8) and , we obtain for the comparison with our simulation (13) Each ground station chooses its serving satellite according to the description in Section IV-B, i.e., it initially uses the satellite 3http://www.isi.edu/nsnam

DONNER et al.: MPLS-BASED SATELLITE CONSTELLATION NETWORKS

447

(a) Fig. 9. Comparison between theoretical and simulation results.

(b) Fig. 8. Route computation effort (R ). The circles denote the values in the first three columns of Table II. (a) “Full” range, up to large numbers of active LSPs P . (b) Focus on the considered numerical example of P = 600.

with maximal elevation. The route in the space segment between origin and destination satellite is calculated with a Dijkstra algorithm, and this route is only recalculated if one of the two serving satellites falls below the minimal elevation and a handover has to be performed. The simulated results in Fig. 9 follow the theoretical expectations very well. It is not surprising that the simulated generation rate of new LSP setups coincides with the prediction, since a plain Poisson process with no particular modifications has been used. On the other hand, the simulated rate of rerouting events shows especially for short LSP durations discrepancies to the expected behavior. The reason is that short-term LSPs have a lower probability to be rerouted since their lifetime is more likely to fall inside an interval with no handovers, which reduces , and this was not considered drastically the rerouting rate in the analytical model. For larger values of the LSP mean holding time the simulated samples cross the prediction (the LSP lifetime does not fit any more into intervals with no handovers) and remain slightly above the theoretical result. As already stated in Section II-B, the advantage of inclined Walker constellations is the permanent

network topology in the space segment, but this does not equivalently mean that each ground station has a similar view of the serving satellites. Users in the very northern or southern hemisphere (latitude greater than the inclination of the satellites) will always have a shorter mean visibility of their serving satellites than a user near to the equator, who can experience a visibility time up to the equivalent to half a footprint diameter. In other words, the mean duration a ground station near the poles stays inside a satellite footprint is smaller than for a terminal near to the equator, and this causes a higher handover rate. In our simulation, we did not consider terminals in the Antarctic or in the region around the north pole (edge of constellation coverage) does not show a considerable deand, thus, the simulated viation from the theoretical result. V. CONCLUSION In this paper, we have developed an MPLS networking concept for use with NGSO satellite constellations. Offering rerouting capabilities, MPLS is well suited for use in dynamic topologies. We have also shown that a global view of the network in terms of a central LSDB with central route calculation is necessary for operating the network with guaranteed QoS requirements, which benefits a lot from the periodicity and regularity of Walker delta constellations. Scenarios 2 and 3 covered this topic in many details, but did not discuss the problem of signaling traffic from this single control station to the network. Other implications like signaling delay and the performance of the label distribution protocols in the satellite environment have to be addressed in future work, as well as solutions like optimization of the control station’s location or the use of multiple synchronized control stations. As routing performance is particularly crucial in the considered dynamic network topology, a generic analytical estimation of route computation effort has been developed. The approach allows to derive qualitative, as well as initial quantitative conclusions that are indicative for the ongoing research work. It has been demonstrated as well that the LSP rerouting effort tends to be dominating over the LSP setup effort for long LSP lifetimes

448

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 22, NO. 3, APRIL 2004

and that the simple analytical estimation coincides with our simulation results very well. Future research work will therefore focus on efficiency and optimization of rerouting algorithms and performance. Other possible research directions are the investigation of (re)routing algorithms for multicast, traffic load balancing in the constellation and for a stateless QoS support over MPLS (such as DiffServ over MPLS). REFERENCES [1] E. Lutz, M. Werner, and A. Jahn, Satellite Systems for Personal and Broadband Communications, 1st ed. New York: Springer-Verlag, 2000. [2] M. Werner, J. Frings, F. Wauquiez, and G. Maral, “Topological design, routing and capacity dimensioning for ISL networks in broadband LEO satellite systems,” Int. J. Sat. Commun., vol. 19, no. 6, pp. 499–527, Nov./Dec. 2001. [3] T. Ors and C. Rosenberg, “Providing IP QoS over GEO satellite systems using MPLS,” Int. J. Sat. Commun., vol. 19, no. 7, pp. 443–461, Sept./Oct. 2001. [4] D. Awduche, M. Networks, L. Berger, T. L. D. Gan, V. Srinivasan, and G. Swallow, “RSVP-TE: Extensions to RSVP for LSP tunnels,” IETF, RFC 3209, Dec. 2001. [5] R. Braden, L. Zhang, S. Berson, S. Herzog, and S. Jamin, “Resource reservation protocol (RSVP)—Version 1, Functional Specification,” IETF, RFC 2205, Sept. 1997. [6] B. Jamoussi, L. Andersson, R. Callon, R. Dantu, L. Wu, P. Doolan, T. Worster, N. Feldman, A. Fredette, M. Girish, E. Gray, J. Heinanen, T. Kilty, and A. Malis, “Constraint-based LSP setup using LDP,” IETF, RFC 3212, Jan. 2002. [7] L. Andersson, P. Doolan, N. Feldman, A. Fredette, and B. Thomas, “LDP specification,” IETF, RFC 3036, Jan. 2001. [8] L. Andersson and G. Swallow, “The multiprotocol label switching (MPLS) working group decision on MPLS signaling protocols,” IETF, RFC 3468, Feb. 2003. [9] D. Katz, D. Yeung, and K. Kompella, “Traffic engineering extensions to OSPF, version 2,” Internet Draft draft-katz-yeung-ospf-traffic-09.txt, work in progress, Oct. 2002. [10] B. S. Davie and Y. Rekhter, MPLS: Technology and Applications. New York: Academic, 2000. [11] F. Le Faucheur, L. Wu, B. Davie, S. Davari, P. Vaananen, R. Krishnan, P. Cheval, and J. Heinanen, “Multiprotocol label switching (MPLS) support of differentiated services,” IETF, RFC 3270, May 2002. [12] J. G. Walker, “Circular orbit patterns providing continuous whole earth coverage,” Royal Aircraft Establishment, U.K., Tech. Rep. 70 211 (UDC 629.195:521.6), Nov. 1970. [13] J. Hutcheson and M. Laurin, “Network flexibility of the IRIDIUM global mobile satellite system,” in Proc. 4th Int. Mobile Satellite Conf. (IMSC’95), Ottawa, ON, Canada, June 1995, pp. 503–507. [14] M. Werner, C. Delucchi, H.-J. Vögel, G. Maral, and J.-J. De Ridder, “ATM-based routing in LEO/MEO satellite networks with intersatellite links,” IEEE J. Select. Areas Commun., vol. 15, pp. 69–82, Jan. 1997. [15] M. Werner, A. Jahn, E. Lutz, and A. Böttcher, “Analysis of system parameters for LEO/ICO-satellite communication networks,” IEEE J. Select. Areas Commun., vol. 13, pp. 371–381, Feb. 1995. [16] T. Dreischer, H. Kellermeier, E. Fischer, and B. Wandernoth, “Advanced miniature optical terminal family for inter-satellite links in space communication networks,” in Proc. 4th European Conf. Satellite Communications (ECSC 4), Rome, Italy, Nov. 1997, pp. 73–78. [17] Motorola Global Communications, Inc., “Application for authority to construct, launch and operate the Celestri multimedia LEO system,” FCC filing, Washington, D.C., June 1997. [18] B. Edmaier, J. Eberspächer, W. Fischer, and A. Klug, “Alignment server for hitless path-switching in ATM networks,” in Proc. XV Int. Switching Symp. (ISS’95), Berlin, Germany, Apr. 1995, pp. 403–407. [19] K. Kar, M. Kodialam, and T. V. Lakshman, “Minimum interference routing of bandwidth guaranteed tunnels with MPLS traffic engineering applications,” IEEE J. Select. Areas Commun., vol. 18, pp. 2566–2579, Dec. 2000. [20] M. Werner, “A dynamic routing concept for ATM-based satellite personal communication networks,” IEEE J. Select. Areas Commun., vol. 15, no. 8, pp. 1636–1648, Oct. 1997.

[21] J. Moy, “OSPF Version 2,” IETF, RFC 2328, Apr. 1998. [22] R. Culton, “The OSPF Opaque LSA Option,” IETF, RFC 2370, Apr. 1998. [23] G. Haßlinger and S. Schnitter, “How service providers can profit from traffic engineering,” in Proc. EURESCOM Summit 2002, Heidelberg, Germany, Oct. 2002. [24] L. Kleinrock, Queueing Systems. New York: Wiley, 1976.

Anton Donner received the Dipl.-Ing. degree in electrical engineering from Munich Technical University, Munich, Germany, in 1999. From 1999 to 2000, he has been working as a Research Assistant at the Institute of Communications Engineering, Munich Technical University on adaptive channel coding systems for progressively source coded data. Since 2000, he has been with the German Aerospace Center (DLR), Oberpfaffenhofen, Germany, in the Digital Networks Group of the Institute of Communications and Navigation. His main research activities are satellite-based reliable multicast solutions, and networking in nongeostationary satellite systems, with focus on signaling and routing issues in dynamic intersatellite link networks.

Matteo Berioli (S’03) received the Laurea degree in electronic engineering (magna cum laude), from the University of Perugia, Perugia, Italy, in 2001, where he is currently working toward the Ph.D. degree in information and electronic engineering. He has worked for the Italian Army as Second Lieutenant in the telecommunication field. Since 2002, he has been with the German Aerospace Center (DLR), Oberpfaffenhofen, Germany, in the Digital Networks Group of the Institute of Communications and Navigation. His main research activities include traffic engineering and routing on IP-based dynamic networks, with a focus on MPLS and nongeostationary satellite systems. He also produced some work on IP mobility and security. He is currently involved in two IST projects: FIFTH and Wireless Cabin.

Markus Werner (M’92–SM’01) received the Dipl.-Ing. degree from Darmstadt Technical University, Darmstadt, Germany, in 1991, and the Ph.D. degree from Munich Technical University, Munich, Germany, in 2002, both in electrical engineering. Since 1991, he has been with the Institute of Communications and Navigation, German Aerospace Center (DLR), Oberpfaffenhofen, Germany, as Research Scientist, Project Manager, and Group Leader. Since 2002, he is also Managing Director of TriaGnoSys GmbH, Wessling, Germany, a consulting company for satellite and aeronautical communications. His project experience includes several national and ESA studies and various projects in the framework of European ACTS, IST, and COST research programs. His main research activities are in the areas of multiservice traffic engineering and capacity dimensioning for satellite systems in general, and networking of nongeostationary satellite systems in particular, with a focus on routing issues in dynamic intersatellite link networks. He lectured on mobile satellite communications at Ilmenau Technical University, Ilmenau, Germany, from 1995 to 1996, and on satellite system design in the M.Sc. Research Seminar Series at University of Bradford, Bradford, U.K., in 2002. He is also a permanent Lecturer at the Carl-Cranz-Gesellschaft (CCG), Oberpfaffenhofen, Germany, teaching satellite communications courses for telecommunications professionals. He is coauthor of Satellite Systems for Personal and Broadband Communications (Berlin, Germany: Springer-Verlag, 2000). Dr. Werner is a member of VDE/ITG.