scheme, TE-QOSPF, exploits the use of non-shortest paths to improve load-balancing and avoid network congestion. Simulation results show that the algorithm ...
SUPERHIGHWAYS TECHNOLOGY AND BROADBAND VPN
Traffic engineering enhancement to QoS-OSPF in DiffServ and MPLS networks S.H. Lim, M.H. Yaacob, K.K. Phang and T.C. Ling Abstract: In the paper, a traffic engineering enhancement to the QoS-OSPF routing protocol is proposed and used as the path selection algorithm in a DiffServ-MPLS network. The proposed scheme, TE-QOSPF, exploits the use of non-shortest paths to improve load-balancing and avoid network congestion. Simulation results show that the algorithm outperforms the QoS-OSPF scheme in terms of loss ratio, link utilisation, and delay.
1
Introduction
The rapid growth of the Internet has given rise to a wide range of applications (e.g. real-time multimedia applications) that demand a certain level of quality of service (QoS) from the network. The Differentiated Services (DiffServ) model [1] was designed to provide the building blocks for a scalable IP QoS solution. It does not use permicroflow states within the network core as in the Integrated Services (IntServ) model [2]. This is achieved by moving complex processing such as traffic classification and conditioning to the edge of the network. Within a DiffServ domain, different per-hop behaviours (PHBs) are used to provide service differentiation among traffic classes. However, resource reservation and the establishment of QoS paths for DiffServ traffic, especially when dynamic service level agreement (SLA) is used, require additional mechanisms such as a bandwidth broker and a signalling protocol [3]. The multiprotocol label switching (MPLS) architecture [4] provides a distributed way for resource reservation and QoS path setup through the use of a label distribution protocol, such as the constraint-based routing label distribution protocol (CR-LDP) [5]. With the capability to setup explicitly routed label switched paths (LSPs), MPLS supports traffic engineering and QoS routing [6]. Since MPLS effectively complements the DiffServ architecture, there have been efforts [7, 8, 9] to standardise the integration of DiffServ and MPLS. Within a DiffServ-MPLS domain, the LSPs may represent aggregated traffic belonging to a certain class of service. The setting up of these LSPs involves routing path selection, where a QoS-aware routing protocol is needed for the dissemination of QoS-related information and the actual QoS path selection [10]. The open shortest path first (OSPF) interior gateway protocol [11] is one of the possible candidates for this purpose. While the original OSPF does not support QoS routing, QoS extensions to OSPF have been proposed and evaluated [12, 13]. r IEE, 2004 IEE Proceedings online no. 20040335 doi:10.1049/ip-com:20040335 Paper first received 24th January and in revised form 16th June 2003 The authors are with the Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, Malaysia IEE Proc.-Commun., Vol. 151, No. 1, February 2004
The proposed routing scheme in [12] (QoS-OSPF) involves the advertising of QoS parameters such as the residual bandwidth and the delay of each network link. The path selection is based on a ‘widest-shortest path’ algorithm. The algorithm first calculates all minimum-hop paths using Dijkstra’s algorithm [14], then select the one with the highest bottleneck bandwidth. The weakness in this scheme is that it uses minimum-hop (shortest) paths as the first selection criterion. This is not always desirable from a traffic engineering point of view, which promotes efficient resource utilisation and load balancing [6]. We propose an enhancement to the QoS-OSPF scheme that allows the use of non-shortest paths under some explicit constraints. The major contribution of this work is to empirically show that this simple traffic engineering extension, which we name TE-QOSPF, can lead to improvement in packet loss ratio, link utilisation, and packet delay. A review of some previous work on QoS routing and traffic engineering is presented. The proposed algorithm is discussed in detail, and the simulation results are presented and analysed. 2
Previous work
Much work has been done regarding QoS routing and traffic engineering in general. The issues of QoS routing and various QoS routing schemes including source routing, distributed routing and hierarchical routing are discussed in [15]. Reference [16] presents a systematic evaluation of four routing algorithms that offer different tradeoffs between limiting the path hop count and balancing the network load, namely widest-shortest path, shortest-widest path, shortest-distance path and dynamicalternative path. [17] introduces mechanisms for pre-computing multiple equal-cost routes to each destination with on-demand route extraction and checking of the suitability of these routes. QoS routing with multiple constraints is generally difficult, where finding a feasible path with two or more independent path constraints (e.g. delay, hop-count) is NP-complete. [19] shows that many of the traffic engineering problems, including resource management and constraint-based routing are intractable. The solutions to these problems often involve heuristics and approximations [20–23]. A hierarchical multilayer QoS routing system in a DiffServ-MPLS network is proposed in [24]. Reference [25] 101
proposed a traffic engineering scheme based on the minimum interference routing algorithm for dynamic setup of guaranteed-bandwidth LSPs. An on-demand LSP setup scheme based on re-routing of existing LSPs is proposed in [26]. 3
Proposed TE-QoSPF algorithm
While many proposed routing algorithm provides solutions for QoS routing and traffic engineering problems, they often involve complex calculation which require significant processing overhead. The mechanisms introduced in [17] only consider minimum-hop paths. The dynamic alternative path evaluated in [16], which is based on dynamic alternative routing [18], is close to our algorithm, but it does not impose constraints for choosing non-shortest paths. Our proposed scheme allows more flexible criteria and constraints on path length while retaining the simplicity of Dijkstra’s algorithm, and does not require any path precomputations. The proposed scheme is based on the assumption that the primary QoS parameter is bandwidth (as in [12]). If a QoS path has the necessary bandwidth for a particular traffic trunk so that packets are never queued, other QoS parameters such as delay and delay variation are generally minimised. However, a delay parameter can still be used to prune high-delay links prior to the execution of the actual path computation. The proposed algorithm alters the path selection criteria used in [12]. Instead of always choosing the minimum-hop paths, the algorithm allows longer paths to be chosen if the path has significantly higher residual bandwidth. The problem is then in determining the actual meaning of ‘significantly higher residual bandwidth’. It is suggested that when a candidate path is one hop longer than another candidate path, its available bandwidth must be twice that of the shorter path. When the hop-count difference is two, the longer path must have three times available bandwidth than that of the shorter path, and so on. In order to limit the use of very long paths, a hop-count difference threshold, c, is defined so that if two paths have a hop-count difference greater than c, the shorter path is always chosen. In other words, let v1 and v2 be two possible candidates for the next node in the shortest path, and the distances (from the source node) of v1 and v2 are d1 and d 2 respectively. The bottleneck bandwidths of the path to v1 and v2 are bw1 and bw2 respectively. If 7d1d 27rc, then the bandwidth ratio, r is calculated as r ¼ bw1=bw2. When d14d 2, r must be greater than a value k for v1 to be chosen. Similarly, when d 24d1, 1/r must be greater than k in order for v2 to be chosen. When d1 ¼ d 2. v1 is chosen if rZ1 and v2 is chosen if ro1. If 7d1d 274c, then the normal shortest path rule is used (whichever has a shorter distance is chosen). The choice of the value k is non-trivial because if it is too large, a longer path has too little chance to be chosen. If it is too small, a slightly wider path with a longer distance might be easily chosen, which may result in inefficient resource utilisation. The optimal value of k depends on many factors, including the current network load pattern, the network topology, and the duration of existing LSPs. The proposed value for k as discussed previously is k ¼ 7d1d 27+1. A function findBetterNode is defined with the inputs v1, v2, d1, d 2, bw1, and bw2 where the output is the better node between the two (v1 and v2) according to the selection criteria as described above. This function will be used in the path selection algorithm, which is basically a modified version of Dijkstra’s algorithm [14] described in [12]. The 102
algorithm represents the network in question as a directed graph where each node is a vertex while each link is an edge. It then finds the best path according to the described selection criteria. The algorithm is listed as pseudocode as follows: Inputs V: set of vertices, labelled 1 to N L: set of edges, labelled as ordered pairs (n, m) of vertex labels For all edges (n, m) in L, b(n, m): available bandwidth s: source vertex t: destination vertex B: requested bandwidth of the QoS path (optional) Type Definition tab_entry: record { hops (integer), bw (integer), parent (integer 1..N), ontree (boolean) } Variables TT[1..N]: topology table, whose n entry is a tab_entry record TTcur: temporary tab_entry record S: list of candidate vertices v: vertex under consideration Function Definition findBetterNode (TT1, TT2 ) Begin If 7TT1.hops TT2.hops7 o ¼ c r: ¼ TT1.bw/TT2.bw k: ¼ 7TT1.hopsTT2.hops7+1 If TT1.hops4TT2.hops If r4k Return TT1 Else Return TT2 End If Else If TT1.hopsoTT2.hops If 1/r4k Return TT2 Else Return TT1 End If Else If r4 ¼ 1 Return TT1 Else Return TT2 End If End If IEE Proc.-Commun., Vol. 151, No. 1, February 2004
Else If TT1.hopsoTT2.hops Return TT1 Else Return TT2 End If End If End The Algorithm Begin For n: ¼ 1 to N Begin TT[n].hops: ¼ infinity TT[n].bw: ¼ infinity TT[n].parent: ¼ null TT[n].ontree: ¼ false End TT[s].hops: ¼ 0 reset S v: ¼ s While v o4 t Begin TT[v].ontree: ¼ true For all (v, m) in L and b(v, m) 4 ¼ B Begin If not TT[m].ontree TTcur.hops: ¼ TT[v].hops+1 TTcur.bw: ¼ min(TT[v].bw, b(v, m)) TTcur.parent: ¼ v TTcur.ontree: ¼ false If TT[m].hopsoinfinity If findBetterNode(TTcur, TT[m]) ¼ TTcur TT[m]: ¼ TTcur Sort S End If Else TT[m]: ¼ TTcur S: ¼ S union { m } /* with proper sort */ End If End If End If S is empty Exit with QoS path not found Else v: ¼ first element of S S: ¼ S{ v } /* remove best candidate from S */ End If End Construct the explicit QoS path End IEE Proc.-Commun., Vol. 151, No. 1, February 2004
It can be seen in the algorithm that there are two occasions in which a sort function is needed. The first is during the addition of a new candidate node into the candidate queue S while the second is when a candidate node changes its parent node. Any sorting algorithm can be used, where the comparison criteria are based on the use of the findBetterNode function, which is fully described in the above pseudocode. Overall, the complexity of the proposed algorithm is the same as the original Dijkstra’s algorithm [14] since the total number of comparisons is only increased by a constant factor. It has been shown that a binary heap implementation for the queue achieves O(7L7+7V7log7V7) [27]. 4
Simulation environment and results
The proposed TE-QOSPF is compared with several other schemes, including the original OSPF with flat cost metric (OSPF-flat), OSPF with inverse capacity cost metric (OSPF-invcap) and the QoS-OSPF. The OSPF-invcap scheme (a widely known scheme used by Cisco) uses static cost metric that is proportional to 1/cap where cap is the total capacity of a link. TE-QOSPF schemes with difference c are referred to as TE-QOSPF+c. All algorithms run in an MPLS-DiffServ environment, using distributed, on-demand routing. The MPLS CR-LDP is used as the label distribution protocol and to reserve path bandwidth for EF-class traffic. The routers are asynchronous transfer mode label switching routers (ATM-LSRs) which use a cell-switching backplane. The simulation environment is built using a discrete-event network simulator, JaNetSim [30], which is based on the NIST ATM/HFC network simulator [31]. A topology based on the MCI Internet backbone (Fig. 1) is used since it represents a typical large ISP topology. The topology contains 18 routers and 32 links with three levels of capacity. The bandwidths are scaled down from the actual values in order to reduce simulation volume. Uniform demand from each site will be used in the simulation, and since the original link capacity for this topology is tailored for a nonuniform distribution of usage, the relative capacity among different links needs to be adjusted to a smaller range. The resulting bandwidths for low, medium and high capacity links are 4 Mbit/s, 5 Mbit/s and 7 Mbit/s respectively. Each router is connected to a customer site representing an aggregate of 6 classes of traffic, including the expedited forwarding (EF) [28], 4 assured forwarding (AF) [29]
18 17 1 10 5
11
9
2
16
6 14 3
15
4 12
7
link capacity Low Medium
8
Fig. 1
13
High
Simulation topology 103
2.5 OSPF-flat OSPF-invcap QoS-OSPF TE-QOSPF+1 TE-QOSPF+2 TE-QOSPF+3
average link utilisation, %
50 45 40 35
30 1.0
Fig. 3
1.1
1.2
1.3 1.4 1.5 normalised load
1.6
1.7
1.8
Average link utilisation
95 90 85 80 75 OSPF-flat OSPF-invcap QoS-OSPF TE-QOSPF+1 TE-QOSPF+2 TE-QOSPF+3
70 65 60 1.0
1.5
Fig. 4
1.1
1.2
1.3 1.4 1.5 normalised load
1.6
1.7
1.8
Peak link utilisation
1.0
0.5
0 1.0
Fig. 2
1.1
1.2
1.5 1.3 1.4 normalised load
1.6
1.7
1.8
Cell loss ratio
One of the objectives of traffic engineering is to optimise the utilisation of network resource. Both the average and the peak link utilisation are measured. For a same level of traffic load, higher average link utilisation generally implies higher throughput (not necessarily true when non-shortest path is used). On the other hand, the peak link utilisation gives an indication of the load distribution among the network links. Figure 3 shows that in general, the TE-QOSPF schemes have the highest average link utilisation while the OSPF-flat has the lowest. On the other hand, Fig. 4 shows that the TE-QOSPF schemes have the lowest peak link utilisation while the OSPF-flat has the highest. Again, the dynamic schemes outperform the static cost metric schemes. In addition, the TE-QOSPF performs better than the QoS-OSPF. Based on the low cell loss ratio of the TE-QOSPF schemes and the possible use of nonshortest paths, it is anticipated that the average link utilisation should be higher and this is verified in Fig. 3. Since the proposed TE-QOSPF scheme may involve nonshortest paths, there is a general expectation that the end-to104
OSPF-flat OSPF-invcap QoS-OSPF TE-QOSPF+1 TE-QOSPF+2 TE-QOSPF+3
55
end delay will be greater than that of the QoS-OSPF scheme. However, it may not be so especially when the network is congested because the queueing delay in a congested network has a much greater effect on the end-toend delay than the propagation delay. Figures 5 and 6 show the average and peak end-to-end delay of packets in each DiffServ class for each routing scheme in a relatively low load network. Figures 7 and 8 show the same information in a high load network (two times the load of the low load 25 OSPF-flat OSPF-invcap
average end-to-end delay, ms
cell loss ratio, %
2.0
60
peak link utilisation, %
classes, and the default DiffServ PHB groups. There are 36 sources of each traffic class, creating a total of 216 traffic sources. On–off sources with Poisson inter-arrival time are used. The transmission size is geometrically distributed. For each of the routing schemes, 12 simulation sessions are performed, each with increasing network load. We increase the overall network load by increasing the average transmission rate of each traffic source, from a noncongested network to a slightly congested network. Each simulation session is run for 180 seconds. The lifetime for each traffic source is scaled down from actual values so that the simulation duration allows a high number of LSP setup/ release events in order to evaluate the performance of each routing scheme. Figure 2 shows the cell loss ratios for each routing scheme. The cell loss ratios for OSPF-flat and OSPF-invcap are the highest, because both schemes use static cost metric and do not adapt to the change of the residual bandwidth of each link. Both QoS-OSPF and TE-QOSPF have significantly lower cell loss ratio than the former two as the network load increases. This shows that dynamic schemes that take consideration of the residual bandwidth significantly outperform those with static cost metric, by a reduction of nearly 50 percent in cell loss ratio. Between the two dynamic schemes, the proposed TE-QOSPF performs better.
20
QoS-OSPF TE-QOSPF+1 TE-QOSPF+2
15
TE-QOSPF+3
10
5
0 EF
AF4
AF3
AF2
AF1
DE
PSC
Fig. 5
Average delay (low load) IEE Proc.-Commun., Vol. 151, No. 1, February 2004
1400 OSPF-flat
1200
OSPF-invcap
peak end-to-end delay, ms
QoS-OSPF
1000
TE-QOSPF+1 TE-QOSPF+2
800
TE-QOSPF+3
600 400 200 0 EF
AF4
AF3
AF2
AF1
DE
PSC
Fig. 6
Peak delay (low load)
TE-QOSPF schemes achieve the lowest end-to-end delay among all the schemes. The TE-QOSPF scheme involves the choice of the value for the hop-count difference threshold, c. In this simulation scenario, the performances of c ¼ 1, c ¼ 2 and c ¼ 3 (TE-QOSPF+1, TE-QOSPF+2 and TE-QOSPF+3) are evaluated. There are differences in the performance for each of them, but they generally outperform the other schemes. The choice for a proper value of c depends on the size of the network. It is suggested that the value of c should not be much greater than the diameter of the network. Additionally, c should be higher in networks with higher number of links among the network nodes to allow higher possibility of utilising alternate, non-shortest paths. Another design choice is the value of k, which is used in the comparison of the residual bandwidth ratio between two candidate paths. The proposed formula for determining k works fine, as shown in the simulation results, but it is very likely that this is not the optimal choice. The determination of optimal c and k is subject to further research. 5
160
Conclusion and future work
OSPF-flat
average end-to-end delay, ms
140
From the simulation results, it can be concluded that the proposed TE-QOSPF scheme has a constantly higher performance than the QoS-OSPF and other static cost metric routing schemes. Furthermore, this performance gain comes with only a very small additional cost. Possible future research includes the search for the optimal values of parameters c and k. In addition, the proposed scheme can still be improved, for example, by incorporating extra QoS parameters, or by applying other techniques including artificial intelligence in performing the routing decision.
OSPF-invcap QoS-OSPF
120
TE-QOSPF+1 TE-QOSPF+2
100
TE-QOSPF+3
80 60 40 20
6
0 EF
AF4
AF3
AF2
AF1
DE
AF2
AF1
DE
PSC
Fig. 7
Average delay (high load)
4500 peak end-to-end delay, ms
4000
OSPF-flat OSPF-invcap QoS-OSPF TE-QOSPF+1
3500 3000
TE-QOSPF+2 TE-QOSPF+3
2500 2000 1500 1000 500 0 EF
AF4
AF3 PSC
Fig. 8
Peak delay (high load)
network). The relative delay pattern among different classes of services is preserved regardless of the network load, while the static cost metric schemes (OSPF-flat and OSPF-invcap) have a generally higher end-to-end delay than the QoS-OSPF and the TE-QOSPF schemes. The proposed IEE Proc.-Commun., Vol. 151, No. 1, February 2004
References
1 Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and Weiss, W.: ‘An architecture for differentiated services’. RFC 2475, Dec. 1998 2 Braden, R., Clark, D., and Shenker, S.: ‘Integrated services in the Internet architecture: an overview’. RFC 1633, June 1994 3 Xiao, X., and Ni, L.M.: ‘Internet QoS: A big picture’, IEEE Netw., 1999, 13, pp. 8–18 4 Rosen, E., Viswanathan, A., and Callon, R.: ‘Multiprotocol label switching architecture’. RFC 3031, Jan. 2001 5 Jamoussi, B. et al. (Eds.).: ‘Constraint-based LSP setup using LDP’. Work in progress (Internet draft), July 2000 6 Awduche, D.: ‘MPLS and traffic engineering in IP networks’, IEEE Commun Mag., 1999, pp. 42–47 7 Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, P., Krishnan, R., Cheval, P., and Heinanen, J.: ‘MPLS support of differentiated services’. Work in progress (Internet draft), Aug. 2000 8 Faucheur, F., Nadeau, T.D., Chiu, A., Townsend, W., Skalecki, D., and Tatham, M.: ‘Requirements for support of Diff-Serv-aware MPLS traffic engineering’. Work in progress (Internet draft), Nov. 2000 9 Faucheur, F., Nadeau, T.D., Chiu, A., Townsend, W., and Skalecki, D.: ‘Extensions to RSVP-TE and CR-LDP for support of Diff-Servaware MPLS traffic engineering’. Work in progress (Internet draft), Nov. 2000 10 Awduche, D., Malcolm, J., Agogbua, J., O’Dell, M., and McManus, J.: ‘Requirements for traffic engineering over MPLS’. RFC 2702, Sep. 1999 11 Moy, J.: ‘OSPF Version 2’. RFC 2328, April 1998 12 Apostolopoulos, G., Kama, S., Williams, D., Guerin, R., Orda, A., and Przygienda, T.: ‘QoS routing mechanisms and OSPF extensions’. RFC 2676, Aug. 1999 13 Apostolopoulos, G., Guerin, R., and Kamat, S.: ‘Implementation and performance measurements of QoS routing extensions to OSPF’. Proc. INFOCOM ‘99, March 1999 14 Dijkstra, E.: ‘A note on two problems in connexion with graphs’, Numer. Math., 1959, 1, (5), pp. 269–271 15 Chen, S., and Nahrstedt, K.: ‘An overview of quality of service routing for next-generation high-speed networks: problems and solutions’, IEEE Netw., 1998, 12, pp. 64–79 16 Ma, Q., and Steenkiste, P.: ‘On path selection for traffic with bandwidth guarantees’. Proc. ICNP 1997, Oct 1997, pp. 191–202 17 Shaikh, A., Rexford, J., and Shin, K.: ‘Efficient precomputation of quality-of-service routes’. Proc. Workshop on Network and Operating Systems Support for Digital Audio and Video, Cambridge, England, July 1998, pp. 15–27 105
18 Gibbens, R.J., Kelley, P.P., and Key, P.B.: ‘Dynamic alternative routing - modeling and behavior’. Proc. 12th ITC, 1988 19 Girish, M.K., Zhou, B., and Hu, J.Q.: ‘Formulation of the traffic engineering problems in MPLS based IP networks’. Proc. Fifth IEEE Symp. on Computers and Communications, 2000, pp. 214–219 20 Wang, Y., and Wang, Z.: ‘Explicit routing algorithms for Internet traffic engineering’. Proc. 8th Int. Conf. on Computer Comm. and Networks, 1999, pp. 582–588 21 Guerin, R., and Orda, A.: ‘QoS-based routing in networks with inaccurate information: theory and algorithms’. IEEE INFOCOM ’97, Japan, Apr. 1997 22 Chen, S., and Nahrstedt, K.: ‘On finding multi-constrained paths’. IEEE ICC ‘98, June 1998 23 Salama, H.F., Reeves, D.S., and Viniotis, Y.: ‘A distributed agorithm for delay-constrained unicast routing’. IEEE INFOCOM ’97, Japan, Apr. 1997 24 Iwata, A., and Fujita, N.: ‘A-hierarchical multilayer QoS routing system with dynamic SLA management’, IEEE J. Sel. Areas Commun., 2000, 18, (12)
106
25 Kodialam, M., and Lakshman, T.V.: ‘Minimum interference routing with applications to MPLS traffic engineering’. Proc. IEEE INFOCOM, 2000, Vol. 2, pp. 884–893 26 Juttner, A., Szviatovszki, B., Szentesi, A., Orincsay, D., and Harmatos, J.: ‘On-demand optimization of label switched paths in MPLS networks’. Proc. 9th Int. Conf. on Computer Comm. and Networks, 2000, pp. 107–113 27 Barbehenn, M.: ‘A note on the complexity of Dijkstra’s algorithm for graphs with weighted vertices’, IEEE Trans. Comput., 1998, 47, (2), p. 283 28 Jacobson, V., Nichols, K., and Poduri, K.: ‘An expedited forwarding PHB’. RFC 2598, June 1999 29 Heinanen, J., Baker, F., Weiss, W., and Wroclawski, J.: ‘Assured forwarding PHB group’. RFC 2597, June 1999 30 Lim, S.H.: UM JaNetSim. Available: http://nrl.fsktm.um.edu.my/ janetsim/ 31 Golmie, N., Mouveaux, F., Hester, L., Saintillan, Y., Koenig, A., and Su, D.: ‘The NIST ATM/HFC Network Simulator: Operation and programming guide’. High-Speed Networks Technologies Group, NIST, US Dept. of Commerce, Dec. 1998
IEE Proc.-Commun., Vol. 151, No. 1, February 2004
