Efficient Buffer Management and Scheduling in a Combined IntServ and DiffServ Architecture: A Performance Study∗ G. Mamais, M. Markaki, G. Politis, I. S. Venieris National Technical University of Athens, Greece e-mail: {gmamais, markaki, gpol}@ telecom.ntua.gr,
[email protected] Abstract : The Differentiated services (DiffServ) architecture promises the provision of QoS enabled networks by following a simple approach, that eliminates scalability concerns and which can be implemented and managed in large networks. However, for end-to-end QoS an appropriate signaling scheme, like the Resource Reservation Protocol (RSVP), should be supported. In this combined approach, signaling messages, which are generated by hosts and interpreted by Edge Routers, are tunneled transparently through the DiffServ core network. The treatment of the signaling messages in the DiffServ cloud is a very important issue, since long delays in their delivery or high percentage of drops could result in unpredictable and undesirable situations. In this paper, we define a DiffServ traffic class, herein called Network Control traffic class, that is used by all signaling messages travelling through the DiffServ network. We provide a simulation analysis which evaluates the key properties of the new DiffServ class including priority level, link-sharing bandwidth and link-sharing structure for an hierarchical class-based resource management mechanism referred to as Class-Based Queuing (CBQ), employed by the DiffServ core routers. Keywords : IP over ATM, Differentiated Services, scheduling, QoS, CBQ. 1. Introduction During the last years, both telecommunications industry and research community have spent a lot of effort on investigating and developing new technologies that could provide QoS over IP based networks. Historically, the first attempts were focused on providing an automatic optimization of IP traffic over switched based networks, such as ATM (e.g. MPOA, IP switching). However, the main disadvantage of these approaches is that the application software does not have an interface which can control the specific capabilities of the underlying network. A different approach, coming from the Internet Engineering Task Force (IETF), is the "Integrated Internet Services” (IntServ) architecture which provides a starting point to establish the necessary infrastructure for advanced multimedia services on top of the IP protocol suite. ∗
Integrated Services architectures have been defined using protocols which are being implemented for IP routers (e.g. RSVP [1]). The basic concept of the IntServ model is the enhancement of the existing IP router with tasks traditionally executed in switch-based networks and thus giving Internet a connection-oriented character. Hence, operations like policing, shaping, admission control and QoS management must be provided by all RSVP routers on per IP flow basis (Figure 1(a)). However, in a large scale network with millions of connected users, the number of IP sessions handled by a core RSVP router can be very large. Therefore, the execution of the above functions for every active IP flow in a core IP router leads to poor performance and to a non-scalable network architecture. Furthermore, many important issues remain unsolved, in particular appropriate charging and admission control mechanisms in order to make an integrated services architecture economically viable. The above considerations have forced the Internet community to define a new model for QoS provisioning over IP networks. The new model, known as Differentiated Services model (DiffServ) [2],[3], defines a set of traffic classes each of which is designed to serve applications with similar QoS demands. A traffic class describes the Per Hop Behavior (PHB) that the packets of this class should experience in each network node. The per hop behavior determines the priority, the maximum delay in the transmission queues, the link-sharing bandwidth and the probability of a packet to be dropped. The DiffServ model ensures high scalability by separating the operations performed in the borders of the network from those accomplished in the core network. Border routers perform more complex tasks such as traffic policing and shaping, marking and prioritized routing (Figure 1(b)). Marking is the process of classifying IP packets belonging to a specific IP flow and assigning them to the appropriate traffic class. All of the above operations are performed on per flow basis as in the IntServ model. However, the small number of active IP flows handled by a border router does not cause the scalability problems that exist in the IntServ architecture. On the other hand, core routers carry out only one simple task, that is prioritized routing.
This paper is partly funded by the European research project ELISA, ACTS AC310
RSVP router
border router
Policing
Policing
Shaping
Shaping
Admission Control
Marking
core router
QoS Management Prioritized Routing
Routing
(a) IntServ model
Prioritized Routing
(b) DiffServ model
Figure 1: Tasks performed in the network elements of an (a) IntServ and (b) DiffServ network DiffServ core routers do not keep any information for the established IP flows. On the contrary, they simply serve packets according to the traffic class that the Ingress border router has assigned to. Hence, each DiffServ core router has to know only the number of traffic classes and the corresponding per hop behavior of each class. However, in the DiffServ model the functions that would allow end users to request network resources in a dynamic manner are missing. In other words, the signaling interface between users and border routers is still not defined. Hence, the natural evolution of the DiffServ and IntServ approaches is a new combined model that inherits the scalability property of the DiffServ architecture and the dynamic network resource allocation feature of the IntServ model [4],[5]. In this new model (Figure 2), users attached to local access networks send reservation requests to the border routers using the IntServ model (e.g. RSVP). In the ingress border router, which is a mixture of RSVP and DiffServ router, the policy and local admission control operations are executed for determining the administrative rights of the user and the availability of local resources for the handling of the particular request, respectively. If both checks succeed, the border router maps the request to an appropriate DiffServ traffic class and forwards the RSVP message to the appropriate bandwidth broker or flow admission control server. Bandwidth brokers and flow admission control servers are responsible for monitoring/controlling the available resources within the DiffServ domain and for the provisioning of end-to-end Network control traffic
QoS. The exact operation of the bandwidth brokers and flow admission control servers has not been specified yet. However, there are already proposals from IETF and from other research institutes that describe a possible interoperation between bandwidth brokers, border routers and DiffServ core routers [6],[7]. The interconnection between bandwidth brokers and between bandwidth brokers and border routers is realized within the DiffServ domain. In other words, there is no separate physical network for carrying network control information between border routers and bandwidth brokers. All network control messages, which are carried transparently through the DiffServ core network as normal IP datagrams, can be RSVP packets or any other protocol messages used for admission control in the DiffServ domain such as the Common Open Policy Service protocol (COPS) [8]. The per hop behavior that the network control messages receive in the DiffServ routers is a very important issue, since long delays or high percentage of drops could result in unpredictable and undesirable situations. Therefore, it is important to define a discrete traffic class, referred to as Network Control (NC) traffic class, for the signaling control messages. This paper focuses on the definition of a Network Control traffic class for DiffServ networks. Key properties of the new DiffServ class like priority, link-sharing bandwidth and link-sharing structure are investigated for a particular resource management implementation, referred to as ClassBased Queuing (CBQ) [10]. The rest of this paper is organized as follows: Section 2 attempts an overview of the basic resource management and packet scheduling mechanisms used in DiffServ routers and highlights the functionality of the CBQ. Section 3 presents the network topology, the source models and the link sharing structures used in our performance evaluation. In Section 4, we present and evaluate the simulation results, while Section 5 gives some concluding remarks. 2. Resource Management and Packet Scheduling One of the greatest challenges coming along with the Internet explosion is managing access to the IP infrastructure, both to prevent inappropriate usage of bandwidth, and to assure critical business applications of the consistently high service quality they need. The solution to this challenge is the implementation of a
Bandwidth brokers cloud
IntServ (RSVP) Host
IntServ (RSVP) border router
border router Access Network
User data traffic
DiffServ domain
Figure 2: Target configuration
Host Access Network
bandwidth management scheme that will complement ATM’s strength in traffic aggregation and high-speed throughput, in order to extend QoS to the IP layer. For this purpose, different bandwidth management algorithms have been proposed starting from simple ones as Priority Queuing (PQ) to more advanced as Weighted Fair Queuing (WFQ) [9] and Class-Based Queuing (CBQ)[10]. PQ is a basic scheme that simply allows designated high priority traffic first access to available bandwidth; it provides no means of controlling the allocation of bandwidth. Also, PQ often results in all but the highest priority applications being completely starved of bandwidth. WFQ overcomes these limitations by providing for each of a small number of traffic classes a fixed proportion of bandwidth. Its drawbacks are that classification is complex and limited, it does not scale in number or granularity of classes and does not ensure explicit rate control for traffic classes. CBQ was developed as a progression of earlier efforts such as WFQ, referenced in the IETF’s DiffServ, to give IP a more flexible set of service parameters. In the remainder of the paper, CBQ will be used as the resource management algorithm and is going to be described next. The CBQ mechanism is based on the notion of controlled link-sharing. Moreover, the user traffic is organized into a tree, or hierarchy, of classes. A class can be an individual flow or aggregation of flows representing different applications, users, organisations, or protocol families. A class is defined by standard IP information such as address, protocol (e.g., TCP and UDP) and application (e.g., ftp, telnet, etc.). Each traffic class is then assigned a committed bandwidth rate and a priority value. At the same time a separate queue is maintained for each traffic class to ensure individual traffic class service requirements are satisfied. CBQ unifies a number of essential elements required at future router designs such as packet classification, packet scheduling and bandwidth management. The classification serves the purpose of assigning arriving packets to the appropriate class for the output link, based on the service level. The packet scheduling of CBQ is decomposed into two types of schedulers, the general scheduler and the linksharing scheduler. The general scheduler is a priority-based scheduler and determines exclusively the scheduling of the packets in the absence of congestion. On the contrary, in the presence of congestion the link-sharing scheduler controls the scheduling of the packets from the different priority classes. The aim of a link-sharing scheduler is twofold: First, to assure that each interior or leaf class in a multilevel structure will receive its allocated bandwidth over appropriate time intervals. Second, to distribute any "excess" bandwidth by following some reasonable guidelines. The overall working of CBQ can be described as follows. When a packet arrives, it is first classified. If the traffic class has not yet used all of its bandwidth (underlimit class), the packet flows immediately to the outbound link and will only experience store and forward router-like latencies by the general scheduler. If a packet arrives and the class is attempting to use more than its committed
bandwidth rate (overlimit class), the link-sharing scheduler is activated and there are two possibilities. First the packet is placed in its queue and then it is either rate shaped onto the outbound link or it is allowed to “borrow” from the currently idle bandwidth of any other traffic class. Borrowing is a unique feature of CBQ that provides bandwidth cost savings with extremely efficient sharing of link bandwidth. Each traffic class can be specifically authorized to borrow, or not borrow, from any idle bandwidth on the link. If a class is allowed to borrow, it can burst above its committed rate to meet periodic bursts in demand. In CBQ, the classes of the same priority are served roundrobin. The service discipline for the classes of the same priority can be either packet-by-packet round-robin (PRR) or weighted round-robin (WRR) [11]. For each class’s turn in the round robin scheduling, the amount of data that the class sends depends on the scheduling algorithm (PRR or WRR) and whether or not that class is classified as overlimit. A class classified as overlimit will be ratelimited to its link-sharing bandwidth. With PRR, classes of the same priority are served in round-robin order, and each class that is not overlimit sends exactly one packet per round. However, the WRR scheduler differs in that it employs weights proportional to the link-sharing bandwidth of each traffic class. The weight determines the number of bytes that a traffic class is allowed to send in a round. If a packet to be transmitted is larger than the traffic class’s weight and the class is underlimit, then the packet is sent, allowing the traffic class to borrow ahead from its weighted allotment for future rounds of the round-robin. 3 Performance Evaluation 3.1 Network topology A typical internetwork consists of multiple routers connected in an hierarchical way. Usually, there are some routers located at the edges of the network, where endusers are attached, and a number of core routers interconnecting the edge routers. Hierarchy levels may exist even in the core network depending on the number of end-users. In our simulation analysis, we used a tree network topology, shown in Figure 3, where each i level router connects α routers of the (i -1) level. Moreover, each router has only one parent except from the router located in the root of the tree, which has no parents. End-users are attached only in the 0 level routers while the height of the tree is equal to n. For the purpose of analysis, we used a network configuration with n=4 and α=3. i=2 core DiffServ router
i=1 edge router
i=0
Figure 3: Example of a tree topology with n=3 and α=3.
Table 1: Source models. Service Video Voice WWW Ftp
Source model CBR Exp Exp Exp
Peak rate (kbps) 500 64 64 64
Packet Size (bytes) 2048 256 1500 1500
3.2 Source models The traffic generated by each host can be categorized in one of the following traffic classes: a) Real-time (RT), b) Network Control (NC), and c) Best effort (BE). Two stream types compose the real time traffic: video streams and audio streams. Although both streams have similar network requirements, they have different demands regarding delay and allocated bandwidth. Moreover, the video stream is transmitted continuously for the whole duration of a videoconference, while the audio stream is transmitted only during the periods that a user speaks. Therefore, audio sources can be modeled as an ON/OFF CBR source, where the mean duration of the ON and OFF periods follows the exponential distribution with a mean value of 10 sec and 5 sec, respectively. During the ON period of an audio source, we assume that the generated traffic is CBR with (peak) rate 64kbps. The video traffic is also CBR, but with higher rate equal to 500kbps. The assumption that the generated video traffic is CBR may sound as invalid, since most of the video compressors do not generate CBR traffic (e.g. VBR MPEG). In our simulation analysis we assume that all video traffic is generated by videoconference of video telephony applications, produced by encoders that produce almost CBR traffic (e.g. motion JPEG). Moreover, we assume that all video and audio streams are enforced at the edges of the network by an appropriate policer which implements a variation of the well-known leaky bucket algorithm adopted to IP traffic. Network Control traffic consists of RSVP messages, which are generated by applications running at the end hosts. There are two main types of RSVP messages: RESV messages which are sent by receivers in order to reserve a specific amount of bandwidth from the network and PATH messages which are send by senders in order to advertise to the RSVP enabled routers (i.e. the edge routers) and receiver hosts the characteristics of their generated traffic. In contrast to other signaling protocols like the ATM UNI, RSVP requires the continuous transmission of signaling messages from both receiver and sender sides even after the establishment of the connection (soft state property of RSVP). In our simulation analysis, we assume that the average length of the RSVP messages is 250 bytes and their refresh rate (i.e. the time interval between two continuous RSVP messages sent for the same IP flow) is 15 seconds.
Ton (sec)
Toff (sec)
10 15 60
10 300 10000
Policer parameters r (kbps) b (bytes) 500 2048 64 256 -
Finally, best effort traffic consists of IP flows generated by file transfer and World Wide Web applications. The WWW and ftp sources are connected into pairs and are uniformly distributed in the network. Table 1 summarizes the sources models used in our simulation along with the selected parameters. 3.3 Link sharing structures In our simulation study, a CBQ-based priority scheduler is adopted with three different link-sharing structures depicted in Figure 4, Figure 5 and Figure 6. In Figure 4 and Figure 5 RT, NC and BE are three distinct traffic classes with different priorities, while in Figure 6 the NC class shares the highest priority level with the real-time subclasses i.e., video and voice. All arriving packets at the core DiffServ router are assigned to one of the leaf classes (such as video, voice, NC); the interior classes (such as RT, BE, Link) are used to designate guidelines about how "excess" bandwidth should be allocated. Thus, if one of the leaf classes i.e., video of the RT non-leaf class has little data to send, the hierarchical link-sharing structure specifies that the "excess" bandwidth will be allocated by the other leaf class i.e., voice. More generally speaking, the "excess" bandwidth will be shared among the rest leaf classes of an interior class in proportion to the relative linksharing allocations of those classes. In Figure 4, Figure 5 and Figure 6 the RT, NC and BE link-sharing allocations are taken equal to 65%, x% and (100-65-x)%, respectively, while the bandwidth allocation of each leaf class is set in respect to the traffic generated by the aforementioned source models. Link
x%
65%
NC
y%
RT
x+y=35
BE
1, x%
video
2, 35%
voice
2, 30%
www
3, 0.8y%
ftp
3, 0.2y%
priority, link-sharing allocation
Figure 4: Priority level 1 is assigned to NC class.
Link
65%
x%
RT
y%
NC
x+y=35
BE
2, x%
video
voice
1, 35%
1, 30%
www
ftp
3, 0.8y%
3, 0.2y%
Scenarios 2 and 5 give the same results concerning the NC class, since both WRR and PRR scheduling are used within classes of the same priority i.e., video, voice for priority 1 and www, ftp for priority 3 and have no impact on the traffic of the NC class. The same holds for the pair 3,6. Therefore, our analysis will be restricted to scenarios 1 to 4. In each scenario, we measure the mean queuing delay at each router and the percentage of dropped packets belonging to the Network Control class. 4.1 Delay and drops as a function of sharing allocation
priority, link-sharing allocation
Figure 5: Priority level 2 is assigned to NC class.
Link
65%
x%
RT
y%
NC
x+y=35
BE
1, x%
video
1, 35%
voice
1, 30%
www
ftp
3, 0.8y%
3, 0.2y%
In these simulation runs, we vary the Network Control class sharing allocation of the CBQ links and observe for the NC class both the mean packet queuing delay and the ratio of dropped packets. Figure 7 shows the percentage of dropped packets that belong to the Network Control class as a function of its sharing allocation. The sharing allocation of the Network Control class ranges from 0.1% up to 1% of the network link. As expected, the percentage of dropped packets is reduced as the sharing allocation of the NC class is increased. As can be easily seen, scenario 1, 3 and 4 perform better than scenario 2, evolving the need to protect the low sharing bandwidth allocation of the NC class by assigning to it the highest priority level.
priority, link-sharing allocation
60
Figure 6: Priority level 1 is shared among NC class and RT subclasses. Drops (%)
In our simulation analysis, we focus on the definition of two key properties of the Network Control class: 1. The selection of an appropriate link sharing structure as discussed in Section 3.3. 2. The service discipline of the CBQ mechanism within classes of the same priority (i.e. selection between Weighted Round Robin and Packet-by-Packet Round Robin). Since the first property has three alternatives and the second two, six simulation scenarios given below can be considered. Table 2: Simulation scenarios Simulation Class priority Scenario 1 RT = NC (Fig. 6) 2 RT > NC (Fig. 5) 3 RT < NC (Fig. 4) 4 RT = NC (Fig. 6) 5 RT > NC (Fig. 5) 6 RT < NC (Fig. 4)
Service Discipline WRR WRR WRR PRR PRR PRR
scenario 2 scenario 3
40
scenario 4
30 20 10 0 0
0,2
0,4
0,6
0,8
1
NC sharing allocation (%)
Figure 7: Drops as a function of sharing allocation. Figure 8 shows the average packet delay of the NC class as a function of its sharing allocation. 700
Delay (ms)
4. Simulation Results
scenario 1
50
scenario 1
600
scenario 2
500
scenario 3 scenario 4
400 300 200 100 0 0
0,2
0,4
0,6
0,8
NC sharing allocation (%)
Figure 8: Delay as a function of sharing allocation.
1
Again, the sharing allocation of the NC class ranges from 0.1% up to 1% of the network link. As expected, the delay is also reduced as the sharing allocation of the NC class is increased. Moreover, if we compare the mean packet queuing delay of the NC class of the four simulation scenarios, we validate the conclusions derived from Figure 7. Thus, scenarios 3 and 4 perform slightly better than scenario 1, which in turn performs better than scenario 2. 4.2 Delay and drops as a function of Best Effort traffic In these simulation runs, we investigate the impact of Best Effort traffic on the NC class, by varying the number of BE sources attached to each Edge Router (ER). We choose a sharing allocation of 1% for the NC class, since in this case there are no dropped packets. Again, we measure the mean packet delay and the dropped packets of the Network Control class. As can be seen in Figures 9 and 10, the increase of best effort traffic has no impact on the Network Control traffic. This happens because best effort class is of lower priority than the network control class and the CBQ mechanism assures that the NC class will always obtain its sharing bandwidth allocation. 1
scenario 1 scenario 2
0,8
4.3 Delay and drops as a function of Real-Time traffic Next, we investigate the impact of Real-Time traffic on the NC class, by varying the number of RT sources attached to each Edge Router (ER). We choose a sharing allocation of 1% for the NC class, since in this case there are no dropped packets. Again, we measure the mean packet delay and the dropped packets of the Network Control class. As can be seen in Figures 11 and 12 the increase of RT traffic has no considerable impact on the Network Control traffic for scenarios 3 and 4. However, this is not the case for scenarios 1 and 2. The negligible impact of the real time traffic to the network control traffic was expected for scenario 3, since in this scenario the NC class has higher priority than the RT class. Moreover, the poor performance of scenario 2, where the NC class has lower priority than RT class, was also expected. The difference on the performance of scenarios 1 and 4, where the priority of the NC class is taken equal to that of the RT class, comes from the employment of different scheduling algorithm i.e., WRR and PRR. The "excess" bandwidth is distributed for WRR in proportion to the bandwidth allocation of the RT and NC classes, while for PRR in proportion to the packet sizes of the RT and NC classes. Hence, PRR scheduling proves to be more appropriate for a traffic class like NC with a low bandwidth allocation and with a packet size near enough to the mean packet size of the RT class. 40
scenario 4
scenario 1
35
scenario 2
30
0,4
Drops (%)
Drops (%)
scenario 3 0,6
0,2 0
scenario 3
25
scenario 4
20 15 10
0
5
10
15
20
5
BE sources / ER
0 0
2
4
Figure 9: Drops as a function of best effort traffic.
8
Figure 11: Drops as a function of Real-Time traffic
100
scenario 1
80 60
scenario 2
160
scenario 3
140
scenario 4
120
scenario 3
100
scenario 4
Delay (ms)
Delay (ms)
6
RT sources / ER
40 20
scenario 1 scenario 2
80 60 40 20
0 0
5
10
15
BE sources / ER
Figure 10: Delay as a function of best effort traffic.
20
0 0
2
4 RT sources / ER
6
Figure 12: Delay as a function of Real-Time traffic
8
5. Conclusions This paper focused on the definition of a new DiffServ class, called Network Control class, used by IP packets that carry signaling information such as RSVP messages. Although, the content of these messages is not processed in the DiffServ domain, they should experience a certain per hop behavior that will guarantee low percentage of dropped packets and short delays. Therefore, we investigated the priority and the appropriate link-sharing structure of this class for the CBQ resource management mechanism. We presented simulation results in respect to the linksharing structures, the link-sharing bandwidth allocation of the NC class and the volume of RT and EF traffic. We concluded that the NC class must get the highest priority, since its low traffic has no significant impact on the performance of the other classes. On the contrary, if RT traffic class gets the highest priority, sources of this class may overload the network resulting in significant packet drops of the NC class. The BE traffic class has no impact on the performance of the other classes due to its lowest priority level. Additionally, in our simulation runs PRR scheduling resulted in better performance than WRR, when used within classes of the same priority. This happens because the distribution of the extra bandwidth is performed for WRR in proportion to the bandwidth allocation of the classes and for PRR in proportion to the packet sizes of the classes. Hence, the PRR scheduling favors the NC more than the WRR scheduling, since the difference between packet sizes is smaller than the difference of the bandwidth allocations between NC and RT classes. 6. References [1] Braden, R., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) - Version 1 Functional Specification", RFC 2205, September 1997. [2] Kathleen Nichols, Steven Blake, “Differentiated Services Operational Model and Definitions” Internet Draft, February 1998. [3] David Black, Steven Blake, Mark Carlson, Elwyn Davies, Zheng Wang, Walter Weiss, “An Architecture for Differentiated Services”, Internet Draft, May 1998. [4] Y.Bernet, R. Yavatkar, P. Ford, F. Baker, L. Zhang, “A Framework for End-to-End QoS Combining RSVP/Intserv and Differentiated Services”, Internet Draft, March 1998. [5] Y.Bernet, R. Yavatkar, P. Ford, F. Baker, L. Zhang, K. Nichols, M. Speer, “A Framework for Use of RSVP with Diff-serv Networks”, Internet Draft, June 1998. [6] A.Detti, M.Listanti, S.Salsano, L.Veltri, "Supporting RSVP in a Differentiated Service Domain: an Architectural Framework and a Scalability Analysis", to be presented in ICC'99. [7] Nichols, K., Jacobson, V. and Zhang, L., "A Two-bit Differentiated Services Architecture for the Internet", Internet Draft, December 1997.
[8] Jim Boyle, Ron Cohen, David Durham, Shai Herzog, Raju Rajan and Arun Sastry, “The COPS (Common Open Policy Service) Protocol”, Internet Draft, August 1998. [9] J. Davin and A. Heybey, “A Simulation Study of Fair Queuing and Policy Enforcement,” Comput. Commun. Rev., vol. 20, October 1990. [10] S. Floyd and V. Jacobson, “Link-sharing and Resource Management Models for Packet Networks,” IEEE/ACM Trans. Networking, vol. 3, August 1995. [11] S. Floyd and M. F. Speer, “Experimental Results for Class-Based Queuing”, unpublished manuscript, November 1998.
