speeds, using, if possible, an implementation on the Intel IXP1200 Gigabit network processor [2]. 5.3 Optional Work. Besides the proposed thesis work, we ...
A Dissertation Proposal on
A Scalable Service Architecture for Quantitative Differentiation Nicolas Christin Department of Computer Science University of Virginia
1 Introduction 1.1 The Need for Quality-of-Service Since its creation in the early seventies, the Internet has adopted a “best-effort” service, which relies on the following three principles: (1) No traffic is denied admission to the network, (2) all traffic is treated in the same manner, and (3) the only guarantee given by the network is that traffic will be transmitted in the best possible manner given the available resources, that is, no artificial delays will be generated, and no unnecessary losses will occur. The best-effort service is adequate as long as the applications using the network are not sensitive to variations in losses and delays (e.g., electronic mail), the load on the network is small and if pricing by network providers is not service-based. These conditions held in the early days of the Internet, when the Internet only consisted of network connections between a handful of universities. However, since the late eighties, these conditions do not hold anymore, for two main reasons. Firstly, an increasing number of different applications, such as real-time video [34], peer-to-peer networking (e.g., napster [4]), or the World-Wide Web [7], to name a few, have been using the Internet. These different applications have different needs in the service they must receive from the network. Secondly, the Internet has switched from a government-supported research network to a commercial entity in 1994, thereby creating a need for service-based pricing schemes that can better recover cost and maximize revenue than a best-effort network. These two factors have created a demand for different levels of service. In some ways, the Internet has been victim of its success, and finding a solution to the problem of providing different levels of services in the network became critical to ensure the survival of the Internet. The issues for providing different levels of services in the network are referred to as providing Quality-of-Service (QoS) guarantees. One could think that QoS issues can be resolved by increasing the capacity of the network. This solution faces two major problems. First, the QoS received by an end-to-end traffic flow is bounded by the QoS received at the link with the smallest capacity (i.e., bottleneck) on the end-to-end path. Thus, augmenting the capacity of some links only moves the bottleneck to another part of the network, and consequently, only changes the location of the problem. Second, the capacity of the network links of the Internet has steadily increased in the past couple of years and only resulted in an increase of the traffic that uses these links, without providing any QoS. In fact, the recent explosion of link capacity in the network, instead of solving the problem of providing service guarantees, has put more stringent requirements on QoS architectures. Routers in the core of a packet network now have to serve millions of concurrent flows at gigabit per second rates, which induces two scalability requirements. First, the state information kept in the core routers for providing QoS should be small. Second, the processing time for classifying and scheduling packets according to their QoS guarantees should be small as well. In addition to these two scalability requirements, the fact the Internet is now mostly a commercial network requires to utilize the existing network resources as efficiently as possible, for instance, maximizing the utilization of the links. A number of service architectures for packet networks have been devised in the last decade in an effort to address these scalability and utilization requirements.
1
1.2 History of Internet QoS
Medium High Low
Complexity of the architecture
The need for service differentiation and QoS became a topic of interest in the late 1980s and early 1990s, with the advent of networked multimedia applications (e.g., [38, 39, 40, 70]). The first solution for service differentiation was the Tenet protocol suite (see for instance [25, 26, 73]), developed by D. Ferrari and the Tenet group at UC Berkeley. After this initial work, the IETF proposed the Integrated Services (IntServ) architecture, specified in [10], as a QoS architecture for IP networks. IntServ, developed in the early and mid-1990s, provides the ability to give individual flows absolute QoS guarantees on packet delays, in terms of upper bound on delays, and packet losses (no loss), as long as the traffic of each flow conforms to a pre-specified set of parameters, e.g., peak sending rate, or maximum burst size [64]. Solutions devised for implementing IntServ require per-flow classification at each router, meaning that, at each router, each incoming packet has to be inspected to determine which service guarantees it must receive. Additionally, IntServ implementations require complex packet scheduling primitives, which run a dynamic priority sorting algorithm. The complexity of the classification and scheduling operations can cause scalability problems if the proposed algorithms have to process a large number of packets within a short period of time. Also, the IntServ architecture relies on a signaling protocol (e.g., RSVP, as described in [72]) for reserving network resources, and on admission control for determining which flows can be admitted with the assurance that no delay bound violation will occur. Both of these mechanisms require that each router keep per-flow state information, which raises additional concerns regarding the scalability of the IntServ architecture. Finally, it has been shown that using admission control mechanisms such as peak-rate allocation could result in under-utilizing the network resources [71]. Due to the open issues discussed above regarding its scalability and efficiency, IntServ has not gained much acceptance. Taking a step back from the IntServ approach, some interest developed in the second half of the 1990s for IntServ new architectures that make a distinction between operations performed in the core of the network, and opEF erations performed at the edges of the network, where a lesser number of flows are present and, thus, more complex operations can be carried out [16]. The design goal "Ideal" Service is to better reflect the original design principle of the DiffServ/AF Internet, which has the source and destination negotiate the communication parameters, while intermediary points along the path from the source to the destination None Weak StrongVery Strong are simply in charge of forwarding the data in the right Service guarantees direction. These recent efforts resulted in the Differentiated Services (DiffServ) architecture [8] which bunFigure 1: The trade-off between strength of service dles flows with similar QoS requirements in classes of guarantees and complexity of the implementation. traffic. The mapping from individual flows to classes of IntServ provides very strong service guarantees at the price of a very traffic is determined at the edges of the network, where high complexity, while DiffServ only provides limited service assur- computational resources are less scarce than in the core. ance, but has low complexity. The “ideal” service should be able to In the core of the network, scheduling primitives only provide strong service differentiation with a low complexity. Note that work with a few classes of traffic, and can thus remain the picture is qualitative. relatively simple. DiffServ currently offers two different types of service in addition to Best-Effort: an Assured Forwarding (AF) service [31] and an Expedited Forwarding (EF) service [24, 41]. The Assured Forwarding service provides qualitative differentiation among the so-called AF classes. For instance, for two AF classes of traffic denoted by class A and B, Assured Forwarding can specify that class B traffic must be dropped more aggressively than class A in times of congestion. Such a qualitative service architecture can be implemented with simple algorithms and does not require per-flow classification or signaling. However, since the AF service only provides qualitative relative guarantees, the service assurance is limited, compared to the IntServ architecture for instance. Conversely, the Expedited Forwarding service offers strong service guarantees on service rates (or delay variations in its most recent revision, [24]) for classes of traffic. In essence, providing the EF service to a flow is equivalent to providing a virtual leased-line to this flow [42] which leads to low resource utilization. Thus, it is envisioned that
2
the EF service can only apply to a very limited number of flows [68]. From these previous research efforts, it appears that a trade-off between simplicity of the implementation and strength of service guarantees has to be made, as shown in Figure 1. On the one hand, IntServ and Expedited Forwarding are too complex to be realized on a large-scale network. On the other hand, the AF service of the DiffServ model only support weak QoS guarantees. As shown in Figure 1, an “ideal” service architecture, that is, a service architecture which can deployed and used on a large network, should provide strong service guarantees with limited complexity. To this date, the most significant advance in devising such an ideal service is probably the SCORE architecture, proposed by Stoica and Zhang [68, 67], and architectures derived from it (e.g., [45, 57]). SCORE tries to reconcile the strength of the IntServ guarantees with the simplicity of the DiffServ architecture, by moving the state information needed to provide IntServ-like service guarantees from network routers to IP packets. SCORE uses this approach to provide end-to-end delay guarantees to flows without requiring per-flow state information at network routers. The basic idea for meeting end-to-end delay requirements is to keep track of the delays experienced by packets along the path from the source to the destination, by storing the values of the experienced delays in the packet headers. The stored information is used for adjusting the priority of packets so that end-to-end requirements are met. However, this approach has two drawbacks. It does not alleviate the need for packet classification, which can be computationally expensive, and it requires a change in the format of the IP header, which questions its potential deployment.
1.3 Thesis Statement and Outline of our Approach Our thesis research will address the trade-off between simplicity and strength of service. Our thesis statement is as follows.
One can design and implement a service architecture with strong guarantees and low complexity. We will therefore present a new point in the design space described in Figure 1, with low computational complexity, and strong service guarantees. To achieve this goal, we believe that one needs to revisit the current tenets of Internet QoS. We note that, until very recently, scheduling and buffer management were treated as orthogonal concepts. However, scheduling and buffer management both address the issue of managing a transmission queue at a given router. The only difference between the two concepts lies in the fact that scheduling manages the head of the transmission queue, deciding which packet will leave next, while buffer management manages the end of the transmission queue, deciding if new packets can be admitted to the queue. Thus, we make the hypothesis that considering buffer management and scheduling as the same problem will enable us to provide strong service guarantees within a simple, and thus scalable, architecture. We will thus devise mechanisms that conjointly address packet dropping and service rate allocation, from which packet scheduling immediately follows. Additionally, mechanisms for providing QoS guarantees have to work in concert with end-to-end mechanisms, such as TCP feedback mechanisms for congestion control and avoidance [36]. However, to the best of our knowledge, with the exception of RIO [16], which builds on the RED algorithm [29] in an effort to reduce the packet drops, none of the proposed service architectures takes into account of the feedback capabilities of TCP traffic. For instance, IntServ was designed with IP multicast traffic [19] in mind. IP multicast traffic cannot use TCP congestion control because of the ACK implosion problem described in [17]. However, it has become increasingly clear that the deployment of IP multicast was facing some major problems, and consequently, a vast majority of the applications on the Internet still rely on unicast/TCP data transfers. 1 Traffic regulation is therefore always realized by admission control mechanisms or traffic policers, which are separate from the scheduling and dropping algorithms. Here again, we make the hypothesis that exploiting TCP feedback mechanisms to regulate the traffic arrivals by dropping “smartly” will alleviate the need for admission control, and will allow us to design a simple, yet effective at providing strong guarantees, service architecture. 1
It is worth noting that ongoing research, for instance [15, 30, 51], is looking into replacement mechanisms for IP multicast.
3
1.4 Organization The remainder of this proposal is organized as follows. In Section 2, we identify related efforts that try to provide scalable services with quantifiable service guarantees. In Section 3, we describe our proposed approach for providing a scalable service with strong guarantees. In Section 4, we illustrate the potential of the proposed approach with the Joint Buffer Management and Scheduling (JoBS) framework. In Section 5, we list the open problems we will tackle. In Section 6, we identify the contributions expected from the thesis research. We summarize the proposal and draw brief conclusions in Section 7.
2 Related Work In this section, we discuss other approaches that are related to our proposal. Due to the amount of literature on the subject, we will only consider class-based service architectures, such as the AF service, where individual flows with similar QoS requirements are grouped into classes of traffic. QoS guarantees are provided on a per-class basis. We present an overview of the related work in this section and refer to [14, 49] for a more thorough discussion of the schemes introduced here. The AF service, and class-based service architectures in general, have the main advantage that they rely on simple algorithms for enforcing QoS guarantees. Also, they only require minor changes to the network architecture, which is an essential advantage with respect to their potential deployment. For both of these reasons, the research community seems to have reached a consensus that class-based service architectures will be a viable solution for providing service guarantees in large-scale networks such as the Internet. However, as pointed out before, the AF service of the DiffServ architecture can only ensure weak service guarantees to classes of traffic, in the sense that only qualitative relative guarantees are supported, because no classes are provided absolute service guarantees, and the difference in the service received by different classes is not quantified. The main focus of the research on QoS networks in the past three years has thus been to strengthen the service assurance that can be given within the context of class-based services without sacrificing their scalability and simplicity. To address this problem, two approaches have emerged: some efforts focused on quantifying the differentiation between classes of traffic, without enforcing absolute service guarantees, while other efforts attempt to provision absolute service guarantees for certain classes, without quantifying the differentiation between other classes. We briefly introduce the two types of approaches. Proportional Service Differentiation In an effort to quantifying the differentiation between classes of traffic, without absolute service guarantees, the best known effort is the proportional service differentiation model, initially proposed by Dovrolis et al. [20, 21], which attempts to enforce that the ratios of delays or loss rates of successive priority classes be roughly constant. For two priority classes such a service could specify that the delays of packets from the higher-priority class be half of the delays from the lower-priority class, but without specifying an upper bound on the delays. This type of service can be implemented through scheduling algorithms and/or buffer management algorithms. The service guarantees are enforced on a per-node basis and do not require any communication between participating nodes. Scheduling and dropping are treated as separate issues. Scheduling As far as the design of scheduling algorithms for proportional differentiation is concerned, two major trends have emerged. [23, 56] use rate-based scheduling algorithms, which are variations on the GPS algorithm [60], which dynamically adjust service rate allocations of classes to meet proportional delay differentiation. The service rate allocation is based upon the backlog of classes. The other trend is to use scheduling algorithms with dynamic time-dependent priorities, in a manner similar to a scheme described by Kleinrock in 1976 ([46], Ch. 3.7). WaitingTime Priority (WTP, [23]), Mean-Delay Proportional (MDP, [57]) and the Hybrid Proportional Delay scheduler (HPD, [20]) all fall into this category. The difference between these three schedulers lies in the way the priority of a given packet is computed, but it shall be noted that all three methods use functions increasing with the queueing delays to assign priorities to packets. 4
Dropping Only few dropping algorithms can support proportional loss differentiation. The Proportional Loss Rate (PLR) dropper [21] is specifically designed to realize proportional differentiated services. PLR enforces that the ratio of the loss rates of two successive classes remains roughly constant at a given value. There are two variants of PLR. PLR(M ) uses only the last M packets for estimating the loss rate of a class, whereas PLR(1) has no such memory constraints. An improvement on PLR [9] has recently been proposed to provide loss differentiation regardless of the timescale chosen for computing the loss rates. Class-Based Services with Absolute Guarantees In a different approach to strengthening class-based services, some proposed schemes attempt to provision absolute guarantees to a subset of all classes of traffic, but only provide qualitative differentiation between other classes. For instance, the Alternative Best-Effort (ABE) service [33] considers two traffic classes. The first class obtains absolute delay guarantees, and the second class has no delay guarantees, but is given a better loss rate than the first class. Loss differentiation is not quantified. The service model in [48] also supports absolute delay bounds, and qualitative loss and throughput differentiation, but no proportional differentiation. We see that all of these previous research efforts only tackle one specific angle (e.g., proportional differentiation with no absolute bounds) of the general problem we are trying to solve, which is, to find the strongest service possible that can be given in the context of class-based services. These models also give limited insight [22, 57] regarding the effect of per-hop, per-class guarantees on end-to-end flows. Our thesis work will address these problems by describing a formal framework that can be used to compare these proposed schemes, and to infer the effect of per-hop, per-class guarantees on end-to-end performance. In addition, we will use this framework to propose a novel, scalable, service architecture with strong QoS.
3 Components of a Scalable Service Architecture In this section, we sketch our proposed solution to the problem of providing strong service guarantees in a scalable manner. We introduce our concept by identifying three key components of a class-based service architecture. The first component is the definition of the set of QoS guarantees offered by the service architecture. The second component addresses the issue of managing the transmission queue of a router in order to enforce the desired service guarantees. The third component is the regulation of traffic arrivals. If the first two components are common to all QoS architectures, the third component in a class-based architecture significantly differs from its counterpart in per-flow architectures. In per-flow QoS, each flow is described by a set of parameters (e.g., maximum burst size, peak sending rate), which requires that a lot of state information be maintained if a large number of flows are present in the network. This set of parameters, or flow profile, serves as the basis for traffic regulation. Two approaches are possible: Either flows that are not conforming to their profile are policed (by dropping traffic) to satisfy to the specified profile, or, for each flow, a binary decision is performed at the ingress node (i.e., where the flow enters the network) to determine if, according to its profile, the flow can be admitted to the network and meet its service guarantees. Note that the two approaches are in fact complementary. In general, each flow which is granted admission to the network is policed if it exceeds its specified profile. As stated above, using per-flow descriptors imposes limit on the scalability of per-flow architectures, due to the amount of state information that has to be stored. Additionally, the computational complexity of checking whether each flow conforms to its profile, and possibly policing it, can be rather high if a large number of flows are present in the network. In a class-based model such as the one we propose, we do not rely on per-flow descriptions, in order to alleviate the aforementioned scalability and complexity concerns. In fact, we take a radically different approach by not making any assumption on traffic arrivals. Instead, we use the capabilities of the network (e.g., TCP feedback mechanisms) to dynamically regulate the traffic arrivals. We next describe separately the approach we propose for each of the three components in more details.
3.1 Service Guarantees The first component of a class-based service architecture is the set of proposed service guarantees. Our goal is to provide a set of service guarantees that can encompass all of AF, Proportional Differentiated Services, ABE, and
5
Sender A
1, 6
Router
Router
Router
5
3 4
Receiver
B
2
Figure 2: Congestion control in TCP. The figure considers the following sequence of events. The sender sends data to the receiver, by first sending data to the first router on the path from A to B (1). At a router in the network, a packet is dropped (2). The receiver infers a packet has been lost (3), and notifies the sender by not acknowledging the missing data (4). The sender realizes a packet has been lost by inspecting the acknowledgments received (5), and reduces its sending rate (6). To simplify the figure, we assume that packets are traveling the same way to go from A to B and to go from B to A, but it does not need to be the case in general.
other class-based services. More generally, we want to be able to enforce any mix of absolute and proportional guarantees at each participating node (i.e., router). We refer to this service as “Quantitative Assured Forwarding” service. Absolute guarantees apply to loss rates, delays, or throughput, and define a lower bound on the service received by each class. Proportional guarantees apply to loss rates and queueing delays. As an example of the guarantees in the Quantitative Assured Forwarding service for three classes of traffic, one could specify service guarantees of the form “Class-1 Delay 2 ms”, “Class-2 Delay 4Class-1 Delay”, “Class-2 Loss Rate 1%”, “Class-3 Loss Rate 2Class-2 Loss Rate”, and “Class-3 Service Rate 1 Mbps”. We require that the algorithms used to implement the service be independent of the choice of specific bounds or proportional coefficients for each class. The proposed service is thus a generalization of all previous research efforts on class-based services. Hence, the Quantitative Assured Forwarding service is the strongest service in a class-based environment, and should therefore be represented by a vertical line in the “Strong” domain in Figure 1.
3.2 Scheduling and Dropping The second component is the scheduling and dropping algorithms required at each router to enforce the desired service guarantees. To ensure scalability and high utilization, these algorithms should not rely on admission control, signaling or traffic policing. We will express the provisioning of per-class QoS within a formal framework inspired by Cruz’s service curves [18]. Using this approach, we will present a reference algorithm called JoBS (Joint Buffer Management and Scheduling), combining scheduling and dropping, which is capable of supporting a wide range of relative, as well as absolute, per-class guarantees for loss and delay, without assuming admission control or traffic policing. The reference algorithm operates as follows. After each arrival, the algorithm predicts delays of the currently backlogged traffic, and then adjusts the service rate allocation to classes and the amount of traffic to be dropped so as to meet the specified service guarantees. A unique feature of the algorithm is that it considers scheduling and buffer management (dropping) together in a single step. The algorithm allocates service rates and drops packets according to the solution to a non-linear optimization problem. While the performance of the reference algorithm with respect to satisfying the service guarantees is excellent, its computational complexity prohibits its implementation at high data rates. Thus, we will devise a heuristic approximation of the reference algorithm. We will apply linear feedback control theory for the design of the heuristic algorithm, and, to this effect, make assumptions which approximate the non-linearities in the system of study, similar to [32, 53, 54, 61].
3.3 Regulating Traffic Arrivals The third component is the end-to-end mechanism in charge of the regulation of traffic arrivals. Clearly, without admission control, it is not feasible to satisfy all absolute guarantees at all times without regulating the traffic arrivals. Admission control makes traffic arrivals conform to a set of parameters that ensure that the specified service guarantees are feasible. One of the main drawback of admission control (or traffic policing) is that the set of parameters is generally statically configured, thereby resulting in an inefficient utilization of the network resources.2 2
This problem of under-utilization can be addressed by statistical admission control (see for instance [11]), or by measurement-based admission control [43] but statistical or measurement-based admission control do not alleviate the problem of storing state information regarding which flows are/are not admitted to the network.
6
Different from existing approaches which either use admission control to regulate traffic arrivals, or perform traffic policing by means of shapers such as leaky-buckets, we will explore the possibility of considering the violation of QoS guarantees as a congestion control problem rather than an admission control problem. Congestion control has been extensively studied in the context of TCP traffic [27, 29, 36, 37, 62], which accounts for more than 90% of the total traffic on the Internet [5]. As illustrated in Figure 2, TCP provides feedback mechanisms that permit to resolve congestion problems, by reducing the sending rate of a source when a packet drop is detected. However, to the best of our knowledge, with the exception of RIO, which only provides qualitative loss differentiation, none of the existing service architectures makes use of the feedback capabilities of TCP traffic, nor of the correlation between packet losses and sending rates. We will therefore investigate the use of a congestion-control based approach, using TCP feedback mechanisms, for dynamically adapting the traffic arrival rates if needed.
4 Joint Buffer Management and Scheduling We briefly present in this section the Joint Buffer Management and Scheduling (JoBS) algorithm that we devised and implemented in the last eighteen months [14, 50]. Over the course of our research, we noted that, paralleling our efforts, there were other incentives to consider scheduling and buffer management as the same problem [52, 69]. However, [69] only uses the concept of combining buffer management and scheduling to provide proportional differentiation, and the experimental results provided are, in our opinion, not conclusive. [52] uses the concept of joint buffer management and scheduling in the context of input-queued switches to address the issue of maximizing both the aggregate throughput and buffer utilization at the same time, revisiting a problem first described by [47], and does not provide any service guarantees. [47, 52] therefore attempt at solving a problem different from ours. The work on JoBS, and the discussion in this section, do not address the use of the feedback capabilities of TCP traffic, however, the results that we will be presenting convinced us that the approach of combining buffer management and scheduling in a single algorithm was the right approach to solve the problem of providing strong service guarantees in a scalable manner. For the discussion in this section, we will assume that no traffic regulation is performed, and that at times when not all service guarantees can be met, some service guarantees are temporarily relaxed. JoBS provides a theoretical framework for reasoning on per-class service guarantees, and formulates the realization of these service guarantees as an optimization problem. Hence, the discussion in this section will provide some elements of answer regarding the first two components of a scalable service architecture, specified in the previous section. We will then turn to a brief description of a heuristic approximation of the optimization problem. The objective of the heuristic is to offer a solution that can be implemented in high-speed routers.
4.1 A Framework for Scalable Service Guarantees We consider an individual router in the network, and propose to provide per-hop, per-class guarantees. We assume that all traffic that arrives to the transmission queue of the output link of this router is marked to belong to one of N classes. We use a convention whereby a class with a lower index receives a better service. We consider a discrete event system, where events are traffic arrivals. We use t n to denote the time of the n-th event in the current busy -th events. We use ai n and li n , period3 , and t n to denote the time elapsed between the n-th and n respectively, to denote the class- i arrivals and the amount of class-i traffic dropped (‘lost’) at the n-th event. We use ri n to denote the service rate allocated to class-i at the time of the n-th event. The service rate of a class i is a fraction of the output link capacity, which can vary over time, and is set to zero if there is no backlog of class- i traffic in the transmission queue. For the presentation of this framework, we assume bursty arrivals with a fluid-flow service, that is, the output link is viewed as simultaneously serving traffic from several classes. Such a fluid-flow interpretation is idealistic, since traffic is actually sent in discrete sized packets. On the other hand, scheduling algorithms that closely approximate fluid-flow schedulers with rate-guarantees are readily available [14, 60, 74]. All service guarantees are enforced over the duration of a busy period. An advantage of enforcing service guarantees over short time intervals is that the output link can react quickly to changes of the traffic load. Further, enforcing guarantees only within a busy period requires little state information, and, therefore, keeps the implementation overhead limited. As a disadvantage, at times of low load, when busy periods are short, enforcing guarantees
( ) ()
3
()
( + 1)
()
The beginning of the current busy period is defined as the last time when the transmission queue at the output link was empty.
7
()
only with information on the current busy period can be unreliable. However, at underloaded links transmission queues are mostly idle and all service classes receive a high-grade service. The following presentation specifies the service differentiation independently for each busy period. Let Ai t define the beginning of the busy period. The arRini rival curve at n-th event, Ai n , is the total traffic that has arrived to the transmission queue of an output link Dropped Routi at a router since the beginning of the current busy period, that is Bi(n)
(0)
Ai (n) =
()
n X k=0
Class- i Traffic
()
ai (k) :
Di(n)
The input curve, Riin n , is the traffic that has been entered into the transmission queue at the n-th event,
Riin (n) = Ai (n)
n X k=0
t(n1)
li (k) :
t(n
2)
t(n) time
Figure 3: Delay and backlog at the transmission queue The output curve is the traffic that has been transmitted of an output link. Ai is the arrival curve, Riin is the input curve, since the beginning of the current busy period, that is Riout is the output curve, Di (n) is the delay, and Bi (n) is the backlog.
nX1 out ri (k)t(k) : Ri (n) = k=0
(3)
In Figure 3, we illustrate the concepts of arrival curve, input curve, and output curve for class- i traffic. At any time t n , the service rate is the slope of the output curve. In the figure, the service rate is adjusted at times t n1 ; t n2 and t n . As illustrated in Figure 3, for event n, the vertical and horizontal distance between the input and output curves, respectively, denote the class-i backlog Bi n and the class-i delay Di n . For the n-th event, we have
()
( ) ( )
()
() Bi (n) = Riin (n)
and
Di (n) = t(n)
() Riout (n) ;
maxfk < n j Riout (n) Riin(k)g :
t
(4)
Eqn. (4) characterizes the delay of the class- i traffic that departs at the n-th event. We define the ‘loss rate’ to be the ratio of dropped traffic to the arrivals. That is
()
A (n) Riin(n) : pi (n) = i Ai (n)
()
(5)
()
Since, from the definition of Ai n and Riin n , the pi n are computed only over the current busy period, they correspond to long-term loss rates only if busy periods are long. We justify our choice with the observation that traffic is dropped only at times of congestion, i.e., when the link is overloaded, and, hence, when the busy period is long. Service Guarantees With these metrics, we can express the service guarantees of a Quantitative Assured Forwarding service. An absolute delay guarantee on class i is specified as
8n : Di(n) di ;
(6)
where di is the delay bound of class i. Similarly, an absolute loss rate bound for class i is defined by
8n : pi(n) Li : 8
(7)
An absolute rate guarantee for class i is specified as
8n : Bi(n) > 0 ; ri(n) i :
(8)
() 0
()
The proportional guarantees on delay and loss, respectively, are defined, for all n such that Bi n > and Bi+1 n >
0, as and
Di+1 (n) Di (n)
= ki
pi+1 (n) pi (n)
= ki0 ;
;
(9)
(10)
where ki and ki0 are constants that quantify the proportional differentiation desired. A certain “slack” is allowed in the proportional service guarantees.
4.2 Service Rate Allocation and Packet Dropping: An Optimization Problem With the framework just defined, we see that enforcing the desired service guarantees is a matter of selecting the service rate allocation to classes and the packet drops to be performed at each event n. In [50], we showed that the service rate allocation and packet drops could be viewed in terms of a non-linear optimization problem, which we summarize here. Each event n, when an arrival occurs, a new optimization is performed. The optimization variable is a timedependent vector xn which contains the service rates ri n and the amount of traffic to be dropped li n ,
() xn = (r1 (n) : : : rN (n) l1 (n) : : : lN (n))T
()
:
(11)
The optimization problem has the form
( ) ( ) = 0; j = 1; : : : ; M ( ) 0; j = M + 1; : : : ; Q;
Minimize F xn Subject to gj xn hj xn
(12)
()
where F : is an objective function, and the gk ’s and hk ’s are constraints.
()
Objective Function Provided that the constraints can be satisfied, the objective function F : will select a solution for xn . Even though the choice of the objective function is a policy decision, we select two specific objectives, which - we believe - have general validity:
Objective 1: Avoid dropping traffic, Objective 2: Avoid changes to the current service rate allocation.
The first objective ensures that traffic is dropped only if there is no alternative way to satisfy the constraints. The second objective tries to hold on to a feasible service rate allocation as long as possible. We give the first objective priority over the second objective. Constraints The constraints of the optimization problem are QoS constraints, which define the desired service differentiation, and system constraints, which describe physical limitations and properties of the output link. The optimization at event n is done with knowledge of the system state before event n, that is the optimizer knows Riin and Riout for all events m < n, and Ai for all events m n. P The first system constraint states that the total backlog cannot exceed the buffer size B , that is, i Bi n B for all events n. The second system constraint enforces that scheduling at the output link is work-conserving. At a P P C holdsPfor all events n where i Bi n > . This constraint is stronger than work-conserving link, i ri n the limit given by the link capacity C , i.e., i ri n C . Other system constraints enforce that transmission rates
()
( )=
() 0
()
9
0 2
4
6 5 4 3 2 1 0
6 8 10 12 14 16 18 20 Simulation Time (s)
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
0 2
4
Ratio of delays
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
Ratio of delays
Ratio of delays
6 5 4 3 2 1 0
6 8 10 12 14 16 18 20 Simulation Time (s)
6 5 4 3 2 1 0
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
0 2
1).
(a) JoBS (optimization).
(b) WTP/PLR(
4
6 8 10 12 14 16 18 20 Simulation Time (s)
(c) MDP/DropTail.
Figure 4: Experiment 1: Relative Delay Differentiation. The graphs show the ratios of the delays for successive classes. The target value is k = 4. 5
3 2 1 00 2
4 6 8 10 12 14 16 18 20 Simulation Time (s)
5
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
4
Ratio of loss rates
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
4
Ratio of loss rates
Ratio of loss rates
5
3 2 1 00 2
1).
(a) JoBS (optimization).
3 2 1 00 2
4 6 8 10 12 14 16 18 20 Simulation Time (s)
(b) WTP/PLR(
Class 2/Class 1 Class 3/Class 2 Class 4/Class 3
4
4 6 8 10 12 14 16 18 20 Simulation Time (s)
(c) MDP/DropTail.
Figure 5: Experiment 1: Relative Loss Differentiation. The graphs show the ratios of loss rates for successive classes. The target value is k = 2. 0
and loss rates are non-negative. Also, the amount of traffic that can be dropped is bounded by the current backlog. Hence, we obtain ri n and li n Bi n for all events n. The QoS constraints result from the service guarantees we want to offer, as described by Eqs. (6)–(10). QoS constraints for classes which are not backlogged are simply ignored. While the absolute loss and throughput guarantees, defined in Eqs. (7) and (8), respectively, can be directly used to define some constraints of the optimization problem, the delay guarantees need to be adapted. Indeed, the delay metric Di n characterizes the delay of the class- i traffic that is leaving the transmission queue at event n, and thus, there is no direct relationship between ri n and Di n . To include the delay guarantees in the optimization problem, the scheduler makes a projection of the delays of all backlogged traffic. We refer to [49] for the mapping from Eqs. (6)–(10) to the expression of the functions gk ’s and hk ’s in function of xn . If not all constraints can be satisfied at the same time, a precedence order on the “importance” of the constraints is used to temporarily relax some constraints.
() 0
0
()
()
()
()
()
This concludes the description of the optimization process in JoBS. The structure of constraints and objective function makes this a non-linear optimization problem, which can be solved with available numerical algorithms [63]. Evaluation We compared JoBS with two other architectures specifically designed for proportional service differentiation, WTP/PLR(1) and MDP/DropTail. We evaluated the performance of the optimization-based approach by performing a single-node simulation with highly variable offered load, represented in Figure 6, oscillating between 70% and 140% of the output link capacity. In Figure 4, we present ratios of delays, averaged over a sliding window of size 0.1 seconds for all three architectures. , and we see that JoBS performs as well as WTP/PLR(1) There are four classes of traffic, with k1 k2 k3 and better than MDP/DropTail. At high loads, the desired differentiation is achieved, and at low loads, none of the three architectures can provide the target ratio, due to the work-conserving constraint. However, JoBS and WTP still manage to provide some differentiation between classes, even at low loads. In Figure 5, we compare ratios of k20 k30 . Again, JoBS performs as well as WTP/PLR(1) and better than loss rates. The target ratio is k10 MDP/DropTail, which does not support proportional loss differentiation. We next evaluate the effectiveness of JoBS in providing absolute delay bounds. We present our results in Figure 7. We ran an experiment identical to the experiment we used for comparing JoBS to other approaches, with ms and k1 k2 k 3 , as well as three different sets of service guarantees. In the first set, we have d1 k30 k20 k10 for the losses. We plot individual packet delays in Fig. 7(a), and see that, except during the intervals s; : s and s; s , JoBS is able to satisfy all delay constraints at the same time. When meeting all
= = =4
= = =2
=
=1
= =4 [2 3 5 ℄ [13 14 ℄
10
=
=
=4
1e3
1e5 Delay ( µ s)
Delay ( µ s)
1e4
100
1e3 100
10
10
1
1
0
2
4
6 8 10 12 14 16 18 20 Simulation Time (s)
Classes: 4 3 2 1
1e5
Classes: 4 3 2 1
1e4
Delay ( µ s)
Classes: 4 3 2 1
1e5
1e4 1e3 100 10
0
2
4
6 8 10 12 14 16 18 20 Simulation Time (s)
1
0
2
4
6 8 10 12 14 16 18 20 Simulation Time (s)
(a) One absolute delay guarantee, three
(b) One absolute delay guarantee, two pro-
(c) No absolute delay guarantee, three pro-
proportional delay guarantees.
portional delay guarantees.
portional delay guarantees.
Figure 7: Experiment 2: Absolute Delay Differentiation. The graphs show the delays of all packets for three different sets of service guarantees.
service guarantees at the same time is not feasible, JoBS first relaxes the proportional guarantees, which explains s; s . We refer to [49] for the plots for the loss rates. We then the transient behaviors at times s; : s and remove the guarantee k1 , and plot the results in Fig. 7(b). We observe that all service guarantees are respected during the whole experiment. For reference purposes, we plot the individual delays for the experiment we used to compare JoBS with related approaches in Fig. 7(c). We showed by evaluation that JoBS was able to 140 provide proportional service guarantees as well as ar120 chitectures specifically designed for that purpose, but, in addition, was able to provide absolute delay bounds. 100 When the set of constraints resulting from the service 80 guarantees is infeasible, JoBS temporarily relaxes some 60 constraints. We note that this study does not make any 40 assumption on traffic arrivals, and does not use the feed20 back capabilities of TCP traffic. Including the feedback 0 capabilities of TCP can only improve the performance 2 4 6 0 8 10 12 14 16 18 20 Simulation Time (s) of the algorithm with respect to the service guarantees provided, and avoids situations where the set of conFigure 6: Offered Load. The plot represents the offered load we straints is infeasible.
[2 3 5 ℄
[13 14 ℄
Offered load in % of link capacity
=4
consider in our evaluation of JoBS.
4.3 A Feedback-Based Heuristic Algorithm The optimization model described above is computationally expensive, and thus, this solution is impractical for an implementation at high speeds. In an effort to provide a scheme that can be implemented at high data rates, we devised a heuristic, based on feedback control, that approximates the optimization described above [14]. As in the optimization model, the service rates ri n and the amount of dropped traffic li n are adjusted at each event n so that the constraints defined by Eqs. (6)–(10) are met. We characterized the service rate allocation and dropping algorithm as two separate feedback control problems, which we summarize in this section.
()
()
Service Rate Allocation We have one feedback loop for each class with proportional delay guarantees. In the feedback loop for class i, we characterize changes to service rate ri n by approximating the non-linear effects of the service rate adjustment on the delays by a linear system, and derive stability conditions for the linearized control loop, following the technique described in [55]. The objective of the feedback loops is to enforce the desired delay and rate differentiation given by Eqs. (6), (8), and (9). We view the computation of ri n in terms of the recursion
()
()
ri (n) = ri (n
()
1) + ri(n) ;
where ri n is selected such that the constraints of proportional delay differentiation are satisfied at event n. From Eqs. (3) and (4), the delay Di n at the n-th event is a function of ri k with k < n. By monitoring Di n we can thus determine the deviation from the desired proportional differentiation resulting from past service rate allocations,
()
()
11
()
Saturation (Absolute Delay and Throughput Guarantees)
∆ri
Controller
Effect of rate adjustment on future delays
Di
Set Point (Proportional Delay Differentiation)
Measurement
Figure 8: Overview of the feedback-based approach. The figure is an overview of the delay feedback loop for class-i. There is one such loop per class.
( ) = ( ( ))
and infer the adjustment ri n f Di n needed to attenuate this deviation. The function f is, in control theory terminology, the “controller”. We illustrate this approach in Figure 8. The set point of the controller is defined by the proportional delay differentiation desired, and the action of the controller is limited by so-called “saturation constraints” which translate limitations on the rate allocated to class- i due to physical constraints, and absolute delay and rate guarantees. The rate adjustment ri n has then an effect on the delays Di k (k > n). The delays of traffic leaving the system are measured and compared to the set point upon each packet arrival. We approximated the effect of ri n on Di k (k > n) by a set of linear relationships, making the assumptions that the backlog Bi n does not change significantly during the time a particular traffic arrival is backlogged, and that the class-i delays do not vary significantly between two events. We then performed an analysis of the linearized control loop, and showed that the adjustment
()
()
()
()
()
ri(n) = K (n) ei (n) ; ()
()
where ei n represents the distance between the set point and the delay of class- i, and K n is a time-dependent proportional coefficient, produced the desired differentiation on delay and rate, provided that K n satisfied to that it is certain constraints. We refer to [14] for details of the derivations. The form K n ei n has the advantage P simple enough to be implemented at high speeds, and satisfies the work-conserving property, as long as i ri C.
()
() ()
Set
1 2 3 4
Enqueue
Dequeue
X
s
X
s
15347 11849 2671 2415
2603 2798 1101 837
4053 3580 3811 3810
912 970 826 858
Pred.
pred
(Mbps) 186 234 557 580
(0) =
If no feasible service rate allocation for meeting all delay guarantees exist at the n-th event, or if there is a buffer overflow at the n-th event, traffic must be dropped, either from a new arrival or from the current backlog. The loss guarantees determine which class(es) suffer(s) traffic drops at the n-th event. To enforce loss guarantees, we rewrite the loss rate, defined by Eqn. (5), as a difference equation
pi (n) = pi (n
Table 1: Overhead Measurements. This table presents, for the four considered sets of service guarantees,
1) AiA(n(n)1) + Ali((nn)) : i
i
the average number of cycles (X ) consumed by the en-
From the last equation, we can determine how the loss rate of class i evolves if traffic is dropped from class i at the n-th event. (s), and the predicted throughput pred (in Mbps) that can Thus, we can determine the set of classes that can suffer drops be achieved. In the 1 GHz PCs we use, one cycle corre- without violating absolute loss guarantees. We define the class- i sponds to one nanosecond. error e0i n as the distance between the value the class-i loss rate should have according to Eqn. (10) and its actual value. We then use hi1 ; : : : ; iR i as an ordering of the class indices from all backlogged classes, that is, Bik n > for k R, such that e0is n e0ir n if is < ir . We drop traffic in the order of hi1 ; i2 ; : : : ; iR i to satisfy to the proportional loss differentiation guarantees. For each class i, we stop dropping traffic when either (1) the loss guarantee Li is reached, queue and dequeue operations, the standard deviation
()
()
() 0
()
12
1
or (2) the buffer overflow is resolved or a feasible rate allocation for absolute guarantees exist and there is no need for dropping traffic anymore.
Implementation We implemented our feedback-based heuristic in 1 GHz Pentium-III PC routers running FreeBSD 4.3 [1] and ALTQ 3.0 [12], and present a brief evaluation of the overhead of the feedback-based algorithms. We have measured the number of CPU cycles consumed by the enqueue and dequeue procedures, which implement the feedback algorithms and the translation of service rates into packet forwarding decisions, respectively, by reading the timestamp counter register of the Pentium processor. We measured the average and standard deviation of the number of cycles over 500,000 datagram transmissions on a heavily loaded link. We compare measurements for a set of four classes with four different set of constraints. The measurements of the number of cycles are shown in the Table 1. The table shows that the overhead for the enqueue operation is significant. At the same time, the numbers indicate that a 1 GHz PC can enqueue and dequeue more than 50,000 packets per second. Considering that the average size of an IP datagram on the Internet is P : bytes [5], this results in a maximum throughput of at least 186 Mbps. We thus showed that our approach was viable at high speeds.
= 451 11
5 Research Issues and Agenda We now summarize the thesis work we have accomplished so far, and describe the issues that we will address as part of the future thesis work, along with some of the possible research tracks we have to solve these open problems. We also identify some work we are interested in pursuing, but where the feasibility is unclear at this point.
5.1 Accomplishments to Date The thesis work completed at this date is as follows. We explored the advances in research related to per-class, perhop QoS architectures. We evaluated a number of scheduling and dropping algorithms devised to provide scalable service differentiation in a simulator. Results from this research are reported in [13]. We outlined our framework for solving the problem of providing absolute and proportional service guarantees, using a joint buffer management and scheduling scheme, and expressed a solution to the problem in terms of an optimization. The work on the optimization model, and its comparison to related approaches resulted in [49, 50]. We devised a feedback-based heuristic for approximating the optimization algorithm from [50] at high-speeds. We implemented the heuristic in the ns-2 network simulator [3], and ran simulation experiments with multiple node topologies, and various types of traffic (TCP, UDP, on-off sources). We also implemented our algorithm in the FreeBSD [1] operating system kernel, using the ALTQ [12] package and ran single and multiple node topology experiments on a testbed of PC-routers. The work on the design, implementation and evaluation of the feedbackbased heuristic resulted in [14]. As a summary of the accomplishments to date, we can state that we have fully defined the first two components of our service architecture, that is, the service guarantees, and the scheduling and dropping algorithms needed to realize them.
5.2 Future Research We next outline the open issues that we will tackle as part of our thesis work. The main open issue is to determine how our per-hop service guarantees can work in conjunction with end-to-end flows. This problem can be decomposed into three subproblems, that we describe separately, along with the solutions that we envision for each problem.
Regulation of traffic arrivals using TCP congestion control mechanisms. We want to use TCP congestion control to regulate traffic arrivals. As described in Section 3, in the context of TCP congestion control, when a packet from a particular traffic flow is dropped, the sender of the traffic flow decreases its sending rate, thereby reducing the traffic arrivals at each router traversed by the traffic flow. We thus need to determine how
13
to choose which packet to drop from a class, so that we can control the per-class arrivals. We think that the solution will require a two-step process. As a first step, we need to determine the traffic mix present in the router. In other words, we need to determine how many flows are present in each class, and what is their contribution to the traffic aggregate of each class. For scalability reasons, we cannot perform per-flow accounting, which would require that excessive state information be maintained in the router. At this time, we envision two possible solutions to the problem. By monitoring when TCP SYN and FIN packets enter the router,4 we may be able to infer how many flows are present in a class. Alternatively, we may perform per-flow monitoring for samples of traffic, i.e., every packet arrivals, and use the samples to infer the traffic mix present in the router at any time, using statistical arguments. Once we have found a way of inferring the traffic mix present in the router, without keeping per-flow state information, we should be able to use existing models of the TCP throughput in function of the packet losses, e.g., [6, 28, 58], to determine the expected arrival rate of each class and the effect of dropping a packet from a particular flow in that class. We note that our solution will require some approximations, since (1) we will not be assured of the accuracy of the traffic mix inferred, because we cannot perform per-flow accounting, and (2) the TCP models presented in the literature for single-flows all rely on some assumptions and approximations. We will therefore need to be able to provide some worst-case bounds on the service received, or at least, clearly state and document the assumptions we are making.
Fairness at the flow level. Also, our research so far does not take into account fairness issues at the flow level. In particular, since the dropping algorithm does not make any distinction between flows belonging to the same class, it may happen, in the worst case, that the same flow is always dropped. We are willing to provide some fairness among flows, without sacrificing scalability. As a point of departure, we conjecture that keeping a small history of the last flows to have suffered drops, as in [59], can help us ensuring a certain fairness. Analyzing the effect of per-class, per-hop service guarantees on per-flow, end-to-end metrics. Finally, we want to evaluate how our per-hop, per-class service translate into end-to-end QoS. For this, we will run extensive experiments, on both local and wide-area networks. We will use our FreeBSD implementation for testing local network behavior, and will use the ns-2 network simulator to simulate wide-area networks with propagation delays.
Pushing the limits of the implementation. In addition to the above, we note that our current implementation of the heuristic can be run at 100Mbps line speeds. We are interested in assessing the viability of the approach at higher speeds, using, if possible, an implementation on the Intel IXP1200 Gigabit network processor [2].
5.3 Optional Work Besides the proposed thesis work, we identify here another problem that may be of interest. However, we are not convinced of its tractability and feasibility at this point, and thus leave it as optional work for the dissertation. Deriving a “true” stability condition for the delay feedback loops. As we explained in Section 4, we used a linearized model of the delay feedback loops to determine how to tune the parameter K n . We derived a stability condition on the linear, approximate, model. However, we cannot assure this stability condition applies to the real system of study for two reasons: (1) we cannot say that the assumptions involved in the linearization always hold, and (2) we had to make an approximation on the linear model itself to get a stability condition computable at high speeds. Empirically, we never saw the delay feedback loops diverge [14], but we cannot formally use this empirical evidence as a proof of correctness. We will thus investigate two other approaches for tuning K n . We will first try to derive a stability condition on the non-linear system, using Lyapunov energy functions [65], and if it is not
()
()
4
TCP SYN and FIN packets are mostly used for establishment and tear-down of a TCP connection, respectively [66].
14
()
feasible, will attempt to see if adaptive control [35] can help us tune K n dynamically. We need to stress out the fact that, at this point, we do not know whether deriving a general stability condition is doable with the tools provided by control theory. Hence, while we will investigate the issue, we cannot promise that we will find a solution. If such is not the case, we point out that the current conditions on K n have experimentally shown to be usable and efficient [14].
()
6 Research Contributions We envision that the key research contributions of this thesis are the following. A New Model for Providing Scalable QoS We will propose a novel way of reasoning about Quality-of-Service guarantees, by addressing the problem of providing all types of per-class service guarantees at the same time, as opposed to all previous efforts, which only tackled one type of service guarantees, for instance, absolute delay differentiation with no loss guarantees. All the previous research efforts have turned out to present major difficulties, with respect to their deployment or their scalability, and after almost two decades, the problem is still not solved. We believe our thesis work will shed a new light on how one can provide service guarantees in a scalable manner, and will present a viable solution for addressing the problem. As explained in Section 1, we believe that such a contribution is essential to the long-term viability of the Internet. Reconciling End-to-End Mechanisms and QoS Mechanisms One of the main reasons for the relative failure of all service architectures proposed so far is that QoS mechanisms (such as scheduling and dropping) ignore the specifics of end-to-end flows, notably, the effect of the TCP congestion control mechanisms. Our thesis will address this issue by taking into account the properties of the transport layer, and will provide a way of reasoning about admission control, in terms of a congestion control problem. In that respect, the thesis work may open a new field of research, and serve as the basis for future efforts in this direction.
7 Summary In this proposed dissertation work, we will explore the design and implementation of a scalable service architecture with strong service guarantees. By combining scheduling, buffer management and traffic regulation in a single framework, we will be able to solve the problem of providing strong service guarantees in a scalable manner. Indeed, we will use readily available resources (e.g., TCP congestion control) to alleviate the need for keeping large state information, and our architecture will not require external components such as traffic policers or shapers. This will be a significant advance in comparison to existing know-how, in the sense that people are convinced that a trade-off between strength of service guarantees and simplicity has to be made. Our dissertation work will provide a proof by example that this trade-off does not impose stringent limits on the service guarantees as is currently believed. Thus, the potential impact of our proposed thesis work will be to offer a scalable solution to the problem of providing strong service guarantees. We believe that our thesis work will pave the way for a fundamental change in the reasoning about service guarantees, by retaining the simple principles that have governed the Internet since its creation. Finally, while this thesis will focus on service differentiation in IP networks, we believe that the proposed work will yield results applicable to other fields in computer science. For instance, computer architects are considering replacing bus architectures by point-to-point connections and crossbar switches in next generation PCs [44]. It is likely that a subset of our thesis work will be applicable to resource management in such a context.
15
References [1] The FreeBSD project. http://www.freebsd.org. [2] Intel’s IXP 1200 network processor. npfamily/ixp1200.htm.
http://developer.intel.com/design/network/products/
[3] ns-2 network simulator. http://www.isi.edu/nsnam/ns/. [4] Napster protocol specification, April 2000. http://opennap.sourceforge.net/napster.txt. [5] Packet sizes and sequencing, May 2001. packetsizes/.
http://www.caida.org/outreach/resources/learn/
[6] F. Baccelli and D. Hong. TCP is max-plus linear. In Proceedings of ACM SIGCOMM 2000, pages 219–230, Stockholm, Sweden, August 2000. [7] T. Berners-Lee, R. Cailliau, A. Luotonen, H. Frystyk-Nielsen, and A. Secret. The world-wide web. Communications of the ACM, 37(8):76–82, August 1994. [8] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss. An architecture for differentiated services. IETF RFC 2475, December 1998. [9] U. Bodin, A. Jonsson, and O. Schelen. On creating proportional loss differentiation: predictability and performance. In Proceedings of IWQoS 2001, pages 372–386, Karlsruhe, Germany, June 2001. [10] R. Braden, D. Clark, and S. Shenker. Integrated services in the internet architecture: an overview. IETF RFC 1633, July 1994. [11] C. S. Chang. Performance Guarantees in Communication Networks. Springer Verlag, 1999. [12] K. Cho. A framework for alternate queueing: towards traffic management by PC-UNIX based routers. In Proceedings of USENIX ’98 Annual Technical Conference, New Orleans, LA, June 1998. [13] N. Christin. SNoOPlet network simulator. http://mng.cs.virginia.edu/snooplet/. [14] N. Christin, J. Liebeherr, and T. Abdelzaher. A quantitative assured forwarding service. Technical Report CS-200121, University of Virginia, August 2001. ftp://ftp.cs.virginia.edu/pub/techreports/CS-2001-21. pdf. Short version in submission to IEEE INFOCOM 2002. [15] Y.-H. Chu, S. Rao, and H. Zhang. A case for endsystem multicast. In Proceedings of ACM SIGMETRICS 2000, pages 1–12, Santa Clara, CA, June 2000. [16] D. Clark and W. Fang. Explicit allocation of best-effort packet delivery service. IEEE/ACM Transactions on Networking, 6(4):362–373, August 1998. [17] J. Crowcroft and K. Paliwoda. A multicast transport protocol. In Proceedings of ACM SIGCOMM’88, pages 247–256, Stanford, California, August 1988. [18] R. Cruz, H. Sariowan, and G. Polyzos. Scheduling for quality of service guarantees via service curves. In Proceedings of the International Conference on Computer Communications and Networks (ICCCN), pages 512–520, Las Vegas, NV, September 1995. [19] S. E. Deering. Host extensions for IP multicasting. IETF RFC 1112, August 1989. [20] C. Dovrolis. Proportional differentiated services for the Internet. PhD thesis, University of Wisconsin-Madison, December 2000. [21] C. Dovrolis and P. Ramanathan. Proportional differentiated services, part II: Loss rate dif ferentiation and packet dropping. In Proceedings of IWQoS 2000, pages 52–61, Pittsburgh, PA., June 2000. [22] C. Dovrolis and P. Ramanathan. Dynamic class selection: From relative differentiation to absolute qos. In Proceedings of ICNP 2001, Riverside, CA, November 2001. To appear. [23] C. Dovrolis, D. Stiliadis, and P. Ramanathan. Proportional differentiated services: Delay differentiation and packet scheduling. In Proceedings of ACM SIGCOMM ’99, pages 109–120, Boston, MA., August 1999. [24] B. Davie (editor), F. Baker, J. Bennet, K. Benson, J.-Y. Le Boudec, A. Charny, A. Chiu, W. Courtney, S. Davari, V. Firoiu, C. Kalmanek, K.K. Ramakrishnam, and D. Stiliadis. An expedited forwading PHB, April 2001. IETF draft, draft-ietfdiffserv-rfc2598bis-01.txt.
16
[25] D. Ferrari. Real-time communication in an internetwork. Journal of High-Speed Networks, 1(1):79–103, January 1992. [26] D. Ferrari and D. Verma. A scheme for real-time channel establishment in wide-area networks. IEEE Journal on Selected Areas in Communications, 8(3):368–379, April 1990. [27] S. Floyd. TCP and explicit congestion notification. ACM Computer Communication Review, 24(5):10–23, October 1994. [28] S. Floyd and K. Fall. Promoting the use of end-to-end congestion control in the Internet. IEEE/ACM Transactions on Networking, 7(4):458–472, August 1999. [29] S. Floyd and V. Jacobson. Random early detection for congestion avoidance. IEEE/ACM Transactions on Networking, 1(4):397–413, July 1993. [30] P. Francis. Yoid: extending the Internet multicast architecture. Available at http://www.isi.edu/div7/yoid/ docs/yoidArch.ps.gz, April 2000. [31] J. Heinanen, F. Baker, W. Weiss, and J. Wroclawski. Assured forwarding PHB group. IETF RFC 2597, June 1999. [32] C. V. Hollot, V. Misra, D. Towsley, and W. Gong. On designing improved controllers for AQM routers supporting TCP flows. In Proceedings of IEEE INFOCOM 2001, volume 3, pages 1726–1734, Anchorage, AK, April 2001. [33] P. Hurley, J.-Y. Le Boudec, P. Thiran, and M. Kara. ABE: providing low delay service within best effort. IEEE Networks, 15(3):60–69, May 2001. [34] RealNetworks Inc. Real video technical white paper, January 1999. http://www.realnetworks.com/devzone/ library/whitepapers/g2overview.html. [35] P. A. Ioannou and J. Sun. Robust Adaptive Control. Prentice-Hall, Englewoods Cliffs, NJ, 1996. [36] V. Jacobson. Congestion avoidance and control. In Proceedings of ACM SIGCOMM ’88, pages 314–329, August 1988. Stanford, CA. [37] V. Jacobson. Modified TCP congestion avoidance algorithm. Note sent to end2end-interest mailing list, April 1990. ftp://ftp.isi.edu/end2end/end2end-interest-1990.mail. [38] V. Jacobson and S. McCanne. vat: LBNL audio conferencing tool. http://www-nrg.ee.lbl.gov/vat/. [39] V. Jacobson and S. McCanne. vic: A video conferencing tool. http://www-nrg.ee.lbl.gov/vic/. [40] V. Jacobson and S. McCanne. wb: LBNL whiteboard tool. http://www-nrg.ee.lbl.gov/wb/. [41] V. Jacobson, K. Nichols, and K. Poduri. An expedited forwarding PHB. IETF RFC 2598, June 1999. [42] V. Jacobson, K. Nichols, and K. Poduri. The “virtual wire” per domain behavior. IETF draft, draft-ietf-diffserv-pdb-vw00.txt, July 2000. [43] S. Jamin, P. Danzig, S. Shenker, and L. Zhang. A measurement-based admission control algorithm for integrated service packet networks. IEEE/ACM Transactions on Networking, 5(1):56–70, February 1997. [44] A. Karve. PCI and future bus architectures. Network Magazine, May 1999. networkmagazine.com/article/NMG20000724S0005.
Online at http://www.
[45] J. Kaur and H. M. Vin. Core-stateless guaranteed rate scheduling algorithms. In Proceedings of INFOCOM 2001, volume 3, pages 1484–1492, Anchorage, AK, April 2001. [46] L. Kleinrock. Queueing Systems. Volume II: Computer Applications. John Wiley & Sons, New York, NY, 1976. [47] G. Lapiotis. Stochastic analysis of joint buffer management and service scheduling in high-speed nodes. PhD thesis, Polytechnic University, 1999. [48] R. R.-F. Liao and A. T. Campbell. Dynamic core provisioning for quantitative differentiated serv ice. In Proceedings of IWQoS 2001, pages 9–26, Karlsruhe, Germany, June 2001. [49] J. Liebeherr and N. Christin. Buffer management and scheduling for enhanced differentiated s ervices. Technical Report CS-2000-24, University of Virginia, August 2000. [50] J. Liebeherr and N. Christin. JoBS: Joint buffer management and scheduling for differentia ted services. In Proceedings of IWQoS 2001, pages 404–418, Karlsruhe, Germany, June 2001. [51] J. Liebeherr and M. Nahas. Application-layer multicast with Delaunay triangulations, November 2001. To appear in Proceedings of IEEE Globecom 2001.
17
[52] D. Liu, N. Ansari, and E. Hou. QLP: A joint buffer management and scheduling for input-queued switches. In Proceedings of the IEEE Workshop on High Performance Switching and Routing (HPSR 2001), pages 164–168, May 2001. [53] C. Lu, J. A. Stankovic, G. Tao, and S. H. Son. Feedback control real-time scheduling: Framework, modeling and algorithms. Journal of Real Time Systems, 2001. Special Issue on Control-Theoretical Approaches to Real-Time C omputing. To appear. [54] Y. Lu, A. Saxena, and T. Abdelzaher. Differentiated caching services; A control-theoretical approach. In Proceedings of the 21st International Conference on Distribute d Computing Systems, pages 615–624, Phoenix, AZ, April 2001. [55] A.M. Lyapunov. The general problem of motion stability, 1892. [56] Y. Moret and S. Fdida. A proportional queue control mechanism to provide differentiated services. In Proceedings of the International Symposium on Computer and Information Systems (ISCIS), pages 17–24, Belek, Turkey, October 1998. [57] T. Nandagopal, N. Venkitaraman, R. Sivakumar, and V. Barg havan. Delay differentiation and adaptation in core stateless networks. In Proceedings of IEEE INFOCOM 2000, pages 421–430, Tel-Aviv, Israel, April 2000. [58] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. Modeling TCP throughput: A simple model and its empirical validation. Proceedings of ACM SIGCOMM ’98, pages 303–314, August 1998. [59] R. Pan, B. Prabhakar, and K. Psounis. CHOKe: A stateless active queue management scheme for approximating fair bandwidth allocation. In Proceedings of IEEE INFOCOM 2000, pages 942–951, Tel-Aviv, Israel, April 2000. [60] A. K. Parekh and R. G. Gallagher. A generalized processor sharing approach to flow control in integrated services networks: The single-node case. IEEE/ACM Transactions on Networking, 1(3):344–357, June 1993. [61] S. Parekh, N. Gandhi, J.L. Hellerstein, D. Tilbury an d T.S. Jayram, and J. Bigus. Using control theory to achieve service level objectives in pe rformance management. Journal of Real-Time Systems, 2001. Special Issue on Control-Theoretical Approaches to Real-Time C omputing. To appear. [62] K. Ramakrishnan and S. Floyd. A proposal to add explicit congestion notification (ECN) to IP. IETF RFC 2481, January 1999. [63] K. Schittkowski. NLPQL: A FORTRAN subroutine solving constrained nonlinear programming problems. Annals of Operations Research, 5:485–500, 1986. Edited by Clyde L. Monma. [64] S. Shenker, C. Partridge, and R. Guerin. Specification of guaranteed quality of service. IETF RFC 2212, September 1997. [65] J.-J. E. Slotine and W. Li. Applied nonlinear control. Prentice-Hall, Englewood Cliffs, NJ, 1991. [66] W. R. Stevens. TCP/IP Illustrated, Volume 1: The Protocols. Addison-Wesley, Reading, MA, 1998. [67] I. Stoica, S. Shenker, and H. Zhang. Core-stateless fair queueing: Achieving approximately fair bandwidth allocations in high speed networks. In Proceedings of ACM SIGCOMM’98, pages 118–130, Vancouver, Canada, 1998. [68] I. Stoica and H. Zhang. Providing guaranteed services without per-flow management. In Proceedings of ACM SIGCOMM ’99, pages 81–94, Boston, MA, August 1999. [69] A. Striegel and G. Manimaran. Packet scheduling with delay and loss differentiation. Computer Communications, 2001. To appear. [70] T. Turletti. The INRIA videoconferencing system (IVS). ConneXions - The Interoperability Report Journal, 8(10):20–24, October 1994. [71] D. E. Wrege, E. W. Knightly, H. Zhang, and J. Liebeherr. Deterministic delay bounds for VBR video in packet-switching networks: fundamental limits and practical trade-offs. IEEE/ACM Transactions on Networking, 4(3):352–362, June 1996. [72] J. Wroclawski. The use of RSVP with IETF integrated services. IETF RFC 2210, September 1997. [73] H. Zhang and D. Ferrari. Rate-controlled static-priority queueing. In Proceedings of IEEE INFOCOM ’93, volume 1, pages 227–236, San Francisco, CA, March 1993. [74] L. Zhang. Virtual clock: A new traffic control algorithm for packet switched networks. IEEE/ACM Trans. Comput. Syst., 9(2):101–125, May 1991.
18
