1
A State Management Protocol for IntServ, Di Serv and Label Switching Hari Adiseshu Guru Parulkar fhari,
[email protected] Applied Research Laboratory Department of Computer Science Washington University in St. Louis St. Louis, MO 63130 Raj Yavatkar
[email protected] Communication Architecture Lab Intel Corp,JF3-206 2111 NE 25th Avenue, Hillsboro, OR 97124 Abstract Providing Quality of Service (qos) in an ecient and scalable manner in the Internet is a topic of active research. The technologies that have drawn the most attention are Integrated Services (IntServ), Di erential Services (Di Serv) and Label Switching(mpls). While these technologies are orthogonal in many respects and can coexist, they are all similar with respect to the fact that some or all the routers in such networks need to be programmed with state information associating data ows with qos classes or priorities or labels. This paper describes a protocol called ssp (State Setup Protocol) which is designed to disseminate and manage this state information. In addition to a simple design which makes fast implementations feasible, ssp provides many features like state aggregation and third party signaling which makes ssp a suitable tool for network con guration, management and provisioning as well. A detailed discussion of ssp is presented along with numerous examples and performance measurements.
1 Introduction Currently there is a great deal of interest in the Internet community in extending the single best e ort class of service to include multiple classes of service. The main motivation is the commercialization of the Internet, prompting network operators to distinguish and prioritize packets belonging to di erent classes of customers, and the development of new multimedia applications which are not loss and delay tolerant
and which require guaranteed network performance. IntServ, Di Serv and Label Switching are the major technologies being pursued to realize qos in the Internet. We begin by describing each scheme brie y and showing that a common factor in each is the requirement for a protocol to install the requisite ow state in network routers.
1.1 IntServ An IntServ reference model de nes the following elements: multiple classes of service, a means for applications to signal to networks and routers their desired class of service, a means for admission control, ow classi cation at each router and packet scheduling to ensure each service class gets its required service. The Internet community has standardized the classes of service in the IntServ wg, and has standardized a signaling protocol called rsvp in the rsvp wg of the ietf. For many reasons, including complexity, scalability and billing, the IntServ scheme and rsvp have not met widespread success or even acceptance in the marketplace. This has lead to a search for a simpler model for IntServ, the direct result being Di Serv or Di erential Services.
1.2 Di Serv Di Serv eliminates the need for applications to signal their individual requirements at run time by letting customers negotiate service pro les with their service providers, which permit customized tailoring of bw usage over a given time period. Providers use pro les to stamp customers' packets with the required qos at the network entry points. The qos information is carried in band within the packet in the ToS (Type of Service or Priority) eld of the ip header. This eliminates the need to maintain per ow state within the interior of the network. While these are the broad details, the speci cs, such as the number of service classes, the nature of the pro les and how out of pro le trac is treated are being resolved in the Di Serv wg of the ietf.
1.3 Label Switching Related to the idea of providing a scalable qos solution is the notion of Tag Switching/Label Switching. At its simplest, label switching adds labels to each packet, and lets routers (called Label Switched Routers or lsrs) use the labels to determine forwarding and scheduling decisions. The association between data ows and the labels is set up explicitly through a label switching protocol. The details of the label switching protocol are being worked out in the mpls wg of the ietf. Label switching can be driven directly by the presence of ows, like ifmp [ea96a], or can be coupled to routing protocols (Tag Switching [ea96b] and aris [A. 97]), or can be driven by intelligent network probes which can detect and/or predict network loading patterns.
1.4 State Setup A common thread running through all these schemes is the need to install state in network routers. In IntServ, rsvp [B. 97] is responsible for installing the < ow, qos > state in the routers along the path of 2
the ow. In mpls, it is the responsibility of the label switching protocol to install the mapping between the packets and the associated label along a speci ed path. Even Di Serv, which explicitly seeks to avoid the complexity of a signaling protocol like rsvp, needs a mechanism to set up the < pro le, ip ToS > state in border routers. We ask:
Is it possible to have a single protocol for installing state in routers for IntServ and Di Serv and Label Switching ?
And if so, is it possible to do so in a simple and ecient manner ?
We believe our state setup protocol, ssp, meets the above two goals. The following sections describe ssp in depth. In addition to a simple design which makes fast implementations feasible, ssp provides many features like state aggregation and third party signaling which makes ssp a suitable tool for network con guration, management and provisioning as well.
1.5 Outline of Paper Section 2 describes the design goals and basic functionality provided by ssp. This section motivates ssp design by describing rsvp features and its de ciencies. ssp speci c notions like lters and routing addresses are introduced. Section 3 describes the use of ssp in IntServ and Label switching networks. Di erent examples are used to show how ssp establishes both unicast and multicast reservations. In particular, the multicast reservations can be tailored to a variety of scenarios, including reservations for a selected subset of senders. Section 4 describes the use of ssp in Di Serv networks to manage state at border routers and as a tool for network management and provisioning. Section 5 provides details of the actual implementation on the NetBSD operating system and performance measurements. The implementation is lightweight enought to support many thousand signaling operations per second. Section 6 describes related work and Section 7 presents our conclusions and plans for future work.
2 Design Goals and Features To understand the design of ssp, it is necessary to rst take a look at rsvp, since ssp is a superset of a simpli ed subset of rsvp.
2.1
rsvp
rsvp is a receiver initiated soft state simplex two phase resource reservation protocol. Receiver initiated means that reservation requests, called resv messages, are issued by the receiver of data rather than the sender of data. Soft state means that the reservation state installed in the network has to be refreshed periodically by the receiver, else the state times out and is deleted. Simplex implies that on signaling from the receiver, the resource reservation is only done on the path from the sender to the receiver. For duplex reservations, it is necessary to issue a reservation request from both ends. Two phase means that although
3
only the receiver initiates the reservation request (resv), the sender also has to send messages periodically (called path messages). The state that rsvp disseminates is the association or binding between a ow and its qos. While rsvp introduced several innovative ideas, including receiver initiated reservations and soft state, its design and implementation are unduly complex, It was this complexity which imparted momentum to Di Serv which was set up explicitly to get rid of signaling. Some of the perceived problems with rsvp include:
Two Pass Approach Although the reservation is made only at the receiving end, it is necessary for the sending end also to periodically send messages. It can be shown that these messages are redundant [Har98b] and can be eliminated, thereby cutting down on latency and message processing.
Heterogeneous qos in Reservations One major source of complexity in rsvp is the ability of di erent
receivers to request di ering qos for the same multicast ow. This can lead to denial of service attacks, necessitating complex counter measures, such as the newly introduced blockade state. Furthermore it increases both size and complexity of state maintained at routers.
Heterogeneous Styles of Reservations A related source of complexity is the ability of receivers to re-
quest di ering styles of reservations which determines which set of senders to a multicast group are entitled to reservations. This can again lead to ambiguities in merging di ering resource requests from di erent senders.
No Support for Label Switching rsvp provides no intrinsic support for label switching. As we show
in later sections, such support is useful for optimizing refresh processing even in the absence of label switched networks.
No Wildcarded Reservations rsvp has been designed purely as an application level signaling protocol.
The application level ow is described in terms of a lter which speci es the ip source and destination address, the protocol type and the upper level ports. All these elds are fully speci ed, i.e., no elds are wildcarded, in traditional rsvp reservations. While rsvp has recently introduced the notion of a cidr style wildcarded source and destination address in lters [Boy97], this is still short of full generalization of all elds. It is much more exible to be able to wildcard arbitrary parts of the lter, allowing a single reservation to describe many application level ows, for example the set of all tcp ows between two networks. We de ne Wildcarded Reservations as the ability to set up a reservation for this type of aggregate ow. This feature is missing in rsvp.
We have brie y listed some of the drawbacks of the current rsvp design. A full writeup on these de ciencies is available elsewhere [Har98b]. These de ciencies of rsvp, coupled with the need to disseminate state in di erent types of networks provided the direct motivation for ssp. ssp is a lightweight ow setup and state setup protocol. Based on a subset of rsvp with many additional features, the design is driven by the following goals:
To eliminate bottlenecks in rsvp described previously. 4
To provide a single protocol for programming state information in routers in IntServ, Di Serv and Label Switched networks.
To do so with minimal number of protocol messages.
2.2 Features We now discuss the features of ssp. While each feature has an associated cost/bene t tradeo , the decision to include each has been taken keeping the above goals in mind.
Soft State Like rsvp, ssp reservations install soft state in routers which needs to be refreshed periodically,
else the state times out. This greatly helps in keeping ssp simple since a number of error prevention and recovery features which are needed in a hard state signaling protocol can be eliminated. For example, ssp messages can be sent using udp datagrams, rather than the more heavyweight tcp. The need for continuous refreshes means that a soft state system must handle a large number of refresh messages in addition to the original reservation message. We describe a simple technique based on label switching to handle refresh messages eciently. The other concern with soft state systems is the need for coordination regarding refresh timeouts, else a system can timeout state before it is refreshed by its neighbor. rsvp solves this by explicitly carrying the timeout in each message. In our development of ssp we have found that a global well known timeout is suitable for all types of reservations, and eliminates the need to carry the timeout explicitly in each message.
Single Pass Operation Unlike rsvp which has messages owing from both the sender and the receiver(s)
before a reservation is set up, ssp is based on a single pass. State programming of routers is driven by a single message originating from the receiver side. This single pass operation cuts down on latency while speeding up processing. In a single pass protocol, the protocol messages can be initiated either from the sender or from the receiver. The advantages of a sender initiated protocol is that the ow of messages follows the default path from the sender to the receiver, and the complexities of merging requests from di erent receivers are avoided. The disadvantage is that a reservation could be set up even when it is not necessary. Also, if the reservation request from the sender is immediately followed by data, there is a possibility that the initial burst of data might be sent before the end to end path is fully set up. In the current implementation of ssp, reservations are initiated by the receiving end. This was chosen as being consistent with the Internet philosophy, in which a best e ort data path is always available, and it is the receiver of data who is in the best position to judge if the qos needs of the data are better served by a signaled path. An issue with receiver initiated single pass reservations is that in case of asymmetric routing, the path set up from the sender to the receiver will not be along the default path, since the path setup is done from the receiver end. The assumption here is that ow speci c routing overrides default routing, so if a router has forwarding state for a ow which is di erent from the default forwarding state for that ip 5
address, then the ow speci c state will be chosen. We expect that with the evolution of QoS aware routing protocols, this will become a non issue. It is important to note however that nothing in the ssp design precludes a sender initiated version of the protocol, and in fact, a sender initiated version is one of the goals of the next phase of implementation.
Third party reservations Unlike other signaling protocols, ssp allows a reservation to be setup not only from the receiver but also from a third party like a Network Operations Center(noc). Furthermore, this reservation can be setup not only along the entire path from a receiver to a sender, but also along any chosen subset of the path. This is essential for Di Serv networks, as well as for general network management and con guration of Virtual Private Networks(vpns). The key concern here is security. A simple solution would be message encryption, another is keeping an access control list enumerating authorized request senders. This issue is not addressed in the current version of ssp.
Single exible style of reservations Unlike rsvp where it is possible to show that di erent styles of
reservations provide complexity without adding value [Har98b], ssp provides a single style of reservation which is general enough to account for reserving resources for di erent senders to a multicast group. The simpli cation is possible by eliminating heterogeneous requests from di erent senders. Until such a time that routers can intelligently discard packets based on application content, we feel this is the best choice to reduce complexity in signaling.
Generalized Filters ssp lters which describe the set of packets requiring a particular qos can be wild-
carded to an arbitrary degree. A single lter can describe many di erent application level ows. This type of state aggregation prevents the state space explosion caused by keeping state on a per ow basis only. Generalized lters can lead to lter con icts, in which a single packet can match multiple lters, leading to ambiguities in mapping packets to lters. Elsewhere [Har98a], we describe a simple geometrical technique to resolve such con icts.
Support for Label Switching ssp provides native support for label switching, thereby eliminating the
need to run separate resource reservation and label switching protocols. This is of special signi cance for label switched routers which typically have a high speed hardware path for label switched trac and a slower software path for non label switched trac. When a resource reservation protocol is invoked before a label switching protocol in such networks, the trac is forced into the slow path initially. ssp eliminates such race conditions.
Dynamic Reservations Since ssp is based on soft state, it is necessary to periodically refresh the state
which has been set up in the network. This provides an opportunity to modify the state based on current requirements, thereby providing optimum utilization of resources. An application or end user can continuously vary its resource requirements based on current requirements by sending its current requirements in each refresh message. This presupposes a mechanism which is suciently lightweight and fast to handle frequent state changes. For example, the current implementation of admission control is based on a simple peak bw allocator. It is not clear if continously varying reservations can coexist with complex admission control algorithms. However, we consider this an important feature in the evolution of signaling, and one which does not add to the overhead of ssp implementation. 6
We now turn to the operation of ssp. In order to understand how ssp works it is necessary to look at some of the basic components of ssp messages, and how they are handled at each hop. IP/UDP header
SSP Setup Message
a) SSP Setup message encapsulated in a UDP datagram
IP address of this hop
IP address of next hop
IP Src Address
IP Dst Address
SSP Reserved Port
SSP Reserved Port
UDP Src Port
UDP Dst Port
b) IP/UDP header of Setup message
Final Unicast Address
BW of flow
Generalized Filter
QoS
Label
Routing Address
Address of previous hop
Phop Address
c) Important fields in a Setup message
Generalized Filter Value
Mask
IP Src IP dst IP proto Sport
Dport IP ToS
IP Src IP dst IP proto Sport
Dport IP ToS
d) Format of Generalized Filter in Setup message
Figure 1: ssp Setup message
2.3 Overview The basic ssp message is the Setup message. The Setup message is propagated from the receiver to the sender, or any subset of the path, and as it propagates it installs state in the intermediate routers. The Setup message is transmitted hop by hop in udp datagrams as shown in Figure 1.a. The ip/udp header shown in Figure 1.b shows that the Setup messages are sent by adjacent ssp peers on the ssp reserved port. Figure 1.c shows the important components of the Setup message. The Generalized Filter describes the packets for which the state is to be setup in the form of the associated qos and label information. Though ssp itself is independent of the nature of qos information, for simplicity, we work with a simple model of qos in which the qos is represented by a single bandwidth(bw) parameter. In addition, the Setup message also contains a routing address which is a unicast address to which the ssp messages are forwarded, as well as the Phop Address which is the ip address of the previous hop. The Phop Address is the same as the ip source address of the ip/udp datagram carrying the Setup message. It is duplicated because user level implementations of ssp may not have access to the ip header of the incoming ssp datagram. The lter carried in ssp is a generalized lter which can be masked to an arbitrary extent as shown in Figure 1.d. It is the hop by hop forwarding of Setup messages towards the routing address which installs the necessary state in routers. Depending on who initiates the Setup messages, the degree of wildcarding of the lters, and the routing address, many di erent scenarios are possible, which are described in detail in the subsequent 7
sections. It is our claim that this approach to programming state is necessary as well as sucient when the following conditions are met:
Information needed to set up state is obtained out of band. To initiate a reservation request,
State is atomic. When any node receives a request, the request is either accepted or denied. It is not
Implicit acknowledgements. The initiator of the reservation gets an error message in case the
the requestor needs to know the desired qos. We assume that this information is available out of band, for example, through a service advertisement protocol, or through well known values associated with individual applications. Thus no signaling messages are needed to obtain this information. modi ed and there are no negotiations. Therefore, no signaling messages are traded back and forth to negotiate a particular qos.
reservation is unsuccessful, but no message otherwise. This eliminates the need to have a separate signaling message for positive acknowledgement.
We observe that installing state in a router associating a set of packets with a particular qos requires that the router receive at least one message. Furthermore, once the message is received by all the relevant nodes, all the programming necessary in the network and end systems is done. In the case of an IntServ network, for example, an end to end path with the desired qos would have been established. Thus, no additional messages are needed.
3
ssp
in IntServ/Label Switching Networks
In this section, we describe how ssp can be used to install state in Label Switching and IntServ networks. We illustrate the use of ssp with examples. Our examples in this section are based on networks which support both Label switching and IntServ, however, it is fairly easy to understand the behavior of ssp in pure Label switched networks and IntServ networks. We describe both unicast and multicast cases. Our examples assume that the end user initiates the signaling request, and are intended to illustrate the use of the lter and the routing address in Setup messages to set up exible reserved label switched paths. Scenarios where the network, as opposed to users, initiates reservations are described in the next section.
3.1 Unicast Reservations Figure 2 shows an IntServ Label Switched network containing end systems A and B, connected by routers R1 and R2. Node A wishes to reserve some speci ed bw for a ow from B to A. It sends a Setup message with the relevant elds as shown in Figure 2. As can be seen, the requested bw is 10 Mbps for a udp ow originating at B from port 5051 and terminating at port 5052 on node A. Node A selects the rst hop label to be 32. ssp works with a model of receiver allocation of labels, in which the downstream end of a link allocates the labels to be used by the sending side. The routing address eld of the Setup message has 8
Setup Message Receiver A
Filter=f1, Label=32, QoS=10 Mbps, Raddr=B
Filter STATE Router R1
f1
QoS 10 Mbps
Incoming Outgoing Phop Label Label 55
32
A
Setup Message Filter=f1, Label=55, QoS=10 Mbps, Raddr=B
Router R2 STATE
Filter f1
QoS 10 Mbps
Incoming Outgoing Phop Label Label 72
55
R1
Setup Message Filter=f1, Label=72, QoS=10 Mbps, Raddr=B
Filter STATE
Sender B
IP Src f1
B
f1
QoS
Incoming Outgoing Phop Label Label
10 Mbps
IP Dst
72
Src protocol Port
A
udp
5051
R2
Dst Port 5052
Raddr = Routing Addr
Figure 2: ssp Setup Message used to setup a unicast reservation from node B to node A been set to B. The Setup propagates hop by hop based on reverse path forwarding to the routing address, namely B, setting up an end to end label switched path with the requested bw provided the Setup passed admission control at each network node. As can be seen, each node along the path selects a new label before forwarding the Setup request, and internally updates its state to maintain the mapping between the incoming and outgoing labels for the ow. Once the Setup reaches B, B can start sending using the label switched path which has been setup from B to A. The state that is set up at each router is also shown in gure, with the incoming and outgoing labels of the data ow. There are several points about ssp operation that we can make at this point.
First, node A gets implicit noti cation of the success of the reservation by the fact that it receives data on the switched path. In case of failure of the Setup, an Error message would have been sent back to the originator of the Setup. The Error message would be routed along the same path as the Setup message, thereby canceling the installed state at the intermediate nodes. In the case of a conventional IntServ network (without lsrs), each router would have programmed its scheduler to let this ow's packets receive 10 Mbps bw. In this case, the label eld would be left blank. In general, it is not necessary to specify both the qos eld as well as the label eld at the same time for networks which do 9
not support both label switching and IntServ.. It is therefore possible, for example, to create an end to end label switched path without any associated bw reservations for a pure label switched network, or to create a reservation without specifying a label for a pure IntServ network. However, as pointed out in Section 7, specifying non zero labels in Setup messages is useful even in non label switched networks for fast processing of Setup refreshes.
Second, in this example, node A requested a reservation for a fully speci ed ow from B. By appropriate wildcarding of the lter in the Setup message, it is possible to ask for reservations for a broader set of packets, for example, all tcp packets, or even all packets from B to A. This example also illustrates the use of the routing address to propagate the Setup message from the receiver of data to the sender based on reverse path forwarding. In the case of asymmetric routing, it might happen that the path set up from the sender to the receiver does not match the default path. One important assumption made in ssp is that ow speci c state overrides default state. So if a router has information regarding both the default path for a ow and the path based on the newly installed state for the ow, and these two are di erent, the assumption is that the ow speci c state will be used.
Third, the state installed in the network is soft state, meaning that it times out unless it is refreshed periodically. The receivers periodically refresh the state by sending Setup messages. If these stop for some time, each node along the path times out the state and releases any reserved resources. It is also possible to explicitly tear down the state using a Tear message described in later sections.
3.2 Multicast Having looked at a simple example of ssp being used as a unicast signaling protocol, we now look at some multicast examples which illustrate the exibility of reservation supported by ssp. Unlike rsvp, in ssp there is no explicit notion of di erent styles of multicast reservations. The multicast ow setup is similar to the unicast setup. Each receiver of a multicast group which requires non-default delivery of data from senders transmits a Setup message to the rst hop router. In this case, the receiver must select a speci c sender as the routing address, so the Setup message propagates to a speci c sender. The receiver is free to send multiple Setup messages to di erent senders. We work with an example scenario of a single receiver and two senders. In the rst example, we show how the receiver can reserve bw for a single sender by sending a Setup message with the routing address set to the desired sender. In the second example, we show how the receiver can reserve bw jointly for both senders, by sending two setup messages with di ering routing addresses, but the same lter speci cation. And nally in the third scenario, we show how the receiver can reserve bw separately for the two senders by sending two separate Setups with separate lters. These examples correspond to the Fixed Filter and the Wildcarded Filter style of reservations in rsvp. Similarly it is possible to construct examples corresponding to the Shared Explicit style of rsvp, all based on the single type of Setup message of ssp. The rst example is shown in Figure 3. This shows a multicast group M with member A, and two senders B and C. In this example the receiver is interested in associating B's packets with a particular qos, in this case 10 Mbps. So A sends a Setup message, with the routing address set to B, and with the ip Filter set to match packets sent by B to the multicast group M. The propagation of these Setup message towards B 10
Multicast Group M Receiver sends Setup message with Routing Address (Raddr) = B Setup Message
Receiver A
Filter=f1, Label=32, QoS=10 Mbps, Raddr=B
Filter Router R1
STATE
f1
QoS
Incoming Outgoing Phop Label Label
10 Mbps
55
32
A
Setup Message Filter=f1, Label=55, QoS=10 Mbps, Raddr=B Filter Router R2
STATE
f1
QoS
Incoming Outgoing Phop Label Label
10 Mbps
72
55
R1
Setup Message Filter=f1, Label=72, QoS=10 Mbps, Raddr=B
Filter
QoS
Sender B
STATE
f1
Incoming Outgoing Phop Label Label
10 Mbps
72
R2
Sender C
IP Src f1
B
IP Dst M
Src protocol Port udp
5051
Dst Port 5052
Figure 3: ssp Setup Message used to setup a reservation for a multicast ow from node B to node A results in the establishment of a label switched path from B to A with a reserved capacity of 10 Mbps, shown in the gure. Packets sent from C to the group M traverse the default path without any reservation. As Setup messages from di erent receivers propagate towards the sender, they are merged at intervening nodes. Thus there is no implosion problem at the sender. ssp works with a model of homogeneous reservations in which reservations from di erent receivers to the same sender are expected to request the same qos, else an error message is returned. Receivers are expected to know the correct bw out of band through service advertising protocols, similar to the way they learn about the existence of senders. It can be shown that heterogeneous reservations, a key feature of rsvp, do not deliver any value without the network knowing the data content of packets, and can lead to denial of service attacks [Har98b]. These problems are avoided with ssp. Now, let us consider the same example with multicast senders B and C, and receiver A, and assume that A wants to reserve bw for all multicast packets from B and C to A, i.e., a shared pipe for data from B and 11
Multicast Group M Receiver sends the same Setup message with Routing Address(RA) = B and Routing Address (RA) = C
Setup Message
Router R2
f1
QoS
Filter
STATE
Incoming Outgoing Phop Label Label
10 Mbps
Setup Message
STATE Filter=f1, Label=55, QoS=10, RA=B
Filter
Router R1
f1
QoS
55
32
A
Incoming Outgoing Phop Label Label
10 Mbps
72
55
R1
e Setu
ssag p Me
Filte
r
Filte
r=f1,
abe =f1, L
Labe
l=72,
l=72,
QoS=
QoS=
10, R
10, R
A=B
A=C
Filter=f1, Label=55, QoS=10, RA=C
Filter=f1, Label=32, QoS=10, RA=C
Filter=f1, Label=32, QoS=10, RA=B
Receiver A
Sender B Sender C
STATE IP Src
f1
Filter
*
f1 IP Dst
QoS
Incoming Outgoing Phop Label Label
10 Mbps
72
Src protocol Port
M
udp
5051
R2
Dst Port 5052
Figure 4: ssp Setup Messages used to setup a reservation for all multicast packets from B and C to A C. In that case we have the scenario shown in Figure 4, in which A sends two separate Setup requests with identical ip Filter elds, but distinct routing addresses as shown. As a result, a switched reserved path is set up from the senders to the receiver, in which the switched path is the same from router R2 to B. As can be seen, the source address in the ip Filter f1 has been wildcarded to let packets from both sources share the reservation. In a label switched network based on cell switching there is a problem of cell interleaving if multiple senders are allowed to send on a single label (vc). In our model, we assume that the lsrs support multiple senders on a single vc without cell interleaving, for example, by using frame reassembly. If not, the lsr should send an error message back to the originator of a Setup message, if the message involves multiple senders. Of course, this issue does not arise with conventional routers, or with lsrs based on non-atm technology. In our examples, we assume lsrs which are able to multiplex multiple senders on a single label 12
without interleaving. Multicast Group M Receiver sends separate Setup message with Routing Address(RA) = B and Routing Address (RA) = C
Setup Message
Router R2
f1 f2
A=B
32 33
A A
Incoming Outgoing Phop Label Label
10 Mbps 10 Mbps
72 72
55 56
R1 R1
Filte
r
Setu
ssag p Me
QoS= l=72,
abe =f1, L
Filter
STATE IP Src
f1 f2
f1 f2
QoS
55 56
e
10, R
10, R QoS= l=72, Labe r=f2, Filte
Sender C
Incoming Outgoing Phop Label Label
10 Mbps 10 Mbps
Filter
STATE
Sender B
QoS
Setup Message
STATE Filter=f1, Label=55, QoS=10, RA=B
Filter
Router R1
A=C
Filter=f2, Label=56, QoS=10, RA=C
Filter=f2, Label=33, QoS=10, RA=C
Filter=f1, Label=32, QoS=10, RA=B
Receiver A
B C
f1 IP Dst
QoS
Incoming Outgoing Phop Label Label
10 Mbps
72
Src protocol Port
M M
udp udp
5051 5051
R2
Dst Port 5052 5052
Figure 5: Setup Message used to setup separate reservations from B and C to A Our nal multicasting example shows A setting up distinct reservations for B and C by sending separate Setup messages with separate ip Filter elds f1 and f2 as shown in Figure 5. This causes two distinct reservations to be made from C to A and from B to A respectively. The state at node C is omitted due to lack of space, but can be easily derived from the Setup message that it receives. As a result of these two Setup messages, separate switched paths are created from B and C to A. The examples that we have discussed showed a single receiver. What happens when there are multiple receivers ? What happens when one receiver makes a wildcarded reservation while another receiver makes a reservation for a single source ? The rule is that two reservations can be merged along a link only if the ip Filter elds of both reservations are identical. If not, two distinct reservations are created and packets are directed to the requisite queue at the output of the router based on mathching the most speci c lter. 13
In this section we have explored how ssp can function as a user reservation protocol in a label switched IntServ network. Through unicast and multicast examples, we have seen how the Setup can be tailored to di erent reservation needs by appropriate wildcarding of the lter, and selection of the routing address. It is interesting to note that all the di erent types of reservations and switched paths have all been set up using only a single type of message. As the next section shows, ssp can be used in scenarios besides user level signaling.
4
ssp
for Network Management and Di Serv
The previous examples showed ssp being used to setup state by end user applications. Of course, the Setup messages could have been initiated by proxy applications, or by routers based on ow detection too. We now turn our attention to another use of ssp, namely for network management and con guring. We show how ssp can be used to set up reservations for aggregate ows, and how reservations can be made remotely by a third party or a network operator. We will also illustrate the use of ssp in Di Serv networks based on ip tos eld. To simplify the examples, we omit Label Switching.
4.1
vpns with qos
A major challenge facing service providers is the provision of Virtual Private Networks (vpns). At its simplest, a vpn de nes a reserved pipe through a service provider, linking di erent customer sites through an overlay network with some qos guarantees. We refer to this as a qos tunnel. Consider Figure 6, which shows two distinct networks A and B. Assume that A and B are remote from each other, and that connectivity is provided by an isp (Internet Service Provider). The isp needs to provide a qos tunnel from B to A. We wish to program the routers of the isp so that packets sent from network B to network A get an assured bw of 20 Mbps. Assume that the border routers are A.7 and B.32 as shown. Setting up a qos tunnel from net B to net A is easily done by sending a Setup message from A.7 to B.32 as shown in Figure 6. The Filter is used to specify all packets originating in network A and destined for network B. This is done by using the mask eld of the ip Filter to mask out all the host related parts of the ip source and destination address, and also to mask out the ip protocol and upper level ports. Also, the routing address is speci ed as B.32 to terminate the tunnel at B.32. This sets up a simplex qos tunnel from net B to net A. The state at router R2 is shown in the gure. For setting up a full duplex qos tunnel, it is necessary for B.32 to send a similar Setup message to A.7. In case the vpn is encrypted, port level information is not available directly from the packet headers without reference to the spi eld. If the qos tunnel is set up for all packets traversing the vpn then the port level information is not needed. If more di erentiation is needed, the lter speci cation of ssp will have to be enhanced to incorporate the spi. In the previous example, the qos tunnel was set up by the border router A.7. In our next example in Figure 7, we will again setup a qos tunnel between the same networks, but do so remotely from the isp's noc( Network Operations Center), located at ip address C.16. In this case we again send the same Setup message that was sent in the previous example but we initiate it from C.16, rather than A.7. The ip packet in which the Setup message is sent is addressed to A.7 as shown in Figure 7. Note that Setup messages are 14
B.*
Setup Filter=f1, QoS=20, RA=B.32
A.*
B.32
R2 R1
A.7
ISP
Filter STATE at R2 IP Src f1
B.*
IP Dst A.*
Src protocol Port *
*
f1
QoS 20 Mbps
Phop R1
Dst Port *
Figure 6: ssp Setup Message used to setup a qos tunnel between two networks interpreted only by routers which directly receive the ip packet in which the Setup message is encapsulated. So intervening routers between C.16 and A.7 do not examine the Setup message. A.7 is the rst router to examine it, and once it examines it, the sequence of operation is the same as in the previous case, leading to a qos tunnel being setup between nets B and A.
4.2 Di Serv We now turn our attention to Di Serv networks. Several isps have already announced support for Di Serv based on the ip ToS eld. In this case, border routers (between the customer and the network) stamp selected packets which t a pro le with a higher value of priority in the ip ToS eld, and internal routers schedule packets based on this eld. The pro le is negotiated out of band by the customer and the service provider. This scheme does away with the need for user level signaling and admission control and packet classi cation. Even though there is no user level signaling in this scheme, we still need a tool which can download the pro les from the isp's noc to the border routers. This is a task ideally suited for ssp. One interesting and powerful aspect of ssp is how ssp can be used to setup state not only along a speci ed path as described in previous examples, but also throughout an entire network or an arbitrary subset. For example, assume a priority based Di Serv network and assume the following:
A customer has negotiated a pro le which lets all the customer's tcp trac come in as high priority.
High priority trac means that the ip ToS byte is set to 3 in ip packets. 15
B.*
Setup Filter=f1, QoS=20, RA=B.32
A.*
B.32
R2 A.7
R1 IP
/U
DP
Se
tup
ISP
IP Dst=A.7
C.16 Network Operations Center (NOC)
IP Src=C.16
Filter STATE at R2 IP Src f1
B.*
IP Dst A.*
Src protocol Port *
*
f1
QoS 20 Mbps
Phop R1
Dst Port *
Figure 7: ssp Setup Message used to remotely setup a qos tunnel between two networks
The customer's ip address is B.32.
All the border routers of the isp are con gured to be part of a multicast group M.
The isp's noc, which keeps track of customers pro les, has address C.16.
Such a network is shown in Figure 8. We wish to program border routers in that network such that all tcp packets to B.32 are stamped with the ip ToS byte set to 3. This can be done using the single Setup message issued from the noc as shown in Figure 9. The Setup message is sent to all the border routers by addressing the ip packet which encapsulates the Setup message to the special multicast group M which includes all the border routers in the net. The relevant elds of the Setup message are shown in Figure 10. Once the Setup message reaches each border router, it does not propagate further, since the routing address is set to 127.0.0.1 which corresponds to the local loopback address. Each router will treat this address as the terminal condition for the propagation of the Setup message, since it matches its own address. The desired pro le is therefore installed in each border router and all trac which meets this pro le, in this case all incoming tcp trac to B.32, will be stamped with the ip ToS set to 3. As can be seen, the lter eld includes the ip ToS. Many variations are possible on this theme, depending on the nature of the pro le negotiated by the customer. In the previous example, if the pro le includes outgoing tcp trac as well, then the noc can also send a Setup message with the lter shown in Figure 10 with the source and destination ip address interchanged to the border router connecting the customer to the isp. It is easy to analyze other scenarios, 16
Network Service Provider Border Routers belonging to mcast group M M
NOC
B.32
C.16 M
ISP M
M
ISP 2
ISP 3
Figure 8: Di Serv network Network Service Provider Multicast distribution path for group M
M
DP
IP/U B.32
C.16
c= IP Sr
M
Setup st=M
IP D
NOC C.16
ISP M
M
ISP 2
ISP 3
Figure 9: ssp Setup for programming border routers for example, where the pro le includes all trac, or is restricted to web trac. In case the pro le varies with time and day, then the noc can modify the lters in its periodic refresh messages to install the new pro les. Of course, once the noc stops the refreshes, the state is automatically removed from the border routers. The aim of illustrating these examples was to show how ssp using a single message can be used to setup state in IntServ, Di Serv and Label Switched networks. The power of Setup messages comes from the great exibility provided by choosing di erent values for the lters and the routing addresses, and by the encapsulation of the Setup message in a suitable ip header. The state that is thus set up needs to be periodically refreshed using Setup messages, else it is torn down. It is also possible to tear the state using explicit Tear messages. Unlike Setup messages which do not need an acknowledgement, Tear messages are acknowledged using Tear Ack messages. Tear Ack messages permit immediate reuse of resources freed by the Tear message, else it is necessary to wait for a timeout period.
17
Setup Message Filter=f1, IP ToS=3, Routing Addr=127.0.0.1 IP Src f1
IP Dst
*
B.32
Src protocol Port tcp
*
Dst Port *
Figure 10: Filter for incoming tcp trac to a customer
5 Implementation Details User Application
1) User issues request SSP
User Space 3) Request sent to remote site
Kernel
UDP
TCP
2) Flow state inserted in kernel IP
Packet Scheduler
Packet Classifier
Network Driver
Control Flow Data Flow
Figure 11: ssp Host Implementation ssp has been implemented as part of a testbed on IntServ label switching networks [Dec95]. The testbed
comprises multiple workstations linked together by label switched routers based on cell switching. The workstations, as well as the control processors of the lsrs are 200 MHz Pentium Pro pcs running the NetBSD operating system. ssp is used as a signaling protocol to reserve resources for application level ows in this environment.
5.1 Host Implementation The ssp implementation architecture on hosts is as shown in Figure 11. ssp is responsible for setting up the control path so that the data path has the requisite qos. Applications are linked with a library which provides the ssp api. The ssp api is both simple and powerful, supporting calls both for issuing as well as receiving Setup requests. There are also special calls for setting up the third party reservations described previously. 18
5.1.1 Control and Data Paths in End Systems At the receiving end, the application issues a Setup request for associating a ow with a particular qos. The request also speci es the routing address towards which the request is to be relayed, which is usually the source of the ow. ssp is responsible for relaying the request to the rst hop router towards the routing address. When the Setup reaches the routing address, this end system programs its packet classi er and scheduler so that the data path with the speci ed qos is set up.
5.1.2 Implementation Issues in the Control Path One way to implement a signaling library is to make it an `in band' library, in which the the library issues ioctl() calls to set up state on sockets previously opened by the application. The advantage of this approach is that the set of packets for which the reservation applies, i.e., the lter, is already de ned. The other option, chosen for the ssp library implementation, is to have an `out of band' library, in which the library explicitly describes the lter, as well as the desired reservation. This has the crucial advantage of permitting third party reservations, which we wanted in our api. An added advantage is that with this design, we can implement the ssp protocol in user space. In fact, the current implementation is a user space daemon. The library communicates with the ssp daemon using reliable socket based communications which does away with the need for soft state within a system.
5.2 Router Implementation The router implementation is substantially the same as the host implementation, except that in this case several additional modules are exercised. The control and data paths in the router are as shown in Figure 12. When a router receives a Setup message, it rst goes through admission control. The admission control module is based on a simple peak bw allocator model, and ensures that the requisite bw in the desired service class is available at the interface the Setup message arrives. For a conventional router, this is followed by programming the packet classi er and scheduler. For an lsr, the switching fabric is instead programmed with the appropriate incoming and outgoing labels. The routing table is then looked up to determine the next hop towards the routing address, and the Setup is relayed to the next hop. These steps set up the data path in the router so that the incoming
ow gets the required qos.
5.3 Implementation Status The ssp implementation for both end systems and routers resides in the user space. The router implementation also incorporates the admission control and for ssp implementations on atm subnets, the switch control library within ssp. The switch control library can control atm switches from Fore Systems and Bay Networks. There is also a vc allocator which is used to allocate vcs to new reservations. Several multimedia applications like nv, vic, ttcp together with a proxy front end called stap have been modi ed to incorporate the ssp api. Using a menu driven interface, it is possible to move from the default path to the qos path 19
SSP daemon Admission Control 2) 3) Program switch Switch Control
SSP
User Space 5) Forward to next hop
1) Incoming Setup
UDP
Kernel
3) Install flow state in kernel
4) Lookup next hop to Routing Addr
Routing Table
IP
Packet Scheduler
Packet Classifier
Label Switched Router Control Flow
Network Driver
Data Flow
Figure 12: ssp Router Implementation and see the perceptible di erence in image quality with a mouse click. The performance of these multimedia applications matches that of native mode atm applications, for example, both nv and vic produce full motion 20 Mbps media streams, with the bottleneck being the media processing and image rendering, rather than the network.
5.4 Experimental Results The signaling throughput and latency were measured for the ssp implementation on NetBSD running on a 200 MHz Pentium Pro based platform. Our measurements indicate that the major overhead in setting up state in a router do not lie in the signaling protocol itself, but rather in the software and hardware components which need to be programmed. The signaling throughput of ssp was measured by creating a special application called the signal generator. The signal generator would create a sequence of alternating Setup and Tear requests with a speci ed qos and lter, and issue them to ssp via the ssp api at a speci ed rate. The throughput was measured by counting the number of signaling messages received at the next hop router. With the packet scheduler and classi er module and the switch control module bypassed, ssp could achieve a sustained throughput of 5600 signaling operations per second. However, throughput slumped to a few hundred signaling operations per second with these modules in place. This motivated us to measure the processing times of individual modules in detail. To isolate the running time of individual modules, the code from each module was extracted and exercised in a stand alone manner. Each code fragment was run 1000 times and the average running time calculated as shown in Table 1. We discuss each of the results below. 20
Module
Time (in ms)
Packet Scheduler and Classi er Module 2 Transmitting a signaling request to the Switch Control 1.5 Library Routing Table lookup 0.2 User to daemon latency (round trip) 0.1 Transmitting a udp packet 0.07 Host atm vc initialization 0.04
Table 1: Processing Overhead for Individual Modules in ssp
Packet Scheduler and Classi er Module The scheduler and classi er module supports two calls. One installs ow speci c state, and the other deinstalls that state. A test program was written to repeatedly install and deinstall state in the kernel. The average time taken for each call to the kernel resident trac control module is 2 ms. We are currently investigating this value to determine the bottleneck, since the overhead of making the system call is only in the order of microseconds.
Switch Control Library This measurement is of interest in label switched environments, where the switch control library is responsible for programming the switch to set up a label switched hardware path between an input port and an output port of the switch. In the current implementation of the Switch Control Library, the ssp daemon sends a request to the library which is in turn passed on to a vendor provided switch management script residing on a remote system. Since the switch management scripts take an unacceptably large amont of time (in the order of tens of milliseconds) to process each switch programming request, the Switch Control Library returns control to the daemon immediately after queueing the request with the management script. This accounts for the low gure of 1.5ms for the signaling setup time. Unlike the other measurements, this is not a reading which can be sustained for multiple back to back requests. Typically, most switches can support sustained signaling rates of only 30-100 requests/s. We are working on a switch control library for a new generation of gigabit atm switches which support switch programming in a few microseconds, which should substantially cut down on switch programming latencies.
Routing Table Lookup ssp needs to forward protocol messages to the next hop towards the routing address carried in the message.
The forwarding is done by querying the kernel resident routing table. The average time to query the table for an ip address is 0.2ms. To cut down on this overhead, the ssp implementation maintains a cache of the least recently invoked lookups. In case a new Setup message refers to the same routing address, the lookup is quickly satis ed through the cache. 21
api
to Daemon Latency
The ssp api is part of a library linked with a user application. Communications with the ssp daemon are via a reliable socket. The round trip latency to send a message from the library to the daemon and back was measured to be 0.1ms, yielding a one way latency of 0.05ms. Most api communication is one way, since ssp responds to Setup requests only in case of failure.
Packet Transmission The time taken to transmit a single udp packet from user space was measured by repeatedly sending a Setup message sized udp packet from a test application (ttcp). The application was able to send about 14000 packets/s, corresponding to a time of about 0.07ms per packet. atm vc
Since our label switching testbed is based on cell switching, it is necessary to initialize the atm driver in the end system each time a new label (vc) is assigned to a ow. The atm driver has a separate call to initialize a new vc. The time taken for this call is 0.04ms. The current measurements have been taken with no optimizations done to the Setup processing code. We expect these readings to come down substantially once the code is optimized. One simple optimization which will speed up Setup processing on refreshes is to lookup the state entry in the ssp database based on the label carried by the Setup message. This provides a single xed length O(1) lookup into the database, avoiding complex lter based lookups. This important bene t hold even for non label switched networks, since there is no reason non label switched networks cannot carry non zero labels in their signaling messages.
6 Related Work There have been several signaling protocols designed earlier, but none with the same scope and state management capabilities as ssp. In the Internet, st and st-2 [L. 95] were examples of a hard state approach to signaling in an IntServ environment, and proved unsuccessful because they depended on a virtual circuit connection oriented service, as opposed to the traditional ip datagram service. The mchip e ort [And91] speci ed a congram approach towards building of an Internet supporting resource reservation. A congram is a service primitive which combines the low overhead of datagram service with the guarantees of a connection oriented service. This scheme, while conceptually simple, su ers from the drawback of having to use a new congram Internet Protocol for data packets as well as for control packets. With the current datagram based Internet infrastructure rmly in place, we do not expect any other data protocol to succeed. The atm Forum has speci ed a signaling protocol - uni 3.1 for setting up a ow between two atm endpoints. The speci cation itself runs into more than 400 pages of closely packed text, and involves a hard 22
state approach to ow setup which requires extreme reliability in network and link hardware in order to function eciently. rsvp was the rst soft-state reservation protocol, and the rst to allow a default best e ort datagram path, but it su ers from the complexity and scalability issues listed earlier. ssp extends rsvp functionality in many ways by adding support for label switching and Di Serv, among other capabilities. In addition, ssp simpli es rsvp by eliminating one of the two passes and the heterogeneous styles of reservations. Recently, the yessir reservation protocol [Pin98] has been proposed as a simple substitute for rsvp. yessir is a sender oriented reservation protocol which reserves resources for rtp streams. Like ssp, it is single phase. Unlike ssp, however, yessir is limited to rtp based trac, and provides no support for label switching or Di Serv nets.
7 Conclusions and Future Work We have described the features of a simple and versatile signaling and state management protocol called ssp. Its operation has been illustrated through many examples. ssp contains a number of features not previously available in any single protocol which make it suitable for both signaling and management of IntServ, Di Serv and Label Switched networks. The features range from a single exible style of reservation to aggregate reservations to generalized lters to single pass operation to third party reservations. ssp is currently being used to setup reservations over an IntServ label switched prototype network testbed. We also propose to use ssp to manage a Di Serv testbed that we are developing, in the manner described in Section 4. Future work includes extensions to ssp to permit sender oriented reservations and duplex reservations. The switch control library will be written to work with a new Gigabit atm switch currently under development. In addition, we plan to extend ssp to propagate Ethernet addresses in addition to labels, so that ssp can be used to manage an Ether switched network.
References [A. 97] A. Viswanathan, N. Feldman, R. Boivie and R. Woundy. ARIS: Aggregate Route-Based IP Switching| Work in Progress. In draft- viswanathan-aris-overview-00.txt, March 1997. [And91] Anderson, Jim, Z. Dittia and G.M. Parulkar. Persistent connections in high speed internets. In Proceedings of the IEEE Globecomm'91, December 1991. [B. 97] B. Braden, L. Zhang, S. Berson, S Herxog and S Jamin. Resource Reservation Protocol (RSVP) { Version 1 Functional Speci cation. In RFC2205, September 1997. [Boy97] Jim Boyle. RSVP Extensions for CIDR Aggregated Data Flows | Work in Progress. In draft-ietf-rsvpcidr-ext-01.txt, December 1997. [Dec95] Decasper D., Waldvogel M., Dittia Z., Adiseshu H., Parulkar G. and Plattner B. Crossbow - A Toolkit for Integrated Services over Cell Switched IPv6. In Proceedings of the IEEE ATM'97 workshop, 1995. [ea96a] P. Newman et. al. Ipsilon Flow Management Protocol Speci cation for IPv4. In rfc1953, May 1996. [ea96b] Yakov Rekhter et. al. Tag Switching Architecture Overview|Work in Progress. In draft-rfced-info-rekhter00.txt, September 1996.
23
[Har98a] Hari Adiseshu and Subhash Suri. A Geometric Technique to Resolve Con icting Filters. In Work in progress, 1998. [Har98b] Hari Adiseshu, Guru Parulkar and Subhash Suri. A Simpli ed Reservation and State Setup Protocol. In http://dworkin.wustl.edu/h~ari/rsvp.ps, 1998. [L. 95] L. Delgrossi and L. Berger. Internet Stream Protocol Version 2(ST2) Protocol Speci cation - version ST2+. In RFC1819, 1995. [Pin98] Ping Pan and Henning Schulzrinne. YESSIR: A Simple Reservation Mechanism for the Internet. In NOSSDAV, 1998.
24
