A Flow Control Algorithm for ACK-based Reliable

A Flow Control Algorithm for ACK-based Reliable Multicast Dah Ming Chiu, Miriam Kadansky, Joe Wesley Sun Microsystems Labs May 1, 1999

Abstract We introduce STBL, a flow control algorithm that uses both window and rate to regulate traffic with smoothed transmission and bounded loss, as well as to compete fairly with TCP traffic. In this paper, we evaluate how STBL behaves as part of a ACK-based reliable multicast transport protocol (TRAM), based on a simulation model. A library of standard reference scenarios is used to drive the simulation. The results are quite promising.

1.0 Introduction The primary benefit of reliable multicast is that it offers an efficient way of disseminating information to many locations (almost) simultaneously. Both error control and flow control in reliable multicast (RM) are difficult problems. The difficulty with error control is that the need to process ACKs (or NACKs) from multiple receivers can create an implosion at the sender as the number of receivers increases. There are many proposals for reliable multicast transport protocols, driven by different application requirements in the areas of: reliability, scalability and performance. These approaches have been reviewed and discussed in [SURVEY]. The problems with flow control are many-fold. Scalability is one of the problems: as the number of receivers increases, the range of suitable transmission rates diminishes and it becomes increasingly difficult to aggregate feedback signals from the receivers. Another problem is with fairness: it is strongly argued that multicast flows must be TCP-friendly which, depending on how religiously you conform, can make flow control more challenging. During periods of congestion, a multicast flow’s repair (retransmission) traffic may come from different nodes, making it harder for the sender take it into account. The following are some of the key factors affecting the design of the flow and congestion control algorithm: • If and how the sender gets feedback information from the receivers • Whether the control is window or rate based1 • Whether there are external knobs that are used to indicate application requirements and performance guarantees agreed to out-of-band • What the fairness goals are The answers to some of these questions depend on the algorithm used to support reliability; for example, whether it is ACK-based or NACK-based. The issue of fairness hinges on larger issues of network resource allocation in the Internet, as is discussed below. In an ACK-based protocol, there is regular feedback from the receivers to the sender for error control purposes. This feedback channel can be used to piggyback flow and congestion information as well, so it is natural to incorporate a flow control algorithm that responds to such feedback information continuously. In a 1

We assume this needs no definition. For example, TCP uses a window-based flow control scheme. Some of the transports proposed for ATM networks use rate-based flow control. 1

NACK-based protocol, there is no regular feedback from the receivers to the sender. This opens the door for the design to use a separate feedback channel that is specifically for flow and congestion control and it has to be justifiably efficient in its own right. This requirement has prompted work on feedback mechanisms based on representative receivers, and prescriptive flow control algorithms based on feedback of longer term network conditions sampling rather than incremental-adjustment flow control algorithms based on almost instantaneous feedback. Window-based flow and congestion control is very natural for an ACK-based protocol due to its simplicity. It interacts well with buffer management policies. Due to years of studies on TCP, there is also a wealth of understanding of how a window-based control algorithm works. The advantage of a rate-based control is that it maps well with application requirements and bandwidth allocation policies. Without regular feedback (ACKs), the control has to be rate-based2. Since the Internet supports primarily best-effort traffic, the issue of fairness is critical as it determines how network resources are allocated to competing users. The fairness of TCP has been studied by many, and over the years TCP has become the primary standard transport protocol as well as the most widely used protocol of the Internet. For these reasons, it has been strongly argued that RM flow control should be TCP-friendly [Mankin]. This is a good design principle. TCP-friendliness, however, should be considered as a qualitative guideline. Fairness of network resource allocation does not have a single straightforward theoretical formulation. Even TCP itself cannot always yield predictably fair resource allocation as TCP fairness defines3. Moreover, it is probably a good design principle to allow some externally configured parameters to help guide the flow and congestion control algorithm when necessary. In TCP’s case, one external parameter is the receiver’s window size determined based on available buffers. This is used to give an upper bound to the congestion window. TRAM is a reliable multicast transport protocol that uses a repair tree to deal with error control [TRAM]. This paper discusses the study of the flow control design for TRAM. The scheme adopted is based on using both window as well as rate for flow control. The use of rate helps to smooth (and shape) the senders’ transmission; the use of window helps to limit the packet losses. The control scheme is referred to as Smoothed Transmission with Bounded Losses (STBL). A primary goal of STBL is to be TCP-friendly. The basic ideas in STBL - the use of both window and rate to control the flow of traffic - are not specific to TRAM. They can be applied to other ACK-based protocols - multicast or unicast.

2.0 Review of RM Flow Control Approaches 2.1 TFMCC One approach for doing multicast congestion control is the TCP Friendly Multicast Congestion Control (TFMCC) proposal. This approach is based on the work by a number of researchers, see [Floyd], [Montgomery], [Bhattacharyya], [Strawman], [Whetten]. [Floyd] introduced the idea of using a TCP formula to set the transmission rate; [Montgomery] and [Bhattacharyya] described how variations of this idea is applied to multicast transport design. Later, [Strawman] unified several similar proposals based on the same approach. [Whetten] addressed some of the theoretical foundations for the TFMCC approach, such as the definition of fairness and the stability of the control system.

2

Without regular feedback, a popular form of control is based on stop-and-go, or also known as on-off control, which is a form of rate-based control as well. 3 At a recent research group meeting, after having finished doing lots of simulation of different RM and TCP flows, Mark Handley concludes semi-jokingly: "TCP is not TCP-friendly". 2

TFMCC is a rate-based flow control. The crux of this approach is to let the sender use a TCP formula that maps packet loss rate to throughput to determine the rate for transmission. Several TCP modeling studies have led to accurate TCP loss-to-rate formula [Mahdavi], [Mathis], [Padhye]. The claim is that the use of such a formula by the sender’s rate controller automatically guarantees that the (multicast) flow competes fairly with other TCP flows. In the multicast case, the application of this approach is more complicated by the fact that packet losses occur at the paths from the sender to the different receivers, and they may be quite different. TFMCC’s solution is to let all receivers measure the loss rate and other parameters in the TCP formula (such as the round trip time); and each calculates the suitable sending rate as if it is the only receiver. The sender then figures out the appropriate rate to use. One of the major advantages of TFMCC is that it is independent of the error control mechanism - whether ACK-based or NACK-based. (In either case, there still needs to be some filtering mechanism so that only the relevant feedback gets back to the sender). This de-coupling makes TFMCC suitable for consideration as a protocol-independent "building block" [Handley]. On the other hand, TFMCC is relatively complicated. The TCP-friendly formula requires measuring the loss rate and the roundtrip time, both of which are quite dynamic. If the measurements are based on very quick sampling, it may not be very representative; if they are based on a longer term sampling, it may reduce the scheme’s responsiveness. Whether the scheme can be made robust enough is still under research.

2.2 Window-based approach Several research studies have investigated adapting the TCP window-based flow control to reliable multicast, for example [Rhee], [Wang]. Rhee described a tree-based reliable multicast protocol, MTCP, which is similar to TRAM. Its windowbased flow control relies on SAs (equivalent to TRAM’s repair nodes) to work with the sender in establishing the values of current data in transit and congestion window. The SA’s also compute a Relative Time Delay for each acknowledged packet to establish a suitable retransmission timer and compute the round trip time at the sender. A TCP-like multicast transport was also studied in [Wang]. There is no tree-based repair and feedback - the scalability issue is not addressed. It contained several interesting ideas otherwise: • Proposed a weaker (perhaps more realistic) notion of TCP fairness - "essentially fair" • Studied a congestion control scheme that reacts to feedback signal probabilistically - a form of filter for redundant feedback for multicast transport

2.3 Theoretical Studies In [Golestani], he made several theoretical observations contrasted rate and window-based flow and congestion control schemes: • Even without dynamically adjusting the window size, a windowing scheme slows down the traffic rate as congestion increases, since the round trip time increases as well • There are two forms of fairness, one tries to achieve equal throughput at the bottleneck resource, and the other achieves relative throughput depending on the round trip time of the flows. He called the former rate-oriented fairness and the latter window-oriented fairness • The better way of implementing congestion control (in particular window based control) is to rely on receivers to collectively participate in the computation - he called this receiver-driven congestion control. Golestani’s observations summarize several key principles in designing rate and window based congestion control and provided us with inspiration.

3

3.0 Smooth Transmission and Bounded Loss (STBL) In this paper, we report the study of a flow control scheme that uses both rate and window to regulate transmission. The rate is used to ensure packets are sent in a smoothed fashion, and bandwidth utilization is fair compared to other competing traffic. The window is used to limit the amount of loss so that the sender does not get too far ahead of the receivers. The algorithm is based on heuristics and is relatively simple, as described below. The rate-based regulator uses a leaky-bucket algorithm. After a packet is transmitted, the regulator determines the time for transmitting the next packet as Next Time = Now + Packet Size / Rate By scheduling packet transmission this way, we expect the average achieved data rate to be approximately that of the rate we use to schedule transmission. In what follows, we use the term rate to refer to the rate we use to schedule transmission. This is also the rate we increase and decrease. The term "achieved rate" is used to mean what we measure that actual data rate (during a congestion window of packets) to be. The regulator also checks to see if the number of "outstanding" packets exceeds the current congestion window size. If so, the transmission is halted until new ACKs come in to reduce the number of outstanding packets to be smaller than the window size. ACKs are sent from receivers once for every ACK Window of packets. ACK Window is set to be half of the congestion window size4. Ideally, the congestion window is established commensurate to the length of the "pipe" and remains unchanged afterwards; the transmission rate is adjusted to compensate for the dynamic conditions of the network (as described in the original short note [STBL]). For example, the following methods can be used: • Based on the knowledge of the path of the connection, a network manager manually configures a fixed congestion window size before the start of the session. • Some sort of network path discovery probe is used to learn the path information for the connection, and uses the information to configure a suitable congestion window size. Without the help of such external configuration, the transport protocol must dynamically determine a suitable congestion window itself, which is discussed below. We considered various ways to determine a suitable congestion window size during a session. Originally, we tried to establish the window size at the beginning of a session, using an algorithm like TCP’s Slow Start. Since the network condition changes continuously, the window size determined during Slow Start would be a function of the network condition only for a particular moment. While this works for a single flow, it did not yield very good fairness among different STBL flows. This led us to consider other alternatives that dynamically adjust the congestion window size after Slow Start. All the variations explored share the basic algorithm as described below: Error correction part of protocol: • receivers send ACKs after receiving ACK Window of packets • ACK Window is set to 1/2 Congestion Window • receivers are informed of current ACK Window size in the data packet header Externally configured parameters: • minACKWindow - used to bound ACKWindow (hence congestion window size) 4

It is possible to make the congestion window > 2 times ACK Window, which we have not experimented with in this paper. 4

• • • •

maxACKWindow - same minDataRate - used to bound data rate maxDataRate - same highWaterMark - used by head to detect congestion based on queue size

Phase 1 (similar to TCP Slow Start): • start with configured/default a rate and window - the initial rate is set to the configured maxDataRate, and the initial ACK Window is set to the configured minACKWindow, whereas the congestion window is 2 times of the ACK Window. • increase congestion window at the rate of doubling it every 2 ACK Window of ACKs; • at the first reported loss, enter phase 2 Phase 2: • cut the rate by half, if there is more loss in the current ACK Window than last • recalculate Rate Increment as 25% of rate difference5 • at the first report of reduced losses, enter phase 3 Phase 3: • increase the rate by Rate Increment, when there is no congestion in a congestion window of packets; • decrease the rate by 50% if there is congestion in congestion window of packets; • measure achieved rate in each congestion window of packets; • (variation 1) recalculate the rate as its average with achieved rate; • (variation 2) adjust congestion window as follows: under congestion, decrease congestion window (by number of packets lost); else (no congestion) if achieved rate / rate < threshold, then increment congestion window The reason for variation 1 is to further improve the "smooth transmission" aspect of the algorithm. When there is a large discrepancy between the transmission rate and the achieved rate, then from a smoothening transmission point of view, it would not hurt to try a slower rate. We call this variation Smoothed rate. Variation 2 is to not trust the congestion window size determined in phase 1 and adjust it dynamically. When n packets are lost (by a receiver), it assumes that some buffer had an overflow of n packets and decreases the congestion window by n. When there is a large discrepancy between the transmission rate and the achieved rate, it assumes this is caused by the congestion window size being too restrictive6 in holding off packets, hence increases the window slowly (by 1 each time). We call this variation Dynamic window. The evaluation of the protocol design is based on simulation. Next we describe a series of reference simulation scenarios proposed by the RM Research Group which we adopted.

4.0 Simulation Scenarios In [RefSim], published by the Reliable Multicast Research Group, a series of scenarios are suggested to help study a RM congestion control scheme. While these scenarios are not suggested to be benchmarks, they do serve as a good starting point of having a few clearly defined cases for comparison.

5 6

We experimented with some different values for this multiplier, but used 25% for the reported results. We experimented with different value of threshold, but used 0.1 for the reported cases. 5

There are 4 sets of scenarios in this library, each aimed at examining the congestion control issue from a different angle.

4.1 Scenario 1 Scenario 1 consists of a 7-node network connecting 1 sender and 3 receivers as shown below:

Sender L1 L2

Receiver 1

L3

Receiver 2

Receiver 3

In this network, all links have bandwidth of 1 Mbits/sec and 10 msec delay. Three sub-scenarios are used to study sensitivity of a congestion control scheme to where packet losses occur: 1. The link L1 has 10% loss 2. The links L2 and L3 both have 10% loss 3. The link L3 has 10% loss Most congestion control algorithms measure congestion based on rate of packet loss. One hypothesis is that when multiple links have independent losses (instead of a single shared link having the same rate of loss) the congestion control algorithm would be fooled to react more, hence lowering throughput. This scenario is designed to study this case.

4.2 Scenario 2 Scenario 2 consists of a 9 node network, where there is a common bottleneck link shared by 3 sessions: 1 RM session from 1 sender to 2 receivers, and 2 TCP sessions each involving 1 sender and 1 receiver:

RM sender

TCP sender 1 TCP sender 2

RM receiver 1 20ms 1Mb/s RM receiver 2 TCP receiver 1 TCP receiver 2

As shown, the bottleneck link has bandwidth 1Mbits/sec. All other links have bandwidth 10Mbits/sec with 10msec delay. This scenario is designed to study if the RM flow competes fairly with TCP flows for network resources (the shared link).

6

4.3 Scenario 3 The network topology for scenario 3 is similar to scenario 2, with one bottleneck link shared by several flows. In this case, all flows are RM flows, except 1 flow is a CBR (Constant Bit Rate) flow7: RM receiver 1

RM sender 1 20ms 1Mb/s

RM receiver 2

RM sender 2

RM receiver 3

RM sender 3

CBR receiver

CBR sender

Again, the non-bottleneck links all have bandwidth of 10 Mbits/sec with 10 msec delay. This scenario is used to evaluate how RM flows share a bottleneck resource among themselves.

4.4 Scenario 4 Finally scenario 4 also has a similar network topology as scenario 2, but consists of n RM flows and n TCP flows, for some value8 of n. The bottleneck link’s bandwidth in this case is a multiple of n:

RM Sender 1 … Sender n

TCP Sender 1 … Sender n

RM Receiver 1 .. Receiver n

20ms n*0.5 Mb/s

5ms 10Mb/s

5ms 10Mb/s

TCP Receiver 1 … Receiver n

The buffer size at the sender end of the bottleneck link is also a multiple of n (10n). This scenario is used to see how the protocols (RM and TCP) fare as the number of flows increase. The detailed descriptions of these scenarios can be found in [RefSim]. Haldar from ISI actually provided some NS scripts for setting up these scenarios in NS [SceLib]. Its use simplified our tasks and should help to ensure we are setting up the experiments in a consistent manner as others.

7 8

When we tested this scenario, we did not include the CBR flow. We have tried setting n up to 10. 7

5.0 Simulation Tool Our simulation is written in NS [NS]. NS is a widely used public-domain simulation package. It comes with various building blocks, and a collection of built-in protocols. The building blocks allow you to easily create network components such as nodes, links and queues. The built-in protocols include TCP/RENO, TCP/SACK and several flavors of IP multicast routing protocols. These built-in protocols allow a researcher to easily simulate his protocol in scenarios various other protocols are also running. The implementation of a protocol’s simulation model is also made easy by the object-oriented nature of NS. The new protocol can be implemented by inheriting from a simpler protocol and extending its functions.

6.0 Simulation Model The congestion control algorithm is studied as a component of TRAM. So the simulation model contains (an abstract version of) most of the mechanisms in TRAM used in the actual operational phase. See [TRAM] for a more detailed technical description and [TRAMID] for a protocol specification. One of the salient features of TRAM is its ability to automatically form a repair tree on-the-fly. This aspect is not simulated using the NS model. The simulation model of TRAM is written as three NS agent objects: • TRAMSENDER • TRAMRECEIVER • TRAMREPAIRER Although these roughly correspond to the objects of TRAM, the simulation model is only an abstraction of the real protocol. The following description of the states and methods in the simulation model serves to give a flavor of the level of detail of the model. For a more rigorous specification of TRAM, the readers should consult an up-to-date version of [TRAMID]. The receiver has the following states: • Parent: where ACKs are sent to • Highest Received: highest sequence number received • Highest Consecutively Received: the highest sequence number such that this packet and all previous packets have been received • Highest Others Received: the highest sequence number received by other receivers under the same parent (this is learned indirectly from parent) • Last ACK: the sequence number contained in the last ACK • Last Sequence Number: the sequence number at which last ACK was sent • Last Loss: the number of losses reported in the last ACK. • ACK Window: the receiver’s knowledge of the current ACK window. The sender has the following states: • For sender functions:  Sequence Number: for next data packet  Stable: the highest sequence number which (and all previous) have been received by all receivers  Phase: switches between different flow control algorithms  ACK Window: number of data packets for which each receiver sends an ACK

8

•

     

Congestion Window: a multiple of ACK Window (2) Window Turn: variable to remember when to adjust window Window Increment: amount to increase congestion window size Rate Increment: amount to increase Data Rate by Rate Decrement: multiplier to decrease Data Rate by Last Rate Adjustment: sequence number at which last rate adjustment was made For Repair functions:  Members: a list of members  Acked(m): the highest sequence number ACKed by member m  Stable(m): the highest sequence number member m and all its children have ACKed  Repair TTL: TTL used for member repair  Inter-Retransmission Time: minimum time between retransmitting the same packet  Retransmission Rate: data rate used for scheduling retransmissions.  HCA: highest sequence number ACKed by all members (this is the minimum of Acked( m))

The receiver has two basic methods: • Receive: This method is called (by NS) whenever a new packet arrives. It updates the various receiver states. • SendACK: This method is called when the Receive method determines that an ACK should be sent: When new Sequence Number >= Last Sequence Number + ACK Window; Or when the repair head’s hello indicates a previous ACK was missing; Or when the last packet (indicated in header) of the session has been received Or when a timer expired (say more about this) The sender is more complicated. Its important methods are: • Queue packets: This method lets the data queue be incremented by some integer. • Begin: This method starts the session. • SendNext: This method sends a packet and based on the current rate schedules the next time SendNext is called. Since the sender is both a repairer as well as the data source, it first tries to send repair messages before sending new data messages. In case the congestion window is exceeded, SendNext will be woken up later as if a data packet has been sent. • SendHello: This method sends a hello message to members. This is relevant because it helps receivers discover lost ACKs. • Receive: This method processes ACKs from receivers; recalculate various error control and flow control states. Most of the flow and congestion control algorithms are implemented as subroutines of this method. • Get Statistics: This method is used to access some performance measurements that depend on the sender’s states. The repair head basically implements the receiver functions and a subset of the sender functions.

7.0 Simulation Results We present the simulation results for the two variations of STBL - Smoothed Rate and Dynamic Window in separate sections below. Recall Smoothed Rate uses the result of actual throughput monitoring to further smooth transmission; so we expect it perform better in terms of congestion control but perhaps not as well in terms of fairness. In the case of Dynamic Window, we expect it to do better in terms of fairness.

9

In all the figures below, we plot the rate (instantaneous bandwidth usage) for different flows against time. The way we derive the rate is by counting the number of acknowledged (i.e. reliably received) packets from the traces.

7.1 Smoothed Rate Frequently, the data rate that is used to schedule packet transmission is much higher than the achieved rate. The intuition is that in this case it is helpful (both to this flow as well as other flows sharing the same bottlenecks) to voluntarily adopt a lower rate - somewhere in between the scheduling data rate and achieved rate. While this smoothing may be good for congestion control, it is also likely to affect the resulting fairness of resource allocation. If the additive-increase and multiplicative-decrease distributed algorithm indeed yields fairness, any change to it would only worsen the fairness. One way to strike a compromise between these goals is to do this smoothing only when the ratio of achieved rate to scheduling data rate is lower than a threshold. In other words, when the achieved rate is not sufficiently different than the scheduling rate, follow the additive-increase and multiplicative-decrease algorithm for adjusting the rate. When the difference is very pronounced, which is the time it is likely to help easing congestion, then smoothen the rate.

7.1.1 Smooth Rate variation in Simulation Scenario 1 The following sequence of plots shows how this algorithm works for the threshold value of 0.1. The scenarios refer to those described in the reference scenario section. Recall the three variations for scenario 1 correspond to having lossy links at different part of the network. The plots show that our algorithm is not sensitive to such placement of lossy links. As the duration of the experiment increases, we can observe that the multicast flow is somewhat less aggressive than the TCP flow. This is intuitively as expected since we smooth the rate whereas TCP does not do that.

Scenario 1-1, Smoothed rate

rate (Mb/sec)

0.8 0.6 tcp 0

0.4

tram 1

0.2 0 1

2

3

4

5

time (second)

In figure above, the shared link L1 (of Scenario 1) has a 10% loss rate.

10

Scenario 1-2, Smoothed rate

rate (Mb/sec)

0.8 0.6 tcp 0

0.4

tram 1

0.2 0 1

2

3

4

5

6

7

8

9 10 11

time (second)

In the figure above, the links L2 and L3 (of Scenario 1) each has 10% loss rate.

Scenario 1-3, Smoothed rate

rate (Mb/sec)

0.8 0.6 tcp 0

0.4

tram 1

0.2 17

15

13

11

9

7

5

3

1

0 time (second)

In the figure above, only link L3 (of Scenario 1) has 10% loss rate.

7.1.2 Smooth Rate variation in Simulation Scenario 2 The result for Scenario 2 is less ideal. First, it should be explained that in this scenario, the three flows do not start at the same time. A TCP flow starts first; then 10 seconds later the TRAM flow starts; then 10 more seconds later the second TCP flow starts. The different starting times seem to have a significant influence on the fair allocation of the achieved throughput. Although first TCP has a head start and consumes all the bottleneck resources initially, when the TRAM flow comes along, its Slow Start (phase 1 of STBL) is more aggressive than the TCP window adjustment algorithm and hence overtakes the first TCP flow’s resource usage. Subsequently, the two TCP flows get somewhat more resources over time but cannot overtake the TRAM flow.

11

1.2 1 0.8 0.6 0.4 0.2 0

tcp 0 tcp 2

46

41

36

31

26

21

16

11

6

tram 1

1

rate (Mb/sec)

Scenario 2, Smoothed rate

time (second)

This result suggests more experimentation with the phase 1 of our algorithm to use a less aggressive window growth strategy.

7.1.3 Smooth Rate variation in Simulation Scenario 3 In Scenario 3, we have three TRAM flows, starting at 0, 10 and 20 seconds. This time, the first flow establishes a dominant share of the bottleneck resources and the late comers cannot overtake it. With closer examination, we notice the window size established by the first flow is significantly higher than the second and third flows. Since in this variation, the congestion window does not change dynamically (only rate does), the first flow holds on to its dominant usage of the bottleneck. This result is anticipated by use. It shows us that the window size is a very significant factor in determining the achievable throughput in a

Scenario 3, Smoothed rate

rate (Mb/sec)

2 1.5

tram 1

1

tram 2

0.5

tram 3 73

64

55

46

37

28

19

10

1

0 time (second)

competitive situation, and led us to consider the dynamic window variation.

7.1.4 Smooth Rate variation in Simulation Scenario 4 In Scenario 4, we have 10 TRAM flows, starting at random times between 0 and 10 seconds; and then 10 TCP flows starting at random times between 10 and 20 seconds.

12

Since this scenario involves 20 flows all together, we plotted the result in 2 figures: one containing the 10 TRAM flows, and the other containing 1 TRAM flow and the 10 TCP flows. (Including the TRAM flow in the second figure is to aid visualizing the comparison).

Scenario 4, Smoothed rate tram 0 rate (Mb/sec)

2.5

tram 1

2

tram 2

1.5

tram 3

1

tram 4 tram 5

0.5

tram 6 55

49

43

37

31

25

19

13

7

1

0 time (second)

tram 7 tram 8 tram 9

Scenario 4, Smoothed rate

tram 0 tcp 10

rate (Mb/sec)

2.5

tcp 11

2

tcp 12

1.5

tcp 13

1

tcp 14 tcp 15

0.5

tcp 16

time (second)

55

49

43

37

31

25

19

13

7

1

0

tcp 17 tcp 18 tcp 19

The result is quite interesting. Perhaps because a relatively large number of TRAM flows all started at times close to each other, none of them ever established a dominant position. As TCP flows are started, they are able to overtake the TRAM flows. In steady state (30-60 seconds), the TRAM flows achieved around 0.1 to 0.2 Mb/sec throughput; whereas the TCP flows achieved averaged around 0.5+ Mb/sec throughput. Given the scenario, ideally all flows should achieve close to 0.5 Mb/sec. While this result is somewhat disappointing, it is not completely unacceptable either. One of the problems with a large number of flows and the potential for heavy congestion is the "throughput-reduce-to-zero" problem. Because of the use of windows to bound losses, we seem to have avoided that. The achieved fairness among TRAM flows is also quite good.

13

7.2 Dynamic window For this variation of the control algorithm, we do not smooth the rate; but instead adjust the congestion window dynamically based on the feedback from the network. As we explained before, the STBL algorithm ideally uses a static window to limit packet losses, and relies on the dynamic rate adjustments to achieve fairness. Instead of assuming correct setting of a static window (requires either manual intervention or some new network path discovery mechanism) which is not realistic, we investigate simple dynamic window adjustment mechanisms. The window is first established in Slow Start (Phase 1) of our algorithm. We want to use this window, unless there is some strong evidence that this window size is affecting fairness and efficiency. Same as before, we detect this condition based on whether the ratio of achieved rate to the scheduling data rate is lower than a threshold. The following plots report the result of using 0.1 as the threshold.

7.2.1 Dynamic Window variation Simulation Scenario 1 For the first scenario, again we notice that the placement of the lossy link does not seem to be a factor. As the time duration of the experiment increases (form 5 to 17 seconds), it is clear that some steady state condition is reached in the first 5 or 6 seconds. In the steady state, the TRAM flow gets a higher share of the resources. But the ratio is not unreasonable9.

Scenario 1-1, Dynamic window

rate (Mb/sec)

1 0.8 0.6

tcp 0

0.4

tram 1

0.2 0 1

2

3

4

5

time (second)

In the figure above, the shared link L1 (of Scenario 1) has loss rate of 10%.

9

[RefSim] discusses some inefficiencies in TCP, and expects the RM protocol to outperform TCP somewhat. 14

Scenario 1-2, Dynamic window

rate (Mb/sec)

1 0.8 0.6

tcp 0

0.4

tram 1

0.2 0 1

2

3

4

5

6

7

8

9 10 11

time (second)

In the above figure, the links L2 and L3 (of Scenario 1) each has loss rate of 10%.

Scenario 1-3, Dynamic window

rate (Mb/sec)

1 0.8 0.6

tcp 0

0.4

tram 1

0.2 17

15

13

11

9

7

5

3

1

0 time (second)

In the above figure, the link L3 (of Scenario 1) has a loss rate of 10%.

7.2.2 Dynamic Window variation Simulation Scenario 2 In Scenario 2, again the different starting time here has a big effect on the achieved fairness. In what appears to be a steady state situation (36-50 seconds), the bottleneck shares achieved were unequal, but not totally unreasonable. Scenario 2 is the one that we are most unhappy with, for both the variations we have tried so far. We suspect the slow start (1st) phase is too aggressive; we will conduct further studies in this area.

15

Scenario 2, Dynamic window

rate (Mb/sec)

1.2 1 0.8

tcp 0

0.6

tcp 2

0.4

tram 1

0.2 46

41

36

31

26

21

16

11

6

1

0 time (second)

7.2.3 Dynamic Window variation Simulation Scenario 3 Scenario 3 is the one with 3 TRAM flows starting at different times. We observe that a steady state is reached quickly. As anticipated, the fairness level achieved is sufficiently better than the case with Smoothed Rate case.

Scenario 3, Dynamic window

rate (Mb/sec)

2 1.5

tram 1

1

tram 2 tram 3

0.5

73

65

57

49

41

33

25

17

9

1

0 time (second)

7.2.4 Dynamic Window variation Simulation Scenario 4 For Scenario 4, the result of using Dynamic Windows yielded similar result as the Smoothed Rate case. The TRAM flows again under-performed the TCP flows (in terms of throughput). The fairness among the TRAM flows is better. The overall fairness between TRAM and TCP is reasonable.

16

Scenario 4, Dynamic window

rate (Mb/sec)

tram 0 2.5

tram 1

2

tram 2

1.5

tram 3 tram 4

1

tram 5

0.5

tram 6 55

49

43

37

31

25

19

13

7

1

0 time (second)

tram 7 tram 8 tram 9

Scenario 4, Dynamic window

tram 0 tcp 10

rate (Mb/sec)

2.5

tcp 11

2

tcp 12

1.5

tcp 13

1

tcp 14 tcp 15

0.5

tcp 16

time (second)

55

49

43

37

31

25

19

13

7

1

0

tcp 17 tcp 18 tcp 19

8.0 Conclusion and Future Studies In this paper, we reported some results on a novel congestion control scheme that uses both window and rate to regulate transmission. The objective is to achieve smoothed transmission with bounded losses, as well as fair resource sharing with TCP traffic. We described our simulation model and used a set of reference simulation scenarios to evaluate the congestion control scheme in the context of a reliable multicast protocol. The result show some promise. The topic of congestion control of multicast flows is complicated and requires further studies. First, we feel there should be more reference scenarios. For example, there should be some scenarios that involve a larger number of links and receivers so that a hierarchical tree-based transport can be evaluated. Even with the given scenarios, we have not simulated a large enough parameter space for our protocol. There are also additional variations to the algorithms we still want to explore. Finally, we look forward to other simulation studies adopting the same scenarios so that simulation results can be compared. Another direction is to look into adopting STBL in unicast transport protocols, as a possible enhancement to TCP, for example.

17

References [Bhattacharyya] Supratik Bhattacharyya et al "The Loss Path Multiplicity Problem in Multicast Congestion Control", TR 98-76, Department of Computer Science, University of Massachusetts, August 1998. [Floyd] Sally Floyd and Kevin Fall "Promoting the Use of End-to-end Congestion Control in the Internet", Technical Report, Lawrence Berkeley Labs, 1998. [Golestani] Jamaloddin Golestani "Fundamental Observations on Multicast Congestion Control in the Internet", Bell Labs, Lucent Technology, paper distributed to RM Research Group in August 1998. [SceLib] "UCB/LBNL Network Simulator: Scenario Generation", http://www-mash.cs.berkeley.edu/ns/nsscengeneration.html [Handley] Mark Handley "The Building Block Approach to RM Standardization", presentation at the Reliable Multicast Transport BOF at the 44th IETF, Minneapolis, March 1999. [JRMS] Kadansky et al "The Java Reliable Multicast Service: A Reliable Multicast Library", Technical Report, Sun Labs, 1998. [Mankin] Allison Mankin et al "IETF Criteria for Evaluating Reliable Multicast Transport and Application Protocols", IETF RFC2357, June 1998. [Mahdavi] J. Mahdavi and S Floyd "TCP-Friendly Unicast Rate-based Flow Control", Technical note sent to the end2end-interest mailing list, 1997. [Mathis] M. Mathis et al "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm", ACM Computer Communication Review, 27(3): 67-82, July 1997. [Montgomery] Todd Montgomery, "A Loss Tolerant Rate Controller for Reliable Multicast", Technical Report: NASA-IVV-97-011, West Virginia University, August 1997. [NS] Kevin Fall and Kannan Varadhan "NS Notes and Documentation", UC Berkeley, LBL, USC/ISI, and Xerox PARC, January 1999. [Padhye] J. Padhye et al "Modeling TCP Throughput: A Simple Model and its Empirical Validation", Proceedings of ACM SIGCOMM 1998. [RefSim] Mark Handley, "Reference Simulations for Reliable Multicast Congestion Control Schemes", RM Research Group paper, published July 1998. [Rhee] Injong Rhee et al, "MTCP: Scalable TCP-like Congestion Control for Reliable Multicast", Technical Report TR-98-01, North Carolina State University, Department of Computer Science, January 1998. [STBL] Dah Ming Chiu, "Congestion Control Using Dynamic Rate and Window", Memo distributed to RM Research Group, November 1998. [Strawman] Mark Handley and Sally Floyd "Strawman Specification for TCP Friendly (Reliable) Multicast Congestion Control (TFMCC)", Memo to Reliable Multicast Research Group, December 1998.

18

[TRAM] Dah Ming Chiu et al "TRAM: A Tree-based Reliable Multicast Protocol", Technical Report, Sun Labs, 1998. [TRAMID] Miriam Kadansky et al "Tree-based ReliAble Multicast protocol (TRAM)", Internet Draft, draft-kadansky-tram-00.txt, October 1998. [SURVEY] Brian Levine and Garcia-Luna-Aceves "A Comparison of Known Classes of Reliable Multicast Protocols", Computer Engineering Department, UCSC, 1997. [Vicisano] Lorenzo Vicisano et al, "TCP-like Congestion Control for Layered Multicast Data Transfer", University College London, and University of Pisa. [Wang] Huayan Amy Wang and Mischa Schwartz, "Achieving Bounded Fairness for Multicast and TCP Traffic in the Internet", ACM SIGCOMM, 1998 [Whetten] Brian Whetten and Jim Conlan "A Rate Based Congestion Control Scheme for Reliable Multicast", Technical White Paper distributed to RM Research Group, October 1998.

© Copyright 1999 Sun Microsystems, Inc. All rights reserved. TRADEMARKS Sun, Sun Microsystems, and Java are trademarks or registered trademarks of Sun Microsystems, Inc.

19