end-to-end semantics of TCP in that packets corrupted while in the cache become ... recover separately from up and down link losses. One may argue that ...
The Use of a Proxy on Board the Satellite to Improve TCP Performance J. Stepanek, A. Razdan, A. Nandan, M. Gerla Computer Science Department, University of California Los Angeles Boelter Hall, Los Angeles CA, 90095 USA Abstract—High errors and high delays create well-known problems with TCP. One established approach to solve this problem involves dividing TCP connections into segments, or splitting the connection. This paper takes this approach one step further by exploring the use of a TCP proxy on board a satellite for the purpose of enhancing end-to-end TCP performance. We will show this approach yields a number of important advantages, especially for small, mobile terminals, multi-segment systems, and multicast applications. As part of this architecture, we introduce a method of TCP backpressure using the advertised window, which optimizes memory usage on the satellite. Using simulation, we demonstrate that performance may improve by as much as three-fold.
I.
INTRODUCTION
Satellites are a viable approach for delivering broadband service. Several satellite systems either currently operating [1,2] or in various stages of development [3-9] intend to support applications with increasing data-rates. Such systems can in fact augment terrestrial services like UMTS—especially in remote areas where terrestrial infrastructure remains prohibitively costly or practically impossible to deploy and operate. Of course, to achieve success, these systems must provide Internet services. While providing Internet services requires supporting Internet protocols in general, TCP has particular significance as the basis of the world-wide-web and many other important client-server applications. Unfortunately, TCP suffers from a number of well-documented performance problems when used with satellite links. The problem grows more severe with longer delay, more frequent errors and larger bandwidth. Many have explored solutions to these problems. Some of these solutions require various architectural modifications and enhanced infrastructure [10] while others involve modifications to operational protocol mechanisms [11]. In fact, the final systems will likely adopt a mixture of approaches depending upon each system’s intended use. In this paper, we focus upon using resources on board the satellite to improve end-to-end TCP performance. In particular, we propose to go one step beyond the standard “split-connection” approach by further subdividing end-toend application paths into separate uplink and downlink TCP connections. We will demonstrate that an on-board proxy approach provides a number of advantages, especially for scenarios that include GEO constellations and multiple satellite segments. To further study this architecture, we
M. Luglio Dipartimento di Ingegneria Elettronica, Università di Roma Tor Vergata Via del Politecnico 1, 00133 Rome, Italy employ the use of network simulation to demonstrate potential performance gains. The remainder of this paper is organized as follows. Section II describes the characteristics of satellites affecting TCP. Section III describes the on-board TCP proxy in more detail. Section IV describes the environment and scenarios used for simulating the on-board proxy. The results are discussed in Section V and the paper concludes with Section VI. II. TCP AND SATELLITES Large propagation delays severely impact TCP performance. Scenarios requiring the provision of broadband service exacerbate this problem, especially if geostationary constellations are utilized, due to the large bandwidth-delay product in these instances. These applications require a large TCP window for efficient use of the satellite link, and a satellite link subject to losses results in frequent “Slow Start” events, which significantly impact throughput. The combination of high-bandwidth, errorprone links, and long delays combine to severely degrade TCP performance. Consequently, applications requiring broadband service from a GEO satellite, for example, may encounter difficulty in using TCP effectively. The effects of link errors and high delays upon TCP are well documented [12-15]. Many have proposed solutions to the problems plaguing TCP performance over satellite [10,11,16]. These solutions may be broadly classified as either involving modified protocol mechanics or enhanced infrastructure and architecture. Changes to the TCP flow-control mechanisms like SACK [17] and Westwood [18] yielded some promising results. Still others have shown that employing architectural solutions such as proxies and gateways also improves performance [19]. In this paper, we focus upon the use of a so-called “split connection” [20,21] in which a separate TCP connection operates across the satellite link. Splitting requires additional TCP proxies or gateways at both the terrestrial uplink and downlink in addition to the application’s source and destination. Since its inception, TCP connection splitting has been further generalized to the use of a so-called “Performance Enhancing Proxy” or PEPs [10]. Between satellite gateways, the splitting approach often uses a modified transport protocol optimized for satellites. This transport may either be a modified TCP or an
altogether different protocol, for example, a reliable UDPbased transport protocol. III. USING AN ON-BOARD TCP PROXY In this paper, we propose a modification to the splitting TCP connections in which the path used by the application from source to destination is further subdivided into uplink and downlink connections across the satellite. This extra subdivision further reduces the size of the congestion window in the same way as in a split connection. Of course, this window reduction comes at the expense of memory and processing on board the satellite. Splitting also violates the end-to-end semantics of TCP in that packets corrupted while in the cache become unrecoverable. To help with this problem, the satellite acknowledges packets in the cache only after an initial successful data transmission on the downlink connection, that is, without waiting for the acknowledgment from the receiving end of the downlink connection. Consequently, only packets both lost on the downlink and subsequently corrupted in memory result unrecoverable packets and a connection failure. Since the allocation of on-board resources remains a prime consideration to satellite designers, the use of an onboard proxy requires careful consideration. Some potential advantages of this approach include the following. 1. Standard TCP Unlike the classical splitting approaches employing either a non-standard or non-TCP transport, an on-board TCP proxy improves performance using standard TCP implementations. Even so, TCP enhancements may further improve performance. Using a non-standard transport may introduce problems for networks with legitimate congestion and fairness issues. While congestion does not enter into scenarios with a single satellite hop, multi-hop or multisegment scenarios may benefit from TCP congestion control. Satellite gateways may also benefit from basic TCP fairness when serving many connections from multiple segments. 2. Simplified Terminal Design Moreover, adding transport resources to the satellite may reduce the cost and complexity of satellite terminals, both in terms of memory and software requirements. For example, a multi-segment system might service a large number of small terminals with the capability to access both GEO and LEO satellites. But these terminals may lack the resources required to include multiple transport protocol implementations, each carefully optimized for different segments. For example, with an on-board TCP cache a small terminal may access both LEO and GEO segments with a single TCP implementation. 3. Shadowing
Shadowing can cause serious problems with TCP performance. In particular, shadowing becomes critical for GEOs with mobility at higher latitudes. Using a separate error-control mechanism on the uplink and downlink also enhances the robustness of the transport. When both the uplink and downlink suffer from shadowing, a successful transmission may require to be unshadowed both on up and down links at the same time. Using an on-board cache increases the opportunities for success when both uplink and downlink suffer from intermittent shadowing. 4. Multicast Finally, using a TCP uplink or downlink allows for multicast on the opposing ground link or in the space segment. For example, TCP may operate on the uplink while a multicast downlink can take advantage of the satellite’s broadcast capabilities. This architecture requires a gateway to handle the transition from TCP to multicast; the multicast may also itself take advantage of the additional reliability provided by the split scheme, and thus recover separately from up and down link losses. One may argue that separate up and down link reliability can also be achieved by introducing an ARQ link layer on up and down links. This scheme however suffer from the major drawback of forcing ALL flows to undergo ARQ ,even the UDP real time flows which have their own built in CRC redundancy and thus do not require retransmissions. In fact, for VoIP applications retransmissions are harmful, especially over satellites. For these reasons, we think split TCP on the satellite is an attractive solution, and explore using an on-board TCP proxy to enhance performance. IV. SIMULATION ENVIRONMENT The simulations were performed using ns-2 (Network Simulator) [22] enhanced to provide better support for satellite environments as described in [23]. These enhancements include satellite handoff and satellite gateways along with a realistic channel model incorporating both shadowing and mobility. To study the split connection architecture proposed here, ns-2 was further enhanced with a new TCP forwarding agent. This agent receives data and acknowledgements from the uplink while sending and retransmitting packets on the downlink. To better use the fixed buffer on the satellite, the forwarding agent uses window advertisements on the uplink. The standard TCP implementations of ns-2 determine the receiver’s buffer size when forming the connection. The current ns-2 advertised window remains constant throughout the life of the connection, and the sender uses the minimum of the advertised window and the congestion window to control the rate of sending packets [24,25]. For our on-board cache, the forwarding agent uses
Beyond showing the effect of using an on-board forwarding agent, this graph clearly shows the effect that the size of the forwarding cache has upon the performance of TCP. The performance of TCP increases with the size of the forwarding agent’s cache in a regular linear fashion. And for larger sizes, TCP performance saturates at a stable value. Figure 2 shows similar results for TCP Westwood [18]. Here, using an on-board forwarding agent improves performance to a lesser extent for saturated cache sizes. This occurs as a result of TCP Westwood’s suitability for lossy links [23]. Consequently, using an on-board splitting has less impact and required larger caches.
500 450
Throughput (packets/s)
a conservative buffer management strategy to prevent dropping packets once they have successfully reach the satellite. Inspired by virtual-circuit networks [26], this strategy sets the advertised window on the uplink connection to one-third the total cache size. At the same time, the satellite limits window on the downlink connection to one-third the cache size. Thus, as packets become backlogged on the satellite, the advertised window decreases, resulting in backpressure on the uplink. For this purpose, the standard TCP implementation of ns-2 was modified to allow for a changing advertised window. Using a dynamic window allows for optimal use of the on-board cache while ensuring that the cache does not overflow and result in dropped packets. With these enhancements, several simulation scenarios were considered. In the first scenario, a TCP connection between two earth stations sends data using a single GEO satellite. Here, each link suffered from Packet Error Rates (PER) ranging from 10-4 to 10-2. These scenarios included both the case of using an on-board cache, or the split case, and the case of using a single connection across the satellite, or the non-split case. In the next set of scenarios, errors were applied only to either the uplink or the downlink, but not both. Again, these scenarios included both the split and non-split cases. In the final set of scenarios, the satellite terminal suffers from shadowing in addition to random packet losses. Here, terminals were placed at 20, 30 and 50 degrees latitude to study the effect of shadowing.
400 350
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
300 250 200 150 100 50 0 0
200
400
600
800
Cache Size (packets)
Figure 2 TCP Westwood Throughput vs. Cache Size
V. SIMULATION RESULTS A. Uniform Uplink and Downlink Errors As stated in the previous section, the first set of results were taken using a fixed PER on both the uplink and the downlink. Figure 1 shows the results for TCP NewReno [27] under various PERs. 500
Throughput (packets/s)
450 400 350
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
300 250 200 150 100 50 0 0
200
400
600
800
Cache Size (packets)
Figure 1 TCP NewReno Throughput vs. Cache Size
The relationship between performance and the size of the cache relates to the optimal window size for the link. Note that in all scenarios, both the uplink and downlink have 2 Mbps capacity and use 512-byte packets. Assuming a GEO one-way delay of 125 ms, the optimal window size for each link becomes 2 ⋅ 106 bps / (8 ⋅ 512 bits per packet) ⋅ 2 ⋅ 0.125 s, or 122 packets. In our case, performance levels once the size of the cache is three times the optimal window size, or 366. Recalling the conservative buffer management strategy, in this case the satellite uses one-third of the cache to store in-flight packets, and thus advertises an optimal window while using an optimal sending window on the downlink connection. The remaining third of the cache can store packets while applying backpressure. This strategy drops very few packets and accounts for the stability of the performance curves even for smaller caches. Figure 3 summarizes performance comparisons for several PERs between use of an on-board split connection and use of a single connection across the satellite. For the split cases, this chart assumes the saturation performance for caches of large enough size as described earlier. This chart shows that PERs of 10-2 yield the highest performance
500
Throughput (packets/s)
450 400 350
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
300 250 200 150
500 450
Throughput (packets/s)
gains from splitting—especially for TCP NewReno which as a three-fold increase. TCP Westwood performs better overall, but gains less from additional splitting.
400 350
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
300 250 200 150 100 50 0
Uplink
Downlink
Errored Channel
100 50
Figure 5 Westwood Throughput with Asymmetric Errors 0
NewReno
Westwood
TCP Version
C. Shadowed Downlink
Figure 3 Comparing Split (Saturated) and Non-Split Throughput
B. Asymmetric Uplink and Downlink Errors The next set of results consider cases in which errors occur only on either the uplink or the downlink, but not both. This follows a scenario in which one channel experiences fading while the other remains clear. Figure 4 shows TCP buffer-saturated throughput for uplink and downlink errors with NewReno. Likewise, Figure 5 shows TCP performance for uplink and downlink errors for TCP Westwood. Once again Westwood gains less from splitting but performs better overall. Here splitting shows the largest gains with downlink errors. Without downlink errors, the split connections behave similarly to a non-split connection. 450
Throughput (packets/s)
400 350 300
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
250 200 150 100 50 0
Uplink
Downlink
Errored Channel
Figure 4 NewReno Throughput with Asymmetric Errors
The previous section focused upon comparing the performance of on-board splitting with non-split connections while only considering random errors. However, by causing long bursts of error of random duration, shadowing effects may greatly reduce the performance of TCP. Thus, it is important to simulate onboard splitting in the presence of a shadowed channel. Figure 6 compares the performance between split and non-split (saturated) TCP connections for 20, 30 and 50 degrees latitude with various PERs. In these scenarios, the downlink moves along an “urban canyon” [23] at 5.0 m/s and experiences shadowing from simulated buildings. In this case, the on-board cache works well for shadowed channels. While TCP normally responds to random packet losses with Fast Recovery or its derivatives, persistent shadowing generally results in Slow Start. As a consequence, the split connections with a shadowed sender only perform Slow Start on the downlink, and the duration of Slow Start is smaller as a result of reduced RTT. The uplink gateway sends packets within the advertised window and does not perform Slow Start as frequently.
200
Throughput (packets/s)
180 160 140
Split; 0.01% PER Non-Split; 0.01% PER Split; 0.1% PER Non-Split; 0.1% PER Split; 1% PER Non-Split; 1% PER
120 100 80 60 40
[5] [6] [7] [8] [9] [10]
[11] [12] [13]
20
[14]
0 20
30
50
Latitude (degrees)
[15]
Figure 6 Comparing Split and Non-Split NewReno Performance with a Shadowed Downlink
[16]
VI. CONCLUSION
[17]
As the simulation results demonstrate, an on-board cache can in some cases significantly enhance TCP performance. The on-board cache works particularly well in cases of intermediate and high error-rates. Finally, the on-board cache works well for shadowed channels by reducing the dependencies between uplink and downlink and reducing the time required for Slow Start. We also find the TCP Westwood gains less from additional splitting but performs much better overall. To achieve the best results, the buffer size must be large enough such that the cache can accommodate both the downlink packets buffered for retransmission and the inflight uplink packets in case of outages on the downlink. In this paper, we employ a conservative buffer management strategy to minimize packets dropped on the satellite. In the future, we intend to explore alternative buffer management strategies for managing buffers between multiple connections.
ACKNOWLEDGMENTS The authors would like to thank T.J. Pannu, Chris Tang and Tak Kin Yung for their invaluable contribution to this work. REFERENCES [1]
[2]
[3] [4]
R. J. Leopold, “Low-Earth Orbit Global Cellular Communications Network”, Mobile Sat. Comm. Conf., Adelaide, Australia, 23 Aug. 1990. R. A. Wiedeman, A. J. Viterbi, “The Globalstar Mobile Satellite System for worldwide Personal Communications”, Proc. of 3rd Int. Mobile Satellite Conference (IMSC '93), June 1993, pp. 285-290. SPACEWAY, http://www.spaceway.com/ CyberStar, http://www.cyberstar.com/
[18]
[19]
[20]
[21]
[22] [23]
[24] [25] [26]
[27]
ASTRA, http://www.astra.lu/ Eutelsat, http://www.eutelsat.com/ SkyBridge http://www.skybridgesatellite.com/ EuroSkyWay, http://www.alespazio.it/program/tlc/eurosk/eurosk.htm Teledesic, http://www.teledesic.com J. Border, M. Kojo, J. Griner, G. Montenegro and Z. Shelby, “Performance Enhancing Proxies intended to mitigate link-related degradations”, RFC 3135, June 2001. M. Allman, D. Glover, L. Sanchez, “Enhancing TCP over Satellite Channels using Standard Mechanism”, RFC 2488, January 1999. C. Partridge, T. J. Shepard, “TCP/IP Performance over Satellite Links”, IEEE Network, September-October 1997, pp. 44-49. M. Allman (editor), “Ongoing TCP Research Related to Satellites”, RFC 2760, Feb. 2000. S. Dawkins, G. Montenegro, M. Kojo, V. Magret, and N. Vaidya, “End-to-end performance implications of links with errors”, RFC 3155, August 2001. M. Allman, C. Hayes, H. Kruse, S. Osterman, “TCP Performance over Satellite Links”, 5th Int. Conference on Telecommunication Systems Modeling and Design, 1997, pp. 1-13. I. F. Akyildiz, G.Morabito, S. Palazzo, “TCP-Peach: a new congestion control scheme for satellite IP networks”, IEEE/ACM Transactions on Networking Volume 9, Issue 3 (2001), Pages 307-321. M. Mathis, J. Mahdavi, S. Floyd, A. Romanow, “TCP Selective Acknowledgment Options”, RFC 2018, October 1996 C. Casetti, M. Gerla, S. Mascolo, M. Y. Sanadidi, and R. Wang, “TCP Westwood: Bandwidth Estimation for Enhanced Transport over Wireless Links,” In Proceedings of Mobicom 2001, Rome, Italy, Jul. 2001. J. Ishac and M. Allman, “On the performance of TCP spoofing in satellite networks”, Proceedings of Milcom 2001, Tysons Corner, McLean, VA, USA, October 28-31, 2001. A. Bakre and B.R. Badrinath, “I-TCP: Indirect TCP for mobile hosts,” Proc. 15th International Conference on Distributed Computing Systems (ICDCS), May 1995. T. Henderson and R. Katz, “Transport protocols for Internetcompatible satellite networks,” IEEE Journal on Selected Areas in Communications, Vol. 17, No. 2, pp. 345-359, February 1999. “Network Simulator (NS-2)”, http://www.isi.edu/nsnam/ns/ P. Loreti, M. Luglio, R. Kapoor, J. Stepanek, M. Gerla, F. Vatalaro, M. A. Vazquez-Castro, “LEO Satellite Systems Performance with TCPIP applications”, Proceedings of Milcom 2001, Tysons Corner, McLean, VA, USA, October 28-31, 2001, session U24. V. Jacobson, “Congestion Avoidance and Control”, Proc. of SIGCOMM '88, 1988, ACM. W. Stevens, “TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms”, RFC 2001, Jan. 1997. Kung, H. T., T. Blackwell and A. Chapman, “Credit-Based Flow Control for ATM Networks: Credit Update Protocol, Adaptive Credit Allocation, and Statistical Multiplexing”, Proceedings of ACM SIGCOMM '94 Symposium on Communications Architectures, Protocols and Applications, August 31-September 2, 1994, pp. 101-114. S. Floyd, T. Henderson, “The NewReno Modification to TCP's Fast Recovery Algorithm,” RFC 2582, April 1999.
