TCP PERFORMANCE USING SPLITTING OVER THE SATELLITE LINK M. Luglio (1), J. Stepanek (2) and M. Gerla (2) (1)
Dipartimento di Ingegneria Elettronica, Università di Roma Tor Vergata Via del Politecnico 1, 00133 Rome, Italy E-mail:
[email protected] (2)
Computer Science Department, University of California Los Angeles Boelter Hall, Los Angeles CA, 90095 USA E-mail:
[email protected],
[email protected] ABSTRACT
Satellites represent a well-assessed approach for delivering broadband service. Several satellite systems either currently operating or in various stages of development intend to support applications with increasing data-rates. Of course, to achieve success, these systems must provide Internet services requiring the support of Internet protocols, especially TCP. Unfortunately, TCP suffers from a number of problems when used with satellite links due to high errors and high delays. The problem becomes more severe if, using GEO architectures, longer delay and more frequent errors occur with larger bandwidth. One possible approach to solve this problem foresees dividing TCP connections into segments, or splitting the connection. This paper takes this approach one step further by exploring the use of a TCP proxy on board a satellite for the purpose of enhancing end-to-end TCP performance, subdividing end-to-end application paths into separate uplink and downlink TCP connections. To this aim we will focus upon using memory resources on board the satellite. We will demonstrate that an on-board proxy approach provides a number of advantages, especially for scenarios that include GEO constellations, multiple satellite segments and multicast applications. To further study this architecture, we employ the use of network simulation to demonstrate potential performance gains. Introduction The idea of ubiquitous broadband access to the Internet continues to expand. Many people have grown to depend upon the Internet and wish to extend this access to other areas of their lives. Satellites provide one viable approach for delivering broadband service. Several satellite systems either currently operating or in various stages of development intend to support applications with increasing data-rates. Of course, to achieve success, these systems must support TCP-based applications. TCP has significance as the basis of the world-wideweb and many other important client-server applications. Unfortunately, TCP suffers from a number of problems when used with satellite links. The problems grow more severe with longer delay, more frequent errors and larger bandwidth. Some proposed solutions to these problems require various architectural modifications and enhanced infrastructure while still others involve modifications to the operational protocol mechanisms. In this paper we propose and explore the adoption of the “splitting” concept to the satellite link, separating the up and down links with a proxy on board the satellite. In particular, section 1 details the problems of TCP with satellites, section 2 describes the operations of the on-board forwarding agent and section 3 describes the simulation environment. Section 4 presents the simulations results and the paper concludes with section 5. 45
1 TCP and Satellites TCP provides a reliable stream of data when using any communication media, including satellites. Understanding some of TCP’s features will help in assessing TCP performance in satellite environments. Several properties of satellite links can degrade TCP performance [1]. First of all, large delays lengthen the duration of the “Slow Start” interval because the Slow Start process depends upon the Round Trip Time (RTT) of the connection [2][3]. Moreover, the optimal size of the window is proportional to both RTT and the maximum sustainable data-rate of the connection. For high RTT and high data-rate, the optimal window size becomes very large. Larger windows further increase the length of “Slow Start”. Larger RTTs also affect the rate at which the window grows during congestion avoidance. During this phase, the window increases by approximately one segment for every RTT. In the error-free case, this has little impact on steady-state performance. But in cases of more frequent losses, this behavior tends to decrease performance. The lossy characteristic of the link creates more problems. The flow-control mechanism of TCP relies upon lost packets as indicators of congestion. While this assumption holds true for wired networks, it fails for many wireless links including satellites. So when the TCP connection experiences relatively frequent losses from link-level errors, TCP performance suffers as a result. 1.1
Proposed solutions
Some solutions involve changes to the protocol mechanisms to accommodate the properties of satellite links [4]. These modifications change the basic error-control and flow-control strategies to improve performance. And sometimes these solutions apply to a wider range of environments beyond satellites. Another class of solutions requires changes to the architecture of the network. In these cases, intermediaries perform processing on behalf of TCP endpoints to the greater benefit of performance. One such technique involves subdividing connections into terrestrial and space segments, or “splitting” the connection. Changes to the TCP flow-control mechanisms like NewReno [5] and Peach [6] offer examples of protocol enhancements. SACK [7] also limits waste of satellite resources upon retransmissions by changing TCP’s error-control mechanism. [8] summarizes a great deal of these efforts in the context of satellites, some of which apply to non-satellite environments as well. Along these lines, at UCLA a modification to the Fast Recovery algorithm was developed, called TCP Westwood [9]. A TCP Westwood sender continuously monitors the effective rate of its connection using ACK interarrival times. This estimation may in fact be lower than the current transmission rate as a result of packets lost at the bottleneck link. The Westwood sender corrects to match the estimated rate by adjusting its window, and thus eliminates losses from congestion. Since Westwood uses estimated bandwidth instead of packet-loss feedback, this scheme represents a significant departure from other flow-control schemes. As previously stated, this proves important for satellite links where lost packets are not a reliable indication of path congestion. As an example of a recently developed satellite specific enhancement, TCP Peach replaces Slow Start and Fast Recovery with Sudden Start and Rapid Recovery. Both Sudden Start and Rapid Recovery use expendable “Dummy’’ packets to probe the characteristics of the network. These packets are sent with a low priority so they do not create congestion. While these packets are not recovered when lost, when acknowledged, the TCP sender increases the window in the manner of acknowledged data. This increases the rate of window expansion when compared with Slow Start and Rapid Recovery. TCP Peach requires that all routers properly treat the IP Precedence field to limit the impact of the Dummy packets.
46
In addition to protocol enhancements, a number of architectural techniques address TCP’s satellite problems. These techniques have been generalized as so-called “Performance Enhancing Proxies” or PEPs [10]. One PEP approach involves segmenting, or “splitting” the TCP connection into satellite and non-satellite segments [11][12]. Splitting uses TCP gateways on ground nodes with satellite access in addition to the application’s source and destination. These satellite gateways use separate connections that operate exclusively on the satellite segment. Between satellite gateways, splitting often uses a transport protocol optimized for satellites. This transport might in fact be either a modified TCP protocol or an altogether different protocol, for example, a reliable UDP-based transport protocol. A related PEP technique involves surreptitiously “capturing” a TCP connection and intelligently processing data packets and acknowledgments. Because the gateway in this case acknowledges data on behalf of the TCP receiver, this approach is commonly referred to as “spoofing” [13][14]. Spoofing divides the connection without the explicit knowledge of the endpoints, and thus processes TCP packets and introduces acknowledgments transparently. In this paper, we investigate an architectural solution along the lines of splitting. In this case, resources on board the satellite improve end-to-end TCP performance. Specifically, we consider adding transport-layer services on the satellite to help with TCP performance and relieve some of the burden of ground nodes. In this case, terrestrial nodes with the capability to both send and receive from the satellite each operate separate TCP connections and the satellite forwards data between connections. Thus, satellites act as application-level gateways between TCP endpoints. This extends the splitting concept by further subdividing the path between TCP endpoints into separate ground-to-space components. Anyway, when allocating resources for transport on board, the cost vs. performance trade-offs deserve careful consideration.
2 The Operation of the On-Board TCP Proxy We propose to extend TCP connection splitting by further subdividing the end-to-end path into separate segments between earth and space. Like canonical splitting, this architecture includes two terrestrial satellite gateways. Unlike normal splitting, each gateway maintains a separate connection with the satellite rather than a single connection with each other. In this context, we refer to the connection carrying data from the ground to the satellite as the “uplink” connection and the connection carrying data from the satellite to the ground as the “downlink” connection. In practice, each connection can send data in either direction and thus act as both uplink and downlink connections. To simplify analysis, we first consider only the cases in which data packets flow in one direction from earth to space and back to earth again, and acknowledgments flow in the reverse. This extra subdivision further reduces the size of TCP’s window in the manner of a split connection. But the reduction in window size comes at the expense of memory and processing on board the satellite. Moreover, like canonical splitting, on-board splitting violates the end-to-end semantics of TCP in that packets corrupted while in the gateway’s cache remain unrecoverable. To help with this problem, the satellite acknowledges packets received on the uplink connection only after the first successful data transmission on the downlink connection, that is, without waiting for the acknowledgment from the receiving end of the downlink connection. And the satellite uses the TCP checksum or some other method to ensure that outgoing packets are not corrupted. Consequently, only packets both lost on the downlink and subsequently corrupted in memory result in unrecoverable packets. In the semantics of TCP, this would represent an effective connection failure. Allocation of on-board processing and memory resources remains a prime consideration to satellite designers. So the use of an on-board proxy as described here deserves careful
47
consideration. To address this, we list the following potential advantages of using an on-board transport service beyond simple window reductions. 1. Standard TCP—The canonical splitting approach normally employs either a non-standard or non-TCP transport protocol between satellite gateways. In contrast, the on-board transport improves performance using any TCP implementations and TCP enhancement will only further improve performance. Typically, non-standard transports use an optimistic approach to flow-control. But the satellite network may also benefit from the TCP approach to congestion. A future satellite system incorporating multiple-hops (and orbits) with packet switching and routing can benefit from standard TCP mechanisms. 2. Flexible Terminal Design—Using a standard TCP won’t impact terminal design. By splitting the connection in space, disparate terminal types can still communicate even though they use different transports. In this case, a disadvantaged terminal may use the same TCP stack for all connections, roaming between wired, wireless, and satellite segments. And this disadvantaged terminal might benefit from an advantaged gateway optimized earth-to-space transport. 3. Shadowing—Shadowing becomes critical for GEOs at higher latitudes providing mobile services. This in turn causes serious problems with TCP performance. To counteract shadowing, using independent error-control mechanisms between earth-to-space connections enhances the robustness of the transport. Normally, a successful transmission requires both up and down links remain unshadowed. But using an on-board transport increases opportunities for success. And independent on-board transports recovers from shadowed losses more quickly. 4. Multicast —Finally, using TCP on the uplink or the downlink allows for multicasting on the opposing ground link or in the space segment. For example, TCP may operate on the uplink while a multicast downlink takes advantage of the satellite’s broadcast capabilities to serve multiple terminals and additional satellites.
3 Simulation Environment We have conducted our exploration of on-board splitting using the well-known ns-2 (Network Simulator) package [15]. While the simulation tool already includes support for satellites, we have added several enhancements to the satellite channel model by incorporating mobility and shadowing. Land Mobile Satellite Channel Model—In this paper, we use a physical-statistical land mobile satellite channel model as described in [16]. This model uses the geometrical projections of buildings surrounding the mobile terminal. In the model, height and width statistical distributions describe these projections [17][18], and the existence or absence of the direct ray defines the state of the channel. This is either line-of-sight (LOS), when a ray exists, or shadowed, when no ray exists. On-Board Forwarding Agent Model—In this study, we primarily focus upon splitting TCP connections using on-board satellite resources. To simulate this architecture, ns-2 was further enhanced with a new TCP forwarding agent. When attached to a satellite, this agent receives data and acknowledgements from the uplink connection while sending and retransmitting packets on the downlink connection. Because an unrestrained sender will quickly overflow the cache space, the forwarding agent limits the advertised window on the uplink connection to one-third of the total space in the cache, reducing this further as the cache fills. At the same time, the satellite limits the window on the downlink connection to one-third of the total space in the cache, limiting the amount of unacknowledged data sent on the downlink connection. For this purpose, the standard TCP implementations of ns-2 were modified to allow for an advertised window that changes over the life of the connection. Limiting windows to one-
48
third of the total cache space allows the satellite to store the “in-flight” packets on the uplink and downlink while leaving one window of additional space for backpressure to take effect.
4 Simulation Scenarios The results of our simulations fall into four broad categories: 1. Single-Hop Unshadowed GEO: Two terminals connect via single GEO satellite. 2. Single-Hop Unshadowed double GEO: Two terminals connect via two GEO satellites. The GEOs communicate using an ISL. 3. Single-Hop Source-Shadowed GEO: As in the first case, but the TCP sender suffer from shadowing as described above. 4. Single-Hop Sink-Shadowed GEO: As in the first case, but the TCP receiver suffers from shadowing. In these experiments, the single-hop scenarios include two terminals with the TCP sender in New York and the TCP receiver in San Francisco. The GEO satellite resides at –96W degrees longitude. In the double-GEO cases the TCP sender lies in Rome with the TCP receiver in Los Angeles. In this case, one GEO resides at –95W and one at zero degrees. The shadowed scenarios use the Land Mobile Satellite Channel as described previously. Here, the mobile terminal moves at 2 m/s. In all scenarios, including the shadowing scenarios, both uplink and downlink suffer from uniformly distributed Bit Error Rates (BER) of 10-6, 10-7 or 10-8. These bit errors were applied independently to both uplink and downlink for both data and acknowledgments. In each scenario, we have considered the “goodput” of both TCP NewReno and TCP Westwood with NewReno enhancements. For the unshadowed cases, the throughput measures were determined using 100 FTP sessions lasting 31 seconds each. Because shadowing introduces greater statistical variation, the shadowed cases use 1000 FTP sessions lasting 31 seconds each. All TCP connections used 1500-byte packets and the TCP window sizes were not limited by the buffer space of the source and sink nodes. 4.1
Single-Hop Unshadowed GEO
Figure 1 includes results for the case without shadowing. Here, losses occur as a result of the uniformly distributed BER. In part due to the low capacity of the link, TCP achieves reasonable performance even when faced with the relatively high error rate of 10-6. For the “split” cases, TCP performance increases with the size of the forwarding cache until reaching a saturation point when the cache does not present a limiting factor. The relationship between performance and the size of the cache relates to the optimal window size for the link. In this scenario, both the uplink and downlink have 1 Mbit/s capacity and use 1500-byte packets. Assuming a GEO one-way delay of 125 ms, the optimal window size for each link becomes 1·106 bit/s/(8·512 bits per packet)·2·0.125 s, or 21 packets. In our case, performance levels once the size of the cache rises above three times the optimal window size, or 63. Figure 2 summarizes the simulation results for the unshadowed single-hop case. For the “Split” case, the forwarding agent used a “saturated” cache size, that is, a cache large enough for the performance to stabilize as seen in Figure 1. From this chart, splitting generally enhances performance especially with increasing bandwidth and error rates. In some cases, splitting achieves over a three-fold increase. Overall, Westwood benefits less from splitting and performs better than NewReno.
49
TCP Performance Unshadowed Geo
TCP Throughput Unshadowed GEO; 10-6 BER; 1 Mb Capacity
2.5
0.6
BER 1.00E-08 BER 1.00E-07
Throughput (Capacity Normalized)
BER 1.00E-06
Throughput (Normalized against 1 Mb)
0.5
0.4
0.3
2
1.5
1
0.5
0.2
50
100
150
200
250
Figure 1 Throughput vs. Cache Size for Unshadowed, 10-6 BER, 1 Mbit/s.
Split
Split
NonSplit
Split
d tw es
W
NonSplit
W
es
tw
oo
d
no
oo
no N
ew
ew
Re
oo tw es
N
W
W
NonSplit
Re
d
d
no
oo
Re
tw
ew
Re N
es
d
no
d
oo tw
N
es
ew
no
oo W
Re
W
es tw
d
no Re N
ew
ew
d
oo es
N
W
W
es
tw
oo tw
Re
Re no
ew
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
1Mb
1Mb
1Mb
1Mb
2 Mb
2 Mb
2 Mb
2 Mb 10 Mb 10 Mb 10 Mb 10 Mb 20 Mb 20 Mb 20 Mb 20 Mb
300
Cache Size (1500-Byte Packets)
4.2
N
ew N
NewReno Non-Split 0 0
no
0 Westwood Split Westwood Non-Split NewReno Split
0.1
Split
NonSplit
Figure 2 Performance for the Unshadowed Scenario
Single-Hop Unshadowed Double GEO
Figure 3 shows TCP throughput for the Single-Hop double GEO scenario. Here, with increased delay, the connection achieves lower throughput overall and performance only levels with much larger cache sizes. Splitting also dramatically increases performance for saturated cache sizes. Figure 4 summarizes the results for the Single-Hop Double GEO cases. For longer delays, TCP Westwood improves performance, with respect to the non split case, and outperforms NewReno substantially more than in the single-hop single GEO case. This holds especially true for large bandwidths. TCP Throughput Unshadowed Single Hop Double GEO; 10-6 BER; 1 Mb Capacity
TCP Performance Unshadowed Single Hop Double Geo
0.45 BER 1.00E-08
2
BER 1.00E-07 BER 1.00E-06
Throughput (Capacity Normalized)
0.4
0.3
0.25
0.2
1.5
1
0.5
0.15
50
100
150 Cache Size (1500-Byte Packets)
200
250
300
Figure 3 Throughput vs. Cache Size for the Unshadowed Single-Hop Double GEO, 10-6 BER, 1 Mbit/s. 4.3
d
d
oo tw es W
W
es
tw
Re
oo
no
no Re ew N
N
ew
d
d W
es
tw
tw
oo
oo
no Re ew
es
N
W
ew
tw es
N
W
Re
oo
no
d
d
no
oo tw es W
N
ew
Re
Re
no
d tw
ew
es W
N
es
tw
oo
oo
no N
ew
Re ew N
NewReno Split NewReno Non-Split
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
1Mb
1Mb
1Mb
1Mb
2 Mb
2 Mb
2 Mb
2 Mb
10 Mb
10 Mb
10 Mb
10 Mb
20 Mb
20 Mb
20 Mb
20 Mb
0 0
Re
no
Westwood Split Westwood Non-Split
d
0
0.1
0.05
W
Throughput (Normalized against 1 Mb)
0.35
Figure 4 Performance for the Unshadowed Single-Hop Double GEO
Single-Hop Source and Sink Shadowed GEO
Figure 5 summarizes the results for the source-shadowed terminal, that is, when the TCP senders moves and suffers from shadowing. Figure 6 summarizes the results for the sinkshadowed case, that is, when the TCP receiver suffers from mobility and shadowing. These results indicate that sink-shadowing causes more harm to TCP than source shadowing. This may be explained by the fact that packets lost from sink shadowing consume transmission resources and satellite buffer space, while packets lost dues to source shadowing do not travel on the downlink or consume space on the satellite. Thus, the relative improvement of splitting appears consistent between source- and sink-shadowed TCP connections.
50
TCP Performance Sink-Shadowed Geo
TCP Performance Source-Shadowed Geo
1
1.6 BER 1.00E-08 BER 1.00E-07 BER 1.00E-06
BER 1.00E-08 BER 1.00E-07 BER 1.00E-06
0.8 Throughput (Capacity Normalized)
1.2
1
0.8
0.6
0.4
0.7 0.6 0.5 0.4 0.3 0.2 0.1
0.2
d
d
no
oo
oo
Re
W
es
tw
tw
ew
es
N
W
d
no Re ew N
tw W
es
tw es W
ew
oo
d
no
oo
no N
N
ew
Re
oo tw es W
Re
d
d
no
oo
Re
tw
ew N
es
Re ew N
W
d
no
d
oo W
es
tw es
tw
oo
no
Re N
ew N
ew
Re
d
d
oo
oo tw
tw es
es W
N ew Re no
W
d
N ew Re no
d
oo
oo
tw
tw
es
es
W
N ew Re no
W
N ew Re no
d
d
oo
oo tw
tw es
es W
N ew Re no
W
N ew Re no
d
d
oo
oo tw
tw es
es W
N ew Re no
W
N ew Re no
no
0
0
W
Throughput (Capacity Normalized)
1.4
0.9
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
1Mb
1Mb
1Mb
1Mb
2 Mb
2 Mb
2 Mb
2 Mb
10 Mb
10 Mb
10 Mb
10 Mb
20 Mb
20 Mb
20 Mb
20 Mb
Figure 5 Performance for the SourceShadowed
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
Split
NonSplit
1Mb
1Mb
1Mb
1Mb
2 Mb
2 Mb
2 Mb
2 Mb
10 Mb
10 Mb
10 Mb
10 Mb
20 Mb
20 Mb
20 Mb
20 Mb
Figure 6 Performance for the SinkShadowed
5 Conclusion In this paper we have examined the idea of adding transport services to the satellite in order to improve the end-to-end performance of TCP over satellite links. We found that the “splitting” of TCP connections into separate uplink and downlink components yields significant performance improvements. Moreover, as the connection suffers from additional problems from mobility and multiple satellites, the on-board cache makes a greater impact. References [1] C. Partridge, T. J. Shepard, “TCP/IP Performance over Satellite Links”, IEEE Network, September-October 1997, pp. 44-49. [2] V. Jacobson, “Congestion Avoidance and Control”, Proc. of SIGCOMM '88, 1988, ACM. [3] W. Stevens, “TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms”, RFC 2001, Jan. 1997. [4] M. Allman, D. Glover, L. Sanchez, “Enhancing TCP over Satellite Channels using Standard Mechanism”, RFC 2488, January 1999. [5] S. Floyd, T. Henderson, “The NewReno Modification to TCP's Fast Recovery Algorithm,” RFC 2582, April 1999. [6] I. F. Akyildiz, G.Morabito, S. Palazzo, “TCP-Peach: a new congestion control scheme for satellite IP networks”, IEEE/ACM Transactions on Networking Volume 9, Issue 3 (2001), Pages 307-321. [7] M. Mathis, J. Mahdavi, S. Floyd, A. Romanow, “TCP Selective Acknowledgment Options”, RFC 2018, October 1996 [8] M. Allman (editor), “Ongoing TCP Research Related to Satellites”, RFC 2760, Feb. 2000. [9] S. Mascolo, C. Casetti, M. Gerla, S. S. Lee, and M. Sanadidi, “TCP Westwood: congestion control with faster recovery”, UCLA CS-Technical Report #200017, 2000. [10] J. Border, M. Kojo, J. Griner, G. Montenegro and Z. Shelby, “Performance Enhancing Proxies intended to mitigate link-related degradations”, RFC 3135, June 2001. [11] T. Henderson and R. Katz, “Transport protocols for Internet-compatible satellite networks,” IEEE Journal on Selected Areas in Communications, Vol. 17, No. 2, pp. 345359, February 1999. [12] M. Karialiopoulos, R. Tafazolli, B. G. Evans, “TCP Performance on Split Connection GEO Satellite Link”, 19th AIAA International Communications Satellite Systems Conference, AIAA19, session 16, vol. 2, April 17-20, 2001, Tolouse, France. 51
[13] J. Ishac and M. Allman, “On the performance of TCP spoofing in satellite networks”, Proceedings of Milcom 2001, Tysons Corner, McLean, VA, USA, October 28-31, 2001. [14] A. Bakre and B.R. Badrinath, “I-TCP: Indirect TCP for mobile hosts,” Proc. 15th Intern. Conference on Distributed Computing Systems (ICDCS), May 1995. [15] “Network Simulator (NS-2)”, http://www.isi.edu/nsnam/ns/ [16] P. Loreti, M. Luglio, R. Kapoor, J. Stepanek, M. Gerla, F. Vatalaro, M. A. VazquezCastro, “LEO Satellite Systems Performance with TCP-IP applications”, Proceedings of Milcom 2001, Tysons Corner, McLean, VA, USA, October 28-31, 2001, session U24. [17] P. Loreti, M. Luglio, R. Kapoor, J. Stepanek, M. Gerla, F. Vatalaro, M. A. VazquezCastro, Throughput and Delay Performance of Mobile Internet Applications Using LEO Satellite Access, International Symposium on 3G Infrastructure and Services, Athens, GR, 2-3 July, 2001, pp.68-73. [18] M. Gerla, R. Kapoor, J.Stepanek, P. Loreti, M. Luglio, F. Vatalaro, M. A. VazquezCastro, Satellite Systems Performance with TCP-IP Applications, Tyrrhenian International Workshop on Digital Communications, IWDC 2001, September 17-20, 2001, Taormina, Italy, pp. 76-90.
52
