So far, TCP fairness over heterogeneous wireless networks has not received ..... Random early marking (REM) currently deployed over the. Internet backbone [9] ...
Enhancing Fairness and Throughput of TCP in Heterogeneous Wireless Networks Fei Peng and Victor C.M. Leung Dept. of Electrical & Computer Engineering, University of British Columbia Vancouver, BC, Canada V6T 1Z4 E-Mail: {feip, vleung}@ece.ubc.ca Abstract - TCP exhibits inherent unfairness towards connections with long round-trip times and connections that have to traverse multiple congested routers. We have previously proposed a TCP Bandwidth Allocation (TBA) algorithm to solve this problem and proved its effectiveness in wireline networks. In this paper, we apply the TBA algorithm to improve TCP fairness over heterogeneous wireless networks with combined wireless and wireline links. In such networks, TCP suffers significant throughput degradations due to its window being frequently shut down, not in response to congestion, but by packet losses due to transmission errors over wireless links. We propose to apply Wireless Explicit Congestion Notification (WECN), a version of ECN enhanced for wireless networks, to decouple congestion control and loss recovery. Further enhancement is also incorporated to smooth traffic bursts. Simulation results show that not only can the combined TBA/WECN mechanism improve TCP fairness, but it can maintain good throughput performance in the presence of wireless losses as well. 1
I. INTRODUCTION As the Internet expands to integrate wireless and wireline networks, traffic volume of wireless users is expected to expand greatly and providing quality of service (QoS) to wireless user will be one of the most important technical issues in the future Internet. It is very important that fairness for endto-end QoS be maintained between wired and wireless users. So far, TCP fairness over heterogeneous wireless networks has not received much attention in the research community. A theoretical analysis was made in [1] for throughput fairness between TCP over wireless link with ARQ (Automatic Repeat ReQuest)-based link layer error recovery and TCP over errorfree wired link. It concludes that unfairness is sensitive to network propagation delay, and variations of wireless link characteristics. Existing proposals such as BA-TCP [2] and TCP-Peach+ [3] improve TCP fairness over wireless networks by, respectively, measuring propagation delay and available bandwidth in the network and conveying this information to the source nodes, and using NIL segments to probe the availability of network resources. In [4], it is stated that TCP Vegas does not discriminate against connections with long propagation delay; however, its implementation requires accurate measurement of propagation delay. Additionally, TCP-Vegas tends to degrade the throughput of multi-hop connections as shown in [4]. To address unfairness suffered by connections with long round-trip times (RTTs) and with multiple congested routers, we have proposed the TCP This work was supported by grants from Telus Mobility, Advanced Systems Institute of BC, and the Canadian Natural Sciences and Engineering Research Council under grant CRD247855-01.
Bandwidth Allocation (TBA) algorithm and illustrated its effectiveness in wired networks [5]. In this paper, we extend and enhance TBA for the heterogeneous wireless environment. In contrast to other proposals, the TBA algorithm does not require core network nodes to provide additional information to end nodes; nor does it require knowledge of propagation delays over the network. It is therefore easy to implement. Another well-known problem of TCP over wireless networks is the severe throughput reduction caused by packet losses over wireless links being misinterpreted by TCP as congestion losses. There are numerous proposals to overcome this problem, which can be classified as link layer protocols, split-connection protocols and end-to-end protocols [6]. Link layer protocols recover wireless losses by local retransmissions but usually are not able to cope with longer disruptions due to handoffs. Split-connection protocols isolate mobility and wireless related problems by splitting a TCP connection between the mobile host and fixed host into wired connection and wireless connection. While effective, these protocols do not maintain end-to-end semantics. End-to-end protocols adjust TCP operations to improve performance in the presence of wireless losses in a way that maintains end-to-end semantics. This can be accomplished by using standard TCP extensions specified in various RFCs, like TCP Reno, New Reno, Sack, ECN, large initial window size, path MTU discovery, etc., or using customized TCP modifications. In the first case, operating system upgrades in end hosts may not be required as they may already support these extensions; whereas in the second case upgrade of TCP software in end hosts is required. An effective approach employs ECN with a few modifications to avoid window reduction caused by wireless errors [7]. We call this approach Wireless ECN (WECN). In this paper, we propose to combine TBA with WECN and show by simulations that this novel method can improve fairness and throughput performance in heterogeneous wireless networks. The organization of the paper is as follows. Section 2 presents the TBA algorithm. Implementation of the TBA algorithm in the access node and its combination with WECN at the end nodes are described in Section 3. Network configurations of the simulations and the metrics for performance evaluations are given in Sections 4 and 5, respectively. The simulation results are analyzed in Section 6. We conclude the paper in section 7. II. TCP BANDWIDTH ALLOCATION ALGORITHM Consider a wireless LAN connecting clients (D1-D6) to a base station (n1) using wireless and wired links. The base station is connected to the backbone network through an access node (n0). We first examine the effects of many competing
Source
Si
Di
Deastination
n0
Access Node
n1
Base Station
Si
ni
Di
Gi
B
D1
Source
S1 S2
D2
Destination Access Node Gateway D2
S3
D3
Base Station
D4
D5
D3 S4
n0
n1
S1
D4
n1
G1
n2
G2
n3
G3
n4
B
D1
S5 D5 S6
: wired link: 1.5 Mb/s, 2.5ms delay :
: wireless link: 1 Mb/s, 2.5ms delay
wired link:
S2
10 Mb/s; S1=0 ms, S2=5 ms, S3=49 ms, S4=95 ms, S5=145 ms, S6=295 ms
connections over a single bottleneck link, so that the backbone network is simplified as a link connecting n0 and n1 directly. This is shown in Fig. 1 as topology 1. Data packets and their corresponding ACKs are forwarded by the access node in opposite directions. Each client Di downloads data from the respective source Si connected to the same access node so that there is a set of J (J=6) connections associated with n0. The sending windows of these sources are determined by the TBA algorithm in access node n0. Then, we use topology 2 shown in Fig. 2 to examine the effects of a wireless link on a TCP connection with long RTT that must also traverse a number of network hops shared by TCP connections over wired links with short RTTs. The sending window for each source in this case is determined by the TBA algorithm in its nearest access node. In both topologies, we assume that congestions for data transfer occur only at the access nodes. The TBA algorithm is applied to each access node over a wired link with bandwidth c. Let xi be the flow rate of the ith connection related to the access node where i∈J. Connection i has an RTT Ti that changes with time. The model is represented by the following equations, xi × Ti = wi , xi ≥ 0, Ti ≥ 0 (1) J
i
≤c
(2)
i =1
Identity (1) means that the flow rate xi is equal to the ratio of the window size wi divided by the RTT Ti. Subject to (2), we establish the proportional rate of each flow through the access node as follows, max p i log x i . (3)
∑ i∈J
Suppose that user i may choose pi, a positive number, as its amount to pay (we use user i interchangeably with connection i in this paper and suppose that the network knows the value of pi). Since the objective function (3) is strictly concave and the feasible region given by (1) and (2) are compact, the optimal solution exists and is unique. The Kuhn-Tucker conditions for * a solution of optimal xi are, c xi∗ = pi (4) pi
∑ i∈J
The rate control algorithm is given by, ∑ pi d xi (t ) = kpi − k i∈J xi dt c
S4
S5
: Wired Link 1.5 Mb/s, 10ms delay : Wireless Link 1 Mb/s, 10 ms delay
:Wired Link 10 Mb/s, 50 ms delay
Fig. 2. Network topology 2 of simulation model
Fig. 1. Network topology 1 of simulation model (Config1)
∑x
S3
D6
(5)
It has been proven [5] that (5) converges to the fair point. Differentiating (1) on both sides, we get, Ti dxi + xi dTi = dwi (6)
We then combine (5) and (6) to obtain the dynamics of window size as follows, ∑ pi dwi dT = (kp i − k i∈J (7) x i )Ti + x i i dt c dt Obviously, the flow rate from the access node will converge to the fair point if source window size of each connection changes with function (7). III. IMPLEMENTATION OF TBA ALGORITHM A. Support in Access Node Fig. 3 shows the implementation of TBA in the access node. Since we only care about TCP throughput and fairness, we focus on the link bandwidth, delay and bit-error rate (BER) and assume that the uplink and downlink have the same characteristics with respect to these parameters. How the wireless bandwidth is shared over the uplink and downlink is beyond the scope of the paper. For clarification, we separate the algorithm into three parts, i.e., TBA Algorithm Calculation, ACK Identification and Writing ACK Option. ACK Identification is required to separate the ACK flow from normal data traffic, identify the connection each ACK belongs to, and detect the ECN bit set by downstream network nodes. After the appropriate window size is determined by TBA Algorithm Calculation, the new window size is written into the ACK by Writing ACK Option, which is then inserted in the return link. In the following we describe the window size calculation by the TBA Algorithm in more detail. To implement (7), we divide time into discrete window cycles. The window size, RTT and average rate of connection i in cycle r(i) are denoted, respectively, as wr(i), Tr(i), and xr(i), where wr(i) is obtained by detecting the number of packets in the window when the sequence number of the backward ACK is equal to that of the first packet sent in this window, Tr(i) is the duration that is recorded and xr(i) is calculated by the formula wr(i)(Tr(i))/Tr(i). The change in window size during Tr(i) cycles is rewritten from (7) as, pi ∑ i∈J (8) ∆wr (i ) (Tr (i ) ) = ((kpi − kxr (i )−1 )Tr (i )−1 )Tr (i ) c + xr (i )−1 (Tr (i ) − Tr (i )−1 ) Each ACK should cause the window size to be changed by ∆wr ( i ) (Tr ( i ) ) / wr ( i ) −1 , since there are wr(i)-1 ACKs from the previous cycle. To get the value of the changed window size from (8), that is, to calculate the changed window size in r(i)+2th cycle for connection i, the average rate of the r(i)th
cycle xr(i) and round-trip time of the r+1th cycle Tr(i)+1 are required. Each time a new window cycle begins, xr(i), Tr(i) and Tr(i)+1 are updated. Due to cumulative characteristics of ACKs, a large number of data segments could be acknowledged by one new ACK after the loss recovery phase and a burst of data packets could be inserted into the network that results in bursts of traffic. In wireless network, recovery of transmission losses can contribute to more traffic bursts if the congestion window is allowed to be large for high bandwidth utilization. To smooth the traffic bursts we modify the TBA algorithm using a variable THRESH to limit the rate of expanding the sending window. We consider two configurations denoted as TBA/ECN and TBA/WECN, explained in more detail later. For TBA/ECN, THRESH is set to half of current window size (CWND) whether ECN or packet loss is indicated by an ACK. For TBA/WECN, THRESH is set to half CWND only when ECN is detected. Additionally, when CWND is detected as 1 or 2, THRESH is reset to 1 or 2 respectively for both configurations. Having determined the value of THRESH, the increment of window size by the TBA algorithm is limited to 4 packets per normal ACK if CWND < 4×THRESH, otherwise the window increment is limited to 1 packet per normal ACK to smooth the burst of data when window size becomes larger than 4×THRESH. For a partial ACK that acknowledges some but not all of the packets that were outstanding at the start of the fast recovery phase, we increase window size by 3/4 of the value calculated by the TBA algorithm. Since the access node is not responsible for loss recovery, this method can be applied to any TCP version. B. Support from TCP Layer With this algorithm, the source congestion window is changed according to the value in the TCP option provided by the TBA algorithm in the access node. TCP Slow Start and Congestion Avoidance are disabled at the source node. With TBA/ECN, normal ECN is applied in the end nodes, and the source node reduces its congestion window in response to both packet losses and ECN. With TBA/WECN, the source node initiates congestion control by reducing its congestion window in response to ECN only, but not packet losses in order to avoid unnecessary throughput degradation. In this case ECN has a higher priority in window control than ACK option field provided by the TBA algorithm. The TCP retransmission mechanism does not have to be modified in any way. IV. SIMULATION CONFIGURATION The link bandwidths and propagation delays are shown in Fig. 1 and Fig. 2. The wireless TCP connection has the same network condition as the wired one except that the wireless link has BER>0 whereas no bit errors occur in the wired links. The TCP sources are assumed to always have data to send, and each TCP segment is 1000 bytes long. The gain parameter k in the TBA algorithm in (7) is 50. In topology 1, to test the fairness of wireless connections with different RTTs, Config1 employs wireless links between n1 and all receivers (Di), as shown in Fig. 1. To test the fairness between wireline and wireless connections, Config2 employs a wireless link between n1 and D6 only, and wired links between n1 and the other destination
Forward Buffer DATA Sever
TBA Algorithm
Changed ACK ACK
Sever
DATA Backward Buffer
Fig. 3. TBA implementation in the access node
nodes. There is only one configuration in topology 2 as shown in Fig. 2, where D1 is connected via a wireless link. Random early marking (REM) currently deployed over the Internet backbone [9] uses two thresholds: minthresh and maxthresh and maintains an estimate of the average queue length, Qavg. When minthresh ≤ Qavg ≤ maxthresh, incoming packets are randomly marked by setting ECN with a probability that increases with Qavg. Incoming packets are randomly dropped with a probability that increases with Qavg when Qavg > maxthresh. When REM and ECN are used with the TBA algorithm, we found that better performance is obtained by setting ECN with probability 1 when Qavg > minthresh, compared to setting ECN with a lower probability. This improves robustness by avoiding the loss of ECN by link errors or congestion. Therefore, at the congested node n0, we set ECN in every incoming packet when the buffer size exceeds minthresh, which is set to half the buffer capacity. The maxthresh is set as buffer capacity. We first examine a range of queue sizes (buffer capacity) from 20 to 100 packets at a high BER of 10-5. Then, we examine a range of BER values with a fixed buffer capacity. Five TCP mechanisms are compared by simulations using the ns-2 simulator [8]. The benchmark mechanism, denoted as Reno, is standard TCP Reno. The ECN mechanism employs ECN based on TCP Reno, with network nodes capable of setting the ECN bit and destination nodes capable of transferring the ECN to the source nodes for the source nodes to take corresponding window control actions. The WECN mechanism is the same as ECN except that window control in response to packet losses is disabled. The other mechanisms, denoted TBA/ECN and TBA/WECN, respectively combine TBA with ECN and WECN. In Config2 of topology 1 and topology 2, the TBA/WECN and WECN mechanisms are evaluated by applying the respective mechanism only in the connection with the wireless link, while the corresponding wired connections employ TBA/ECN and ECN, respectively. However, in Config1 for topology 1, since all the connections incorporate wireless links, the same mechanism is applied to all the connections for comparisons. V. PERFORMANCE METRICS A number of metrics for quantifying fairness have been proposed, but no single metric has common acceptance. In this paper, assuming pi = 1, each user has the right to 1/nth of the capacity of the bottleneck link for n users proportionally sharing that link. Jain’s metric is applicable in this context. With connection i receiving a fraction bi of the bandwidth on a given link, the fairness of the allocation is defined as:
Fig. 4(a) Fairness vs. queue size, topology 1, Config.1
(∑i =1 bi ) n
Fairness ≡
2
n × (∑i =1 bi ) n
2
This metric ranges continuously in value from 1/n to 1, with 1 corresponding to equal allocation for all users, which is the best result in proportional fairness. Utilization is another important metric, since a high fairness is of little use if the link capacity is grossly underutilized. Utilization is defined as the number of original bits (i.e., not counting retransmissions) successfully transferred over a link during some time interval divided by the product of link rate and time interval. The closer the utilization metric approaches 1, the higher the link capacity is utilized. Utilizatio n ≡
( segments _ acked ) * ( segment _ size ) (link _ rate ) * time
The third metric is the utilization deviation. This metric is used for topology 2. The utilization deviation is defined as, Connection s
Utilizatio n _ Deviation ≡
∑ Utilizatio n
i
− 0 .5
i =1
Connection s
where Connections represents the number of connections in topology 2 connected to the same gateway. From Fig. 2, we see that the ideal bandwidth shared by each sender is half the bandwidth of the bottleneck link. If both the utilization and the fairness are good, the Utilization_Deviation is very small; otherwise, it can be very large. VI. SIMULATION RESULTS AND ANALYSIS Figs. 4 and 5 compare the (a) fairness and (b) utilization of the different schemes over a range of queue size for Config1
Fig. 5(a) Fairness vs. queue size, topology 1, Config.2
Fig. 4(b) Utilization vs. queue size, topology 1, Config.1
and Config2, respectively, in topology 1 with BER=10-5 over the wireless links. These figures show that considering both fairness and utilization, TBA/WECN outperforms the other schemes over a wide range of queue size. Fig. 4 shows that TBA/WECN and TBA/ECN have better fairness than the other scheme, but the utilization of TBA/ECN is much lower than TBA/WECN since TBA/ECN cannot avoid throughput degradation caused by link errors. Likewise TBA/WECN and WECN have better utilization than the other schemes, but TBA/WECN has much better fairness than WECN. While WECN avoids unnecessary window reductions due to wireless losses, it suffers from poor fairness typical of TCP due to its inherent bias toward connections with long RTT or traversing multiple congested routers. The TBA algorithm in combination with either WECN or ECN can alleviate this bias and give significantly better fairness performance compared to WECN, ECN and RENO. Applying TBA/WECN to the single wireless connection in Config2 yields a more consistent improvement in fairness over TBA/ECN, as shown in Fig. 5(a), compared to the case of multiple wireless connections in Config1 shown in Fig. 4(a). This shows that TBA/WECN is more helpful to a wireless connection competing for bandwidth with wired connections than several wireless connections sharing the same congested link. Fig. 5(b) shows that the utilization of TBA/ECN and ECN is much closer to that of TBA/WECN and WECN since the wired connections can occupy the rest of bandwidth lost by the shared wireless connection. Compared to topology 1, Fig. 6(a) shows that TBA/WECN yields significantly better fairness in topology 2 where the wireless connection traverses multiple nodes and shares links with wired connections. This illustrates that TBA/WECN is quite robust in heterogeneous wireless networks where connections suffer from extremely unfairness due to high BER,
Fig. 5(b) Utilization vs. queue size, topology 1, Config.2
Fig. 6(b) Utilization_deviation vs. queue size, topology 2
Fig. 6(a) Fairness vs. queue size, topology 2
long transmission delay and traversing multiple nodes. On the other hand, Fig. 6(b) shows that the Utilization_Deviations of TBA/WECN and TBA/ECN are much better than the other schemes. Overall, Fig. 6 shows that the performance of TBA/WECN is much better than the other schemes. From Figs. 4-6, it is apparent that to realize the performance advantages of TBA/WECN, a sufficiently large buffer size (> 50) should be used. However, a buffer size that is too large (>100) is not helpful as the congestion windows can get very large resulting in a high rate of congestion losses under heavy traffic. We further evaluate performance of the different schemes over a range of wireless link BER from 10-5 to 10-9. Figs. 7(a) and 7(b) show the fairness and utilization, respectively, for Config1 in topology 1 with buffer size = 60. Results for the other configurations and topologies are very similar. When BER of the wireless link falls below 10-6, the utilizations of all the different schemes become similarly high, and TBA/WECN and TBA/ECN achieve the same fairness that is much better than the other schemes. With minimal wireless losses, TBA/ECN is just as effective as TBA/WECN, as both benefit from the fairness enhancement of TBA. In comparison, TCP Reno generally yields lower utilization and fairness performance in the presence of congestion or wireless losses.
depend on knowledge of the propagation delay. Although we have only presented results of applying these techniques to TCP Reno, simulations show that these techniques can similarly enhance the performance of other TCP versions. In this paper, WECN is applied to avoid throughput degradations due to link errors, and TBA is applied to improve fairness. These techniques can also work with TCP split connection protocols and link layer protocols. For further work, it is of interest to investigate the local stability and convergence of the equilibrium point of the TBA algorithm in the presence of link errors, and to apply the algorithm to conditions where congestion occurs at other than access nodes. REFERENCES [1] [2] [3] [4] [5]
VII. CONCLUSIONS [6]
We have presented simulation results to show that the combination of TBA algorithm and WECN, enhanced by the traffic burst smoothing function, can improve TCP fairness and utilization in heterogeneous wireless networks in which the wireless link suffer from a high BER. A desirable feature of our algorithm that eases implementation is that it does not
Fig. 7(a) Fairness vs. queue size, topology 1, Config.1
[7] [8] [9]
T. Matsuda and M. Yamamoto, “Performance analysis of TCP fairness between wired and wireless sessions”, Proc. IEEE PIMRC’02, 2002. M. Gerla, W.J. Weng and R. Lo Cigno, “BA-TCP: A bandwidth aware TCP for satellite networks”, Proc. Int’l Conf. on Computer Communications and Networks, pp. 204–207, 1999. I.F. Akyildiz, X. Zhang and J. Fang, “TCP-Peach+: Enhancement of TCP-Peach for satellite IP networks”, IEEE Communications Letters, vol. 6, pp. 303–305, July 2002. C.Q. Hua and T.-S. P Yum, “The fairness of TCP Vegas in networks with multiple congested gateways”, Proc. IEEE Int’l Conf. High Speed Networks and Multimedia Communications, pp. 115–121, 2002. F. Peng and V.C.M. Leung, “A novel fair bandwidth allocation algorithm for TCP window control”, Proc. IEEE IPCCC’03, Phoenix, AZ, April 2003,. H. Elaarag, “Improving TCP performance over mobile networks”, ACM Computing Surveys, vol. 34, pp. 357–374, September 2002. F. Peng, S.D. Cheng and J. Ma, “A Proposal to apply ECN into wireless and mobile networks”, Proc. INET’00, 2000. The Network Simulator-ns-2, http://www.isi.edu/nsnam/ns. S. Floyd, R. Ramakrishna and S. Shenker, “Adaptive RED: An algorithm for increasing the robustness of RED's active queue management”, Aug. 2001, under submission.
Fig.7(b) Utilization vs. queue size, topology 1, Config.1
