QoS Performances of IEEE 802.11 EDCA and DCF - Aston University

QoS Performances of IEEE 802.11 EDCA and DCF: A Testbed Approach Saty Mukherjee, Xiao-Hong Peng

Qiang Gao

School of Engineering & Applied Science Aston University Birmingham, UK {mukherjs,x-h.peng}@aston.ac.uk

School of Electronic and Information Engineering Beihang University Beijing, China [email protected]

Abstract— There has been considerable research into the performance of the enhanced IEEE 802.11e standard in recent years. Majority of the performance analysis has been based on simulations and analytical models, with little testing done in a real-world environment. We investigate the behaviors of the Enhanced Distributed Channel Access (EDCA) and legacy Distributed Coordination Function (DCF) based on a testbed operating in a realistic working environment. Both TCP/UDP streams representing traffic types such as FTP and VoIP are tested. Quality of Service (QoS) metrics such as Throughput and Delay are calculated directly from a passive capture of the channel. These metrics can also be converted into physical layer requirements such as Signal to Noise ratio. EDCA and DCF are shown to have similar performances with single streams of traffic, which are contributed directly by the loss effect at the MAC layer.

network. In fact, frame losses are regular occurrences in wireless networks, in contrast to wired environments. More recently there has been some pioneering work on EDCA modification in [8, 9] in which results have been verified using a real hardware testbed. Similarly in [10] the EDCA mechanism is implemented in a field programmable gate array (FPGA). The focus of their work [8-10] is still on the modification and optimisation of EDCA. In our work, however, we aim to show the effect of frame errors on QoS performances using both DCF and unmodified EDCA. Section II gives an overview of the operation of the legacy DCF and enhanced EDCA MAC protocols. Section III describes our real world test-bed environment, followed by our test results and analysis in Section IV. The paper is concluded in Section V.

Keywords; DCF, EDCA, Frame Loss, QoS, TCP, UDP.

I.

INTRODUCTION

Quality of Service (QoS) for IEEE 802.11a/b/g networks emerges as a critical issue since multimedia applications such as VoIP and video streaming through WLANs are becoming increasingly popular. Therefore, a range of technologies are required to ensure that system bandwidth, packet loss rates and latency are properly managed or controlled. In order to know the real need for each of the technologies, understanding the behaviour of the protocols currently available is key to the achievement of the QoS performances required. There has been considerable work in performance analysis of wireless LAN technology. Early work was carried out by Bianchi in [1] using a Markov chain model to approximate the behaviour of the distributed coordination function (DCF) mechanism. A number of other mechanisms to provide QoS were evaluated in [2] before the finalisation of the standard in 2005. During this time a number of performance evaluations [3-7] of the enhanced distributed channel access (EDCA) mechanism were conducted using analytical models and simulations in NS-2 and OPNET. One of the deficiencies of such work was the lack of a real hardware testbed to evaluate the accuracy of the simulation and analytical results. Performance evaluations conducted on the MAC layer contention mechanism assume perfect channel conditions. While this may be necessary in order to simplify analytical and simulation work, it is not a true representation of a real wireless

II.

MAC PROTOCOLS

In our study we concentrate on two distributed access methods: Distributed Coordination Function from legacy 802.11 [11] and Enhanced Distributed Channel Access from 802.11e [12]. The centralised access methods, Point Coordination Function (PCF) [11] and Hybrid Controlled Channel Access (HCCA) [12] are not considered as they are rarely implemented in hardware devices. A. Distributed Coordination Function (DCF) DCF is a random access scheme based on the carrier sense multiple access with collision avoidance (CSMA/CA) scheme. A legacy DCF station with a packet to send will first sense the medium for activity. If the channel is idle for a distributed interframe space (DIFS), the station will attempt to transmit after a random back-off period. This period is referred to as the contention window (CW). The value for the contention window is chosen randomly from a range [0, 2n-1], i.e. CW min ≤ CW ≤ CW max

(1)

where n is PHY dependent. Initially, CW is set to the minimum number of slot times CWmin, which is defined per PHY in microseconds [11]. The randomly chosen CW value, referred to as the back-off counter, is decreased each slot time if the medium remains idle. If during any period the medium becomes busy, the back-off counter is paused and resumed only when the medium becomes idle. On reaching zero, the station

978-1-4244-3693-4/09/$25.00 ©2009 IEEE Authorized licensed use limited to: ASTON UNIVERSITY. Downloaded on January 20, 2010 at 14:18 from IEEE Xplore. Restrictions apply.

transmits the packet onto the physical channel and awaits an acknowledgment (ACK). The transmitting station then performs a post back-off, where the back-off procedure is repeated once more. This is to allow other stations to gain access to the medium during heavy contention. If the ACK is not received within a short interframe space (SIFS), it assumes that the frame was lost due to collision or being damaged. The CW value is then increased exponentially and the back-off begins once again for retransmission. This is referred to as the Automatic Repeat Request (ARQ) process. If the following retransmission attempt fails, the CW is again increased exponentially, up until the limit CWmax. The retransmission process will repeat for up to 4 or 7 times, depending on whether the short retry limit or long retry limit is used. Upon reaching the retry limit the packet is considered lost and discarded. The retry limit is manufacturer dependent and can vary considerably. We will show in this paper that the impact of this is two fold: affecting both the delay and throughput of a station. B. Enhanced Distributed Channel Access (EDCA) The enhanced access method EDCA builds on the legacy DCF process and introduces four different access categories (ACs) or traffic classes for service differentiation at the MAC layer. This is achieved by varying the size of CW in the backoff mechanism on a per category basis. Service differentiation is provided by the following methods: i) Arbitration Interframe Space (AIFS): This is similar to the DIFS used in DCF, except the AIFS can vary according the access category; ii) Variable Contention Window: By giving higher priority traffic smaller contention windows, less time is spent in the back-off state, resulting in more frequent access to the medium. iii) Transmission Opportunity (TxOP): This allows a station that has access to the medium to transmit a number of data units without having to contend for access to the medium. In effect this is a form of frame bursting. The TxOP limit is defined per traffic class. Multiple AC queues can exist on a single station, contending with each other for the physical medium. This is regarded as virtual contention. The reference model is shown below in Fig. 1.

III.

EMPIRICAL TESTBED SETUP

When designing our IEEE 802.11 testbed, we wanted the setup to be simple and reflect a typical wireless environment as realistic as possible. All the tests were conducted in our research lab, configured as a typical office environment. The room layout is open plan with the access point installed 2m above ground on a wall. In all tests, clients had line of sight to the access point. There were no significant obstacles in the radio path. Using a Yellow Jacket [13] portable spectrum analyser, we checked the test area and found a typical noise power of -85dBm to -95dBm, which was within the bounds for an office environment. An overview of the system setup is shown in Fig. 2. 54Mbps 802.11g

Dell Server

#

Proxim Access Point

Full Duplex 100Mbps Ethernet

Laptop Clients AirPcap® Passive Sniffer

End to End TCP/UDP Connection

Figure 2. Wireless test-bed setup

A. Hardware environment Both the server and clients were mid range devices. The processor and the size of RAM used are P4 3.06 GHz and 1 GB for the server, and P4 1.5 GHz and 512 MB for the clients. An infrastructure wireless LAN was established within the research lab. The data transmission rate was fixed at 54Mbps (64-QAM) using the 802.11g PHY, disabling the auto fallback mechanism. Wi-Fi Protected Access 2 (WPA2) encryption was enabled on the wireless link and beacon broadcast was disabled. The access point used was an enterprise class Proxim AP4000 (Firmware 3.63), equipped with the AR5002AP-2X chipset. The laptop clients were fitted with Proxim ORiNOCO 11a/b/g PC Cards, based on the Atheros AR5001X+ chipset. Manufacturer specific features such as Turbo G and Extended Range were disabled. The server was connected to the access point using a full duplex fast Ethernet switch; this eliminated any bottlenecks on the wired side. DCF and EDCA parameters on the access point were left at default as prescribed in [11, 12]. The AirPcap USB device [14] is a passive packet capture device for 802.11 networks in the 2.4GHz band. All control, management and data frames transmitted on a particular 802.11b/g channel can be captured in real time and displayed in Wireshark [15]. This combination provides the results for our analysis in Section IV.

Figure 1. EDCA Reference Model [12]

The EDCA configuration used in our testing is described in Section III.

B. Sofware configuration and integration Both client and server utilised the Windows XP operating system with SP2. In order to test TCP/UDP connections over the testbed, we used the iPerf utility [16]. In the tests with TCP, we used the default value of 64k for the receive window, and 1460 byte payloads. With UDP, the receive buffer was 8k with the default 1470 byte payload.

Authorized licensed use limited to: ASTON UNIVERSITY. Downloaded on January 20, 2010 at 14:18 from IEEE Xplore. Restrictions apply.

RESULTS AND ANALYSIS

25

To date various methods have been developed and introduced to measure throughput at different layers in a protocol stack. We introduce a way of deriving the throughput directly from a raw 802.11 frame stream capture. Using Wireshark, we isolate the required stream by filtering traffic between the two required MAC addresses. Based on this capture, we used a combination of filters and custom scripts to determine the following:

Individual DCF Individual EDCA Total DCF

20

Total EDCA

η Throughput (Mbps)

IV.

Individual DCF SIM Individual EDCA SIM

15

10

5

•

Total number of frames sent, N.

•

Frame retransmission distribution, R(i).

•

Frame arrival delay, τ .

0 0

The time duration of the data capture, T, was derived from the timestamps in the capture file. Let R(i) (i = 1,2,…..n) be the number of frames being retransmitted i times before they are accepted at the receiver, where n is the retry limit for the device (in our case a nonstandard value of n = 10 is set). If a frame was retransmitted we assume that it was lost due to noise, collision or failure to pass the frame check sequence (FCS) at the receiver. From this, we can calculate the total number of frame losses Nl during T, as n

N l = ∑ iR (i )

(2)

i =1

2 3 Number of Clients

4

in Fig. 3. we can see that the simulated throughput is higher that what we found in our experiment. This reinforces our point that frame losses should be considered when evaluating wireless networks. A single retransmission can be regarded as the effect of independent random errors on the channel. This could also be attributed to the increased contention effect created by increasing the number of stations. Burst errors are classified by retransmission of two or more consecutive frames of the same sequence. 3

N λ = l ×100 N

2.5

Let Nr denote the total number of frames passed on to the higher layer at the receiver. Given the TCP data payload size LTCP , the average TCP throughput η during T can then be worked out by

2

n

)

τ Delay (ms)

(3)

N − ∑i =1 iR(i ) LTCP N L ( N − N l ) LTCP η = r TCP = = T T T

5

Figure 3. TCP Throughput as wireless clients increase

The loss rate λ in percent is given by

(

1

τ DCF τ EDCA

1.5

1

0.5

(4)

This calculation was based on the assumption that the last retransmission was successful. We had compared the captured value of Nr with the calculated result N - Nl as given in (4) and (2), and found that they were identical. In our first test we examine the performance of both DCF and EDCA using TCP traffic. The wireless clients had an average signal to noise ratio (SNR) of 46 dB at a fixed 3m distance from the access point. Fig.3 shows the average individual and overall TCP throughput. As expected there was a significant drop in individual TCP throughput as the number of clients increases. This is due to the shared medium in 802.11 and the distribution of radio resources amongst the contending clients. There is also a 5% difference in performance between DCF and EDCA with a single client. We attribute this difference to the larger default contention window for the Best Effort traffic class. We also have simulation results in OPNET 14.0, focussing only on the effect of contention, using a perfect channel. From the results

0 0

1

2 3 Number of Clients

4

5

Figure 4. MAC Delay

In Fig. 4, we show the MAC layer delay τ as we increase the number of TCP wireless clients, which was calculated by averaging the time difference between successive unique accepted frames on the passive frame capture. A clear linear relationship is observed between the MAC layer delay and the number of clients due to the effect of contention. Note that this does not include any processing delay on the client. Moving on to our second test, we evaluate the correlation between throughput and the signal to noise ratio (SNR). This test involved moving the client device further away from the AP, to reduce SNR. Our test results exhibit a strong correlation between throughput and SNR as shown in Fig. 5. In our field test the throughput dropped to 0 at the point around 17dB, when the received power was below the receiver sensitivity.


4000

25 η DCF

R(i) Retransmissions

20 η Throughput (Mbps)

DCF d = 3m SNR = 45dB DCF d = 6m SNR = 34dB DCF d = 9m SNR = 27dB DCF d = 15m SNR = 17dB

3500

η EDCA

15

3000 2500 2000 1500 1000 500

10

0 1

2

5

3

4

5

i

6

7

8

9

10

Figure 6. DCF Retransmission distribution 4000

0 10

15

20

25 SNR (dB)

30

35

40

45

50

Figure 5. Throughput vs SNR

If we set the AP to a slower PHY (e.g. 9Mbps BPSK), rather than 54 Mbps 64 QAM for the above results, we can expect a lower throughput at the same SNR due to the lower spectral efficiency of the modulation scheme used. This is an area of further work that may be explored. QoS requirement for throughput at a higher layer can be converted into physical layer requirements such as SNR. This can be useful for planning and design of a WLAN where the requirements at the higher and lower layers will be jointly considered. The retransmission distributions R(i) for different scenarios in terms of distance and SNR are shown in Fig. 6 and Fig. 7. Lower SNR at the receiver causes more frames to be lost or corrupted as the distance from the AP increases. This in turn triggers the ARQ mechanism to retransmit the frame for a number of times, as shown in Fig. 6 and Fig. 7. We can see that when SNR is high a frame is mostly retransmitted once or twice if required, while the number of retransmissions required increases significantly with distance when the SNR is lower. The differences between the distributions for EDCA and DCF are attributed to varying channel conditions as the tests were conducted sequentially. In our final test we demonstrate the effect of frame loss on a 2Mbps UDP traffic stream. The stream is indicative of a high quality MPEG video from the server to the client via UDP. The payload size is set to a default value of 1470 bytes. Both DCF and EDCA MAC protocols are tested. Note that no background traffic was present on the link during the test. The UDP throughput was calculated using the same principal as in (4) for TCP except for the default payload size: LUDP = 1470 bytes, i.e. η=

(N − ∑

n i =1

)

iR (i) LUDP T

(5)

As there was no fragmentation, the calculated UDP throughput from (5) was identical to that measured at the transport layer by iPerf. The data transmission rate γ, which includes the retransmitted frames is defined by N L FRAME γ = T

EDCA d = 3m SNR = 45dB EDCA d = 6m SNR = 34dB EDCA d = 9m SNR = 27dB EDCA d = 15m SNR = 17dB

3500

(6)

R(i) Retransmissions

5

3000 2500 2000 1500 1000 500 0 1

2

3

4

5

i

6

7

8

9

10

Figure 7. EDCA Retransmission Distribution

From the data in Fig. 8, we can see that UDP throughput remains relatively constant at 2Mbps for both MAC protocols. However, the UDP data transmission rate increases rapidly at low SNR values, due to the increased number of retransmissions. At high SNR values, there are far fewer retransmissions required, so the difference between the data transmission rate and UDP throughput is significantly reduced. 8

η UDP DCF η UDP EDCA γ UDP DCF γ UDP EDCA

7 6 γ and η (Mbps)

0

5 4 3 2 1 0 0

5

10

15

20

25 SNR (dB)

30

35

40

45

50

Figure 8. UDP data transmission rate and throughput

Fig. 9 shows the frame loss rate for the UDP test, which is correlated with the transmission rate shown in Fig. 9. Here the transmission rate is counted for both the original transmission and all subsequent retransmissions. Fig. 10 shows the packet loss rate at the transport layer versus corresponding SNR. It can be seen that the large frame error rate has a very small effect on packet loss rate, even at low SNR values. This is mainly due to the retransmission mechanism adopted at the MAC layer that can recover the majority of lost frames of the UDP streams at a relatively low rate (2 Mbps). The UDP packet jitter is of great importance than delay when used for non/near real-time video streaming, as data is usually buffered by the video application. The packet jitter


80

4.5 λ UDP DCF

Jitter DCF

λ UDP EDCA

70

Jitter EDCA

4

50

Jitter (ms)

λ Frame Loss Rate (%)

60

40

3.5

3

30 20

2.5

10 2

0 0

5

10

15

20

25 SNR (dB)

30

35

40

45

0

50

Packet Loss Rate (%)

[1]

0.1

[2]

0.08 0.06

[3]

0.04 0.02

[4]

0 5

10

15

20

25 SNR (dB)

30

35

40

45

50

Figure 10. UDP packet loss rate

(variation in delay) in Fig. 11 shows an increase at low SNR values. Although the packet losses are minimal, they can have considerable effect on a video stream, causing video corruption or frame freezing. In the case of higher bit rate UDP streams, the MAC layer is unlikely to be able to cope with a large number of retransmissions; and consequently large scale packet loss is likely. V.

20

25

30

35

40

45

50

VI. REFERENCES

0.12

0

15

Figure 11. UDP packet jitter

UDP Packet Loss DCF UDP Packet Loss EDCA

0.14

10

SNR (dB)

Figure 9. Frame loss rate for UDP test 0.16

5

CONCLUSIONS

We aimed to highlight in this paper that frame loss has an effect on QoS metrics such as throughput, delay and jitter. This research will serve as a basis for future studies on frame loss and its effect on higher protocol layers. Our wireless testbed showed that TCP throughput was greatly affected by frame loss caused by low SNR values. With a single TCP stream, legacy DCF and EDCA were both affected by retransmissions and have similar performances. The effect on relatively low rate UDP streams was limited as the retransmission mechanism in the MAC layer was able to recover the majority of losses. Further work needs to be done to establish whether the MAC layer can recover from losses in a high bit rate UDP stream, without affecting delay and jitter. Further investigation and experimentation into frame loss with various modulation schemes and the upcoming IEEE 802.11n standard is strongly recommended. Frame loss with newer ACK policies such as Block-ACK and No-ACK [12] should also be empirically tested.

[5]

[6]

[7]

[8]

[9]

[10] [11] [12] [13]

[14] [15] [16]

G. Bianchi, “Performance analysis of the IEEE802.11 distributed coordination function,” IEEE Journal on Selected Areas in Communications, vol. 18, pp. 535-547, 2000. A. Lindgren, A. Almquist, and O. Schelen, "Evaluation of quality of service schemes for IEEE 802.11 wireless LANs," in Proc. IEEE 26th Annual Conference on Local Computer Networks, 2001. J. del Prado Pavon and P. Wienert, "IEEE 802.11e Performance Evaluation," Phillips, 2003. K. Xu, W. Quanhong, and H. Hassanein, "Performance analysis of differentiated QoS supported by IEEE 802.11e enhanced distributed coordination function (EDCF) in WLAN," in Proc IEEE GLOBECOM, 2003. J. Prado Pavon and S. N. Shankar, "Impact of frame size, number of stations and mobility on the throughput performance of IEEE 802.11e," in Proc. IEEE Wireless Communications and Networking Conference, 2004. X. Yang, "Performance analysis of IEEE 802.11e EDCF under saturation condition" in Proc. IEEE International Conference on Communications, 2004. E. E. Paal and N. Olav, "Non-saturation and saturation analysis of IEEE 802.11e EDCA with starvation prediction," in Proc. ACM 8th international symposium on Modeling, analysis and simulation of wireless and mobile systems. Montreal, Canada, 2005. I. Dangerfield, D. Malone, and D. J. Leith, "Experimental Evaluation of 802.11e EDCA for Enhanced Voice over WLAN Performance," in Proc. 4th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks, 2006. P. Clifford, K. Duffy, J. Foy, D. J. A. L. D. J. Leith, and D. A. M. D. Malone, "Modeling 802.11e for data traffic parameter design," in Proc. 4th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks, 2006. Y.-S. Yoo, J.-R. Kim, Y.-J. Kim, and J.-D. Huh, "Analysis of EDCA MAC throughput for the IEEE 802.11e WLANs" in Proc. International Conference on Convergence Information Technology, 2007. IEEE-SA, "IEEE 802.11 Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications," 1999. IEEE-SA, "IEEE 802.11e Amendment: Medium Access Control (MAC) Quality of Service (QoS) Enhancements," 2005. BVSystems, "Yellow Jacket," http://www.bvsystems.com/Products/WLAN/YJ802.11bg/YJ802.11bg.htm 2008. CACE-Technologies, "AirPCap," http://www.cacetech.com/products/airpcap.htm, 2008. G. Combs, "Wireshark," http://www.wireshark.org/, 2008. NLANR/DAST, "Iperf - TCP/UDP Bandwidth Measurement Tool," http://dast.nlanr.net/Projects/Iperf/, 2008.
