unexpected outdoor link-layer performance of 802.11 Atheros chipset based cards [1] with Madwifi driver due to a driver. Research supported in part by National ...
Implication Of MAC Frame Aggregation On Empirical Wireless Experimentation Gautam Bhanage∗ , Rajesh Mahindra† , Ivan Seskar∗ , Dipankar Raychaudhuri∗ ∗ WINLAB, Rutgers University, RT 1 South, North Brunswick, NJ 08902, USA † NEC Labs America, Princeton, NJ 08540, USA {gautamb, seskar, ray}@winlab.rutgers.edu, {rajesh}@nec-labs.com
Abstract— Wireless network emulator testbeds have become increasingly important for realistic, at-scale experimental evaluation of new network architectures and protocols. Typically, wireless network performance measurements are made at multiple layers of the wireless protocol stack, i.e. link layer, MAC layer and network layer. This study highlights the impact of layer 2 frame aggregation that is enabled by default in the software drivers for commodity wireless 802.11 devices while it is still not a part of the core 802.11 standard. Using experimental measurements, it is shown that this feature has an impact across a diverse set of wireless experiments and should be considered while comparing results. Measurements on the ORBIT testbed show that throughput measurements can vary up to a startling 25% for certain packet sizes and the variance in receiver side interframe delays can almost double if MAC aggregation and preset transmission opportunities are not taken into consideration. Further results for VoIP traffic show a deterioration in jitter of up to 8 times when coupled with MAC layer aggregation in 802.11. Index Terms— Frame aggregation, fast-framing, txop, madwifi implementation, 802.11e.
I. I NTRODUCTION Testbeds for wireless networking research such as ORBIT [9], MiNT [10], EMULAB [19], and Roofnet [6] aim to provide the ability to control and measure important network parameters on actual wireless devices with a high degree of accuracy, reproducibility, and efficiency. Wireless emulation has been recently used for evaluating the performances of 802.11 protocols [13][21], ad hoc routing protocols [8], managed mode performance of networks [12], quality of service evaluations [25], power and rate control experiments [23], [20], [27], [7], and a host of other studies. The common factor in the success of the wireless testbeds mentioned above is the use of commodity wireless hardware devices (e.g., 802.11 Atheros/Intel NICs) along with open-source driver softwares (e.g., Madwifi driver [5]) that allows for flexible configuration and customization. This flexibility gives users a better control over protocols and softwares used over the radio nodes. Despite having several advantages, certain driver implementations might have unexpected features running by default to further optimize standard 802.11 behavior. In a recent work [11], the authors provide experimental evidence of unexpected outdoor link-layer performance of 802.11 Atheros chipset based cards [1] with Madwifi driver due to a driver Research supported in part by National Science Foundation Grant # CNS072505.
specific antenna diversity algorithm. Hence, executing experiments on such wireless testbeds requires careful monitoring throughout the experiment and an awareness of unexpected non-802.11 conforming operations of the hardware-driver. Such behavior of the driver should either be disabled or accounted for in the eventual analysis. Through this study we show that though MAC-frame aggregation or fast framing1 is not a part of the 802.11 standard, open-source drivers can and do perform frame aggregation under certain circumstances (besides having aggregation enabled by default) which may not lead to conforming results with scientific experiments. Through our experiments we evaluate the frame aggregation functionality that is a part of the latest Madwifi drivers and report deviation from expected results. Contributions of this study can be summarized as follows: 1) We describe frame aggregation as implemented in the Madwifi driver and show that savings in overheads are determined only by the MAC back-off and the transmission rate, 2) Describe prerequisites for aggregation on the basis of reverse engineering of the Madwifi driver source code, 3) We show that throughput, inter-frame delay, frames transmitted and jitter vary significantly from the expected value (as per the standard). 4) Finally, we present three test cases that show a significant difference in performance for scientific experiments with default frame aggregation behavior. Section II starts by describing the basics of frame aggregation, followed by its quantitative analysis and conditions for its working based on the Madwifi driver. Section III presents a detailed discussion on the results from experiments of first order measurements such as throughput and interframe delay. Section IV shows the practical implication of frame aggregation on scientific experiments through three case studies. Finally, Section V provides the concluding remarks. II. F RAME AGGREGATION Observation of internet traffic studies [22] showed that more than half of the packets were smaller than 200 bytes leading to considerable control overhead and degraded efficiency. A few studies like [18] and [26] aim to alleviate this problem by performing frame aggregation above the MAC - SAP. Other 1 This term in Madwifi world is based on the fast framing support from the ATHEROS hardware. Frame aggregation was considered as a part of 802.11eD1.0, but is not a part of the final 802.11e standard.
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
DIFS Backoff A)
DIFS Backoff
SIFS
802.11 MAC Frame
ACK
DIFS Backoff Aggregated MAC Frame
PHY
MAC 24
LLC 0 8
ACK
MAC Sub-Frame II
MAC Sub-Frame I
4
D)
Aggregation MAC 16
E)
ACK
SIFS
B)
C)
SIFS
802.11 MAC Frame
0 Destination MAC 2
6
LLC UDP/IP 8
Source MAC 6
DATA
28 LEN 2
Fig. 1. Frame aggregation for the creation of a lumped MAC frame which has the two aggregated frames as the payload. This approach generates savings on difs, backoff times and physical layer headers. Subparts (A) and (B) of the figure compare timelines for transmission with and without aggregation while (C), (D), and (E) describe the payload of an aggregated frame.
approaches like [24], [15] have been proposed to perform MAC frame aggregation to improve efficiency of real-time traffic transmissions over 802.11. Another study in [14] uses MAC frame aggregation in 802.11 mobile terminals for energy savings and an increased battery life. Recently, the 802.11n standard [3] has incorporated frame aggregation as a built in feature. Fast Framing is a technique employed by Atheros chip manufacturers to achieve higher throughput values as an optimization for improving 802.11a/b/g system performance by utilizing MAC frame aggregation (allowing frame sizes up to 3000Bytes)2,3 . This paper considers Madwifi’s implementation for frame aggregation on Atheros wireless cards. This technique may or may not be used by other manufacturers and their implementation details may be different. However, popularity of this driver/hardware combination prompted us to delve deeper into its working.
A. Basics Frame aggregation in Madwifi combines two MAC frames as shown in Figure 1 and sends them as the payload of a single congregated frame. This single aggregated frame contains two sub-frames encapsulated in a 802.11 PHY and MAC header prior to transmission. Each sub-frame is a UDP/IP payload encapsulated by the Link-Layer control header. A custom Aggregation MAC header is added by the driver to each sub-frame containing details about the source and destination MAC addresses and the length of the subsequent sub-frame in bytes. This aggregation header helps the receiver to detect aggregation and push the congregated frame as two independent frames upward through the stack. 2 We noticed this feature after considerable sniffing and investigation. Though this feature may be well known in industry, the scientific community does measurements largely unaware of this mechanism. 3 Packet trace analysis softwares like Wireshark or Ethereal are not able to identify these frames. However, a study of the source code, comparison of frame sizes, and transmissions confirm frame aggregation.
B. Performance Estimation To determine performance gains due to frame aggregation we estimate the channel time which includes the channel access time and transmission time. The total channel time (Ttotal ) using standard 802.11a protocol is given as the sum of Tbackof f + Tdif s + Tphy + Tmac + Tllc + Tdata + Tack + Tsif s . where Tdif s , Tsif s , and Tbackof f are taken in accordance with the 802.11a standard [2]. Time taken to transmit PHY header/preamble Tphy and a MAC ACK Tack are fixed irrespective of data transmission rates. Transmission time for the MAC header Tmac , Link-Layer control Header Tllc and UDP/IP payload Tdata depends on the physical layer transmission rate used. The total channel time for an aggregated frame will change to: Ttotal
aggr
= Tbackof f + Tdif s + Tphy + Tmac + Tllc
+ 2 × (Taggr
mac
+ Tllc + Tdata ) + Tack + Tsif s (1)
where Taggr mac is the time spent to transmit the Madwifi Aggregation header. The aggregation header is smaller than a standard MAC header since most details are provided by the header of the aggregated frame. Savings with aggregation can be calculated as: Savingswith
aggr
= Tbackof f + Tdif s + Tphy
+ Tmac + Tack + Tsif s − (2 × Taggr
mac
+ Tllc ) (2)
Since the Taggr mac and Tllc overheads added due to aggregation (as shown in Figure 1) are always lesser than the times spent in sending multiple ACKs, PHY, MAC headers, and channel contention, aggregation always yields bounded savings in channel time. It is interesting to note that this value is not determined by the frame size, rather it varies only with the Tbackof f and the transmission rate. The backoff is decided on the number of slots after which a node can transmit its frame is largely correlated with the contention on the channel. The transmission rate of a frame decides the Tmac and the Tllc since these headers are transmitted at the same rate as data. Figure 2 shows the efficiency of payload delivery in 802.11a seen for two different transmission rates with and without MAC frame aggregation. The effect of PHY header, ACK transmissions and inter-frame spacing is more pronounced for higher PHY rates since the PHY headers and MAC acknowledgements are always transmitted at 6M bps. Substituting standard values, we observe that the protocol overhead reduction with frame aggregation results in savings of 90 to 370μsecs of channel time. This value varies depending upon the amount of backoff determined by contention. C. Preconditions For Aggregation Enabling frame aggregation (which is done by default when loading a driver) in itself does not result in aggregation of every transmitted frame. Rather, it is a decision made on a per frame basis, which depends on a certain set of conditions. Studying these conditions is crucial since they enable us to understand when we may observe aggregation and thus, attribute certain performance characteristics to it. Frame aggregation mechanism is possible when the driver senses that
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
Ant - 4
54
36
20dB Attenuation
C1
24 Payload UDP/IP Headers MAC Headers Preamble + PHY Headers MAC ACK IFS (SIFS+DIFS)
12
0
C2
AP
48
20m
Effective Channel Utilization
Ant - 1
36M
36M−Ag
54M
54M−Ag
Fig. 2. Bandwidth utilization with aggregation for different transmission rates (36M and 54M in this case) of 1024byte frames. Since the transmission rate of Physical layer headers are at a constant speed of 6M in 802.11a, the benefits of aggregation are more at higher physical layer rates.
Given: Fragmentation threshold, channel rate, transmit queue priority. Relevant Checks Performed: 1: if (if depth txQueue < aggregation thresh), goto RET0. 2: if (fragmentation threshold < 2346), goto RET0. 3: if (mode is neither station nor AP), goto RET0. 4: if (in ap mode() and is ether multicast), goto RET0. 5: if (fast frames are enabled and current rate is less than the minimum rate for aggregation), goto RET0. 6: if (txoplimit is defined and the time for aggregated frame transmission > txoplimit), goto RET0. RET1: Aggregation possible. RET0: Aggregation not possible.
S Ant - 3
Ant - 2 20m
Fig. 4. Experiment setup on the ORBIT radio grid. Figure indicates scaled relative position of entities for measurements on the ORBIT grid. C1 and C2 represent the clients sending traffic to the AP. The four noise injection antennae (Ant - *) are located at the four corners of the grid with only Ant - 1 used to pump noise at the receiver running on the AP. S is a sniffer used for inter-frame delay measurements.
P arameter Experiment Duration Averaging Duration Operation Mode Frequency Chipset Driver Tx Diversity Virtual CS
V alue 120 secs Per sec 802.11a Infrastructure 5.18GHz Atheros AR5212 MadWiFi 0.9.3 Disabled Disabled
Fig. 5. Common experimental parameters used with ORBIT nodes. Other parameters such as channel rate and packet sizes which may vary with experiments are mentioned explicitly.
study some simple means of disabling frame aggregation are: by a private ioctl to disable fast framing, source code change or by making the fragmentation threshold smaller than 2346. III. F IRST O RDER R ESULTS
Result:Boolean - Aggregation possible or not. Fig. 3. Factors affecting decision making in the driver for MAC frame aggregation as per the Madwifi 0.9.3 driver source code.
the channel is approaching saturation. Figure 3 shows some of the relevant checks performed in the driver before a decision is made to aggregate a MAC frame4 . One of the first checks is to determine if the user has enabled fragmentation by setting the threshold to a value lesser than the MTU. The second check ensures that MAC aggregation works only in the access point and station modes and will not affect experiments performed in the ad hoc mode. Aggregation is also disabled with multicast. Another important requirement is that the transmission queue must have at least aggregation thresh MAC frames ready for transmission for frame aggregation to occur. A default value of 3 is used for the aggregation thresh by the Madwifi driver. The driver also makes sure that the aggregated frame will be transmitted within the txoplimit set for the interface to conform with the 802.11e standard. Based on this 4 Information provided in the figure is based on source code for the Madwifing drivers and may vary for implementations.
We begin our empirical evaluation with results that illustrate the impact of aggregation using metrics like: saturation throughput, MAC frames, inter-frame arrival times, and their variance. A. TestBed And Topology We use the ORBIT testbed facility [9] which consists of 400 802.11 wireless nodes arranged in a 20m × 20m grid. As shown in the Figure 4, four noise injection antennae are incorporated in the testbed that allow controlled injection of AWGN noise at desired power and frequency. The generic topology consists of a single wireless link running between an AP and a client C1 which acts as the sender as shown in Figure 4. To measure performance with contention we add another client C2 sending traffic to the AP. Care was taken in choosing the position of the wireless nodes such that the RSSI at the receiver was similar for the different nodes to avoid PHY layer capture that could affect our results. A node was chosen to operate in monitor mode to verify the aggregation of MAC frames over the wireless link. Figure 5 depicts the generic experimental parameters used for conducting the experiments. We generate traffic using Iperf [4] internet traffic generator tool.
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
30
20
15
Aggregation − Contention No Aggregation − Contention Aggregation − No Contention No Aggregation − No Contention
10
5 5
10
15
20 25 Offered Load (Mbps)
30
35
25 20 15 10
Aggregation − Contention No Aggregation − Contention Aggregation − No Contention No Aggregation − No Contention
5 0 64
40
30
Percentage improvement in throughput due to aggregation decreases with packet size
Observed Throughput (Mbps)
Channel Saturation Triggers Aggregation
Observed Throughput (Mbps)
Observed Throughput (Mbps)
25
128
(a)
256 512 Packet Sizes (Bytes)
1024
20
15 Aggregation − Contention No Aggregation − Contention Aggregation − No Contention No Aggregation − No Contention
10
5 9M
1400
Higher difference in throughput at higher rates due to aggregation
25
12M
(b)
18M 24M 36M PHY Channel Rate (Mbps)
48M
54M
(c)
Fig. 6. (a)Observed throughput with varying offered load with and without frame aggregation. Retries due to contention are costlier with aggregation. Hence the difference in throughput with and without contention scales with aggregation. (b) Throughput with varying packet sizes with and without frame aggregation. Results show that for smaller frame sizes throughput almost doubles, while the advantages diminish for larger frames. (c) Throughput measurements for varying transmission rates. We see that benefits of aggregation are more at higher transmission rate.
5500 5000
1500
Agregation − Contention No Aggregation − Contention Aggregation − No Contenion No Aggregation − No Contention
1000
500 5
10
15
20 25 Offered Load (Mbps)
30
(a)
35
40
3000 MAC Frame Transmissions
2000
MAC Frame Transmissions
MAC Frame Transmissions
Drop in frames transmitted 2500 due to aggregation
4500 4000 3500 3000 2500 2000 1500 1000 64
Aggregation − Contention No Aggregation − Contention Aggregation − No Contention No Aggregation − No Contention 128
256 512 Packet Sizes (Bytes)
2500
More frames are transmitted with aggregation at higher rates
2000 1500 1000 Agregation − Contention No Aggregation − Contention Aggregation − No Contenion No Aggregation − No Contention
500
1024
1400
(b)
0 9M
12M
18M 24M 36M PHY Channel Rate (Mbps)
48M
54M
(c)
Fig. 7. (a)Observed MAC frames transmitted with varying offered load with and without frame aggregation. Frames transmitted in saturation with aggregation drop to almost half those without aggregation. (b) MAC frames transmitted with varying packet sizes with and without frame aggregation. Number of MAC frames transmitted drop by half for larger frame sizes, while for smaller frame sizes the reduction in transmitted frames is lesser. (c) MAC frames transmitted as a function of the transmission rate. We observe that higher number of frames are transmitted for higher rates and the savings increase significantly.
B. Throughput Measurements Our first experiment measures the throughput performance with aggregation as a function of offered load on the link. Figure 6(a) plots the throughput measured on a single link and a link with two senders (with contention) as a function of the offered load. Packet size throughout the experiment is maintained at 1024 bytes, transmission rate at 36Mbps, and other experiment parameters are maintained as shown in Figure 5. Aggregation is explicitly enabled or disabled by controlling the fast framing option in the driver. The results show that below saturation, performance of all 4 cases is similar. However, saturation throughput settles at different values for each of the test cases as offered load is increased. Maximum throughput is observed with the case of aggregation and no contention since it sees minimum overheads. Contention reduces throughput both with and without aggregation. However, the results are different due to higher costs associated with MAC retries in aggregation. With the same experiment setup we also measure the number of MAC frames transmitted. Figure 7(a) shows the number of frames transmitted as a function of the offered load at the sender(s). As with the throughput measurements, results below saturation from all four cases are comparable. However, as the offered load is increased further, the number of MAC frames transmitted drop to almost half with the use of aggregation. Another interesting measurement revealed that as the channel is on the verge of being saturated we see partial aggregation.
Packet traces captured at an offered load of 25Mbps with a channel rate of 36Mbps revealed that some frames were being aggregated while some were not. This conforms with our earlier investigation of the driver which reveals that a decision to aggregate is made on a per frame basis. This observation is particularly important since experiments making measurements with the channel barely saturated may see varying throughput depending on the amount of aggregation. Our next experiment measures the pattern in throughput with varying application packet sizes. Figure 6(b) plots the observed throughput with varying packet sizes and a fixed offered load of 50Mbps. These measurements show that throughput nearly doubles with aggregation for small frame sizes of 64bytes, while it shows comparatively lesser gains for larger frame sizes. As proven earlier savings in overheads are constant and only determined by the backoff and transmission rate. Since the relative overheads seen for small frames are higher, we observe better performance gains for small frame sizes and vice versa. It is to be noted that for large frame sizes, even though the throughput does not double with aggregation, the change is significant (approximately 25%) to affect scientific experiments. It can also be seen that the effect of contention is seen more with large frame sizes since the cost of retransmissions is higher. Figure 7(b) shows the number of MAC frames transmitted for different frame sizes. We observe that the number of frames transmitted for large frame sizes (say 1400bytes) is halved
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
Mean Inter−frame Delay (milliseconds)
Observed Interframe space (microsecs)
2000
1500
1000
500 No Aggregation With Aggregation − Receiver With Aggregation − Sniffer 0 0
5
10 15 Frame Number
20
25
2
No Aggregation With Aggregation − Receiver With Aggregation − Sniffer
1.5
Size
NoAg
Ag − R
128
5e-4
5.5e-2
Ag − S 7e-4
256
6e-4
12.4e-2
1.2e-4
1
0.5
0
64
(a)
128
256 512 Frame Size (bytes)
768
1024
(b)
512
8e-4
34.2e-2
1.6e-4
1024
2.1e-4
108.9e-2
4.7e-4
(*all values in msecs)
(c)
Fig. 8. (a)Observed interframe space (µsecs) between MAC frames as seen with and without aggregation for the first 25 MAC frames. Frame size is maintained at 1024bytes in all scenarios. (b) Mean interframe arrival times as seen at the receiver and sniffer with and without frame aggregation. (c)Variance in inter-frame arrival times without aggregation at the receiver (NoAg), with aggregation at the receiver (Ag-R), and aggregation as seen by the sniffer (Ag-S).
with the use of aggregation. As the frame size reduces, the percentage reduction in the frames transmitted decreases. As discussed previously, the savings in channel time due to aggregation are fixed irrespective of the frame sizes used. Hence, it is possible to transmit more frames of smaller size because of the lesser transmission times when compared to large frames. To sum up, maximum advantage with throughput (doubling) is seen with smaller frame sizes while the maximum advantage with frame sizes (halving) is seen with larger frames. Thus, these measurements with frame sizes provides an insight into the inverse relationship between how aggregation affects the throughput and frames transmitted for different frame sizes. Transmission rate is one of the major factors for determining the effect of physical layer overheads and hence the savings achieved due to aggregation. Figure 6(c) shows the measured throughput for various transmission rates at a fixed frame size of 1024bytes. As expected and seen through the formal analysis, benefits of aggregation are more at higher transmission rates since we see a greater effect of the savings on higher overheads. Figure 7(c) shows the number of MAC frames transmitted for different frame sizes. We see an expected trend where the number of frames transmitted with and without aggregation increase with channel rate. C. Inter-frame Arrival Measurements Delay measures the difference in absolute time between when a frame is transmitted and the time when it is received. However, we measure interframe arrival times since delay measurements require explicit synchronization5 . Let us assume that the delay for two consecutive frames i and j is given as Di and Dj . Each Dk is evaluated as the difference in the time stamp at the receiver Rk from the sender Sk . We can evaluate the difference in delay of two consecutive frames i and j, given by Dj − Di as: (Rj − Sj ) − (Ri − Si ) = (Rj − Ri ) − (Sj − Si )
(3)
Hence the difference in inter-frame arrival times given by Rdif f = Rj − Ri is evaluated as: Rdif f = (Dj − Di ) + (Sj − Si ) = Ddif f + δ
(4)
5 Synchronization across machines is possible to a granularity of a few msecs using the network timing protocol daemon. However, achieving higher granularity is difficult using standard tools. Delay measurements can easily replace the inter-frame arrival measurements if this issue is fixed.
where Ddif f represents difference in delay of consecutive frames and δ denotes difference in time stamps of consecutive frames at the sender. As long as the frames are sent at a constant rate over a single link, with static channel conditions, δ appears as a constant value. Thus we can use difference in inter-frame arrival times as an approximation for difference in delays of consecutive frames. To achieve accurate measurements with considerably slow CPUs at the sniffers, all temporal measurements are made at 9M . While absolute values of delay might change, the observed trends only scale with transmission rate. Figure 8(a) shows the inter-frame delay seen for 25 sequential frames at the receiver. We observe that without aggregation the interframe arrival times are constant at around 1msec throughout the experiment. It is interesting that the measurements for aggregation vary depending on the point of measurement. The arrival times at the sniffer are homogeneous since it sees continuous flow of aggregated MAC frames. However, at the receiver, after the aggregated MAC frame is received they are split and pushed up the stack. Hence the arrival time for the first frame within the aggregated frame is equal to the arrival time of the aggregated frame which is approximately 2msecs, and that of the second frame is the time required to simply push it up the stack which is around 2μsecs. Similar observations can be made from the Figure 8(b) which measures the mean of inter-arrival times between frames over approximately 25 thousand sniffed MAC frames for different frame sizes. Results show that the mean delays are comparable for the case with no aggregation and the measurement made at the receiver. This relationship holds because even with aggregation, though the first MAC frame is received after considerable delay the second MAC frame (which is a part of the aggregated frame) is available almost instantaneously. Mean inter-arrival time with the aggregation at the sniffer is approximately double than that without aggregation since the sniffer essentially measures the inter-arrival time for the aggregated frame. Figure 8(c) displays the variance in inter-arrival frame times for different frame sizes. These results are derived from measurements shown in Figure 8(a) which shows that there is little or no variance in inter-frame arrival times for the cases with no aggregation and aggregation measured at the sniffer. However, the variance is high at the receiver since the inter-
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
since the PER increases with frame size. This has a particular importance with topology creation since creating topologies with noise requires injection of just the adequate amount of noise (so that it does not impact other links). However, if the frame sizes vary due to aggregation this may introduce another dimension to the topology mapping problem [16].
Percentage Of Saturation Throughput (%)
100
80
60
40
20
0 −50
1024Byte, aggregation 1024Byte, no aggregation 64Byte, aggregation 64Byte, no aggregation −48
B. Case 2: VOIP Performance
−46 −44 −42 −40 Injected Noise At Receiver (db)
−38
−36
Fig. 9. Performance with MAC frame aggregation in noisy environments. Results presented with emulation on an indoor setup.
No Aggregation Aggregation
VoIP Jitter (milisecs)
20
15
10
5
0
9
12
18 24 36 PHY Channel Rate (Mbps)
48
54
Fig. 10. Measurement of VOIP jitter with different channel rates with and without aggregation. Jitter significantly increases with aggregation.
frame arrival time is high for the first frame and negligible for the second frame within the aggregated frame. The sniffer which measures performance under aggregation does not see the two separate frames and hence sees a very low variance in inter-frame times. These measurements are important since they show that point of measurement (at the sniffer or receiver) would produce a dramatic difference in delay and jitter with frame aggregation.
Voice over IP (VOIP) is a protocol for voice packet delivery over IP. An interesting feature of aggregation is that it can also aggregate frames across multiple IP flows. If appropriate tagging is not used at the IP layer, real time and best effort frames could be aggregated resulting in unexpected performance. At this time, most services like Gtalk, Skype, or VLC media player do not set the TOS (type of service) byte. Hence channel saturation by running other flows such as data, or streaming video could result in aggregation of small VOIP frames with possibly large data frames. This could cause a higher variation in the observed delays with the VOIP frames. To measure performance under these conditions we setup a link between an access point running a VoIP and a data receiver and a client running the corresponding senders. VoIP traffic was emulated using the G.711.2 codec with 50pkts/sec of real time traffic and a frame size of 92(64+overheads) bytes. For different channel rates the measured jitter is as shown in the Figure 10. As seen, the jitter is always higher for the case with aggregation. At lower rates, the jitter with aggregation is 8 times higher than that without aggregation and at least 6.5 times higher at higher channel rates. Thus, real time traffic is severely affected by aggregation in terms of the perceived jitter. Upon sniffing the wireless link, we observed that about 33% of the total VoIP packets were aggregated with the 1400byte packets of the competing data flow. C. Case 3: Rate Adaptation Algorithms
IV. C ASE S TUDIES Our case studies show the effect of MAC frame aggregation on possible scientific experiments. We present a representative set of results which are by no means exhaustive but provide an insight into possibly inaccurate measurements or inferences that could be made. A. Case 1: Topology Creation Wireless emulation frequently requires the creation of multihop topologies [8], [28], mobility emulation and emulation of non-zero packet error rate (PER) links. Creation of multihop topologies in ORBIT is supported by noise injection in four corners of the grid. Studying impact of noise injection on topology creation is important with aggregation since noise affects varying frame sizes differently. For our experiments we run a single link with the receiver closer to a noise antenna while the sender is as far as possible. Noise injection on all other antennae (except near the receiver) is turned down by introducing attenuation. Figure 9 shows the percentage of saturation throughput seen on the link with varying levels of noise injected at the receiver. We see that for the same frame sizes, throughput falls faster with aggregation
Performance and stability of 802.11 rate adaptation algorithms is widely proven through implementation on madwifi drivers [23], [20], [27], [7]. Each of these algorithms rely on channel estimation metrics such as ETT (expected transmission time) [23], packet loss [20], delivery ratio [27], throughput [7], or SINR estimates [17]. Through this study we will study the impact of aggregation on rate selection. We setup an experiment to measure the rate selected by the sample rate adaptation algorithm [7] over a single link with maximum transmit power. We select and plot the rate selected per frame for a consecutive sample of 250 sniffed frames in Figure 11(a). Since these measurements are taken under a controlled environment, the channel conditions have been shown to be consistent and repeatable. For the same conditions we observe that results without aggregation settle at 54M most of the time while with aggregation we see an oscillation between 54M and 48M . This performance is attributed to higher frame loss with aggregation and becomes clearer with results in Figure 11(b). This figure plots the distribution of frames transmitted at different rates. We observe that for different transmit powers, the rate adaptation algorithm behaves consistently better without aggregation, sending more
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
48
Aggregation No Aggregation 40 0
50
100 150 Experiment Duration (secs)
200
250
1
0.8
0.6
0.4
0.2
0
No AG
AG
MINIMUM
(a)
No AG
AG
TRANSMIT POWER
No AG
AG
MAXIMUM
(b)
Percentage of Transmitted MAC frames
Percentage of Transmitted MAC Frames
Observed Packet Tx Rate (Mbps)
54
54Mbps PHY Rate 48Mbps PHY Rate 36Mbps PHY Rate
1.2
1.2
54M
48M
36M
24M
18M
12M
9M
1
0.8
0.6
0.4
0.2
0
No AG AG No Noise
No AG AG −39 dBm
No AG AG −37 dBm
(c)
Fig. 11. Measurements made for performance with auto rate adaptation using the sample rate adaptation scheme that is default with the Madwifi drivers. (a)The time plot shows a random snapshot of rates of 250 consecutive transmissions. We observe that the adaptation algorithm settles comfortable with no aggregation at a rate of 54M but does not work so well when aggregation is enabled. (b)Shows the distribution of the rates at which the 50K sniffed packets were sent. We observe that this number varies widely with (AG) and without aggregation (No AG) for different transmit power levels. (c) A similar measurement is made here with varying levels of injected noise at the receiver to emulate deteriorating link conditions. We see a distinct difference in rate selection with (AG) and without (No AG) aggregation.
frames at a higher rate. A comparison with varying noise at the receiver in Figure 11(c) shows a similar trend. For the same injected noise at the receiver, we observe a lower rate selection for aggregation due to a possibly higher packet error rate. Thus rate adaptation algorithms should consider aggregation while comparing results obtained with their evaluations. V. C ONCLUSION A ND F UTURE D IRECTIONS This study highlights conditions under which MAC aggregation occurs and quantifies typical performance variations seen with and without aggregation using both a theoretical model and experimentation. We show that the default aggregation used in commodity 802.11 devices can produce measurements which differ from those of a properly configured controlled experiment, and may thus lead to misleading conclusions. Specifically, we observe significant differences in throughput, number of frames transmitted, sniffed inter-frames times, and inter-frame times as seen at receiver. Topology creation becomes harder with noise injection since link PERs are aggregation dependent. Also, for real time services such as audio and video, despite increase in throughput, jitter increases with aggregation. Future work will involve detecting clients with non-conforming MAC behavior in 802.11 hotspots. R EFERENCES [1] Atheros 802.11 wlan cards. http://atheros.com/. [2] Ieee 802.11a standard. http://standards.ieee.org/ getieee802/download/802.11a-1999.pdf. [3] Ieee 802.11n standard. http://www.ieee802.org/11/index. shtml. [4] Iperf-internet traffic generator. http://dast.nlanr.net/ Projects/Iperf/. [5] Multiband atheros driver for wifi. http://madwifi.org. [6] D. Aguayo, J. Bicket, S. Biswas, G. Judd, and R. Morris. Link-level measurements from an 802.11b mesh network, August 2004. [7] J. Bicket. Bit-rate selection in wireless networks. In Masters thesis, M.I.T., Cambridge, MA, 2005. [8] E. Borgia and F. Delmastro. Effects of unstable links on AODV performance in real testbeds. EURASIP Journal on Wireless Communications and Networking, Jan. 2007. [9] D. Raychaudhuri, I. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa, H. Liu, and M. Singh. Overview of the ORBIT radio grid testbed for evaluation of next-generation wireless network protocols. In IEEE WCNC , March 2005.
[10] P. De, A. Raniwala, S. Sharma, and T. Chiueh. MiNT: a miniaturized network testbed for mobile wireless research. Proceedings of IEEE (INFOCOM), pages 2731–2742, Mar. 2005. [11] Domenico Giustiniano and Giuseppe Bianchi and Luca Scalia and Ilenia Tinnirello. An Explanation for Unexpected 802.11 Outdoor Link-level Measurement Results. In Proceedings of IEEE INFOCOM, April 2008. [12] M. A. Ergin, K. Ramachandran, and M. Gruteser. Understanding the Effect of Access Point Density on Wireless LAN Performance. In Proceedings of ACM MOBICOM, pages 350–353, September 2007. [13] S. Ganu, K. Ramachandran, M. Gruteser, I. Seskar, and J. Deng. Methods for restoring mac layer fairness in 802.11 networks with physical layer capture. In Proceedings of the REALMAN workshop. [14] J. Lorchat and T. Noel. Energy saving in IEEE 802.11 communications using frame aggregation. In IEEE GlobeCom, Dec 2003. [15] K. Lee, S. Yun, I. Kang, and H. Kim. Hop-by-Hop Frame Aggregation for VoIP on Multi-Hop Wireless Networks. In IEEE ICC, May 2008. [16] S. Kaul, M. Gruteser, and I. Seskar. Creating Wireless Multi-hop Topologies on Space-Constrained Indoor Testbeds Through Noise Injection. Tridentcom, 4:2731–2742, Mar. 2006. [17] T.-S. Kim, H. Lim, and J. C. Hou. Improving spatial reuse through tuning transmit power, carrier sense threshold, and data rate in multihop wireless networks. In In the proceedings of MobiCom. [18] Y. Kim, S. Choi, K. Jang, and H. Hwang. Throughput enhancement of IEEE 802.11 WLAN via frame aggregation. In In the proceedings of VTC, pages 3030–3034, September 2004. [19] L. Stoller and R. Ricci and S. Guruprasad and M. Newbold. An integrated experimental environment for distributed systems and network. In Proceedings of OSDI. [20] M. Lacage, M. H. Manshaei, and T. Turletti. Ieee 802.11 rate adaptation: A practical approach. In In proceedings of MSWIM, 2004. [21] R. Mahindra, G. Bhanage, G. Hadjicristofi, I. Seskar, D. Raychaudhuri, and Y. Zhang. Space versus time separation for wireless virtualization on an indoor grid. Proceedings for IEEE NGI, 2008. [22] S. McCreary and K. Claffy. Trends in wide area IP traffic patterns - A view from ames Internet exchange. [23] K. Ramachandran, R. Kokku, M. Gruteser, and H. Zhang. Synchronous Two-phase Rate and Power Control in 802.11 WLANs. In In proceedings of ACM MobiSys, pages 350–353, July 2008. [24] S. Yun, H. Kim, H. Lee, and I. Kang. 100+ VoIP Calls on 802.11b. In IEEE Journal SELECTED AREAS IN COMMUNICATIONS, May 2007. [25] V. A. Siris and G. Stamatakis. Optimal cwmin selection for achieving proportional fairness in multi-rate 802.11e wlans: Test-bed implementation and evaluation. In in the proceedings of ACM WINTECH, 2006. [26] T. Li, Q. Ni, D. Malone, D. Leith, T. Turletti, and Y. Xiao. Aggregation with Fragment Retransmission in Very High-Speed WLANs. In IEEE/ACM Transcations on Networking, March 2006. [27] S. H. Y. Wong, H. Yang, S. Lu, and V. Bharghavan. Robust rate adaptation for 802.11 wireless networks. In in the proceedings of ACM MobiCom. [28] Z. Wu, S. Ganu, and D. Raychaudhuri. IRMA: Integrated Routing and MAC Scheduling in Multi-hop Wireless Mesh Networks. WiMesh, 2006.
978-1-4244-4148-8/09/$25.00 ©2009 This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2009 proceedings.
