The LevelOne model is based on the Atheros 11b/g chipset and the Z-Com model carries the popular Prism chipset. The Cisco device is a popular AP commonly ...
Monitoring wireless networks: performance assessment of sniffer architectures M. Portol´es, M. Requena, J. Mangues, M. Cardenete
Publication: Vol.: pp.: No.: Date:
in Proc. of the IEEE International Conference on Communications (ICC 2006) Istanbul (Turkey). June 11-15, 2006
This publication has been included here just to facilitate downloads to those people asking for personal use copies. This material may be published at copyrighted journals or conference proceedings, so personal use of the download is required. In particular, publications from IEEE have to be downloaded according to the following IEEE note: c °2007 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Monitoring wireless networks: performance assessment of sniffer architectures* Marc Portoles-Comeras, Manuel Requena-Esteso, Josep Mangues-Bafalluy, Marc Cardenete -Suriol Centre Tecnològic de Telecomunicacions de Catalunya (CTTC) Parc Mediterrani de la Tecnologia (PMT) Av. Canal Olímpic S/N 08860 Castelldefels - Barcelona - Spain {marc.portoles, manuel.requena, josep.mangues, marc.cardenete}@cttc.es Abstract—This paper assesses the performance of wireless sniffers built using off-the-shelf hardware and software. Such an analysis tool is commonly found in wireless testbeds, given its low cost. The goal of this study is to provide a framework to assess the performance of measurements obtained using this popular tool. Among various contributions, the study shows the importance of precisely checking the behavior of the hardware used for communications and for monitoring in order to validate the traces obtained. It also presents a methodology to assess the performance of wireless sniffers based on obtaining a correlation factor between traces captured by independent devices. The paper reveals, in addition, a new type of loss in sniffer captures, the saturation loss, that completes previous work on this topic. Finally, the paper also studies the behavior of architectures using several sniffers in a single computer. It reveals how, again in this case, the hardware chosen is critical to obtain desired performance. Keywords: wireless networking, monitoring, tuning, exprimentation
meaurements,
sniffers,
I. INTRODUCTION Wireless networking research is progressively moving to an experimental plane. There is the need for experimentally validating the extensive research effort done during recent years. This research has mainly been based on simulation studies but now, in its path to industrial production, requires more realistic deployments. An important component of experimental research are measurements. Traditionally networking distinguishes two types of measurements: active and passive. The former are those based on injecting synthetic traffic flows in a network and the latter, those based on (transparently) collecting and analyzing the traffic observed at a certain point of the network. Active measurements are used to characterized path segments while passive measurements serve to characterize the traffic and other operational parameters (e.g loads) at any particular point of the network. Both may be seen as complementary. Wireless sniffers are the most common passive measurement instruments used in current WLAN networking research. These are packet capture engines that passively monitor the wireless medium capturing (non-intrusively) passing traffic. Their functionality is, essentially, the same as their LAN counterparts but challenged by the specific additional constraints that wireless networking imposes (e.g.
*Part of this work has been partially funded by Generalitat de Catalunya under grant number SGR2005-00690 (Grup de Recerca Singular)
randomness of the channel propagation behavior or resource constraints of the measurement machines). A commonly used wireless sniffer setup consists in attaching a WLAN device to a computing system (desktop, laptop, PDA,…). The WLAN device is then configured to run in “monitor” mode by means of which it passes up to the computing system all packets observed in the channel under observation. These are called off-the-shelf wireless sniffers. Despite the advantages of using an off-the-shelf wireless sniffer built up in such a manner (e.g. low cost, high availability,…), analyzing the traces obtained must be done with care. A pioneering study done at the University of Maryland by Yeo et al. (see [1], [2] and [3] ) reveals that this monitoring architecture is prone to losing an important part of the traffic being actually transmitted over the wireless medium. This loss may depend on factors such as the hardware being used, the propagation environment, the nature of the transmitting source, the position of the monitor, etc. The work presented in this paper extends the work of Yeo et al. identifying new possible reasons for traffic loss and providing a more detailed view on the performance of off-theshelf wireless sniffers. Additionally, the study covers a wider range of wireless measurement architectures adding more than one sniffer in a single machine and monitoring more than one channel at a time. The paper is organized as follows. The next section presents some related work that has analyzed performance of wireless measurement devices. Section 3 provides a description of the system setup used for the experimental study and presents some preliminary work done to appropriately tune the performance of the devices and tools used. Section 4 covers the results obtained in this study presenting the several contributions of the paper. Finally the paper concludes in section 5 and highlights some future research lines. II. BACKGROUND AND RELATED WORK Wireless measurements have gained an important role in current experimental wireless networking research. This importance has led to the creation of specific workshops (e.g. WiNMEE [4]) that try to put together diverse experiences on the topic. An increasing number of papers use measurements in the studies presented, and some of them use off-the-shelf wireless
sniffers as a measurement tool. However, not many of them have paid attention on validating the actual performance of the monitoring tool itself.
chipset and the Z-Com model carries the popular Prism chipset. The Cisco device is a popular AP commonly found WLAN enabled spaces.
A pioneering work done at the University of Maryland (see [1], [2] and [3] ) shows that care should be taken when analyzing traces obtained using an off-the-shelf wireless sniffer. The study reveals that sniffers may lose an important part of the traffic due to various causes. Specifically the authors identify three loss sources: (1) frame loss, frames present in the air but not detected by the sniffer; (2) type loss, packet types that the sniffer is inherently unable to capture; and (3) AP loss, sniffers that lose all packets coming from specific AP hardware models. As shown later in the paper we have identified a new type of loss called saturation loss that refers to the fact that some sniffers can only capture up to a certain packet rate level.
Wireless devices are controlled using computer nodes of the EXTREME cluster. In all cases these nodes are Pentium IV PCs with a 3GHz processor, 512MB of RAM memory and running Linux OS, with kernel 2.4.26. EXTREME automation system makes an extensive use of the wireless extensions API [8] to configure and control wireless devices. The Madwifi driver [9] supports this API and controls LevelOne cards. The hostap driver [10] also supports this API and controls Z-COM devices. Besides, a customized set of tools have been developed to automate the configuration of the AP1200 following EXTREME model. These tools are based on perl scripting and the Net::Telnet::Cisco library [11].
The study at Maryland continues observing that by combining trace captures of various sniffers one can obtain a better picture of the traffic present on the wireless medium. The authors propose a novel method to combine these traces taking advantage of the low-level properties of the IEEE 802.11 WLAN protocol to synchronize them. In the present paper we show that the effectivity of such a technique can be assessed determining the (un)correlation between the traces captured by the different sniffers. Another preliminary study that has influenced the current one is the one carried out at Intel Research Cambridge and presented in [6]. Some important observations can be drawn from that study. The first one is that correctly tuning devices used to take measurements can make a significant difference in the conclusions derived from experimental results. The second one is that not only the passive elements (e.g. wireless sniffers) but also the active elements (e.g. wireless traffic probes) have to be correctly tuned before fulfilling effective experimentation. Additionally, and more specifically, the authors in [6] reveal that some devices (based on the popular Prism chipset) when acting as sniffers, cannot work in a single computer without affecting each other. This observation is further developed here. III.
SYSTEM DESCRIPTION
This section details the system setup prepared for experimentation and provides some hints and observations on the performance and tuning of the tools used. A. Scenario set up All experiments have been carried out within the EXTREME framework (see [7]). This is a multi-purpose networking experimental platform currently under development within the Centre Tecnològic de Telecomunicacions the Catalunya (CTTC) in Barcelona. The main advantage of this platform is its high automation capabilities that allow automatic execution, data collection and data processing of several repetitions of a experiment. Three types of wireless devices have been used. These are: Cisco AP1200, LevelOne WNC-0300 and Z-COM ZDC XI626. The LevelOne model is based on the Atheros 11b/g
The study presented here focuses on the efficiency of system architectures for carrying out wireless measurements rather than on the effects of channel propagation on these measurements. This is the reason why all communications between wireless devices are transmitted through coaxial wiring. As can be seen in Figure 1 all wireless devices are connected to a central bank of splitters and combiners. This bank of splitters replicates with very low attenuation (in comparison to open-air propagation) all signal inputs in each of its ports to the rest of ports. The bank of splitters and combiners is composed of minicircuit ZX10-4-27 splitters (with 4 ports) and minicircuit ZFSC-2-10G splitters (with 2 ports). EXTREME Bank of splitter/ combiners
Transmitter(AP)
Sniffer
Receiver(STA)
Sniffer
Experiment Server and Data Collector
Figure 1. Scenario setup
The scenario setup used during experimentation is depicted in Figure 1. In all cases a transmitter sends a UDP stream to a sink receiver. UDP streams are sent at different packet rates using packets of minimum size (The application used in our case allows a minimum of 64 bytes of application payload). The bank of splitters/combiners replicates the UDP stream to a group of wireless sniffers that store packet captures for data analysis at the end. The receiver also stores the packets received for the analysis at the end. As is shown below in the paper, various combinations of transmitter/receiver/sniffer are considered to fulfill study purposes and using the varied hardware available. Finally, the application used to generate UDP streams is the Multi-GENerator toolset (MGEN [11]). The reason to choose this application is double fold. On one side the traffic source can activate an option called “precise on” that efficiently controls real-time generation of packets to guarantee user-
demanded generation rates. On the other side the traffic sink is able to store received packets for later processing. The minimum (application layer) payload size that this application allows is of 64 bytes.
allows higher operation throughputs. Both are considered in the figure, where one may notice the difference on the maximum packet rate achieved depending on whether pre-backoff is applied or not.
All results in this paper are obtained by repeating 10 times the same experiment in identical conditions.
The parameters used to generate the figure (obtained from the standard) are summarized in the following table,
B. Preliminary assessment and tuning of experimentation tools This section details some preliminary work done to tune the performance of the tools and equipment that is used during the experimental study. Specifically we analyze the performance of the traffic generation tool together with the hardware available for wireless networking communications. This results are compared to the theoretical bounds imposed by IEEE 802.11 standard protocol.
Table 1. Parameters used in numerical representation Parameter 802.11b
802.11b Parameter 802.11b 802.11b (short (short preamble) preamble)
Tphy
48µs
24µs
CWmin
31
31
Tslot
20µs
20µs
