node monitoring and results evaluation, as well as a set of methods used to gather and ... Testbed). In addition to the tools that are already parts of the testbed environ- ment and ... cO Springer-Verlag Berlin Heidelberg 2012 ..... file, it is possible to prepare all necessary software on the server side and then distribute it to all ...
wnPUT Testbed Experimentation Framework Adam Nowak, Przemyslaw Walkowiak, Andrzej Szwabe, and Pawel Misiorek Institute of Control and Information Engineering Poznan University of Technology M. Curie-Sklodowskiej 5, 60-965 Poznan, Poland {Adam.Nowak,Przemyslaw.Walkowiak,Andrzej.Szwabe, Pawel.Misiorek}@put.poznan.pl Abstract. In this paper, we present a MANET experimentation framework developed at Poznan University of Technology within the EU FP7 OPNEX project including its main part called the wnPUT testbed. The presentation is complemented by description of the state-of-the art in research on wireless testbed development and experimentation methodology. The main goal of the wnPUT testbed environment is to support the first stage of experimental research on backpressure-based wireless networks. From this perspective, wnPUT is a small-scale testbed with the aim of ‘preliminary’ experimentation supporting target activities realized subsequently in large-scale testbed environments. The paper contributes with a description of the testbed development process, tested functionalities, as well as hardware and software characteristics. In particular, we analyze experiment execution phases such as network configuration, node monitoring and results evaluation, as well as a set of methods used to gather and visualize statistics. Keywords: MANET, wireless testbeds, delay-awareness, multi-path routing, home networking, heterogeneous networks, IMS.
1
Introduction
A large part of studies on Mobile Ad-hoc Networks (MANETs) has been based on experiments in simulated network environments. Along with advances in the research on network algorithms, the need for developing non-simulation-based testbeds rise significantly. As the number of experimental case-studies grows, the experiment execution itself gets more and more complex. Therefore, it is hard to achieve results of satisfying reliability without the use of any specialized tools that allow to effectively manage the entire experimentation process. The wnPUT testbed was designed to support the experiments evaluation within studies on backpressure-based wireless network control algorithms constituting the core of Delay-Aware Utility Maximization System (DANUMS) [14]. Furthermore, the wnPUT testbed was used as a platform for preliminary experimentation performed before remote evaluation of DANUMS and multi-path OLSR in other, large-scale testbeds (i.e., within the NITOS testbed and DESTestbed). In addition to the tools that are already parts of the testbed environment and presented further in this paper, we also describe a framework which L. Bononi et al. (Eds.): ICDCN 2012, LNCS 7129, pp. 367–381, 2012. c Springer-Verlag Berlin Heidelberg 2012
368
A. Nowak et al.
may be used to introduce additional source code classes extending the testbed functionality. From the perspective of the authors of this paper, using a physical testbed environment was crucial – results of the experiments conducted previously in ns2 simulation environment [7] have not been satisfying in terms of compatibility with real behavior of nodes. Experiments performed by designers of wnPUT were focused on backpressurerelated scheduling and multi-path routing. The current state of wnPUT is based on the experience gained with efforts realized in [10, 11]. Scenarios, which are the main target for wnPUT experiments are related to the DANUMS application in a home-based context, where many of wireless devices exist in a narrow space. The structure of the wnPUT, which consists of notebook devices, enables realizing the scenarios unavailable in other testbeds due to their remote-only accessibility. Additionally, the wnPUT testbed enables to experimentally evaluate scenarios involving (and requiring) interaction with users, such as audio-video conversations or other actions requiring the user activity. This paper presents wnPUT testbed which continues best practices gathered from other researchers’ experiences. The introduced framework enables experiment evaluation based on scenario description, which is manageable in an easy way. In addition to the remote execution of experiments, the wnPUT testbed permits the user presence in experiment evaluation process, enabling experiments on audio-video conversations in a specified topology and declared conditions. The remainder of this paper is structured as follows: Section 2 describes the related work in the area of testbed environments designed in the last years. Methodology of research and experimentation performed on the wnPUT testbed is described in Sec. 3. The detailed description of wnPUT testbed environment is presented in Sec. 4. The presentation of wnPUT is followed by presentation of the wnPUT experimentation phases in Sec. 5 and description of a sample experiment execution in the testbed in Sec. 6. The last part of the paper contains its summary presented in Sec. 7.
2
Related Work
The most of the existing testbeds are strictly connected with education and commercial facilities, where researchers are focused on improving the algorithms and technologies. The DES-Testbed at Freie Universit¨at in Berlin [21] and NITOS testbed in Volos [20] are examples of such environments. Social ventures, such like the Freifunk [17] also become more and more popular and get interest from city authorities which may help in their expansion and improvement. However, such solutions should not be strictly regarded as testbeds – they play the role of experimental networks which feature some of the testbed abilities, such as performing experiments and gathering statistics. In the last 5 years, there have been developed at least a few testbeds which serve as an experimentation environments for the purposes of networking researchers. As mentioned by the authors of [1], the trend towards testbed
wnPUT Testbed Experimentation Framework
369
environments rises constantly since 2001 when one of the first ad hoc-based testbed – Ad hoc Protocol Evaluation (APE) has been released in order to evaluate protocols for MANETs [15]. Another example of solution developed in that period of time is MiNT [16], which is known from its miniaturized 802.11 testbed gathered in the close area. The connections between nodes and topology itself are established thanks to the application of the limited radio range reduced by antenna attenuators [3]. Open Access Research Testbed for Next-Generation Wireless Networks (ORBIT) [22] is a project hosted at WINLAB in Rutgers University, which began in 2005. The project uses a large two-dimensional grid of 400 802.11 radio nodes, enabling various topologies thanks to the functionality of an experiment scenario definition. The Distributed Embedded Systems Testbed (DES-Testbed) located in Computer Systems & Telematics (CST) at the Institute of Computer Science, Freie Universit¨ at Berlin is a hybrid wireless multi-transceiver testbed for long-term studies, consisting of a wireless mesh network (WMN) and a wireless sensor network (WSN). Currently, there is 115 wireless mesh routers equipped with three or more IEEE 802.11a/b/g network adapters and the same number of wireless sensor nodes of the MSB-A2 type. Testbed offers a full remote access to experimentation – researchers may provide their own system image files and experiment descriptions defined in DES-Cript language [5]. DES-Testbed provides repeatable and reliable experimentation in a real world scenarios. One of CERTH laboratories – NITLab – has developed a wireless testbed called Network Implementation Testbed using Open Source platforms (NITOS). This platform is available to researchers in order to enable the evaluation of experimental protocols working in a wireless environment. Devices of the NITOS testbed are based on commercial Wifi cards and Linux open source drivers. Management is realized be means of cOntrol and Management Framework (OMF) [8] open-source software. NITOS wireless testbed has been developed as a part of the wireless facilities of the European project OneLab2 [19] and is deployed outside of the University of Thessaly (UTH) campus building.
3
Experimentation Methodology
This section describes experimentation methodology used by authors when developing the wnPUT testbed environment and evaluating the experiment scenarios with DANUMS. 3.1
The Purpose of the wnPUT Testbed
Despite the availability of remotely-accessible large-scale testbeds, we have decided to develop our own small experimental environment. This enabled us to demonstrate that development of a local small-scale testbed improves the effectiveness of experimentation performed in remote large-scale testbeds like DESTestbed [21]. The main reason for this decision was the fact that in its early implementation stages our experimental software was faced with software ‘bugs’,
370
A. Nowak et al.
which severely affected the performance of the testbed environment, frequently leading to problems and errors at the kernel level of the operating system. By performing our experiments locally, we were able to evaluate our research ideas in a more effective way and adjust our algorithms frequently within a cyclic performance evaluation process. As the scale of our testbed was small, it was possible for us to achieve a relatively high controllability of the experimentation process. This way, in comparison with DES-Testbed or the NITOS testbed, our wnPUT testbed can be seen as a ‘pre-testbed’ solution. 3.2
Research and Experimentation Cycles
Figure 1a presents the process of developing the DANUM system [14] (the main OPNEX solution evaluated in our testbed) as a research and experimentation cycle. Special attention was paid here to revealing the feedback from the analysis of the experimentation results. The experimentation results were used to improve the model and the next-stage of implementation. From the perspective of theoretical research, our preliminary objective was to investigate how the backpressure-based Max-Weight Scheduling (MWS) may be exploited for throughput maximization in MANET networks. The results of the first experiments exposed the fact that the application of MWS in the MANET environment typically requires modifications of the MAC layer providing schedulers with direct access to the queuing system [14]. Initially we followed the approach of the classical Network Utility Maximization (NUM) [6], for which the optimization goal was to maximize the summarized network utility [4]. In this approach, only the utility functions of rate are used, since the classical NUM model does not explicitly express the delay-dependent utility factor [4]. As a consequence, such a solution is not effective when applied to a system serving simultaneously files and multimedia streams. In order to resolve this issue, the Delay-Aware NUM problem was formulated [14]. The new formulation was based on the introduction of a new optimization variable and a novel definition of a virtual queuing system transporting virtual ‘utility units’. With the formulated Delay-Aware NUM problem, the delay-aware utility (as a function of the new optimization variable) was chosen as the main evaluation criterion. At the final stage, the utility functions for different classes of traffic were redefined [13], [14]. Thanks to the experimental evaluation performed for each version of the implementation, we were able to confirm the overall utility gain at each of the late DANUMS development phases [14]. In particular, the ability of ‘soft’ admission control of the DANUM system was shown [14]. Experiments conducted at the first development stage were focused on the scenario of serving TCP flows in networks of various topologies. When the per-flow queuing control system was implemented, the experiments were extended in order to investigate the simultaneous service of files and multimedia streams. At this very important phase of the DANUMS development, utility was introduced as the key performance measure [13], [14]. The final experimentation concentrated on the use of DANUMS in various topologies in IMS-based audiovisual streaming scenarios.
wnPUT Testbed Experimentation Framework
371
Theoretical research Implementation Experimentation and analysis Backpressurebased MWS throughput m maximization
Queue level reporting protocol and the MWSbased scheduler
TCP flows on various topologies
A typical implementation of MWS: the lack of compatibility with some network devices
Estimation of the MAC layer congestion
TCP flows on various topologies Theoretical research and analysis
Implementation
Experimentation
Unconstrained backpressure approach
Implementation of a backpressure MWS module and a queue level reporting protocol
Testing streaming media; Delay and packet loss ratio as performance measures
Multi-path extension of the OLSR used for backpressureoriented 'preliminary routing'
Network capacity and transmission reliability used as performance measures in a small testbed
Adjustment of protocol parameters including messageexchange intervals
Effective operation of the multi-path extension in largescale testbeds (DESTestbed, NITOSTestbed)
Summarized throughput not corresponding to the maximization of user-perceived utility
Per-flow queuing control giving fine-grained service differentiation capability
Simultaneous service of files and multimedia streams
Ineffectiveness of the classical NUM in a multi-service network
Virtual queuing, end-to-end delay and rate monitoring, sender-side virtual units rate control
The first DANUM system controlling networks of various topologies
Feedback: large delays even in an empty network; routing loops issues; backward traffic Countermeasures: anew multi-path algorithm based on OLSR protocol
Optimized delayaware utility functions for different classes of traffic
Adjustment of system parameters and utility function parameters
Evaluation of the second generation DANUM system; IMS scenarios
Feedback: increased reliability confirmed; No capacity gain in a small network
(a)
(b)
Fig. 1. (a) Methodological perspective of research and experimentation on the DANUM system; (b) Research and experimentation on multi-path backpressure-controlled routing and packet forwarding
In parallel to the activities aimed at the development of the DANUMS system, the wnPUT testbed was also employed in the work on multi-path routing solutions [11]. Figure 1b presents the process of the development of a multipath OLSR extension supported by the experimentation performed within the wnPUT testbed. The first multi-path packet forwarding solution provided as a result of this process was based on the classical, i.e. unconstrained backpressure algorithm. After identifying the shortcomings of the method, we developed a modified algorithm for a multiple routes calculation [10]. This solution was evaluated as a convenient means for supporting backpressure scheduling [11]. Initially, our experiments were conducted in a small-topology network. Streaming media flows were used as an experimentation base. Delay and the packet loss ratio were applied as the key performance measures. As soon as the multi-path extension of OLSR was implemented, our experiments were aimed at measuring the end-to-end throughput and evaluating transmission reliability. Such an approach unveiled a higher invulnerability to node failures. This was achieved
372
A. Nowak et al.
thanks to the availability of less outdated information about alternative paths. With confirmed benefits from multi-path routing applied in a small-scale testbed, the focus of the experimentation shifted to activities within a larger DES-Testbed platform, which enabled the remote evaluation of the proposed solutions in scenarios with more complex network topologies. The results obtained this way allow us to state that the ‘preliminary’ experimentation on wnPUT made it possible to effectively use our gradually gathered experience to provide valuable output of experiments involving the application of the backpressure-based multi-path OLSR in a dense-topology network [12]. 3.3
Testbed Development Cycles
The OPNEX experimentation activities iteratively influenced the wnPUT testbed development. Here we focus on the video-over-wireless scenario (see Fig. 2a) and on the issue of the experimental network scalability (see Fig. 2b). Figure 2a presents dependencies between the process of wnPUT testbed development and the gradually identified requirements imposed by the 3G IMS video-calling scenarios. At the first phase, we deployed an IMS video-calling application using the DANUM system in a small network. At this stage, basic experimentation tools like Wireshark and Iperf were employed for packet monitoring and traffic generation. The testbed was divided into two separate components: (i) a server responsible for experiment management and monitoring, (ii) autonomous wireless network nodes. The wnPUT server provided a set of tools (described in detail in Sec. 4) necessary for the experimentation process. The DANUM system was installed on each testbed node. It is worth noting that the DANUMS node implementation may be deployed in any Linux-kernel-based operating system, e.g., it may be installed on the bootable USB flash drive. In order to evaluate IMS 3G video-calling scenarios, the Open Core IMS was integrated with the wnPUT testbed. In particular, the wnPUT nodes (playing the role of IMS video-calling clients) used the OLSR HNA [2] packet functionality for communication with an external network [9]. The IMS video-calling scenarios require additional testbed functionalities. In order to analyze the performance of video-calling from the delay-aware perspective, it was necessary to monitor additional variables such as queue states (including virtual queues) and end-to-end delay. Moreover, the IMS video-calling scenario required the incorporation of the Open Core IMS framework within the wnPUT testbed [9]. In the next phase of the testbed development, we observed that experiments should be conducted in scenarios as close to ‘real-world’ as possible, especially in the case of experiments with networks serving heterogeneous traffic (i.e., file transmissions and audio-video streaming). The experiments allowed to confirm the ability of the DANUM system to ensure ‘fair coexistence’ of media streams and file transfers [13]. Moreover, the experiments showed that the DANUM system can realize ‘soft’ admission control, and increase the overall utility for networks controlled by DANUMS [14].
wnPUT Testbed Experimentation Framework Assumptions, evaluation, analysis
Testbed structure and management
Scenarios oriented on multimedia streaming (video over wireless)
First implementation of the DANUM System in a small-scale network: monitoring realized by Wireshark, Iperf used as a traffic generator
Need for the testbed structure enabling effective experimentation
Testbed architecture divided into: management server and wireless network nodes
Need for a node-internal monitoring: queue levels, end-to-end delay, rate; Need for tool enabling testbed and experiment management
Deployment of custom tools monitoring the required parameters; Testbed and experiment management
Media streaming experiments showing the ability of 'soft' admission control; Experiment scenarios similar to 'real-world' applications; Audio-video over wireless
Iperf used to simulate RTP/UDP streams and TCP files
The basic level of TCPfriendliness of media streams assured by the DANUM system; The need for experiments in `real world' scenario based on the IMS technology
Open Core IMS wired network installation integrated with a wireless wnPUT testbed (playing the role of an access network)
Assumptions, evaluation, analysis
Testbed structure and management
Video over wireless in networks serving streams and files;
NS-2 implementation in various topologies of different scales
Delay-awareness imposing the application of DANUMS in networks with limited number of hops;
Experiments in single-hop mesh networks
Overall utility gain and basic IMS-readiness in fully-connected 1-hop mesh networks; 'Real-world' applicability limited to small networks
Multi-hop topology with the number of hops on routing paths limited to 2 (wnPUT testbed)
Overall utility gain and the basic IMS-readiness in 2-hop networks; Applicability still limited to small networks
Multi-hop topology (remote-accessible DESTestbed and NITOSTestbed); Custom tools for topology monitoring and visualization.
373
Experiments confirming the overall utility gain and the basic IMSreadiness of DANUMScontrolled MANETs
Basic IMS-readiness of DANUMS-controlled MANET;
(a)
(b)
Fig. 2. Implementing and evaluating scenarios on the wnPUT testbed: (a) ‘real world’ video over wireless scenario – 3G IMS video-calling; (b) the scalability perspective
Figure 2b presents the influence of the scalability requirements on some key decisions made during the wnPUT testbed development process. Each step presented in the figure assumed the necessity to deal with a larger topology than the one used in the previous step, beginning from single-hop fully connected mesh networks and finishing at multi-hop networks containing more than 20 nodes. At the beginning, experiments were realized by means of network simulator (ns-2) [10]. When the first version of the wnPUT testbed was successfully deployed, experimentation in a real-world environment was possible [11]. After the experiments conducted on a 2-hop wireless network controlled by DANUMS, we started experiments on more complex network topologies based on remotely-accessible testbeds located in Berlin (DES-Testbed ) and Volos (NITOSTestbed ) [12]. In [12], we present the results from the final experimentation stage, which confirmed that in the case of a saturated network, the application of the proposed backpressure-based multi-path extension of the OLSR protocol allows for more stable multi-hop transmission than the standard OLSR does.
374
4
A. Nowak et al.
wnPUT Infrastructure
Testbed developed at PUT, called wnPUT, is aimed at targeting mainly home networking scenarios, which are characterized by a limited number of nodes and a small number of hops (mainly one or two) between them. The main purpose of this approach is to emulate the properties of small (but dense) networks, which are widely used these days. Two types of network have been introduced to the wnPUT in order to ensure the separation between wireless nodes and the management server. The first one – the wired management network is responsible for the management process affecting both the server and nodes. This includes communication, software and data synchronization, scenario upload, etc. The main goal of the second part of the testbed (the wireless one) is to constitute an experimentation platform, where all actions defined in the scenario are conducted. As presented in Fig. 3a, both networks coexist simultaneously and play the role which corresponds to their purposes.
wnPUT server
management data exchange
wnPUT w nPUT node management network
experiment network
(wired)
wnPUT node wnPUT Server
(wireless)
(a)
(b)
Fig. 3. (a) The wnPUT architecture represented by two kinds of networks: wireless – experimental (operational) and wired (lying beneath the wireless infrastructure) – for management purposes; (b) The location of wnPUT testbed nodes
Nodes of the wnPUT testbed are distributed in various office rooms in one of the Poznan University of Technology buildings, as presented in Fig. 3b. A set of tools has been prepared in order to realize tasks of distinct operation phases, and to enable performing new experiments with a minimal group of actions required from a testbed operator. This set of tools enables convenient way for collecting the experiment results, without a need for specialized knowledge in the area of advanced Linux functions. The universal logging API have been prepared for the purposes of data collecting. This API enables saving selected information from experiment evaluation in
wnPUT Testbed Experimentation Framework
375
a log file. ‘Printing’ the information may be performed by including the methods from the log API in the source code. Such an approach guarantees a high level customization of gathered informations and their possible usage. The wnPUT Builder tool intended to build the appropriate version of software, has been designed in order to support various architectures of nodes available in the network. Therefore, it is not required to have the same Linux kernel version nor the same architecture for each node. Thanks to the advanced Makefile, it is possible to prepare all necessary software on the server side and then distribute it to all nodes during the synchronization process. Each new node may be added to the testbed and extend its network topology without high costs or requirements. The set of demands is limited to a working Linux distribution (Ubuntu 10 is preferred), Ethernet and wireless network cards. Once the wired connection is established, the operator needs to execute the wnPUT Welcome script, which will add the new node to the database and synchronize appropriate software using remote ssh commands.
5
Experiment Execution Phases
The experiment evaluation process within the wnPUT testbed consists of the following steps: Init Phase (Management System Initialization), Sync Phase (Testbed Software Synchronization), NetInit Phase (Network Initialization), Exp Phase (Experiment Execution and Evaluation), Monitoring Phase (Network Monitoring), Viz Phase (Results Visualization). The order of the phases execution is shown in Fig. 4. In order to prepare and evaluate the testbed, an XML-config file need to be delivered. The solution is similar to the idea of DES-Cript [5], which may be seen as the abstraction layer for the experiment evaluation, encapsulating all the data required by the particular ISO/OSI layers in a simple and straightforward format. The key was to separate the issues of experiment description from the technical matter. Another advantage of XML-based config file is the fact, that the entire testbed configuration is available in one place. This file contains the entire data required by a particular preparation or evaluation step. Init Phase: In the Init Phase, the wired management network (independent of the proper wireless network) has to be set up before any action is taken. IP addresses within this network are assigned using the DHCP protocol. The DHCP daemon configuration file is generated according to the testbed XML-based description, which provides information about nodes and their Ethernet Network Interface Cards (NICs). The mentioned file is used by the DHCP daemon on the server connected to the management network. As the results of actions performed in this phase, a node is ready for remote management and may be utilized by operator using the management tools included in the testbed. Sync Phase: During the Sync Phase, required software and configuration are updated on each node. This is realized by checking if the server does not host a newer version of the software than the one which is used on wireless testbed
376
A. Nowak et al.
- establishing management network - enabling remote access
Init
- software synchronized - experiment config sent
Sync
- setting experiment topology - loading software & kernel modules
NetInit
Exp
- evaluating experiment according to actions enclosed in scenario - gathering experiment logs
Monitoring
- monitoring parameters - observing the current network state
Viz
- visualizing summarized results - analyzing experiment
Fig. 4. wnPUT experiment evaluation phases and their results
nodes. Configuration data and testbed software are distributed by means of the management network and are synchronized thanks to the application of the rsync command, which uses ssh to operate properly. Using ssh ensures a unified approach for the authorization purposes according to the server private key. The rsync module minimizes the network load and the transfer duration by means of a comparison mechanism. Moreover, the synchronization of the software and settings is performed simultaneously on each node. Thanks to this property, the time necessary for this phase does not increase proportionally to the number of nodes. The testbed XML-based configuration file contains the list of files common to each wireless node as well as the list of files specific to a given node, which enables the support for nodes of various architectures and Linux kernel versions. NetInit Phase: The NetInit Phase consists of all the remaining processes necessary to connect node with the wireless testbed. The wireless interface of each node is configured (which includes its operation mode, tx power and additional options) and an IP address is assigned. Moreover, the initial network topology is set by using commonly available and used commands (i.e. iwconfig, ip, iptables). Furthermore, the required Linux kernel modules are loaded (e.g., netconsole or any experiment-related modules). The wireless network topology is simulated according to the scenario description modeled by means of appropriate iptables rules. Exp Phase: The proper experiment execution is possible only if all the previous phases (i.e. Init Phase, Sync Phase and NetInit Phase) are completed. The Exp Phase is controlled by two factors: scenario configuration files and testbed operator’s actions. Configuration files contain the description of experiment scenarios, which are responsible for handling both topology data and node behavior during the experiment execution. Such descriptions support managing the entire experiment process by enabling the customization of the network in many
wnPUT Testbed Experimentation Framework
377
ways (e.g., links between nodes can be turned on/off, transmission parameters can be modified whenever necessary). The iperf is used to generate UDP and TCP traffic with a specified throughput and duration, which allows to perform various traffic generation scenarios. Monitoring Phase: In Monitoring Phase nodes states are supervised. As a result, various parameters may be visualized. This phase timeline is connected with the experiment time itself – observation is made in real time. Monitoring is based on Kst, which is described as one of the most effective plotting tool dedicated to real-time data visualization [18]. Moreover, Kst features in many powerful built-in components and is expandable with plugins and extensions. The Kst plotting tool acquires the data directly from log files provided by the Logger, mentioned later in this paper. Thanks to the fact that this application may read directly from the selected stream and real-time monitoring of nodes parameters may be achieved. Viz Phase: Viz Phase is the final stage of experimentation. In this phase, the plots with desired data are gathered and are available for the testbed operator to be used as a basis for final experiment analysis.
6
Experimentation Process
In this section we present a sample experiment performed within the wnPUT testbed. It is worth being noted that in order to conduct any experiment within the testbed, all the phases mentioned in Sec. 5 need to be completed.
UDP1 source
UDP destination
TCP source
N1
N7
N4
N2
N6
UDP2 source
N3 N5 TCP destination
Wireless links UDP Traffic direction TCP Traffic direction
Fig. 5. Topology used in a sample experiment
378
6.1
A. Nowak et al.
Experiment Preparation
Before the start of the actual experiment execution, the first three phases (see Fig. 4) must be completed. Firstly, the management network has to be set up, in a way specified in the description of Init Phase (see Sec. 5), and then the software has to be synchronized in the Sync Phase. At this stage, a scenario description containing the desired topology schema and all the steps to be executed during the experiment should be provided. According to the NetInit Phase definition, the topology may be set when the file with a experiment description is provided. Listing 1.1 presents the entries of experiment description which has been realized for to the topology presented in Fig. 5. In the presented example, the experiment has been planed as the transmission of two UDP flows and one TCP flow competing for the network resources. This specific experiment has been arranged in order to illustrate the benefits of DANUMS in a wireless mesh network. ... i p e r f −s −u i p e r f −c N7 −u −b 10m Listing 1.1. Fragment of a sample scenario file
6.2
Experiment Execution
Experiment execution – the core of Exp Phase is limited to selection of a previously designed scenario description (an XML file) and the proper experiment execution. During this process, various parameters of the experiment may be observed, including the throughput and queues levels. The preview of the monitoring tool is presented in Fig. 6.
wnPUT Testbed Experimentation Framework
379
Fig. 6. Node monitoring realized by kst
Each experiment may be interrupted at any time, e.g., when the current results suggest that the system does not work in a proper way. 6.3
Experiment Results Processing
After the end of the experiment, log files containing nodes’ activity are made available on the server. All further processing actions are based on the data recorded in these files. A sample content of the log file is presented in Listing 1.2. [ 2 1 6 7 2 . 9 2 8 9 0 6 ] OPX: [ flowID 1 7 2 . 1 7 . 1 7 . 1 7 7 : 4 1 0 5 7 1 7 2 . 1 7 . 1 7 . 1 6 1 : 5 0 0 1 11 queue 1858 vqueue 1857 p r i c e 1 d u t i l i t y 56523 u t i l i t y 52012 r a t e 0 d e l a y 56 l 2 q 42 ] [ urgency s e l f 1857 n e i g h 0 perm 1 ] Listing 1.2. Fragment of a log file created during the experiment
wnPUT Logger utilizes the data from log files and converts it on tabular information – Comma Separated Values (CSV) file format which is easy to read and widely utilized via tools like gnuplot. Whenever a need to introduce any other
380
A. Nowak et al.
parameter to analyze appears, a minor modification of Logger is needed. However, once it is done, the newly introduced element is available for all consecutive experiments. The DANUM system requires the additional monitoring of both end-to-end throughput and delay in order to use it for per-flow utility derivatives calculations required by the indirect flow control component [14]. The presented logs (see Listing 1.2) and monitoring snapshots (see Fig. 6) contain information about mentioned-above parameters. 6.4
Analysis of Experimentation Effectiveness
Accuracy and repeatability of the experiments were the main benefits from experimentation performed locally within the wnPUT testbed environment. Without a proper management tools, like the ones included in wnPUT framework, it would not be possible to achieve reliable and repeatable results. On the other hand, synchronization level provided by automated tools, like the ones used in wnPUT and other testbeds guarantee the valid execution of experiments. Results collected in the wnPUT testbed are far more useful than the ones achieved in a standard non-automated way – largely because the testbed environment provides mechanisms ensuring repeatability of highly-controllable experiments.
7
Conclusions
The wnPUT testbed has been built and developed for the purposes of research aimed at verifying the applicability of the backpressure principle to MANETs, including the evaluation of a backpressure variation based on virtual queues being utilized by the DANUM system. The main reason for the decision to arrange the wnPUT testbed was the local character of the initial research on DANUM and high vulnerability to errors of the implementation. Developing a small-scale testbed proved to be a necessary step of the research on DANUMS – it helped to analyze the performance and simplified the process of experiment repetition realized within large-scale testbeds (DES-testbed and NITOS-Testbed). Results of DANUMS development realized in wnPUT have been successfully ‘transformed’ into valuable results of the experimentation performed in DES-Testbed, demonstrating advantages of using backpressure-based control system in a dense largescale MANET [12]. Acknowledgement. This work was supported by Poznan University of Technology under grant 45-085/11 DS-PB.
References 1. Blywis, B., G¨ unes, M., Juraschek, F., Schiller, J.: Trends, Advances, and Challenges in Testbed-Based Wireless Mesh Network Research. ACM/Springer Mobile Networks and Applications (MONET) 15, 315–329 (2010)
wnPUT Testbed Experimentation Framework
381
2. Clausen, T., Jacquet, P.: Optimized Link State Routing Protocol (OLSR). RFC 3626 (Experimental) (October 2003), http://www.ietf.org/rfc/rfc3626.txt 3. De, P., Raniwala, A., Krishnan, R., Tatavarthi, K., Modi, J., Syed, N.A., Sharma, S., cker Chiueh, T.: MiNT-m: An Autonomous Mobile Wireless Experimentation Platform. In: Proc. of Mobisys 2006, pp. 124–137 (2006) 4. Georgiadis, L., Neely, M.J., Tassiulas, L.: Resource allocation and cross-layer control in wireless networks. Foundations and Trends in Networking, 1–149 (2006) 5. G¨ une¸s, M., Juraschek, F., Blywis, B., Watteroth, O.: DES-CRIPT - A Domain Specific Language for Network Experiment Descriptions. In: International Conference on Next Generation Wireless Systems, NGWS 2009, Melbourne, Australia (2009) 6. Kelly, F., Maulloo, A.K., Tan, D.: Rate Control in Communication Networks: Shadow Prices, Proportional Fairness and Stability. Journal of the Operational Research Society 49 (1998) 7. The Network Simulator NS-2, http://www.isi.edu/nsnam/ns/ 8. Rakotoarivelo, T., Ott, M., Jourjon, G., Seskar, I.: OMF: a Control and Management Framework for Networking Testbeds. SIGOPS Oper. Syst. Rev. 43, 54–59 (2010) 9. Szwabe, A., Misiorek, P., Walkowiak, P.: IMS-Based Performance Analysis of a MANET Controlled by the Delay-Aware NUM System. In: 2011 20th Annual Wireless and Optical Communications Conference (WOCC), pp. 1–6. IEEE Press, Newark (2011) 10. Szwabe, A., Misiorek, P.: Integration of Multi-path Optimized Link State Protocol with Max-weight Scheduling. In: ICIMT 2009. International Conference on Information and Multimedia Technology, pp. 458–462. IEEE Press (2009) 11. Szwabe, A., Misiorek, P., Nowak, A., Marchwicki, J.: Implementation of Backpressure-Based Routing Integrated with Max-Weight Scheduling in a Wireless Multi-hop Network. In: 2010 IEEE 35th Conference on Local Computer Networks (LCN), pp. 983–988 (2010) 12. Szwabe, A., Misiorek, P., Urbanski, M., G¨ une¸s, M., Juraschek, F.: Multi-Path OLSR Performance Analysis in a Large Testbed Environment, Technikal Report IAII-609, Poznan University of Technology, Institute of Control and Information Engineering (2011) 13. Szwabe, A., Misiorek, P., Walkowiak, P.: DANUM System for Single-hop Wireless Mesh Networks. In: Proceedings of 2010 International Conference on Future Information Technology (ICFIT 2010), vol. 1, pp. 365–369. IEEE Press, Changsha (2010) 14. Szwabe, A., Misiorek, P., Walkowiak, P.: Delay-Aware NUM System for Wireless Multi-hop Networks. In: 11th European Wireless Conference 2011 - Sustainable Wireless Technologies (EW 2011), Vienna, Austria, pp. 530–537 (2011) 15. Ad-hoc Protocol Evaluation Testbed, http://apetestbed.sourceforge.net/ 16. An Autonomic Reconfigurable Miniaturized Mobile Wireless Experimentation Testbed, http://www.ecsl.cs.sunysb.edu/mint/ 17. Freifunk – City Mesh, http://www.freifunk.net 18. kst - Visualize Your Data, http://kst-plot.kde.org/ 19. OneLab: Open Federated Laboratory for Future OneLab: Open Federated Laboratory for Future Internet Research, http://www.onelab.eu 20. NITOS Wireless Testbed - Network Implementation Testbed Laboratory, http://nitlab.inf.uth.gr/NITlab/index.php/testbed 21. The Distributed Embedded Systems Testbed (DES-Testbed) Webpage, http://www.des-testbed.net/ 22. Orbit-lab Testbed (March 2011), http://www.orbit-lab.org/
