ing protocols may be implemented as Click graphs and easily moved between ... tiated services. ...... Software Testing and Analysis, pages 2â13, 2000.
Nsclick: Bridging Network Simulation and Deployment Michael Neufeld, Ashish Jain, Dirk Grunwald University of Colorado Department of Computer Science Boulder, CO 80309 {neufeldm, ajain, grunwald}@cs.colorado.edu
Abstract
1. INTRODUCTION
Ad hoc network protocols are often developed, tested and evaluated using simulators. However, when the time comes to deploy those protocols for use or testing on real systems the protocol must be reimplemented for the target platform. This usually results in two, completely separate code-bases that must be maintained. Bugs which are found and fixed under simulated conditions must also be fixed separately in the deployed implementation, and vice versa. There is ample opportunity for the two implementations to drift apart, possibly to the point where the deployed and simulated version have little actual resemblance to each other. Testing the deployed version may also require construction of a testbed, a potentially time-consuming and expensive endeavor. Even if constructing an actual testbed is feasible, simulators are very useful for running large, repeatable scenarios for tasks such as protocol evaluation and regression testing. Furthermore, since the implementation may require modification of the kernel network stack, there’s a good chance that a particular implementation may only run on specific versions of specific operating systems. To address these issues, we constructed the nsclick simulation environment by embedding the Click Modular Router inside of the popular ns-2 network simulator. Routing protocols may be implemented as Click graphs and easily moved between simulation and any operating system supported by Click. This paper describes the design, use, validation and performance of nsclick.
Experimentation and development of ad hoc routing protocols is, for the most part, performed exclusively in a simulated environment. This is with good reason, since constructing realistic testbeds can be an expensive, time-consuming business [10]. However, the time does eventually come for the new protocol or variant to be deployed. At that point, the code from the simulated environment must be reimplemented for the target platform. Since ad hoc routing protocol implementations generally require modifications to the kernel network stack, the target platform is often limited to a single operating system, or even particular versions of a single operating system. The Click Modular Router [8] by itself provides a layer of abstraction which fixes this. The same routing code may be run at kernel level on both Linux and FreeBSD, and at user level on any OS that provides libpcap [2] functionality. However, it doesn’t change the fact that code has to be developed separately for the simulator and for deployment. Our research group is examining system-level designs for lowpower ad hoc networks as part of a broader research program investigating power-aware systems. Since we are interested in a system level analysis, we need to be able to both evaluate and deploy the computing systems we are designing since aspects of our work cut across applications, operating systems and routing algorithms. Our early attempts to deploy existing ad hoc routing implementations met with disappointment - we found few implementations and even fewer that had been tested and validated on a large scale. Thus, we decided to build a single implementation that could be used in multiple operating systems and that would allow easier experimentation and protocol modification. Early in the design, we decided to extend the Click Modular Router rather than develop yet another routing framework. As the name suggests, the Click Modular Router provides a framework for constructing and connecting “routing elements”; a full router is configured by composing element in different configurations. The existing Click implementation supports a wide range of router elements supporting a number of wired routing configurations, including components for full ICMP support, simple Ethernet switches, Random Early Detection (RED) and differentiated services. These components can be loaded into the Linux kernel to provide a dynamically configured, adaptable router with good performance. We extended the existing Click tools by constructing routing components that support different ad hoc routing protocols. Then, to test and validate our implementations, we created the nsclick simulator, which combines ns-2 [6] and Click. In this paper, we describe the design of nsclick and compare aspects of an ad hoc routing protocol to a comparable protocol implemented
Categories and Subject Descriptors I.6 [Simulation and Modeling]: Model Development
General Terms Measurement,Design,Experimentation,Verification,Performance
Keywords Click,ns-2,ad hoc,simulation
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MSWiM’02, September 28, 2002, Atlanta, Georgia, USA. Copyright 2002 ACM 1-58113-XXX-X/02/0009 ...$5.00.
FromDevice(eth0)
FromDevice(eth1)
FromDevice(eth0) HostEtherFilter(addr_eth0)
Print(eth0)
HostEtherFilter(addr_eth1)
Classifier
Classifier
ARP Query,ARP Response,Other
ARP Query,ARP Response,Other
ARPResponder(addr_eth0)
ARPQuerier(addr_eth0)
ARPQuerier(addr_eth1)
Classifier IP,Other
Strip(14)
Count
Discard
ARPResponder(addr_eth1) Classifier IP,Other
Discard
Discard
CheckIPHeader
Strip(14)
CheckIPHeader
MarkIPHeader
MarkIPHeader
GetIPAddress(16)
GetIPAddress(16)
IPClassifier
IPClassifier
For eth1,Other
For eth0,Other
Discard
Discard
Figure 1: Click Sniffer
using the standard ns-2 simulator. As part of the comparison, we demonstrate some advantages to our approach, since our simulator tests allowed us to find specific flaws in the IETF MANET AODV draft 10 specification [12]. In particular, we found that the draft RFC specification for the AODV routing protocol using explicit “HELLO” messages can lead to routing loops that reduce performance. This is important since most implementation of the AODV protocol use “HELLO” messages, while most simulations use link-level information for the same purpose. There are other benefits of using Click rather than the standard ns-2 implementation of such protocols. For example, one of our techniques to reduce transmission power uses “packet aggregation” to encapsulate several small packets into a single larger packet; this can easily be done by creating encapsulation and deencapsulation components in Click. Likewise, we can combine the existing suite of traffic shaping, buffering and dropping components with new routing protocols. In Section 4, we first briefly describe the Click and ns-2 tools and discuss the changes that were necessary to combine them. Then, in Section 3, we compare the performance of nsclick with ns-2 for a specific ad hoc routing protocol.
2. DESIGN Our new simulation infrastructure combines two extant networking tools: click and ns-2. We describe each in turn and then show how they have been integrated.
2.1 The Click Modular Router The Click Modular Router is a system designed for constructing and deploying fast, configurable, and flexible packet routers. Broadly speaking, it consists of two main parts: a router configuration, and an OS-specific layer that handles the details of running the router configuration on a particular system. A router configuration is a graph made up of individual elements, each of which performs some action on a packet, and then hands it on to the next element in line. Elements usually tend to perform fairly simple tasks, e.g. decrement the IP TTL field, but may also do much more complicated things, e.g. using ARP to maintain an IP to MAC address map. A very simple example is shown in Figure 1. All of the packets arriving on the network interface eth0 are displayed, counted, and discarded, effectively constructing a crude packet sniffer. The Click script corresponding to Figure 1 is shown below:
Queue
Queue
ToDevice(eth0)
ToDevice(eth1)
Figure 2: Click IP Router
// // A simple Click script which dumps and counts packets // coming in off of eth0 // FromDevice(eth0) −> Print(eth0) −> Count −> Discard; The FromDevice element grabs packets from the specified Ethernet device and pushes them to the Print element. The Print element displays information about the packet, and sends the packet to Count. Count simply counts each transiting packet and then sends those packets to the Discard element. A more complicated routing graph is in Figure 2. This graph functions as an IP router between two separate networks. There are four things to notice about this example. First, routing functionality is constructed using components, enabling significant reuse from one routing framework to another. Second, the components operate on actual IEEE 802.2 format frames – almost all of the current Click framework is focused on devices with Ethernet framing and encapsulation. Third, the components are configurable; for example, the Strip() element simply removes bytes from the beginning of a frame – by changing the value of “14” to another value, the same component could be used to e.g. strip IPv6 headers. Lastly, only specific elements interact with the external environment. In this example, those elements receive and send packets to an actual network interface. These characteristics (composability, wire format, paramatarizability and external interfaces) are exploited when we use Click to specify routing actions within a simulator. In particular, we can reuse existing click components to assist in specific simulations. For example, Click has a wide collection of queuing and classification components suitable for implementing DiffServ or QoS mechanisms. Second, each simulated network node will need a different configuration – the ability to parameterize the Click elements will let us use the same routing specification with different parameters. Third, the use of actual wire-formats has a large implication for a simulator. It means messages may take more memory, but wire-format messages are necessary if the same code is to be used in an implementation. Lastly, the limited external interface helps reduce the number of modifications needed to integrate Click into a simulator.
2.2 ns The ns-2 simulator is a discrete event simulation framework commonly used to evaluate networking routing protocols, flow control and congestion control mechanisms. Loosely speaking, most ns-2 simulations are structured as graphs with nodes representing stations or routers with specific connectivity. Most ns-2 simulations use specific “message types” that abstract issues about the actual link layer. Network specifications typically do not specify addresses for nodes; instead, the addressing is made implicit by “connecting” communication sources and destinations. Unless specific routing protocols are being evaluated, a global routing phase establishes an optimal routing table. The standard ns-2 distribution includes a tool (“nam”) that is useful for visualizing simulation outcomes and performance. Recent versions of ns-2 include a physical layer model that attempts to model the 802.11b wireless Ethernet MAC layer and power constraints. At the top level, ns-2 simulations of mobile networks are specified using scenario files that encode the motion of nodes within a network. There are large collections of mobile node scenarios that are used by the ad hoc networking community to validate new protocols. When modeling mobile networks, ns-2 provides a standardized evaluation framework that combines reasonable physical and power models with a large collection of mobility scenarios. As we shall see, we have been able to retain many of the useful aspects of ns-2.
2.3 Integration Integrating Click with ns-2 required changes to both programs. Click normally uses system calls to determine the current time and transfer packets to and from Ethernet devices and the kernel. It also was constructed with the implicit assumption that it would run steadily and regularly as either a user-level process or as part of a kernel. The ns-2 simulator doesn’t work with real wire format packets (except in emulation mode), and expects to handle packet routing either globally or via routing agents attached to each node. Fortunately, both Click and ns-2 are modular enough to permit a marriage between the two of them without extensive changes to either. Broadly speaking, there are three general problems that needed to be solved to integrate Click into ns-2: 1. Managing time 2. Interfacing simulated devices 3. Parameterizing And Addressing Nodes
2.3.0.1 Time. Click uses a time interface to determine schedule specific networking events. For example, the DelayShaper element causes packets to be delayed for a specific time and the Shaper element allows packets to be forwarded at a specific rate. Both rely on the ability to measure time and schedule events for the future. In an implementation, this time management is performed using the gettimeofday system call and alarm interrupts. Since Click is normally used to implement routers on computers in a distributed network, the returned time is unique to that computer and the times for different machines may not agree. The ns-2 simulator also uses time to schedule the use of communication links, control transmission routines and the like. Since
ns-2 is implementing a discrete event simulator, the time across all nodes is identical and increases at the same rate. Mechanically, we had to modify the Click operating system interface to use a new application interface we created for ns-2. This interface provides mechanisms for Click to determine the current time and schedule future interrupts and for ns-2 to configure and query specific properties of the Click router. None of the API calls are specific to ns-2. In fact, early testing of the abstraction layer was carried out with a simple test harness completely unrelated to ns-2. Within ns-2, we created a new node type that instantiates the data necessary for the API, such as the MAC and IP addresses of the specific node. An equally important change was to introduce a Jitter element into Click. Since Click uses localized time, specific scheduled events tend to occur at physically distinct times. For example, if all nodes within radio contact receive a message and schedule a transmission to occur 250ms later, the actual occurrence of those events will typically not occur at the same time in a physical network; however, they occur at precisely the same time in nsclick. This can lead to higher loss rates and retransmissions than would normally occur. The Jitter element is normally specified immediately prior to the output device in the Click routing graph; it delays a packet a random period of time to avoid problems arising from the synchronized simulator clock.
2.3.0.2 Simulated Devices. Within ns-2, we defined a new node type that provides an interface between Click and ns-2. Click routing graphs use specific elements to receive and send messages (ToDevice and FromDevice); the nsclick routers simply substitute the new elements ToSimDevice and FromSimDevice. The simulated device transmits “raw packets”, which are simply standard Ethernet frames. Those raw packets can be transmitted using the ns-2 physical layer models; this means nsclick can use the 802.11b MAC layer and power models provided by ns-2. A secondary issue that arises with the use of wire format network frames is trace file content. Normally it is trivial to dump very specific packet information into the standard ns-2 trace files since the relevant information is simply stored as fields in the simulated packet structure. With wire format packets, the job is a little more difficult. The raw data must be examined to determine the specific type, along the same lines as a network traffic analyzer (e.g. The Ethereal Network Analyzer [1]). Currently, nsclick does not do this, and all packets are shown as being the same type in the trace files. This can be a problem when using tools designed to analyze these files. For example, VeriSim [3] is a system for verifying formal properties of routing protocols. To analyze AODV, it relies on being able to distinguish between different types of AODV control packets in the trace file. Since nsclick currently does not make this distinction, we can yet use tools such as VeriSim. However, simulations running under nsclick do produce both .nam and .tr style tracefiles which may be used to gather basic routing statistics such as goodput, latency, and hopcount. We are in the process of building a trace converter that converts wire format traces into the standard ns-2 trace format so that tools that depend on the ns-2 trace file format (such as VeriSim) can be used. In addition to ns-2-style tracefiles, nsclick can produce trace information of its own. The standard Click printing elements (e.g. Print and IPPrint) still function and may be used to dump packet data at opportune points. The ToSimDump
element dumps packets into a libpcap-compatible file. This file may then be viewed using any number of applications, e.g. Ethereal. Since there is an equivalent packet dumping element (ToDump) which runs on real systems, it’s possible to make direct comparisons between traces produced in the simulator and those produced on an actual system.
2.3.0.3 Parameterizing And Addressing. In the ns-2 simulator, most configuration and parameterization is done using TCL scripts. When used on an actual Linux system, the Click router presents a configuration interface using the /proc filesystem that can be used to modify parameters of individual elements. For example, that interface could be used to change the duration of a DelayShaper element without changing the overall router configuration. Currently nsclick doesn’t provide an interface between ns-2 and this configuration interface. This may reduce the functionality and flexibility of some Click configurations. We are currently in the process of adding this support to nsclick. Setting IP and MAC addresses is another issue. When running Click, such information can either be hard coded or pulled directly from the network interface. Hard coding is awkward at best, and means that the script must be modified for each system it runs on. When only a single instance of Click is running on a machine (as is normal in an actual system), this is a minor inconvenience. When running tens or hundreds of nodes (as is normal in simulation), this can be a major problem. A different version of the Click graph would have to be produced for each node in the simulation, meaning that a minor change to the routing graph would have to be applied to many files, producing a maintenance nightmare. Pulling the information from the network interface makes more sense, but standard ns-2 doesn’t use regular IP and MAC addresses, and only stores one address per node, not per network interface. To fix this, nsclick nodes have IP and MAC addresses just like actual systems would. These addresses are set from the TCL configuration script, and can be accessed from the Click script via the AddressInfo element. This allows a single Click graph to be used for all of the nodes in the simulation but still have unique IP and MAC addresses for each node. Of course, it is still possible (and often very useful) to run different Click graphs in different nodes in the simulation, but it isn’t required. Creating scenarios for nsclick is very similar to creating them for mobile nodes under regular ns-2. Node motion files used in regular ns-2, e.g. those generated by setdest, may be used without modification in nsclick. Network traffic files, however, are a little different. For example, Figure 3 starts a constant bit rate (CBR) source under regular ns-2 while Figure 4 shows the configuration of the same data source under nsclick. Both scripts are fairly similar, but there are some key differences to review. First, note the use of the Agent/Raw class instead of Agent/UDP. This forces the use of wire-format UDP packets which Click can process. The second thing to note is the lack of a connect statement and the addition of source and destination IP addresses and ports. Since nsclick uses regular IP addresses for routing, the destination must be stated explicitly, rather than making an implicit connection between the two nodes as in ns-2. Specifying network topology with nsclick is also a little different than with ns-2 mobile networking. Figure 5 shows the logical structure of a nsclick simulated Ethernet interface on a
# # 6 to 18 at 995.012550273234 # set udp (0) [new Agent/UDP] $ns attach−agent $node (6) $udp (0) set null (0) [new Agent/Null] $ns attach−agent $node (18) $null (0) set cbr (0) [new Application/Traffic/CBR] $cbr (0) set packetSize 64 $cbr (0) set interval 0.25 $cbr (0) set random 0 $cbr (0) set maxpkts 4000 $cbr (0) attach−agent $udp (0) $ns connect $udp (0) $null (0) $ns at 995.012550273234 "$cbr_(0) start"
Figure 3: Scenario File In ns-2 # # 6 to 18 at 995.012550273234 # set raw (0) [new Agent/Raw] $ns attach−agent $node (6) $raw (0) set null (0) [new Agent/Null] $ns attach−agent $node (18) $null (0) set cbr (0) [new Application/Traffic/CBR] $cbr (0) set packetSize 64 $cbr (0) set interval 0.25 $cbr (0) set random 0 $cbr (0) set maxpkts 4000 $cbr (0) attach−agent $raw (0) $raw (0) set−srcip [$node (6) getip eth0] $raw (0) set−srcport 5000 $raw (0) set−destport 5000 $raw (0) set−destip [$node (18) getip eth0] $ns at 995.012550273234 "$cbr_(0) start"
Figure 4: Equivalent Scenario File In nsclick
LAN. This is very similar to the standard ns-2 mobile node network interface, except that packets are directed to Click instead of a regular routing agent, and there is no ARP module attached. As in ns-2 mobile networking, all network interfaces which are attached to the same Channel object are on the same LAN. For wired Ethernet this corresponds to nodes on the same hub; for the wireless network, this corresponds to nodes on the same channel or frequency. The difference arises primarily at node creation. With regular ns-2, mobile nodes are typically created with a single wireless Ethernet interface with all of the nodes on the same channel. Under nsclick, all network interfaces must be explicitly added after creation, and the MAC and IP addresses of the interfaces must be specified. Interfaces may be either wired or wireless, depending on the type of MAC and Channel used. Mixing wired and wireless network interfaces on the same node is trivial, for example the scenario in Figure 6. The hybrid node could be a standard IP router as shown in Figure 2, a node running an ad hoc routing protocol, a gateway running Mobile IP, or perhaps just a simple Ethernet bridge. The hybrid node could also have multiple wired and wireless interfaces and route traffic between
40
Click
ClickQueue
MAC
Propagation
NetIF
Percentage Throughput
LL
Stock NS-2, 2 Nodes Modular AODV & NSClick, 2 Nodes Stock NS-2, 6 Nodes Modular AODV & NSClick, 6 Nodes
20
Channel
Figure 5: Nsclick Ethernet Interface 0
0
5
10 Node Mobility (m/s)
15
20
Figure 7: Stock ns-2 and nsclick AODV Comparisons, Under 10 Transmitters
Figure 6: Simple Nsclick Wired/Wireless Hybrid Scenario all of them. Furthermore, a scenario could easily have multiple hybrid nodes, all routing traffic simultaneously. Writing the TCL code to allocate IP addresses and keep track of which nodes are on which networks could involve a lot of planning and typing for complicated configurations, but is still a straightforward procedure. This is one of the places where the flexibility of Click and ns-2 really pays off. With Click, it’s easy to run different routing graphs in each node. This means that having special gateway nodes is as simple as determining the functionality of the gateway and implementing it in Click. The ns-2 simulator provides convienent scripting using TCL as well as models of the physical layer. It is possible to do hybrid wired/wireless scenarios in regular ns-2, but our experience has been that it is much easier to do so in nsclick.
3. VALIDATION AND PERFORMANCE COMPARISON While nsclick may be used to many kinds of different Click scripts, the original motivation behind it was to assist the development, debugging, and validation of ad hoc routing protocol implementations. We wanted to combine those protocols with QoS and traffic shaping mechanisms in order to experiment with power-performance trade-offs. In this section, we assess the utility of nsclick in two ways. First, we demonstrate that we can construct non-trivial extensions to the Click framework – in this case we briefly discuss our experience building an implementation of the AODV ad hoc routing protocol as an element within Click. We then compare the performance and memory demands of ns-2 and nsclick.
3.1 Building A Modular AODV Implementation The standard ns-2 implementation of AODV uses specific message types rather than formatted “raw” packets. We wanted an implementation of the protocol that could be combined with other traffic shaping elements and could be used on an actual
wireless network. This posed two problems; first, we needed to build and verify an implementation of AODV that met the IETF MANET AODV draft 10 specifications of the AODV routing protocol [12]. Second, the implementation of the protocol needed to depend on features commonly available in actual wireless networks. The AODV protocol has two variants. The first variant detects “link breakage” by using feedback from lower layers of the protocol stack. For example, 802.11b sends acknowledgements when data packets are received. If such an acknowledgement isn’t received after sending a packet to a neighboring node, the link is broken. However, the availability of this information in the real world is contingent on the use of specific network card and driver combinations, and often may not be accessible. In order to run on as broad a set of platforms as possible, we elected to use the second AODV variant that uses explicit “HELLO” messages to maintain link connectivity. Nodes broadcast “HELLO” beacons on a regular basis. Link breakage is indicated when a “HELLO” message has not been received for a specific period of time. The Internet Draft specification for AODV [12] provides suggested starting values to try for the “beacon interval” and “beacon lifetime.” We configured the standard ns-2 implementation and the nsclick implementation to use the same values. We generated motion and traffic scenarios as suggested by Camp et al [4]: fifty nodes total in a 300m by 600m space, and a transmission range of 100m. The node mobility model used was the random waypoint model, with the node speed to each subsequent destination being set randomly to a value ±10% of a particular average speed for each scenario. We used average speeds of 1 m/s, 5 m/s, 10 m/s, 15 m/s and 20 m/s. For network traffic, we used a peer-to-peer model. Sources were paired up randomly and sent CBR traffic to each other (4 packets/s, 64 bytes/packet). No node was paired with more than one other node. We ran scenarios with 2, 6, 10, 14, and 20 nodes sending packets. Simulations ran for 2000 seconds, with data traffic starting 1000 seconds into the simulation and running for a further 1000 seconds. This was intended to allow the nodes to mix together thoroughly before starting traffic and avoid effects caused by initial clumping of nodes. For the stationary scenarios, the delay allowed nodes to gather neighbor connectivity information using “HELLO” messages. We also ran ten random variations on each node motion configuration, resulting in a grand total of 300 scenarios. Figures 7 and 8 show the mean and 95%
40
FromSimDevice(eth0)
Percentage Throughput
Classifier
Stock NS-2, 14 Nodes Modular AODV & NSClick, 14 Nodes Stock NS-2, 20 Nodes Modular AODV & NSClick, 20 Nodes
IP
non−IP
ARP
Discard
FromKernel Classifier Data
Control
20
Buffer
ToKernel AODV
EncapEther
Elements changed Simulation−specific Elements
Log
Jitter
0
0
5
10 Node Mobility (m/s)
15
20
Figure 8: Stock ns-2 and nsclick AODV Comparisons, Over 10 Transmitters
confidence intervals for both implementations. Two factors stand out; first, the results are almost statistically identical, except for the highest mobility scenario with 20 nodes. We believe that some of that variation is due to the use of the Jitter element to avoid simultaneous simulator events, which resulted in different packet loss rates due to collisions. The second notable factor is that the throughput is very low. This is partially explained by the choice of parameters for “HELLO” messages by the draft RFC interacting with our scenarios. The recommended parameters cause excessive network congestion. There are also problems with the AODV specification which result in temporary routing loop formation due to sequence number problems when using “HELLO” messages. As mentioned, the ns-2 AODV implementation typically uses lower level network feedback to determine link breakage while most implementations use “HELLO” beacons. Several flaws with the handling of “HELLO” messages have been discovered by us and other practitioners. Other researchers working on formal verification of network protocols [3] have found flaws in the protocol in the past. Some of the flaws could have been discovered more quickly in implementations by more rigorous regression testing using explicit “HELLO” beacons in the ns-2 simulator. When we corrected the various errors concerning the “HELLO” beacon protocol, we observed an increase in throughput. Note that we would expect to see similar improvement when we include the AODV modular router components in a running Linux kernel using Click. One of the strong benefits of nsclick is that we were able to test and debug our implementation under nontrivial circumstances before deploying it. There is no way we could have constructed a testbed which would have stressed our implementation as thoroughly and easily as we did under simulation. This allowed us to find and fix problems before we ever ran our implementation on an actual system. The only other system we know of which provides this feature is the AODV implementation created at Uppsala University [11]. Recent versions of it may run in ns-2 as well as on actual Linux systems. Our implementation met our initial goal, since the core routing protocol can be moved between deployment and simulation. Figure 9 shows the changes to the routing configuration necessary when moving the routing protocol from the simulation environment to a deployed network. We remove the Jitter element since that is only useful to skew the simulator clicks and would
ToSimDevice(eth0)
Figure 9: Click Routing Graph For AODV Router Configuration Showing Elements That Change When Moving From Simulation To Deployment
impose a slight performance penalty in an actual network. We also need to change the device specification for the input and output devices, i.e. FromSimDevice becomes FromDevice and ToSimDevice becomes ToDevice. Thus we see that there are usually very few changes required to move from simulation to deployment. This implementation illustrates some problems to be overcome with nsclick. In particular, it highlighted the need to capture hardware-specific information from simulated network devices, e.g. detecting link breakage. Some work is already underway for this; the standard Click distribution now contains an element (AiroInfo) that reports information about link signal strength. We simply need to provide a similar interface within nsclick, and construct a similar Click element to get the information. Systemdependent code such as this makes it more difficult to move code between simulation and reality.
3.2 Memory Consumption And Simulator Performance In order to compare the performance of nsclick with ns-2, we looked at two different types of scenarios: a simple wired LAN configuration and a subset of the wireless mobile AODV scenarios used for comparing our Click AODV implementation with the stock ns-2 implementation. We ran the scenarios both with and without network traffic. The scenarios without network traffic were intended to indicate basic overhead, i.e. the relative costs incurred simply bringing up all of the nodes. The scenarios with network traffic were intended to show relative overhead in more typical use cases. The performance numbers for the LAN scenarios were taken on a Dell PowerEdge 4300 with dual 933MHz Pentium III processors, and 512MB of RAM, running Mandrake Linux 8.1. The performance numbers for the AODV scenarios were taken on a Dell PowerEdge 2300 with dual 1.266GHz Pentium III processors, 2GB of RAM, also running Mandrake Linux 8.1. The ns-2 version of the LAN scenario used regular ns-2 LAN routing, and the nsclick version used a simple Click script which simply uses ARP to find nodes on a local network, shown in Figure 10. We varied the number of nodes, but the scenarios with traffic only ever had one traffic source which sent unicast traffic to a single destination, 10 packets/s, 1000 bytes/packet, running for a total of 30 seconds. The results are shown in Figures 11
20
50000 Memory - nsclick Memory - ns-2
FromSimDevice(eth0)
Strip(14)
CheckIPHeader
HostEtherFilter
CheckIPHeader
GetIPAddress(16)
Classifier (Arp Query, Arp Reply, Other)
MarkIPHeader
ARPResponder
ARPQuerier
GetIPAddress(16)
FromSimDevice(tap0)
Time - nsclick Time - ns-2
40000
30000 10 20000
IPClassifier
Queue
Execution Time (s)
Memory Used (KB)
15
5
(For Me, Not For Me)
10000 ToSimDevice(tap0)
ToSimDevice(eth0)
Discard
0
0
100
50
Figure 10: Click LAN Router 50000 Time - nsclick Time - ns-2
10
40000
5 10000
0
50
100
150 Nodes
200
250
Time - nsclick Time - ns-2 8
6 30000
4 20000
2
10000
0
0
Figure 11: LAN Scenario Performance Without Traffic
Execution Time (s)
20000
Memory - nsclick Memory - ns-2
40000 Memory Used (KB)
10 30000
Execution Time (s)
Memory Used (KB)
50000
0
0
250
Figure 12: LAN Scenario Performance With Traffic
15 Memory - nsclick Memory - ns-2
200
150 Nodes
0
100
50
200
150 Nodes
250
0
Figure 13: AODV Scenario Performance - Without Traffic
ference isn’t as large as it is for the LAN scenarios. The second thing to note is that nsclick actually runs faster in all cases. We believe that this is likely due to the fact that after the Click routing graph is created, the Click code runs as straight C++, while the routing agents in ns-2 have to spend more time in the OTCL interpreter. The third thing to note is that memory consumption with nsclick varies quite a bit more than with ns-2. This makes sense since, as mentioned previously, nsclick memory consumption is affected much more than ns-2 by the number of packets in flight. In summary, we feel that these benchmarks indicate that nsclick performs quite reasonably when compared to ns-2. Memory consumption is greater, but not to an excessive degree, and in many cases nsclick will run more quickly than ns-2.
400
30000 Memory - nsclick Memory - ns-2
25000
Time - nsclick Time - ns-2
20000
200
15000
10000
Execution Time (s)
300 Memory Used (KB)
and 12. There are some interesting things to observe in the graphs. First, nsclick is quicker than ns-2 when no traffic is being sent. We believe that this is due to the fact that regular ns-2 has to spend time pre-calculating routing information for all of the nodes, while nsclick does not. However, the situation reverses when network traffic is added. Stock ns-2 is quicker at routing packets, and makes up for the routing overhead pretty quickly. The second thing to notice is memory overhead. There is a fairly sizable additional overhead with nsclick. However, we feel that the overhead is not excessive. Consuming 50MB of memory for a 250 node simulation doesn’t seem excessive when most laptop computers come with at least 128MB of RAM standard. Actual nsclick memory consumption in general depends on both the number of in-flight packets as well as the size of those packets since it transmits raw Ethernet frames. This makes overall memory consumption less predictable than ns-2, since the packet sizes in nsclick depend on the actual “wire size” of the packet, not just on the ns-2 headers. Also note that adding traffic doesn’t have much of an effect on the overall memory consumption for either ns-2 or nsclick. The packets in this scenario only live for a short time in the network, resulting in small fluctuations above the baseline memory consumption. Finally, we compare the relative performance of ns-2 and nsclick when running mobile scenarios routing with AODV. For the scenarios without traffic, we simply varied the overall number of nodes present in the system, much like the corresponding LAN scenarios. For the scenarios with traffic, we kept the total number of nodes constant, but varied the number of transmitters in order to see how varying the network load affected memory consumption and speed. The results are shown in Figures 13 and 14. The first interesting point to note is that once again nsclick requires more memory than regular ns-2. However, for the scenarios without traffic, the dif-
100 5000
0
0
5
10 Nodes Transmtting
15
20
0
Figure 14: AODV Scenario Performance - With Traffic
4. RELATED WORK The notion of being able to run the same network code both in simulators and actual systems is not new. The usual approach is to start with an actual system kernel, virtualize parts of it as needed, and then simulate the network underneath it. This is the approach taken by the Harvard TCP/IP network simulator [14], ENTRAPID [7], Dummynet [13], and Alpine [5]1 . Interestingly, there is an element distributed with Click which appears to work along these lines. The RadioSim element simulates aspects of an 802.11 radio network, and is included along with elements used to implement the Grid ad hoc routing protocol [9]. However, the simulations used for publication about the Grid protocol were conducted using a separate implementation under ns-2. The nsclick simulation environment resembles these other systems in that it also began with a working real-life system which was adapted to function with some simulated parts. However, there are two key differences. First, nsclick exploits the existing abstraction provided by Click. Rather than running an entire kernel, it only has to provide simulated services for transferring packets to and from network interfaces. This is a much simpler task than virtualizing a kernel, but also more restrictive. Since there is no kernel to provide services, Click elements which attempt to directly use kernel services, e.g. access the routing table or get information from a network driver, simply won’t work. This also means that nsclick is limited to using models of application traffic from within the ns-2 simulator rather than running the actual applications themselves. The second key difference is that nsclick runs entirely within a discrete event simulator 2 , whereas the others run in real time. Another system similar in spirit to nsclick is the implementation of AODV built at Uppsala University [11]. As of release 0.5, it is possible to run it both in Linux as well as in ns-2. The primary difference between it and nsclick is that nsclick is a general purpose environment for experimentation rather than an extension designed for a single AODV implementation.
5. CONCLUSIONS AND FUTURE WORK We have found nsclick to be a very effective tool for developing and experimenting with network routing protocols. Typically, though not always, the simulations we run go faster than the equivalents in regular ns-2, and being able to easily stress test our protocol implementations before deploying them results in a higher confidence level in our code and less time wasted debugging out in the field. It does place heavier demands on memory than ns-2, but the price paid seems light when we consider the benefits we get from using it. While nsclick works well, there is certainly room for improvement. An obvious limitation is the restriction to CBR (UDPonly) network traffic. This is acceptable at this time for evaluating the performance of ad hoc routing protocols, but is too confining in general. There are also more simulated hardware capabilities it would be nice to have. For example, link layer detection of a lost connection and being able to monitor received signal strength would be highly useful in wireless scenarios. Another problem is the lack of information in the trace files. Being able to distinguish between various packet types would greatly increase their utility, and also open up interesting possibilities 1 While not a simulator, Alpine does provide a user-level network stack which may be used to run regular network applications to assist with network protocol development. 2 Currently only ns-2, but other simulators could be supported.
with formal verification using tools like VeriSim [3]. A further improvement would be to allow the use of Click read and write handlers. When running on an actual system, information may be sent to and read from individual Click elements by reading and writing to special files in the /proc/click filesystem. Being able to control these from the simulator would allow for greater customization and dynamic control of Click routers during simulation. We are currently working on making these improvements, as well as integrating our changes to Click into the standard Click distribution from MIT. The nsclick simulation tool is available for download at http://systems.cs.colorado.edu/Networking/nsclick.
6. REFERENCES [1] Ethereal network analyzer. available from http://www.ethereal.com. [2] tcpdump/libpcap. available from http://www.tcpdump.org. [3] K. Bhargavan, C. Gunter, M. Kim, I. Lee, D. Obradovic, O. Sokolsky, and M. Viswanathan. Verisim: Formal analysis of network simulations. In Int. Symposium on Software Testing and Analysis, pages 2–13, 2000. [4] T. Camp, J. Boleng, B. Williams, L. Wilcox, and W. Navidi. Performance comparison of two location based routing protocols for ad hoc networks. In Proc. of IEEE INFOCOM, 2002. [5] D. Ely, S. Savage, and D. Wetherall. Alpine: A user-level infrastructure for network protocol development. In Proc. of USITS ’01, 2001. available from http://alpine.cs.washington.edu. [6] K. Fall and K. Varadhan. ns notes and documentation. available from http://www.isi.edu/nsnam/ns, Nov 1997. [7] X. W. Huang, R. Sharma, and S. Keshav. The ENTRAPID protocol development environment. In INFOCOM (3), pages 1107–1115, 1999. [8] E. Kohler, R. Morris, B. Chen, J. Jannotti, and M. F. Kaashoek. The click modular router. ACM Transactions on Computer Systems, 18(3):263–297, August 2000. [9] J. Li, J. Jannotti, D. S. J. D. Couto, D. Karger, and R. Morris. A scalable location service for geographic ad hoc routing. In Proc. of ACM Mobicom 2000, 2000. [10] D. Maltz, J. Broch, and D. Johnson. Experiences designing and building a multi-hop wireless ad hoc network testbed. Technical Report CMU-CS-99-116, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, March 1999. [11] E. Nordstr¨om, H. Lundgren, and B. Wiberg. Aodv-uu aodv implementation created at uppsala university. available from http://www.docs.uu.se/˜henrikl/aodv/. [12] C. Perkins, E. Royer, and S. Das. Ad hoc On-Demand Distance Vector (AODV) Routing. INTERNET DRAFT 10, IETF MANET group, January 2002. [13] L. Rizzo. Dummynet: a simple approach to the evaluation of network protocols. ACM Computer Communication Review, 27(1):31–41, 1997. [14] S. Y. Wang and H. T. Kung. A simple methodology for constructing extensible and high-fidelity tcp/ip network simulators. In Proc. IEEE INFOCOM’99 (The Conference on Computer Communications), pages 1134–1143, 1999.
