Optimizing QoS In FRACTEL: A TDMA Based WiFi ...

Optimizing QoS In FRACTEL: A TDMA Based WiFi Mesh Network For Rural Digital Connectivity Vishal Sevani Advisor: Prof. Bhaskaran Raman Department of CSE, IIT Bombay

1

Abstract WiFi mesh networks are an attractive option to provide rural digital connectivity, due to their low cost. However one of the major disadvantages of WiFi technology is lack of QoS for essential real time applications such as remote education, telemedicine, etc. In this report we outline FRACTEL MAC protocol which makes use of TDMA based mechanism to provide QoS support in multi-hop mesh networks. FRACTEL MAC supports fine grained synchronization accuracy of the order of microseconds and has support for efficient QoS provisioning. For ease of deployment and maintenance, it has features like automated node join, reconfigurability, etc. To ascertain the robustness of the FRACTEL MAC protocol design, we have carried out it’s evaluation on an outdoor campus wide wireless testbed. The results are encouraging with UDP throughput close to the expected value and jitter values within 30ms, even at four hops. However, to efficiently utilize the capacity in mesh networks there is need to utilize techniques such as spatial reuse, multi-channel and multi-radio operation, etc. Also in TDMA mesh networks, scheduling plays a key role in ensuring QoS for real time applications. With a view to incorporating these techniques in FRACTEL MAC and design efficient scheduling algorithms, we present a literature survey of present work in these areas. We investigate the applicability of current techniques for these different mechanisms in the context of FRACTEL MAC protocol. We identify the issues that need to be tackled to incorporate these mechanisms in FRACTEL MAC, laying down the plan for our future work.

Introduction

Use of WiFi based mesh networks has been explored for providing last mile connectivity over several alternatives such as GPRS, 3G, WiMax, etc. The major advantage of WiFi mesh networks is low cost compared to the other technologies. Also by appropriately modifying the MAC protocol and using directional antennas, it is possible to extend connectivity to over several hundreds of kilometers [1, 2], thereby proving the viability of these for providing low cost rural digital connectivity. However one of the major drawbacks of WiFi mesh networks is lack of assured QoS which is essential for supporting essential real time applications, such as remote education, telemedicine, etc. It is well known that contention based access mechanism of 802.11 CSMA/CA protocol is not suitable for providing the desired QoS in mesh networks and use of TDMA based mechanism is considered more appropriate. Employing TDMA based mechanism for mesh networks involves several challenges, such as accurate time synchronization, efficient scheduling of time slots, etc. Also for efficiently utilizing the capacity in mesh networks several mechanisms such as spatial reuse, multi-channel and multi-radio operation, etc have been proposed in literature. However efficient and practical realizability of these has proven to be challenging. In this report we highlight the issues concerning efficient functioning of TDMA mesh networks. First we outline FRACTEL, which is a TDMA based MAC protocol for WiFi mesh networks designed specifically to provide QoS support for various applications mentioned above. We wish to extend FRACTEL MAC for our future work and so to ascertain it’s robustness, we carried out it’s experimental evaluation, the results of which are presented next. Then, we present literature survey for the different mechanisms men1

2.1.1

tioned above for improving network capacity, ie. spatial resuse, multi-channel and multi-radio operation, along with scheduling in TDMA mesh networks, highlighting the open problems that we wish to tackle in our research work. Finally we conclude with summary, laying down the roadmap for the work planned ahead.

2

Attaining strict multi-hop time synchronization in TDMA based mesh networks has proven to be a challenging task primarily due to hardware limitations. However in FRACTEL MAC we show that with simple and proper design and implementation it is possible to attain micro-second granularity time synchronization even in commodity WiFi hardware.As mentioned, FRACTEL makes use of tree topology to attain synchronization in which every node is synchronized with it’s parent.The synchronization information is propagated from the root node in the tree towards the leaves, in the control slots of the frame. Use of tree topology for synchronization and schedule dissemination, though may not be optimal in terms of overhead, but is simple and robust approach as is demonstrated by practical implementation and evaluation of the protocol. The synchronization information primarily consists of the three important parameters, global slot start, hardware time stamp, and offset. Global slot start is the start time of next slot as propagated by the root node in it’s schedule packet. Hardware time stamp is the time at which packet transmission begins while offset is the time difference in clock of the node with respect to the root. So when the control packet is received by a child node, it first calculates it’s offset with respect to the parent node based on it’s receive timestamp and the hardware timestamp contained in the packet. It then calculates it’s offset with respect to the root node using the offset of the parent node contained in the packet and then adjusts the start time of next slot based on the value of the global slot start. This synchronization mechanism is further illustrated in fig.2 below. As regards the implementation of the protocol, FRACTEL TDMA MAC has been implemented by modifying the opensource madwifi drivers [6] available for 802.11 CSMA/CA. To incorporate the TDMA mechanism, following modifications have been carried out in the madwifi driver, ie. disabling of CCA, RTS/CTS, backoff, MAC level ACK’s, etc. Hardware clock of micro-second level granularity has been used to attain time synchronization accuracy of the order of micro-seconds, while kernel of millisecond granularity have been used to attain slot granularity of the order of milliseconds. The

FRACTEL TDMA MAC protocol

Primary goal of the FRACTEL project is to provide low cost rural digital connectivity with QoS support for various real time applications such as remote education, telemedicine, etc[3]. With this view FRACTEL TDMA MAC protocol is designed to support long distance communication to extend connectivity from semi-urban areas to rural regions over tens of kms, along with providing desired QoS. The details of the protocol are specified below.

2.1

Time synchronization mechanism and implementation

TDMA MAC protocol design and implementation

As mentioned the primary goal of FRACTEL MAC is to provide efficient QoS, along with keeping the overall design approach simple and practically feasible. With this view the MAC protocol is designed with following key features, 1. TDMA based access mechanism 2. Centralized routing and scheduling 3. Tree topology to attain pairwise time synchronization and disseminate schedule information 4. Use of soft state to maintain time synchronization and network topology 5. Support for automated node join and reconfigurability of topology in case of node failure To facilitate synchronization, schedule dissemination, etc FRACTEL MAC defines three different types of slots ie. schedule, contention and data slots. These slots are combined together into a frame which repeats periodically. More details regarding the protocol can be found in [4]. The TDMA frame structure from [4] is reproduced here in fig. 1 for sake of understanding. Below we briefly describe the time synchronization mechanism followed in FRACTEL, along with it’s implementation details. 2

Figure 1: Frame format of the FRACTEL MAC protocol [4] tic transmission. Root node on receiving the node join request, selects the parent node for the new node based on link quality information contained in the node join request. Although currently the node with best link quality is selected as the parent, going ahead we intend to incorporate load balancing mechanism wherein the load in the network is also taken into account while deciding the parent for new node. This same mechanism is used for reconfiguring the topology when an intermediate node fails. A child node periodically keeps track of the schedule packet received from parent node. If the schedule packet is not received from parent node for more than certain period of time, as specified by timeout interval, the node rejoins the network ie. assuming it has connectivity with nodes in the network. This soft state mechanism for maintaining time synchronization in the network, along with reconfiguring the topology is one of the key features of the FRACTEL MAC which makes it more robust as is ascertained by it’s evaluation on the outdoor testbed.

Figure 2: Time synchronization mechanism of FRACTEL MAC protocol [5] work regarding the design and implementation of the FRACTEL MAC protocol has been done by past year MTech students Ashutosh Dhekne and Nirav Uchat and the following references [7, 5, 4] give complete details. Below we describe the node join mechanism and it’s implementation that we carried out along with outdoor evaluation of the FRACTEL MAC protocol implementation. 2.1.2

Node join implementation

Node join design and implementation

As the node join request is propagated in contention slots, we maintain a separate queue to store node join requests. Before transmitting a node join join request, a node generates a random number and if it’s value is within a certain range as defined by probability of transmission in the slot, the node join request is transmitted to the parent. When the node join request reaches the root node, it selects the parent for the new node, based on link quality as mentioned above, and accordingly updates the routing tree. Also as the topology get changed, it updates the routing table entries in the kernel routing table. For updating the kernel routing table we make use of user program, fractel routing daemon. The madwifi driver code interacts with the user pro-

Node join design For easier maintenance, FRACTEL MAC has provision for automated node join and reconfigurability of topology in case of failure. Node join is the mechanism wherein a newly started node becomes part of the network without any manual intervention. When a node first starts (or reboots in case of failure), it waits to hear schedule packet from neighbouring nodes, so as to synchronize with the network. It then propagates the node join request consisting of link quality, in terms of RSSI values, from neighbouring nodes towards the root node. This node join request is propagated towards the root only in contention slots, which supports random contention access mechanism with probabilis3

testbed comprises of a 5 node, 4 hop linear topology, with distance between the nodes varying from 150m to over a km. The hardware used for experimentation consists of Mikrotik RB433AH, and Ubiquiti SR2/XR2 wireless cards. Mikrotik boards were running openwrt kamikaze v8.09 with linux kernel version 2.6.26.5. The bandwidth and jitter measurements were carried out using iperf v2.04. Fig. 4 below, depicts our outdoor campus wide wireless testbed. We present comprehensive results of the evaluation, alongwith certain corrections required in the implementation below.

gram using the kernel proc file system. As the routing tree gets changed on joining of a new node, the driver code updates the new routing tree into the proc file, fractel routing entries. The user program, fractel routing daemon reads these entries and accordingly updates the kernel routing table. It makes entries in the routing table for each node in the tree, making the child node as the gateway node for the descendents which it can reach through that child, and parent node as the gateway node for all other nodes which it cannot reach through any of it’s child node. For this, the user program fractel routing daemon traverses the subtree rooted at each of it’s child node and makes the child node as gateway for all nodes in the subtree rooted at that child node. For all other nodes in the routing tree, not covered so far, it makes the parent node as the gateway. This procedure of updating routing table entries is further illustrated in fig. 3. Also this updating of routing table entries is carried out at all the nodes in the network as and when they receive the schedule packet containing the updated topology. The experimental evaluation of node join is dealt with in next section.

Figure 4: Outdoor campus wide wireless testbed

gateway node for all other nodes Parent

gateway node for all it's descendants

3.1 Node N

Child 1

gateway node for all it's descendants

Before carrying out the evaluation, we observed that there were several anomalies in the implementation of the MAC protocol that needed to be rectified. Firstly in the current implementation, a node stops propagating schedule information to it’s child nodes, if it does not receive the schedule packet from it’s parent node. This anomaly causes errors occurring up the tree to propagate downwards and may lead to inefficient performance, especially in wake of high wireless losses.This small change took up considerable effort and time, especially given the unconducive development environment for madwifi driver code which makes debugging considerably difficult. Moreover in addition to this anomaly there was a bug in the implementation, wherein a node was transmitting for all it’s scheduled data slots in the frame, but last. This bug was causing the throughput to be measured slightly less than theoretical. The fig. 5 illustrates the change in throughput after removing the bug. As can be seen the measured throughput in the later case is close to

Child 2

Figure 3: Routing entries update at node N

3

Corrections to the prior implementation

FRACTEL TDMA MAC protocol evaluation

To ascertain the robustness of FRACTEL TDMA MAC protocol design and implementation, we have carried out it’s evaluation on an outdoor wireless testbed in the campus. This work was carried out along with Lokendra Singh, an MTech student. The 4

expected value after accounting for wireless losses.

Throughput Vs Hops at 6Mbps (802.11g) 2000

Expected UDP TCP

(1.42%)

Throughput (Kbps)

(1) (1)

Throughput (Kbps)

Expected UDP After Bug Removal Before Bug Removal

2000

1500 (6) (10)

1000

(12) (13) 500

1500 (10.1%)

1000

(13.5%) (18.0%)

500 0

(16) (18)

1

2

3

4

Hops

(a) 0 1

2

3

4

Throughput Vs Hops at 12Mbps (802.11g)

Hops

4000 3500

3.2 3.2.1

Throughput (Kbps)

Figure 5: Throughput improvement after removing the bug

Throughput, jitter and node join delay measurements

3000 2500 (15.17%)

2000 1500

(26.6%) (30%)

1000 500

Measurement for throughput

0 1

Firstly, we measured the performance of the network in terms of throughput obtained by varying the number of hops at two different data rates of 6Mbps and 12Mbps. The parameters for the FRACTEL MAC protocol are highlighted in the table 1. For scheduling, we had slots assigned to each node sequentially, ie. for eg. at 3 hops, first slot was alloted to root node, second to node 1, third to node 2 and this sequence was repeated. Fig.6 shows the results for TCP and UDP throughput for these two different data rates. Expected throughput is calculated taking into account overheads, such as packet headers. guard intervals, etc and assuming that there are wireless errors. Slot duration No. of control slots No. of contention slots No. of data slots Guard interval Frame duration Probability of transmission in contention slot

Expected UDP TCP

(1.03%)

2

3

4

Hops

(b)

Figure 6: Measured throughput at different hops for two different 802.11g phy rates of 6 and 12Mbps (figures in parentheses indicate the observed error rates) As can be seen the measured UDP throughput, is quite close to the expected value after accounting for the losses. However the measured TCP throughput is significantly less which is due to high error rate observed on the wireless links. The likely cause of high losses was the presence of significant external interference in the campus.

3ms 3 2 30 100µs 105ms 0.7

3.2.2

Measurement for jitter

An important parameter to evaluate the perceived QoS is delay and jitter. Typically for most of the real time applications, delay and jitter values are required to be low, for eg. for real time voice application the delay is required to be less than 150ms with low jitter. Delay and jitter depend to a considerable extent on number of nodes in the network and the scheduling policy used, as well as on other factors such as wireless losses. Here we evaluate

Table 1: FRACTEL MAC parameter values

5

Node Join Delay Time(ms)

jitter for the linear topology mentioned above with scheduling scheme as mentioned. The jitter values, as obtained using the iperf tool,for two different data rates of 6Mbps and 12 Mbps are shown in fig. 7. As can be seen jitter values are below 30ms even for 4 hops which is indicative of the robustness of the TDMA implementation. 50 Minimum at 6Mbps Minimum at 12Mbps Maximum at 6Mbps Maximum at 12Mbps Average at 6Mbps Average at 12Mbps

Jitter (ms)

40

2500 2000 1500 1000 500

1

2

3

4

Hops

30

Figure 8: Measured node join delay at different hops

20

protocols. Some of the problems that we encountered include mal-functioning of the wireless cards, mikrotik router boards, power supplies, etc, sudden deterioration in the link quality due to changes in antenna alignment, etc. Troubleshooting and identifying the correct cause of failure can be very difficult as we realized based on our experience. For eg. in one instance, the card was functioning properly when tested indoors but when deployed on SOM building terrace, we could not make the link between SOM and Kresit work. We resorted to several other changes, such replacing the antenna, PoE, etc before identifying that the wireless card had malfunctioned and was transmitting with much less power than it was set to. Likewise another problem we faced was that, while testing our TDMA protocol for any changes that we made in the code, we more often ended with loss of connectivity due to faulty changes or misconfiguration. As we did not have ethernet connectivity to the mikrotik router boards deployed on remote buildings, we had to personally go there to rectify the errors resulting in loss time. The solution that we devised for this problem was that we had watchdog running in each node, which periodically monitored for connectivity with any of the nodes in the network. If it could not connect with any other node in the network, it reverted to fail safe mode. The fail safe mode that we used is CSMA 802.11g 6Mbps, as we observed that all our links perform well at that data rate. In short it can be stated that working on outdoor wireless tested can be pretty challenging, and debugging the troubleshooting the problems requires considerable experience.

0 1

2

3

4

Hops

Figure 7: Measured Jitter values at different hops

Measurement for node join delay

Node join delay is the time from which the node receives first schedule packet, till it becomes part of the network. It is essential that the node join delay be not too high, especially since the same mechanism is used for reconfiguring the topology in case of failure. In order to measure node join delay at different hops, we used following methodology. We first shut down the entire network, and then start the nodes one after other, first the root node, then hop 1 node and so on. Likewise, we repeated this entire experiment five times and the average readings for node join delay are shown in fig. 8. The probability of transmission in contention slots was kept as 0.7. As can be seen the node join delay is within few seconds, even at four hops. Considering the high wireless losses, such low values of node join delay justify the design choice of transmission of node join requests in contention slots.

3.3

Minimum Maximum Average

0

10

3.2.3

3000

Experiences with the testbed

For evaluation of the FRACTEL MAC protocol, we carried out measurements on the outdoor testbed set up in the campus, which enabled us to gain an insight into practical difficulties in outdoor experimental evaluation of the wireless networking 6

4

Literature survey

outlines a TDMA implementation with support for reconfiguring the slot times, by making use of a kernel mode driver, built on top of the madwifi driver. Although they report synchronization accuracy of 25 microseconds, it is not evident that the synchronization mechanism proposed is applicable to multihop networks. Of the couple of implementations reported for multi-hop networks, [11] outlines the design and implementation of the TDMA MAC protocol which is based on in-band time synchronization. It proposes a framework which takes into account hardware bottlenecks such as clock drift, processing delay, turnaround time for changing the device state while switching from transmission to reception, etc for calculating the guard interval, slot duration, etc. It takes into account wireless errors for calculating the synchronization period, ie. time interval after which synchronization needs to be done. It formulates a linear program to minimize overheads, given the values of different parameters such as clock drift, processing delays, etc. Using proprietary WiLD MAC platform, it attains synchronization accuracy of the order of microseconds, as well as slot granularity is of the order of milliseconds. It also gives comprehensive measurement evaluation to justify their design procedure. Likewise, [12] proposes a pairwise synchronization algorithm similar to FRACTEL, and takes into account propagation delay to calculate clock difference between two nodes. It makes use of a guard band to account for processing delays encountered while packet transmission, and carries out measurements to empirically tune the value of the guard band. Based on the measurements carried out on an indoor testbed, it reports synchronization accuracy of few microseconds at 3 hops. Although both these implementations for multihop networks report time synchronization accuracy of the order microseconds, which is good enough, the major difference between these and FRACTEL MAC is that the latter has support for various other essential features also, such as schedule dissemination, routing, topology maintenance, etc, which makes it a more generic protocol. Moreover the evaluation of both these protocols has been carried out on an indoor testbed, whereas FRACTEL MAC has been evaluated on an outdoor wireless testbed which further confirms the robustness of the protocol design to operate in wake of varied and diverse conditions. Table 2 below further highlights the

In this section we present a brief survey of available literatue dealing with various issues concerning optimizing the performance of wireless mesh networks. We first outline different TDMA MAC implementations available in literature and compare them with FRACTEL MAC. We then briefly review different techniques proposed in literature for improving the capacity of the network, such spatial re-use, multichannel and multi radio operation. We then deal upon the problem of efficient scheduling of time slots in TDMA mesh networks, taking into account the QoS requirements of different applications. We also highlight the open problems in all these areas, stressing upon the issues that we wish to tackle in our research work.

4.1 4.1.1

TDMA MAC protocol implementations Related Work

Several TDMA MAC protocols have been proposed in literature for wireless mesh networks, however very few works give implementation based evaluation of the protocol. This is largely due to technical challenges involved in time synchronizing the individual nodes, especially at fine granularity. Even of the few TDMA MAC implementations available in literature, most are for single hop [2, 8, 9, 10]. To the best of our knowledge very few implementation based evaluations exist for multi-hop networks, which include [11, 12]. Of the various single hop tdma implementations, [9] proposes a flexible research platform for software defined radios and builds upon a system that allows researchers to modify the custom 802.11 protocol as per their needs. It demonstrates that it is possible to implement TDMA protocol for commodity hardware by appropriately modifying open source madwifi driver code. However they do not lay emphasis on tight time synchronization and present no results for synchronization accuracy. [8] outlines the hardware specific challenges involved in implementing a TDMA MAC protocol and describe the changes required in open source madwifi driver for the same. They also provide support for multi-channel operation and report channel switching latency of 4-5 ms. [2] proposes an implicit synchronization mechanism which is an improvement over [1], to reduce the idle period and hence improve utilization. [10] 7

similarities and differences between these TDMA all the links originating at those two nodes calculate the effect of interference using the metric, BIR, multi-hop MAC protocols. defined as BIR = 4.1.2

Proposed Work

AB is the delivery rate of packets at B from where RAC A when both the nodes A and C are transmitting. Using this methodology, the number of measurements reduces to from O(n4 ) to O(n2 ). The authors compare the performance of the method proposed with the method that carries out all O(n4 ) measurements for several different scenarios, such as with auto-rate disabled, with reduced transmit power and so on. Based on the results authors conclude that proposed method gives quite accurate results. However it can be noted that even though the proposed method reduces the number of measurements to O(n2 ), but still the overhead can be large even for medium sized network and can incur significant network downtime for measurements. The previous work focussed only on the pairwise interference between links in the network, however [14] notes that it is essential to consider the cumulative effect of more than one interferer. Based on the measurements carried out, they observe that k of the nodes when active in isolation may not interfere with transmission from a given node c, but when these k nodes are active simultaneously, they may together cause interference to the transmission from node c. They term this interference as k-way interference and they note that number of measurements grows as O(nk ), if one needs to account for this k-way interference. Likewise another study [15] makes an interesting observation that different interferers act independently. They give measurement based validation for the same and highlight that using this property of independence of interferers, it is possible to calculate the k-way interference in O(n2 ) measurements. Another work [16], proposes a novel approach to reduce the measurement overheard. They show that if the two packets are transmitted at precisely the same time by two nodes, then by noting the effect of collision on the packet transmission it is possible to gauge the effect of interference. This is an considerable improvement over the technique outlined by [13], which requires the transmission between two nodes to last for atleast 20-30 seconds to observe the effect interference. Whereas using the technique mentioned, the effect of interference can

Although time synchronization and node join are in place in current FRACTEL MAC implementation, other functionalities such as flow request, topology updates, etc have yet to be implemented. Moreover to ascertain the robustness of the protocol to provide QoS guarantees for different applications, there is need for it’s thorough evaluation. So our plan for proposed work for TDMA MAC protocol can be stated as, 1. Implement other designed functionalities of the FRACTEL MAC, ie. flow request, schedule dissemination and multi-hop routing for the data path. 2. Carry out thorough evaluation of the FRACTEL MAC protocol on outdoor wireless testbed for different QoS applications, such as real time voice, video streaming, etc, over sustained duration of time so as to ascertain it’s robustness.

4.2 4.2.1

AC +RAC RAB CD RAB +RCD ,

Spatial reuse and interference modeling Related Work

Spatial resuse is one of the mechanisms for improving the capacity of wireless mesh networks, however incorporating spatial reuse requires accurate interference modeling which can be challenging. Several works have been proposed in the literature which focus on this problem of interference modeling in wireless networks. We present a brief summary of some of the key works in this area, highlighting their shortcomings. [13] is one of the first papers to propose a measurement based methodology to estimate link interference in wireless mesh networks. It highlights that for calculating the effect of interference between each pair of links, potentially 0(n4 ) are required. Such huge number of measurements can induce significant network downtime. So to reduce the number of measurements they propose a methodology wherein a given pair of node broadcasts in turn and 8

Synchronization accuracy Slot granularity Hardware used Support for routing

Support for schedule dissemination and topology maintenance Outdoor testbed evaluation with 5 node, 4 hop linear topology

FRACTEL MAC 4-5 µs over single hop ms Commodity 802.11 hardware Support for generic routing

Soft TDMAC [12] 4-5 µs over single hop

Yes

µs Commodity 802.11 hardware Implements distance vector routing along with the MAC; No support for generic routing No

Yes

No

TDM MAC [11] 3-4 µs over single hop µs Proprietary hardware No

No

No

Table 2: Comparison of multi-hop TDMA MAC protocols to receiver with SIR. They carry out indoor experiments to observe the dependence of SIR on error rate and report that there is transition region beyond which error rate is either 0 or 100 %. So if SIR for a given sender and interferer is in 0 % region, then both sender and interferer can operate simultaneously. They also carry out experiments in outdoor environment and find the similar relationship between SIR versus error relationship. However, for the controlled experiments carried out in indoor environment they find that the transition region for SIR(beyond which the error rate is 0 or 100%), is as wide as 30dB, while for the outdoor experiments they find that the transition region to be around 10-15 dB. They attribute the cause of this disparity to the capture effect, where in the receiver locks to one of the transmitting nodes even if the signal strength of the other transmitting node is higher. [17] also proposes a method to calculate the effect of adjacent channel interference using the spectral mask specified by 802.11 standard. Although it can be noted that even though the overhead of this SIR based methodology for interference modeling is much less, the prediction accuracy is seemingly lower than some of the other techniques described above.

be calculated in a single packet transmission, requiring atmost few milliseconds. The authors term their technique as micro-probing. However the major disadvantage of this technique is that it requires fine grained synchronization accuracy between the transmitting nodes of the order of microseconds, which can be difficult to achieve. For observing the effect of carrier sense interference, they make use of MAC service time which is the time difference between the event the packet is handed over to the MAC layer for transmission and the actual packet transmission begins. In order to ensure that the two nodes do not transmit at the same time, they make one of the nodes transmit a small interval of time before than the other node and observe the MAC service time for the packet transmission at the other node. They carry out comprehensive measurements to evaluate their methodology against the technique proposed in [13] and find that the two techniques give comparable results with the overhead of the former being 400 times less. However as noted by the authors, one of the major disadvantage of the technique is need for tight time synchronization of the order of microseconds which is an engineering challenge. Likewise another work [17, 18], propose the use of RSSI(received signal strength indicator) based metric to calculate the effect of interference. Based on the RSSI of sender and interferer at the receiver node, they calculate the SIR (signal to interference ratio) and correlate delivery probability from sender

Below we present a comparison of some of the key techniques outlined above, based on prediction accuracy, overhead, etc. Also we note that none of the techniques account for effect of interference at different channel or rates. ie. none of the tech9

niques predict if the interference map generated for divided into fixed size cycles with each cycle cona particular channel/rate will applicable at different sisting of two slots of equal length. Each node operates in one of the channels in first slot and other channel/rate. Table 3 highlights these details. channel in the second slot. For routing they make use of a spanning tree, as they argue that most of the traffic is directed from a wireless node to4.2.2 Proposed Work wards a root, which acts as a gateway for the wired As can be seen by comparison, each of these tech- connection. They also incorporate load balancing niques has certain disadvantages and moreover no into routing wherein a node switches to a new partechnique exists that accounts for interference at ent node, if load on the current parent increases. different channels and at different rates. Moreover They carry out measurements to evaluate the efmost of these techniques have been outlined for fectiveness of their scheme to counter the presence CSMA protocol. Although the same technique may of external interference. By operating the nodes be applicable for TDMA mesh networks, there is a on non-interfering channels, they note their scheme need for measurement based validation for the same causes no reduction in throughput. They also study especially in outdoor testbed environment (as the the impact of slot time on throughput and effeclikely deployment scenario for FRACTEL is out- tiveness of their scheme for efficient load balancdoor environment). In our proposed work we wish ing. They observe that their scheme distributes the load evenly over multiple gateways under heavy to tackle these issues load conditions. However their work does not out1. Carry out a systematic measurement based line any channel assignment methodology and also study on an outdoor testbed environment, to the slot granularity achieved by them is of the order investigate the applicability and the effective- of milliseconds with channel switching time being ness of the three key techniques outlined above, about 2 ms. Such large slot sizes are not approin the context of TDMA mesh networks. priate for providing QoS guarantees to real time 2. Investigate an efficient mechanism that ac- applications. [20, 21] outline the practical difficulties in using counts for adjacent channel interference, predicts the effect of interference at different chan- multi-radio and multiple channels. [20] highlight nels (based on the measurements done at a par- that when using multiple interfaces within a sinticular channel) and at different rates as well. gle node, the performance may infact get affected due to board crosstalk, radiation leakage, inade3. Incorporate efficient interference modeling quate separation between the antennas, etc. They mechanism in the FRACTEL TDMA MAC carry out throughput measurements and find that protocol utilizing the techniques mentioned when the three or more interfaces were active, the above with appropriate modification, address- throughput is less than when only one or two ining the implementation issues for the same. terfaces were active. They also carry out measurements to study the effect of channel separation on 4.3 Multi-channel and multi-radio oper- performance and note that even if there is sufficient separation between the channels at which the two ation interfaces operate, the performance does not neces4.3.1 Related Work sarily improve. Also they note that when one inUse of multiple channels and multiple radios can terface is transmitting while other is receiving, the significantly improve the capacity of the network, receiving interface’s throughput degrades considerhowever using both these mechanisms involve cer- ably. Also they find that if the antenna separation tain issues which need to be resolved. Below we between two interfaces is less than 35dB, then there present summary of few important works in these is no significant improvement in performance. Likewise [21] highlights that even for a single area highlighting the open issues. [19] highlights the need for multi-channel opera- interface, the performance is different at different tion to mitigate the effect of external interference. channels. They carry out measurements to ascerThey propose a architecture wherein nodes are syn- tain link quality in terms of RSS (received signal chronized to each other and radio channel time is strength) and note that RSS for a given link at dif10

BIR [13] Prediction accuracy Measurement overhead

Implementation complexity Outdoor testbed evaluation with 3-4 nodes Prediction for adjacent channel interference Prediction for effect of interference at different channels Prediction for effect of interference at different rates

SIR [17]

High Very high; O(n2 ) measurements with 20-30s for each Low No

micro-probing [16] High Moderate; O(n2 ) measurements with 10-15ms for each High No

No

No

Yes

No

No

No

No

No

No

Low Low; O(n) measurements with 1015ms for each Low Yes

Table 3: Comparison of interference modeling techniques ferent channels is different. For nodes with multiple interface, they note that different interfaces may behave differently even at the same channel. They attribute this disparity to manufacturing variations or insufficient antenna separation. They highlight that by appropriately making use of these observations in the channel assignment algorithms, it is possible to improve the performance significantly. [22] highlight that there is a need to use multiple radios in a node in collaborative manner and outline the design guidelines for the same. These design guidelines include, design for choice ie. use of radios with possibly different characteristics, design for flexibility ie. the availability of multiple radios should be abstracted from the application layer, and design for separation ie. control path and data path should make use of separate radios preferably. They describe several practical systems, which incorporate these design guidelines thereby illustrating their efficacy. [23] outline a practical mesh network based system that makes use of dual radios in a node to improve the capacity of the network. They first note that even if the two interfaces within a node are operated at two channels that are far apart but in the same frequency band(say 802.11b/g or 802.11a), then there is no improvement in performance unless the antennas is separated by sufficiently large distance of 18 inches or more. Since increasing the antenna separation increases the form factor, they propose operating the two radios in two different frequency bands ie. one in 802.11a and the other in 802.11b/g. They also note that any channel as-

signment strategy needs to take into account short term variations in the link quality as well as external interference. They highlight that existing metric such as ETT is incapable of tracking short term variations in link quality and propose changes to incorporate the same. For this purpose they keep track of average and mean deviation in link delivery ratio as follows, pa = pa + g.(p − pa ) pv = pv + g.(|p − pa | − pv ) where pa and pv are the average and mean deviation in the link delivery ratio, while g is the gain factor whose value is set to 0.2. For channel assignment, they consider a mesh network scenario consisting of a gateway node and mesh access nodes connected to the gateway node, via one or more hops. The channel assignment strategy they propose is for the scenario wherein the gateway node has a single radio while the mesh nodes have two radios each. The primary intent of their channel assignment strategy is to minimize intra-path interference. Accordingly the gateway selects a channel sequence, consisting of series of channels alternating from two frequency bands, 802.11a and 802.11b/g, for example b1,a1,b2,a2,b3,a3. Based on it’s distance from the gateway node, the mesh access node assigns the channels to it’s interfaces, for eg. a node i hops away from the gateway assigns channel (ci , ci+1 ), to it’s interfaces. Thus as the two interfaces of a node are operating on two channels in different frequency bands, and the channels are not repeated for the

11

links unless they are geographically far apart, the intra-path interference is minimized. Based on the measurement results they illustrate that their strategy yields significant improvement in performance in comparison to several other strategies. They also carry out measurements to highlight the efficacy of using a metric that takes into account short term variations in the link quality, as well as external interference. However as mentioned, the primary intent of their channel assignment strategy is to minimize intra-path interference, but under heavy load inter-path interference can also significantly affect the performance.Moreover they do not specify detailed channel assignment strategies for the case of gateway nodes with multiple radios or multiple gateway nodes in the network. 4.3.2

Proposed work

One of the major limiting factor for using multichannel operation in single radio systems is that channel switching latency of the commodity 802.11 hardware is of the order of milliseconds [19, 24]. Such high channel switching latency limits the efficiency and may affect the performance of QoS sensitive applications. However as we have observed, there is provision in madwifi driver for fast channel switching with delay of around 500 microseconds, for some of Atheros chipset based cards. But we need carry out further investigations to incorporate the functionality in the FRACTEL MAC protocol. While for multi-radio operation, as noted by [20, 21], there are considerable practical difficulties which limit the applicability of this technique.Moreover as noted by [22], there is need for appropriate architecture which suitably optimizes the performance of multi-radio systems. So our future work can be stated as, 1. Investigate the fast channel switching mechanism available in the madwifi driver code and incorporate it in the FRACTEL MAC protocol. 2. Devise an efficient channel assignment algorithm in context of FRACTEL MAC protocol taking into account the channel switching latency, ie. if channel switching with low latency (of the order of microseconds) is not feasible, then the algorithm should optimize the performance taking into account the high channel switching latency.

3. Investigate the applicability of multi-radio systems in context of FRACTEL MAC protocol, devising an efficient architecture for the same.

4.4

Scheduling in TDMA based mesh networks

Above we focussed on the various techniques for improving the capacity of the network, but to efficiently utilize the available capacity such that the QoS requirements of different applications are fulfilled requires the use of efficient TDMA scheduling algorithms. The problem of unicast scheduling in TDMA based networks is equivalent to vertex colouring problem in graphs [25, 26], which is a well known hard problem and for which no polynomial time algorithm is known to exist. Moreover the problem becomes even more difficult if the delay requirements of different applications have to be satisfied. In order to fulfill the delay requirements, the links over a given path have to be scheduled such that the delay between the packet transmission from source to destination is less that maximum delay specified by application. Again if the scheduling algorithm takes into account multi-channel operation as well as joint routing, the complexity of the problem further increases. Below we highlight some of the related work for the problem of scheduling in context of TDMA based mesh networks, for these different variations. 4.4.1

Related Work

[26] deals with the problem of both unicast and broadcast scheduling in TDMA based multi-hop networks. For unicast scheduling they devise an optimal polynomial algorithm for tree networks. For graphs, they devise an approach wherein a graph is divided into several oriented graphs, which can be coloured easily. An in-oriented graph is one which consists of atmost one outgoing edge, while an out-oriented graph is one which consists of atmost one incoming edge. They outline an algorithm to colour oriented graphs using 0(βρ) colours, where ρ is the degree of the graph, while β is the maximum number of neighbours for a vertex, with labels lower than itself. Using this algorithm for oriented graphs, the authors devise an algorithm, with performance guarantee 0(1) for planar graphs and 0(θ2 ) for arbitrary graphs, where θ is the thickness of the graph. Thickness of a graph is the mini-

12

mum number of planar graphs that it can be partitioned into. Although [26] outlines efficient polynomial time algorithms for the TDMA scheduling in multi-hop networks, they do no consider multichannel operation, nor do they consider the problem of routing. Also they do not take into account delay requirements of different applications. [27] tackles the problem of scheduling taking into account the delay requirements. It follows a novel approach, wherein it divides the network into a bipartite graph such that every node is scheduled to transmit in every alternate time slot. For dividing the network into a bipartite graph, they formulate the routing problem as an integer linear program (ILP), with the constraint that every node is scheduled to transmit in every alternate time slot. For efficiently solving the ILP so formulated, the authors propose 3 heuristics, one based on the shortest path of the node from the root, while other two heuristics are based on the load at the node, where load is defined as maximum of incoming or outgoing traffic. Given this scheduling mechanism wherein every node transmits in every alternate slot, they argue that the scheduling delay for any given path is atmost twice that obtained using any scheduling policy devised for wireline networks such weighted fair queuing (WFQ), round robin, etc. This approach of dividing the network into a bipartite with the objective of providing assured performance is similar to that followed in [1]. However some of the disadvantages of [27] are that, they consider only tree topology for transfer of data wherein data transmission occurs between a node in the tree and the root node. However in more general cases, the data transfer can be between any two nodes in the network. Moreover the heuristics proposed by them may not always yield a solution to the ILP, even though one may exist. [25], [28] also focus on the problem of delay aware scheduling and devise algorithms to minimize scheduling delay. [28] considers the TDMA model of 802.16 and 802.11, where the time is divided into slots of fixed duration and these slots are grouped into frames. The frame consists of two different types of slots, control and data slots. The authors in [28] show that the problem of delay aware scheduling reduces to the problem of finding shortest path if the relative order of transmissions between every pair of nodes is known. By relative order they mean that a given a pair of nodes, it is known which node transmits earlier and which

later. They also formulate ILP’s to find the relative order of transmissions and to minimize the maximum delay over all source destination pairs. They propose a polynomial time algorithm for tree topologies to find a schedule with delay of at most one frame. Likewise [25] follows graph theoretic approach and formulates a polynomial time algorithm that schedules the links so that there is no scheduling delay over any given path. It considers a tree topology with two hops and evaluates the performance guarantee of the algorithm to be 3/2 times the optimal. However, one of the shortcomings of both the works is that they consider only the case where the network is organized as a tree.[28] does formulate the problem of delay aware scheduling in general graphs as an ILP, but it does not give any polynomial time algorithm for it. [29] presents an interesting classification of TDMA scheduling algorithms based on whether they are centralized or distributed, whether they take into account multi-channel operation, spatial reuse, etc, whether they solve the joint problem of routing, etc. It specifically studies the problem of scheduling real time voice/video in the context of mesh networks and notes that there is lack efficient algorithm which jointly solves the problem of routing taking into account multi-channel operation and spatial reuse and suitably fulfills the QoS requirements of these different applications. 4.4.2

Proposed Work

As noted by [29], there are no efficient algorithms available for voice/video scheduling especially in context of FRACTEL MAC protocol. Moreover there is no concrete implementation based evaluation of the scheduling algorithms proposed in the literature. Efficient implementation of the algorithm with running time of few milliseconds is a challenging aspect in itself, especially since the processing power of the hardware used for WiFi routers is typically much less. So our proposed work can be stated as,

13

1. Devise an efficient scheduling algorithm, in the context of FRACTEL MAC protocol, that takes into account spatial reuse and multichannel operation and satisfies the delay requirements of voice and video applications. Also the algorithm should jointly solve the problem of routing thereby maximizing the use of available resources.

(of the order of microseconds) is not feasible, then the algorithm should optimize the performance taking into account the high channel switching latency.

2. Carry out implementation based evaluation of the same.

5

Roadmap work

for

the

proposed

3. Investigate the applicability of multi-radio systems in context of FRACTEL MAC protocol, devising an efficient architecture for the same.

Here we outline the roadmap for the proposed work. Feb 2010 - March 2010 : TDMA MAC protocol implementation

December 2010 onwards : Scheduling in 1. Implement other designed functionalities of the TDMA based mesh networks FRACTEL MAC, ie. flow request, schedule dissemination and multi-hop routing for the 1. Devise an efficient scheduling algorithm, in data path. the context of FRACTEL MAC protocol, that takes into account spatial reuse and multi2. Carry out thorough evaluation of the protochannel operation and satisfies the delay recol on outdoor wireless testbed for different quirements of voice and video applications. QoS applications, such as real time voice, video Also the algorithm should jointly solve the streaming, etc, over sustained duration of time problem of routing thereby maximizing the use so as to ascertain it’s robustness. of available resources. April 2010 - July 2010 : Spatial reuse and interference modeling 2. Carry out implementation based evaluation of 1. Carry out a systematic measurement based study on an outdoor testbed environment, to investigate the applicability and the effectiveness of the three key techniques outlined above, in the context of TDMA mesh networks. 2. Investigate an efficient mechanism that accounts for adjacent channel interference, predicts the effect of interference at different channels (based on the measurements done at a particular channel) and at different rates as well. 3. Incorporate efficient interference modeling mechanism in the FRACTEL TDMA MAC protocol utilizing the techniques mentioned above with appropriate modification, addressing the implementation issues for the same. August 2010 - Nov 2010 : Multi-channel and multi-radio operation 1. Investigate the fast channel switching mechanism available in the madwifi driver code and incorporate it in the FRACTEL MAC protocol. 2. Devise an efficient channel assignment algorithm in context of FRACTEL MAC protocol taking into account the channel switching latency, ie. if channel switching with low latency

the same.

6

Conclusion

FRACTEL MAC protocol makes use of TDMA based mechanism to provide QoS support in wireless mesh networks. Based on the evaluation of the FRACTEL MAC in outdoor wireless testbed, we conclude that the protocol design is robust enough to support essential real time applications. However to further increase the performance of the FRACTEL protocol there is need to incorporate spatial resuse, multi-channel and multi-radio operation. But there are several challenges in practically realizing these techniques, such as need for accurate and efficient interference modeling with minimal overhead, fast and efficient channel switching implementation, efficient channel assignment algorithms, etc. Also for efficiently utilizing the available capacity of the network and to fulfill the varying QoS requirements of different applications, there is need to devise efficient scheduling algorithms that take into account spatial reuse, multi-channel and multiradio operation. We intend to investigate the issues and challenges involved in practically employing these various mechanisms in the context of FRACTEL MAC protocol as a part of our proposed work.

14

References [1]

[2]

[3]

[4]

[5]

[12] P. Djukic and P. Mohapatra. soft-tdmac a software tdma based mac over commodity 802.11 B. Raman and K. Chebrolu. Design and evalhardware. In IEEE Infocom, 2009. uation of a new mac protocol for long-distance 802.11 mesh networks. In ACM MOBICOM, [13] Jitendra Padhye, Sharad Agarwal, Venkata N. Padmanabhan, Lili Qiu, Ananth Rao, and 2005. Brian Zill. Estimation of link interference in R. Patra, S. Nedevschi, S. Surana, A. Sheth, static multi-hop wireless networks. In Internet L. Subramanian, and E. Brewer. Wildnet: DeMeasurement Conference, 2005. sign and implementation of high performance wifi based long distance networks. In NSDI [14] S. Das, D. Koutsonikolas, Y. Hu, and D. Peroulis. Characterizing multi-way interference in ’07, pages 87–100, 2007. wireless mesh networks. In ACM WiNTECH Kameswari Chebrolu and Bhaskaran Raman. Workshop, 2005. Fractel: a fresh perspective on (rural) mesh networks. In NSDR ’07, pages 1–6, 2007. [15] D. Niculescu. Interference map for 802.11 networks. In Proceedings of the 7th ACM SIGA. Dhekne, N. Uchat, and B. Raman. ImpleCOMM conference on Internet measurement, mentation and evaluation of a tdma mac for 2007. wifi-based rural mesh networks. In NSDR ’09, 2009. [16] N. Ahmed, U. Ismail, S. Keshav, and K. Papagiannaki. Online estimation of RF interference. Nirav Uchat. Design, Implementation and In Proceedings of the 2008 ACM CoNEXT Evaluation of Multi-hop TDMA system. Conference, 2008. MTech Thesis, IIT Bombay, 2009.

[6] http://madwifi-project.org/.

[17] Rahul Jain. SIR based Interference Mapping. MTech Thesis, IIT Bombay, 2009.

[7] Ashutosh Dhekne. Multi-hop Time Synchronization and Power Saving Mechanisms for [18] Pradeep Gopaluni. Interference Estimation and Automated Generation of Spatial Re-use FRACTEL. MTech Thesis, IIT Bombay, 2009. Map for Wireless Mesh Networks. MTech The[8] A. Sharma and E.M. Belding. FreeMAC: sis, IIT Bombay, 2008. framework for multi-channel mac development on 802.11 hardware. In Proceedings of the [19] G. Wu, S. Singh, and T. Chiueh. Implementation of dynamic channel switching on IEEE ACM workshop on Programmable routers for 802.11-based wireless mesh networks. In Proc. extensible services of tomorrow, 2008. if International Conference on Wireless Inter[9] Michael Neufeld, Jeff Fifield, Christian Donet, 2008. err, Anmol Sheth, and Dirk Grunwald. SoftMAC - Flexible Wireless Research Platform. [20] J. Robinson, K. Papagiannaki, C. Diot, X. Guo, and L. Krishnamurthy. Experimenting In HotNets-IV, 2005. with a multi-radio mesh networking testbed. [10] A. Sharma, M. Tiwari, and H. Zheng. MadIn 1st workshop on Wireless Network MeasureMAC: Building a reconfigurable radio testbed ments (WiNMee), 2005. using commodity 802.11 hardware. In Proc. First IEEE Workshop on Networking Tech- [21] A.P. Subramanian, J. Cao, C. Sung, and S.R. Das. Understanding Channel and Interface nologies for Software Defined Radio (IEEE Heterogeneity in Multi-channel Multi-radio SECON 2006 Workshop). Wireless Mesh Networks. In Proceedings of the [11] D. Koutsonikolas, T. Salonidis, H. Lundgren, 10th International Conference on Passive and P. LeGuyadec, Y.C. Hu, and I. Sheriff. Tdm Active Network Measurement. Springer, 2009. mac protocol design and implementation for wireless mesh networks. In ACM CoNEXT, [22] P. Bahl, A. Adya, J. Padhye, and A. Walman. Reconsidering wireless systems with multiple 2008. 15

radios. ACM SIGCOMM Computer Communication Review, 34(5):39–46, 2004. [23] A. Dhananjay, H. Zhang, J. Li, and L. Subramanian. Practical, distributed channel assignment and routing in dual-radio mesh networks. ACM SIGCOMM Computer Communication Review, 39(4):99–110, 2009. [24] A.P. Subramanian, H. Gupta, S.R. Das, and J. Cao. Minimum interference channel assignment in multiradio wireless mesh networks. IEEE Transactions on Mobile Computing, 2008. [25] D. Panigrahi and B. Raman. TDMA Scheduling in Long-Distance WiFi Networks. Dept. of CSE, IIT Bombay, TR-CSE-2008-16, Tech. Rep., Aug, pages 2008–16, 2008. [26] S. Ramanathan and E.L. Lloyd. Scheduling algorithms for multihop radio networks. IEEE/ACM Transactions on Networking (TON), 1(2):166–177, 1993. [27] G. Narlikar, G. Wilfong, and L. Zhang. Designing multihop wireless backhaul networks with delay guarantees. In INFOCOM 2006. 25th IEEE International Conference on Computer Communications. Proceedings, 2006. [28] P. Djukic and S. Valaee. Delay aware link scheduling for multi-hop TDMA wireless networks. IEEE/ACM Transactions on Networking (TON), 17(3):870–883, 2009. [29] Vijay Gabale. Scheduling in Wireless Mesh Networks: A Survey. PhD seminar report, IIT Bombay, 2009.

16