Simulation of Adaptive Statistically Multiplexed Routing in Ad Hoc Networks G. R. Dattatreya and Sarvesh S. Kulkarni Department of Computer Science University of Texas at Dallas email:
[email protected]
Abstract? This paper develops a solution to the problems of discovery, maintenance, and use of multiple routes in ad hoc networks. Performance criterion is the average time taken by a packet to reach its destination through multiple hops. A source node considers each of its neighbors (reachable by direct wireless transmission) as a gateway for every possible destination. The e ect of delay at a gateway and beyond, until the packet reaches its destination, is approximately modeled as an M/M/1 queuing system. Potential gateways at every node provide multiple routes. Multiple routes are statistically multiplexed to distribute the load as well as to deal with changes in data and network con guration. The potential of each gateway of a node is estimated on-line and the proportions of trac routed through multiple gateways are also updated adaptively. Simulation results are presented and discussed. Index Terms? Rapidly deployable mobile networks, equivalent queuing systems, adaptive parameter estimation, load balancing, and maintenance of multiple routes. I. INTRODUCTION
changes in con gurations include a node being congested or otherwise not being able to serve as a relay station for some time intervals. We rst develop a common problem for load balancing and maintenance of ecient multiple routes between source-destination pairs. The statistical model of trac, switching, and performance are approximate to ensure simplicity. Deviations from our model are expected to be re ected as corresponding variations from the ideal without causing instability, operational problems, or functionality. Our next idea is to maintain and update multiple routing information in a local fashion. Information for routing includes estimates of performance gures for various routes, in addition to estimates of e ective control parameters to achieve good performance. The term \e ective" is used to denote that at a source (or store and forward) node, the actions subsequent to forwarding to a neighbor are not exactly known, but their overall statistical e ect is being estimated and used. The third idea we use in our approach is to generate and send control packets at random time intervals, re ecting small proportions of the actual trac. The advantage of these control packets is that they can explore alternative paths and collect cumulative data for estimation of control parameters for good performance. Control packets are not used in a ooding manner. Control packets can be lost. Packet losses can be due to a node being out of range, temporarily nonfunctional, or congested. Indeed, we treat all forms of non-receipt of a packet over a link by the same statistical means. II. THE ROUTING AND LOAD BALANCING MODEL In our network model, a set of wireless communication stations (nodes) S1 ; : : :; SM , each with a unique identi cation (ID) is spread out in a geographic area. Each Si has a unique communication channel through which it can transmit to and receive from another station Sj in the set N(Si ), the set of possible neighboring nodes of Si . If a node Si strays into a far area where the same channel is used, the unique IDs help in detecting the con ict. A higher level algorithm should resolve the con ict. This is not the topic of the present paper. A node Si may not be able to communicate with its neighbor Sj 2 N(Si ) all the time, due to possible mobility, congestion, and breakdown of the latter. Indeed we expect that many possible neighbors may never directly communicate with each other due to the distance between them. Each Si also attempts to develop and maintain a list of active neighbors A(Si ) N(Si ), with each of
Nomadic computing encompasses a wide variety of computing and communication scenarios wherein the hosts are mobile, may function intermittently, and the network organization itself is ad hoc, somewhat inecient, but caters to the need of possibly a temporary application in environments with very little control. Numerous potential applications of such networks are cited in Kleinrock [1]. Research interest in modeling and development of solution algorithms for various functions of such networks is rapidly growing. The main problem addressed in this paper is the distributed construction and maintenance of multiple routes. Although attention is on ad hoc networks, some of the techniques, especially statistical multiplexing are applicable to other network types also. Johnson [2] suggests separate route discovery and maintenance protocols for routing in ad hoc networks. Haas and Pearlman (see [3]) propose the \Zone Routing Protocol" for large at-routed wireless networks, wherein updates are propagated only locally . Details are omitted here due to page limitation. In this paper, we concentrate on adaptive algorithms for maintaining and updating multiple routes in changing network con gurations, in ad hoc networks. Examples of
1
which, it can directly communicate. Each A(Si ) is time varying. We assume that variations are slow, that each such variation does not a ect the overall topology of the network drastically and that many variations do not take place within a small time interval. Obviously, the time scales of these variations are relative to the data rates. As mentioned earlier, average data rates are not very high and we do not expect the chain of a few successive topology variations to trigger a sequence of events leading to network instability. Data packets of varying lengths arrive at (or are generated at, or submitted to) each station for possible relay transmission to speci ed destinations. Each station is a possible source, destination, as well as a store and forward relay station. The rate of arrival of packets at source Si for destination Sj is ij . The statistical nature of the large number of types and lengths of data packets, sources and destinations allow us to model each store and forward (transmit) station Si as an Exponential server with service rate . Each station has a bu er to handle the trac. Each station Si is an M/M/1 queuing system. Arrivals from di erent sources (either generated at Si itself or forwarded from nodes in N(Si )) merge into one single queue at Si . Delay in a station is the sum of queuing and service delays. Any physical link delay is negligible and is included in the service delay. At the source, a packet generated enters the queue. At the destination, the packet enters the queue and is \processed" or \consumed" when it gets into the service mode, instead of being transmitted. The overall system is a network of M/M/1 queues. A path from source Si to destination Sj is a sequence of stations (with no repetitions) through which relay transmission is feasible, as dictated by existing interconnections. The de nition of a path is intended to aggregate the delays in the sequence of nodes.
forward the packet to its neighbor Sm is given by qX n(jm) 1 ? qn(jl)
(1)
Sl 2
The probability values play a crucial role in the routing strategy. At Sn , even though the number of active neighbors of Sn may be several, we expect that only a few of qn(jm) are reasonable probability values and the rest are negligible, if not actually 0, for a given destination Sj . In this strategy, since a node is visited at most once, it is possible for a packet to get routed in a wrong direction to the point that it gets \cornered" in a wrong node, having visited all the neighbors of such a corner. If a packet gets cornered, it may be reset to so it can extricate itself out by revisiting a node it had once visited. Some limit on the number of such resets will ensure that packets do not wander around for ever. The above probabilistic routing attempts to avoid congestion by distributing the trac and sharing the burden of queuing and forwarding over alternative paths. The use of simple probability matrix at every node simpli es the task of picking from alternative paths right at the source. Routing decisions become local. Given the set of Qn, n = 1; : : :; M, the actual probability of every path for every source-destination pair can be easily computed. Clearly, ij , i , and Qn determine the overall performance measures such as the average cumulative delay of a random packet, delay of a packet given its source-destination pair, the average queue lengths at nodes, etc. We assume that for the operating parameters of loads ij and i , there exists a solution for the load balancing problem. This simply implies that we have enough resources to handle the overall communication requirements. B. The Load Balancing Model
The performance optimization problem with the above routing model is the determination of optimal probability matrices at every node. Simple special cases of such problem and corresponding solutions appear in literature on load balancing in networks of loosely coupled computers [4], [5], [6], and [7]. As a simple example, if a trac stream of unit rate has two alternative paths to a destination and each path is an M/M/1 queue with service rates of 2.25 and 1, routing 90% of trac through the faster server and 10% through the slower is optimal. In a general multi-hop network, we would need to identify all possible paths for all possible source-destination pairs, use probability variables for di erent paths for every trac type (source-destination pair), write a global performance index of \expected delay of a packet," and determine optimal probability values for all paths. We will then have to translate these path probabilities into link probabilities, that is, the entries in the Qn matrices. Clearly, this is a very complicated nonlinear optimization problem with no simple solution algorithm for the general case. The number of parameters to be identi ed and solved for is unrealistically large. Besides, such a solution is useful provided the network con guration, all the trac densities, and service rates are known in advance and remain constant. In the next section we develop practical approaches and an adaptive operational algorithm that the individual nodes can run in a distributed fashion.
A. The Routing Strategy
We would like the path for a packet from source Si to destination Sj to be nondeterministic to balance the ow of trac among many paths. In general, the number of possible paths between a source-destination pair would be quite large even for a sparsely connected network. In deterministic routing, each station maintains a routing list of neighboring nodes as a function of possible destination nodes. The size of the list at each node will be M ? 1 for an M node network. In our model of probabilistic routing, let Sn be a node with a packet to be transmitted for an eventual destination Sj . Node Sn picks a node Sm 2 A(Sn ) N(Sn ) with a probability qn(jm). Thus each node maintains a probability matrix Qn = [qn(jm)] with M ? 1 rows and kn columns where kn = jN(Sn )j is the number of possible neighbors of Sn . A straightforward probabilistic routing described above has the limitation of a packet being routed through a loop. To avoid that, we assume that each node stamps the packets with the ID of the nodes, in the order it visits them. An intermediate node forwards a packet to one of its neighbors not yet visited by the packet. The probabilities of picking neighbors are updated under the above condition, as follows. Let a packet from source Si meant for destination Sj reach an intermediate node Sn after visiting the set of nodes denoted by . The probability with which Sn should 2
III. DEVELOPMENT OF SOLUTION As a simple approximation, we model the e ect of the succession of all the queuing systems in a path as some equivalent M/M/1 queuing system. Thus, if a packet from a trac stream with the original source node of Sn and nal destination of Sj is at an intermediate node Si , then Si considers routing it to one of its active neighbors in A(Si ), one that has not already been visited by the packet. If all other activities in the network are statistically stationary, these gateways in A(Si ) o er di erent M/M/1 choices for the trac stream, as a crude approximation. By routing proportions of the trac to di erent gateways, the load can be balanced to reduce (minimize, under ideal conditions) the expected delay of such trac. Buzen and Chen [5] rst studied such an optimization problem. Dattatreya and Venkatesh [4] study the same problem and develop adaptive estimators for the optimum proportions under the conditions of unknown trac and server parameters. Consider the total trac at Si with the destination Sj . Portions of these may have originated at other nodes. For the moment, ignore the nodes already visited by this trac so that we can consider all the gateways of Si to route the trac. Let the current estimate of the arrival rate of such trac be ^. Let node Si have Gl 2 A(Si ), l = 1; : : :; ki as potential gateway neighbors to which it (that is, the node Si ) can route trac to destination Sj . Initially, A(Si ) = N(Si ), the set of all nodes with which Si shares independent channels. Control packets with destination Sj sent out by Si provide an estimate of delay. Let Rl be the estimated response time of control packets routed through Gl . Let the current routing probabilities used by the control packets (as well as by the data packets) be ^ l , l = 1; : : :; ki. Equation (1), Section II describes how to deal with with packets that have already visited some nodes. The estimate of the equivalent service rate of the gateway Gl evaluates to 1 + ^ ^: (2) R^l l
IV. SIMULATION EXPERIMENTS
A. The Role of Control Packets
Each node delegates the task of collecting information to \control packets." Each node generates and routes control packets for various destinations. The idea is to let the traf c of control packets represent a xed small percentage of actual data trac generated at a node. The statistics of trac generated at a node can be estimated at the node itself. The required parameters are the rates of trac and the average delay from the current source to a destination through a gateway. The estimation approach can be any one of cumulative averages, running averages over a sliding window, exponentially forgetting cumulative averages [7], etc. In our experiments, cumulative delays are updated in an exponential forgetting form, with a weight of 0.1 for the most recent delay observation and a weight of 0.9 for the previous estimate. At every node, the control packets are routed with approximately the same routing policy as that used over the packets of real data. A small fraction of the control packets are routed to all its possible neighbors. That is, Si routes such small fractions of its generated control packets to each node in N(Si ) ? A(Si ) also. This is to monitor currently inactive neighbors. The delays through a currently inactive neighbor can change at a later time to make it a competing active neighbor. A control packet gets stamped with the ID of every node it visits, in order. When the control packet reaches its intended destination, it reverses its path to reach the source. The source thus obtains the cumulative delay experienced by the control packet and uses half the value as the delay from source to destination After a round trip from the source to destination and then back to the source, the control packet gets extinct. If an attempted multi-hop transmission of a control packet from node Si to destination Sj does not result in Si receiving the control packet back within a speci ed time-out period, the control packet is considered lost. At the start-up time, control packets are generated at some predetermined high rates. Routing probabilities are also prespeci ed, possibly with equal values for all gateways. As the network operates, possibly with only control packets at the beginning, the e ects of various gateway We consider ki of the gateways of Si with the largest esti- neighbors for various destinations are estimated by a node. mated equivalent service rates for routing the trac. De ne Routing probabilities are estimated from such information and updated periodically, using (4). 1 d^l = ^ ^ : (3) B. Simulation Setup Rl We chose a 10 node graph shown in Fig. 1 to conduct simulation experiments. This graph provides several mulThen, 3 2 tiple routes to demonstrate route multiplexing. It is well k X connected (but not strongly connected) and re ects a typi66 1 + d^m 77 cal geographic layout. At the start of simulation, only con(4) trol packets are generated (in our experiment, with half the ^l = 666 km=1 777 d^2l ? d^l expected average per-node trac) with equally likely desti4 X d^2 5 nations and routed equally likely to all possible neighbors. m After four updates, 90% of control packets are routed to m=1 three best neighbors and the remaining 10% are routed is the updated estimate of routing probabilities for each l = the to inactive neighbors, to monitor possible changes in their 1; : : :; ki, for load balancing. See Dattatreya and Venkatesh status. At this time, the network is open for transmission [4] for details. of data packets The control packet generation rate The overall approach we propose is for each node to is then modi ed also. and maintained adaptively estimate routing proportions separately for each tional level of the data trac rate.at a predetermined fracstream of trac (based on nal destinations of packets). The routing probability estimates are updated regularly. The next section describes simulation of such a network The updates are based on the cumulative average delay inoperation which also illustrates several implementation de- formation collected by all control packets sent out by node tails. Si to destination Sj . If a control or data packet reaches i
i
3
\Times of Changes" \Node" \Rate" \Link"
Figure 1: Experimental Network
2
1 0 0 1
1 0
11 0
15 0 06 1
3
Figure 2: Total Data Packet Loss Vs. Time
9
11 00 00 11 0
14 0
1 0
11 00
40000 35000 30000 25000 20000 15000 10000 5000 0
7
11 00 008 11
node m and has already visited all of m's neighbors, it is said to be cornered. In the present experiment, such packets are counted as lost (to estimate the fraction of packets that are so lost). Both control and data packets are dropped if they encounter a full bu er. Data packets are treated as datagrams and are not acknowledged at this layer of network operation. The data packet rate at di erent nodes are kept somewhat di erent to be more realistic. They are 4, 6, 8, 4, 4, 12, 4, 2, 4, 3 per time unit, at nodes 0 through 9, respectively. Let be the per-node average of data generated at all the nodes. For the above data rates, = 5:1 per time unit. Routing probability estimates are updated regularly at intervals of about 30 control packet generations per node (125 time units). Data packets are generated after running the network with only control packets for 500 time units (about 1,250 control packets per node). After the data packets start owing, the rate of control packets is xed at 5% of the data packet rate. The rate of packet arrivals at any node is several times since control and data packets generated at many nodes pass through that node. Destinations are random and equally likely. Let be the total service rate of every node (since nodes are expected to be identical hardware devices). Each server has a bu er of size 50 packets. We vary to re ect load variations. We found ratios of 0:167 to represent a fairly stable system with very few instances of congestion problems, whereas ratios greater than 0:33 were found to exhibit severe congestion at almost all nodes. We chose a ratio of about 0.22 to examine the behavior of our algorithm. The time-out value was set at 10 time units.
0
5000
10000
15000
due to cornering were about 1.7% and losses due to fullbu er were about 20%. The losses due to cornering can be reduced considerably by resetting a cornered packet and letting it visit nodes it had earlier visited. We conducted several experiments with changing environments. The following are chosen for illustration. The level of congestion was moderate ( = 0:22). Three different experiments were conducted. The rst experiment is a heavily used node (node 7) becoming nonfunctional and regaining functionality later (shown in faint dotted lines in the plots); the second experiment is a link (between nodes 0 and 1) breaking and being restored later (shown in heavy dotted line). These two are instances of mobility. The third experiment induces a sudden change in data rate at node 7 (to 300% of the original value, shown in solid lines); the date rate returns to the original value later. The plots show variations in data packet losses, mean delay, and mean number of hops as a function of time. The time instants of network changes are shown in the plots with vertical lines (labeled as \Time of Changes" in the legend). When the heavily used node, 7, suddenly stops functioning, all the data sent through it are lost until the neighbors update the routing tables. The data destined to it are lost until the node functions again. In this experiment, no attempt is made to stop sending data with node 7 as the nal destination. Once the node functions again, the data packet losses do not increase. For the link and data rate changes, we see that the data losses due to the transients occur for a very short time period, as seen in Fig. 2; the plot of packet loss due to data rate change is hardly distinguishable from the time axis. In Figs. 3 and 4, the mean packet delay and mean number hops increase sharply due to node failure. But at the next change, back to the original situation, they fall down to the original lev-
C. Simulation Results
The routing probabilities reach stable values within 4 6 updates even for higher than moderate congestion. The average number of hops for a packet was around 2.3 for high congestion loads and about 1.9 for low to moderate congestion loads. For low congestion loads, the routing probabilities converge to 0 or 1 indicating that the adaptive algorithm nds and maintains more or less unique routes. For higher loads, we see very good splits in probability values indicating good statistical multiplexing. For low to moderate congestion, packet cornering was less than 0.03% and there was no loss of data packets due to full bu ers. For very high congestion loads, = 0:33, packet losses 4
tion is available to take advantage of such an approach. Each node can at most know its currently active neighboring nodes which are in its direct transmission range. The rates of di erent classes of trac incident to a node can be estimated. A node can send out control packets with various destinations, route them through its neighbors (gateways), and expect that other nodes will be functioning similarly. We use this principle and evaluate the potential of each neighbor gateway for each class of trac, by the delay experienced by control packets routed through di erent gateways. This approach is used adaptively and the proportions of a class of trac to be routed through di erent gateways is maintained and frequently updated. This provides statistically multiplexed multiple routes for each class of trac. The store and forward activity at each node is modeled as an M/M/1 queue. The e ect of a statistical mix of a sequence of such M/M/1 queues is also approximated to be an M/M/1 queuing system. The advantage of such an approximation is that the average delay at and beyond a gateway is the only statistic required to update proportions of the trac to be routed to the different gateways. The approach is illustrated with simulation experiments and results. The model is statistical and coarse grained; as such, any other operational requirement can be imposed without creating logical con icts. In [8], we present preliminary results of our approach in networks with simulated self-similar trac. REFERENCES [1] L Kleinrock, \Nomadic computing - An opportunity, ACM SIGCOMM, Computer Communication Review, vol. 25, pp. 36- 40, 1995. [2] D. B. Johnson, \Routing in ad hoc networks of mobile hosts," Proceedings of Workshop on mobile computing systems and applications, Santa Cruz, CA, Dec. 1994, pp. 1-6. [3] Z. J. Haas and M. R. Pearlman, \The performance of the zone routing protocol in recon gurable wireless networks," IEEE J. Select. Areas Commun., June 1999. [4] G. R. Dattatreya and R. Venkatesh, \Adaptive performance optimization of loosely coupled processors," IEEE Trans. Syst., Man, Cybern., vol. 21, pp. 607-619, May/June 1991. [5] J. P. Buzen and P. P. S. Chen, \Optimal load balancing in memory hierarchies," in Information Processing '74 Amsterdam, North Holland, pp. 271-275. [6] A. N. Tantawi and D. Towsley, \A general model for optimal static load balancing in star network con gurations," in Performance '84 E. Gelembe, Ed., NY: North Holland, 1984. [7] A. Kumar, \Adaptive load control of the central processor in a distributed system with a star topology," IEEE Trans. Computers, vol. C-38, pp. 15021512, November 1989. [8] S. S. Kulkarni and G. R. Dattatreya, \Statistically multiplexed adaptive operation of ad hoc networks with self-similar trac," 1999 IEEE Emerging Technologies Symposium on Wireless Communications and Systems, Richardson, TX, Apr. 1999.
Figure 3: Mean Data Packet Delay Vs. Time 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05
0
5000
10000
15000
Figure 4: Mean Number of Hops per Data Packet Vs. Time 2.4 2.2 2 1.8 1.6 0
5000
10000
15000
els. In the other two cases of network changes (link failure and data rate change) the changes in mean number of hops and mean delays are not very signi cant and they fall back to the original levels after the second change. The vertical scales in Figs. 3 and 4 are expanded to distinguish between the three plots. However, the increase in the mean delay as well as in the mean number of hops after the network change is quite low for the cases of link failure and data rate change.
V. CONCLUDING REMARKS
In ideal multi-hop packet switched networks, minimizing the expected delay of a packet from source to destination intrinsically solves all the problems of network operation such as nding the best routes, eliminating congestion, etc. Such a static optimization problem is very complex even when all the system parameters are known in advance. Furthermore, in practical ad hoc networks, very little informa5
