An Approach to Flexible QoS Routing with Active

An Approach to Flexible QoS Routing with Active Networks Michael Welzl University of Innsbruck Computer Science Department A-6020 Innsbruck, Austria [email protected]

Alfred Cihal University of Linz Telecooperation Department A-4040 Linz, Austria [email protected]

Abstract Active Networks (AN) are investigated since several years, attempting to satisfy the increasing needs for highly customizable protocol mechanisms in an era where the Internet is threatened to become immutable due to its huge installed base. AN seem to be particularly useful in the context of QoS support. This paper investigates AN for On-demand QoS routing and proposes an approach called AQR. Simulation experiments are reported which show the (varying) benefits of AQR under various network conditions and traffic characteristics.

1. Introduction The Internet has grown to hundreds of millions of nodes; the huge installed base of routers renders changes to the core protocols nearly impossible. At the same time, the characteristics of Internet applications proliferate and the increasingly varying characteristics of data streams calls for adaptable and wide-ranging support by the underlying network. In this context, Active Networks (AN) have gained a lot of attention recently. The goal of AN is to support customized protocol mechanisms that can be introduced in a running network. Differences in known AN approaches concern, e.g., the question of whether remote applications should be able to download protocol mechanisms to a node (router) or whether this right should be reserved to the operators of nodes, and the question of whether the code for these mechanisms should be carried as an additional payload by the data packets in transit or whether shipping and installing such code should be separated from the issue of data transfer. Three major concepts and terms in the context of AN will be referred to frequently in the remainder: i) Active Applications (AA) denote user-provided communicating applications which make use of (and usually, customize) AN; ii) the abstract machine on which all developers of customizations for an AN (i.e. for a particular AN approach)

Max M¨uhlh¨auser Technical Univ. Darmstadt Telecooperation Department Darmstadt, Germany D-64289

can rely is called NodeOS; iii) the runtime system available on an AN node is coined as Execution Environment (EE). As to the above-mentioned varying characteristics of data streams, the well-established scientific domain of Quality-of-Service (QoS) support offers mechanisms both for describing these characteristics (translated into corresponding requirements for underlying services) and for providing adequate support. However, due to the static “installonce-execute-many” nature of the “world” of communication protocols, such mechanisms cannot be exploited and adapted to the desired degree. In particular, QoS routing — a term used for routing mechanisms which consider QoS — suffers from the static nature of networking today. QoS routing is bound to the use of common metrics and procedures which usually rely on distributed network performance data (increasing network traffic) and sophisticated algorithms (increasing the processor load on routers) but usually yield considerable improvements only for certain classes of applications. The present paper combines the concept of AN with suitable QoS routing mechanisms to form a novel approach called “Active QoS Routing” (AQR); it builds upon an earlier version, which was published in [1]. The use of Active Networks in the context of QoS is not new. Chapter two will briefly revise the state of the art in this respect. Following a necessary NodeOS extension in chapter three, the AQR approach proposed in this paper will be introduced in chapter four. Major simulation experiments are discussed in chapter five.

2 QoS Support in Active Networks Providing good QoS implies that a certain set of requirements must not be violated on the path from the sending application to the receiver. Therefore, QoS provisioning demands special mechanisms from various network layers. Difficulties in mapping QoS layers to the corresponding OSI network layers have led to the creation of numerous QoS architectures, most of which are not clearly related to

the OSI model [2]. Sometimes, the necessary mechanisms cannot even be associated with a particular layer; MPLS, for example, does not fit into the OSI model at all [3]. At first sight, this may seem to make Active Networks a good candidate for QoS provisioning: Active Networks, too, do not follow the traditional OSI network layer model. An AA may implement services ranging from the application to the network layer, limited only by the abstractions offered in the NodeOS specification. These abstractions — thread pools, memory pools, channels, files and flows — encapsulate resources for computation, memory, communication and persistent storage [4]. In other words, the NodeOS may prevent an AA from allocating an arbitrary amount of memory or reserving unlimited bandwidth on a link. Further, access to NodeOS resources can and should be limited by the Execution Environment. Aside from supporting security, the NodeOS abstractions provide the basis for stacking an AN on top of any other network, as is the case for the probably largest AN testbed, the “ABone”. Here, Active packets are transmitted between Internet hosts running a regular TCP/IP stack. While this overlay mode of operation makes it easy to deploy and test an AN, it renders QoS provisioning exceedingly difficult — we explain this fact by taking a look at Active Networks from a QoS perspective. If we consider the fundamental AN concepts, we can identify some commonalities:

 Active Networks provide means for injecting code into the network  This code may cause packets to be created by an arbitrary Active Node  The injected code may decide where these packets go  The injected code may carry out computations on packets received Thus, the lowest candidate layers for AN are the network and transport layers, where we find mechanisms such as routing and flow / congestion control. Possible utilizations of AN to support QoS roughly fall into the following categories:1 1. Mechanisms which transfer application layer functionality into the network [6]: An example would be scaling of multimedia content within the network as in [7] or intelligent dropping of packets that correspond to specific frames of a video stream [12]. 2. Mechanisms which are usually associated with layers 3 or 4: Here, we find Active Congestion Control [8], 1 A similar conclusion is drawn in [5], where architectures are described as falling in three main areas: application level, packet level, and programmable hardware level.

which reduces the feedback delay for congestion control mechanisms by moving endpoint algorithms into the network. Other examples focus on load balancing [9] and routing of application layer data [10], [11] (arguably, the latter could also be put in the first category). Our QoS routing mechanism also falls in this category. 3. Mechanisms which rely on non-active QoS provisioning mechanisms: Here, AN are merely used to add greater flexibility to the specification of a QoS request, while the actual QoS provisioning mechanism may be regular DiffServ [16] or RSVP [17]. We only cited a few Active QoS provisioning mechanisms which we consider representative; a more complete overview can be found in [13] and [21].

3 AQR assumptions: A proposal to extend the NodeOS Spec Apart from Active Congestion Control and application layer oriented (category 1) approaches, all of these mechanisms appear to rely on access to some kind of lower layer performance metrics. We therefore propose to extend the NodeOS specification as follows: 1. A set of relevant performance parameters (such as “state of congestion” or “nominal link bandwidth”) must be defined. For AQR, these parameters are delay and available bandwidth. While delay calculation needs no explicit support from the network (and therefore has the advantage of compatibility with existing NodeOSes and Execution Environments), we need to assume that the “available bandwidth” channel property is available for reading. The term “available bandwidth” is vague: there may be numerous different ways to calculate what could be called the “available bandwidth” of a link, including measurements over any interval and methods of bandwidth prediction based on mathematical traffic modelling. We therefore either need a more precise definition or more universal performance parameters which allow an AA to do the calculations itself. The latter can be achieved by allowing access to the MIB (“Management Information Base”) [20] objects ifInOctets and ifOutOctets — byte counters which, if combined with a timestamp and subtracted from the nominal link bandwidth (MIB object ifSpeed), represent the available bandwidth during the time between two measurements [24]. 2. For each of these parameters, a channel is given the properties of the underlying network. If this is not possible due to the mode of operation (e.g., in an overlay

network), an AA which tries to access the parameter must receive an error message. 3. For security reasons, it must be possible to selectively allow read and/or write access to performance parameters. The major advantage of using Active Networks lies in the immediate availability of new network services without the need for standardization. The actual benefit of an active variant of a QoS provisioning mechanism as opposed to a standardized one can thus be found in greater flexibility. In the case of AQR, this flexibility manifests itself in the fact that a user can individually choose QoS parameters and their calculation (note that AQR is sender-initiated).

4 AQR operation In our scheme, the source floods a query into the network, where shortest path routing is already available via a link state routing protocol such as OSPF. The link state routing protocol may be implemented as an Active Application; otherwise, an interface to routing as in the “ASP” Execution Environment is required [17]. This limitation allows us to dramatically reduce the amount of flooding packets without installing per-flow state in any nodes other than the source and destination, thereby increasing the mechanism’s scalability. Theoretically, it should be easy to extend our mechanism to be applicable to ad hoc networks by dropping this requirement, but we did not consider such a case in our simulations (cf. [30] for a related example). To further decrease the load imposed on the network, we require an Execution Environment to be capable of Demand Loading, a mechanism which caches the code of an AA for repeated execution, resulting in much fewer bits to be sent (data packets must only carry AA code if mechanisms change). Demand Loading is used, e.g., in the “ANTS” Execution Environment [18]. The AQR algorithm can roughly be described as follows (see the appendix for a description in pseudocode): 1. The AQR sender calculates all non-cyclic paths to the destination from the link state routing table. 2. A probing packet carrying the QoS requirements, code for QoS calculation, the sender and receiver’s addresses and a list of visited nodes is sent to each first hop of these paths. 3. Upon receiving an AQR probing packet, an AQRcompliant transit node executes the AA code (contained in the packet or cached), which

 checks if the minimum QoS requirements found in the packet can be met (if a threshold — e.g.,

a maximum delay — is exceeded, the packet is dropped),

 compares and updates the QoS data,  adds itself to the list of already visited nodes, and  executes the code of the AQR sender, starting at step 2 — except that no probing packets are sent to the source or to any other already visited node (packets are multicast in the proper direction at each AQR-compliant transit node). 4. Only packets which conform to the minimum QoS requirements reach the AQR receiver, where a list of valid paths is generated. After a predefined period, the best path is chosen and communicated to the sender. If there is more than one best path, the traffic is split among the best paths. This process needs to be repeated in order to be able to react to changes in the network utilization. The repetition frequency, the maximum delay threshold and the time the receiver waits before it informs the sender of the current best path should be a function of the (shortest path routing based) round-trip time (RTT). For our simulations, we chose twice the RTT for the probing frequency and path choice period and one RTT for the maximum delay threshold. As an alternative to the algorithm described here, each node could be provided with its relevant subtree as a data structure in probing packets; clearly, this is a trade-off between computational complexity and scalability (as dictated by the size of probing packets). We have also run experiments with a tolerance parameter (defined as a percentage of the best QoS found) to facilitate traffic splitting, but the results were not satisfactory.2 Thus, for the results presented here, this tolerance was set to zero. As of yet, we only studied AQR with constant bit rate payload; traffic splitting may be advantageous with reactive payload, but then, scalability is questionable.

5 Simulation of AQR We performed two series of simulations with the “ns” network simulator [19]. Figure 1 shows the topology which we used for the first series, the goal of which was to study the behaviour of delay based AQR in a somewhat realistic scenario. Obviously, the perfect path in terms of delay will often be the shortest path (minimal number of hops); thus, in comparison with traditional routing, AQR could only yield satisfactory results if the type and amount of background traffic caused the QoS of the shortest path to 2 In most simulations, delay increased significantly if the tolerance was higher than approx. 8% because the disadvantage of using an additional path with more delay could not be outweighed by the load reduction along the optimal path.

Figure 3. The topology used for series 2

Figure 1. The topology used for series 1

Figure 2. AQR (delay) vs. shortest path routing

degrade significantly; therefore, although we carried out a large number of simulations [13], we do not present average values but instead only some key results from simulations which we consider representative. One such result is depicted in fig. 2. We chose this scenario because it shows a significant delay reduction (approx. 20%) despite a number of path changes (peaks in the AQR line); the four visible delay levels correspond with four paths. In this simulation, the sender was at node 9, the receiver was at node 45, and the background traffic consisted of greedy TCP sources. We also tried Pareto sources to account for web traffic, but in these experiments, AQR only changed the path if we increased the amount of background traffic to a degree which we consider unrealistic. In all of our simulations, the nominal bandwidth of all links was 1.5 Mbit/s, packet sizes of all packets including measurement packets were 500 bytes,3 delay between prob3 Note that without knowledge of the EE, it is impossible to determine the correct size of measurement packets. We therefore carried out studies

ing packet “waves” was set to approximately 2 RTTs (3.5 seconds in series 1, 1.5 seconds in series 2), and we generally used a simulation time of 360 seconds — some of our graphs show a shorter period for better visibility. Only the type of background traffic and QoS specifications were varied for the simulation traces presented in this paper. Changing the type of background traffic allowed us to isolate the behavior of AQR with responsive background traffic from its behavior with unresponsive background traffic. In our earlier paper [1] we discussed already that remarkably good results in terms of delay and throughput can be obtained with delay being the only QoS parameter. For the present paper based on Active Networs, we considered the available bandwidth (as measured over the period between two probing intervals) in the QoS parameter space in addition to delay. For these simulations, we chose to use a somewhat less realistic but more controllable scenario by means of a 15-node topology, which is shown in fig. 3. Using node 5 as a sender and node 13 as a receiver, we studied the behaviour of AQR both with (greedy) TCP background traffic and exponentially distributed UDP background traffic. Figure 4 shows the delay of a constant bit rate AQR stream with TCP background traffic. Probably to the reader’s surprise, AQR based on bandwidth measurements alone not only increases the average delay but also jitter. The results from this simulation, which are shown in tables 1 and 2, may seem disappointing at first sight: the differences in throughput are small, which is probably caused by TCP’s adaptive behaviour. Delay increased by approx. 9% in comparison with shortest path routing, jitter increased by 44%. We achieved better results when the background traffic consisted of UDP packets. Here, the throughput increased by 27% in comparison with shortest path routing. On the other hand, the average delay increased by 8% and jitter increased by 87%. A closer look at the simulation traces reveals why we could not get better results with the Bandwidth parameter alone: Jitter was a result of AQR choosing very long paths. The available bandwidth on these paths may have been higher, but was not fully saturated by our constant bit with varying sizes, but the results did not change significantly.

Avg. delay (ms) Min. delay (ms) Max. delay (ms) Max. jitter (ms) Throughput (kb/s)

Bandwidth 680 353 1075 722 48.40

Delay 492 353 827 474 48.76

Shortest P. 623 353 856 502 48.46

Delay in sec. Bandwidth Delay Shortest path

1.10 1.05 1.00 0.9 0.90 0.85 0.80 0.75 0.70 0.65

Table 1. Results of the QoS Parameters “Bandwidth” and “Delay” (TCP bg. traffic)

0.60 0.55 0.50 0.45 0.40

Avg. delay (ms) Min. delay (ms) Max. delay (ms) Max. jitter (ms) Throughput (kb/s)

Bandwidth 660 353 1328 975 42.21

Delay 440 353 872 519 46.55

Shortest P. 609 353 874 521 33.33

Table 2. Results of the QoS Parameters “Bandwidth” and “Delay” (UDP bg. traffic)

0.35 4

6

8

10

12

Time in sec.

Figure 4. QoS parameters “Delay” and “Bandwidth”, TCP bg. traffic Delay in sec. Shortest path AQR- new

0.60 0.58 0.56 0.54 0.52 0.50

rate stream.4 These preliminary results lead to the conclusion that the QoS parameter “Delay” always yields better results for a constant bit rate stream, whereas switching to very long paths may cause the “Bandwidth” variant of AQR to lose. We now focus on a mixture (called “AQR-new”) of both parameters, where a delay threshold limits the choice of paths. “AQR-old” denotes AQR solely relying on delay. For TCP background traffic, AQR-new shows faster convergence to the optimum path (see figs. 5 and 6). We have seen this effect in many other tests under different circumstances and believe it to be a result of AQR finding the optimal trade-off between reducing delay by choosing a “faster” path and reducing delay by means of a traffic load (and thus, queuing delay) reduction. There was no other drastic change in the delay or throughput results (see table 3); as could be expected, the average delay was notably – 15% – smaller than the average delay of shortest path routing. The maximum jitter value depicted calls for closer consideration: this value (for both AQR-new and AQR-old) is due to an outlier resulting from the “positive” effect of AQR finding the optimal path very early in the lifetime of a data stream. Unresponsive background traffic yields a different result, which is depicted in figure 7. While the smoothing of the delay curve in comparison with Shortest Path routing is still significant, it is disturbed by regular outliers. The main advantage of AQR-new with unresponsive background traffic 4 We expect that this parameter alone could be used advantageously for adaptive streams which increase bandwidth usage until packet loss is detected; on the other hand, it is questionable whether such an approach will scale

0.48 0.46 0.44 0.42 0.40 0.38 0.36 4

6

8

10

12

Time in sec.

Figure 5. The “Bandwidth / Delay” combination with TCP bg. traffic

lies in a throughput enhancement which was as high as 33% in our simulations; this enhancement is due to a smaller packet loss ratio. The average delay was reduced by 36% (see table 4).

6 Conclusions, future and related work We have proposed AQR as an approach to combining Active Networks with QoS routing. In the variant finally proposed, AQR combines a consideration of both bandwidth and delay for finding optimal paths. This variant showed considerable improvements over shortest-path routing under various load combinations and characteristics. The nature of these improvements varies, however; this observation gives rise to further research efforts. In our further work, we also plan to consider multi-domain routing and constraints with respect to application behavior and customization in the light of scalability. Usage of reactive traffic sources instead of CBR for AQR payload also requires

Delay in sec. AQR- new AQR- old

0.60

Avg. delay (ms) Min. delay (ms) Max. delay (ms) Max. jitter (ms) Throughput (kb/s)

0.58 0.56 0.54 0.52 0.50 0.48

AQR-new 410 353 599 246 48.76

AQR-old 411 353 599 246 48.73

Short. P. 471 353 584 231 48.46

0.46

Table 3. Results of the “Bandwidth / Delay” combination (TCP bg. traffic)

0.44 0.42 0.40 0.38 0.36

6

4

8

10

12

Time in sec.

Avg. delay (ms) Min. delay (ms) Max. delay (ms) Max. jitter (ms) Throughput (kb/s)

Figure 6. AQR-new vs. AQR-old, TCP background traffic Delay in sec. Shortest path AQR- new

0.90 0.85

AQR-new 446 353 872 519 46.47

AQR-old 443 353 872 519 46.35

Short. P. 610 353 874 520 33.38

Table 4. Results of the “Bandwidth / Delay” combination (UDP bg. traffic)

0.80 0.75 0.70 0.65 0.60 0.55 0.50 0.45 0.40 0.35 0.30 140

160

180

200

220

240

Time in sec.

Figure 7. The “Bandwidth / Delay” combination with UDP background traffic

By removing the third feature, the applicability of AQR could easily be extended to ad hoc wireless networks. While there are numerous other QoS routing approaches (the most closely related approach appears to be the ondemand routing mechanism for ad hoc networks described in [30]), some of which utilize Active Networks to implement some form of distributed intelligence [23] [9], AQR appears to be the only method that combines sourceinitiated QoS routing with Active Networks to add a new degree of flexibility to QoS routing.

A AQR pseudocode further consideration, especially when combined with the tolerance parameter for traffic splitting. At first sight, the scalability of AQR may seem questionable because it floods the network with measurement packets. We would like to note that AQR shares this property with any on-demand routing mechanism for ad hoc networks (see [25], [27] and [28] for examples) and should therefore show similar properties with respect to scalability and fairness. Some advantages of source routing in a smaller environment such as a campus network are described in [29]. Moreover, its scalability is enhanced via three important features: 1. It requires no state at intermediate network nodes

This code, which is a simplified version of the ns code that was used in our simulations, uses the QoS parameter “delay”. The complete code is available from our project web site at [22]. //

globals

// routing table matrice for ‘‘visit’’ static int a[maxNodes][maxNodes]; // is the current path valid? int validPath=0; /*----------------------------------*/ /* CALLED ONCE: Initialization */ /*----------------------------------*/

2. Flooding packets are dropped when a QoS threshold is exceeded (this is only possible for arbitrary QoS parameters through active code in each flooding packet)

for(i=0; ipath = path; pkt->pathcopy = path; pkt->waveid = waveid;

// // // //

time of sending the ‘‘ssr’’ path complete copy for receiver number of packet wave

/****** The Active Code ******/ pkt->code = { if(NOW - pkt->time < max_delay) { REMOVE FIRST ENTRY FROM path; send(pkt, dest=FIRST ENTRY IN path); } else drop(pkt); send(pkt, dest=FIRST ENTRY IN path); } // //

Procedure to send a wave of probe packets altered version of visit in [31]

void visit(int k, int destination, int depth, int val[], Packet* p) { if(k == destination) { validPath=1; return; } val[k] = 1; for(int t = 0; tpath = pkt->pathcopy; replypkt->waveid = pkt->waveid; send(replypkt, dest = source of pkt); } /*--------------------------------------------*/ /* EVENT: Handle reply packet at the sender */ /*--------------------------------------------*/ recv(replypkt) { if( (replypkt->delay < smallestDelay) && (replypkt->waveid == waveid) ) { smallestDelay = replypkt->delay; COPY replypkt->path TO bestpath; } }

References [1] Michael Welzl, Alfred Cihal, and Max M¨uhlh¨auser, “Active QoS Routing”, KiVS 2001, Hamburg, Germany, 20-23 February 2001. [2] Cristina Aurrecoechea, Andrew T. Campbell, and Linda Hauw, “A Survey of QoS Architectures”, Multimedia Systems Journal, Special Issue on QoS Architecture, Vol. 6 No. 3, pg. 138-151, ACM / Springer, May 1998. [3] Yakov Rekhter, “MPLS and the Evolution of IP Routing”, talk “Business Application Session BA11”, ICC 2000, New Orleans, Louisiana, USA, June 18-22, 2000. [4] AN Node OS Working Group, Node OS Interface Specification, Edited by Larry Peterson, January 24, 2000. [5] J. Crowcroft, M. Fry, D. Hutchinson, I. Marshall, M. Sloman and I Wakeman, “Editorial: Active networks and services”, Computer Networks Vol 36, Issue 1, June 2001, Elsevier Science B. V. [6] A. Ghosh, M. Fry, and G. MacLarty, “An infrastructure for application layer active networking”, Computer Networks Vol 36, Issue 1, June 2001, Elsevier Science B. V. [7] Anand Balachandran, Andrew T. Campbell, and Michael E. Kounavis, “Active Filters: Delivering Scaled Media to Mobile Devices”, Proceedings of NOSSDAV 1997, St. Louis, Missouri, May 19-21, 1997. [8] T. Faber, “Active Congestion Control”, IEEE Network, May/June 1998, pp. 61-65 [9] Giuseppe Di Fatta, Salvatore Gaglio, Giuseppe Lo Re, and Marco Ortolani, “Adaptive Routing in Active Networks”, Proceedings of OpenArch 2000, Tel Aviv, 2000.

[10] S. Bhattacharjee, K. Calvert, and E. Zegura, “SelfOrganizing Wide-Area Network Caches”, Proceedings of Infocom 1998, San Francisco, CA, March/April 1998.

[24] Michael Welzl, “A Stateless QoS Signaling Protocol for the Internet”, Proceedings of IEEE ICPADS 2000, July 2000.

[11] Sally Floyd, “Adaptive Web Caching and Active Networking”, talk at DARPA Active Networking PI meeting, June 19-20, 1997.

[25] Sung-Ju Lee, William Su, Julian Hsu, Mario Gerla, and Rajive Bagrodia, “A Performance Comparison Study of Ad Hoc Wireless Multicast Protocols”, Proceedings of Infocom 2000, Tel Aviv, Israel, March 2000.

[12] S. Bhattacharjee, K. Calvert, and E. Zegura, “Congestion Control and Caching in CANES”, Proceedings of ICC 1998, Atlanta, GA, 1998.

[26] Seungjoon Lee and Chongkwon Kim, “A new wireless ad hoc multicast routing protocol”, Computer Networks Vol 38 (2002), pp. 121-135, Elsevier Science B. V.

[13] Alfred Cihal, Dienstg¨uteunterst¨utzung mit Active Networks, Master’s Thesis at the Johannes Kepler University of Linz, Austria, March 2001. [14] Jonathan M. Smith, Kenneth L. Calvert, Sandra L. Murphy, Hilarie K. Orman, and Larry L. Peterson, “Activating Networks: A Progress Report”, IEEE Computer, April 1999. [15] Andrew T. Campbell, Herman G. De Meer, Michael E. Kounavis, Kazuho Miki, John B. Vicente, and Daniel Villeia, “A Survey of Programmable Networks”, ACM SIGCOMM Comp. Commun. Review, April 1999 [16] Sheng-Yih Wang and Bharat Bhargava, “An Adaptable Network Architecture for Multimedia Traffic Management and Control”, IEEE International Conference on Multimedia and Expo (III) 2000, pp. 1615-1618 [17] Bob Lindell, “Active Signaling Protocol (ASP) Execution Environment”, Presentation to OPENSIG 2000 meeting, Napa, CA, October 11-12, 2000. [18] David J. Wetherall, John Guttag, and David L. Tennenhouse: “ANTS: A Toolkit for Building and Dynamically Deploying Network Protocols”, IEEE OPENARCH 1998, San Francisco, CA, April 1998. [19] S. McCanne and S. Floyd, “ns Network Simulator”, http://www.isi.edu/nsnam/ns/ [20] K. McCloghrie and M. T. Rose, Management Information Base for Network Management of TCP/IP-based internets: MIB-II, STD17, RFC 1213, March 1991. [21] G. Di Fatta, G. Lo Re, “Active Networks: the State of the Art”, CERE-CNR Tech. Rep. 2/01, March 2001. [22] http://www.tk.uni-linz.ac.at/micky/AQR.html [23] I. W. Marshall and C. Roadknight, “Provision of quality of service for active services”, Computer Networks Vol 36, Issue 1, June 2001, Elsevier Science B. V.

[27] Chunhung Richard Lin, “An On-demand QoS Routing Protocol for Mobile Ad Hoc Networks”, Proceedings of Infocom 2001, Anchorage, Alaska, April 2001. [28] Samir Das, Charles Perkins, and Elizabeth Royer, “Performance Comparison of Two On-demand Routing Protocols for Ad Hoc Networks“, Proceedings of Infocom 2000, Tel Aviv, Israel, March 2000. [29] Jerome H. Saltzer, David P. Reed, and David D. Clark, “Source Routing for Campus-wide Internet Transport”, in Local Networks for Computer Communications, Anthony West and Philippe Janson, editors, North Holland, Amsterdam, 1981, pages 1-23. An earlier version was published as Internet Engineering Note IEN-144 (March, 1980). [30] Atsushi Iwata, Ching-Chuan Chiang, Guangyu Pei, Mario Gerla, and Tsu-Wei Chen, “Scalable Routing Strategies for Ad Hoc Wireless Networks”, IEEE JSAC 1999, August 1999. [31] Robert Sedgewick, Algorithmen, Addison-Wessley (Germany) GmbH, 2. corrected edition, 1995.