To bootstrap the system, during an initialisation phase the ants explore G with a uniform random proportional rule psd ij,t = 1/Ni ; âi where Ni is the number of ...
Distributed and Scalable Path Management by a System of Cooperating Ants 1
Vebjørn Kjeldsen 1 , Otto J Wittner 2 , Poul E Heegaard 1 Department of Telematics, Norwegian University of Science and Technology. 2 Centre for Quantifiable Quality of Service in Communication Systems, Norwegian University of Science and Technology.
Abstract - Path management in next generation networks will be computationally excessive if guarantied quality of service is to be offered. Route computations subject to many and changing requirements set forth by a range of applications is far from straight forward when applying state-of-the-art routing systems. Robust and adaptive swarm based systems are candidates to handle future path management challenges. One such system, the Cross Entropy Ants System (CEAS), provides stochastic, asynchronous and truly distributed path management. In most management systems there is a trade-off between performance and management overhead. This paper presents an improved version of CEAS, denoted Subpath CEAS. Significant savings are observed in memory usage, and in the number of control packets generated, without loss of performance or added processing. To achieve this, end-toend paths with the same destination and QoS requirements are made to share and exploit information about their common sub-paths. Keywords: CE-ants, CEAS, Path Management, Routing, Swarm Intelligence, Cross Entropy Method
1. Introduction In order to maximise resource utilisation in the next generation multi-service Internet, and at the same time provide quality of service guarantees, management of paths between peering nodes is required. Paths from source to destination nodes should be chosen to ensure an overall good utilisation of the network resources, and at the same time provide services with high throughput, low loss and low latency. The available spare capacity must be utilised such that a failure results in a minimum disturbance in traffic flows. Hence, proper path management requires that: a) the set of operational paths should be continuously updated as the traffic load changes, b) new paths should become almost immediately available between communicating nodes when established paths are affected by failures, and c) new or repaired network elements should be put into operation without unnecessary delays. Near immediate and robust failure handling advocates distributed local decision-making on how to deal with failures, though truly distributed decision-making typically yields solutions that
are less fine tuned with respect to optimal resource utilisation. The combinatorial optimisation aspects of path management are typically NP-hard; see for instance [1]. Nevertheless, considerable knowledge has been acquired for planning paths in networks, and insight and practical methods for obtaining such paths by mathematical programming are available. For an overview, see Pióro and Medhi [2] and references therein. Several stochastic optimisation techniques have been proposed [3], [4], [5], [6]. They deal with path finding as an optimisation problem where the “solution engine” has a global overview of the problem and they assume that the problem is unchanged until a solution is found. Hence these techniques are applied off-line and provide management only on longer time-scales (minutes-hours). Even so off-line techniques combined with protocols like (G)MPLS, RSVP, OSPF, ISIS and BGP is how path management in the Internet is realised today. Thus robust path management applying online, fully automated, distributed and adaptive techniques rather than complex manual operations, is not yet the reality. Dorigo & al. [7] introduced a multi-agent system, today known as Ant Colony Optimisation (ACO), with a behaviour inspired by ants to solve combinatorial optimisation problems. In Schoonderwoerd & al. [8] a similar system is designed that addresses load sharing in telecommunication networks. These systems are referred to as swarm intelligence systems [9], and have been studied further by many researchers, with Dorigo and Di Caro’s AntNet system [10] being one of the most cited among swarm intelligence systems for path management in telecommunications. Self-management by swarm intelligence is a candidate to meet the aforementioned requirements for proper path management and to overcome some of the drawbacks of the current path and fault management strategies. In [11] Helvik and Wittner introduce a cross-entropy based ant system for path management in communication networks. The system is formally founded, is fully distributed, asynchronous and adaptive, and is even able to find near optimal solutions to problems with NP-hard complexity. The system is today known as the Cross-Entropy Ant System (CEAS). In [12] Heegaard, Wittner, and Helvik show how CEAS may perform adaptive multi-path load sharing and stochastic routing with fast restoration on link failures. In [13] CEAS
is shown to perform better than an ACO system in a multiconstraints QoS routing context. In all ant systems there is a trade-off between management overhead (number of management packets) and performance. Improvements to CEAS have been made to reduce overhead without sacrificing performance. In [14] elitism is introduced and in [15] self-tuned packet rate control is applied. This paper presents a new novel version of CEAS, denoted Subpath CEAS, which significantly reduces the required amount of management traffic by an approach totally different from what is described in [14], [15]. The approach has great potential in large-scale scenarios since signaling information is shared among end-to-end paths. The rest of the paper is organised as follows. Section 2 presents the foundations for CEAS as well as the latest enhancements. Section 3 presents the new version of CEAS, Subpath CEAS. Section 4 describes two case studies and presents results showing how Subpath CEAS enables reduced management overhead. And finally, Section 5 presents some concluding remarks and indications of future work.
2. The Cross Entropy Ant System The Cross Entropy Ant System (CEAS) was first introduced in [11]. CEAS is designed for robust and adaptive path management in communication networks. CEAS is a swarm intelligence system inspired by the foraging behaviour of ants. It is a multi-agent system similar to the Ant Colony Optimization (ACO) system [7], where agents, denoted (artificial) ants, search iteratively for paths in a network. A stream of ants is generated in a source node, and each ant has a two phase life cycle. First, a forward ant searches for a path between the source node and a destination. At each node visited during the search, the ant’s next hop is decided by applying a random proportional rule. Path quality is evaluated on arrival at the destination node. Second, a backward ant backtracks along the reverse path, back to the source node, leaving pheromones (messages) at nodes to guide future ants in their search for the same destination. The better the path, the stronger the pheromone updates. Also whenever a pheromone value is updated in a node, the rest of the pheromone values in the same node is reduced by a factor. This is known as pheromone evaporation. Note that the formal expressions in the following subsections are indexed such that rewrite or reprint is unnecessary in Section 3.
2.1. Formal Foundations
from node i, i.e. which link (i, j) to traverse, according to a random proportional rule psd ij,t = !
(1)
sd ! τil,t (i,l)∈E
sd ! for is the pheromone value of link (i, j) ∈ E where τij,t paths between s and d, and t is “time” in terms of number sd of updates on τij ; ∀ij since system start-up. The random sd proportional rule matrix is denoted p!sd ! #×#V ! #. t = [pij,t ]#V To bootstrap the system, during an initialisation phase the ! with a uniform random proportional rule ants explore G sd pij,t = 1/Ni ; ∀i where Ni is the number of neighbours to node i. The random proportional rule in (1) is equivalent to what is applied by most systems base on Ant Colony Optimization (ACO) [16]. However, in the case of CEAS (1) stems from the formal foundations inherited from the Cross Entropy (CE) method, first introduced by Rubinstein in [6]. Rubinstein’s basic notion is that finding the best path between a source s and a destination d by a ! has a very low probability when random search in G the path sample space is large, i.e. it is a rare event. Hence the CE method applies an importance sampling technique [17] where the set of random proportional rules sd p!sd t = pij,t ; ∀ij are gradually changed according to the importance (e.g. the cost) of the various paths found. The objective is to minimise the cross entropy between the random proportional rule sets p!sd !sd t , subject to t−1 and p sd ! some path cost history Lt . In the CE method, “time” t represents iterations. For each iteration m sample paths are found by applying p!sd t−1 . This outputs the cost vector, sd ! sd denoted cost history, L ω1sd ), · · · , L(! ωm )}. The t = {L(! sd minimum value of a control parameter γt , denoted the temperature, is then chosen such that when the performance function ! sd /γ sd
sd −Lt ! sd H(L t , γt ) = e
t
(2)
is applied to all m samples, the expected overall performance satisfies sd ! sd sd h(! psd !t−1 (H(Lt , γt )) =≥ ρ t−1 , γt ) = Ep
(3)
Ep!t−1 (X) is the expected value of X s.t. the rules in p!t−1 , and ρ is a parameter (denoted search focus) close to 0 (typically 0.05 or less). Finally, a new updated set of rules, p!t , is determined by minimising the cross entropy sd ! sd between p!t−1 and p!t with respect to γtsd and H(L t , γt ). Minimised cross entropy is achieved by applying the random proportional rule in (1) for ∀ij with sd τij,t =
m "
k=1
Ants are sampling paths between node s and d in a ! = (V ! , E) ! where V ! is the set of nodes connected graph, G ! (vertices) and E is the set of links (edges). A path sample is denoted ω ! sd = {(s, i1 ), (i1 , i2 ), · · · , (ik−1 , d)}, where ! (i, j) ∈ E denotes the link connecting node i and j, and k is the number of hops in path ω ! sd . When searching for a path between s and d, ants sample their next hop to travel
sd τij,t
I((i, j) ∈ ω ! ksd )H(L(! ωksd ), γtsd )
where I(x) = 1 if x is true, 0 otherwise. See [6] for further details and proof. To avoid centralised control and synchronised batch oriented iterations, in CEAS a path cost value L(! ωtsd ) sd is calculated immediately after a sample path ω ! t is found, i.e. as soon as a single forward ant arrives at its destination. Hence “time” t, the total number of
updates, now represents the total number of backward ants returned. To achieve such a per-path operation, instead of (3) CEAS applies an auto-regressive overall performance function ht (γtsd ) = βht−1 (γtsd ) + (1 − β)H(L(! ωtsd ), γtsd ) approximated by ht (γtsd ) ≈
1−β 1 − βt
t "
β t−i H(L(! ωisd ), γtsd )
(4)
t 1 − β " t−i | β H(L(! ωisd ), γ sd ) = ρ} (5) 1 − β t i=1
However (5) is a complicated (transcendental) function that is both storage and processing intensive since all observations up to the current path sample, i.e. the entire ! sd path cost history L ω1sd ), · · · , L(! ωtsd )}, must be t = {L(! stored, and weights for all observations recalculate. In an on-line operation of a network node, such resource requirements are impractical. Instead it is assumed, given a β close to 1, that the changes in γtsd are typically small from one iteration to the next. This enables a (first order) Taylor expansion of (5), providing γtsd ←
bsd ωtsd )e t−1 +L(!
sd )/γ sd −L(! ωt t−1
sd )/γ sd L(! ω sd ) −L(! ωt t−1 +asd −ρ 1−βt (1+ sdt )e t−1 1−β γ t−1
(6)
sd sd where asd ωtsd )/ ln ρ, and 0 = b0 = 0 and γ0 = −L(! sd asd t ← β(at−1 + (1 +
bsd t
←
β(bsd t−1
+
L(! ωtsd ) − )e γtsd
sd ) L(! ωt γtsd
sd ) L(! ωt − γtsd
L(! ωtsd )e
)
(7)
)
where the performance function in (2) is adopted. Similar to the overall performance function (4), in CEAS the pheromone values are also made to be a ! sd function of the entire history of path cost values, L t = sd sd ωt )}. Pheromone values, and hence the {L(! ω1 ), · · · , L(! corresponding random proportional rules, are also updated for every path sample made, i.e. for every arrival of a forward ant to its destination a backward ant starts to traverse the reverse path and perform updates. Updates are made by applying the performance function in (2) based on the last cost value L(! ωtsd ) and the temperature sd γt calculated by applying (6). Pheromones are updated by sd τij,t =
t "
k=1
sd τij,t
≈ I((i, j) ∈
− ω ! tsd )e
sd ) L(! ωt γtsd
+ Asd ij +
where
i=1
where β ∈< 0, 1 > is a memory factor weighting (geometrically) the output of the performance functions. As for the CE method, the temperature γtsd is determined by minimising it subject to h(γ) ≥ ρ. In [11] it is shown that the temperature equals γtsd = {γ sd
However, as for (5), excessive processing and storage requirements also apply for (8). Hence a (second order) Taylor expansion of (8) is appropriate, giving
I((i, j) ∈ ω ! ksd )β
Pt
x=k+1
sd I((i,·)∈! ωx )
H(L(! ωksd ), γtsd ) (8)
The memory factor β weights (geometrically) the output of the performance functions, i.e. implements evaporation. The exponent of β is the number of ants that has updated node i at “time” t since “time” k when a path ω ! ksd was found.
−
sd Asd ! tsd )e ij ← β(Aij + I((i, j) ∈ ω
B sd ij − sd + γ t
−
sd ) L(! ωt γtsd
sd ) L(! ωt − γtsd
sd sd ← β(Bij + I((i, j) ∈ ω ! tsd )e Bij
−
sd sd Cij ← β(Cij + I((i, j) ∈ ω ! tsd )e
sd ) L(! ωt γtsd
sd 2 (Bij ) sd 4Cij
(1 +
sd Cij (γtsd )2
(
