Multi-Colony Ant Algorithm for Continuous Multi ... - Semantic Scholar

Water Resour Manage (2007) 21:1429–1447 DOI 10.1007/s11269-006-9092-5

Multi-Colony Ant Algorithm for Continuous Multi-Reservoir Operation Optimization Problem M. R. Jalali & A. Afshar & M. A. Mariño

Received: 30 December 2005 / Accepted: 31 August 2006 / Published online: 25 November 2006 # Springer Science + Business Media B.V. 2006

Abstract Ant Colony Optimization (ACO) algorithms are basically developed for discrete optimization and hence their application to continuous optimization problems require the transformation of a continuous search space to a discrete one by discretization of the continuous decision variables. Thus, the allowable continuous range of decision variables is usually discretized into a discrete set of allowable values and a search is then conducted over the resulting discrete search space for the optimum solution. Due to the discretization of the search space on the decision variable, the performance of the ACO algorithms in continuous problems is poor. In this paper a special version of multi-colony algorithm is proposed which helps to generate a non-homogeneous and more or less random mesh in entire search space to minimize the possibility of loosing global optimum domain. The proposed multi-colony algorithm presents a new scheme which is quite different from those used in multi criteria and multi objective problems and parallelization schemes. The proposed algorithm can efficiently handle the combination of discrete and continuous decision variables. To investigate the performance of the proposed algorithm, the wellknown multimodal, continuous, nonseparable, nonlinear, and illegal (CNNI) Fletcher– Powell function and complex 10-reservoir problem operation optimization have been considered. It is concluded that the proposed algorithm provides promising and comparable solutions with known global optimum results. Key words ant colony . optimization . multi-colony . multi-reservoir M. R. Jalali (*) Iran University of Science and Technology (IUST) and Mahab Ghodss Consulting Engrs., Tehran, Iran e-mail: [email protected] A. Afshar Department of Civil Engineering and Center of Excellence for Fundamental Studies in Structural Mechanics, Iran University of Science and Technology, Tehran, Iran e-mail: [email protected] M. A. Mariño Hydrology Program and Department of Civil and Environmental Engineering, University of California, Davis, CA 95616-8628, USA e-mail: [email protected]

1430

Water Resour Manage (2007) 21:1429–1447

The following symbols are used in this paper: BFi C D G Gkgb Ii L NC NR NT Ri Si Smax Smin !, β ηij ρ t0 tij(t) ðn Þ bi kgb Pij(k,t) q q0

benefit function for cell or element i in a trial solution string set of costs associated with the options {cij} set of decision points {di} Graph (D,L,C) value of the objective function for the ant with the best performance within the past total iterations inflow to the reservoir at time period i set of options {lij} number of release intervals (or classes) number of reservoirs number of time periods release at time period i reservoir storage volume at time period i maximum storage allowed minimum storage allowed parameters that control the relative importance of the pheromone trail versus a heuristic value heuristic value representing the cost of choosing option j at decision point i pheromone evaporation coefficient initial value of pheromone concentration of pheromone on arc (i,j) at iteration t benefit coefficients of reservoir n at time period i ant with the best performance within the past total iterations probability that ant k selects option lij for decision point i at iteration t random variable uniformly distributed over [0, 1] tunable parameter ∈ [0, 1]

1 Introduction Ant colony optimization algorithms (ACO) have been successfully applied to various combinatorial optimization problems. As heuristic algorithm, ACO’s were first proposed by Dorigo (1992) and Dorigo et al. (1996) as a multi-agent approach to different combinatorial optimization problems including the traveling salesman and the quadratic assignment problem. Later Dorigo and Di Caro (1999) introduced a general ant colony optimization algorithm, namely ant colony metaheuristic, which enabled the algorithm to be applied to other engineering problems provided that the problem can be properly formulated. Recently, Dorigo et al. (2000) reported the successful application of ACO algorithms to a number of benchmark combinatorial optimization problems. Successful application to the Quadratic Assignment problem (e.g. Maniezzo and Colorni 1999; Gambardella et al. 1999a), the Shortest Common Supersequence problem (Michels and Middendorf 1999), the Job Shop Scheduling problem (Colorni et al. 1994), or the Resource-Constraint Project


1431

Scheduling problem (Merkle et al. 2000) have also been reported. Application of ACO algorithms to water resources problems, however, is of quite recent origin. Abbaspour et al. (2001) employed ACO algorithms to estimate hydraulic parameters of unsaturated soils. Maier et al. (2003) used ACO algorithms to find a near global optimal solution to a water distribution system, indicating that ACO algorithms may form an attractive alternative to genetic algorithms for optimal design of water distribution systems. Zecchin et al. (2003) compared the performance of the original ant system (Dorigo et al. 1996) with that of a min–max ant system, a modified version of the ant system proposed by Stützle and Hoos (1997a, 1997b), for optimization of water distribution networks. Simpson et al. (2001) discussed the selection of parameters employed in ant algorithms for optimizing pipe network systems. Jalali et al. (2006) employed ACO algorithms to solve the problem of optimal reservoir operation. Also, Jalali et al. (2005a) introduced an improved version of the ACO algorithm in single reservoir operation optimization. They included explorer ants, and a local search technique in a standard ACO algorithm. Linking a transient flow simulation model in a pressure system with an ACO algorithm, Abbasi (2005) developed an optimization model for water supply pipe design considering transient flow. Optimum operation of multi-reservoir systems has received much attention during the last three decades. Labadie (2004) presented a state-of-the-art review of the optimal operation of multi-reservoir systems with mathematical and heuristic optimization algorithms. He discussed some applications of genetic algorithms, artificial neural networks, and fuzzy-based approach to the multi-reservoir optimization problem; however, ACO algorithms were not included in the review. To partially overcome the dimensionality problem in dynamic programming, evolutionary algorithms have been used. There have been several applications of genetic algorithms (GAs) to multi-reservoir operation problems (Esat and Hall 1994; Fahmy et al. 1994; Oliveira and Loucks 1997). Esat and Hall (1994) clearly demonstrated the advantages of GAs over standard dynamic programming techniques in terms of computational requirements. Recently, Wardlaw and Sharif (1999) applied GAs to four-reservoir system operation, concluding that algorithm with real value coding performs significantly faster than the one that employs binary coding. They extended the formulation to a more complex 10-reservoir problem. Being at its early stages of development, Honey Bees Mating Optimization (HBMO) metaheuristic algorithm was applied to a single reservoir operation problem with promising results (Bozorg Haddad and Afshar 2004). A suitable approach for coarse grained parallelization is a multi-colony ant algorithm in which every processor holds a colony of ants (Middendorf et al. 2002). In a multi-colony ant system, after each generation, the colonies may exchange information about their solutions. After information exchange, the new pheromone information may be stored in a pheromone matrix. Limited work has been reported so far where the colonies exchange information in order to give the colonies a chance to evolve different pheromone matrices and thus search in different regions of the search space (Middendorf et al. 2002; Calégari 1999). ACO’s are basically developed for discrete optimization and hence their application to continuous optimization problems require the transformation of a continuous search space to a discrete one by discretization of the continuous decision variables. Thus, the allowable continuous range of decision variables is usually discretized into a discrete set of allowable values and a search is then conducted over the resulting discrete search space for the optimum solution (Abbaspour et al. 2001). Up to now, only a few ant approaches for continuous optimization have been proposed in the literature. The first method – called Continuous ACO (CACO) – was proposed by Bilchev and Parmee in 1995, and also later used by some others (Wodrich and Bilchev 1997 and

1432


Mathur et al. 2000). Other methods include the API algorithm by Monmarche et al. (2000), and Continuous Interacting Ant Colony (CIAC), proposed by Dreo and Siarry (2002). Although both CACO and CIAC claim to draw inspiration from the ACO metaheuristic, they do not follow it closely. All the algorithms add some additional mechanisms (e.g. direct communication – CIAC and API – or nest – CACO) that do not exist in regular ACO. They also disregard some other mechanisms that are otherwise characteristic of ACO (e.g. stigmergy – API – or incremental construction of solutions – all of them). CACO and CIAC are dedicated strictly to continuous optimization, while API may also be used for discrete problems. In this paper, a multi-colony scheme is incorporated into the original ACO algorithms to address the optimum operation of a multi-reservoir system with continuous search space, as well as investigating their merits in solving continuous-nonseparable, nonlinear, and illegal (CNNI) optimization problems. The proposed multi-colony scheme is successfully applied to a 10-reservoir system with continuous search space and the well-known Fletcher–Powell function as a CNNI optimization problem.

2 ACO Algorithms: General Aspects The ACO mimics the natural behavior of a colony of ants. When ants are traveling, they deposit a substance called pheromone, forming a pheromone trail as an indirect means of communication. As more ants choose a path to follow, the pheromone on the path builds up, making it more attractive for other ants to follow. In the ACO algorithm, artificial ants are permitted to release pheromone while developing a solution or after a solution has been fully developed, or both. As stated, the amount of pheromone deposited is made proportional to the goodness of the solution an artificial ant develops. Rapid drift of all ants toward the same part of the search space is avoided by including a stochastic component in the choice decision policy and by means of numerous mechanisms such as pheromone evaporation, explorer ants, and local search. For the successful application of ACO algorithms to combinatorial optimization problems, one must project the problem on a graph. Consider a graph G=(D,L,C), in which D={di} is the set of decision points at which some decisions are to be made, L={lij} is the set of options j=1...NC, at each decision point i=1...NT, and C={cij} is the set of costs associated with option L={lij}. A feasible path on the graph is called a solution (8)k and the path with minimum cost is called the optimum solution (8*)k. The transition rule used in the original ant system is defined as follows (Dorigo et al. 1996): 8 ! h iβ > > C ð t Þ ηij > ij > > < J if j 2 allowed k ! h iβ P ð1Þ Pij ðk; t Þ ¼ C ð t Þ η ij ij > > j¼1 > > > : 0 otherwise where Pij(k,t) is the probability that ant k selects option lij for decision point i at iteration t; Cij(t) is the concentration of pheromone on arc (i,j) at iteration t; ηij =1/cij is the heuristic value representing the cost of choosing option j at decision point i; and ! and β are two parameters that control the relative importance of the pheromone trail and heuristic value. The heuristic value ηij is analogous to providing the ants with sight and is sometimes called visibility. This value, in static problems, is calculated once at the start of the algorithm and is not changed during the computation.


1433

Let q be a random variable uniformly distributed over [0,1], and q0 ∈ [0,1] be a tunable parameter. The next node j that ant k chooses to go is (Dorigo and Gambardella 1997): 8 n o < arg max ½Cil ðt Þ! ½ηil β if q q0 l2allowedk ð2Þ j¼ : J otherwise where J is a value of a random variable selected according to the probability distribution of Pij(k,t) [Equation (1)]. Equations (1) and (2) provide a probabilistic decision policy to be used by the ants to direct their search towards the optimal regions of the search space. The level of stochasticity in the policy and the strength of the updates in the pheromone trail determine the balance between the exploration of new points in the state-space and the exploitation of accumulated knowledge (Dorigo and Gambardella 1997). To simulate pheromone evaporation, the pheromone evaporation coefficient (ρ) is defined which enables greater exploration of the search space and minimizes the chance of premature convergence to sub-optimal solutions upon completion of a tour by all ants in the colony. The global trail updating is done as follows: Cij ðt þ 1Þ ¼ ð1 ρÞ:Cij ðtÞ þ ρ:ΔCij ðtÞ

ð3Þ

where Cij(t+1) is the amount of pheromone trail on option j of the ith decision point at iteration t+1; 0≤ρ≤1 is the coefficient representing the pheromone evaporation and ΔCij(t) is the change in pheromone concentration associated with arc (i,j) at iteration t. The amount of pheromone Cij(t) associated with arc (i,j) is intended to represent the learned desirability of choosing option j when at decision point i. Various methods have been suggested for calculating the pheromone changes. The method used here was originally suggested by Dorigo and Gambardella (1997) in which only the ant which produced the globally best (gb) solution from the beginning of the trail is allowed to contribute to pheromone change: 8 k gb < 1=G if ði; jÞ 2 tour done by ant kgb ð4Þ ΔCij ðt Þ ¼ :0 otherwise

where Gkgb is value of the objective function for ant kgb , which is the ant with the best performance within the past total iterations. Other methods of pheromone change and update may be found elsewhere (Jalali et al. 2006). Multi colony approach is a good candidate for parallelization in which several colonies of ants cooperate to find good solutions. Two different approaches have been studied in the literature, namely heterogeneous and homogeneous. The heterogeneous approach is to have colonies of ants with a behavior that differs between the colonies. This approach has been used for multi criteria optimization problems where the colonies work with different optimization criteria (see e.g. Gambardella et al. 1999b). In homogeneous approach all the ants show a similar behavior (see e.g. Michels and Middendorf 1999), and information is exchanged between the colonies after several iterations. Bolondi and Bondaza (1993) implemented a very fine-grained parallelization in which every processor holds only a single ant. Better results have been obtained with a coarser grained variant by Dorigo (2003, unpublished manuscript). Bullnheimer et al. (1998) proposed a parallelization where an information exchange between contributing ant colonies was done every k iteration(s) for some fixed k. Their simulations show how much the running time of the algorithm decreases with an increasing interval between the information exchanges. However it was

1434


not discussed how it may influence the solution quality. A parallel ant algorithm for the Quadratic Assignment problem was implemented by Talbi et al. (1999). A fine-grained master-worker approach was used, where every worker holds a single ant that produces one solution. Every worker then sends its solution to the master. The master computes the new pheromone matrix and sends it to the workers. Stützle (1998) compared the solution quality obtained by several independent short runs of an ant algorithm with the solution quality of one long run whose running time equals the total running times of the short runs. Under some conditions the short runs proved to give better results. Also, they have the advantage that they can run in parallel and also it is possible to use different parameter values for the runs. In Michels and Middendorf (1999) model every processor holds one colony of ants where after every fixed number of iterations and the locally best solution are exchanged between the colonies. If after exchange process, a colony receives a new solution better than the its own best solution, the received solution becomes the new best found solution for this colony. This solution influences the colony because an elitist strategy is applied where during trail update some pheromone is always put on the trail that corresponds to the best found solution. Krüger et al. (1998, unpublished manuscript) showed that for the traveling salesman problem (TSP) it is better to exchange only the best solutions found so far than to exchange whole pheromone matrices and add the received matrices multiplied by some small factor to the local pheromone matrix. Different information exchange strategies between the colonies have been studied by Middendorf et al. (2000, 2002). These strategies were defined as: (a)

Exchange of globally best solution: The globally best solution is sent to all colonies where it becomes the new locally best solution. (b) Circular exchange of locally best solutions: A virtual neighborhood is established between the colonies so that they form a directed ring. Every colony sends its locally best solution to its successor colony in the ring. The variable that stores the best found solution is updated accordingly. (c) Circular exchange of migrants: As in (b) the processors form a virtual directed ring. In an information exchange step every colony compares its mb best ants with the mb best ants of its successor colony in the ring. The mb best of these 2mb ants are then used to update the pheromone matrix. (d) Circular exchange of locally best solutions plus migrants: Combination of strategies (b) and (c).

3 Proposed Multi-Colony Approach Most of the previous researches on utilization of multi-colony system have concentrated on information exchange and updating pheromone matrices in a parallel system. This paper benefits from the inherent potential in multi-colony ant system to approach a continuous optimization problem. In fact, utilization of a multi-colony system with heterogeneous discretization scheme with possibly of information exchange between the colonies help to provide a non-homogeneous and dynamic discretization scheme in the search space. The main objective is then, to benefit from multi-colony approach, to develop a scheme to search for real solution in a continuous search space with a non-homogenous discretized scheme. As stated by Jalali et al. (2005b), in small scale problems, discrete-refining (DR) approach may result in near optimum solution in continuous search space. This was in fact


1435

Decision

Decision

illustrated with few mathematical problems as well as a real world single reservoir operation problem. However, in complex real world problems with large continuous search space, it is very often possible to loose the global optimum domain in the early stages of DR process forever. In the above mentioned single reservoir problem, due to predefined target release, the probability of such an outcome was quite low, hence the results were satisfactory. To overcome this problem, a special version of multi-colony algorithm is proposed which helps to generate a non-homogeneous and more or less random mesh in entire search space to minimize the possibility of loosing global optimum domain when DR mechanism is applied. To illustrate the process of information exchange in the proposed multi-colony algorithm, Figure 1 is presented for a three-colony system. As is clear, each colony has its own discretization scheme. When information exchange condition is attained, the best solution of each colony is determined, and for any decision point, new search space is developed which is bounded by the minimum and maximum values of the decision variable from the best solution of the colonies. Therefore, a new search space will form which is smaller and may contain the global optimal. The new search space is then discretized considering the different discretization pattern of each colony. After any colony information exchange process, the whole decision space must be re-discretized. To save the best available solution, in each decision stage, one of the points in new discretization scheme must coincide with that of the best available solution. Therefore, while generating a more or less random and non-homogeneous discretization scheme, it is guaranteed that, in each information exchange stage, the best solution will surely be transformed to the next stage. The proposed multi-colony algorithm presents a new scheme which is quite different from those used in multi criteria problems and parallelization schemes. In most of the

State

State Colony 1

State

t lu So

n io

Decision

st Be

Decision

Decision

Colony 1

State

State

Colony 2

Colony 2

Decision

Decision

Information Exchange

State

State Colony 3

Figure 1 Schematic of information exchange in proposed multi-colony ACO algorithm.

Colony 3

1436


available multi-colony algorithms, in the stage of information exchange, in addition to exchange of the best solution between the colonies, pheromone matrices are also updated. In the proposed scheme, which is the most suitable for solution to continuous domain, due to the limited stages of information exchange, pheromone matrix will be re-initiated for all colonies after any information exchange. The proposed algorithm can efficiently handle the combination of discrete and continuous decision variables. For continuous decision variables, the DR mechanism is employed whereas for discrete decision variables, it is disregarded. To investigate the performance of the proposed algorithm, the well-known multimodal, continuous, nonseparable, nonlinear, and illegal (CNNI) Fletcher–Powell function was selected as: f ð xÞ ¼

n X

ðAi Bi Þ2

i¼1

Ai ¼

n X

aij sin a j þ bij cos aj

j¼1

Bi ¼

n X

aij sin xj þ bij cos xj

j¼1

: xj :ði ¼ 1; 2; :::; nÞ Thirty-dimensional Fletcher–Powell function has previously been tested with different algorithms (Bäck 1996). As one may note, the minimum value of the function is zero where xj =!j for all values of j (Figure 5). Jalali et al. (2005b) approached this function using ACO with pheromone re-initiation (PRI), partial path replacement (PPR), discrete refining (DR) mechanism, and very powerful nonlinear programming (NLP) solver (i.e. Lingo software from Lindo corporation). The best solution found with ACO and classic NLP solver was 21,550 and 89,993.2, respectively. In this paper three colonies were employed. Specifications of the colonies are presented in Table I. Flow diagram of the developed scheme including PRI, PPR, DR, and information exchange between different colonies is presented in Figure 2. As is clear from Figure 2, combination of previously developed and implemented improving mechanisms are also included in the new multi-colony algorithm. Variable information exchange steps were used. The first exchange was scheduled after 10th PRI, therefore, information exchange steps were reduced by three. High rate of convergence during the initial periods of computation is the main reason for more frequent employment of PRI mechanism and variable information exchange steps. Statistical measures of the results obtained for Fletcher–Powell function are presented in Table II. Convergence pattern for the three colonies are presented in Figures 3 and 4. The best result obtained is 9,943.55 which is 117% better than Jalali’s previous result (Jalali et al. 2005b). To give an indication of the desirability of the results, values of !j and xj are plotted in Figure 5, where xj =!j being the global optimal.

Table I Specifications of colonies for Fletcher–Powell function

Colony no. No. of ants No. of classes

1 75 6

2 100 11

3 100 20


1437

Pheromone initiation and Heuristic value computation for each decision at each decision point for all colonies

Building the string of decisions by all of the ants of colony

PPR mechanism implementation (some new ant generation)

Colony

Iteration

Fitness value determination for all of the ants of colony

Selecting the best solution (ant) of colony

Updating pheromone of each decision at each decision point of colony

PRI condition ?

Yes

Pheromone re-initiation for each decision at each decision point of colony

No

DR condition ?

Yes

Discrete refining of search space for each decision point of colony

No No

All colonies are covered ? Yes Information exchange?

Yes

Selecting best solution, Discrete refining of search space for each decision point and Pheromone reinitiation for all colonies

No No

End condition? Yes End

Figure 2 Flowchart of proposed multi-colony ACO algorithm.

1438


Table II Statistical parameters of the solutions to Fletcher– Powell function by the proposed multi-colony ACO algorithm (averaged over 15 runs)

Parameter

Mean The best The worst SD

Colony 1

2

3

30,103.99 9,943.55 57,629.16 13,246.98

28,999.91 9,944.80 56,970.92 13,411.36

28,874.90 9,943.55 56,675.63 13,197.46

4 Application to a 10-Reservoir Operation Problem Consider the following 10-reservoir system (Figure 6) which was first introduced by Murray and Yakowitz (1979), and most recently was solved by Wardlaw and Sharif (1999) employing GA with real coding. This problem is complicated not only in terms of size, but also because of many time dependent constraints on storage. The problem was formulated such that it remained solvable by linear programming, and Murray and Yakowitz (1979) presented a solution using constrained differential dynamic programming. The problem is beyond the capacity of traditional DP and is difficult with variants such as DDDP, but is relatively simple to solve by linear programming (LP). The system comprises reservoirs in series and in parallel, and a reservoir may receive supplies from one or more upstream reservoirs. Operation of the system is to be optimized over 12 operating periods to maximize hydropower production. Decision variables for the problem are reservoir releases in each operating period (decision point). Inflows are defined for each of the most upstream reservoirs, and initial storage and target storages at the end of the operating period are specified for each reservoir. In addition, there are minimum operating

500000 450000 400000 Colony 1: 6 Classes

Objective Value

350000

Colony 2: 11 Classes

300000


250000 200000 150000 100000 50000 0 0

500

1000

1500

2000

2500

3000

Iteration

Figure 3 Fletcher–Powell function value evaluation by proposed multi-colony ACO algorithm (averaged over 15 runs).


1439

Objective Value

500000 450000

Colony 1: 6 Classes

400000


350000


300000 Information Exchange Steps 250000 200000 150000 100000 50000 0 0

500

1000

1500

2000

2500

3000

Iteration

Figure 4 Fletcher–Powell function value evaluation by proposed multi-colony ACO algorithm (best of 15 runs).

storages in each reservoir that must be satisfied, as well as constraints on minimum and maximum reservoir releases. Details are given by Murray and Yakowitz (1979). The continuity constraints for each reservoir over each operating period i are: Siþ1 ¼ Si þ Ii þ M:Ri

8i

ð5Þ

4

Xj 3 2 1 0 -4

αj

-3

-2

-1

0

1

-1

2

3

αj

4

-2 -3

Xj -4 Global optimum

Best of proposed ACO

Figure 5 Decision variables of the best result of the Fletcher–Powell function optimization by proposed multi-colony ACO.

1440


3

2 R3

1 R2

5

6

8

R5 4

R1

R6 R8

R4

7

9

R9

R7

10

R10 Figure 6 Schematic of 10-reservoir problem.

where Si is the vector of reservoir storages at time i in NR reservoirs; Ii is the vector of reservoir inflows in time period i to NR reservoirs; Ri is the vector of reservoir releases in time period i from NR reservoirs; and M is a NR×NR matrix of indices of reservoir connections: 3 2 1 0 0 0 0 0 0 0 0 0 7 6 6 0 1 0 0 0 0 0 0 0 0 7 7 6 6 0 0 1 0 0 0 0 0 0 0 7 7 6 7 6 6 0 1 1 1 0 0 0 0 0 0 7 7 6 6 0 0 0 0 1 0 0 0 0 0 7 7 6 7 6 6 0 0 0 0 0 1 0 0 0 0 7 7 6 6 1 0 0 1 1 1 1 0 0 0 7 7 6 7 6 6 0 0 0 0 0 0 0 1 0 0 7 7 6 6 0 0 0 0 0 0 0 1 1 0 7 5 4 0 0 0 0 0 0 1 0 1 1


1441

1200 1190

Mean

Function Value

1180 1170 1160 1150 1140 1130 1120 1110 1100 1

2

3

4

5

No. of Colonies

(a) Average 1200 1190

The Best

Function Value

1180 1170 1160 1150 1140 1130 1120 1110 1100 1

2

3

4

5

4

5

No. of Colonies

(b) The best 0.012

Coefficient of Variation

Function Value

0.010 0.008 0.006 0.004 0.002 0.000 1

2

3 No. of Colonies

(c) Coefficient of variation Figure 7 Statistical measures of 10-reservoir operation problem versus number of colonies.

1442


Table III Statistical parameters of the solutions to 10-reservoir problem with proposed multicolony ACO algorithm (averaged over 10 runs)

Parameter

Colony

Mean The best The worst SD

1

2

3

1,185.18 1,192.09 1,180.88 3.60

1,185.18 1,192.39 1,180.73 3.63

1,185.22 1,192.30 1,181.12 3.60

The objective function to be maximized can be written as: Max

I¼

10 X 12 X

ðnÞ

ðnÞ

bi :Ri þ

n¼1 i¼1

10 X

h i ðn Þ g ðnÞ S13 ; f ðnÞ

ð6Þ

n¼1

ðnÞ

ðn Þ

where Ri is the release from reservoir n in time period i, S13 is ending storage of reservoir ðnÞ n, f (n) is the target ending storage of reservoir n. The benefit function bi was tabulated by (n) Murray and Yakowitz (1979). The penalty function g for deviation from target ending storage of reservoir n is expressed as: h i h i2 ðn Þ ðnÞ g ðnÞ S13 ; f ðnÞ ¼ 60 S13 f ðnÞ

for

ðn Þ

S13 f ðnÞ

n ¼ 1:::NR

ð7Þ

and h i ðnÞ gðnÞ S13 ; f ðnÞ ¼ 0

for

ðnÞ

S13 f ðnÞ

n ¼ 1:::NR

ð8Þ

1200

Objective Value

1100

1000

Colony 1: 4 Classes for each reservoir Colony 2: 6 Classes for each reservoir Colony 3: 4 Classes for reservoirs 1,2,3,6,8, and 9 5 Classes for reservoirs 4 and 5 10 Classes for reservoirs 7 and 10

900

800 0

500

1000

1500

2000

2500

3000

Iteration

Figure 8 Objective value evaluation of 10-reservoir operation by proposed multi-colony ACO algorithm (averaged over 10 runs).


1443

1200

1100 Objective Value

Information Exchange Steps

1000 Colony 1: 4 Classes for each reservoir Colony 2: 6 Classes for each reservoir 900 Colony 3: 4 Classes for reservoirs 1,2,3,6,8, and 9 5 Classes for reservoirs 4 and 5 10 Classes for reservoirs 7 and 10 800 0

500

1000

1500

2000

2500

3000

Iteration

Figure 9 Objective value evaluation of 10-reservoir operation by proposed multi-colony ACO algorithm (the best of 10 runs).

Diverse range of possible storages and releases for different reservoirs is a unique specification of the problem which imposes serious limitation on any discrete algorithm. As an example, release for reservoir number 3 is bounded by [0.005, 2.12], whereas that of reservoir number 10 has a wider range of variation (i.e. 0.01, 18.9). Therefore, assigning the same number of release classes for all reservoirs will not yield suitable results. In fact, Wardlaw and Sharif (1999) employed a real coding GA to get as close as 99.8% of the global optimal (1,194.44 unit). Jalali et al. (2005b) approached the same 10-reservoir problem using ACO with pheromone re-initiation (PRI), partial path replacement (PPR), and discrete refining (DR) mechanism, getting as close as 98.7% of the global optimal. For a highly complex system, definition of an appropriate heuristic function becomes more and more difficult. In fact, by defining an inappropriate heuristic function one may mislead the ants by providing them with wrong sight or vision. This was tested defining: ηij ¼ ðBF Þi :rj

i ¼ 1; :::; NR NT; j ¼ 1; :::; NC

ð9Þ

in which BFi is the benefit function for cell or element i in a trial solution string and, for different values of β (Jalali et al. 2005b). It was concluded that β=0 might be the best choice for complex systems such as multi-reservoir problems. In the proposed multi colony algorithm, the final solution and the convergence pattern may be influenced by the number of colonies for any number of given ants. To test the impact of number of colonies, the problem was solved for number of colonies ranging from one to five for 300 ants. Statistical measures of the results for 10 runs are depicted in Figure 7. Even though the desirable number of colonies depend on the solution domain, for the problem in hand three colonies may be considered as the best. For colony number 1, all releases from reservoirs were discretized in four classes employing 100 ants. Second colony has a population of 150 ants with six discrete levels.

1444


20

Release

18 16

LP

14

Proposed Multi-Colony ACO

12 10 8 6 4 2

Res. 1

Res. 2

Res. 4 Res. 5 Res. 6

Res. 3

Res. 8

Res. 7

120

108

96

84

72

60

48

36

24

12

0

0 Res. 9 Res. 10

Period

(a) Reservoir Releases 35

Reservoir Storage

30

LP

25

Proposed Multi-Colony ACO

20 15 10 5

Res. 1

Res. 2

Res. 3

Res. 4 Res. 5 Res. 6

Res. 7

Res. 8

120

108

96

84

72

60

48

36

24

12

0

0 Res. 9 Res. 10

Period

(b) Reservoir Storages Figure 10 Comparison between the best result of proposed multi-colony ACO and result of LP model (10reservoir problem).

Third colony with 150 ants has variable discrete classes for different reservoirs. Specifically, release from reservoir number 7 and 10 were discretized into 10 classes, reservoirs 4 and 5 into five classes, and remaining reservoirs were discretized into four classes. Information exchange takes place when all colonies have at least two pheromone re-initiation experiences. In the process of information exchange, the best solution between all colonies is selected and used for developing new discretization scheme for each colony.


1445

At the time of information exchange, pheromone re-initiation mechanism is implemented, providing a better initial solution for all colonies to start with. Results obtained for 10 different runs (using different seed numbers) are presented in Table III for different colonies. The best solution had an objective function of 1,192.39 which is approximated as 99.8% of the known global solution resulted from LP. The best solutions so far reported by different researchers for the same problem are 1,190.625 (Murray and Yakowitz 1979 using constrained DDP) and 1,190.25 (Wardlaw and Sharif 1999 using GA real coded). Relatively low standard deviation for 10 runs and three colonies may be considered as a strength point of the proposed algorithm. Rate of convergence of the results, averaged over 10 runs, for different colonies are presented in Figure 8. Figure 9 presents the rate of convergence of the best run for different colony structures, highlighting the information exchange and discrete refining stages. It is quite clear that after 2,500 iterations using 150 agents, full convergence has been occurred. Release from different reservoirs and storage at different periods for the best solution are presented in Figure 10 (a, b). To have a notion of the global optimal, results of the LP solution are also indicated on the same figures.

5 Concluding Remarks Poor performance of the ACO algorithms for continuous optimization problem is due to discretization of the search space. In small-scale problems, discrete refining (DR) approach may result in near optimum solution in continuous search space. However, in complex real world problems with large continuous search space, it is very often possible to loose the global optimum domain in the early stages of DR process forever. Generating a non-homogeneous and more or less random mesh in the entire search space may minimize the possibility of loosing global optimum domain when discrete refining mechanism is applied. The proposed multi-colony algorithm generates a non-homogeneous, randomly space discretized scheme which efficiently handles the combination of discrete and continuous decision variables. For continuous decision variables, the DR mechanism is employed whereas for discrete decision variables, it is disregarded. Application of the proposed algorithm to well-known multimodal, continuous, nonseperable, nonlinear, and illegal (CNNI) Fletcher–Powell function and complex 10-reservoir problem operation optimization revealed its potential to improve final results in highly complex continuous optimization problems. For the Fletcher–Powell function, the best result obtained is 9,943.55 which is 117% better than Jalali’s previous result (Jalali et al. 2005b). The best objective value of the 10-reservoir problem is approximated as 99.8% of the known global solution.

References Abbasi H (2005) Ant colony optimization algorithm for supply pipe design considering transient flow. MSc thesis, Iran University of Science and Technology, Tehran, Iran Abbaspour KC, Schulin R, van Genuchten MT (2001). Estimating unsaturated soil hydraulic parameters using ant colony optimization. Adv Water Resour 24(8):827–841 Bäck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, New York Bilchev G, Parmee IC (1995) The ant colony metaphor for searching continuous design spaces. In: Fogarty TC (ed) Proceedings of the AISB workshop on evolutionary computation, vol 993 of LNCS. Springer, Berlin Heidelberg New York, pp 25–39

1446


Bolondi M, Bondaza M (1993) Parallelizzazione di un algoritmo per la risoluzione del problema del comesso viaggiatore. Master’s thesis, Politecnico di Milano, Milan, Italy. Bozorg Haddad O, Afshar A (2004). MBO (Marriage Bees Optimization), a new heuristic approach in hydrosystems design and operation. In: Proceedings of the 1st international conference on managing rivers in the 21st century: issues and challenges. Penang, Malaysia, 21–23 Sep 2004, pp 499–504 Bullnheimer B, Kotsis G, Strauss C (1998) Parallelization strategies for the ant system. In: De Leone R, et al. (eds) High performance algorithms and software in nonlinear optimization, applied optimization, vol 24. Kluwer, Dordrecht, pp 87–100 Calégari PR (1999) Parallelization of population-based evolutionary algorithms for combinatorial optimization problems. PhD thesis, D´epartment D’Informatique, Ecole Polytechnique Fédérale De Lausanne Colorni A, Dorigo M, Maniezzo V, Trubian M (1994) Ant system for job shop scheduling. Belg J Oper Res 34:39–53 Dorigo M (1992) Optimization, learning and natural algorithms. PhD thesis, Politecnico di Milano, Milan, Italy Dorigo M, Di Caro G (1999). The ant colony optimization metaheuristic. In: Corne D, Dorigo M, Glover F (eds) New ideas in optimization. McGraw-Hill, London, pp 11–32 Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1(1):53–66 Dorigo M, Maniezzo V, Colorni A (1996) The ant system: optimization by a colony of cooperating ants. IEEE Trans Syst Man Cybern 26:29–42 Dorigo M, Bonabeau E, Theraulaz G (2000) Ant algorithms and stigmergy. Future Gener Comput Syst 16:851–871 Dreo J, Siarry P (2002) A new ant colony algorithm using the hierarchical concept aimed at optimization of multiminima continuous functions. In: Dorigo M, Caro GD, Sampels M (eds) Proceedings of the 3rd international workshop on ant algorithms (ANTS 2002), vol 2463 of LNCS. Springer, Berlin Heidelberg New York, pp 216–221 Esat V, Hall MJ (1994) Water resources system optimization using genetic algorithms. In: Hydroinformatics ’94, Proceedings of the 1st international conference on hydroinformatics. Balkema, Rotterdam, The Netherlands, pp 225–231 Fahmy HS, King JP, Wentzel MW, Seton JA (1994). Economic optimization of river management using genetic algorithms (paper no 943034). International summer meeting, American Society of Agricultural Engineers, St Joseph, MI Gambardella LM, Taillard ED, Dorigo M (1999a) Ant colonies for the quadratic assignment Problem. J Oper Res Soc 50:167–176 Gambardella LM, Taillard ED, Agazzi G (1999b) MACS-VRPT: a multiple ant colony system for vehicle routing problems with time windows. In: Corne D, Dorigo M, Glover F (eds) New ideas in optimization. McGraw-Hill, pp 63–76 Jalali MR, Afshar A, Mariño MA (2005a). Improved ant colony optimization algorithm for reservoir operation (technical report). Hydroinformatics Center, Civil Engineering Department, Iran University of Science and Technology, Tehran, Iran Jalali MR, Afshar A (2005b) Semi-continuous ACO algorithms (technical report). Hydroinformatics Center, Civil Engineering Department, Iran University of Science and Technology, Tehran, Iran Jalali MR, Afshar A, Mariño MA (2006). Reservoir operation by ant colony optimization algorithms. Iran J Sci Technol 30(B1):107–117 Labadie JW (2004) Optimal operation of multireservoir systems: state of-the-art review. J Water Resour Plan Manage, ASCE 130(2):93–111 Maier HR, Simpson AR, Zecchin AC, Foong WK, Phang KY, Seah HY, Tan CL (2003) Ant colony optimization for design of water distribution systems. J Water Resour Plan Manage 129(3):200–209 Maniezzo V, Colorni A (1999) The ant system applied to the quadratic assignment problem. IEEE Trans Knowl Data Eng 11:769–778 Mathur M, Karale SB, Priye S, Jyaraman VK, Kulkarni BD (2000) Ant colony approach to continuous function optimization. Ind Eng Chem Res 39:3814–3822 Merkle D, Middendorf M, Schmeck H (2000) Ant colony optimization for resource-constrained project scheduling. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2000). Morgan Kaufmann, San Mateo, CA, pp 893–900 Michels R, Middendorf M (1999) An ant system for the shortest common supersequence problem. In: Corne D, Dorigo M, Glover F (eds) New ideas in optimization. McGraw-Hill, New York, pp 51–61 Middendorf M, Reischle F, Schmeck H (2000) Information exchange in multi colony ant algorithms. In: Rolim J (ed) Parallel and distributed computing, proceedings of the 15 IPDPS 2000 workshops, third


1447

workshop on biologically inspired solutions to parallel processing problems (BioSP3), 1–5 May 2000, Cancun, Mexico, LNCS, vol 1800. Springer, Berlin Heidelberg New York, pp 645–652 Middendorf M, Reischle F, Schmeck H (2002) Multi colony ant algorithms. Journal of Heuristics 8:305–320 Monmarche N, Venturini G, Slimane M (2000) On how Pachycondyla apicalis ants suggest a new search algorithm. Future Gener Comput Syst 16:937–946 Murray DM, Yakowitz S (1979) Constrained differential dynamic programming and its application to multireservoir control. Water Resour Res 15(5):1017–1027 Oliveira R, Loucks D (1997) Operating rules for multireservoir systems. Water Resour Res 33(4):839–852 Simpson AR, Maier HR, Foong WK, Phang KY, Seah HY, Tan CL (2001) Selection of parameters for ant colony optimization applied to the optimal design of water distribution systems. In: Proceedings of the international congress on modeling and simulation, Canberra, Australia, pp 1931–1936 Stützle T (1998) Parallelization strategies for ant colony optimization. In: Eiben AE, Bäck T, Schoenauer M, Schwefel H-P (eds) Parallel problem solving from Nature-PPSN V, LNCS, vol 1498. Springer, Berlin Heidelberg New York, pp 722–731 Stützle T, Hoos HH (1997a) The MAX–MIN ant system and local search for the traveling salesman problem. In: Baeck T, Michalewicz Z, Yao X (eds) Proceedings of IEEE-ICEC-EPS ’97, IEEE international conference on evolutionary computation and evolutionary programming conference. IEEE, pp309–314 Stützle T, Hoos HH (1997b) Improvements on the ant system: introducing MAX–MIN ant system. In: Proceedings of the international conference on artificial neural networks and genetic algorithms. Springer, Berlin Heidelberg New York, pp 245–249 Talbi E-G, Roux O, Fonlupt C, Robillard D (1999) Parallel ant colonies for combinatorial optimization problems. In: Rolim J et al (eds) Parallel and distributed processing, 11 IPPS/SPDP ’99 workshops, LNCS, vol 1586. Springer, Berlin Heidelberg New York, pp 239–247 Wardlaw R, Sharif M (1999) Evaluation of genetic algorithms for optimal reservoir system operation. J Water Resour Plan Manage, ASCE, 125(1):25–33 Wodrich M, Bilchev G (1997) Cooperative distributed search: the ant’s way. Control Cybern (3):413–446 Zecchin AC, Maier HR, Simpson AR, Roberts A, Berrisford MJ, Leonard M (2003) Max–min ant system applied to water distribution system optimization. In: Modsim 2003 – international congress on modelling and simulation, Modelling and Simulation Society of Australia and New Zealand Inc, Townsville, Australia, 2, 795–800