Evaluation of Covariance Matrix Adaptation Evolution Strategy ...

chemical engineering research and design 9 0 ( 2 0 1 2 ) 2051–2071

Contents lists available at SciVerse ScienceDirect

Chemical Engineering Research and Design journal homepage: www.elsevier.com/locate/cherd

Evaluation of Covariance Matrix Adaptation Evolution Strategy, Shuffled Complex Evolution and Firefly Algorithms for phase stability, phase equilibrium and chemical equilibrium problems Seif-Eddeen K. Fateen a,∗ , Adrián Bonilla-Petriciolet b , Gade Pandu Rangaiah c a b c

Department of Chemical Engineering, Cairo University, Egypt Department of Chemical Engineering, Instituto Tecnológico de Aguascalientes, Mexico Department of Chemical & Biomolecular Engineering, National University of Singapore, Singapore

a b s t r a c t Phase equilibrium calculations and phase stability analysis of reactive and non-reactive systems play a significant role in the simulation, design and optimization of reaction and separation processes in chemical engineering. These challenging problems, which are often multivariable and non-convex, require global optimization methods for solving them. Stochastic global optimization algorithms have shown promise in providing reliable and efficient solutions for these thermodynamic problems. In this study, we evaluate three alternative global optimization algorithms for phase and chemical equilibrium calculations, namely, Covariant Matrix Adaptation-Evolution Strategy (CMA-ES), Shuffled Complex Evolution (SCE) and Firefly Algorithm (FA). The performance of these three stochastic algorithms was tested and compared to identify their relative strengths for phase equilibrium and phase stability problems. The phase equilibrium problems include both multi-component systems with and without chemical reactions. FA was found to be the most reliable among the three techniques, whereas CMA-ES can find the global minimum reliably and accurately even with a smaller number of iterations. © 2012 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved. Keywords: Covariant Matrix Adaptation Evolution Strategy; Shuffled Complex Evolution; Phase stability analysis; Phase equilibrium calculations; Chemical equilibrium calculations

1.

Introduction

The calculations of phase and chemical equilibrium are an essential component of all process simulators. The search for better methods and techniques to solve these often-difficult thermodynamic problems is still ongoing. Current methods

have their own deficiencies and sometime fail to find the correct solutions for difficult problems such as calculation of simultaneous phase and chemical equilibrium for systems containing many components near the critical point of the mixture and the phase boundaries (Zhang et al., 2011a). Actually, novel processes handle complex mixtures, severe

Abbreviations: CMA-ES, Covariant Matrix Adaptation-Evolution Strategy; DE, Differential Evolution; DETL, Differential Evolution with Tabu List; ES, Evolution Strategy; FA, Firefly Algorithm; GA, Genetic Algorithm; GSR, Global Success Rate; IDE, Integrated Differential Evolution; IDE N, IDE without Tabu List and Radius; MFA, Modified Firefly Algorithm; MFlops, million floating-point operations; NFE, number of function evaluations; PEC, phase equilibrium calculations; PS, phase stability; PSO, Particle Swarm Optimization; PSO-CNM, PSO with Nelder–Mead Simplex Method (PSO-CNM); PSO-CQN, PSO with Quasi-Newton Method; rPEC, Reactive Phase Equilibrium Calculations; RPS, Repulsive Particle Swarm; RT, Random Tunneling; SA, Simulated Annealing; SC, stopping criteria; SCE, Shuffled Complex Evolution; SQP, Sequential Quadratic Programming; SR, Success Rate; TPDF, tangent plane distance function; TS, Tabu Search; UBBPSO, Unified Bare-Bones Particle Swarm Optimization. ∗ Corresponding author. Tel.: +20 1114008888; fax: +20 237497646. E-mail addresses: [email protected] (S.-E.K. Fateen), [email protected] (A. Bonilla-Petriciolet), [email protected] (G.P. Rangaiah). Received 12 November 2011; Received in revised form 11 April 2012; Accepted 24 April 2012 0263-8762/$ – see front matter © 2012 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.cherd.2012.04.011

2052

c D F, Fobj G g* I i, j k Keq m n

N

np nref p q r Rg SCmax

T x y z


number of components dimension of the problem objective function Gibbs free energy global minimum brightness of a firefly in FA index of the component, or index of the firefly in FA index of the problem dimension in FA equilibrium constant of a reaction number of points in a complex in SCE number of moles in the thermodynamics equations, or population size parameter in the stochastic algorithms matrix of stoichiometric coefficients of a set of reference components chosen from the reactions number of problems column vector of moles of each of the reference components penalty term or number of complexes in SCE number of points in a sub-complex in SCE number of reactions universal gas constant maximum number of successive iterations/generations without improvement in the best objective function value temperature, K decision variable vector mole fraction mole fraction

Greek letters ˛ number of consecutive offspring generated by each sub-complex in SCE, or parameter of the random term in FA move equation ˇ transformed decision variables used instead of mole fractions, or attractiveness of a firefly in FA ε vector of random numbers in FA ϕ fugacity coefficient of a pure component fugacity coefficient of a component in a mixture ϕˆ activity coefficient, or parameter in the attractiveness equation in FA parameter of the extra term in FA chemical potential stoichiometric coefficient of a component in a reaction number of phases Subscripts F feed I index for the components in the mixture index for the iterations in FA K minimum value min O initial value of the parameter at composition y Y at composition z Z Superscripts 0 pure component

operating conditions, or even incorporate combined unit operations (e.g., reactive distillation, extractive distillation, etc.). Therefore, wrong estimation of the thermodynamic state may have negative impacts on the design, analysis and operation of such novel processes. The prediction of phase behavior of a mixture involves the solution of two thermodynamic problems: phase stability (PS) analysis and phase equilibrium calculations (PEC). PS problems involve the determination of whether a system will remain in one phase at the given conditions or split into two or more phases. This type of problems usually precedes the PEC problem, which involves the determination of the number, type and composition of the phases at equilibrium at the given conditions. During the analysis of a chemical process, PS and PEC problems need to be solved numerous times. Solving both these types of thermodynamic problems involves the use of global optimization. In particular, PS analysis requires the minimization of the tangent plane distance function (TPDF), while the Gibbs free energy function needs to be minimized for PEC (Srinivas and Rangaiah, 2007b). For PS problems, finding a local minimum for TPDF is not sufficient; and the global minimum must be identified for determining the correct stability condition. The number and type of phases, at which Gibbs free energy function achieves the global minimum, are usually unknown in PEC problems, and so several calculations may have to be performed using different phase configurations to identify the stable equilibrium state, which increases the complexity of the optimization problem. In general, high non-linearity of thermodynamic models, non-convexity of TPDF and Gibbs free energy functions and the presence of a trivial solution in the search space make PEC and PS problems difficult to solve. Moreover, both these thermodynamic problems may have local optimal values that are very comparable to the global optimum value, which makes it challenging to find the global optimum (Bonilla-Petriciolet et al., 2011). Note that a Reactive Phase Equilibrium Calculation (rPEC) or chemical equilibrium calculation is performed if any reaction is possible in the system under study, and the objective function (i.e., Gibbs free energy) must satisfy the chemical equilibrium constraints. Further, complexity and dimensionality of these calculations increase significantly. Hence, PS, PEC, and rPEC problems require a reliable and efficient global optimization algorithm. Many deterministic and stochastic optimization algorithms have been proposed and tested for finding the global optimum in PS, PEC and rPEC problems, particularly in the past two decades (Avami and Saboohi, 2011; Bonilla-Petriciolet et al., 2011; Bonilla-Petriciolet and Segovia-Hernández, 2010; Burgos-Solórzano et al., 2004; Jalali and Seader, 1999; Jalali et al., 2008; Rangaiah, 2001; Reynolds et al., 1997; Rossi et al., 2009; Srinivas and Rangaiah, 2007a; Wasylkiewicz and Ung, 2000). A comprehensive review of these techniques can be found in (Zhang et al., 2011a). Deterministic global optimization methods have been applied to different PEC, PS and/or rPEC problems. For example, homotopy continuation methods have been applied to PEC and PS problems (Jalali et al., 2008; Sun and Seider, 1995). Although homotopy-continuation algorithm guarantees global convergence to a single solution, it does not guarantee global convergence to multiple solutions. Even using complex search spaces, the success of continuation methods in finding all solutions cannot be assured. On the other hand, interval analysis has been used to solve nonlinear equations to find all stationary points of TPDF for phase stability analysis (Tessier et al., 2000; Xu et al., 2002).


This approach has been combined with other numerical procedures for performing phase equilibrium calculations. For example, Burgos-Solórzano et al. (2004) applied interval Newton method for solving the PEC and rPEC problems under high pressure. In general, the interval analysis method can solve nonlinear equations to find all solutions lying within the variable bounds. It requires an interval extension of the Jacobian matrix, and involves setting up and solving the interval Newton equation for a new interval. However, it is quite a challenge to find all solutions and the Jacobian matrix for the complex systems, and the computational time is significant for multicomponent systems. Recently, Rossi et al. (2011) applied convex analysis method to PEC and rPEC problems. This method employs the CONOPT solver in General Algebraic Modeling System (GAMS). The proposed method can solve PEC problems with high efficiency and reliability but it requires the convexity of the model. Branch and bound methods have been applied to many chemical engineering optimization problems including PS and PEC problems (Cheung et al., 2002; Harding and Floudas, 2000). In general, these methods are often slow and require a significant numerical effort that grows exponentially with problem size (Nichita et al., 2002a; Wakeham and Stateva, 2004). Besides, branch and bound methods require certain properties of the objective function, and problem reformulation is usually needed to guarantee the global convergence. Note that the problem reformulation can be very difficult to perform for complex thermodynamic models such as equations of state with nontraditional mixing rules. Finally, Nichita et al. applied the tunneling method to perform stability analysis of various systems (Nichita et al., 2002b, 2008) and to PEC problems (Nichita et al., 2002a, 2004). Their results suggest that tunneling method is a robust and efficient tool for these applications even for difficult cases. However, it requires feasible and improved initial estimates for reliability and computational efficiency respectively (Nichita et al., 2002a). For an unknown system, it is difficult to provide a feasible and good initial estimate for the algorithm. In summary, the deterministic methods can guarantee convergence to the global optimum only when certain properties such as continuity are satisfied or a priori information of the system is available. Also, reformulation of the problem may be needed depending on the characteristic of the thermodynamic models, and the computational time grows exponentially with problem size. In contrast, stochastic methods are quite simple to implement and use. They do not require any assumptions or transformation of the original problems, can be applied with any thermodynamic model, and yet provide a high probabilistic convergence to the global optimum. They can often locate the global optimum in modest computational time compared to deterministic methods (Bonilla-Petriciolet et al., 2011). In recent years, several stochastic global optimization techniques have been applied to solve the PS and PEC problems in non-reactive and reactive systems (Bonilla-Petriciolet et al., 2006, 2011; Bonilla-Petriciolet and Segovia-Hernández, 2010; Rahman et al., 2009; Rangaiah, 2001; Reynolds et al., 1997; Srinivas and Rangaiah, 2006, 2007a,b). These algorithms include Simulated Annealing (SA), Genetic Algorithms (GA), Tabu Search (TS), Differential Evolution (DE), Random Tunneling method (RT), Particle Swarm Optimization (PSO), Repulsive Particle Swarm (RPS) and hybrid stochastic algorithms. In particular, Srinivas and Rangaiah (2007a, 2010) studied DE and TS for non-reactive mixtures, and proposed two versions of DE with Tabu List (DETL), in order to improve the performance

2053

of the optimization algorithm. Srinivas and Rangaiah (2006) evaluated the RT on a number of medium sized problems including vapor–liquid, liquid–liquid and vapor–liquid–liquid equilibrium problems. RT can locate the global optimum for most of the examples tested but its reliability is low for problems having a local minimum comparable to the global minimum. Rahman et al. (2009) have concluded that RPS can reliably locate the global minimum of TPDF problems. In a recent study, Bonilla-Petriciolet and Segovia-Hernández (2010) tested different versions of PSO for PS and PEC for both reactive and non-reactive systems, and their results show that classical PSO is a reliable method with good performance. Finally, hybrid stochastic global optimization such as bare-bones Particle Swarm Optimization and Integrated Differential Evolution with Tabu List have been successfully applied to phase equilibrium and phase stability problems (Zhang et al., 2011b). Systematic and comprehensive comparison of different global optimization methods is challenging. However, some comparison of stochastic with deterministic algorithms for phase equilibrium calculations can be found in the literature. Teh and Rangaiah (2002, 2003) compared the performance of GA and TS with several deterministic algorithms such as Rachford-Rice-Mean value theorem-Wegstein’s projection method, accelerated successive substitution method, Nelson’s method, simultaneous equation-solving method, linearly constrained minimization method, GLOPEQ and enhanced interval analysis method for solving phase equilibrium calculations. Their comparison shows that some stochastic methods can be more efficient than deterministic algorithms. Most of the stochastic methods have some parameters to be tuned for different problems in order to improve the convergence to the global optimum. Selection of proper parameter values for different problems usually require a lot of effort, and an improper choice can result in computational inefficiency or poor reliability. In order to overcome such difficulties, this work evaluates three global optimization algorithms that have fewer algorithm parameters, for PEC, rPEC and PS problems involving multiple components, multiple phases and popular thermodynamic models. The performance of Covariant Matrix Adaptation-Evolution Strategy (CMA-ES), Shuffled Complex Evolution (SCE) and Firefly Algorithm (FA) on PS, PEC and rPEC problems are compared and discussed based on both reliability and computational efficiency using practical stopping criteria. The remainder of this paper is organized as follows. The three algorithms, CMA-ES, SCE and FA, are presented in Section 2. Description of PEC, PS and rPEC problems is given in Section 3. Implementation of the three algorithms is covered in Section 4. Section 5 presents the results and discusses the performance of CMA-ES, SCE and FA on PEC, PS and rPEC problems. Finally, the conclusions of this work are summarized in Section 6.

2. Description of stochastic global optimization techniques In this study, the global optimization problem to be solved is: Minimize F(X)

(1)

with respect to D decision variables: X = (X1 , X2 , . . ., Xd , . . ., XD ). The upper and lower bounds of these variables are

2054


Fig. 1 – Simplified algorithm of CMA-ES. Adapted from Hansen (2006).

1 , X2 , . . . , Xd , . . . , 1 2 d D Xmax , Xmax , . . . , Xmax , . . . , Xmax and Xmin min min D Xmin respectively. Three different stochastic global optimization techniques, the Covariant Matrix Adaptation Evolution Strategy (CMAES), the Shuffled Complex Evolution (SCE) and the Firefly Algorithm (FA), were evaluated for the phase stability and equilibrium problems in this study. The first two techniques belong to the Evolutionary Methods category, while the third technique is one type of Swarm Intelligence

techniques, according to the classification presented by Rangaiah (2010). Those methods were selected because no attempts have been reported in the literature to evaluate their performance in solving the phase and chemical equilibrium problems and their performance could be superior to other stochastic methods. Each of these methods is briefly described in the following sections. More details of these stochastic optimization methods can be found in the cited references.


2055

Step 1: Set parameters and Initialize Set parameters to their default values An initial population of points is sampled randomly from the feasible solution space (Ω). The selected population is partitioned into one or more complexes, each containing a fixed number of points. Step 2: Main evolution loop Each complex evolves according to a competitive complex evolution (CCE) algorithm. CCE algorithm employs a downhill simplex method in generating offsprings. The entire population is periodically shuffled and points are reassigned to complexes to share the information from the individual complexes. Evolution and shuffling are repeated so that the entire population is close to convergence criteria, and are stopped if the convergence criteria are satisfied. Local optimization starting from the best solution found by the global search Output: Solution found by local optimizer.

Fig. 2 – Simplified algorithm of SCE. Adapted from Duan et al. (1992).

2.1.

The CMA-ES method

Evolution Strategy (ES) is a stochastic search algorithm, in which search steps are taken by stochastic variation (mutation) of points found so far. The best out of a number of new search points are selected to continue. The mutation is usually carried out by adding a normally distributed random vector. Pair-wise dependencies between the variables in this distribution are described by a covariance matrix. For most objective functions, the mutation covariance matrix needs to be adapted continually during optimization. The covariance matrix adaptation evolution strategy (CMA-ES) was proposed to adapt the mutation covariance by gathering information about successful search steps, using that information to modify the covariance matrix of the mutation distribution in a goal directed, de-randomized fashion (Hansen and Ostermeier, 2001). Changes to the covariance matrix are such that variances in directions of the search space that have previously been successful are increased while those in other directions passively decrease. The accumulation of information over a

number of search steps makes it possible to adapt the covariance matrix even when using small populations. CMA-ES has found many engineering applications (e.g., feedback control of combustion (Hansen et al., 2009), calibration of scientific instruments (Wilson et al., 2008), solving design problems in hydrogeology (Bayer and Finkel, 2007), parameter estimation (Hohm and Zitzler, 2007)), and many others. A simplified algorithm, taken from (Hansen, 2006), is presented in Fig. 1. All the parameters needed for CMA-ES were made as functions of the population size, n, and overall standard deviation, (Hansen and Ostermeier, 2001). The method adapts itself and hence no need to fine-tune any additional parameter. In this study, we used the modification proposed by Ros and Hansen (2008) that reduces the internal time and space complexity from quadratic to linear. We also enabled the active CMA-ES option (Jastrebski and Arnold, 2006), which uses the information about the unsuccessful offspring candidate solutions to actively reduce variances of the mutation distribution in unpromising directions of the search space.

Step 1: Set parameters and Initialize Generate initial population of n fireflies x i Light intensity Ii at x iis determined by f(x) i Define light absorption coefficient γ Step 2: Main evolution loop while (t < MaxGeneration) for i=1:n for j=1:n if (Ii < Ij ), Move firefly i towards j; end if Vary attractiveness with distance r via exp[-γr] Evaluate new solutions and update light intensity end for j end for i Rank the fireflies and find the current best solution. end while Local optimization starting from the best solution found by the global search Output: Solution found by local optimizer.

Fig. 3 – Simplified algorithm of FA. Adapted from Yang (2007).

2056

Table 1 – Details of PEC and PS problems studied. System

Feed conditions

1

n-Butyl acetate + water

nF = (0.5, 0.5) at 298 K and 101.325 kPa

2

Toluene + water + aniline

nF = (0.29989, 0.20006, 0.50005) at 298 K and 101.325 kPa

3

N2 + C1 + C2

nF = (0.3, 0.1, 0.6) at 270 K and 7600 kPa

4

C1 + H2 S

nF = (0.9813, 0.0187) at 190 K and 4053 kPa

5

C2 + C3 + C4 + C5 + C6

nF = (0.401, 0.293, 0.199, 0.0707, 0.0363)at 390 K and 5583 kPa

6

C1 + C2 + C3 + C4 + C5 + C6 + C7–16 + C17+

nF = (0.7212, 0.09205, 0.04455, 0.03123, 0.01273, 0.01361, 0.07215, 0.01248) at 353 K and 38,500 kPa

7

C1 + C2 + C3 + iC4 + C4 + iC5 + C5 + C6 + iC15

8

C1 + C2 + C3 + C4 + C5 + C6 + C7 + C8 + C9 + C10

nF = (0.614, 0.10259, 0.04985, 0.008989, 0.02116, 0.00722, 0.01187, 0.01435, 0.16998) at 314 K and 2010.288 kPa nF = (0.6436, 0.0752, 0.0474, 0.0412, 0.0297, 0.0138, 0.0303, 0.0371, 0.0415, 0.0402) at 435.35 K and 19,150 kPa

Thermodynamic models

NRTL model and parameters reported by Rangaiah (2001) NRTL model and Model parameters reported by McDonald and Floudas (1995) SRK EoS with classical mixing rules and parameters reported by Bonilla-Petriciolet et al. (2006) SRK EoS with classical mixing rules and parameters reported by Rangaiah (2001) SRK EoS with classical mixing rules and parameters reported by Bonilla-Petriciolet et al. (2006) SRK EoS with classical mixing rules and parameters reported by Harding and Floudas (2000) SRK EoS with classical mixing rules. Model parameters reported by (Rangaiah, 2001). SRK EoS with classical mixing rules and parameters reported by Bonilla-Petriciolet et al. (2006)

Global optimum for Equilibrium

Stability

−0.020198

−0.032466

−0.352957

−0.294540

−0.547791

−0.015767

−0.019892

−0.003932

−1.183653

−0.000002

−0.838783

−0.002688

−0.769772

−1.486205

−1.121176

−0.0000205


PEC & PS no.

Table 2 – Details of rPEC (chemical equilibrium) problems studied. System

Feed conditions

Thermodynamic models

Global optimum

Reference(s)

1

A1 + A2 ↔ A3 + A4

nF = (0.5, 0.5, 0.0, 0.0) at 355 K and 101.325 kPa

NRTL model and ideal gas. Keq = 18.670951

−2.05812

McDonald and Floudas (1995), Bonilla-Petriciolet et al. (2011), Bonilla-Petriciolet and Segovia-Hernández (2010)

nF = (0.3, 0.3, 0.0, 0.4) at 373.15 K and 101.325 kPa

Wilson model and ideal gas.

−1.434267

Bonilla-Petriciolet et al. (2011)

3

(1) Ethanol (2) Acetic acid (3) Ethyl acetate (4) Water A1 + A2 ↔ A3, and A4 as an inert component (1) Isobutene (2) Methanol (3) Methyl ter-butyl ether (4) n-Butane A1 + A2 + 2A3 ↔ 2A4

4

(1) 2-Methyl-1-butene (2) 2-Methyl-2-butene (3) Methanol (4) Tert-amyl methyl ether A1 + A2 ↔ A3 + A4

5

2

6

G0 rxs /R = −4205.05 + 10.0982T − 0.2667T ln T ln Keq = − G0 rxs /R where T is in K nF = (0.354, 0.183, 0.463, 0.0) at 355 K and 151.95 kPa

Wilson model and ideal gas. Keq = 1.057 × 10−04 e4273.5/T . Where T is in K

−1.226367

Bonilla-Petriciolet et al. (2006), Bonilla-Petriciolet et al. (2011)

nF = (0.3, 0.4, 0.3, 0.0) at 298.15 K and 101.325 kPa

UNIQUAC model and ideal gas. ln Keq = 450/T + 0.8

−1.10630

Wasylkiewicz and Ung (2000) and Bonilla-Petriciolet et al. (2011)

(1) Acetic acid (2) n-Butanol (3) Water (4) n-Butyl acetate A1 + A2 ↔ A3

nF = (0.6, 0.4, 0.0)

−0.144508


A1 + A2 + 2A3 ↔ 2A4 with A5 as inert component

nF = (0.1, 0.15, 0.7, 0.0, 0.05) at 335 K and 151.9875 kPa

Margules solution model. gE /Rg T = 3.6x1 x2 + 2.4x1 x3 + 2.3x2 x3 Keq = 0.9825 Wilson model and ideal gas. Keq = 1.057*10−04 e4273.5/T where T is in K

−0.872577


Margules solution model Keq = 3.5 NRTL model Keq = 4.0

−0.653756

Wasylkiewicz and Ung (2000) and Bonilla-Petriciolet et al. (2011) Wasylkiewicz and Ung (2000) and Bonilla-Petriciolet et al. (2011)

7

(1) 2-Methyl-1-butene (2) 2-Methyl-2-butene (3) Methanol (4) Tert-amyl methyl ether (5) n-Pentane A1 + A2 ↔ A3

8

A1 + A2 ↔ A3 + A4

nF = (0.52, 0.48, 0.0) at 323.15 K and 101.325 kPa nF = (0.048, 0.5, 0.452, 0.0) at 360 K and 101.325 kPa

−0.311918


rPEC no.

2057

2058


Table 3 – Selected values of the parameters used in the implementation of CMA-ES, SCE and FA. Method

Parameter

Selected value

CMA-ES

n p m q ˛ n ˛o ˇmin n

0.5 10D n/(2D + 1) (2D + 1) (D + 1) 30 10D 0.5 0.2 1 10D

SCE

FA

2.2.

The SCE algorithm begins with a population of points sampled randomly from the feasible space. The population is partitioned into several communities, each of which evolves based on a statistical reproduction process that uses the “simplex” geometric shape to direct the search in an improvement direction. At periodic stages in the evolution, the entire population is shuffled and points are reassigned to communities to ensure information sharing. As the search progresses, the entire population converges toward the neighborhood of global optimum. The SCE parameters include the number of points in a complex (m), the number of points in a sub-complex (q), the number of complexes (p), and the number of consecutive offspring generated by each sub complex (˛). The basic algorithm of the SCE method is presented in Fig. 2.

The SCE method 2.3.

SCE is another population-based stochastic global optimization algorithm proposed by Duan et al. (1992). The SCE algorithm and its variants have been used in many engineering applications such as the assessment and optimization of hydrologic model (Vrugt et al., 2003a,b). However, to the best of our knowledge, this method has not been applied for performing thermodynamic calculations.

The Firefly Algorithm

FA is a nature-inspired meta-heuristic stochastic global optimization method that was developed by Yang (2007). It is a relatively new method that has gained popularity in finding the global minimum of diverse applications. It was rigorously evaluated by Gandomi et al. (2011), and has been recently used to solve the flow shop scheduling problem (Sayadi et al., 2010)

Fig. 4 – Global Success Rate, GSR, versus iterations for PS problems using CMA-ES, SCE and FA with SC-1 (a) stochastic method only and (b) stochastic method combined with local optimization.

2059


Table 4 – Success rate (SR) and number of function evaluations of CMA-ES, SCE and FA for PS problems using SCmax with NP of 10D. PS no.

PS-1

PS-2

PS-3

PS-4

PS-5

PS-6

PS-7

PS-8

SCmax

10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50

Average GSR and total NFE

CMA-ES

SCE

FA

SR

NFE

SR

NFE

SR

NFE

36 39 40 30 61 57 100 100 100 95 95 95 7 100 100 100 100 100 98 100 99 23 27 100 75.1

494 1076 3022 1105 2722 3453 2535 3737 4249 817 1569 2557 551 5997 9033 881 2081 4081 13,690 2081 4081 2592 3350 27,311 103,065

55 55 55 99 100 100 100 100 100 100 100 100 20 94 99 29 35 37 10 9 9 29 38 49 63.4

726 956 2596 1912 2513 3388 2159 2589 4734 760 1676 3259 922 6040 9706 7287 11,832 17,183 22,377 25,090 29,219 2327 13,215 23,928 196,394

51 66 79 57 32 21 100 100 100 100 100 100 60 57 38 30 36 43 99 97 98 53 60 61 68.25

517 1094 1988 1087 2083 3665 712 1701 3191 442 1046 2158 1382 3179 5902 2562 5482 13,659 4536 9690 18,967 3243 7127 18,158 113,569

tiveness is proportional to brightness and, thus, for any two flashing fireflies, the less bright one moves toward the brighter one. A simplified algorithm for the FA technique is presented in Fig. 3. In this algorithm, the attractiveness of a firefly is determined by its brightness, which is equal to the objective

and financial portfolio optimization (Giannakouris et al., 2010). The FA algorithm imitates the mechanism of firefly communications via luminescent flashes. In the FA algorithm, the two important issues are the variation of light intensity and the formulation of attractiveness. The brightness of a firefly is determined by the landscape of the objective function. Attrac-

Table 5 – Success rate (SR) and number of function evaluations (NFE) of CMA-ES, SCE and FA for PEC problems using SCmax with NP of 10D. PEC no.

PEC-1

PEC-2

PEC-3

PEC-4

PEC-5

PEC-6

PEC-7

PEC-8


SCmax

10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50 10 25 50

CMA-ES

SCE

SR

NFE

88 98 98 61 100 100 100 100 100 75 78 78 6 82 100 32 100 100 100 100 100 23 36 100 81.5

1034 2486 3485 2962 4463 5036 2080 2905 5276 326 661 1487 618 5750 7856 2078 12,520 15,308 14,064 17,988 21,685 2637 4516 19,920 157,141

SR 43 50 98 96 97 97 100 100 100 72 73 74 11 87 100 55 98 100 100 99 98 27 64 96 80.6

FA NFE

SR

NFE

334 1598 3485 2381 3077 4513 2275 3392 4908 1027 1880 3347 516 4855 8539 3224 11,516 15,669 39,012 41,355 46,197 961 12,511 24,557 241,129

51 82 93 4 0 0 100 100 100 55 86 97 95 97 89 100 100 100 100 100 100 100 100 100 81.2

499 1127 2068 829 1832 3449 677 1515 3339 463 1187 2533 1890 3337 5887 2856 5462 10,323 3077 7047 17,608 3455 6409 15,100 101,974

2060


Fig. 5 – Global Success Rate, GSR (plot a) and average NFE (plot b) of CMA-ES, SCE and FA for PS problems using SC-2 (SCmax = 10, SCmax = 25 and SCmax = 50) and SC-1 (1500 iterations). function. The brightness of a firefly at a particular location x was chosen as I(x) = f(x). The attractiveness is judged by the other fireflies. Thus, it was made to vary with the distance between firefly i and firefly j. The attractiveness was made to vary with the degree of absorption of light in the medium between the two fireflies. Thus, the attractiveness is given by ˇ = ˇmin + (ˇo − ˇmin )e−r

2

(3)

k=1

The movement of a firefly attracted to another more attractive (brighter) firefly j is determined by xi = xi + ˇ(xj − xi ) + ˛εi

˛k = 0.9939˛k−1

(5)

Thus the randomness is decreased gradually as the optima are approached. This formula was adapted from Yang (2011).

(2)

The distance between any two fireflies i and j at xi and xj is the Cartesian distance:

d 2 (xi,k − xj,k ) rij = xi − xj =

decrease with the increase in the iteration number, k, in order to reduce the randomness according to the following formula:

(4)

The second term is due to the attraction, while the third term εi is a vector of random numbers drawn from a uniform distribution in the range [−0.5, 0.5]. In our implementation, we used the value of 1 for ˇo , 0.2 for ˇmin and ˛ was made to

3. Formulation of PS, PEC and rPEC problems A brief description of the global optimization problems including the objective function, decision variables and constraints, for PEC, PS and rPEC problems is given in the following sections.

3.1.

Phase stability problems

Solving the PS problem is usually the starting point for the phase equilibrium calculations. The theory used to solve this problem states that a phase is stable if the tangent plane generated at the feed (or initial) composition lies below the molar Gibbs energy surface for all compositions. One common implementation of the tangent plane criterion (Harding and Floudas, 2000; Michelsen, 1982; Sun and Seider, 1995) is to


2061

Fig. 6 – Global Success Rate, GSR versus iterations for PEC problems using CMA-ES, SCE and FA with SC-1 (a) stochastic method only and (b) stochastic method combined with local optimization. minimize the tangent plane distance function (TPDF), defined as the vertical distance between the molar Gibbs energy surface and the tangent plane at the given phase composition. Specifically, TPDF is given by

TPDF =

c

yi ( i |y − i |z )

(6)

i=1

where i |y and i |z are the chemical potentials of component i calculated at compositions y and z, respectively. For stability analysis of a phase/mixture of composition z, TPDF must be globally minimized with respect to composition of a trial phase y. If the global minimum value of TPDF is negative, the phase is not stable at the given conditions, and phase split calculations are necessary to identify the compositions of each phase. The decision variables for minimizing TPDF in phase stability problems are mole fractions, yi for i = 1, 2, . . ., c, each in the range [0, 1], and the constraint is that summation of these mole fractions is equal to 1. The constrained global optimization of TPDF can be transformed into an unconstrained problem by using decision variables ˇi instead of yi as follows: niy = ˇi zi nF

i = 1, ..., c

(7)

and yi =

niy

j=1 c

i = 1, . . . , c

njy

(8)

where nF is the total moles in the feed mixture used for stability analysis, and niy are the conventional mole numbers of component i in trial phase y. The number of decision variables is still c for the unconstrained minimization of TPDF. Thus, the unconstrained global optimization problem for phase stability analysis becomes: min TPDF(ˇi ) 0 ≤ ˇi ≤ 1,

i = 1, . . . , c

(9)

The calculation of TPDF is straightforward with almost any thermodynamic model because: i − 0i Rg T

= ln

∧

xi i i

= ln(xi i )

(10)

where Rg is the universal gas constant, i is the chemical potential of component i at the mixture, and 0i is the chemical potential of pure component i. More details on PS problem

2062


Fig. 7 – Global Success Rate, GSR (plot a) and average NFE (plot b) of CMA-ES, SCE and FA for PEC problems using SC-2 (SCmax = 10, SCmax = 25 and SCmax = 50) and SC-1 (1500 iterations). formulation can be found in Rangaiah (2001). The characteristics of PS problems used in this study are summarized in Table 1.

3.2.

Phase equilibrium calculation problems

A mixture of substances at a given temperature, T, pressure, P and total molar amount may separate into two or more phases. The composition of the different substances is the same throughout a phase but may differ significantly in different phases at equilibrium. If there is no reaction between the different substances, then it is a phase equilibrium problem. There are mainly two approaches for PEC: equation solving approach and Gibbs free energy minimization approach. The former involves solving a set of non-linear equations arising from mass balances and equilibrium relationships. The latter involves the minimization of the Gibbs free energy function. Although the first approach seems to be faster and simple, the solution obtained may not correspond to the global minimum of Gibbs free energy function. Moreover, it needs a priori knowledge of phases existing at equilibrium (Rangaiah, 2001). Classic thermodynamics indicate that minimization of Gibbs free energy is a natural approach for calculating the equilibrium state of a mixture. Hence, this study uses Gibbs free energy minimization for PEC, which was used to determine phase compositions at equilibrium in several works

(McDonald and Floudas, 1995; Rangaiah, 2001; Reynolds et al., 1997; Teh and Rangaiah, 2003). The mathematical formulation involves the minimization of Gibbs free energy subject to mass balance equality constraints and bounds that limit the range of decision variables. In a non-reactive system with c components and phases, the objective function for PEC is

g=

c

nij ln(xij ij ) =

j=1i=1

c

nij ln

j=1i=1

xij ˆ ij i

(11)

where nij , xij , ij , ˆ ij and ϕi are the moles, mole fraction, activity coefficient and fugacity coefficient of component i in phase j, and the fugacity coefficient of pure component, respectively. Eq. (11) must be minimized with respect to nij taking into account the following mass balance constraints:

nij = zi nF

i = 1, ..., c

0 ≤ nij ≤ zi nF

i = 1, ..., c

(12)

j=1

j = 1, ...,

(13)

where zi is the mole fraction of component i in the feed and nF is the total moles in the feed.


2063

Fig. 8 – Global Success Rate, GSR versus iterations for rPEC problems using CMA-ES, SCE and FA with SC-1 (a) stochastic method only and (b) stochastic method combined with local optimization.

To perform unconstrained minimization of Gibbs energy function, one can use new variables instead of nij as decision variables. The introduction of the new variables eliminates the restrictions imposed by material balances, reduces problem dimensionality and the optimization problem is transformed into an unconstrained one. For multi-phase non-reactive systems, new variables ˇij ∈ (0, 1) are defined and employed as decision variables by using the following expressions: ni1 = ˇi1 zi nF

i = 1, ..., c

nij = ˇij

zi nF −

j−1

(14)

nim

i = 1, ..., c;

j = 2, ..., − 1

(15)

m=1

ni = zi nF −

−1

nim

i = 1, ..., c

(16)

m=1

Using this formulation, all trial compositions satisfy the mass balances allowing the easy application of optimization strategies (Bonilla-Petriciolet and Segovia-Hernández, 2010; Srinivas and Rangaiah, 2007a). For Gibbs energy minimization, the number of decision variables ˇij is c ( − 1) for non-reactive systems.

The details of PEC problems used in this study are also in Table 1. In most of the reported studies, PEC problems tested assumed that the number and type of phases are known; such problems are also known as phase split calculations. In this study too, the same assumption is made, and the problems tested are simply referred to as PEC problems.

3.3.

Reactive Phase Equilibrium Calculation problems

In rPEC problems, also known as chemical equilibrium problems, reactions increase the complexity and dimensionality of phase equilibrium problems, and so phase split calculations in reactive systems are more challenging due to non-linear interactions among phases and reactions. The phase distribution and composition at equilibrium of a reactive mixture are determined by the global minimization of Gibbs free energy with respect to mole numbers of components in each phase subject to element/mass balances and chemical equilibrium constraints (Burgos-Solórzano et al., 2004; Seider and Widagdo, 1996). The expressions for Gibbs free energy and its mathematical properties depend on the structure of the thermodynamic equation(s) chosen to model each of the phases that may exist at equilibrium.

2064


Table 6 – Success rate (SR) and number of function evaluations of CMA-ES, SCE and FA for rPEC problems using SCmax with NP of 10D. rPEC no.

SCmax

rPEC-1

6D 12D 24D 6D 12D 24D 6D 12D 24D 6D 12D 24D 6D 12D 24D 6D 12D 24D 6D 12D 24D 6D 12D 24D

rPEC-2

rPEC-3

rPEC-4

rPEC-5

rPEC-6

rPEC-7

rPEC-8


CMA-ES

SCE

SR

NFE

SR

NFE

SR

NFE

100 100 100 100 100 100 100 100 100 0 0 0 98 100 100 98 100 100 25 52 60 100 100 100 80.5

1776 3560 5839 2366 4751 5526 1300 2986 5484 2453 3454 75,001 2092 4507 4860 5158 6972 7967 1319 3095 4246 2245 6151 6867 169,975

100 100 100 99 99 100 99 99 100 8 8 8 75 75 75 95 92 93 34 34 34 100 99 99 76.04

4377 5162 6334 4613 5251 6114 4936 5563 7593 4677 5343 7064 3044 3673 5026 12,249 12,995 14,799 2721 3398 4228 8265 7450 9074 153,949

100 100 100 100 100 100 100 99 100 0 0 0 95 95 89 92 92 92 9 11 13 99 100 100 74.4

1268 1987 3414 1201 1950 3484 1325 1986 3486 1226 1961 3234 877 1368 2332 1554 2403 4012 997 1505 2607 1209 1854 3080 50,317

Recently, Bonilla-Petriciolet et al. (2011) concluded that the constrained Gibbs free energy minimization approach has the advantage of requiring smaller computing time compared to the unconstrained approach, is straightforward and suitable for chemical equilibrium calculations. In summary, for a system with c components and phases subject to r independent chemical reactions, the objective function for rPEC is

Fobj = g −

ln Keq N−1 nref,j

(17)

j=1

where g is given by Eq. (11), ln Keq is a row vector of logarithms of chemical equilibrium constants for r independent reactions, N is an invertible, square matrix formed from the stoichiometric coefficients of a set of reference components chosen from r reactions, and nref is a column vector of moles of each of the reference components. This objective function is defined using reaction equilibrium constants, and it must be globally minimized subject to the following mass balance restrictions (Bonilla-Petriciolet et al., 2011):

(nij − vi N−1 nref,j ) = niF − vi N−1 nref,F

FA

i = 1, . . . , c − r

(18)

j=1

rearranged using the following expression:

ni = niF − vi N−1 (nref,F − nref, ) −

(nij − vi N−1 nref,j )

j=1

i = 1, ..., c − r

(19)

Using Eq. (19), the decision variables for rPEC are c ( − 1) + r mole numbers (nij ). Then, the global optimization problem can be solved by minimizing Eq. (17) with these decision variables and the remaining c − r mole numbers (ni ) are determined from Eq. (19), subject to the inequality constraints ni > 0. In constrained optimization problems, the search space consists of both feasible and infeasible points. For rPEC, feasible points satisfy all the mass balance constraints, Eq. (18), while infeasible points violate at least one of them (i.e., ni < 0 where i = 1, . . ., c − r). The penalty function method is used to solve the constrained Gibbs free energy minimization in reactive systems because it is easy to implement and is considered efficient for handling constraints in the stochastic methods (Bonilla-Petriciolet et al., 2011). For handling these constraints, an absolute value of constraint violation is multiplied with a high penalty weight and then added to the objective function. In case of more than one constraint violation, all constraint violations are first multiplied with the penalty weight, and all of them are added to the objective function. Specifically, the penalty function is given by

where ni,F is the initial moles of component i in the feed. These mass balance equations can be rearranged to reduce the number of decision variables in the optimization problem and to eliminate equality constraints, which are usually challenging for stochastic optimization methods. Thus, Eq. (18) is

−1

Fr =

Fobj

if ∀nij > 0 i = 1, ..., c

Fobj + p

otherwise,

j = 1, ..., ,

(20)

where p is the penalty term whose value is positive. In case of infeasible solutions (i.e., ni < 0), Gibbs free energy function of


2065

Fig. 9 – Global Success Rate, GSR (plot a) and average NFE (plot b) of CMA-ES, SCE, and FA for rPEC problems using SC-2 (SCmax = 6D, SCmax = 12D and SCmax = 24D) and SC-1 (1500 iterations). phase cannot be determined due to the logarithmic terms of the activity or fugacity coefficients. So, the penalty function used for handling infeasible solutions in rPEC is given by Bonilla-Petriciolet et al. (2011). n

p = 10 ·

unf

|ni |

(21)

i=1

where ni is obtained from Eq. (19) and nunf is the number of infeasible mole numbers (i.e., ni < 0 where i = 1, . . ., c − r). In this study, the resulting constrained Gibbs free energy minimization for a reactive system is solved using CMA-ES, SCE and FA algorithms. The details of the rPEC problems are shown in Table 2.

4.

Implementation of the methods

In this study, all the optimization algorithms and thermodynamic models were coded in MATLAB. CMA-ES, SCE and FA codes were obtained from Hansen (2011), Donckels (2011) and Yang (2007), respectively. All the codes were modified to allow for the two stopping criteria used in this study as discussed below. The parameters used for the algorithms were tuned to give sufficiently good results for a problem from each of

the three categories of problem, and then they were fixed for all problems tested in order to compare the robustness of the algorithms; see Table 3 for the parameter values used. Further, n = 10D for the three methods. Altogether, we studied 24 problems consisting of 8 PEC, 8 PS and 8 rPEC problems, whose details can be found in Tables 1 and 2. All these problems are multimodal with number of decision variables ranging from 2 to 10. Each problem was solved 100 times independently with a different random number seed for robust performance analysis. The performances of stochastic algorithms were compared based on Success Rate (SR) and average number of function evaluations (for both global and local searches) in all 100 runs (NFE), for two stopping criteria: SC-1 based on the maximum number of iterations and SC-2 based on the maximum number of iterations without improvement in the best objective function value (SCmax ). SC-2 is an improvementbased stopping criterion. Note that NFE is a good indicator of computational efficiency since function evaluation involves extensive computations in application problems. Further, it is independent of the computer and software platform used, and so it is useful for comparison by researchers. SR is the number of times the algorithm located the global optimum to the specified accuracy, out of 100 runs. A run/trial is considered successful if the best objective function value obtained after

2066


Fig. 10 – Global Success Rate, GSR of CMA-ES, SCE, FA and IDE for all problems using stochastic method combined with local optimization. the local optimization is within 1.0E-5 from the known global optimum. Also, Global Success Rate (GSR) of different algorithms is reported for all the problems. It is defined as

GSR =

np SR

i

np

(22)

i=1

where np is the number of problems and SRi is the individual success rate for each problem. At the end of each run by each stochastic algorithm, a local optimizer was used to continue the search to find the global optimum precisely and efficiently. This is also done at the end of different iteration levels for performance analysis; however, global search in the subsequent iterations is not affected by this. Since all algorithms were implemented in MATLAB, Sequential Quadratic Program (SQP) was chosen as the local optimizer. The best solution at the end of the stochastic algorithm was used as the initial guess for SQP, which is likely to locate the global optimum if the initial guess is in the global optimum region. In the small number of cases when the local optimizer diverged to a larger value of the objective function, the output of the stochastic algorithm was retained. All computations were performed on 64-bit HP Pavilion dv6 Notebook computer with an Intel Core i7-2630QM processor, 2.00 GHz and 4 GB of RAM, which can complete 1344 MFlops (million floating-point operations) for the LINPACK benchmark program that uses the MATLAB “backslash” operator to solve for a matrix of order 500 (Burkardt, 2008).

5.

Results and discussion

5.1.

Performance of algorithms on PS problems

On PS problems, similar tests using the three stochastic algorithms were performed. The results were collected at different iteration levels, starting from 10-iteration level, after local optimization at each of these iteration levels. As expected, GSR of CMA-ES, SCE, and FA for all PS problems using SC-1 improves with increasing number of iterations (Fig. 4a). The highest GSR was 68.9% obtained by the FA algorithm without the local optimization. The selected PS problems were somewhat difficult to optimize, which is reflected in the relatively low GSR without using the local optimizer. At 10 and

25 iterations, SCE obtained the best GSR, but from 50 to 750 iterations, the CMA-ES obtained the best GSR. At the termination of the iterations, FA obtained the highest GSR. Despite relatively faster convergence, SCE has the tendency to be trapped at a local optimum. The reliability of CMA-ES and SCE did not improve much beyond the 100th iteration. However, reliability of FA kept improving until the end. Fig. 4(b) shows the improvements in GSR with the use of a local optimization technique at the end of the stochastic techniques. The best GSR, which was obtained by CMA-ES, increased from 65.9% without local optimization to reach 91.9%. The performance of CMA-ES showed slightly more reliability compared to that of FA. However, GSR of the SCE was about 20% less. Another interesting observation from Fig. 4(b) is that GSR of SCE using local optimization decreased with iterations. Since PS problems are particularly difficult, local optimization techniques may diverge even with an improved initial point obtained by further iterations in the stochastic method. This result was peculiar only to SCE method, which sheds an additional negative light on its use with SQP for the solution of phase stability problems. In stochastic global optimization, it is necessary to use a suitable stopping criterion for the optimization algorithm to stop at the right time incurring least computational resources without compromising reliability of finding the global optimum. Results on the effect of stopping criterion, SC-2 with SCmax = 10, 25 and 50 on CMA-ES, SCE and FA for all PS problems are presented in Table 4; GSR and NFE reported in this table are for stochastic followed by local optimization. They show that, in general, reliability of the algorithm and NFE increase with increasing SCmax . For PS problem 6, CMA-ES obtained the best reliability. For PS problem 2, SCE obtained the best reliability. For PS problems 1, 5, 7 and 8, FA obtained the best reliability. The three techniques obtained 100% reliability for problem 3, while only SCE and FA obtained 100% reliability for problem 4. However, reliability obtained using SC-1 is always higher than that obtained by SC-2, which is shown in Fig. 5(a) that summarizes GSR of the three techniques with the three SC-2 stopping criteria compared with SC-1 for all PS problems. Fig. 5(b) shows the NFE of the three techniques with the four stopping criteria for all PS problems. For SCE, use of SC-2 with maximum 50 iterations gave similar GSR results to using SC-1, yet it required 8 times less NFE. Therefore, it is recommended to use SC-2 with SCE. For CMA-ES and FA,


2067

Fig. 11 – Global Success Rate, GSR (plot a) and average NFE (plot b) of CMA-ES, SCE, FA and IDE for all problems using SC-2 (SCmax = 10, SCmax = 25 and SCmax = 50 except for rPEC problems: SCmax = 6, SCmax = 12 and SCmax = 24) and SC-1 (1500 iterations). use of SC-1 gave slightly more reliable performance but at the expense of much higher NFE as shown in Fig. 5(b). Thus, use of SC-2 for both CMA-ES and FA may still be desirable because of considerable savings in the computational needs, despite the small loss in reliability. Problems 6, 7 and 8 were identified as challenging since at least one of the methods failed to achieve 50% GSR even with the subsequent local optimization. These problems were solved again with a higher population size, and these results will be discussed later. In general, stochastic optimization methods provide only a probabilistic guarantee of locating the global optimum, and their proofs for numerical convergence usually state that the global optimum will be identified in infinite time with probability 1 (Niewierowicz et al., 2003; Rudolph, 1994; Weise, 2008). So, better performance of stochastic methods is expected if more iterations and/or larger population size are used.

5.2.

Performance of algorithms on PEC problems

GSR values for all PEC problems by CMA-ES, SCE and FA algorithms with n of 10D using SC-1 are illustrated in Fig. 6. As expected, GSR improves with increasing number of iterations, particularly at lower iteration levels. After 250 iterations, GSR does not improve significantly for both CMA-ES and SCE. However, GSR kept improving for FA. In general, subsequent iterations without improvement in the results are

waste of computational resources. For example, for stochastic optimization only, GSR of CMA-ES is 88% at 50 iterations; it increases to 90.75% at 250 iterations and stays the same until 1500 iterations. Results in Fig. 6(a) show that CMA-ES has higher reliability at low NFE compared to SCE and FA, for PEC problems when global stochastic optimization only is used. When the performance of the three methods with local optimization at the end of the global search is compared, FA gives the highest reliability with GSR close to 100% at 1500 iterations. The effect of stopping criterion, SC-2 on CMA-ES, SCE and FA algorithms has also been studied on PEC problems. Table 5 summarizes SR and NFE obtained by these algorithms with SCmax = 10, 25 and 50 along with the maximum allowable iterations of 1500 (to avoid indefinite looping), all using n = 10D. For PEC problems 3, 6 and 7, the three algorithms obtained 100% reliability. CMA-ES obtained the best reliability for problem 2, while FA obtained the best reliability for problem 4. Both these methods obtained 100% reliability for problem 8. CMA-ES and SCE obtained the best reliability for problems 1 and 5. Fig. 7 summarizes GSR and NFE of CMA-ES, SCE and FA algorithms with four stopping criteria. We obtained the same conclusion of higher reliability with higher SCmax . It can be observed in Fig. 7(a) that the use of SC-2 gives similar GSR compared to that with SC-1 for CMA-ES and SCE but lower GSR for FA. NFE values in Fig. 7(b) shows that SCE uses most NFE to terminate the global search by SC-2 compared to CMA-ES and FA. In general, SC-2 requires significantly fewer NFE compared to SC-1, which

2068


confirms the need for a good termination criterion. Especially, with SCmax = 50, SR obtained by the algorithms is comparable to that obtained with SC-1 but uses much fewer NFE (Fig. 7). CMA-ES achieved better reliability with fewer NFE when SC-2 was used. When SC-1 was used, FA achieved better reliability with fewer NFE compared to the other two methods.

5.3.

Performance of algorithms on rPEC problems

GSR of CMA-ES, SCE and FA algorithms for all rPEC problems using SC-1 is illustrated in Fig. 8(a), when global stochastic optimization was used without the subsequent use of local optimization. GSR generally improves with increasing number of iterations for these problems as well. The highest GSR is 88.5% obtained by FA. At 50 iterations, CMA-ES obtained best GSR, but from 250 to 1500 iterations, its GSR did not improve. On the other hand, FA obtained better GSR at higher iterations. GSR of SCE was 74% at 250 iterations and remained almost constant until 1500 iterations. GSR of FA remained very low until 750 iterations when it climbed steadily until it reached the highest GSR of 88.5% at 1500 iterations. In short, when comparing the stochastic optimizations without the use of subsequent local optimization, FA is more reliable than SCE, which itself is more reliable than CMA-ES. When the results of the stochastic global optimization after 1500 iterations followed by local optimization are compared (Fig. 8b), FA comes in top in terms of reliability with GSR of 96.5% compared to 82.6% for CMA-ES and 76.5% for SCE. The performance of CMA-ES and SCE were very good since both have reached very close to the final GSR after only 50 iterations although no significant improvement was obtained in subsequent iterations. In short, FA is the most reliable whereas CMA-ES is the most effective to find the global optimum in small number of iterations. Results obtained on the effect of stopping criteria on the three algorithms using SC-2 with SCmax = 6D, 12D and 24D, for rPEC problems are summarized in Table 6. Note that SCmax values used for each rPEC problem were those used by BonillaPetriciolet et al. (2011) so that their and the present results can be compared. Table 6 shows that reliability of the algorithm increases with SCmax but requires more NFE. For rPEC-1, 2, 3 and 8, CMA-ES, SCE and FA algorithms obtained 100% SR. For rPEC problem 4, FA obtained the best reliability followed by SCE and CMA-ES. This is considered one of the difficult problems, and will be separately discussed later. CMA-ES obtained the best reliability for problems 5, 6 and 7, with FA close behind. Among the three algorithms, NFE required by FA is much lower than that by CMA-ES and SCE. As shown in Table 6, total NFE required by FA for all rPEC problems is 50,317 compared to 169,975 by CMA-ES and 153,949 by SCE. The average GSR is 74.4% for FA, 80.7% for CMA-ES and 76.0% for SCE. Thus, these results again show that SCE is less reliable than CMA-ES and FA. Fig. 9 shows the performance of GSR and NFE of CMA-ES, SCE and FA with different stopping criteria for rPEC problems. Again, we conclude that the higher the SCmax , the better the reliability of the algorithm is except for FA, and that the use of SC-2 gives similar GSR compared to SC-1 (Fig. 9a) except for FA. The use of SC-2 for CMA-ES and SCE will bring about similar reliability compared to the use of SC-1 but with much higher efficiency. This is not the same conclusion that can be drawn for FA as SC-1 gave the highest reliability (96.5%) as shown in Fig. 9(a).

5.4. Comparison with the reported performance of other stochastic methods Recently, Zhang et al. (2011b) reported the performance of Unified Bare-Bones Particle Swarm Optimization (UBBPSO), Integrated Differential Evolution (IDE) and IDE without Tabu List and radius (IDE N). They also analyzed the performance of UBBPSO, IDE and IDE N, and compared them with other published results such as classical PSO with Quasi-Newton Method (PSO-CQN), classical PSO with Nelder–Mead Simplex Method (PSO-CNM), Simulated Annealing (SA), Genetic Algorithm (GA), and Differential Evolution with Tabu List (DETL). All these stochastic algorithms were run 100 times independently, and at the end of every run, a deterministic local optimizer was activated. Zhang et al. (2011b) reported that IDE gave better performance across the entire spectrum of problems. Hence, it is sufficient to compare the performance of CMA-ES and FA with IDE for the three categories of problems, with the different stopping criteria. Fig. 10 shows the average GSR of CMA-ES, SCE and FA for the 24 problems as compared with the average GSR of IDE, at different iterations. CMA-ES shows the best convergence rate as its average GSR reaches about 87.9% after 50 iterations only. The reliability of IDE is superior to all evaluated algorithms until the 750th iteration. At larger iterations, the GSR of FA was the highest, reaching 95% at the 1500th iteration compared to 92.8% for IDE. Fig. 11 shows the average GSR of CMA-ES, SCE and FA for the 24 problems as compared with the average GSR of IDE, when SC-2 was used; IDE is superior over the other three algorithms in terms of its reliability as shown in Fig. 11a. GSR of IDE for SCmax = 50 for PS and PEC and SCmax = 24D for rPEC is 92.0% as opposed to GSR of CMA-ES, which was 88.6%. Its NFE, however, was significantly more than that of CMA-ES: 19,600 and 11,700 (Fig. 1b). Since FA showed better reliability than CMA-ES and SCE, the following modification was made to enhance its reliability and efficiency. Its reliability supersedes that of IDE at large number of iterations. The moves of the fireflies in FA algorithm are not affected by the position of the best firefly (xbest). This strategy may prevent FA algorithm in falling into local optima and leaves open to continue the search for all available minima. However, this strategy also results in slower convergence toward the global optimum in comparison with other techniques. The modification that we attempted was based on a suggestion by Yang (2007) to improve the efficiency by adding an extra term εi (xi − g* ) to the move equation, Eq. (4). In the case of this Modified Firefly Algorithm (MFA), the global optimum becomes a factor in determining how all fireflies move. The parameter determines how significant the contribution of the location of the best firefly to the direction of the moves of the other fireflies. In our case, was taken as 1. We tested the performance of MFA algorithm in the most challenging phase stability and equilibrium problems. In particular, nine of the twenty-four problems were identified to be the most challenging as their success rates were less than 50%, for at least one of the methods when the first stopping criterion was applied. These problems were PS 6, 7 and 8; PEC 4, 6, 7 and 8; and rPEC 4 and 7. In the first instance, the solution of these problems were attempted with a larger population size (20D) using the three algorithms under evaluation. No significant impact of the population size on the performance of all the three algorithms was found. Fig. 12 shows the average GSR for all 9 problems, when the four algorithms were used with 20D population size up to 750 iterations. The


2069

Fig. 12 – Global Success Rate (GSR) versus iterations for the most challenging problems using CMA-ES, SCE, FA and MFA with SC-1 (a) stochastic method only and (b) stochastic method combined with local optimization. comparison of GSR, when global stochastic optimization is used alone as depicted in Fig. 12(a), shows that CMA-ES is still the most effective algorithm to find the global in 50 iterations. MFA increases the effectiveness of FA at 250 iterations, and it is the most reliable method with the highest GSR at 750th iteration. However, the addition of the local optimization at the end of the stochastic optimization considerably improves the reliability of all techniques as shown in Fig. 12(b). In this case, FA is the most reliable technique and CMA-ES is the most effective. MFA using the local optimizer did not improve FA performance in this case. Other types of improvements that may be considered are the inclusion of other meta-heuristics such as Tabu List or Simulated Annealing, to improve the efficacy of the search or the modification of the move equation by accepting or rejecting the moves with certain probability. These ideas will be attempted in future research projects.

6.

Conclusions

In this study, three stochastic global optimization algorithms, namely, CMA-ES, SCE and FA have been evaluated for solving the challenging phase stability, and phase and chemical equilibrium problems. Performance at different iteration levels and the effect of stopping criterion have also been analyzed. CMAES was found to be the most effective algorithm to find the

global minimum reliably and accurately in 50 iterations. FA was found to be the most reliable technique across different problems tried but requires relatively more computational effort. The modification of a term that relates the firefly move to the location of the global optimum did not significantly improve the efficiency of FA. The stopping criterion, SC-1 gives slightly better reliability than SC-2 at the expense of computational resources, and the use of SCmax can significantly reduce the computational effort for solving PEC, rPEC and PS problems without significantly affecting the reliability of the stochastic algorithms studied. Comparison of the performance of CMAES, SCE and FA with the results in Zhang et al. (2011b) shows that IDE is more reliable among the algorithms tested.

References Avami, A., Saboohi, Y., 2011. A simultaneous method for phase identification and equilibrium calculations in reactive mixtures. Chem. Eng. Res. Des. 89, 1901–1908. Bayer, P., Finkel, M., 2007. Optimization of concentration control by evolution strategies: formulation, application, and assessment of remedial solutions. Water Resour. Res. 43, W02410. Bonilla-Petriciolet, A., Rangaiah, G.P., Segovia-Hernández, J.G., 2011. Constrained and unconstrained Gibbs free energy minimization in reactive systems using genetic algorithm and

2070


differential evolution with tabu list. Fluid Phase Equilib. 300, 120–134. Bonilla-Petriciolet, A., Segovia-Hernández, J.G., 2010. A comparative study of particle swarm optimization and its variants for phase stability and equilibrium calculations in multicomponent reactive and non-reactive systems. Fluid Phase Equilib. 289, 110–121. Bonilla-Petriciolet, A., Vázquez-Román, R., Iglesias-Silva, A.G., Hall, K.R., 2006. Performance of stochastic global optimization methods in the calculation of phase stability analyses for nonreactive and reactive mixtures. Ind. Eng. Chem. Res. 45, 4764–4772. Burgos-Solórzano, G.I., Brennecke, J.F., Stadtherr, M.A., 2004. Validated computing approach for high-pressure chemical and multiphase equilibrium. Fluid Phase Equilib. 219, 245–255. Burkardt, J., 2008. The LINPACK Benchmark. http://people.sc. fsu.edu/∼jburkardt/m src/linpack bench/linpack bench.html (accessed 08.09.11). Cheung, A., Adjiman, C.S., Kolar, P., Ishikawa, T., 2002. Global optimization for clusters of flexible molecules–solvent–solute interaction energy calculations. Fluid Phase Equilib. 194, 169–183. Donckels, B., 2011. Optimization Algorithms. http://biomath. ugent.be/∼brecht/index.html (accessed 08.07.11). Duan, Q., Sorooshian, S., Gupta, V., 1992. Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resour. Res. 28, 1015–1031. Gandomi, A.H., Yang, X.-S., Alavi, A.H., 2011. Mixed variable structural optimization using Firefly Algorithm. Comput. Struct. 89, 2325–2336. Giannakouris, G., Vassiliadis, V., Dounias, G., 2010. In: Konstantopoulos, S., Perantonis, S., Karkaletsis, V., Spyropoulos, C., Vouros, G. (Eds.), Experimental Study on a Hybrid Nature-Inspired Algorithm for Financial Portfolio Optimization Artificial Intelligence: Theories, Models and Applications. Springer, Berlin/Heidelberg, pp. 101–111. ˜ Hansen, N., 2006. In: Lozano, J., Larranaga, P., Inza, I., Bengoetxea, E. (Eds.), The CMA Evolution Strategy: A Comparing Review – Towards a New Evolutionary Computation. Springer, Berlin/Heidelberg, pp. 75–102. Hansen, N., 2011. The CMA Evolution Strategy. http://www.lri.fr/∼hansen/cmaesintro.html (accessed 08.09.11). Hansen, N., Niederberger, A.S.P., Guzzella, L., Koumoutsakos, P., 2009. A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans. Evol. Comput. 13, 180–197. Hansen, N., Ostermeier, A., 2001. Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9, 159–195. Harding, S.T., Floudas, C.A., 2000. Phase stability with cubic equations of state: a global optimization approach. AIChE J. 46, 1422–1440. Hohm, T., Zitzler, E., 2007. Multicellular pattern formation: parameter estimation for ODE-based gene regulatory network models. IEEE Eng. Med. Biol. 28, 52–57. Jalali, F., Seader, J.D., 1999. Homotopy continuation method in multi-phase multi-reaction equilibrium systems. Comput. Chem. Eng. 23, 1319–1331. Jalali, F., Seader, J.D., Khaleghi, S., 2008. Global solution approaches in equilibrium and stability analysis using homotopy continuation in the complex domain. Comput. Chem. Eng. 32, 2333–2345. Jastrebski, G.A., Arnold, D.V., 2006. Improving evolution strategies through active covariance matrix adaptation. In: IEEE Congress on Evolutionary Computation, Vancouver, BC, pp. 2814–2821. McDonald, C.M., Floudas, C.A., 1995. Global optimization for the phase and chemical equilibrium problem: application to the NRTL equation. Comput. Chem. Eng. 19, 1111–1141. Michelsen, M.L., 1982. The isothermal flash problem. Part I. Stability. Fluid Phase Equilib. 9, 1–19.

Nichita, D.V., Garcia-Sanchez, F., Gomez, S., 2008. Phase stability analysis using the PC-SAFT equation of state and the tunneling global optimization method. Chem. Eng. J. 140, 509–520. Nichita, D.V., Gomez, S., Luna-Ortiz, E., 2004. Multiphase equilibria calculation by direct minimization of Gibbs free energy using tunneling global optimization method. J. Can. Petrol. Technol. 43, 13–16. Nichita, D.V., Gomez, S., Luna, E., 2002a. Multiphase equilibria calculation by direct minimization of Gibbs free energy with a global optimization method. Comput. Chem. Eng. 26, 1703–1724. Nichita, D.V., Gomez, S., Luna, E., 2002b. Phase stability analysis with cubic equations of state by using a global optimization method. Fluid Phase Equilib. 194, 411–437. Niewierowicz, T., Escarela-Perez, R., Campero-Littlewood, E., 2003. Hybrid genetic algorithm for the identification of high-order synchronous machine two-axis equivalent circuits. Comput. Electr. Eng. 29, 505–522. Rahman, I., Das, A.K., Mankar, R.B., Kulkarni, B., 2009. Evaluation of repulsive particle swarm method for phase equilibrium and phase stability problems. Fluid Phase Equilib. 282, 65–67. Rangaiah, G.P., 2001. Evaluation of genetic algorithms and simulated annealing for phase equilibrium and stability problems. Fluid Phase Equilib. 187–188, 83–109. Rangaiah, G.P., 2010. Stochastic Global Optimization: Techniques and Applications in Chemical Engineering. World Scientific, Singapore. Reynolds, D., Mulholland, A.J., Gomatam, J., 1997. Calculation of chemical and phase equilibria via simulated annealing. J. Math. Chem. 22, 25–27. Ros, R., Hansen, N., 2008. A simple modification in CMA-ES achieving linear time and space complexity. In: 10th International Conference on Parallel Problem Solving from Nature, Dortmund, Germany, pp. 296–305. Rossi, C.C.R.S., Berezuk, M.E., Cardozo-Filho, L., Guirardello, R., 2011. Simultaneous calculation of chemical and phase equilibria using convexity analysis. Comput. Chem. Eng. 35, 1226–1237. Rossi, C.C.R.S., Cardozo-Filho, L., Guirardello, R., 2009. Gibbs free energy minimization for the calculation of chemical and phase equilibrium using linear programming. Fluid Phase Equilib. 278, 117–128. Rudolph, G., 1994. Convergence analysis of canonical genetic algorithms. IEEE Trans. Neural Net. 5, 96–101. Sayadi, M.K., Ramezanian, R., Ghaffari-Nasab, N., 2010. A discrete firefly meta-heuristic with local search for makespan minimization in permutation flow shop scheduling problems. Int. J. Ind. Eng. Comput. 1, 1–10. Seider, W.D., Widagdo, S., 1996. Multiphase equilibria of reactive systems. Fluid Phase Equilib. 123, 283–303. Srinivas, M., Rangaiah, G.P., 2006. Implementation and evaluation of random tunneling algorithm for chemical engineering applications. Comput. Chem. Eng. 30, 1400–1415. Srinivas, M., Rangaiah, G.P., 2007a. Differential evolution with tabu list for global optimization and its application to phase equilibrium and parameter estimation problems. Ind. Eng. Chem. Res. 46, 3410–3421. Srinivas, M., Rangaiah, G.P., 2007b. A study of differential evolution and tabu search for benchmark, phase equilibrium and phase stability problems. Comput. Chem. Eng. 31, 760–772. Srinivas, M., Rangaiah, G.P., 2010. Differential evolution with tabu list for global optimization: evalution of two versions on benchmark and phase stability problems. In: Rangaiah, G.P. (Ed.), Stochastic Global Optimization: Techniques and Applications in Chemical Engineering. World Scientific, Singapore, pp. 465–504. Sun, A.C., Seider, W.D., 1995. Homotopy-continuation method for stability analysis in the global minimization of the Gibbs free energy. Fluid Phase Equilib. 103, 213–249.


Teh, Y.S., Rangaiah, G.P., 2002. A study of equation-solving and gibbs free energy minimization methods for phase equilibrium calculations. Trans. IChemE 80, 745–759. Teh, Y.S., Rangaiah, G.P., 2003. Tabu search for global optimization of continuous functions with application to phase equilibrium calculations. Comput. Chem. Eng. 27, 1665–1679. Tessier, S.R., Brennecke, J.F., Stadtherr, M.A., 2000. Reliable phase stability analysis for excess Gibbs energy models. Chem. Eng. Sci. 55, 1785–1796. Vrugt, J.A., Gupta, H.V., Bastidas, L.A., Bouten, W., Sorooshian, S., 2003a. Effective and efficient algorithm for multiobjective optimization of hydrologic models. Water Resour. Res. 39, 1214. Vrugt, J.A., Gupta, H.V., Bouten, W., Sorooshian, S., 2003b. A Shuffled Complex Evolution Metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters. Water Resour. Res. 39, 1201. Wakeham, W.A., Stateva, R.P., 2004. Numerical solution of the isothermal, isobaric phase equilibrium problem. Rev. Chem. Eng. 20, 1–56. Wasylkiewicz, S.K., Ung, S., 2000. Global phase stability analysis for heterogeneous reactive mixtures and calculation of reactive liquid–liquid and vapor–liquid–liquid equilibria. Fluid Phase Equilib. 175, 253–272.

2071

Weise, T., 2008. Global Optimization Algorithm: Theory and Application. http://www.it-weise.de/projects/book.pdf (accessed 09.08.11). Wilson, J.W., Schlup, P., Lunacek, M., Whitley, D., Bartels, R.A., 2008. Calibration of liquid crystal ultrafast pulse shaper with common-path spectral interferometry and application to coherent control with a covariance matrix adaptation evolutionary strategy. Rev. Sci. Instrum. 79, 033103. Xu, G., Brennecke, J.F., Stadtherr, M.A., 2002. Reliable computation of phase stability and equilibrium from the SAFT equation of state. Ind. Eng. Chem. Res. 41, 938–952. Yang, X.-S., 2011. Firefly Algorithm. http://www.mathworks.com/ matlabcentral/fileexchange/29693-firefly-algorithm (accessed on 08.09.11). Yang, X.S., 2007. Firefly Algorithm, Nature-Inspired Metaheuristic Algorithms. Wiley Online Library, pp. 79–90. Zhang, H., Bonilla-Petriciolet, A., Rangaiah, G., 2011a. A review on global optimization methods for phase equilibrium modeling and calculations. Open Thermodyn. J. 5 (Suppl. 1-M7), 71–92. Zhang, H., Fernández-Vargas, J.A., Rangaiah, G.P., Bonilla-Petriciolet, A., Segovia-Hernández, J.G., 2011b. Evaluation of integrated differential evolution and unified bare-bones particle swarm optimization for phase equilibrium and stability problems. Fluid Phase Equilib. 310, 129–141.