Additive Sequential Evolutionary Design of Experiments B. Balasko, J. Madar, and J. Abonyi Department of Process Engineering, University of Veszprem Veszprem P.O.box 158.,H-8201 HUNGARY
[email protected] www.fmt.vein.hu/softcomp
Abstract. Process models play important role in computer aided process engineering. Although the structure of these models are a priori known, model parameters should be estimated based on experiments. The accuracy of the estimated parameters largely depends on the information content of the experimental data presented to the parameter identification algorithm. Optimal experiment design (OED) can maximize the confidence on the model parameters. The paper proposes a new additive sequential evolutionary experiment design approach to maximize the amount of information content of experiments. The main idea is to use the identified models to design new experiments to gradually improve the model accuracy while keeping the collected information from previous experiments. This scheme requires an effective optimization algorithm, hence the main contribution of the paper is the incorporation of Evolutionary Strategy (ES) into a new iterative scheme of optimal experiment design (AS-OED). This paper illustrates the applicability of AS-OED for the design of feeding profile for a fed-batch biochemical reactor.
1
Introduction
Process models play important role in computer aided process engineering since most of advanced process monitoring, control, and optimization algorithms rely on the process model. Unfortunately often some of the parameters of these models are not known a priori, so they must be estimated from experimental data. The accuracy of these parameters largely depends on the information content of the experimental data presented to the parameter identification algorithm [1]. Optimal Experiment Design (OED) can maximize the confidence on model parameters through optimization of the input profile of the system. For parameter identification of different dynamic systems and models, this approach has been already utilized in several studies [2]- [6]. OED is based on an iterative algorithm where the optimal conditions of the experiments or the optimal input of the system depends on the current model, which parameters were estimated based on the result of the previously designed experiment. Consequently, experiment design and parameter estimation are solved iteratively, and both of them are based on nonlinear optimization of cost functions. L. Rutkowski et al. (Eds.): ICAISC 2006, LNAI 4029, pp. 324–333, 2006. c Springer-Verlag Berlin Heidelberg 2006
Additive Sequential Evolutionary Design of Experiments
325
That means in practice, the applied nonlinear optimization algorithms have great influence on the whole procedure of OED, because for nonlinear dynamical models the design of the experiment is a difficult task. This problem is usually solved by several gradient-based methods e.g. nonlinear least squares method or sequential quadratic programming. Several gradient computation methods are described in [7]. In [8] extended maximum likelihood theory is applied for optimizing the experiment conditions. As a population-based effective optimization algorithm, this paper proposes the application of evolutionary strategy (ES). In [9] it has been shown that ES results in more satisfactory parameter values than classical sequential quadratic programming (SQP) or nonlinear least square (NLS) methods. On the presented application example, a fed-batch biochemical reactor, some results were already shown in e.g. [10], but these results assume that model parameters or model structure are perfectly known. The drawback of OED is that the experiment design uses only information from the current experiment and parameter identification relies only on one experiment while there are previous experiment information available. In this paper an additive sequential evolutionary OED was proposed, which uses the results of the previous experiment designs and parameter estimations. The aim of this paper is to illustrate the usefulness of AS-OED, independently from the model structure, hence a simplified monotonic Monode model was used. The paper is organized into five sections: the first, second and third sections review the theoretical background of experiment design, additive design and evolutionary strategy, respectively. The fourth section presents an application example and finally conclusions are given in the fifth section.
2
Classical Optimal Experiment Design
The case study considered in this paper belongs to the following general class of process models: dx(t) = f (x(t), u(t), p) dt y(t) = g(x(t)),
(1) (2)
where u is the vector of the manipulated inputs, y is the output (vector), x represents the state of the system, where p denotes the model parameters. The p parameters are unknown and should be estimated based on data taken from experiments. The estimation of these parameters is based on the minimization of the square error between the output of the system and the output of the model: texp 1 min Jmse (u(t), p) = (eT (t) · Q(t) · e(t))dt) (3) p texp t=0 e(t) = y˜(u(t)) − y(u(t), p)
(4)
326
B. Balasko, J. Madar, and J. Abonyi
in which y˜(u(t)) is the output of the system for a certain u(t) input profile, and y(u(t)) is the output of the model for the same u(t) input profile with p parameters, Q is a user supplied square weighting matrix that represents the variance measurement error. The basic element of the experiment design methodology is the Fisher information matrix F , which combines information on (i) the output measurement error and (ii) the sensitivity of the model outputs y with respect to the model parameters: texp 1 ∂y ∂y F (p0 , u(t)) = ( (u(t), p)|p=p0 )T · Q(t) · ( (u(t), p)|p=p0 )dt (5) texp t=0 ∂p ∂p The sensitivities are calculated based on the partial derivatives of the model parameters. As the true parameters p∗ are unknown during experiment design, the derivatives are calculated near to the so-called nominal parameters po , which can be given by some initial guess, extracted from literature or estimated from the previous experiments. The optimal design criterion aims the minimization of a scalar function of the F matrix. several optimal criterion exist, we present D-optimal and E-optimal criterion suggested by Bernaerts et al. [1]: – D-optimal criterion minimizes the determinant of the covariance matrix and thus minimizes the volume of the joint confidence region: JD = min(det(F )) u(t)
(6)
– E-optimal criterion minimizes the condition number of F , i.e. the ratio of the largest to the smallest eigenvalue of the Fisher matrix: JE = min u(t)
λmax (F ) λmin (F
(7)
If the po nominal parameters are far from the p∗ true parameters, convergence cannot be guaranteed after a first optimal design. So an iterative design scheme is needed to obtain convergence from po to p∗(Fig.2(a)). Both the parameter estimation and the experiment design steps of this iterative scheme represent a complex nonlinear optimization problem, hence the effectiveness of the applied optimization algorithms have great influence on the performance of the whole procedure. The classical solution is to use nonlinear least squares (NLS) algorithm for parameter estimation eq. (3), and sequential quadratic programming (SQP) for the experiment design eq. (7). In this paper the application of evolutionary strategy (ES) is proposed for this purpose, which can be used for non-linear optimization problems. The main drawback of the classical iterative approach is that the Fisher information matrix F contains information only about the current experiment regardless of the information content of the previous experiments, and the parameters are identified from this experiment. It is useful to include the information from previous experiments within the parameter estimation and the experiment design steps, too.
Additive Sequential Evolutionary Design of Experiments
(a)
327
(b)
Fig. 1. Design schemes of the classical (a) and the Additive Sequential (b) OED method
3
Additive Sequential Experiment Design
The paper proposes a new additive sequential experiment design approach. The goal of the additive sequential design is to include all available information within the iterative design scheme, so the Fisher information matrix is calculated for the current AND the previous experiments. It means that in every iteration step, the new experiment is designed considering the previous experiments too: F i (p0i , ui (t)) =
i
1
k=1
texp,k
texp,k
( t=0
∂y (uk (t), p)|p=p0i )T · ∂p
·Q(t) · (
(8)
∂y (uk (t), p)|p=p0i )dt ∂p
where ui is the input vector of the ith experiment with texp,i experiment time. In this way, the parameters might be identified very effectively since every available experimental data is used to design a set of new informative (and independent) experiments. The iterative scheme of this Additive Sequential Experiment Design is shown in Fig.2(b). The main advantage of this new approach is that in this way the experiment design becomes more robust and effective.
4
Evolutionary Strategy
This paper proposes the application of evolutionary strategy (ES) instead of the utilization of NLS and SQP. ES is a stochastic optimization algorithm that
328
B. Balasko, J. Madar, and J. Abonyi
uses the model of natural selection [11]. The advantage of ES is that it has proved particularly successful in problems that are highly nonlinear, that are stochastic, and that are poorly understood [11]. The design variables in ES are represented by n-dimensional vector xj = [xj,1 , . . . , xj,n ]T ∈ Rn , where xj represents the jth potential solution, i.e. the j-th the member of the population. The mutation operator adds zj,i normal distributed random numbers to the design variables: xj,i = xj,i + zj,i , where zj,i = N (0, σj,i ) is a random number with σj,i standard deviation. To allow for a better adaptation to the objective functions’s topology, the design variables are accompanied by these standard deviation variables, which are so-called strategy parameters. Hence the σ j strategy variables control the step size of standard deviations in the mutation for j-th individual. So an ES-individual aj = (xj , σ j ) consists of two components: the design variables xj = [xj,1 , . . . , xj,n ]T and the strategy variables σ j = [σj,1 , . . . , σj,n ]T . Before the design variables are changed by mutation operator, the standard deviations σ j are mutated using a multiplicative normally distributed process: (t)
(t−1)
σj,i = σj,i
exp(τ N (0, 1) + τ Ni (0, 1)) .
(9)
The exp (τ N (0, 1)) is a global factor which allows an overall change of the mutability, and the exp (τ Ni (0, 1)) allows individuals to change of their mean step sizes σj,i . So τ and τ parameters can be interpreted as global learning rates. Schwefel suggests to set them as [12]: 1 1 τ = √ , τ = √ . 2n 2 n
(10)
Throughout this work discrete recombination of the object variables and intermediate recombination of the strategy parameters were used: xj,i = xF,i σj,i = (σF,i +
xM,i
(11)
σM,i ) /2 ,
or
(12)
where F and M denotes the parents, j is index of the new offspring. The Evolutionary Strategy function in Matlab environment has three important parameters that one has to adjust carefully in order to find the most reliable solution with the least computation time: (i) number of generations, (ii) size of a population in a generation and (iii) number of individuals with the best fit values, which appear unchanged in the next generation.
5
Application Example
This paper illustrates the applicability of the proposed approach for the design of a feeding profile of a fed-batch biochemical reactor with monotonic kinetics. Prior knowledge on the microbial dynamics is generally lacking, the parameters of the applied kinetic model are usually determined by using a method
Additive Sequential Evolutionary Design of Experiments
329
which minimizes the difference between the measured response of the reactor and the predicted response of the model. The following equation describes the mass balance of the reactor: ⎡ ⎤ ⎡ ⎤ ⎤ ⎡ S −σX CS,in d ⎣ ⎦ ⎣ X = μX ⎦ + ⎣ 0 ⎦ u (13) dt V 1 0 where S is the mass of the substrate [g], X is the mass of the micro-organism [g DW], V is the volume [L], u is the inlet flowrate [L/h], CS,in = 500 g/L is the substrate concentration in the inlet feed, σ = μ/YX/S + m is the specific substrate consumption rate, where, YX/S = 0.47 g DW/g, m = 0.29 g/g DW h, while μ [1/h] is the kinetic rate. The initial conditions: S(t = 0) = 500 g, X(t = 0) = 10.5 g DW, V (t = 0) = 7 L. The maximum volume is Vmax = 10 L, and the maximum inlet flowrate is umax = 0.3 L/h. The following monotonic kinetic (Monode) model of the μ kinetic rate was considered: μM (CS ) = μmax
CS KS + CS
(14)
The system was simulated with μmax = 0.1 1/h and KS = 1 g/L as real parameter values, i.e. p∗ = [0.1 1]T . The goal was to find the unknown parameters of the model: μmax and KS (the other parameters were assumed to be accurately known). The μmax and KS parameters were estimated by optimization, see eq. (3). It was assumed that only the substrate concentration CS is measured. Conse˜ quently, in this application example, the system output is y˜ = VS[g] ˜ [L] , which was generated by the simulation of (13) and (14). We applied the iterative OED methodology to design the feeding profile u(t) with E-optimal criterion equations (5) (8) and (7). It has been shown in [9] that ES seems to be the best optimization algorithm for this non-linear problem. The aim of the paper is to prove that an additive (memory-effect type) evolutionary experiment design can accord better results for this application. The parameter error (Err) was defined as the sum of the absolute difference between the ending output parameter results and the ’true’ μmax =0.1 1/h and KS =1 g/L values, normalized with the true values(Eq.(15)) where npar denotes the number of optimized parameters. p0 − p∗ Err = p∗ npar
(15)
i=1
Both the classical and the additive evolutionary ED was initialized with the pinit = [0.15 0.5] parameter vector (50 percent error for each), 3 independent runs were made with 9 iteration during each of them. The length of one experiment
330
B. Balasko, J. Madar, and J. Abonyi
was 20 h, the sample time was 1 h. Ten input profile values were optimized with linear extrapolation between them. The parameters of ES function were adjusted to 40 generations with a population size of 25 individuals and the best 10 individuals appear unchanged in the next generation. With all these experimental parameters the runtime of the algorithm is ca. 2 minutes. Note, that our task here is not to maximize the system output, i.e. the biomass concentration but to find an input profile which maximizes the precision of the parameter estimation. For comparison, Fig.2 (a) shows a manually selected input step profile and the response of the system. The ES optimized parameters for this profile are pmanual = [0, 0997 0, 8968] with an Err value of 0.1053. The parameter space does not converge to a global minima if this input profile is selected as shown in Fig.2 (b). Identification cost function is defined as the square error of the model output with respect to its parameters around the p0 nominal parameters.
1.1
0.2
L/h
0.15 0.1
1.05
0 0
5
10 Time (h)
15
20
KS/K0S
0.05
1
150 100 g/L
0.95
50 0 0
5
10 Time (h)
(a)
15
20
0.9 0.9
0.95 μ
1 /μ0
max
1.05
1.1
max
(b)
Fig. 2. Manually selected input profile and system output (a) and contour plot of the identification cost (b)
The results are shown in Table 1., Fig.3. and Fig.4. Table 1. shows the sum of Err values of the ending parameters for the 3 independent runs, and their mean values; pinit is shown on the figure as the 0th experiment. As one can see, the additive sequential evolutionary experiment design results in better parameters at the end of the experiments and almost in every iteration step, it is more robust and computationally just a bit more expensive. The iterative sequential design has its uncertainty in the output because of the large searching space and the lack of information from previous experiments. As Fig.3. shows, in additive design, less iteration cycles and evolutionary computation would result also in reliable parameters, hence computation time excess could be spared without any quality loss in parameter estimation. E-optimized input profiles are presented on Fig.4. for the runs with the best ending parameter estimations.
Additive Sequential Evolutionary Design of Experiments
331
OED results − Err and J values
Err values
1
0.5
0 0
2
4
2
4
6
8
10
6
8
10
mse
0.01
J
0.005
0 0
(a) OED results − Err and J values
Err values
2 1.5 1 0.5 0 0
2
4
6
8
10
2
4
6
8
10
Jmse
0.1
0.05
0 0
(b) OED results − Err and J values
Err values
1
0.5
0 0
2
4
6
8
10
2
4
6
8
10
0.01
J
mse
0.015
0.005 0 0
(c)
Fig. 3. Err and Jmse values (dotted line with ’o’ marker for classical and continous line with ’x’ marker for AS-OED) for the three independent runs
332
B. Balasko, J. Madar, and J. Abonyi Table 1. Sum of error of the ending parameters and mean values 1st run 2nd run 3rd run Err Sequential Ev. Design 0.1283 0.0094 0.0256 0.0544 Additive Seq.Ev.Design 0.0091 0.0449 0.0785 0.0441
Additive Sequential ED − Feed profile and biomass 0.4
0.3
0.3 L/h
L/h
Sequential ED − Feed profile and biomass 0.4
0.2 0.1 0 0
0.1 5
10 Time (h)
15
0 0
20
150
5
10 Time (h)
15
20
5
10 Time (h)
15
20
20 15 g/L
g/L
100 50 0 0
0.2
10 5
5
10 Time (h)
(a)
15
20
0 0
(b)
Fig. 4. E-Optimized input profiles and system outputs generated by (a) classical sequential OED and (b) Additive Sequential OED
6
Conclusion
The aim of this paper was to develop a robust and effective additive evolutionary experiment design strategy and to demonstrate its usefulness on an industrial example. This approach uses Evolutionary Strategy for both the experiment design and the parameter identification steps of the iterative scheme. Additive Sequential Evolutionary Experiment Design uses information from the previous experiments and although it is computationally just a little more expensive than the ES aided classical Optimal Experiment Design because Fisher matrices and the input profiles of the previous experiments already exist so they do not need to be calculated again. Computation time could be lowered by decreasing the number of iteration cycles what does not worsen parameter estimation quality. It has been proven that this new approach results in more reliable and confidential values of the identified parameters.
References 1. BERNAERTS, K., SERVAES, R.D., KOOYMAN, S. and VERSYCK, K.J. and VAN IMPE, J.F.: Optimal temperature design for estimation of the Square Root model parameters: parameter accuracy and model validity restrictions, Int. Jour. of Food Microbiology, 73, 145-157, 2002 2. BERNAERTS, K., VAN IMPE, J.F.: Optimal dynamic experiment design for estimation of microbial growth kinetics at sub-optimal temperatures: Modes of implementation, Simulation Modelling Practice and Theory, 13, 129138, 2005
Additive Sequential Evolutionary Design of Experiments
333
3. VERSYCK, K.J., BERNAERTS, K., GEERAERD, A.H., VAN IMPE, J.F.: Introducing optimal experimental design in predictive modeling: A motivating example, International Journal of Food Microbiology 51, 3951, 1999 4. BERNAERTS, K., GYSEMANS, K.P.M., MINH, T.N., VAN IMPE, J.F.: Optimal experiment design for cardinal values estimation: guidelines for data collection, International Journal of Food Microbiology, 100, 153 165, 2005 5. CHEN, B.H., BERMINGHAM, S., NEUMANN, A.H., KRAMER, H.J.M., and ASPREY, S.P.: On the Design of Optimally Informative Experiments for Dynamic Crystallization Process Modeling, Ind. Eng. Chem. Res. 43, 4889-4902, 2004 6. COHN, D.A.: Neural Network Exploration Using Optimal Experiment Design, Neural Networks, Vol. 9, Issue 6, 1071-1083, 1996 7. POINT, N. VANDE WOUWER, A., and REMY, M.: Practical Issues in Distributed Parameter Estimation: Gradient Computation and Optimal Experiment Design, Control Engineering Practice, Vol. 4, Issue 11, 1553-1562, 1996 8. A.F. EMERY, A.F., NENAROKOMOV, A.V., FADALE, T.D.: Uncertainties in parameter estimation: the optimal experiment design, International Journal of Heat and Mass Transfer, 43, 3331-3339, 2004 9. MADAR, J., SZEIFERT, F. AND ABONYI, J.: Evolutionary Strategy in Iterative Experiment Design, the Hungarian Journal of Industrial Chemistry, Special Issue on Recent advances in Computer Aided Process Engineering 10. SMETS, I.Y.M, VERSYCK, K.J.E and VAN IMPE, J.F.: Optimal control theory: A generic tool for identification and control of (bio-)chemical reactors, Annual Reviews in Control, 26, 57-73, 2002 11. MADAR, J. and ABONYI, J.: Evolutionary Algorithms, Chapter 2.10. in Instrument Engineers’ Handbook, 4th Edition, Volume 2 - Process Control, editor: B. Liptak, CRC Press, 2005 12. SCHWEFEL, H.: Numerical optimization of computer models. Wiley, Chichester, 1995
