Time delay system identification based on ...

Time delay system identification based on optimization approaches Ahlem SASSI, Sa¨ıda BEDOUI, Kamel ABDERRAHIM Numerical Control of Industrial Processes, National School of Engineers of Gabes, University of Gabes, St Omar Ibn-Khattab, 6029 Gabes, Tunisia. E-mail: [email protected], [email protected], [email protected] Abstract—In this paper, the problem of estimating the time delay and dynamic parameters of monovariable time delay discrete is addressed. This problem involves both the estimation of the time delay and the dynamic parameters from input-output data. In fact, we have considered our previous method which consists in minimizing a quadratic criterion using either the gradient method or the Levenberg-Marquardt method. The used criterion is deduced from a formulation allowing to define the time delay and the dynamic parameters in the same estimated vector and to build the corresponding observation vector. In order to improve the performance of this approach, we advocate the use of the quasi-Newton approach based on the BroydenFletcher-Goldfarb-Shanno (BFGS) algorithm. Simulation results are presented to illustrate the performance of the new solution.

I.

I NTRODUCTION

This paper deals with the problem of identification of time delay systems. This problem consists in building mathematical models of processes using observed input-output data [1]. It has received a great attention in the last years, since the delay time is a physical phenomenon which arises in most control loops of industrial processes [2],[30]. In fact, the delay may be an inherent property of the system such as in the transport processes or in the accumulation of time lags in a large number of low order systems connected in series. It may also be introduced by the time response of control loop devices such as sensors, actuators, controllers, networks, etc. The time delay identification problem involves both the estimation of the dynamic parameters and the delay time. This is one of the most difficult problems that represent an area of research where considerable work has been done in the last years. The proposed methods for the identification of time delay systems can be classified in different ways depending on : the type of used time (continuous or discrete), the form of solution (linear or nonlinear), the type of data processing (batch or iterative or recursive), etc [2]-[4],[7]-[11],[13],[15],[16],[18],[22],[23]. Only the iterative identification approach for monovariable discrete time delay systems is considered in this paper. This approach has found applications in many fields, such as signal processing and control [1],[19],[20]. The early iterative method [2] developed in the literature consists in using the parametrisation approach. It is based on two main steps. The first consists in using the least square algorithm to estimate the parameters of a system with a known delay which is greater than the real delay. The second allows to deduce the delay from the first zero coefficients of the numerator. In practice, it is difficult or rather impossible to have zero coefficients from experimental c 2013 IEEE 978-1-4799-2228-4/13/$31.00

data. Thus, a threshold must be selected. This selection leads to poor results when the input-output measurements are noisy. Another method is proposed in [16]. It is based on the use of the least square approach to identify the parameters assuming that the delay is known. Then, it estimates the delay either by maximizing the correlation function, or by minimizing the quadratic error. This method assumes that the domain range of the delay is known. We cite also our approach which consists in defining the time delay and the dynamic parameters in the same estimated vector and in building the corresponding observation vector [22],[23]. This formulation is then used to propose a method to identify these systems by minimizing a quadratic criterion using either the gradient method or the LevenbergMarquardt method. The gradient algorithm usually converges very slowly [25],[32],[33]. In fact, this algorithm cannot be recommended. The Levenberg-Marquardt switches between the GaussNewton approach and the gradient method [32],[33]. It allows us to find a solution even if the initial conditions are far to the final minimum. However, it finds only a local minimum, not a global minimum. In this paper, we propose the use the quasiNewton approach for solved the obtained problem. The idea of quasi-Newton method consists in approximating the inverse Hessian [29],[32],[33]. This represents the important advantage of the quasi-Newton method since no matrix inverse has to be performed. It can be mentioned that the Broyden-FletcherGoldfarb-Shanno (BFGS) formula is more often adopted to approximate the inverse Hessian. This choice can be justified by the following reasons: no requirement for second order derivatives, very fast convergence and recommended for medium sizes problems (100 parameters). This paper is organized as follows. Section II presents the model and its assumptions. Section III recalls the gradient and the Levenberg-Marquardt methods for the identification of time delay systems. In section IV, we propose a new solution based on the quasi-Newton BFGS algorithm. Sections IV and V illustrate the performance of this method a simulation example and an experimental validation carried out on a level control process II.

M ODEL AND

ASSUMPTIONS

We consider the problem of estimating the time delay and the parameters of the following system: A(q−1 )y(k) = q−d B(q−1 )u(k) + v(k)

(1)

where u(k) and y(k) are the system input and output, respectively, v(k) is a white noise and d is the time delay. A(q−1 )

ones in the same estimated vector θˆG which we associate the corresponding observation vector φG [22],[23] such as:

and B(q−1 ) are two polynomials with the unit backward shift operator q−1 [i.e. q−1 y(k) = y(k − 1)] having the orders na and nb respectively, defined by: A(q

−1

na

) = 1 + ∑ ai q

−i

= 1 + a1 q

−1

+ . . . + a na q

ˆ θˆG (k) = [aˆ1 (k), . . . , aˆna (k), bˆ 1 (k), . . . , bˆ nb (k), d(k)]

−na

−1

nb

B(q ) = ∑ bi q

−i

= b1 q

−1

+ . . . + b nb q

     φG (k) =      

−nb

i=1

Using these expression, equation (1) can be rewritten as: nb

na

y(k) = − ∑ ai y(k − i) + ∑ bi u(k − i − d) i=1

(2)

i=1

The model described above will be studied under the following assumptions: A1. A2. A3. A4. A5.

(7)

∂ J(k) ∂ θˆG (k − 1)

(8)

∂ J(k) ∂ e(k) ∂ y(k) ˆ = e(k) = − e(k) ∂ θˆG (k − 1) ∂ θˆG (k − 1) ∂ θˆG (k − 1)

(9)

Therefore, we get:

∂ J(k) = −φG (k)e(k) ˆ ∂ θG (k − 1)

(10)

METHODS

1 1 J(k) = e2 (k) = [y(k) − y(k)] ˆ 2 2 2

Thus, the gradient algorithm is given by:

θˆG (k) = θˆG (k − 1) − µ (k)φG (k)e(k)

ˆ

θˆG (k) = θˆG (k − 1) − µ (k)[H(k) + λ I]−1

(3)

H(k)

nb

y(k) ˆ = θ (k − 1)φ (k)

=

∂ 2 J(k) ∂ θˆG (k − 1)∂ θˆG (k − 1)T

The second derivatives of the criterion (3) was approximated in [22] using the small residual algorithm in [25] as:

(4)

The estimated output y(k) ˆ can be rewritten as: ˆT

∂ J(k) ˆ ∂ θG (k − 1)

where λ is a scalar and H(k) the Hessian of J(k) as:

y(k) ˆ = − ∑ aˆi (k − 1)y(k − i) + q−d(k−1) ∑ bˆ i (k − 1)u(k − i) i=1

(11)

where µ (k) is a positive scalar called step size. Then, we have proposed another method which minimizes the same criterion (3) using the Levenberg-Marquardt search:

where y(k) ˆ is the estimate of y(k) at iteration k defined by:

i=1

          

Deriving the quadratic criterion J(k) by the estimated parameter vector θˆ (k − 1), we obtain the following expression:

Several approaches have been proposed in the litterature of the simultaneous identification of time delay and dynamic parameters of time delay systems[4]-[6],[9],[10],[12],[14],[16][18],[22]-[24],[26]. These methods can be classified in different categories of solution such as the least square solution, etc. Only the optimization solution is considered in this paper. It consists in minimizing a quadratic criterion in term of the prediction error e(k) given by:

na

ˆ

q−d(k−1) u(k − nb) ˜ nb ˆ − ∑i=1 bi (k − 1)q−d(k−1) ∆u(k − i)

θˆG (k) = θˆG (k − 1) − µ (k)

Problem statement: The goal is to develop an algorithm to estimate, simultaneously, the time delay d and the parameters (a1 , . . . , ana , b1 , . . . , bnb ). E XISTING



where: ∆u(k − i) = u(k − i) − u(k − i − 1) (see appendix). We have proposed methods which consist in minimizing the quadratic criterion (3) using, in a first time the gradient search. It leads to the following iterative algorithm of computing θˆG (k) as follows:

The polynomials A(q−1 ) and B(q−1 ) are coprime. The orders na and nb of the model are known. The input sequence u(k) is a stationary ergodic process, independent of v(k) and is persistently exciting. The disturbance v(k) is a sequence of independent, identically distributed random variable with zero mean and finite variance σv The input, the output and the noise are causal, i.e. u(k) = 0, y(k) = 0 and v(k) = 0 for k ≤ 0.

III.

−y(k − 1) .. . −y(k − na) ˆ q−d(k−1) u(k − 1) .. .



i=1

(6)

H(k) ≈ φG (k)φGT (k) (5)

(12)

And using the expression (10), the Levenberg-Marquardt algorithm is given by:

where θˆ (k) the estimated parameter vector given by:

θˆG (k) = θˆG (k − 1) + µ (k)[φG (k)φGT (k) + λ I]−1 φG (k)e(k) (13)

θˆ (k − 1) = [aˆ1 (k − 1), . . . , aˆna (k − 1), bˆ 1 (k − 1), . . . , bˆ nb (k − 1)]T φ (k) = [−y(k − 1), .., −y(k − na), u(k − 1 − d), .., u(k − nb − d)]

Note: If λ is too high comparing with the Hessian, this algorithm converges to the gradient one. Else, it converges to the Newton algorithm.

In order to identify simultaneously the time delay and the parameters of a delay system, we have proposed to define these 474

IV.

P ROPOSED

V.

APPROACHES

We present now two examples to illustrate the effectiveness of proposed approaches. Therefore, the main purpose of these simulation examples is to compare the performance of the proposed approaches with that of gradient and LevenbergMarquardt approaches.

The BFGS algorithm is one of the most efficient quasiNewton methods for unconstrained optimization [27],[28]. It is by far the most popular quasi-Newton update formula. This algorithm was proposed by Broyden, Fletcher, Goldfarb, and Shanno individually. It allows to compute θˆG as follows:

θˆG (k) = θˆG (k − 1) − µ (k)[H(k)]−1

∂ J(k) ˆ ∂ θG (k − 1)

R ESULTS

A. Example 1

(14)

The simulations are performed under the following conditions:

Using (10), this expression can be rewritten as follows: (15)

•

The system to be identified is persistently excited by a pseudo random binary sequence (PRBS).

The main idea is to approximate the Hessian H(k) by B(k) as follows:

•

The estimation starts with zero initial values for the parameters and the time delay.

W (k)W T (k) B(k − 1)V (k)V T (k)BT (k − 1) + W T (k)V (k) V T (k)B(k − 1)V (k) (16) where B(k) is called the Broyden and W (k) is the gradient variation given by:  ∂ J(k) − ∂ θˆ∂ J(k)   ∂ θˆG (k) G (k−1) W (k) = (17) µ (k)   −1 V (k) = [B(k − 1)] φG (k)e(k)

•

The additive noise v(k) is a white noise sequence with zero mean and constant variance σv2 = 10−4.

•

The evaluation of the parameter error is considered to study the performance of each studied method.

θˆG (k) = θˆG (k − 1) + µ (k)[H(k)]−1φG (k)e(k)

B(k) = B(k − 1)+

δ= •

, we opt for deriving J(k) In order to compute the term ∂∂θˆJ(k) G (k) ˆ by the expression (15) of θG (k). Using the series expansion of this derivation, the second order approximation is given by:

∂ J(k) ∂ θˆG (k)

=

∂ J(k) ∂ θˆG (k − 1)

+

∂ 2 J(k) (µ (k)[H(k)]−1 ϕ (k)e(k)) ∂ θˆG (k − 1)∂ θˆGT (k − 1)

kθ − θˆ k kθ k

The simulated example is a second-order plus time delay process defined by: G(q−1 ) = q−d

b1 q−1 + b2q−2 1 + a1q−1 + a2 q−2

where a1 = −0.2, a2 = 0.36, b1 = 0.5, b2 = 0.35 and d = 2. The simulation results are given in Fig.1, Fig.2, Fig.3 and Fig.4. 1

0

Using (7) and (12), The obtained formula of W (k) is:

−0.2 0.5

(ϕ (k)ϕ T (k))(µ (k)[H(k)]−1 ϕ (k)e(k)) W (k) = µ (k)

−0.4

(18)

0

a1 a ˆ1

−0.6 −0.8

T

200

400

k

600

0

200

k

400

600

1

1

Data: u, y, θG0 . N: the measurement number. choose µ (k). begin For k = 1 : N Construct the observation vector φG (k) using (7) Compute: V (k) = [H(k − 1)]−1ϕ (k)e(k) T −1 ϕ (k)e(k)) W (k) = (ϕ (k)ϕ (k))(µ (k)[H(k)] µ (k) T

−0.5 0

The proposed algorithm, deduced from equations (14)-(18), is summarized in algorithm 1.

a2 a ˆ2

0.5

b2

0

ˆb2

0.5

b1 ˆb1

−0.5 −1

0 0

200

400

k

600

0

200

k

400

600

2

d 1

d˜

0 0

T

200

k

400

600

Fig. 1. Real and estimated evolutions of the time delay and parameters

(k)W (k) (k)V (k)H (k−1) H(k) = H(k − 1) + W − H(k−1)V W T (k)V (k) V T (k)H(k−1)V (k) θˆG (k) = θˆG (k − 1) + µ (k)[H(k)]−1ϕ (k)e(k)

using the gradient method.

Algorithm 1: BFGS Algorithm

475

0

1

verge to their true values

0.5

•

−0.5 0

a1 a ˆ1 −1 0

200

400

600

a2 a ˆ2

−0.5 0

200

400

k 1

B. Example 2 We consider a level control process. This process is composed of a set of eight identical tanks illustrated in Fig.5. It is easy to remark that the output of tank (j), q j represent the input of tank (j+1), q j+1 . Using this notation and assuming that the levels in the tanks are in an operating point, we can approximate the dynamic behaviour of the level in each tank by a simple linear model:

b2 ˆb2

0.6

0.5

600

k

0.8

0.4 0

b1 ˆb1

0.2

−0.5 0

200

400

0

600

0

k

200

400

600

k 2 1

d d˜

S

0 0

The best results in term of speed of convergence are given by the proposed method.

200

400

dh j = q j − q j+1 dt

(19)

600

k

q j+1 = Kh j

Fig. 2. Real and estimated evolutions of the time delay and parameters

where S is the section of the tank and K is a constant representing the properties of the tank.

using the Levenberg-Marquardt method. 1

0 −0.2

0.5 −0.4 0

a1 a ˆ1

−0.6 −0.8 0

200

400

600

−0.5 0

k

0.6

a2 a ˆ2 200

400

600

k

1

0.4

b2

0.2

ˆb2

0.5

b1 ˆb1

0

0

−0.2 0

200

400

0

600

200

400

600

k

k 2

d 1

d˜

0 0

200

400

600

k

Fig. 3. Real and estimated evolutions of the time delay and parameters

using the BFGS method. Fig. 5.

Level control process.

2 Gradient Levenberg-Marquardt BFGS

1.5

Thus, the transfer function of each tank is given by: H j (p) =

1

1 q j (p) K(T p + 1)

where

0.5

T=

(20)

S K

0 0

Fig. 4.

100

200

300

400

500

Then, we can deduce that the transfer function relating q1 (t) with the level in the tank eight h8 (t) by:

600

The parameter estimation errors δ .

H8 (p) =

Based on the results presented in figures Fig.1, Fig.2, Fig.3 and Fig.4, we observe that: •

Ke q1 (p) (T p + 1)8

where Ke = f rac1k

The estimated parameters of the three methods con476

(21)

The above system is a connexion in series of eight first order systems. Consequently, it can be approximated by a first order system with delay [30] defined by: H8 (p) = e−τ p

Ka q1 (p) 1 + Ta p

(22)

b1 q−1 + b2q−2 1 + a1q−1

0.8

−0.2

0.6

−0.4

0.4

−0.6

0.2

−0.8

0 0

500 k

1000

0

500 k

0

500 k

1000

0.8

ˆb2

0.6 0.4

(23)

1

0.5

0.2

d˜ 0

As a numerical example, we consider the step response of this systems with T = 1, K = 1 and sampling time Te = 2s (Fig.6). In this case, the transfer function representing the dynamic behaviour of the global system is: 1 H(p) = (p + 1)8

0 0

500 k

1000

1000

Fig. 7. Real and estimated evolutions of the time delay and parameters

using the gradient method.

(24)

Fig.6 shows the presence of an important time delay. It

0

0.8

−0.2

0.6

−0.4

0.4

1.2

−0.6

1

−0.8

0.2

ˆb1

a ˆ1 0 0

0.8

500 k

1000

0

500 k

0

500 k

1000

0.8

0.6 0.4

0.6

0.2

0.4

0

0.2

10

20

30

40

50

60

1

ˆb2

0.5

d˜ 0

Fig. 6.

ˆb1

a ˆ1

The delay τ , is introduced by the accumulation of time lags of the first order systems and Ta and Ka are the constant time and the constant of the time delay system. The discrete transfer function of (22) is given by: H(q−1 ) = q−d

0

0 0

The step response.

is therefore possible to approximate the model of a very high-order complex dynamic process with a simplified model defined by (22). In that way, we apply the three methods (gradient, Levenberg-Marquardt and BFGS) in order to estimate the time delay and the parameters. The obtained results are presented in Fig.7, Fig.8 and Fig.9 which show that the BFGS method gives the best results in terms of speed of convergence and precision.

500 k

1000

Fig. 8. Real and estimated evolutions of the time delay and parameters

using the Levenberg-Marquardt method. 0

0.8

−0.2

0.6

−0.4

0.4

−0.6

VI.

1000

0.2

ˆb1

a ˆ1

C ONCLUSION

−0.8

0 0

In this paper, we have considered the problem of simultaneous identification of the time delay and dynamic parameters of monovariable time delay discrete. Indeed, we have recalled our previous approach which consists in defining the time delay and the dynamic parameters in the same estimated vector and in building the corresponding observation vector. Then, this formulation is used to develop a method to identify the delay and the dynamic parameters by minimizing a quadratic criterion using either the gradient method or the LevenbergMarquardt method. To improve the performance of this approach, we have advocated the use of the quasi-Newton BFGS algorithm to solve the obtained system. Simulation and experimental examples are presented to illustrate the effectiveness of the proposed method and to compare their performance in terms of convergence speed and precision.

500 k

1000

0

500 k

1000

0.8 1

ˆb2

0.6 0.4

0.5

0.2

d˜ 0

0 0

500 k

1000

0

500 k

1000

Fig. 9. Real and estimated evolutions of the time delay and parameters

using the BFGS algorithm.

A PPENDIX A A PPROXIMATION OF Ln(q) 477

The shift operator and the backward difference are given by respectively (25) and (26): qu(k) = u(k + 1)

(25)

∆u(k) = u(k) − u(k − 1)

(26)

[16]

∆u(k) = (1 − q−1)u(k)

(27)

[17]

We can infer the identity between the shift operator and the backward difference [21], then:

[18]

Therefore:

∆ = 1 − q−1 ⇔ q−1 = 1 − ∆ Which we apply the Logarithm function as follows:

[19]

Ln(q) = −Ln(1 − ∆) Using the series expansion of Ln(1 − ∆), the first order approximation of the shift operator is given by:

[20]

Ln(q) = ∆ = 1 − q−1

[21]

(28)

[22]

ACKNOWLEDGMENT This work was supported by the Ministry of the Higher Education and Scientific Research in Tunisia.

[23]

R EFERENCES [1] [2] [3]

[4]

[5]

[6]

[7]

[8]

[9] [10]

[11]

[12]

[13]

[14]

[15]

[24]

T. Soderstrom and P. Stoica, System identification, Prentice Hall International, Series in systems and control engeneering, 1989. J.P. Richard, Time delay systems: an overview of some recent advances and open problems, Automatica, vol. 39, pp. 1667-1694, 2003. V.B. Kolmanovskii, S.I. Niculescu and K. Gu, Delay effects on stability: A survey, In proceeding of the 38th IEEE Conference on Decision and Control, vol.2, pp. 1993-1998, 1999. X.M. Ren, A.B. Rad, P.T. Chan and W.L. Lo, Online identification of continuous-time systems with unknown time delay, IEEE Transactions on Automatic Control, vol.50, no.9, pp. 1418-1422, 2005. S.V. Drakunov, W. Perruquetti, J.P. Richard and L.Belkoura, Delay identification in time-delay systems using variable structure observers, Annual Reviews in Control, vol.30, no.2, pp .143-158, 2006 Y. Orlov, L. Belkoura, J.P. Richard and M. Dambrine, Adaptive identification of linear time-delay systems, International Journal on Robust and non-linear Control, vol.13, no.9, pp. 857-872, 2003. M. De la Sen, Robust adaptive control of linear time-delay systems with point time varying delays via multiestimation, Applied mathematical modelling, vol.33, no.2, pp. 959-977, 2009. Q.G. Wang and Y. Zhang, Robust identification of continuous systems with dead time from step responses, Automatica, vol.37, no.3, pp. 377390, 2001. H. Kurz and W. Goedecke, Digital parameter adaptive control of process with unknown dead time, Automatica, vol.17, no.1, pp. 245-252, 1981. P.J. Gawthrop and M.T. Nihtila, Identification of time delays using polynomial identification method, Systems and Control Letters, vol.5, no.4, pp. 267-271, 1985. S.W. Sung and I.B. Lee, Prediction error identification method for continuous time processes with time delay, Industrial and Engeneering Chemistry Research, vol.40, pp. 5743-5751, 2001. O. Gomez, Y. Orlov and I.V. Kolmanovsky, Online identification of SISO linear time-invariant delay systems from output measurements, Automatica, vol.43, no.12, pp. 323-331, 2006. S. Ahmed, B. Huang and S.L. Shah, Parameter and delay estimation of continuous time models using a linear filter, Journal of Process Control, vol.43, no.12, pp. 5743-5751, 2001. A.B. Rad, W.L. Lo and K.M. Tsang, Simultaneous online identification of rational dynamics and time delay: A correlation based approach, IEEE Transactions on Control Systems Technology, vol.11, no.6, pp. 957-959, 2003. W. Gao, M.L. Zhou, Y.C. Li and T. Zhang, An adaptive generalized predictive control of time varying delay system, In proceeding of the second International Conference on Machine Learning and Cybernetics, pp. 878-881, 2004.

[25] [26] [27] [28] [29] [30] [31] [32] [33]

478

A. Elnaggar, G.A. Dumont and A.L. Elshafei, New method for delay estimation, In proceeding of the 29th IEEE Conference on Decision and Control, vol.3, pp. 1929-1930, 1990. L. Xie, Y. Liu and H. Yang, Gradient based and least squares based iterative algorithms for Matrix equations AXB + CX T D = F, Applied Mathematics and Computation, vol.217, pp. 2191-2199, 2010. B. Bao, Y. Xu, J. Sheng and R. Ding, Least squares based iterative parameter estimation algorithm for multivariable controlled ARMA system modelling with finite measurement data, Mathematical and Computer Modelling, vol.53, no.910, pp. 1664-1669, 2011. D. Wang and F. Ding, Least squares based and gradient based iterative identification for Wiener nonlinear systems, Signal Processing, vol.91, no.5, pp. 1182-1189, 2011. D. Wang and F. Ding, Input-output data filtering based recursive least squares identification for CARARMA systems, Digital Signal Processing, vol.20, no.4, pp. 991-999, 2010. L. Lapidus and G.F. Pinde, Numerical Solution of Partial Diffenrential Equations in Science and Engeneering, John Wiley and sons,1982. Bedoui. S, Ltaief. M and Abderrahim. K, New results on discrete time delay systems. International Journal of Automation and Computing, vol.9, no.6, pp. 570-577, December 2012. S. Bedoui , M. Ltaief and M. Abderrahim, A new recursive algorithm for simultaneous identification of discrete time delay systems. In 16th IFAC Symposium on system identification, vol.16, no.1, pp. 1502-1507, 2012. Z. Yang, Z. Sun and C. Andersen, Nonlinear FOPDT Model Identification for the Superheat Dynamic in a Refrigeration System, Proceeding of IEEE, pp. 634-639, 2011. O. Nelles, Non linear system identification: from classical approach to neural networks and fuzzy models. Editions Springer 2001. E. Walter et L. Pronzato, Identification de modles paramtriques partir de donnes exprimentales. Editions MASSON, Paris 1994. QI. Chunhong, K.A. Gallivan and P.A. Absil, Recent Advances in Optimization and its Applications in Engineering. pp. 183-192, 2010. J. Nocedal and S.J. Wright, Numerical Optimization. Editions Springer 1999. D. Yu-Hong, Convergence properties of the BFGS algorithm. Chinese Academy of Sciences 2002 J.E. Normey-Rico and E.F. Camacho, Control of Dead-time Processes Editions Springer, London 2007. R.O. Duda et P.E. Hart, Pattern Classification and Scene Analysis. New York : Wiley, 1973. R. Fletcher, Practical Methods of Optimization, Wiley, 2000. P.E. Gill, W. Murray, M.H. Wright: Practical Optimization, Academic Press, London. 1981.