Combining inflation-free and iterative ensemble Kalman filters for

Nonlin. Processes Geophys., 19, 383–399, 2012 www.nonlin-processes-geophys.net/19/383/2012/ doi:10.5194/npg-19-383-2012 © Author(s) 2012. CC Attribution 3.0 License.

Nonlinear Processes in Geophysics

Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems M. Bocquet1,2 and P. Sakov3 ´ Paris-Est, CEREA joint laboratory Ecole des Ponts ParisTech and EDF R&D, France Paris Rocquencourt research center, France 3 Nansen Environment and Remote Sensing Center, Bergen, Norway

1 Universit´ e 2 INRIA,

Correspondence to: M. Bocquet ([email protected]) Received: 22 January 2012 – Revised: 30 April 2012 – Accepted: 22 May 2012 – Published: 25 June 2012

Abstract. The finite-size ensemble Kalman filter (EnKF-N) is an ensemble Kalman filter (EnKF) which, in perfect model condition, does not require inflation because it partially accounts for the ensemble sampling errors. For the Lorenz ’63 and ’95 toy-models, it was so far shown to perform as well or better than the EnKF with an optimally tuned inflation. The iterative ensemble Kalman filter (IEnKF) is an EnKF which was shown to perform much better than the EnKF in strongly nonlinear conditions, such as with the Lorenz ’63 and ’95 models, at the cost of iteratively updating the trajectories of the ensemble members. This article aims at further exploring the two filters and at combining both into an EnKF that does not require inflation in perfect model condition, and which is as efficient as the IEnKF in very nonlinear conditions. In this study, EnKF-N is first introduced and a new implementation is developed. It decomposes EnKF-N into a cheap two-step algorithm that amounts to computing an optimal inflation factor. This offers a justification of the use of the inflation technique in the traditional EnKF and why it can often be efficient. Secondly, the IEnKF is introduced following a new implementation based on the Levenberg-Marquardt optimisation algorithm. Then, the two approaches are combined to obtain the finite-size iterative ensemble Kalman filter (IEnKF-N). Several numerical experiments are performed on IEnKF-N with the Lorenz ’95 model. These experiments demonstrate its numerical efficiency as well as its performance that offer, at least, the best of both filters. We have also selected a demanding case based on the Lorenz ’63 model that points to ways to improve the finite-size ensemble Kalman filters. Eventually, IEnKF-N could be seen as the first brick of an efficient ensemble Kalman smoother for strongly nonlinear systems.

1

Introduction

Let us first introduce two recently developed and complementary ensemble Kalman filters. 1.1

An ensemble Kalman filter without the intrinsic need for inflation

The finite-size ensemble Kalman filter (EnKF-N), introduced in Bocquet (2011), relies on the statistical modelling assumption that the ensemble used in the analysis is a collection of samples of the prior probability density function p(x|x1 , x2 , . . . , xN ), where xk is the k-th member of the Nmember forecast ensemble. The idea behind EnKF-N is that the empirical moments of the ensemble, x=

N 1 X xk , N n=1

P=

N 1 X (xk − x) (xk − x)T , N − 1 k=1

(1)

do not necessarily match the mean xb and the error covariance matrix B of the prior probability density function (pdf). In the large ensemble size limit N → ∞, we expect that they coincide. But for any finite N , sampling errors may induce a discrepancy. An effective form for the prior pdf was proposed. It is the result of an integration over all possible xb and B. The effective prior pdf which is advocated to be used in the analysis step of the ensemble Kalman filter is: − N 2 p(x|x1 , x2 , . . . , xN ) ∝ (x − x) (x − x)T + εN (N − 1)P ,

(2)

rather than the Gaussian prior

Published by Copernicus Publications on behalf of the European Geosciences Union & the American Geophysical Union.

384

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter

  1 p(x|x1 , x2 , . . . , xN ) ∝ exp − (x − x)T P−1 (x − x) 2

(3)

which is implicitly assumed in traditional EnKF. Notation |X| designates the determinant of matrix X. Note that the determinant and the inverse operators in formula Eqs. (2) and (3) are meant to operate in the vector space spanned by the anomalies {xk − x}k=1,...,N . Otherwise they would often be zero for the determinant and undefined for the inverse. The constant εN depends on the assumptions made to derive the prior. Two classes of filter that depend on these assumptions have been introduced. For the first, both xb and Pb are uncertain, and εN = 1 + N1 . For the second, it is assumed that the empirical mean x is a fine approximation of xb . In this case: εN = 1. The analysis step of EnKF-N follows from Bayes rule: p(x|y, x1 , x2 , . . . , xN ) ∝ p(y|x)p(x|x1 , x2 , . . . , xN ) ,

(4)

where y ∈ Rd is the observation vector at a given update step. It is assumed that the observational likelihood is Gaussian:   1 (5) p(y|x) ∝ exp − (y − H (x)) T R−1 (y − H (x)) , 2 where H is the observation operator and R is the observation error covariance matrix. In Bocquet (2011), the filter was seen as a deterministic filter and, in particular, the focus was on its ensemble transform variant. It is assumed that the system state x we would like to estimate, can be decomposed into x = x + Aw ,

(6)

where A = [x1 − x, . . . , xN − x] is the matrix of the anomalies. The weights w ∈ RN are the (redundant) coordinates of x in ensemble space. The filter follows the same algorithm as the ensemble transform Kalman filter of Hunt et al. (2007), except that the optimal vector of weight wa at the analysis step is obtained from the minimisation of the cost function 1 Je(w) = (y − H (x + Aw)) T R−1 (y − H (x + Aw)) 2  N  + ln εN + wT w , 2

(8)

where H is the Jacobian matrix of H as is done in the traditional scheme, is based on the Hessian of Je in Eq. (7)
εN + wT w IN − 2wwT T −1 e = (HA) R HA + N H , (9)
2 εN + wT w where IN is the identity matrix in ensemble space. The complete algorithm is recalled in algorithm 1. In this algorithm, U is an arbitrary orthogonal matrix in ensemble space that preserves the ensemble mean (Sakov and Oke, 2008). Algorithm 1 Finite-size ensemble Kalman filter. Require: The forecast ensemble {xk }k=1,...,N , the observations y and error covariance matrix R 1: Compute the mean x and the anomalies A from {xk }k=1,...,N . 2: Compute Y = HA, δ = y − Hx 3: Find the minimum: n  o wa = min (δ − Yw)T R−1 (δ − Yw) + N ln εN + wT w w

 −1
ε +wT w I −2w wT 4: Compute a = YT R−1 Y + N N a a TN
2 a a 5: Compute xa = x + Awa . 6: Compute Wa = ((N − 1)a )1/2 U 7: Compute xak = xa + A Wak

εN +wa wa

The filter was, in particular, tested on the Lorenz ’95 toymodel (Lorenz and Emmanuel, 1998), using the root-meansquare error of the analysis as a criterion. In this context, EnKF-N was used without inflation which is usually required to stabilise or optimise the performance of such system (Anderson and Anderson, 1999). As a consequence the burden of tuning inflation was avoided. Yet, EnKF-N (εN = 1 type) systematically levelled or slightly outperformed EnKF with optimally tuned inflation, for a large range of time intervals between updates. The extra numeral cost of using EnKF-N instead of EnKF was deemed marginal, especially for highdimensional systems. This statement will be confirmed and clarified in the present article.

(7)

where H is the observation operator and R is the observation error covariance matrix. The tilde symbol indicates that the function is defined in ensemble space. This variational analysis step has similarities with that of Zupanski (2005); Harlim and Hunt (2007). The other modification is the computation of the posterior error covariance matrix, which, instead of being based on the Hessian in ensemble space

Nonlin. Processes Geophys., 19, 383–399, 2012

e = (HA) T R−1 HA + (N − 1)IN , H

1.2

An ensemble Kalman filter for strongly nonlinear systems

The extended Kalman filter propagates the error covariance matrix from time t1 to time t2 , based on a tangent linear model that has been computed at the previous analysis step (0) around the analysis x1 . However, when new observations y2 are assimilated at time t2 , a new estimation of the state at time t1 conditional on the future observations y2 can be obtained. For strongly nonlinear systems and large update www.nonlin-processes-geophys.net/19/383/2012/

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter intervals, this new estimation may significantly differ from (0) x1 , and the tangent linear model should be corrected and obtained at this new state. One would usually solve for the optimal analysed state at time t2 by minimizing J2 (x2 ) =

1 (y2 − H (x2 )) T R−1 (y2 − H (x2 )) 2    1 f f + x2 − x2 P−1 x − x 2 2 2 , 2

(10)

where P2 is the background error covariance at time t2 . Instead, it is preferable, in strongly nonlinear conditions, to minimise for the (re-)analysed state x1 conditional on the future observations: 1 J1 (x1 ) = (y2 − H (M(x1 ))) T R−1 (y2 − H (M(x1 ))) 2    1 (0) (0) + x1 − x1 P−1 (11) x − x , 1 1 1 2 where M is the transition model from time t1 to time t2 , and P1 is the background error covariance at time t1 obtained by a forecast from the assimilation step prior to t1 and does not depend on future observations. This is equivalent to solving a one-lag extended Kalman smoother. Even with a linear observation operator, cost function J1 is difficult to solve when the model is nonlinear, since as opposed to J2 , J1 is nonquadratic, so that this cost function needs to be iteratively optimised. This approach has been suggested by (Wishner et al., 1969; Jazwinski, 1970; Tarantola, 2005). Cost function J1 can be, for instance, optimised using a Newton approach (p+1)

x1

(p)

(p)

−1 = x1 − H(p) ∇J1 (x1 ) ,

(12)

where p is the iteration index and where the gradient and the Hessian are respectively given by   (p) (p) ∇J1 = −MT(p) HT R−1 y2 − H M(x1 )   (p) (0) +P−1 x1 − x1 (13) 1 T T −1 H(p) = P−1 1 + M(p) H R HM(p) ,

(14) (p)

where M(p) is the tangent linear model computed at x1 . More recently it has been suggested to use a similar approach, but with the EnKF (Gu and Oliver, 2007). Kalnay and Yang (2010) suggested repeating the assimilation cycle by applying the ensemble transform calculated at t2 to the ensemble from the previous iteration at t1 until the best fit to observations is obtained. One advantage of the EnKF framework is that the use of the tangent linear model and its adjoint is replaced with the use of the ensemble. It has been properly formalized as the iterative ensemble Kalman filter (IEnKF) in the framework of the deterministic filters and tested on the Lorenz ’63 and ’95 by Sakov et al. (2012). At the cost of additional iterations and, hence, of additional propagations www.nonlin-processes-geophys.net/19/383/2012/

385

of the ensemble, this IEnKF was shown to significantly outperform EnKF especially for large time interval between updates. Two versions of the filter were put forward. The first one mimics the use of the tangent linear model by the propagation of a rescaled ensemble, a bundle of trajectories. It is called IEKF by Sakov et al. (2012). In this study, we shall not use IEKF, but a close variant that will be called the bundle variant. It will turn out to offer a performance improvement over the IEKF. The second one consists in transforming the ensemble before its propagation, using the ensemble transform  −1/2 T(p) = (N − 1)IN + YT(p) R−1 Y(p) ,

(15)

obtained at the previous iteration. The inverse transformation is applied after propagation. It performs a form of preconditioning of the optimisation problem in ensemble space. We shall call it here the transform variant. The IEnKF still requires inflation. In strongly nonlinear conditions, the inflation factor may be large, although Sakov et al. (2012) remark that the sensitivity to it is weaker and that the required magnitudes are smaller than in non-iterative schemes. Note that the minimisation scheme implicitly adopted by Sakov et al. (2012) is the iterative Newton scheme. Because the Newton method is one of many schemes to minimise cost function Eq. (11), there is a large freedom in choosing the iterative scheme. In this article, we shall choose the Levenberg-Marquardt algorithm (Levenberg, 1944; Marquardt, 1963), in order to have a better control of the optimisation and safely generalize the iterative ensemble Kalman filter of Sakov et al. (2012) to an inflation-free iterative ensemble Kalman filter, which is the final goal of this article. 1.3

Outline

In this article, the focus is on the deterministic EnKFs rather than the stochastic EnKF. The variants of the filters with localization are deliberately not studied. It is well beyond the objective of this article and, to some extent, a disconnected topic. To start with, we shall come back to the EnKF-N and IEnKF filters. Our starting points are the results of Bocquet (2011) and Sakov et al. (2012), and we develop on these two filters. In Sect. 2 we shall first give another interpretation of EnKF-N. It makes an explicit connection with inflation and provides an efficient way to optimise cost function Eq. (7). Besides it sheds light on the use of inflation and why it can be surprisingly efficient when accounting for sampling errors. In Sect. 3, we introduce an implementation of IEnKF that is based on a Levenberg-Marquardt algorithm. As a precursor of the trust-region methods, it allows the controlling of the update which will be useful in the generalization to a finite-size version of the filter, which we introduce in Sect. 4. Nonlin. Processes Geophys., 19, 383–399, 2012

386

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter

Numerical experiments are carried out on the new systems in Sect. 5. Conclusions and leads for improvements are given in Sect. 6. 2

The finite-size ensemble Kalman filter

In this section, we give a new insight on the EnKF-N, whose principles have been recalled in the introduction. For the sake of simplicity, the observation operator H is assumed linear. The generalization to a nonlinear H is not difficult and will be treated in the framework of iterative EnKFs anyway (Sects. 3 and 4). Equation (7) can be written 1 Je(w) = (δ − Yw) T R−1 (δ − Yw) 2  N  + ln εN + wT w , (16) 2 where Y = HA is the observation anomaly matrix and δ = y − H (x) is the innovation. The minimisation of Je is performed over ensemble space, which is numerically efficient in high-dimensional applications. However, this cost function was shown to be non-convex. Besides it has one global minimum and possibly additional local minima. In practice, in Bocquet (2011), the L-BFGS-B minimizer of Byrd et al. (1995) was used. The quasi-Newton algorithm prevents from forming a Hessian with a negative eigenvalue, which might occur with the joint use of such cost function and of a Newton optimisation method. 2.1

Dual scheme

Although this path is successful and efficient, we would like to find a more explicit scheme in order to establish stronger connections with traditional EnKF with inflation. √We wish to split the radial degree of freedom of w, that is wT w, from √ its angular degrees of freedom, that is w/ wT w. To alleviate the notations, we define the functions: g(w) = (δ − Yw) T R−1 (δ − Yw) , f (ρ) = N ln (εN + ρ) .

(17) (18)

Having in mind Je(w) of Eq. (16), a related Lagrangian is introduced:  1 1 1  L(w, ρ, ζ ) = g(w) + ζ w T w − ρ + f (ρ) . (19) 2 2 2 The dual cost function, that we define for ζ > 0, is given by: D(ζ ) = inf sup L(w, ρ, ζ ) . w ρ≥0

(20)

It is easy to check that the maximum and minimum that define D(ζ ) exist. The dual problem consists in minimizing this dual cost function: 1 = inf D(ζ ) , ζ >0

Nonlin. Processes Geophys., 19, 383–399, 2012

(21)

whereas the original problem 5 = inf Je(w) w

(22)

is called the primal problem. In Appendix A, we demonstrate that the two global minima of D and Je coincide. In particular 5 = 1. This is a remarkable non-quadratic case of so-called strong duality (Borwein and Lewis, 2000). This means that the problem of finding the global minimum of our original (primal) problem can rigorously be traded with the problem of finding the global minimum of the dual function. To connect the corresponding optimal ζ ? , ρ ? and w? , and to compute the dual cost function, we write the condition of stationarity of the Lagrangian over ρ and over w. It yields ζ? =

N df ? (ρ ) = , dρ εN + ρ ?

ζ ? w? = ∇w g(w? ) = −Y T R−1 (δ − Yw? ) .

(23) (24)

Since ρ ? ≥ 0, Eq. (23) implies that ζ ? belongs to [0, N/εN ], the interval from 0 (excluded) to N/εN (included). The solutions of Eqs. (23) and (24) are N − εN , ζ?  −1 w? = YT R−1 Y + ζ ? IN YT R−1 δ . ρ? =

(25) (26)

By inserting these solutions in the Lagrangian, one obtains the dual cost function D(ζ ) = L(w? , ρ ? , ζ ) −1 1  = δ T R + Yζ −1 YT δ 2 εN ζ N N N + + ln − . 2 2 ζ 2

(27)

The dual cost function is a function of one single variable. Hence, it is easy to find its global minimum, even in the presence of several minima. As a result, instead of minimizing Je(w) over w, one can equivalently: 1. find the global minimum ζ ? of D(ζ ) in ]0, N/εN ],
−1 T −1 2. compute w? = Y T R−1 Y + ζ ? IN Y R δ. The implementation of the corresponding EnKF-N is detailed in algorithm 2. 2.2

Assets of the dual approach

Let us discuss this alternate minimisation. It has several advantages. Firstly, even though the primal problem seems to be efficiently solved with the help of a quasi-Newton minimizer, it is only guaranteed to find one minimum, not necessarily the global one. On the contrary, because the search of the global www.nonlin-processes-geophys.net/19/383/2012/

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter Algorithm 2 Dual finite-size ensemble Kalman filter. Require: The forecast ensemble {xk }k=1,...,N , the observations y, and error covariance matrix R 1: Compute the mean x and the anomalies A from {xk }k=1,...,N . 2: Compute Y = HA, δ = y − Hx 3: Find the minimum:   −1 ζa = min δ T R + Yζ −1 YT δ ζ ∈]0,N/εN ]

+εN ζ + N ln

N −N ζ

 (28)

 −1 4: Compute a = YT R−1 Y + ζ a IN 5: 6: 7: 8:

Compute wa = a YT R−1 δ. Compute xa = x + Awa . Compute Wa = ((N − 1)a )1/2 U Compute xak = xa + AWak

minimum is reduced to a one-dimensional problem, the dual problem allows to easily find the global minimum. We have performed numerical tests about this issue on the Lorenz ’63 model (Lorenz, 1963) which are discussed in Sect. 5. We found that in marginal cases, the dual approach (global optimisation) would perform better than the primal scheme (local optimisation). However, for most of the tests (Lorenz ’63 and Lorenz ’95) we found no significant performance differences between the dual and primal approaches. Specifically for the numerical tests we have performed with Lorenz ’95, the primal and dual algorithm systematically led to the same optimum. Secondly, the algorithm clearly exhibits the extra cost of EnKF-N against EnKF: minimizing Eq. (27) over scalar ζ . Note that the inverse matrix in the first term of Eq. (27) can be obtained from a singular value decomposition that can also be used in the gain computation Eq. (26). In practice, for the experiments of Sect. 5, we found that the extra cost is negligible. Thirdly, this algorithm parallels the traditional EnKF scheme. Comparing Eq. (26) with Eqs. (20) and (21) of Hunt et al. (2007), it is clear that ζ replaces N − 1 found in the traditional deterministic filters. That is why ζ can be seen as the effective size of the ensemble: the mean number of members that truly contribute in the effective prior. The Lagrange multiplier ζ is also connected to an inflation of the prior error covariance matrix. It can be absorbed into a rescaling of √ the ensemble anomalies by a factor (N − 1)/ζ , so that the analysis Eq. (26) coincides with the usual formula. Therefore, if we define the inflation operation as: xk −→ x + λ(xk − x) ,

(29)

EnKF-N forecasts the following optimal prior inflation factor s N −1 λ? = . (30) ζ? www.nonlin-processes-geophys.net/19/383/2012/

387

That EnKF-N can equivalently be rewritten as a a traditional EnKF with an optimal (prior) inflation factor sheds light as to why inflation can be so successful in dealing with sampling errors. At a fundamental level, the introduction of ζ was possible because of the rotational invariance of the prior in ensemble space. In short, the inflation works well to compensate sampling errors because of the exchangeability of members in the ensemble. Finally, we found the dual approach to be more stable than the primal one. Indeed, in the primal algorithm the BFGS iterations cannot lead to a singular Hessian by construction. However, at the end of the minimisation, one has to generate the new ensemble by computing a in algorithm 1. In infinite machine precision, the Hessian is positive-definite. However, it might be singular in finite numerical conditions, in very demanding conditions, that we only found in the severe Lorenz ’63 test case of Sect. 5. The dual algorithm does not meet this problem because the value of ζa that enters the definition of a is controlled and is guaranteed to be positive, since D(ζ ) goes to +∞ when ζ vanishes. 3

The iterative ensemble Kalman filter

In this section, we follow the steps of Sakov et al. (2012). One minor difference is in the formulation of the iterative ensemble filter, which is written here in ensemble space. Another one is an improvement in terms of performance over the IETKF of Sakov et al. (2012), whose updated version will be called the bundle IEnKF. Another significant difference consists of noticing that one has the freedom to choose any iterative optimisation scheme. Here we shall choose the Levenberg-Marquardt scheme. 3.1

The IEnKF in ensemble space

Let us note x the ensemble mean at time t1 and A the anomaly ensemble matrix at time t1 . Equivalently to using cost function Eq. (11) to perform the analysis, one can use a cost function depending on the coordinates w of x1 = x + Aw in ensemble space: 1 Je(w) = (y2 − H (M(x + Aw))) T R−1 2 1 × (y2 − H (M(x + Aw))) + (N − 1)wT w , 2

(31)

The derivation of the background term can be read in Hunt et al. (2007). The iterative minimisation of the cost function following a Newton algorithm reads: e−1 ∇ Je(w(p) ) , w(p+1) = w(p) − H (p)

(32)

where p is the iteration index and where the gradient and the Hessian are given by

Nonlin. Processes Geophys., 19, 383–399, 2012

388

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter

T ∇ Je(p) = −Y(p) R−1

e(p) H



 y2 − H M(x + Aw(p) )

+(N − 1)w(p) , T = (N − 1)IN + Y(p) R−1 Y(p) ,

(33) (34)

where Y(p) = [H MA]0(p) is the tangent linear of the operator from ensemble space to the observation space that propagates the ensemble anomalies A through the model M and the ob(p) servation operator H , and computed at x1 = x + Aw(p) . 3.2

Levenberg-Marquardt algorithm

The Levenberg-Marquardt algorithm (Levenberg, 1944; Marquardt, 1963) is a precursor of the trust-region method in the sense that it seeks to determine when the (superlinear) Newton method is applicable and when it is not, and should be replaced by the slower but safer gradient descent method (also known as steepest descent). The distinction between the two regimes is obtained by the ratio θ=

(2012), and we found this value to be suitable for the experiments described in Sect. 5. After propagation, the ensemble is inflated by a factor 1/. Note that in the transform variant, the last propagation of the ensemble is often unnecessary. But when the algorithm exits on, e.g., the maximum iteration criterion, it may be necessary to propagate the ensemble, because it may not have been updated at the latest w (as opposed to the Gauss-Newton implementation of Sakov et al., 2012). Therefore, it is possible to avoid this last propagation. However, for the sake of simplicity we have preferred to keep the simpler implementation.

Je(w) − Je(w0 ) , L(0) − L(w0 − w)

(35)

where w0 is the new tentative vector, and L is the quadratic local expansion of Je: 1 e . L(1w) = Je(w) + (1w)T ∇ Je + (1w) T H1w 2

(36)

Instead of determining a region of confidence as in modern trust-region methods, it shifts the Hessian in ensemble e −→ H e + µIN , where space used in the Newton method: H µ is a positive constant. If the quadratic expansion of the cost function allowed by the gradient and the Hessian matches the behaviour of the exact cost function, which corresponds to a large θ, then µ is reduced, otherwise µ is increased (small or negative θ). When µ is small the algorithm is close to a Gauss-Newton method which has a superlinear convergence. When µ is large enough the algorithm is close to a gradient descent method. A textbook and a clear synthesis on this well-established technique are given by Nocedal and Wright (2006) and by Madsen et al. (2004). In the following, the Levenberg-Marquardt algorithm described in Madsen et al. (2004) is adapted to our ensemble Kalman filter context. In the part of the algorithm which seeks a satisfying µ, a single model propagation is necessary. At each successful Newton step, the propagation of the full ensemble is required. The resulting IEnKF scheme is detailed in algorithm 3 of Appendix B. The algorithm describes both the bundle and the transform variants. When algorithmic steps differ in the two variants, the variant is explicitly mentioned. In the bundle variant, the ensemble is shrunk by a small  factor before propagation. It is chosen to be  = 10−4 throughout this article. It is the same as that of Sakov et al. Nonlin. Processes Geophys., 19, 383–399, 2012

4

Combining the finite-size and iterative ensemble Kalman filters

The EnKF-N and IEnKF are complementary since, when looking at the underlying cost function of the analysis, the EnKF-N modifies the prior likelihood, while IEnKF modifies the observational likelihood. Combining the outcome of the previous sections, the cost function used in the analysis as a function of coordinates w of x1 = x + Aw should be 1 Je(w) = (y2 − H (M(x + Aw))) T R−1 2 × (y2 − H (M(x + Aw)))  N  + ln εN + wT w , 2

(37)

Je has a global minimum in RN since it is a positive function and since it goes to infinity as w T w goes to infinity. Several strategies are certainly possible to minimise this cost function. 4.1

Dual approach

The first one consists in using the dual transformation put forward for the EnKF-N in Sect. 2. The derivation holds if one replaces g with g(w) = (y2 − H M(x + Aw)) T R−1 × (y2 − H M(x + Aw)) .

(38)

In particular, the strong duality holds (this can be checked going through Appendix A). As a consequence, it demonstrates that there should be an optimal inflation factor in this context. However, each one of the subproblems, indexed by ζ,   inf g(w) + ζ wT w (39) w

which were equivalent to solving the traditional EnKF analysis in ensemble space Eq. (26), is now equivalent √ to solving an IEnKF analysis, with a prior inflation factor (N − 1)/ζ . Hence, such a path may be numerically costly. Solutions such as primal-dual algorithms may be contemplated, but we www.nonlin-processes-geophys.net/19/383/2012/

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter prefer to explore a more straightforward way to minimise Eq. (37). However, the existence of an optimal inflation factor is proven in this context. 4.2

Primal approach

The second and more direct way to minimise Eq. (37) is to minimise with respect to the w coordinates, with the sole guarantee to find a local minimum. The gradient and Hessian are ∇ Je = −Y T R−1 (y2 − H M(x + Aw)) w +N , εN + wT w
ε + wT w IN − 2wwT e=N N + Y T R−1 Y . H
2 T εN + w w

r λ =


N −1 εN + wT? w? . N

N IN + Y T R−1 Y , εN + wT w

(43)

(40) (41)

(42)

For instance, the statistics of ζ ? over a long data assimilation experiment can tell us about the need to enforce adaptive inflation or not. Cases where ζ ? is strongly fluctuating are cases where IEnKF-N may outperform IEnKF with optimal but constant inflation. The details of this primal scheme are given in algorithm 4 of Appendix B. In severe conditions, such as the one studied www.nonlin-processes-geophys.net/19/383/2012/

at the end of Sect. 5, it might be tedious to define a program in which µ is always large enough so as to guarantee a positive-definite Hessian. One rigorous way around it is to truncate the Hessian to a positive-definite matrix in the iterative minimisation, while the gradient is left unchanged. This truncation of the Hessian does not apply when building the new ensemble. This is equivalent to a slightly sub-optimal pre-conditioning of the minimisation problem. For instance, one can choose: e= H

The main drawback is that the Hessian may have a nonpositive eigenvalue. A priori, this precludes Newton approaches. One solution around it is to use a quasi-Newton minimizer such as L-BFGS-B (Byrd et al., 1995). Only the gradient is provided to the minimizer. This approach was successfully tested. However, on very rare occasions and only for very non-Gaussian systems, the filter can break because the approximate tangent linear leading to an approximate adjoint leads the minimizer into re-initializing the quasiHessian, which may break the filter. This is also a very economical approach, as part of the work is carried out by the minimizer, known to be very efficient. However, for this article, we prefer to report results obtained in a more controlled environment. We can use the Levenberg-Marquardt algorithm as it uses a shifted Hessian, which can be made positive-definite, for large enough µ. Besides, we know that at the minimum of the cost function, the Hessian is positive-definite, so there is a neighborhood around the minimum where the Hessian is positivedefinite. This means that in the vicinity of the minimum, the Levenberg-Marquardt can safely rely on the Hessian of the cost function. As a diagnostics, an intermediate result of the dual approach can be used. Indeed, from Eq. (23), and using the fact that ρ ? = wT? w? at the saddle-point, an equivalent optimal prior inflation factor can be obtained from the analysis in ensemble space: ?

389

which is an always positive-definite substitute matrix for the Hessian.

5

Numerical experiments

Most numerical tests will be performed on the Lorenz ’95 toy-model (Lorenz and Emmanuel, 1998). It has M = 40 variables and its dynamics reads for m = 1, . . . , M: dxm = (xm+1 − xm−2 )xm−1 − xm + F dt

(44)

where F = 8, and the boundary conditions are chosen cyclic. It is integrated using a fourth-order Runge-Kutta scheme with a time-step of 0.05. With this choice of F , the model is strongly chaotic with 13 positive Lyapunov exponents and a doubling time of 0.42 time unit. Hence, it is difficult to control by filtering techniques and can offer simple but severe tests for new methods. Since the model is a simplistic representation of a mid-latitude band of the global atmosphere, a time step of 0.05 in the model’s time represents 6 h of physical time. The data assimilation experiment setup we have chosen consists in computing a reference run of the model, defined as the truth (Sakov and Oke, 2008; Bocquet, 2011; Sakov et al., 2012). This truth is observed for every variable each 1t. Each observation is perturbed by an independent draw from a Gaussian variable of mean 0 and variance 1. Because we do not want to use localization that would mask some of the properties of the filters, the ensemble size is chosen to be N = 40 in the rank-sufficient regime. Nevertheless, one can contemplate building local versions of the filters similarly to what was done by Hunt et al. (2007); Bocquet (2011). The performance of a data assimilation run is assessed computing the average root-mean-square of the difference between the data assimilation analysis and the truth (denoted rmse in the following), for a sufficiently long run. In the following, the runs’ duration, that does not include the burn-in period, is 5 × 104 days (physical time), and we have checked that the convergence of the statistical indicators, such as the rmse, is satisfying. Nonlin. Processes Geophys., 19, 383–399, 2012

tenth percentile, and percentile of the efficient rank ζ for a long run of the EnKF-N, and an ensemble size of N = 40. The dashed line represents the efficient rank diagnosed from the inflation of EnKF leading to the best analysis rmse.

8390

M. Bocquet and P. Sakov: TheThe iterative finite-size ensemble M. Bocquet, P. Sakov: iterative finite-size ensembleKalman Kalmanfilter filter

5.2 T t

40

40

Analysis rmse Optimal efficient rank of the prior ζ

Efficient rank of the prior

a

0.9

30

20

Mean Median Tenth percentile Percentile Efficient rank from opt. inflation

10

0

0

0.1

0.2

0.3

0.4

0.5

0.6

Time interval between updates

Fig. 1. Mean, Mean, standard standard deviation (shown by errors bars), median, Fig. a tenth percentile, percentile and tenth and percentile percentile of of the the efficient efficient rank rank ζζ a for for a long run of the EnKF-N, and an ensemble size of N = 40. The dashed line of the EnKF-N, and an ensemble size of N = 40. represents the efficient rank diagnosed from the inflation of EnKF represents leading to the best analysis rmse. leading

a

5.1 40 Complementary experiments on EnKF-N EnKF-N has been tested on the Lorenz ’63 and Lorenz ’95 models in Bocquet (2011). Here, we wish to report some 30 complementary experiments related to the dual implementation of EnKF-N, introduced in Sect. 2, in its εN = 1 variant. The efficient rank ζ a of the ensemble prior, as estimated 20 by the EnKF-N formalism, is computed using Eq. (23), and ρ a = wTa wa at optimality: Optimal efficient rank of the prior ζ

interval same re tion.

10

ζa =

N , εN + wTa wa

(45)

for 0a0 long run200of EnKF-N applied 400 800 ’95, with 1000 600 to Lorenz of Lorenz model the setup described Time above. Note’95that ζ a = N − 1 would correspond to a deterministic EnKF without inflation. The 2.time interval betweenranks updates is run varied: 1t = Fig. Sequence of efficient ζ a in 1t a long of EnKF-N 0.05, 0.10, . . . , 0.60, so aswith to probe the critical where applied to Lorenz ’95 model, ∆t = 0.30, and withcases an ensemble inflation strong (when ζ a /N is small). The statistics of ζ a of N = 40ismembers. are plot in Fig. 1: mean, standard deviation, tenth percentile and percentile. This allows to study the variability of the efficient rank, why we believe EnKF-N may outperform EnKF(as with optior, indirectly, of the optimal prior inflation factor seen by mized constant inflation, when the system becomeswith signifiEnKF-N). It is clear that the efficient rank decreases 1t. cantly nonlinear. Using Eq. (42),increases the efficient has been More importantly, the variability veryrank significantly diagnosed frompercentile the optimal inflation of the since the tenth decreases below 20EnKF for 1tobtained ≥ 0.50. from selecting best rmseofconfiguration. It is the alsooptimal plotted Because of thisthe variability the rank in time, in Fig. 1, factor and shows a similar as not the efficient inflation diagnosed by evolution EnKF-N is constant rank and of EnKF-N. SinceThat the mean not differ much, varies with time. is whyand wemedian believedo EnKF-N may outitperform also suggests that, optimised most of the time, the efficient rankthe is EnKF with constant inflation, when rather that it occasionally drops. the systemconstant, becomesbut significantly nonlinear.suffers Usingsudden Eq. (42), We have rank checked this examining of ζ a . In Fig. 2 efficient has been diagnosed sequences from the optimal inflation is displayed a typical sequence in the case ∆t = 0.30. Nonlin. Processes Geophys., 19, 383–399, 2012

0.8 0.7 30 0.6

N=40

0.5 0.4 20

0.3

EnKF (opt. infl.) IEnKF (bundle, opt. infl.) IEnKF (transform, opt. infl.)

10

0.2 0 0.15 00

200 0.1

0.2

400

0.3

600 0.4

Time interval of Lorenz ’95 model Time between updates

800 0.5

0.6 1000

Fig. Sequence of efficient of EnKF-N Fig. Sequenceroot rankserror ζ a in long run of EnKF-N Fig. 2. 3. Analysis mean square of athe IEnKF (bundle and applied to Lorenz ’95 model, with ∆t = 0.30, and with an ensemble applied to Lorenz ’95 model, with 1t = 0.30 and with an ensemble transform versions) for the Lorenz ’95 model and for several time of N = 40∆t of N members. intervals between two subsequent updates. For comparison the same results for EnKF (ETKF version) are given for optimal inflation.

of the EnKF obtained from selecting the best rmse configurawhy we believe EnKF-N may outperform EnKF with optition. It is also plotted in Fig. 1, and shows a similar evolution mized constant inflation, when the system becomes signifias efficientIEnKF, rank of EnKF-N. Since the meanimplementaand median 5.2the Testing cantly nonlinear. UsingLevenberg-Marquardt Eq. (42), the efficient rank has been do nottion differ much, it also suggests that, most of the time, diagnosed from the optimal inflation of the EnKF obtained the efficient rank is rather constant, but that it occasionally from selecting the best rmse configuration. It is also plotted suffers sudden drops. We have checked this examining seThe IEnKF first tested in itsevolution Levenberg-Marquardt in Fig. 1, andisshows a similar as the efficientimplerank quences of ζ a . In Fig. 2 is displayed a typical sequence in mentation, the the bundle variants. Since the of EnKF-N.for Since meanand andtransform median do not differ much, the case 1t = 0.30. on the that, ability of the IEnKF handle strong nonitfocus also is suggests most of the time,tothe efficient rank is linearity, the time interval between updates ∆t is varied: rather constant, but that it occasionally suffers sudden drops. 5.2 Testing IEnKF, Levenberg-Marquardt a ∆t have = 0.05,0.10,...,0.60. The filters are run We checked this examining sequences of ζwith . Inoptimal Fig. 2 implementation inflation factors leading to the best rmses are isinflation: displayedthea typical sequence in the case ∆t = 0.30. selected. The termination criteria of the iterative minimizaThe IEnKF is first tested in its Levenberg-Marquardt imtion are such that the maximal number of iterations max = √ is pSince plementation, for the bundle and transform variants. 40, and the precision has reached ||∆w|| = wT w ≃ e2 , the focus is on the ability of the IEnKF to handle strong with e2 = 10−3 . Provided pmax is large enough (especially nonlinearity, the time interval between updates 1t is varfort large ∆t), the results are largely independent from this ied: 1t = 0.05, 0.10, . . . , 0.60. The filters are run with opchoice. The choice of the influential criterion e2 is more timal inflation: the inflation factors leading to the best rmcritical. A compromise must be found between precision ses are selected. The termination criteria of the iterative minand limiting the number of iterations. This leads to choosimisation are such that the maximal number of iterations is √ ing e = 10−3 , found to be satisfying for all experiments repmax 2= 40, and the precision has reached ||1w|| = wT w ' ported in this article. The condition on the gradient was not e2 , with e2 = 10−3 . Provided pmax is large enough (espeimplemented (see algorithm 3), which means that e1 = 0. We cially fort large 1t), the results are largely independent from also chose τ = 10−3 which has an influence on the iteration this choice. The choice of the influential criterion e2 is more number, since it tells whether the minimization at the begincritical. A compromise must be found between precision ning is closer to a Newton method (small τ ) or to a steepand limiting the number of iterations. This leads to choosest descent method. We found that in the weakly non-linear ing e2 = 10−3 , found to be satisfying for all experiments reregime of small ∆t, a smaller τ could decrease the number of ported in this article. The condition on the gradient was not required iterations. This is consistent with the intuition that implemented (see algorithm 3), which means that e1 = 0. We a steepest descent method is unnecessary in this regime and also chose τ = 10−3 which has an influence on the iteration favoring Newton’s method results in a finer convergence. number, since it tells whether the minimisation at the beginThe area reported Fig. 3.(small As aτcomparison, the ning is results closer to Newton in method ) or to a steeprmse for themethod. EnKF with optimalthat inflation also reported. est descent We found in the isweakly nonlinearIt clearly shows that, cost ofτ cycling the ensemble proparegime of small 1t,atathe smaller could decrease the number gation, the more nonlinear the propagation is the more IEnKF www.nonlin-processes-geophys.net/19/383/2012/

The IE mentat focus lineari ∆t = 0 inflatio selecte tion ar 40, an with e fort lar choice critical and lim ing e2 ported implem also ch numbe ning is est des regime require a steep favorin The rmse f clearly gation

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter M. Bocquet, P. Sakov: The iterative finite-size ensemble Kalman filter 0.9 0.8

N=40

0.7

Analysis rmse

0.6 0.5 0.4

0.3

EnKF (opt. infl.) IEnKF (bundle, opt. infl.) IEnKF (transform, opt. infl.)

0.2

0.15

0.6

0

0.1

0.2

0.3

0.4

0.5

0.6

Time interval between updates

median, ong run ed line EnKF

1000

nKF-N semble

h optiignifis been tained plotted nt rank much, ank is drops. Fig. 2

Fig. mean square error Fig.3.3. Analysis Analysisroot root-mean-square error of of the the IEnKF (bundle and transform transform versions) versions) for for the the Lorenz Lorenz ’95 model and for several time intervals∆t 1t between between two two subsequent subsequent updates. updates. For comparison the intervals sameresults results for for EnKF EnKF (ETKF (ETKF version) version) are given for optimal inflasame tion. tion.

of required iterations. is consistent with theimplementaintuition that 5.2 Testing IEnKF, This Levenberg-Marquardt a steepest descent method is unnecessary in this regime and tion favouring Newton’s method results in a finer convergence. The results are reported in Fig. 3. TheAs IEnKF is first tested in itsfor Levenberg-Marquardt implea comparison, the rmse the EnKF with optimal inmentation, for the bundle and transform variants. Since the flation is also reported. It clearly shows that, at the cost of cyfocus is on the ability of the IEnKF to handle strong noncling the ensemble propagation, the more nonlinear the proplinearity, the time interval between updates ∆t is varied: agation is the more IEnKF outperforms EnKF. ∆t As = 0.05,0.10,...,0.60. filters we are did runnot with opposed to Sakov et The al. (2012), findoptimal a siginflation: the inflation factors leading to the best rmsesvariare nificant difference between the transform and bundle selected. The termination criteria of the iterative minimizaants for small and moderate 1t, as far as analysis rmses tion are such thatThe themain maximal number is pmax = are concerned. difference is of notiterations in the √ LevenbergTw ≃ e , w 40, and the precision has reached ||∆w|| = 2 Marquardt implementation, but in the fact that after the anal−3 with e = 10 . Provided p is large enough (especially 2 max ysis full cycle, we propagate the ensemble from t1 to t2 avoidfort the results are largely fromThis this ing large to use∆t), the rescaled ensemble used independent within the cycle. choice. The choice of the influential criterion e is more 2 final propagation of the non-rescaled ensemble makes the critical. A compromise must beno found between precision bundle scheme used in this study longer based on the exand limiting thefilter, number of IEKF iterations. Thisetleads to choostended Kalman as the in Sakov al. (2012). ingHowever, e2 = 10−3note , found be satisfying experiments rethattofor 1t ≥ 0.5, for the all transform outperported article. The As condition gradient not forms in thethis bundle variant. opposedontothe Sakov et al.was (2012) implemented (seeof algorithm 1 = 0. We implementation IEnKF, 3), wewhich foundmeans that that the ermses are also chose τ = 10−3 which1thas influence the iteration rapidly increasing beyond ≥ an 0.60. Beyondon this time internumber, since it tellsthan whether the minimization at dynamical the beginval, which is more the doubling time of the ning is closer to aminima Newton τ ) the or underlying to a steepsystem, multiple aremethod likely to(small form in est method. We that weakly asfound pointed outinbythe Pires et al.non-linear (1996). costdescent function Eq. (11) regime small number ∆t, a smaller τ could number of The of average of model runs decrease used untilthe convergence required iterations. is bundle consistent the intuition is reported in Fig. 4 This for the andwith transform variants.that In aaddition steepesttodescent method unnecessary regime and the N = 40, anisensemble of Nin=this 25 members is favoring Newton’s in a finercompare convergence. also considered in method order toresults quantitatively with the results of Sakov al. (2012). However, did not notice The results are etreported in Fig. 3. As awecomparison, the any qualitative change between the outcomes of reported. the two enrmse for the EnKF with optimal inflation is also It sembleshows configurations. clearly that, at the cost of cycling the ensemble propagation, the more nonlinear the propagation is the more IEnKF www.nonlin-processes-geophys.net/19/383/2012/

391

It is clear that the bundle variant requires less iterations than the transform variant, between 1 to 2 fewer iterations. This is different from the implementation of Sakov et al. (2012), who showed that the IEKF variant usually requires slightly more iterations than the transform variant, when not using the Levenberg-Marquardt framework. Note that in the transform variant algorithm, as opposed to the bundle variant, it is legitimate to skip the computation of the forecasted ensemble (lines 41–44) because it has been done earlier in line 29. This could lead to a gain of up to one iteration. However, in practice, we found it was inefficient in our test cases, since it had a negative impact on the precision which may have required additional iterations at a later cycle. The large number of iterations needed at small 1t, between 3 to 4, is not very significant because the LevenbergMarquardt is not designed to optimally perform a convergence of a very few iterations. In this case, the more straightforward Gauss-Newton method, such as the implementation of Sakov et al. (2012) is preferable. 5.3

Testing IEnKF-N

Here, the IEnKF-N, bundle and transform versions, are tested on the same configuration as IEnKF. The results are reported in Fig. 5. First the two variants of IEnKF-N offer the same rmses in this configuration over the full range of 1t. Secondly, in this example, they are essentially equally performing or better than the IEnKF with optimal inflation. This shows that, as hoped for, some of the properties of the finite-size EnKF apply as well to iterative variants of the IEnKF. The average number of model runs used until convergence is reported in Fig. 4 for the bundle and transform variants of IEnKF-N. Like for the IEnKF experiments, the same termination criterion is chosen (pmax = 40, e1 = 0, e2 = 10−3 , and τ = 10−3 ) for all runs. The finite-size versions of the filters require a similar number of iterations (or less) as their optimised inflation counterparts. Here again, the bundle variant is doing better than the transform variant: it requires 1 to 2 less ensemble propagations on average required by the transform variants to achieve the same precision, using the same termination criterion. Similarly to Fig. 1 in the case of EnKF-N, the statistics of ζ a are plot in Fig. 6: mean, standard deviation, median, tenth percentile, and percentile. The results are qualitatively similar. However, as could be expected, the efficient rank is larger in the IEnKF-N case than EnKF-N in the same configuration. Even though the percentile remains high below 1t ≤ 0.5, it rapidly falls off beyond. Consistently this is where the IEnKF-N starts to slightly outperform IEnKF with optimal inflation.

Nonlin. Processes Geophys., 19, 383–399, 2012

Number of ensemble propagations Number of ensemble propagations Number of ensemble propagations

11 11 11 10 10 10 9 9 9 8 8 8 7 7 7 6 6 6 5 5 5 4 4 4 3 3 3 2

M. Bocquet and P. Sakov: The iterative M. Bocquet, P. Sakov: The iterativefinite-size finite-sizeensemble ensembleKalman Kalman filter filter M. M.Bocquet, Bocquet,P.P.Sakov: Sakov:The Theiterative iterativefinite-size finite-sizeensemble ensembleKalman Kalmanfilter filter

N = 40 NN = 40 = 40

IEnKF (transform), opt.infl. IEnKF-N (transform) IEnKF (transform), opt.infl. IEnKF (transform), opt.infl. IEnKF (bundle), opt.infl. IEnKF-N (transform) IEnKF-N (transform) IEnKF-N (bundle) IEnKF (bundle), opt.infl. IEnKF (bundle), opt.infl. IEnKF-N (bundle) IEnKF-N (bundle)

2 2 1 1 1 0 0 0 0 0 0

0.1

0.2

0.10.1

Time 0.2 0.2

0.3

0.4

0.5

0.6

interval updates 0.50.5 0.30.3between 0.40.4 Time interval between Time interval betweenupdates updates

0.60.6

11 1111 10 1010 9 9 9 8 8 8 7 7 7 6 6 6 5 5 5 4 4 4 3 3 3 2

N = 25 NN= =2525

Number of ensemble propagations Number of ensemble propagations Number of ensemble propagations

392 10 1010

IEnKF (transform), opt.infl. IEnKF-N (transform) IEnKF (transform), opt.infl. IEnKF (transform), opt.infl. IEnKF (bundle), opt.infl. IEnKF-N (transform) IEnKF-N (transform) IEnKF-N (bundle) IEnKF (bundle), opt.infl. IEnKF (bundle), opt.infl.

2 2 1 1 1 0 0 0 0 0 0

IEnKF-N (bundle) IEnKF-N (bundle) 0.1

0.10.1

0.2

0.3

0.4

0.5

Time between0.4 updates 0.20.2 interval 0.30.3 0.50.5 0.4 Time Timeinterval intervalbetween betweenupdates updates

0.6

0.60.6

Fig. 4. Average number of model runs, divided by the size of the ensemble for the two variants of the Levenberg-Marquardt IEnKF, for the Fig. Average number of model runs, dividedby∆t bythe the sizeofoftwo the ensemble for the two variants ofofthe the Levenberg-Marquardt IEnKF, for the Fig. 4.4.4. Average number ofof model runs, divided size the the variants of the IEnKF, for the Fig. Average number model runs, divided theensemble ensemblefor for thetwo two variants theensemble Levenberg-Marquardt IEnKF, for the Lorenz ’95 model and for several time intervals between subsequent updates. On the left, size is N = 40, whereas on Lorenz ’95 model and for several time intervals 1t between two subsequent updates. On the left, the ensemble size is N = 40, whereas on Lorenz ’95 and onon Lorenz ’95 and severaltime intervals ∆t ∆tbetween betweentwo twosubsequent subsequentupdates. updates.On Onthe theleft, left, the ensemble size isNN==40, 40,whereas whereas the right, itmodel ismodel N= 25.forforseveral the right, itisN is N N = 25. thethe right, it itis == 25. right, 25. 40 0.9

0.7 0.70.6

N=40

N=40 N=40

30

3030

0.6 0.60.5

Analysis rmse Analysis Analysis rmsermse

Efficient rankrank of the Efficient of prior the prior Efficient rank of the prior

0.9 0.90.8 0.8 0.80.7

4040

0.50.5 0.4

20

0.40.4

2020

0.3

EnKF-N EnKF (opt. infl.) IEnKF (bundle, opt. infl.) EnKF-N EnKF-N EnKF (opt. infl.) EnKF (opt. infl.) IEnKF-N (transform) IEnKF (bundle, infl.) IEnKF (transform, opt. infl.) IEnKF (bundle, opt.opt. infl.) IEnKF-N (bundle) IEnKF-N (transform) IEnKF-N (transform) IEnKF (transform, infl.) IEnKF (transform, opt.opt. infl.) IEnKF-N (bundle) IEnKF-N (bundle)

0.30.3 0.2

0.20.2 0.15 0

0.1

0.15 0.15 0 0

0.10.1

0.2

0.3

0.4

0.5

0.6

0.50.5

0.60.6

Time interval between updates 0.20.2

0.30.3

0.40.4

Time interval between updates Time interval between updates

Fig. 5. Analysis root mean square error of the IEnKF-N (bundle and transform versions) for the Lorenz ’95 andIEnKF-N for (bundle several time Fig. 5.Analysis Analysis root-mean-square error ofIEnKF-N the (bundle Fig. root mean square error ofthe the IEnKF-N (bundle and Fig. 5.5.Analysis root mean square error ofmodel and intervals ∆t between two subsequent updates. For comparison the and transform versions) for the Lorenz ’95 model and for several transform versions) the Lorenz ’95 model andfor for several time transform versions) forfor the Lorenz ’95 model and several time same are forsubsequent EnKF-N, IEnKF (bundle andcomparison transform time results intervals 1tgiven between two subsequent updates. For intervals ∆tbetween between two subsequent updates. Forcomparison comparison the intervals ∆t two updates. For the variants) with optimal inflation (repeated from Fig. 3). the same results are given for EnKF-N, IEnKF (bundle and transsameresults resultsare aregiven givenforforEnKF-N, EnKF-N,IEnKF IEnKF(bundle (bundleand andtransform transform same form variants) with optimal inflation (repeated from Fig. 3). variants) withoptimal optimal inflation (repeated fromFig. Fig.3).3). variants) with inflation (repeated from

The results are first obtained for three non-iterative ensemble Kalman filters. EnKF withfor optimally tuned inflation (the The results arefirst obtained for threenon-iterative non-iterative ensem5.4 Focusing onfirst the weakly nonlinear regime The results are obtained three enseminflation factor that yields the best analysis rmse), EnKFbleKalman Kalmanfilters. filters.EnKF EnKFwith withoptimally optimallytuned tunedinflation inflation(the (the ble N usingfactor its primal implementation, and EnKF-N using its inflation factor thatyields yieldsresults thebest best analysis rmse), EnKFinflation the analysis EnKFTo mitigate thethat optimistic obtained on rmse), Lorenz ’95, we dual implementation are considered. As discussed in Bocusinglike, primal implementation, and EnKF-N usingitsits NNwould using itsitsprimal and EnKF-N in thisimplementation, section, to illustrate and discussusing apparent quet (2011) about theare application of the EnKF-N filters to dual implementation considered. As discussed Bocdual implementation are considered. As discussed ininBoclimitations of the formalism. To do1so, we choose the quite variant should be used the Lorenz ’63 model, the ε = 1+ N quet (2011)about about theapplication applicationLorenz ofthe theEnKF-N EnKF-N filters demanding case ofthe 3-variable ’63 toy model, with quet (2011) ofN filters toto for such a small ensemble where the1 1analyzed state vector is variant should be used the Lorenz ’63 model, the ε = 1+ an ensemble of 3 members. It might not be directly relevant the Lorenz ’63 model, the εNN= 1+ NNvariant should be used not well estimated by the ensemble mean. The fundamental for sucha asmall smallensemble ensemble wherethe theanalyzed analyzed state vector tosuch high-dimensional geophysical systems. Butstate the goal of this for where vector isis difference between the the primal and dual formalism is that the not well estimated by ensemble mean. The fundamental section is to find out about flaws or inconsistencies of the not well estimated by the ensemble mean. The fundamental primal implementation picks upand one minimum for is thethat analschemes and to point directions of possible improvement. difference between theto primal dual formalism the difference between the primal and dual formalism is that the ysis, whereas the dual implementation determines the global primal implementation picks up one minimum for the analprimal implementation picks up one minimum for the analysis,whereas whereasthe thedual dualimplementation implementationdetermines determinesthe theglobal global ysis, Nonlin. Processes Geophys., 19, 383–399, 2012

10

1010

0 0 00 00

Mean Median Tenth percentile Mean Mean Median Median Percentile Tenth percentile Tenth percentile Efficient rank from opt. inflation Percentile Percentile Efficient rank from opt. inflation Efficient rank from opt. inflation 0.1 0.2 0.3

0.10.1

0.4

0.5

Time interval between updates 0.20.2

0.30.3

0.40.4

0.50.5

0.6

0.60.6

Timeinterval intervalbetween betweenupdates updates Time

Fig. 6. Mean, standard deviation (shown by errors bars), median, tenth percentile, and percentile of the efficient ζ abars), for a long run Fig. Mean, standard deviation (shown by errors median, Fig.6.6. 6.Mean, Mean,standard standard deviation (shown errors bars), median, Fig. deviation (shown bybyrank errors bars), median, aa for a ofa N of the IEnKF-N, bundle variant, and an ensemble size = 40. tenth percentile and percentile of the efficient rank ζ long run tenthpercentile, percentile,and andpercentile percentileofofthe theefficient efficientrank rankζ ζ for fora along long run tenth run The dashed line represents the efficient rank diagnosed from the40. of the IEnKF-N, bundle variant and an ensemble size of N = of the IEnKF-N, bundle variant, and an ensemble size of N = of the IEnKF-N, bundle variant, and an ensemble size of N =40. 40. inflation of EnKF leading to the analysis rmse. The dashed line represents efficient rank diagnosed Thedashed dashed linerepresents represents thebest efficient rank diagnosedfrom fromthe the The line the efficient rank diagnosed from the inflation of EnKF leading to the best analysis inflationofofEnKF EnKFleading leadingtotothe thebest bestanalysis analysisrmse. rmse. inflation rmse.

minimum of the analysis cost function. 5.4.1 Diagnosis The analysis root mean square errors are reported on minimum theanalysis analysis cost function. minimum ofofthe cost function. Fig. 7. First of all, it is clear that the finite-size filters The Lorenz ’63 model (Lorenz, 1963) is defined the setunof Theanalysis analysisroot rootmean meansquare squareerrors errors areby reported on The are reported on derperform below some threshold equals to ∆t = 0.16 for three ordinary differential equations: Fig. 7. First of all, it is clear that the finite-size filters unFig. 7. First of all, it is clear that the finite-size filters un2 2 σderperform ∆t = some 0.14 for σobs = 8. Theytoultimately obs = 1, and below threshold equals ∆t==0.16 0.16difor derperform below some threshold equals to ∆t for dx verge for too small ∆t. This divergence is not directly due todi2 2 = σ (y − x) , 2 2 σ = 1, and ∆t = 0.14 for σ = 8. They ultimately σdt = 1, and ∆t = 0.14 for σ = 8. They ultimately diobs obs obs the quasi-linearity of∆t. the model inobs that regime, but to thedue fact verge fortoo toosmall small Thisdivergence divergence notdirectly directly verge for ∆t. This isisnot due toto dy that even in that regime it still diverges without a properly = ρx − y − xzof,ofthe thequasi-linearity quasi-linearity themodel modelininthat thatregime, regime,but buttotothe thefact fact the dt tuned inflation. Indeed weit have checked without that the aEnKF-N that even in that regime still diverges properly that that regime it still diverges without a properly dz eventoina large behaves extent similarly an EnKFthat on athe linear ad=inflation. xy − βz , Indeed (46) tuned inflation. Indeedwe wehave haveto checked EnKF-N tuned checked that the EnKF-N dt vection model (LA model) used in Sakov and Oke (2008). behavestotoa alarge largeextent extentsimilarly similarlytotoananEnKF EnKFon ona alinear linearadadbehaves vection model (LA model) used in Sakov and Oke (2008). vection model (LA model) used in Sakov and Oke (2008). www.nonlin-processes-geophys.net/19/383/2012/

M.Bocquet, BocquetP.and P. Sakov: The iterative finite-size ensemble M. Sakov: The iterative finite-size ensemble KalmanKalman filter filter

EnKF optimal inflation EnKF-N dual EnKF-N primal

1.0 0.8 0.6 0.4

σ

0.2 0

0

0.1

0.2

0.3

EnKF optimal inflation EnKF-N dual EnKF-N primal

3.0

0.4

2 obs

=1

0.5

Time interval between updates

Analysis root mean square error

Analysis root mean square error

1.2

393 11

2.5 2.0 1.5 1.0

σ

0.5 0

0

0.1

0.2

0.3

0.4

2 obs

=8

0.5

Time interval between updates

Fig. 7. Analysis root mean square error for three filters applied to the Lorenz ’63 model (N = 3): the EnKF with optimally tuned inflation, 2 2 Fig. 7. Analysis root-mean-square error for three filters appliedthe to the Lorenz ’63and model =panel 3): theillustrates EnKF with the primal EnKF-N, and the dual EnKF-N. Left panel illustrates σobs = 1 case the (N right theoptimally σobs = 8 tuned case. inflation, 2 2 the primal EnKF-N and the dual EnKF-N. Left panel illustrates the σobs = 1 case and the right panel illustrates the σobs = 8 case.

It is therefore likely that this underperformance could be whereback σ = to 10,the ρ= 28, and β = of 8/3. model is chaotic traced determination theThis optimal prior inflawiththrough a doubling time of 0.78 of time is integrated ustion the minimization Eq.unit. (27) Itthat may become ing a fourth-order Runge-Kutta scheme a time-step of inaccurate in that regime. If one trusts thewith Bayesian formal0.01 time unit.EnKF-N, In the data ahead, ism underlying thenassimilation according toexperiments the discussion in all three(2011) variables observed every These observations Bocquet thisare inaccuracy can be1t. ascribed to either (i) normal uncorrelated errors of standard σobs . anhave inappropriate choice of the hyperprior whichdeviation is at the heart are 5×10 -cycle with additional peofAll theruns derivation of 5Eq. (2) long and in fineanspecifies thespin-up particular riod of of Eq. 5 × (27) 104 cycles. an prior apparent limitation form , (ii) orTo theillustrate use of the p(x|x ) 1 ,...,xNof the finite-size formalism in),either rather than p(x|y,x (iii) orthe theEnKF use ofora the localiterative mini1 ,...,xN EnKF, we than vary the theglobal time interval between updates between mum rather minimum. Point (iii) can be ruled 1t because = 0.05 (nearly linearstudy regime between solved update this steps) and out, the present essentially prob1t = 0.50 (strongly nonlinear regime). The error covariance lem. 2 2 = 8. will eithertime be σinterval obs = 1, or obs0.15, Beyond ∆tσ≃ the EnKF-N primal and results are first obtained for three non-iterative dualThe implementations significantly outperform the ensemEnKF ble Kalman filters. EnKF with optimally tuned inflation (the with optimally tuned inflation. As should have been exinflation factor that yields the best analysis rmse), EnKFpected, the dual implementation is better than the primal imN using its primal implementation, and EnKF-N using its plementation. Note that when ∆t ≤ 0.15, the primal varidual implementation are considered. As discussed in Bocant can beat the dual one. However, since it is in a regime quet (2011) about the application EnKF-N filters where the formalism breaks down forof1thethe likely reasons givento the Lorenz ’63 model, the ε = 1+ variant should be used N N above, this difference is essentially irrelevant. for such a small ensemble where the analysed state vector is A similar experiment wasensemble performed but for = 9, closer not well estimated by the mean. TheNfundamental todifference the asymptotic ensemble size limit. In that case, thethat turnbetween the primal and dual formalism is the ing point is at about ∆t ≃ 0.04. primal implementation picks up one minimum for the analThewhereas same study was implementation conducted for the iterative ensemble ysis, the dual determines the global Kalman filters. resultscost are function. reported in Fig. 8. Note that minimum of theThe analysis we The have analysis not introduced any dual variant thereported IEnKF-N. root-mean-square errorsofare on That 7. why only the IEnKF with optimally tuned inflation Fig. is and First the IEnKF-N its primal been of all, it isinclear that theimplementation finite-size filtershave underper2 qualtested. The bundle variant was chosen. results form below some threshold equals to 1tThe = 0.16 forare σobs = 1, 2 non-iterative itatively to the filters. However, since and 1t similar = 0.14 for σobs = 8. They ultimately diverge for too the flow1t. between updates is made linear the small This divergence is not more directly duethanks to the to quasiiterative the in regime pushedbut toward qualilinearitycorrections, of the model that is regime, to thethefact that tative of small ∆t.diverges Consistently turning point even behavior in that regime it still withoutthe a properly tuned beyond which the IEnKF-N outperforms, is pushed towards www.nonlin-processes-geophys.net/19/383/2012/

higher ∆t. inflation. Indeed we have checked that the EnKF-N behaves 5.4.2 An extent empirical solution to a large similarly to an EnKF on a linear advection model (LA model) used in Sakov and Oke (2008). It is, thereExploring small regime and the could behavior of the back dual fore, likelythe that this ∆t underperformance be traced cost function Eq. (27), we found that, in that regime, the arto the determination of the optimal prior inflation through ⋆ gument ζ of its minimum is mostly given by the maximum the minimisation of Eq. (27) that may become inaccurate in of theregime. interval, thattrusts is N/ε . But if N/ε N asymptotically that If one theN Bayesian formalism underlying behaves like N − 1, it is bigger than N − 1 at finite N , for EnKF-N, then according to the discussion in Bocquet (2011) εthis = 1 or ε = 1 + 1/N . For instance in the case N =3 N inaccuracy N can be ascribed to either (i) an inappropriate and ε = 1 + 1/N , one has N/ε = 2.25 as compared to N of the hyperprior which isNat the heart of the derivachoice N − 1 = 2. Hence, in that regime, EnKF-N tends to often tion of Eq. (2) and in fine specifies the particular form of implicitly the use ensemble, whichp(x|x may 1lead overEq. (27), deflate (ii) or the of the prior , . . . ,to xNan ) rather confident Kalman filter and to its divergence. than p(x|y, x1 , . . . , xN ), (iii) or the use of a local minimum Fromthan thisthe educated we propose to cap rather global guess, minimum. Point (iii) canthe be argument ruled out, ⋆ ζbecause of the the minimum of Eq. (27) at N − 1. Note that the case present study essentially solved this problem. ζ⋆ = N − 1 corresponds to an absence of inflation and Beyond time interval 1t ' 0.15, the EnKF-N primaldoes and not guarantee the filter’s stability. Moreover, duala priori implementations significantly outperform the rather EnKF than ζ ⋆ , tuned we prefer to modify (27)have in such a way withcapping optimally inflation. As Eq. should been ex⋆ that the maximum value of ζ cannot exceed N −1. The first pected, the dual implementation is better than the primal reason for doing so is that when the duality of appendix A implementation. Note 1t ≤ result 0.15, the primal variisant derived on the full interval ]0,N/ε ], and an abrupt truncan beat the dual one. However,Nsince it is in a regime cation invalidate the duality equivalence. The second wherewould the formalism breaks down for the likely reasons given ⋆ reason is that a direct capping of ζ requires an access to above, this difference is essentially irrelevant. Eq.A(27), which is not straightforward thefor case of9,the prisimilar experiment was performedinbut N= closer mal schemes. For these two reasons, we propose to renorto the asymptotic ensemble size limit. In that case, the turning malize εN = − 1). That way, it is easy to check point isεN at to about 1tN/(N ' 0.04. ⋆ through Eq. (23) thatwas the maximum of ζiterative is ζ ⋆ =ensemN −1 The same study conductedvalue for the ⋆ atbleρ Kalman = 0. In filters. the following capping will referintoFig. this8.renorThe results are reported Note malization of εnot not to the truncation of Eq. (27) that we have introduced anyabrupt dual variant of the IEnKFN , and toN.the interval ]0,N − 1]. That is why only the IEnKF with optimally tuned inflaThe resulting performances of theimplementation capped EnKF-N, primal tion and the IEnKF-N in its primal have been and dualThe variants, displayed Fig. 9.The With our setup, the tested. bundleare variant was in chosen. results are qualitatively similar to the non-iterative since dual EnKF-N outperforms or equals filters. EnKF However, with optimally the flow between is made more thanks toThe the tuned inflation overupdates the enlarged range ∆tlinear ∈ [0.05,0.50]. benefit of the dual filter over the primal variant is now even Nonlin. Processes Geophys., 19, 383–399, 2012

12394

M. Bocquet and P. Sakov: The iterative M. Bocquet, P. Sakov: The iterativefinite-size finite-sizeensemble ensembleKalman Kalman filter filter

0.8 IEnKF optimal inflation IEnKF-N primal EnKF optimal inflation EnKF-N primal

0.6 0.5 0.4 0.3 0.2

σ 0.1 0

0

0.1

0.2

0.3

0.4

2 obs

=1

0.5

Time interval between updates

Analysis root mean square error

Analysis root mean square error

0.7

IEnKF optimal inflation IEnKF-N primal EnKF optimal inflation EnKF-N primal

2.5

2.0

1.5

1.0

σ

0.5

0

0

0.1

0.2

0.3

0.4

2 obs

=8

0.5

Time interval between updates

Fig. 8. Analysis root mean square error for two iterative filters applied to the Lorenz ’63 model (N = 3): the IEnKF, bundle variant with 2 Fig. 8. Analysis root-mean-square erroralgorithm for two iterative applied to the LorenzLeft ’63 panel modelillustrates (N = 3): the the σIEnKF, variant with optimally tuned inflation, and the primal for the filters IEnKF-N, bundle variant. case and the right obs = 1bundle 2 = 1 case and the right 2 optimally tunedtheinflation, theThe primal algorithm forFig. the 7IEnKF-N, bundle variant. Left panel illustrates the σobs panel illustrates σobs = 8and case. EnKF results of are reported for comparison. 2 = 8 case. The EnKF results of Fig. 7 are reported for comparison. panel illustrates the σobs

clearer in the almost linear regime. In that regime, ζ ⋆ remains close to Ncorrections, − 1, with occasional excursions smallerthevalues. iterative the regime is pushedtotoward qualiThese are mostly by transitions of thepoint dytativeevents behaviour of smallprovoked 1t. Consistently the turning namics the Lorenz ’63 model between lobes. Because of beyondofwhich the IEnKF-N outperforms, is pushed towards the short1t. ∆t, it is only on these occasions that the ensemble higher departs from the control run whereas most of the time the system is in a quasi-linear regime where almost no inflation 5.4.2 An empirical solution is needed. The behavior of ζ ⋆ is illustrated in Fig. 10 in the case ∆t = 0.05. Exploring the small 1t regime the behaviour of the The corresponding results for and the bundle IEnKF-N aredual recost function (27),itswe foundvariant. that, in that regime, thethe arported in Fig. Eq. 11 for primal Even though gumentessentially ζ ? of its minimum mostly given by thediagnosed maximum capping solves theisdivergence problem of the IEnKF-N interval, that is N/εN . But if N/ε asymptotically N optimal earlier, underperforms IEnKF with inflabehaves N linear − 1, itregime. is bigger than N 1 at finiteprimal N, for tion in the like nearly Guided by−the capped εN dual = 1 EnKF-N or εN = results, 1 + 1/N. instance, inathe N =of3 and weFor conjecture that dualcase variant andcapped εN = IEnKF-N 1 + 1/N, would one hasat N/ε = 2.25 as compared the leastN partially close this gap.to N −1 =implementing 2. Hence, in that EnKF-N is tends to often imHowever theregime, dual IEnKF-N certainly chalplicitlyand deflate thean ensemble, which may lead to an overconlenging left as open question. fident Kalman to its divergence. In addition to filter beingand empirical, the capping solution cannot Fromsatisfying this educated propose to tuning. cap theHowargube fully since guess, it leadswe back to some ? of the minimum of Eq. (27) at N − 1. Note that ment ζ ever in the context of this numerical experiment, no scalar the tuned. case ζ ?It=was N −only 1 corresponds to distinguish an absence the of inflation was necessary to weakly and does not a priori guarantee the filter’s stability. Morenon-linear regime from the rest of the ∆t range. This also ? , we prefer to modify Eq. (27) over, rather than capping ζ suggests that, in the context of this experiment, a genuinely in such a way that the valueofofinflation ζ ? cannot exceed satisfactory solution thatmaximum avoids tuning or distinN −1. The first reason for doing so is that the duality result of guishing between regimes, would necessitate a fine modeling Appendix A is derived on the full interval ]0, N/ε ], and an N the parof the hyperprior, beyond Jeffrey’s prior that leads to abruptform truncation would ticular of the dual costinvalidate function the Eq. duality (27). ?equivalence. The second reason is that a direct capping of ζ requires an access to Eq. (27), which is not straightforward in the case the primal schemes. For these two reasons, we propose 6 of Conclusion to renormalize εN to εN = N/(N − 1). That way, it is easy that more the maximum of ζ ? is check through Eq. (23) Intothis article, we have further explored value two recently ? ? ζ = N −ensemble 1 at ρ =Kalman 0. In thefilters following will in refer developed meantcapping to operate highto dimensional geophysical systems. We have presented and Nonlin. Processes Geophys., 19, 383–399, 2012

justified their algorithms. They have been mostly tested on the ’95 chaoticoftoy thisLorenz renormalization εN model. , and not to the abrupt truncation ofThe Eq. first (27) to the interval ]0, N −ensemble 1]. filter, the finite-size Kalman filter, is The resulting of the capped primal built to reduce theperformances impact of sampling errors,EnKF-N, and was shown, dual variants, displayed in Fig. 9. With our setup, the inand a perfect modelare context, to significantly reduce the need dual EnKF-NWe outperforms or that equals for inflation. have shown the EnKF schemewith canoptimally be made tuned inflation over the enlarged rangeEnKF, 1t ∈ [0.05, 0.50]. equivalent to a traditional deterministic but with an The benefit of the dual filter over the primal variant is now inflation of the prior, whose value is determined by the mineven clearer in the almost linear regime. imum of a one-dimensional non-necessarily convex cost? remains close to N − 1, with occasional In that regime, ζ function. Assuming that EnKF-N significantly reduces the excursions to smaller values. for These events errors, are mostly proneed for inflation accounting sampling this sugvoked by transitions of the dynamics of the Lorenz ’63 model gests why inflation is usually so efficient in accounting for between lobes. of the short 1t, it ismathematically only on these sampling errors.Because It tells that this efficiency occasions that the ensemble departs from the control run, comes from the invariance of the ensemble by permutation whereas most of the time the system is in a of its members. Through a dual transformation, itquasi-linear was shown regime where almost no inflation is needed. The behaviour how to find the global minimum of the EnKF-N analysis cost ? is illustrated in Fig. 10 in the case 1t = 0.05. of ζ function. This solves one of the open questions raised by The corresponding results for the bundle IEnKF-N are reBocquet (2011). ported in Fig. 11 for its primal variant. Even though the The second filter, the iterative ensemble Kalman filter outcapping essentially solves the divergence problem diagnosed performs EnKF in strongly nonlinear regime, at the cost of earlier, IEnKF-N underperforms IEnKF with optimal inflaiterating the ensemble propagation between updates. The imtion in the nearly linear regime. Guided by the capped primal plementation of Sakov et al. (2012) corresponds to a Newton and dual EnKF-N results, we conjecture that a dual variant of method in solving the underlying analysis cost function. In the capped IEnKF-N would at least partially close this gap. this article, we have proposed to use a Levenberg-Marquardt However, implementing the dual IEnKF-N is certainly chalimplementation of the filter instead. It lenging and left as an open question. offers a better control on In theaddition convergence, the empirical, positive-definiteness the Hessian to being the cappingofsolution canand seems to require less iterations in strongly nonlinear connot be fully satisfying since it leads back to some tuning. ditions. However is less of efficient in mild nonlinear condiHowever, in the it context this numerical experiment, no tions where the number of required iterations is small. scalar was tuned. It was only necessary to distinguish The the transform and bundle variant IEnKF similar weakly nonlinear regime fromofthe rest oflead the to 1tvery range. This results and we suggest that they should be considered as also suggests that, in the context of this experiment, a varigenants of asatisfactory same IEnKF filter. that avoids tuning of inflation uinely solution Exploiting their complementarity, the two filtering approaches have been combined into an inflation-free iterative www.nonlin-processes-geophys.net/19/383/2012/

M.Bocquet, BocquetP.and P. Sakov: The iterative finite-size ensemble M. Sakov: The iterative finite-size ensemble KalmanKalman filter filter

395 13

M. Bocquet, P. Sakov: The iterative finite-size ensemble Kalman filter EnKF optimal inflation Capped dual EnKF-N Capped primal EnKF-N EnKF optimal inflation Capped dual EnKF-N Capped primal EnKF-N

1.01.2 0.81.0

0.8 0.6 0.6 0.4

σ

0.4 0.2

0

0

2

σ

0.2 0

0.1 0

0.1

0.2

EnKF optimal inflation Capped dual EnKF-N Capped primal EnKF-N primal EnKF optimal inflation Capped dual EnKF-N Capped primal EnKF-N primal

3.0

0.3

0.4

Time interval between updates 0.2 0.3 0.4 Time interval between updates

obs

=1

2 obs

=1

0.5 0.5

Analysis root square mean square Analysis root mean error error

Analysis root mean square error Analysis root mean square error

1.2

13

3.0 2.5 2.5 2.0 2.0 1.5 1.5 1.0

σ

1.0 0.5 0.5 0

0

0

0

σ 0.1 0.1

0.2

0.3

0.4

Time interval between updates 0.2 0.3 0.4 Time interval between updates

2 obs

2 obs

=8

=8 0.5

0.5

Fig. 9. Analysis root mean square error for three filters applied to the Lorenz ’63 model (N = 3): the EnKF with optimally tuned inflation, Fig. 9. Analysis error filterswith applied to the Left Lorenz ’63illustrates model (Nthe = 3): tunedillustrates inflation, the primal EnKF-Nroot-mean-square with capping, and the for dualthree EnKF-N capping. panel σ 2 the=EnKF 1 casewith and optimally the right panel Fig.2 9. Analysis root mean square error for three filters applied to the Lorenz ’63 model (N = 3):obs with optimally tuned inflation, 2the=EnKF theσobs primal 1 case and the right panel illustrates the = 8EnKF-N case. with capping and the dual EnKF-N with capping. Left panel illustrates the σ2obs the primal with capping, and the dual EnKF-N with capping. Left panel illustrates the σobs = 1 case and the right panel illustrates 22 = 8EnKF-N the theσσobs = 8 case. case. Optimal efficient rank of the prior ζ x-axis a Optimal efficient rank of the prior ζ x-axis

a

obs

2 2

1.5 1.5

1 1

0.5 0.5

0 20 0 10 20

0 10 -10 0 -20-10 0 -20

200 0

200

400

600

Time of Lorenz ’63 model 400

600

800 800

1000 1000

Time of Lorenz ’63 model a

Fig. 10. Sequence of efficient ranks ζ in a long run of a capped 2 dual appliedoftoefficient the Lorenz withrun σobs 1 and Fig.EnKF-N 10. Sequence ranks’63 ζ amodel, in a long of a=capped 2 a capped Fig. 10. Sequence of efficient ζ a model, inlinear a long runσof dual EnKF-N applied to the Lorenz ’63 with = 1with and ∆t = 0.05 which corresponds toranks a nearly regime, and obs 2 = 1 and the Lorenz ’63 model, with σobs ∆t =EnKF-N 0.05 which to The a nearly linear regime, andthe with andual ensemble of applied N =corresponds 3 to members. lower graph displays x 1t 0.05 which to a The nearly linear regime, andthe with an = ensemble of Ncorresponds =’63 3 members. lower graph displays variable of the Lorenz reference trajectory (the truth). Note thex an ensemble N = 3’63 members. lower graph displays variable of theofLorenz (the truth). thex good correlation between thereference periodThe oftrajectory relative stability forNote ζ ⋆the and ⋆ variable of the Lorenz ’63 reference trajectory (the truth). Note the good correlation between the period of relative stability for ζ and the presence of the true state in orbit within a lobe. good correlation between the in period relative stability for ζ ? and the presence of the true state orbit of within a lobe. the presence of the true state in orbit within a lobe.

ensemble Kalman filter. The Levenberg-Marquardt scheme ensemble filter.control The Levenberg-Marquardt provides theKalman necessary on the minimization scheme of the or distinguishing between regimes, would necessitate a fine provides the necessary control on the minimization of the non-convex underlying cost function. Their performances modelling of the hyperprior, beyond Jeffrey’s prior that leads non-convex underlying cost function. Their performances are close to that of the IEnKF with optimized inflation. to particular form dual cost function Eq. (27). arethe close toof that of of thethe IEnKF with optimized inflation. The number ensemble propagation required by IEnKF-N The number of ensemble propagation required by IEnKF-N seems to be very similar to that of IEnKF in the same context. seems to be very similar to that of IEnKF in the same context. In this article, we have deliberately eluded the rank prob6 In Conclusion this article, we have deliberately eluded the rank problem aspect of the EnKF. To go beyond, and extend the results lem aspect of the EnKF. To go beyond, and extend the results toInlow-rank ensemble, onefurther shouldmore additionally article, we have exploredbuilt twolocalizarecently to this low-rank ensemble, one should additionally built localization into this algorithm as was for instance done for EnKF-N developed ensemble Kalman filters meant to operate in high tion into this algorithm as was for instance done for EnKF-N in Bocquet (2011). But it should bring us far from the prein Bocquet (2011). But it should bring us far from the prewww.nonlin-processes-geophys.net/19/383/2012/

liminary objectives of this article. dimensional geophysical systems. We have presented and liminary objectives of this article. A possible improvement over the iterative filters in their justified their algorithms. They have been mostly tested on Levenberg-Marquardt wouldfilters be to re-shape possible over the iterative in their theALorenz ’95improvement chaoticimplementation, toy model. the algorithm so that it avoids extra fundamentally unnecesLevenberg-Marquardt implementation, would be to re-shape The first filter, the finite-size ensemble Kalman filter, is sary iterations for small time interval between updates. One the it avoids extra fundamentally unnecesbuiltalgorithm to reduceso thethat impact of sampling errors, and was shown, has to keep in mind that the Levenberg-Marquardt method sary iterations for small time interval between updates. One in a perfect model context, to significantly reduce the need was designed reduce the large number of iterations needed has to keep into mind that the Levenberg-Marquardt method for inflation. We have shown that the scheme can be made in the optimization of a system built on a nonlinear model, was designed to reduce the large number of iterations needed equivalent to a traditional deterministic EnKF, but with an and, theofpresent form, is not optimal system on a in theinoptimization ofwhose a system built onfor a nonlinear model, inflation the prior, value is determined bybuilt the minmildly model. and, in nonlinear the apresent form, is not non-necessarily optimal for system built on a imum of one-dimensional convex costmildly nonlinear model. function. EnKF-N significantly reduces the AnotherAssuming previouslythat identified challenge is to built an optineed for inflation accounting for sampling errors, this sugmization algorithm foridentified the cost function dualan IEnKFAnother previously challengeofis the to built optigests whyalgorithm inflationfor is usually efficient for mization the cost so function of in theaccounting dual IEnKFN. sampling errors. It tells that this efficiency mathematically N. comes the invariance the ensemble by permutation In thefrom last section of this of article, we have identified a deof In itsthe members. Through a dual transformation, it wasregime shown last section this article, weexactly) have identified a demanding regime, theof almost (but not linear how atosmall find the global minimum ofexactly) the EnKF-N manding regime, the almost (but linear analysis regime with ensemble, where the not finite-size (iterative or not) cost afunction. This solves one the questions raised with small ensemble, where theoffinite-size (iterative not) EnKFs do not apparently succeed in open determining aorproper by Bocquet (2011). EnKFs do not apparently in adetermining a proper effective inflation. We havesucceed proposed motivated but empireffective inflation. proposed a motivated but empirThe second filter,We thehave iterative ensemble filter, outical solution which consists in capping theKalman diagnosed value ical solution consists cappingregime, the valueofa performs EnKF in strongly nonlinear cost of degrees ofwhich freedom in thein ensemble at diagnosed N −at1the through of degrees incost thefunction ensemble Nupdates. − For 1 through a iterating theofensemble between The immodification offreedom the dualpropagation Eq.at(27). the cases modification ofofthis the dual function Eq. (27). For the cases plementation Sakov etcost al. (2012) corresponds a Newton under scrutiny, offers a satisfying solution intothe EnKFunder this the offers a satisfying solution in function. theHowever EnKF-In method in solving underlying cost N casescrutiny, and a promising one in the analysis IEnKF-N case. N case and a promising one in the IEnKF-N case. However article, we have proposed to usesuch a Levenberg-Marquardt itthis surely requires a robust argument as the derivation of itimplementation surely requires acost robust argument as the of thefunction filter instead. offers a derivation better control the modified dualof fromsuch aItmore fitted hyperprior. the modified dual cost function from a more fitted hyperprior. on the convergence, the positive-definiteness of the Hessian Additionally it was shown in this special case that knowledge Additionally was shown in special case that knowledge and toit require less in strongly nonlinear conof theseems global minimum ofiterations thethis analysis, which is provided by of the global minimum of the isbetter provided by ditions. However, it leads is less in which mild nonlinear condithe dual algorithm, toefficient aanalysis, systematically perforthe algorithm, leads a systematically perfortionsdual where theprimal number oftorequired iterationsbetter is small. The mance than the algorithm. mance than the primal algorithm. transform and bundle variant of IEnKF lead to very similar For the term, we that this results andlonger we suggest thatbelieve they should be finite-size considerediterative as variFor the longer term, we believe that this finite-size iterative ensemble Kalman filter can be seen as a first elementary oneants of a same IEnKF filter. ensemble Kalman filter can be seen as a first elementary onelag brick of an efficient ensemble Kalman smoother. lag brick of an efficient ensemble Kalman smoother. Nonlin. Processes Geophys., 19, 383–399, 2012

396 14

M. Bocquet and P. Sakov: The iterative M. Bocquet, P. Sakov: The iterativefinite-size finite-sizeensemble ensembleKalman Kalmanfilter filter

0.9

0.7

3.0

IEnKF optimal inflation Capped primal IEnKF-N EnKF optimal inflation Capped dual EnKF-N

0.6 0.5 0.4 0.3 0.2

σ

2 obs

=1

Analysis root mean square error

Analysis root mean square error

0.8

IEnKF optimal inflation Capped primal IEnKF-N EnKF optimal inflation Capped dual EnKF-N

2.5 2.0 1.5 1.0

σ

0.5

2 obs

=8

0.1 0

0

0.1

0.2

0.3

0.4

0.5

0

0

0.1

Time interval between updates

0.2

0.3

0.4

0.5

Time interval between updates

Fig. 11. Analysis root mean square error for two iterative filters applied to the Lorenz ’63 model (N = 3): the IEnKF, bundle variant with 2 Fig. 11. tuned Analysis root-mean-square erroralgorithm for two iterative filters applied the Lorenz ’63 variant. model (N = 3): theillustrates IEnKF, bundle variant optimally inflation, and the primal for the IEnKF-N withtocapping, bundle Left panel the σobs = 1 with case 2 = 1 case 2 optimally tuned andthe theσobs primal the IEnKF-N with capping, bundleand variant. Left panelwith illustrates σobs and the right panelinflation, illustrates = 8 algorithm case. Thefor EnKF with optimally tuned inflation dual EnKF-N cappingthe results of Fig. 7 2 = 8 case. The EnKF with optimally tuned inflation and dual EnKF-N with capping results of Fig. 7 are forpanel comparison. andreported the right illustrates the σobs are reported for comparison.

Appendix A Exploiting their complementarity, the two filtering approaches have been an inflation-free iterative Strong duality of thecombined EnKF-Ninto optimization problem ensemble Kalman filter. The Levenberg-Marquardt scheme Inprovides this appendix, a proof is given thatminimisation the dual andofprithe necessary control on the the mal problems lead to the same global minimum. We denon-convex underlying cost function. Their performances are fine thetoparticular transform of aThe function close that of theLegrendre-Fenchel IEnKF with optimised inflation. numαber > 0of→ensemble h(α) as the function βrequired > 0 → hby∗(β) definedseems by to propagation IEnKF-N be very similar to that of IEnKF in the same context. h∗ (β) = inf (αβ −we h(α)) (A1) In this article, have. deliberately eluded the rank probα>0 lem aspect of the EnKF. To go beyond, and extend the results Applying this transform to function f defined by Eq. (18), it to low-rank ensemble, one should additionally built localizais easy to check that: tion into this algorithm as was for instance done for EnKF-N  in BocquetN(2011). But it N should bring us far from the pre− εN ζ − N ln ∗ ζ for ζ ∈]0,N/εN ] (A2) fliminary (ζ) = objectives of this article. −N lnεN for ζ > N/εN , A possible improvement over the iterative filters in their and, using the concavityimplementation, of the ln function, andbeintoparticular Levenberg-Marquardt would re-shape for ρ ≥ 0 so that it avoids extra fundamentally unnecestheany algorithm sary iterations for small time N interval between updates. One Nhas ln(ε + ρ) in ≤N lnεNthat + theρ,Levenberg-Marquardt method (A3) toNkeep mind ε was designed to reduce the N large number of iterations needed itinis the found that for anyofρa>system 0 optimisation built on a nonlinear model and, in the present form, is not optimal for system built on a f ∗∗ (ρ) = N ln(εN + ρ). (A4) mildly nonlinear model. Anotherf previously identified challenge is Therefore coincides with its bi-conjugate f ∗∗to. build an optimisation algorithm for the cost function of the problem dual IEnKFTo prove strong-duality, we consider the dual and N. we gradually transform it into the primal one: In the last section of this article, we have identified a de∆manding = inf D(ζ) regime, the almost (but not exactly) linear regime ζ>0  the finite-size (iterative or not)  ensemble, where with a small EnKFs do not apparently succeed in determining a proper = inf inf sup L(w,ρ,ζ) w ρ≥0 ζ>0 effective inflation. We have proposed a motivated but empir  ical solution which consists in capping the diagnosed value = inf inf sup L(w,ρ,ζ) ζ>0 w

ρ≥0

Nonlin. Processes Geophys., 19, 383–399, 2012

  1 = inf inf g(w) + ζwT w − inf (ζρ − f (ρ)) w freedom in the ensemble ρ≥0 2 ζ>0 of of degrees at N − 1 through a
1 modification of the dual cost T function ∗ Eq. (27). For the cases = inf inf g(w) + ζw w − f (ζ) under2scrutiny, ζ>0 w this offers a satisfying solution in the EnKF  N case
case. However, 1 and a promising one Tin the IEnKF-N ∗ g(w)a+robust inf ζw w − f such (ζ) as the derivation of = infrequires it surely argument ζ>0 2 w the modified dual cost function
from a more fitted hyperprior. 1 ∗∗ Additionally it was this special case that knowledge = inf g(w) + fshown (wTinw) w 2 of the global minimum of the analysis, which is provided by
1 the=dualinfalgorithm, to a systematically better perforg(w) + fleads (wT w) w mance2 than the primal algorithm. = (w) term, we believe that this finite-size iterative Forinf theJelonger w ensemble one= Π. Kalman filter can be seen as a first elementary (A5) lag brick of an efficient ensemble Kalman smoother. The key assumptions that make this derivation possible are the following. First the infima of D(ζ), and Je(w) in Eq. (A5) are attained and make the derivation meaningful. Appendix we A used the fact that f defined by Eq. (18) coSecondly, incides with its bi-conjugate: f ∗∗ = f , as proven above. It Strong of thethat EnKF-N optimisation can alsoduality be checked the infimum of D(ζ)problem over ζ > 0 is attained in ]0,N/εN ]. In this appendix, proof is given that the dual and primal problems lead to the same global minimum. We define the particular Legrendre-Fenchel transform of a function α > 0 → Appendix B h(α) as the function β > 0 → h∗ (β) defined by Algorithms of the iterative filters h∗ (β) = inf (αβ − h(α)) . (A1) α>0

Acknowledgements. The authors thank two anonymous reviewers for their useful comments.

Applying this transform to function f defined by Eq. (18), it is easy to check that: References  N − εN ζ − N ln Nζ for ζ ∈]0, N/εN ] ∗ f (ζ ) = (A2) Anderson, J.−N L. and Anderson, S.for L.:ζ A>Monte Implemenln εN N/εN Carlo ,

tation of the Nonlinear Filtering Problem to Produce Ensemble

www.nonlin-processes-geophys.net/19/383/2012/

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter and, using the concavity of the ln function and, in particular, for any ρ ≥ 0 N ln(εN + ρ) ≤ N ln εN +

N ρ, εN

(A3)

it is found that for any ρ > 0 f ∗∗ (ρ) = N ln(εN + ρ) .

(A4)

Therefore, f coincides with its bi-conjugate f ∗∗ . To prove strong-duality, we consider the dual problem and we gradually transform it into the primal one: 1 = inf D(ζ ) ζ >0

! = inf inf sup L(w, ρ, ζ ) ζ >0

w ρ≥0

! = inf inf sup L(w, ρ, ζ ) ζ >0 w

ρ≥0

  1 = inf inf g(w) + ζ w T w − inf (ζρ − f (ρ)) ρ≥0 2 ζ >0 w   1 = inf inf g(w) + ζ w T w − f ∗ (ζ ) 2 ζ >0 w    1 T ∗ = inf g(w) + inf ζ w w − f (ζ ) ζ >0 2 w  1  = inf g(w) + f ∗∗ (w T w) 2 w  1  = inf g(w) + f (w T w) 2 w = inf Je(w) w

= 5.

(A5)

The key assumptions that make this derivation possible are the following. First the infima of D(ζ ), and Je(w) in Eq. (A5) are attained and make the derivation meaningful. Secondly, we used the fact that f defined by Eq. (18) coincides with its bi-conjugate: f ∗∗ = f , as proven above. It can also be checked that the infimum of D(ζ ) over ζ > 0 is attained in ]0, N/εN ]. Appendix B Algorithms of the iterative filters

www.nonlin-processes-geophys.net/19/383/2012/

397

Algorithm 3 Levenberg-Marquardt IEnKF. Require: Transition model from t1 to t2 ; M, an observation opf erator H . Algorithm parameters: , τ , e1 , e2 , pmax . E1 , the forecast ensemble at t1 , y the observation at t2 f 1: Compute x and A from E1 2: p = 0, ν = 2, w = 0 3: x1 = x + Aw 4: y2 = H M(x1 ) 5: T = IN (transform) 6: E1 = x1 + A (bundle), E1 = x1 + AT (transform) 7: E2 = M(E1 ) 8: Y2 = (H (E2 ) − y2 )/ (bundle), Y2 = (H (E2 ) − y2 )T−1 (transform) T 9: Je = 12 (y − y2 )T R−1 (y − y2 ) + N−1 2 w w T −1 10: ∇ Je = (N − 1)w − Y2 R (y − y2 ) e = (N − 1)IN + YT R−1 Y2 11: H 2

ekk 12: flag = ||∇ Je||∞ > e1 , µ = τ max H 13: while flag and p < pmax do 14: p := p + 1
e + µIN 1w = −∇ Je 15: Solve H 16: if ||1w|| ≤ e2 then 17: flag = FALSE 18: else 19: w0 = w + 1w 20: x1 = x + Aw0 21: y2 = H M(x1 )
22: L = 12 1wT µ1w − ∇ Je Je0 = 21 (y − y2 )T R−1 (y − y2 ) + N 2−1 w0T w0 23: 24: θ = (Je − Je0 )/L 25: if θ > 0 then Je = Je0 26: 27: w = w0 28: E1 = x1 + A (bundle), E1 = x1 + AT (transform) 29: E2 = M(E1 ) 30: Y2 = (H (E2 ) − y2 )/ (bundle), Y2 = (H (E2 ) − y2 )T−1 (transform) −1 31: ∇ Je = (N − 1)w − YT 2 R (y − y2 ) e = (N − 1)IN + YT R−1 Y2 32: H 2 1 e− 2 (transform) 33: T=H
34: flag = ||∇ Je||∞ > e1 n o 35: µ := µ max 13 , 1 − (2θ − 1)3 , ν = 2 36: else 37: µ := µ ν, ν := 2 ν 38: end if 39: end if 40: end while 41: x1 = x + Aw 1 e− 2 (bundle) 42: T = H 43: E1 = x1 + AT 44: E2 = M(E1 ) 45: E2 := x2 + λ (E2 − x2 )

Nonlin. Processes Geophys., 19, 383–399, 2012

398

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter

Algorithm 4 Levenberg-Marquardt IEnKF-N. Require: Transition model from t1 to t2 : M, an observation opf erator H . Algorithm parameters: , τ , e1 , e2 , pmax . E1 , the forecast ensemble at t1 , y the observation at t2 f 1: Compute x and A from E1 2: p = 0, ν = 2, w = 0 3: x1 = x + Aw 4: y2 = H M(x1 ) 5: T = IN (transform) 6: E1 = x1 + A (bundle), E1 = x1 + AT (transform) 7: E2 = M(E1 ) 8: Y2 = (H (E2 ) − y2 )/ (bundle), Y2 = (H (E2 ) − y2 )T−1 (transform)   9: Je = 12 (y − y2 )T R−1 (y − y2 ) + N2 ln εN + wT w w 10: ∇ Je = N − YT R−1 (y − y2 ) 2 εN +wT w
ε +wT w IN −2wwT N −1 e 11: H = N + YT
2 2 R Y2 εN +wT w



ekk 12: flag = ||∇ Je||∞ > e1 , µ = τ max H 13: while flag and p < pmax do 14: p := p + 1
e + µIN 1w = −∇ Je 15: Solve H 16: if ||1w|| ≤ e2 then 17: flag = FALSE 18: else 19: w0 = w + 1w 20: x1 = x + Aw 21: y2 = H M(x1 )
22: L = 12 1wT µ1w − ∇ Je   23: Je0 = 12 (y − y2 )T R−1 (y − y2 ) + N2 ln εN + w0T w0 24: θ = (Je − Je0 )/L 25: if θ > 0 then 26: Je = Je0 27: w = w0 28: E1 = x1 + A (bundle), E1 = x1 + AT (transform) 29: E2 = M(E1 ) 30: Y2 = (H (E2 ) − y2 )/ (bundle), Y2 = (H (E2 ) − y2 )T−1 (transform) w T −1 (y − y ) 31: ∇ Je = N 2 T − Y2 R εN +w w


32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44:

T T −1 e = N εN +w w IN −2ww H + YT
2 2 R Y2 εN +wT w 1

e− 2 (transform) T=H
flag = ||∇ Je||∞ > e1 o n µ := µ max 13 , 1 − (2θ − 1)3 , ν = 2 else µ := µ ν, ν := 2 ν end if end if end while x1 = x + Aw 1 e− 2 (bundle) T=H E1 = x1 + AT E2 = M(E1 )

Nonlin. Processes Geophys., 19, 383–399, 2012

Acknowledgements. The authors thank two anonymous reviewers for their useful comments. Edited by: J. Duan Reviewed by: two anonymous referees

References Anderson, J. L. and Anderson, S. L.: A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts, Mon. Weather Rev., 127, 2741–2758, 1999. Bocquet, M.: Ensemble Kalman filtering without the intrinsic need for inflation, Nonlin. Processes Geophys., 18, 735–750, doi:10.5194/npg-18-735-2011, 2011. Borwein, J. M. and Lewis, A. S.: Convex analysis and nonlinear optimization: theory and examples, Springer, 2000. Byrd, R. H., Lu, P., and Nocedal, J.: A Limited Memory Algorithm for Bound Constrained Optimization, J. Sci. Stat. Comput., 16, 1190–1208, 1995. Gu, Y. and Oliver, D. S.: An Iterative Ensemble Kalman Filter for Multiphase Fluid Flow Data Assimilation, Soc. Petrol. Eng. J., 12, 438–446, 2007. Harlim, J. and Hunt, B.: A non-Gaussian ensemble filter for assimilating infrequent noisy observations, Tellus A, 59, 225–237, 2007. Hunt, B. R., Kostelich, E. J., and Szunyogh, I.: Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter, Physica D, 230, 112–126, 2007. Jazwinski, A. H.: Stochastic Processes and Filtering Theory, Academic Press, 1970. Kalnay, E. and Yang, S.-C.: Accelerating the spin-up of Ensemble Kalman Filtering, Q. J. Roy. Meteor. Soc., 136, 1644–1651, 2010. Levenberg, K.: A Method for the Solution of Certain Problems in Least Squares, Quart. Appl. Math., 2, 164–168, 1944. Lorenz, E. N.: Deterministic nonperiodic flow, J. Atmos. Sci., 20, 130–141, 1963. Lorenz, E. N. and Emmanuel, K. E.: Optimal sites for supplementary weather observations: simulation with a small model, J. Atmos. Sci., 55, 399–414, 1998. Madsen, K., Nielsen, H. B., and Tingleff, O.: Methods for nonlinear least square problems, Tech. rep., Informatics and Mathematical Modelling, Technical University of Denmark, 2nd Edn., 2004. Marquardt, D.: An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math., 11, 431–441, 1963. Nocedal, J. and Wright, S. J.: Numerical Optimization, Springer Series in Operations Research, Springer, 2006. Pires, C., Vautard, R., and Talagrand, O.: On extending the limits of variational assimilation in nonlinear chaotic systems, Tellus A, 48, 96–121, 1996. Sakov, P. and Oke, P. R.: Implications of the Form of the Ensemble Transformation in the Ensemble Square Root Filters, Mon. Weather Rev., 136, 1042–1053, 2008. Sakov, P., Oliver, D., and Bertino, L.: An iterative EnKF for strongly nonlinear systems, Mon. Weather Rev., 140, 1988– 2004, doi:10.1175/MWR-D-11-00145.1, 2012.

www.nonlin-processes-geophys.net/19/383/2012/

M. Bocquet and P. Sakov: The iterative finite-size ensemble Kalman filter Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM, 352 pp., 2005. Wishner, R. P., Tabaczynski, J. A., and Athans, M.: A Comparison of Three Non-Linear Filters, Automatica, 5, 487–496, 1969.

www.nonlin-processes-geophys.net/19/383/2012/

399

Zupanski, M.: Maximum Likelihood Ensemble Filter: Theoretical Aspects, Mon. Weather Rev., 133, 1710–1726, 2005.

Nonlin. Processes Geophys., 19, 383–399, 2012