Moving-Horizon Estimation for Linear System affected ...

Chapter

Moving-Horizon Estimation for Linear System affected by Outliers

Moving-horizon state estimation is addressed for discrete-time linear systems with disturbances acting on the dynamic and measurement equations. In particular, the measurement noises can take abnormal values, usually called outliers. For such systems, one can adopt a Kalman filter with estimate update that depends on the result of a statistical test on the residuals. As an alternative to such a method, a robust moving-horizon estimator is proposed. Such a method provides estimates of the state variables obtained by minimizing a set of least-squares cost functions by leaving out in turn all the measurements that can be affected by outliers. At each time instant, the estimate that corresponds to the lowest cost is retained and propagated ahead at the next step, where the procedure is repeated with the new batch of measures. The stability of the estimation error for the proposed moving-horizon estimator is proved under mild conditions concerning the observability of the free-noise dynamic equations and the selection of a tuning parameter in the cost function. The effectiveness the proposed method as compared with the Kalman filter is shown by means of simulations and estimation errors comparison.

2

68 2. Moving-Horizon Estimation for Linear System affected by Outliers

Figure 2.1: Moving-Horizon Strategy - N + 1 window

• Moving-Horizon Estimation (MHE) is an online optimization-based strategy for process monitoring and state estimation.

• Investing the success of receding horizon strategies in the estimation of dy-

namic states and parameters, for linear and nonlinear systems, recent attention has been concentrated on the MHE. [97], [99], [113] and [114].

• The interest of the MHE is the possibility of dealing with a limited amount

of data, instead of using all the information available from the beginning. Only a fixed amount of most recent information is used to solve the optimization problem, so that the oldest measurement sample is discarded as a new sample becomes available [99], [96], [97], and [101].

• In this framework the strategy of MHE is to transform the estimation prob-

lem into a quadratic program using a moving fixed-size estimation window. The fixed-size estimation window is necessary to bound the size of the quadratic program.

• Robustness is a fundamental requirement in the design of filters for uncertain systems. [104] and [115].

• Outliers are particular type of uncertainty.cite our paper

2.1. Introduction

2.1

69

Introduction

In a number of industrial applications the measurements of state variables may be affected by gross observation errors due to sensor malfunctions, wrong replacement of measures, or simply large stochastic noise. Such data are usually called outliers, and a vast literature exists on signal-processing methods to attenuate their effect or detect them for the purpose of rejection (see, e.g., [93] and the references therein). In this chapter, we move from a different point of view, as we deal with the problem of estimating the state variables of a linear system by means of measures that may be corrupted by outliers. As is well-known, under the assumption that initial state and disturbances are white Gaussian stochastic processes, the best estimator in the sense of the minimization of the expected quadratic estimation error is the Kalman filter. Such an estimator is recursive, and thus well-suited to treating the new measures by iterating the estimate update based on the current residual, i.e., the output error given by the difference between the measure and its prediction based on the state estimate. Thus, it is not difficult to account for outliers by checking abnormal residuals via a threshold test and avoiding to use them for the Kalman estimate update. Such a method can be derived by reformulating the Kalman filtering as maximum-likelihood optimization problem with additional constraints on the amplitude of the noise. A similar idea is applied in [94] to solve the problem of the so-called event-based estimation, i.e., Kalman filtering with distributed sensors and a centralized estimator subject to the reduction of sensor-to-estimator communications for the purpose of energy saving. Moving-horizon estimation (MHE) is a research topic that has gained a lot of attention in the last decades in parallel with the success of model predictive control. First ideas behind MHE on performing state estimation for dynamic systems by using a limited amount of most recent information are presented in the pioneering work of Jazwinski [95]. Such estimates can be obtained by minimizing a least-squares estimation function defined on a batch of inputs and outputs according to a moving-horizon strategy. One of the advantages of MHE is the capability of accounting constraints on the state variables (see, e.g., [96–99, 101, 102, 112]). Outliers are particular type of uncertainty, which in general prevent an estimator from ensuring guaranteed performances [103]. Robustness is thus a fundamen-

70 2. Moving-Horizon Estimation for Linear System affected by Outliers

Figure 2.2: Leave-out measures strategy - N + 1 window

tal requirement in the design of filters for uncertain systems. Clearly, the basic requirement for an estimator to be robust to outliers is the stability of the estimation error. Conditions for the stability of moving horizon estimators for uncertain linear systems are reported in [104, 105], where explicit bounding sequences are provided thanks to the adoption of worst-case cost functions. Unfortunately, such cost functions are not helpful in the case of measurements affected by outliers, thus a different criterion is proposed here. More specifically, at each time instant we separately minimize a set of least-squares cost functions, where the measurements that can be affected by outliers are left out in turn in such a way to generate a robust estimate (see Fig. 2.2). As a resulting estimate, we choose the minimizer associated with the lowest cost, and such an estimate is propagated ahead at the next time instant according to the moving horizon strategy. The chapter is organized as follows. In Section 2.2, we consider the problem of constructing a Kalman filter with estimate update that depends on a residual check to account for possible outliers. Section 2.3 concerns the proposed moving-horizon estimation method with a proof of stability given in Section 2.4. In Section 2.5, simulation results are presented for the purpose of comparison with the Kalman filter and more simulations results in Section 2.6. comments and application view are in Section 2.7. Finally, the conclusions are drawn in Section 2.8. Let us denote by λ(P ) and

71

2.1. Introduction

¯ ) the minimum and maximum eigenvalues of a symmetric matrix P , respecλ(P tively; moreover, P > 0 denotes that it is positive definite. Given a generic matrix $
⊤ M ) 1/2 . Given a sequence of vectors v , v ¯ M , kM k := λ(M i i+1 , . . . , vj for i < j, j j let us define vi := col (vi , vi+1 , . . . , vj ) and vi := col (vl , l = i, i + 1, . . . , j, i 6= k)

with i ≤ k ≤ j.

k

Figure 2.3: Kalman filtering set-up

The Kalman filter (KF) is a widely used tool for estimating the state of a dynamic system, given noisy measurement data. It is the optimal linear estimator for linear Gaussian systems, giving the minimum mean squared error. Using state estimates, the filter can also estimate what the corresponding (output) data are. However, the performance of the Kalman filter degrades when the observed data contains outliers. To address this, many previous works have tried to make the Kalman filter more robust to outliers with different class approaches. In our scope we use a KF with threshold parameters for the mean of comparison with the proposed MHF. The required tuning of threshold parameters for optimal performance, maybe difficult to assure correct or accurate estimates because of bad choices of the weights, and thus, may lead to deteriorated performance property and that is shown in Section 2.5.Any errors and deviations from assumptions that are not captured, causing the least squares estimator underlying the KF to yield strongly biased results in the presence of just a single observation, innovation, or structural outlier. Furthermore, generally just two deviating observations among a set of 1000 points are sufficient for ℓ2 -norm to be less efficient than ℓ1 -norm. So again, while optimal at the Gaussian distribution in the mean-squared sense, the KF is not robust or highly efficient under contamination.

72 2. Moving-Horizon Estimation for Linear System affected by Outliers

2.2

Recursive estimation in the presence of outliers

Consider the system xt+1 = A xt + B ut + wt

(2.1a)

yt = C x t + v t

(2.1b)

where t = 0, 1, . . . is the time instant, xt ∈ Rn is the state vector, ut ∈ Rm is

the control vector, wt ∈ Rn is the system noise vector. For the sake of simplicity,

we suppose to deal with scalar measures yt ∈ R, and let vt ∈ R the measurement

noise. We assume that the initial state x0 is Gaussian with mean x ¯0 and covariance P0 = P0⊤ > 0; wt is zero-mean Gaussian with covariance Q = Q⊤ > 0. Moreover,

x0 , {wt }, and {vt } are assumed to be uncorrelated. The measurement noise vt is zero-mean Gaussian with covariance r > 0 except in case of outliers that occur

in an unpredictable way and have covariance that is unknown and much larger than r. Under the foregoing assumptions, a possible approach to the estimate of the state variables consists in applying a maximum likelihood criterion at each t by maximizing the conditional probability density w.r.t. x0 , . . . , xt with given measures y0 , . . . , yt , while rejecting too large noises that may be ascribed to the occurrences of outliers. The decision on rejection can be taken on the basis of a check on the residual error as compared with a suitable threshold σ > 0. Thus, at each time t, we have to solve the following problem: ¯0 ) min (x0 − x

xt0 ,w0t ,v0t

⊤

P0−1 (x0

  for   0, t−1 X +  wi⊤ Q−1 wi , for  

−x ¯0 ) +

t X

vi2 /r

i=0

t=0 t = 1, 2, . . .

(2.2a)

i=0

s.t.

xi+1 = A xi + B ui + wi , i = 0, 1, . . . , t − 1

(2.2b)

yi = C xi + vi , i = 0, 1, . . . , t

(2.2c)

− σ ≤ vi , i = 0, 1, . . . , t

(2.2d)

vi ≤ σ , i = 0, 1, . . . , t .

(2.2e)

Unfortunately, the solution of the problem (2.2) entails the exploration of the combinations of all the active sets (2.2d) and (2.2e), and hence such a method

73

2.2. Recursive estimation in the presence of outliers

turns out to be too computationally demanding, as the computational burden increases exponentially with t. Because of this difficulty, one can resort to a suboptimal approach that is based on the idea of performing only a one-step ahead optimization at each time t. Specifically, according to [94] where this problem is addressed though formulated in a different way, one can solve the following: min (x0 − x ¯0 )

⊤

xt0 ,w0t ,v0t

P0−1 (x0

−x ¯0 ) +

  for   0, t−1 X +  wi⊤ Q−1 wi , for  

t X

vi2 /r

i=0

t=0 t = 1, 2, . . .

(2.3a)

i=0

s.t.

xi+1 = A xi + B ui + wi , i = 0, 1, . . . , t − 1

(2.3b)

yi = C xi + vi , i = 0, 1, . . . , t

(2.3c)

− σ ≤ vi , i = 0, 1, . . . , t

(2.3d)

vi ≤ σ , i = 0, 1, . . . , t .

(2.3e)

xi = x ˆi|i , i = 0, 1, . . . , t − 1

(2.3f)

wi = w ˆi|i , i = 0, 1, . . . , t − 2

(2.3g)

vi = vˆi|i , i = 0, 1, . . . , t − 1

(2.3h)

where x ˆi|i , w ˆi|i , and vˆi|i are the solutions obtained at time i. In line with [94][Theorem 3], it is straightforward to prove that such a solution can be computed recursively by using a Kalman filter with covariance update driven by a threshold check. In other words,   ˆt|t−1 + Pt|t−1 C ⊤ (r + CPt|t−1 C ⊤ )−1 (yt   x x ˆt|t = −C x ˆt|t−1 ) , if |yt − C x ˆt|t−1 | < σ    x ˆt|t−1 , otherwise

(2.4a)

for t = 0, 1, . . ., where

x ˆt+1|t = A x ˆt|t + B ut ,

t = 0, 1, . . .

Pt|t−1 = APt−1|t−1 A⊤ + Q , t = 1, 2, . . .  ⊤ ⊤ −1    Pt|t−1 − Pt|t−1 C (r + CPt|t−1 C ) Pt|t−1 , Pt|t = if |yt − C x ˆt|t−1 | < σ t = 0, 1, . . .    P , otherwise t|t−1

(2.4b) (2.4c)

(2.4d)

74 2. Moving-Horizon Estimation for Linear System affected by Outliers with P0|−1 = P0 and x ˆ0|−1 = x ¯0 . The Kalman filter is an efficient algorithm to perform the estimation but some remarks are in order. First, the choice of σ is critical in general. In principle, the Gaussian assumptions allow one to fix the threshold in a convenient way but, in real problems, such assumptions are not easy to be verified or do not hold. Second, the covariances of the both initial state and noises may be unknown. Third, a typical application of estimation involves the use of a linearized model of the nonlinear plant of interest. Under these conditions the Kalman filter may suffer from robustness issues. The aforesaid motivates the contribution of the next section, where an approach based on a moving horizon strategy is thereby proposed.

2.3

Moving-horizon estimation robust to outliers

Let us consider the linear discrete-time dynamic system (2.1) under some different assumptions, i.e., xt+1 = A xt + B ut + wt yt = C x t + v t

(2.5a) (2.5b)

where t = 0, 1, . . . is the time instant, and, likewise in (2.1), xt ∈ Rn is the state

vector, ut ∈ Rm is the control vector, wt ∈ Rn is the system noise vector, yt ∈ R is the measure, and vt ∈ R is the measurement noise. Such a noise is assumed to

be small except on rare occurrences.

Assumption 2.1. There exists rv > 0 such that |vt | ≤ rv , t = 0, 1, . . . with t 6= t¯i , where t¯i ∈ N>0 , i = 0, 1, . . ., is a strictly increasing sequence and vt¯i ≫ rv . In any case, the abnormal noises at the instants t¯i are assumed to be bounded, i.e., there exists M ≫ rv such that, for all i = 0, 1, . . ., vt¯i ≤ M .

Assumption 2.2. We assume that wt is “small” as compared with the dynamics, i.e., bounded and taking zero or around zero values. More precisely, there exists rw > 0 such that, for all t = 0, 1, . . ., kwt k ≤ rw . We shall address a state estimation problem by using information obtained only in the recent past according to a moving-horizon strategy. Such an approach consists in deriving a state estimate at the current time t by using the information given by yt−N , . . . , yt , ut−N , . . . , ut−1 . More specifically, we aim at estimating

75

2.3. Moving-horizon estimation robust to outliers

xt−N , . . . , xt on the basis of such information and of a “prediction” x ¯t−N of the state xt−N at the beginning of the moving window. We denote the estimates of xt−N , . . . , xt at time t by x ˆt−N |t , . . . , x ˆt|t , respectively. As compared with the previous literature on MHE (see, e.g., [101, 102, 104– 106]), here we consider explicitly the occurrence of outliers in the measures. In such a setting, a natural criterion to derive the estimator consists in resorting to a least-squares approach by explicitly trying to reduce the effect of the outliers. Though in principle we can deal with an arbitrary number of outliers, the present study is restricted to the case of at most only one measurement affected by outlier in the batch of measures included in sliding window, thus assuming what follows. Assumption 2.3. System (2.5) is such that, inf i≥0 (t¯i+1 − t¯i ) > N + 1.

for all

i

=

0, 1, . . .,

If an outlier corrupts the k-th measure of the batch 1, 2, ..., N + 1, a leastsquares cost function that leaves out such a measure is

Jtk (xt−N ) = µ k xt−N − x ¯t−N k2 +

1 N

t X

i=t−N i6=t−N +k−1

(yi − Cxi )2

(2.6)

where k = 1, 2, . . . , N + 1 and µ ≥ 0. Of course, the cost (2.6) is to be

minimized together with the constraints1

xi+1 = Axi + Bui , i = t − N, . . . , t − 1 .

(2.7)

We account for the case in which no outlier affects the measurements by using

Jt0

(xt−N ) = µ k xt−N

t X 1 −x ¯t−N k + (yi − Cxi )2 N +1 2

(2.8)

i=t−N

to be minimized under the constraints (2.7). In practice, at each time t = 1

If additional constraints such as the state belong to a set X ⊂ Rn hold, we need to account

for both (2.7) and xi ∈ X, i = t − N, . . . , t in the optimization of the cost function. The formulation of the estimation problem in this more general setting does not entail conceptual difficulty, but it is omitted to avoid notational overhead.

76 2. Moving-Horizon Estimation for Linear System affected by Outliers N, N + 1, . . . we have to solve N + 2 problems as follows: min Jtk (x) , k = 0, 1, . . . , N + 1

x∈Rn s.t. (2.7) holds

and compare the optimal costs (2.6) and (2.8). The best of such costs is associated with the estimate. Thus, the estimate of xt−N at time t is k(t)∗

x ˆt−N |t = x ˆt−N

(2.9a)

where k(t)∗ ∈

argmin k=0,1,...,N +1

  Jtk x ˆkt−N

x ˆkt−N ∈ argmin Jtk (x) . x∈Rn s.t. (2.7) holds

(2.9b) (2.9c)

Note that such an estimate is not unique in general. To complete the estimation, we can determine the remaining estimates at time t by using (2.7) as follows: x ˆt−N +i+1|t = Aˆ xt−N +i|t + But−N +i , i = 0, . . . , N − 1. In solving such problems, we need to assign x ¯t−N , for which various choices can be made. For example, we can choose the result of the corresponding estimate at previous step, i.e., x ¯t−N = x ˆt−N |t−1 . Another possibility consists in propagating the value of x ˆt−N −1|t−1 as follows: x ¯t−N = A x ˆt−N −1|t−1 + But−N −1 . To simplify the stability analysis, we will adopt this last choice. Of course, we need to select an a-priori prediction of x0 for x ¯0 . Remark 2.1. As compared with the cost functions considered in [101,102,105,106], in (2.6) and (2.8) we explicitly normalize to contribution of the least-squares interpolation by dividing for the number of addends in such a way as to fairly compare the costs in (2.9b). Remark 2.2. Concerning the extension of the proposed approach to the case with multiple outliers in the batch, it is easy, though tedious, to replace Assumption 3.4 with a more general setting. If k outliers affect the N + 1 measures, we consider all permutations of k measures taken from the set of the N + 1 ones in the batch. Thus, for example, we need to consider a number of   2 n2 = N +1

77

2.4. Stability of the moving-horizon estimator

costs to discriminate among the various cases with two outliers. Of course, we need to consider the “no outlier” setting and all the case that corresponds to “one outlier” in the batch; thus, the number of cost functions is on the overall given by       2 2 1 . =N +2+ + n0,1,2 = 1 + N +1 N +1 N +1 In general, we have to account for up to k outliers in the batch by using  k  X i n0,1,...,k = N +1 i=0

costs to perform the comparisons according to (2.9a).

2.4

Stability of the moving-horizon estimator

If we consider the collections of measures at time steps t − N, t − N + 1, . . . , t,

it is straightforward to obtain

t t = Fk xt−N + Hk wt−1 + vt−N , yt−N t−N k k k k = 0, 1, . . . , N + 1

with Fk and Hk for k 6= 0 obtained from F0 and H0 by deleting the k-th block

row, respectively, where 

C   CA  F0 :=  ..  .  C AN

      



    H0 :=     

0

...

C

0

CA .. .

C .. .

 0    ... 0  . ..  .. . .   ... C ...

CAN −1 CAN −2

0



0

Based on the aforesaid, we can state the following theorem on the stability of the estimation error et−N := xt−N − x ˆt−N |t . Theorem 2.1. Suppose that Assumption 3.4 holds, Fk is of full rank, k =   0, 1, . . . , N + 1, and let δ := min λ Fk⊤ Fk > 0. Then the sequence {ζt } k=0,1,...,N +1

given by

2



(2.10a)

ζt+1 = a(µ) ζt + b(µ) , t = 0, 1, . . .

(2.10b)

ζ0 =

µ+

δ N +1



µ kx0 − x ¯ 0 k2 + c

78 2. Moving-Horizon Estimation for Linear System affected by Outliers is such that ket−N k2 ≤ ζt for t = N, N + 1, . . ., where

a(µ) =

8µ kAk2 µ + N δ+1

2(2N + 1) c= N (N + 1)

b(µ) = 

N 2 rw2

8µrw2 + 2c µ + N δ+1 max

k=0,1,...,N +1

2

kHk k + (N rv + M )

2



.

If µ is chosen such that a(µ) < 1, the sequence {ζt } converges to b(µ)/(1 −

a(µ)), and is strictly decreasing if ζ0 > b(µ)/(1 − a(µ)).

Proof. After focusing on a first preliminary bound, we will derive two (lower and upper) bounds on the optimal cost function, which will be combined to prove that the estimation error of the minimizer satisfies (2.10). First, using



t−1 t−1 w ≤ kH k

Hk wt−N

k t−N k ≤ kHk k N rw k from the bound

t

vt−N ≤ N rv + M k

t

t

t−1 t−1 t

yt−N − Fk xt−N = H w H w ≤ + v

k t−N k

k t−N k + vt−N k t−N k k

we obtain



t

2

yt−N − Fk xt−N 2 ≤ 2 N 2 rw k

max

k=0,1,...,N +1

 kHk k2 + (N rv + M )2 .

(2.11)

Second, the optimal cost at time t is if k(t)∗ = 0

t

2 1

yt−N − F0 x Jt∗ (ˆ xt−N ) = µ k x ˆt−N − x ¯t−N k2 + ˆt−N N +1 if k(t)∗ ∈ {1, . . . , N + 1}

2

t 1

yt−N − F0 x xt−N ) = µ k x Jt∗ (ˆ ˆt−N − x ¯t−N k2 + ˆt−N N +1

and hence Jt∗ (ˆ xt−N ) ≥ µ k x ˆt−N − x ¯t−N k2 +

2

t 1

yt−N ∗ − Fk∗ x ˆt−N k N +1

(2.12)

79

2.4. Stability of the moving-horizon estimator k(t)∗

where, from now on, Jt∗ and k ∗ stand for Jt

and k(t)∗ , respectively. Since



t t − Fk x + yt−N ˆt−N kFk xt−N − Fk x ˆt−N k = Fk xt−N − yt−N k k

t

t + yt−N − Fk x ≤ Fk xt−N − yt−N ˆt−N , k k

we have

2

t

2 t + 2 yt−N − Fk x

kFk xt−N − Fk x ˆt−N k2 ≤ 2 Fk xt−N − yt−N ˆ t−N k k

and hence

t

2 1

t 2

yt−N − Fk xt−N 2 ,

≥ kFk xt−N − Fk x

yt−N − Fk x ˆ k − ˆ t−N t−N k k 2 k = 0, 1, . . . , N + 1 Using (2.11) and since kFk xt−N − Fk x ˆt−N k2 ≥ δ kxt−N − x ˆt−N k2

(2.13)

from (2.13) for k = k ∗ we obtain

where



2 1 δ t

yt−N ∗ − Fk ∗ x ˆt−N ≥ kxt−N − x ˆt−N k2 − c1 k N +1 2(N + 1) 2 c1 :=



2 N 2 rw

max

2

k=0,1,...,N +1

kHk k + (N rv + M )

N +1

2



and, using this inequality in (2.12), Jt∗ (ˆ xt−N ) ≥ µ k x ˆt−N − x ¯t−N k2 +

δ kxt−N − x ˆt−N k2 − c1 . 2(N + 1)

(2.14)

Since kxt−N − x ˆt−N k ≤ kxt−N − x ¯t−N k + k¯ xt−N − x ˆt−N k

(2.15)

and hence kxt−N − x ˆt−N k2 ≤ 2kxt−N − x ¯t−N k2 + 2k¯ xt−N − x ˆt−N k2 , we have 1 ˆt−N k2 − kxt−N − x ¯t−N k2 kˆ xt−N − x ¯t−N k2 ≥ kxt−N − x 2

(2.16)

and, using this inequality in (2.14), we finally obtain   µ δ ∗ Jt (ˆ xt−N ) ≥ + kxt−N − x ˆt−N k2 − µkxt−N − x ¯t−N k2 − c1 . 2 2(N + 1) (2.17)

80 2. Moving-Horizon Estimation for Linear System affected by Outliers Third, let us consider the following inequalities, which hold for the various definitions we have introduced so far: Jt∗ (ˆ xt−N ) ≤ Jtk (ˆ xkt−N ) ≤ Jtk (xt−N ) = µ kxt−N − x ¯t−N k2 

2

if k = 0  1 y t t−N − F0 xt−N N +1

2

+

t  1 N yt−N k − Fk xt−N if k ∈ {1, . . . , N + 1}

1 t − Fk xt−N 2

yt−N ≤ µ kxt−N − x ¯t−N k2 + k N

for k = 0, 1, . . . , N + 1. Thus, the previous inequality provides Jt∗ (ˆ xt−N ) ≤ µ kxt−N − x ¯t−N k2 + c2 where 2 c2 :=



2 N 2 rw

max

k=0,1,...,N +1

2

kHk k + (N rv + M )

(2.18)

2

N



.

Using (2.17) and (2.18), it follows   µ δ + ket−N k2 ≤ 2 µ kxt−N − x ¯t−N k2 + c 2 2(N + 1)

(2.19)

where c := c1 + c2 . Since xt−N = Axt−N −1 + But−N −1 + wt−N −1 and x ¯t−N = Aˆ xt−N −1 + But−N −1 , we obtain kxt−N − x ¯t−N k2 ≤ kA(xt−N −1 − x ˆt−N −1 ) + wt−N −1 k2

2 ≤ 2 kAk2 kxt−N −1 − x ˆt−N −1 k2 + 2 rw .

(2.20)

Summing up, if we combine (2.19) and (2.20), it follows   δ µ 2 + ket−N k2 ≤ 4 µ kAk2 ket−N −1 k2 + 4µrw +c 2 2(N + 1) and hence ket−N k2 ≤

2 + 2c 8 µ kAk2 8µrw 2 ke k + t−N −1 µ + N δ+1 µ + N δ+1

for all t = N, N + 1, . . .. After fixing the initial condition for t = N , from the above inequality it is straightforward to set the sequence {ζt } according to (2.10), thus concluding the proof.

The condition a(µ) < 1 can be always satisfied via a convenient choice of a sufficiently small value of µ.

81

2.5. Simulation example

2.5

Simulation example

We consider the problem of estimating the state variables of a second-order oscillating system by using only the measures of the first variable to compare the Kalman filter based on the residual check according to (2.4) (KF), and the proposed method (2.9) (denoted by MHF). Such a system with damping ratio ξ, (undamped) natural pulsation ω, and no input is described by a discrete-time equation where A=

1

T

−T ω 2

−2ωξT + 1

!

C = (1

0)

and T > 0 is the sample time. We choose ξ = 0.2, ω = 5 rad/s, and T = 0.001 s. The distributions of initial state, system and measurement noises are taken according to [107][Example 4.3, p. 156], i.e., they are zero-mean white Gaussian processes with covariances P0|−1 = diag(2, 2), Q = diag(0, 4.47), and r = 0.01 except in case of outlier occurrence, for which the covariance was chosen much larger than r, i.e., with dispersion equal to 10. The KF was designed by using such values of P0|−1 , Q, and r and by using different residual threshold by scaling the covariance of the √ residuals, namely, St = r + CPt|t−1 C ⊤ (for example, choosing σt equal to St or √ √ √ 5 St or 10 St or 100 St ). The estimates of the MHF were obtained by solving (2.9) with N = 3 and different choices of µ. In practice, we set µ to tune the MHF, just as we did with σt for the KF. We will evaluate the performances of such filters by the root mean square error (RMSE): RM SE(t) =

M X ket,i k2 i=1

M

!1/2

where et,i is the estimation error at time t in the i-th simulation run, and M is the number of simulation runs. We simulated multiple synthetic outliers randomly positioned over 100 time steps. The results of such tests with a MHF having µ = 0.6 and a KF with different threshold values σt are shown in Fig. 2.4. The RMSE’s boxplot of MHF and KF with different choices of µ and σt are shown in Fig. 2.9. Tables 2.2 and 2.3 show the RMSE statistic of the MHF and KF for different choices of µ and σt , respectively. The computational time In Table2.1

82 2. Moving-Horizon Estimation for Linear System affected by Outliers

Figure 2.4: x1 and its estimates using MHF with µ = 0.5 and KF with σt =

√

St

Figure 2.5: Subjected measurements with multiple random outliers

√ Figure 2.6: x1 and its estimates using MHF with µ = 0.5 and KF with σt = 5 St

83

2.5. Simulation example

√ Figure 2.7: x1 and its estimates using MHF with µ = 0.5 and KF with σt = 10 St

√ Figure 2.8: x1 and its estimates using MHF with µ = 0.5 and KF with σt = 100 St

Table 2.1: Means of Computational time in seconds over 100 runs

r 0.01

µ-MHE 0.1

0.5

1

10

5.6883

5.6833

6.4414

6.6624

√

St

0.0053

σt -KF √ 5 St

√ 10 St

0.0083

0.0056

100

√

St

0.0069

84 2. Moving-Horizon Estimation for Linear System affected by Outliers

Figure 2.9: RMSEs of MHF and KF over 100 simulation runs.

Table 2.2: Statistic of RMSEs for x1 over 100 runs with r = 0.01

.

µ-MHE

√

St

σt -KF √ 5 St

√ 10 St

√ 100 St

0.1

0.5

1

10

min

0.004

0.000

0.000

0.005

1.218

1.154

2.460

0.583

median

0.065

0.064

0.062

0.089

1.295

1.442

3.380

3.370

max

0.079

0.081

0.071

0.136

1.598

1.504

3.512

5.853

Table 2.3: Statistic of RMSEs for x2 over 100 runs with r = 0.01

.

µ-MHE

√

St

σt -KF √ 5 St

√ 10 St

√ 100 St

0.1

0.5

1

10

min

0.005

0.000

0.000

0.000

1.385

1.471

1.572

1.464

median

13.13

14.89

17.62

13.79

12.74

11.34

10.2233

104.1

max

19.08

19.94

17.62

17.93

26.74

28.28

65.54

144.4

RMSE’s tables show the robustness of the proposed MHF, one can realize the range of estimation errors for each proposed estimators by picking-up the medians of the rmses. Though a convenient choice of µ makes the MHF performs much better in term of RMSE, its computational effort is larger.

2.6. Discussion on the simulation results

2.6

85

Discussion on the simulation results

In this section we show different simulations run with different choices of σt and µ (see fig. 2.10–2.13). We choose the same parameters as in 2.5, system and measurement noises are taken according to [107][Example 4.3, p. 156]. We simulate the MHF with µ equal to 0.1, 0.5, 1.0, 10 and the KF with no threshold √ √ √ value, and σt equal to 5 St or 10 St or 100 St ) respectively. The outliers are randomly generated and positioned over 100 time steps at each simulation run. The simulation results show the robustness of the proposed MHF with different µ. Some choices of σt for KF behave well according to outliers such as Fig. 2.12 and Fig. 2.13 while some of them are not (hand-tuned problem).   C    CA    F0 :=   ..   .   C AN t t , k = 0, 1, . . . , N + 1 = Fk xt−N + Hk wt−1 + vt−N yt−N t−N k k k

Implementation

86 2. Moving-Horizon Estimation for Linear System affected by Outliers

Figure 2.10

Figure 2.11

Table 2.4: Means of RMSE and computational time in seconds over 100 runs

Filter

Figure

Parameter

x1 RMSE

x2 RMSE

CPU

MHF

2.10

µ = 0.1

0.0447

6.8275

0.4269

2.11

µ = 0.5

0.0420

6.8693

0.4703

2.10

No threshold √ σt = 5 St

0.7076

51.598

0.0247

0.6507

4.2748

0.0256

KF

2.11

87

2.6. Discussion on the simulation results

Figure 2.12

Figure 2.13

Table 2.5: Means of RMSE and computational time in seconds over 100 runs

Filter

Figure

Parameter

x1 RMSE

x2 RMSE

CPU

MHF

2.12

µ=1

0.0476

5.3863

0.4045

2.13

µ = 10 √ σt = 10 St √ σt = 100 St

0.0424

6.8693

0.4155

1.2258

4.3907

0.0237

1.3730

33.422

0.0225

KF

2.12 2.13

88 2. Moving-Horizon Estimation for Linear System affected by Outliers

2.7

Comments and applications note

The MHE method was proposed and successfully applied to various types of dynamical systems, in unconstrained linear systems [115], constrained linear systems [97], [106], hybrid systems [118], [98], hybrid systems with unknown inputs [119], uncertain linear system [104], [105], singular system [114], nonlinear systems [112], [101], and nonlinear systems with neural network [102]. In this chapter we have proposed a new aspect of research of MHE. Investing the success of MHE in the state estimation and process monitoring for linear and nonlinear systems, efficient solution of MHE approach is important for solving the problem of outlier in industrial system. Hedengren in [128] has introduced an example application in oil and gas industry of considering the problem of determining the flow of mud through the return annulus of drilling pipe. The results with data outliers clearly indicate that all state estimate except

Figure 2.14: Outlier effect on different estimators, see [128]

ℓ1 -norm MHE are significantly affected by the bad data points. See Fig. 2.14. Our method as shown in this chapter has solved the sensitivity of ℓ2 - norm MHE to outliers. The need of such method is due to the diversity of the fields that deal with outliers. Some of these fields are process control [121], clinical alerting and heart surgery [124], intrusion detection [123], environmental monitoring [124], positioning [125], cloud management [126], and fault detection [127].

2.8. Conclusion

2.8

89

Conclusion

We have addressed the problem of state estimation for linear systems with measurements affected by outliers by using moving horizon estimation. We have proved the stability of the proposed moving horizon estimator and compare its performance with that of the Kalman filter having an estimate update driven by a threshold test on the current output error. Such a Kalman filter turns out to be quite sensitive to the choice of the threshold, which is difficult to be set. Indeed, the proposed moving-horizon filter is much easier to be tuned via the selection of µ and more robust to outliers. However, it is more demanding from the computational point of view as compared with the Kalman filter.

C

References

[1] O. Maimon, and L. Rokach ”Data Mining and Knowledge Discovery Handbook” O. Maimon and L. Rokach Editors, 3rd edition, Springer. Springer New York Dordrecht Heidelberg London, 2010. [2] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. ”From Data Mining to Knowledge Discovery in Databases” AI Magazine American Association for Artificial Intelligence, pp.37–54, Fall 1996. [3] H. W. Ian , F. Eibe, A. M. Hall. ”Data Mining: Practical Machine Learning Tools and Techniques (3rd edition).” A Elsevier, 30 January 2011. [4] S. Sayad. ”Real Time Data mining”. Self-Help Publishers books, Canada, 2011. [5] . Marbn, G. Mariscal, and J. Segovia. ”A Data Mining Knowledge Discovery Process Model.” In Data Mining and Knowledge Discovery in Real Life Applications, Book edited by: J. Ponce and A. Karahoca, pp. 438–453, February 2009, I-Tech, Vienna, Austria. [6] M. J. Druzdzel and R. R. Flynn. ”Decision Support Systems.” Encyclopedia of Library and Information Science, Allen Kent (ed.), New York: Marcel Dekker, Inc., 2002. [7] Acrotec http://www.acrotec.it/?lang=en. ”Projects: Integrated Network for Emergencies”, 2012.

146

References

[8] J. Spate, K. Gibert, M. Sanchez, E. Frank, J. Comas, and I. Athanasiadis. ”Data Mining as a Tool for Environmental Scientist. ” Al Magazine, vol 17, 1996. [9] P. Keen. ”Decision support systems : a research perspective” Cambridge, Mass. : Center for Information Systems Research, Alfred P. Sloan School of Management, 1980. [10] R. Sprague ”A Framework for the Development of Decision Support Systems.” MIS Quarterly, 4(4),pp.1–25, 1980. [11] A. P. Sage. ”Decision Support Systems Engineering”.

John Wiley Sons,

Inc., New York, 1991. [12] P. E. Lehner, T. M. Mullin, and M. S. Cohen.”A probability analysis of the usefulness of decision aids.” Uncertainty in Artificial Intelligence, Elsevier, pp.427–436, 1990. [13] J.G. Borges, E.M. Nordstrm, G. Gonzalo, J. Hujala, and T. Trasobares. ”Computer-based tools for supporting forest management. The experience and the expertise world-wide” Community of Practice Forest Management Decision Support Systems, Ume, Sweden, 2014. [14] K.A. Delic, L. Douillet, and U. Dayal. ”Towards an architecture for realtime decision support systems:challenges and solutions” IEEE International Symposium on Database Engineering and Applications, pp.303–311, 2001. [15] G. Piatetsky-Shapiro ”Knowledge Discovery in Real Databases: A Report on the IJCAI-89 Workshop.” AI Magazine, 11(5), pp. 68–70, 1991. [16] J. Shrager, and P. Langley ”Computational Models of Scientific Discovery and Theory Formation.” San Francisco, Calif.: Morgan Kaufmann., 1990. [17] W. Kloesgen, and J. Zytkow ”Knowledge Discovery in Databases Terminology. In Advances in Knowledge Discovery and Data Mining” AAAI Press, Menlo Park, Calif , pp. 569–588, 1996. [18] P. Spirtes, C. Glymour, and R. Scheines Search” Springer-Verlag., New York, 1993.

”Causation, Prediction, and

147

References

[19] J. Elder, and D. Pregibon ” A Statistical Perspective on KDD. In Advances in Knowledge Discovery and Data Mining” AAAI Press, Menlo Park, Calif eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, pp. 83–116, 1996. [20] C. Glymour, D. Madigan, D. Pregibon, and P. Smyth ” Statistics and Data Mining” Communications of the ACM (Special Issue on Data Mining), November 1996. [21] U. M. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. ” From Data Mining to Knowledge Discovery: An Overview. In Advances in Knowledge Discovery and Data Mining”, eds. U. Fayyad, G. Piatetsky- Shapiro, P. Smyth, and R. Uthurusamy, 130. Menlo Park, Calif.: AAAI Press., 1996. [22] U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy ” Statistics and Data Mining” AAAI Press.,Menlo Park, Calif, 1996. [23] J. Han, and M. Kamber. ”Data mining: concepts and techniques”. Morgan Kaufmann, 2006. [24] D.T. Larose. ”Discovering knowledge in data: an introduction to data mining”.

John Wiley and Sons, 2005.

[25] S. I. Weiss, and C. Kulikowski. ”Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Networks, Machine Learning, and Expert Systems”. San Francisco, Calif.: Morgan Kaufmann, 1991. [26] D.J. Hand. ”Discrimination and Classification”. Chichester, U.K.: Wiley, 1981. [27] A. K. Jain, , and R. C. Dubes. ”Algorithms for Clustering Data”.

Engle-

wood Cliffs, N.J.: Prentice Hall, 1988. [28] D. M. Titterington, A. F. M. Smith, and U. E. Makov. ”Statistical Analysis of Finite-Mixture Distributions”.

Chichester, U.K.: Wiley, 1985.

[29] P. Cheeseman, and J. Stutz. ” Bayesian Classification: Theory and Results”,In Advances in Knowledge Discovery and Data Mining, eds. U.

148

References

Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 7395. Menlo Park, Calif.: AAAI Press., 1996. [30] B. Silverman ”Density Estimation for Statistics and Data Analysis”. Chapman and Hall, New York, 1986. [31] R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and I. Verkamo. ” Fast Discovery of Association Rules”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 307328. Menlo Park, Calif.: AAAI Press., 1996. [32] R. Zembowicz, and J. Zytkow. ” From Contingency Tables to Various Forms of Knowledge in Databases”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 329351. Menlo Park, Calif.: AAAI Press., 1996. [33] C. Glymour, R. Scheines, P. Spirtes, and K. Kelly.”Discovering Causal Structure”.

Academic, New York, 1987.

[34] D. Heckerman ”Bayesian Networks for Knowledge Discovery”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. PiatetskyShapiro, P. Smyth, and R. Uthurusamy, 273306. Menlo Park, Calif.: AAAI Press., 1996. [35] D. Berndt, and J. Clifford. ”Finding Patterns in Time Series: A Dynamic Programming Approach.”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 229248. Menlo Park, Calif.: AAAI Press., 1996. [36] O. Guyon, N. Matic, and N. Vapnik. ”Discovering Informative Patterns and Data Cleaning”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 181204. Menlo Park, Calif.: AAAI Press., 1996. [37] W. Kloesgen ”A Multipattern and Multistrategy Discovery Assistant”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 249271. Menlo Park, Calif.: AAAI Press., 1996.

149

References

[38] C. Matheus, G. Piatetsky-Shapiro, and D. McNeill. ”Selecting and Reporting What Is Interesting: The KEfiR Application to Healthcare Data”,In Advances in Knowledge Discovery and Data Mining, eds. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, 495516. Menlo Park, Calif.: AAAI Press., 1996. [39] M. Basseville, and I. V. Nikiforov. ”Detection of Abrupt Changes: Theory and Application”.

Englewood Cliffs, N.J.: Prentice Hall, 1993.

[40] V. Mallet and D.E. Keyes and F.E. Fendell ”Modeling wildland fire propagation with level set methods” Computers Mathematics with Applications, Elsevier, 57(7), pp. 1089–1101, 2009. [41] F.E. Fendell, M.F. Wolff ”Forest Fires Behavior and Ecological Effects” Academic Press, Chapter Wind-aided fire spread, pp. 171–223, 2001. [42] J. Spate, K. Gibertb, M. Sanchez-Marr, E. Frank, J. Comase, I. Athanasiadis, R. Letcher. ”Data Mining as a Tool for Environmental Scientists” Al Magazine, vol. 17, 1996. [43] I. N. Athanasiadis, and P. A. Mitkas ”An agent-based intelligent environmental monitoring system” International Journal of Management of Environmental Quality, 15(3), pp. 238–249, 2004. [44] M. Poch, J. Comas, I. Rodriguez-Roda, M. Sanchez-Marre, U. Cortes. ”Designing and building real environmental decision support systems” Environmental Modelling Software, Elsevier, 2003. [45] I. Ben-Gal ”Outlier Detection” In: Maimon O. and Rockach L. (Eds.) Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, Kluwer Academic Publishers, 2005. [46] G. J. Williams ,R. A. Baxter ,H. X. He ,S. Hawkins, and L. Gu ”A Comparative Study of RNN for Outlier Detection in Data Mining” IEEE International Conference on Data-mining (ICDM’02), Maebashi City, Japan, CSIRO Technical Report CMIS-02/102, 2002.

150

References

[47] H. Liu, S. Shah, and W. Jiang. ”On-line Outlier Detection and Data Cleaning” Journal of Computers and Chemical Engineering, vol. 28, pp. 1635– 7647, 2004. [48] D. Hawkins ”Identification of Outliers”. [49] C. C. Aggarwal ”Outlier Analysis”.

Chapman and Hall, 1980.

Kluwer Academic Publishers.

[50] N. Devarakonda, S. Pamidi, V. V. Kumari, A. Govardhan ”Outliers Detection as Network Intrusion Detection System Using Multi Layered Framework” Advances in Computer Science and Information Technology Communications in Computer and Information Science. Springer Vol. 131, pp. 101–111, 2011. [51] K. Prakobphol, and J. Zhan ”A Novel Outlier Detection Scheme for Network Intrusion Detection Systems ” IEEE International Conference on Information Security and Assurance Vol. 131, pp. 555–560, 2008. [52] A. Juvonen, and T. Hamalainen. ”An Efficient Network Log Anomaly Detection System Using Random Projection Dimensionality Reduction” IEEE 6th International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–5, 2014. [53] W.-F. Yu, and N. Wang

”Research on Credit Card Fraud Detection

Model Based on Distance Sum” IEEE Conference on Artificial Intelligence JCAI’09, pp. 353–356, 2009. [54] S.B.E. Raj, and A.A. Portia. ”Analysis on Credit Card Fraud Detection Methods” IEEE International Conference on Computer, Communication and Electrical Technology (ICCCET), pp. 152–156, 2011. [55] M. Haiying, and L. Xin. ”Application of Data Mining in Preventing Credit Card Fraud” IEEE International Conference on Management and Service Science MASS’09, pp. 1–6, 2009. [56] O. Ghorbel, M.W. Jmal, W. Ayedi, H. Snoussi, and M. Abid. ”An overview of outlier detection technique developed for wireless sensor networks” IEEE 10th International Multi-Conference on Systems, Signals Devices (SSD), pp. 1–6, 2013.

151

References

[57] C. Franke, M. Gertz. ”Detection and Exploration of Outlier Regions in Sensor Data Streams” IEEE International Conference on Data Mining Workshops ICDMW’08 , pp. 375–384 , 2008. [58] S. Cateni, V. Colla, and M. Vannucci ”Outlier Detection Methods for Industrial Applications” Advances in Robotics, Automation and Control, Book edited by: Jess Armburo and Antonio Ramrez Trevio, I-Tech, Vienna, Austria, 2008. [59] S. Ahmad, N.M. Ramli, H. Midi. ”Outlier detection in logistic regression and its application in medical data analysis” IEEE Colloquium on Humanities, Science and Engineering (CHUSER), pp.503-507, 2012. [60] Y. Mao, Y. Chen, G. Hackmann, M. Chen, L. Chenyang, M. Kollef, and T.C. Bailey. ”Medical Data Mining for Early Deterioration Warning in General Hospital Wards” IEEE 11th International Conference on Data Mining Workshops (ICDMW), pp.1042-1049, 2011. [61] V. J. Hodge, and J. Austin ”A Survey of Outlier Detection Methodologies.”. Kluwer Academic Publishers, 2004. [62] V. Chandola, A. Banerjee, and V. Kumar ”Outlier Detection : A Survey”. University of Minnesota. [63] S. Papadimitriou, H. Kitawaga, P.G. Gibbons, C. Faloutsos. ”LOCI: Fast Outlier Detection Using the Local Correlation Integral” Intel Research Laboratory. Technical report no. IRP-TR-02-09, 2002. [64] F. Grubbs ”Procedures for detecting outlying observations in samples”. ”Technometrics”, 11(1), pp. 1–21, 1969. [65] R. O. Duda, P. E. Hart, and D. G. Stork. ”Pattern Classification (2nd Edition)”.

Wiley, 2000.

[66] L. R. Rabiner, and B. H. Juang ”An introduction to Hidden Markov Models”. ”IEEE ASSP Magazine”, 3(1), pp. 4–16, 1986. [67] V. Barnett, T. Lewis ”Outliers in Statistical Data”.

Wiley, 1994.

152

References

[68] L. Davies , and U. Gather ”The identification of multiple outliers”. ”Journal of the American Statistical Association”, 88(423), pp. 782–792, 1993. [69] F. R. Hampel ”A general qualitative definition of robustness”. ”Annals of Mathematics Statistics”, vol. 42, pp. 1887–1896, 1971. [70] F. R. Hampel ”The influence curve and its role in robust estimation”. ”Journal of the American Statistical Association”, vol. 69, pp. 382–393, 1974. [71] J. W. Tukey ”Exploratory Data Analysis”.

Addison-Wesley, 1977.

[72] I. Ben-Gal, G. Morag, and A. Shmilovici ”CSPC: A Monitoring Procedure for State Dependent Processes”. ”Technometrics”, 45(4), pp. 293–311, 2003. [73] P. C. Mahalanobis ”On the generalised distance in statistics”. ”Proceedings of the National Institute of Sciences of India”, 2(1), pp. 49–55, 1936. [74] C. J. Dunn ”A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters”. ”J. Cybernet”, Vol. 3, pp. 32–57, 1973. [75] J. C. Bezdek ”Pattern Recognition with Fuzzy Objective Function Algorithms”.

Kluwer Academic Publishers, Norwell, MA, USA, 1981.

[76] S. Chattopadhyay, D. K. Pratihar, and S. C. De Sarkar ”A Comparative Study of Fuzzy C-Means Algorithm and Entropy-based Fuzzy Clustering Algorithms”. ”Computing and Informatics”, Vol. 30, pp. 701–720, 2011. [77] Matlab and Statistics Toolbox Release 2012b The MathWorks Inc., Natick, Massachusetts, United States. Available online on http://www.mathwork. com Matlab and Statistics Toolbox Release 2012b, The MathWorks, Inc., Natick, Massachusetts, United States. [78] G. Miner, R. Nisbet, J. Elder ”Handbook of Statistical Analysis and Data Mining Applications”.

Academic Press, Elsevier, 2009.

[79] C. J. Schewart ”Analysis of Covariance - ANCOVA” http://www.stat. sfu.ca/~cschwarz/CourseNotes, Course 20-10-2014.

153

References

[80] G. Shan, and C. Ma ”A Comment on Sample Size Calculation for Analysis of Covariance in Parallel Arm Studies”. ” Biometrics and Biostatistic”, Vol 5, issue 1-1484, 2014. [81] S. W. Huck ”Reading Statistics and Research (4th ed.)”.

Boston, MA:

Allyn and Bacon, 2004. [82] E. Ostertagova ”Modelling using polynomial regression ”. ”Procedia Engineering, Elsevier”, Vol 48, pp. 500–506, 2012. [83] A. D. Aczel ”Complete Business Statistics”.

Irwin, ISBN 0-256-05710-8,

1989. [84] K. Suzuki ”Artificial Neural Networks: Architectures and Applications ”. InTech, 2013. [85] D. Michie, and D. J. Spiegelhalter ”Machine Learning, Neural and Statistical Classification”.

Ellis Horwood, 1994.

[86] P. Joseph, I. Bigus, and M. Rochester ”Data mining with neural networks: solving business problems from application development to decision support”. McGraw-Hill, Inc. Hightstown, NJ, USA, 1996. [87] S. Nirkhi ”Potential use of Artificial Neural Network in Data Mining ”. ”The 2nd International Conference on Computer and Automation Engineering (ICCAE)”, pp. 339–343, 2010. [88] Y. Bengio, J. M. Buhmann, M. Embrechts, and J. M. Zurada. ”Introduction to the special issue on neural networks for data mining and knowledge discovery”. ”IEEE Transaction on Neural Networks”, Vol 11, pp. 545–549, 2000. [89] A. Roy. ”Artificial neural networks - a science in trouble”. ”ACM SIGKDD Explorations”, Vol 1, pp. 33–38, 2000. [90] T. Kohonen ”Self-Organizing Maps”. Series in Information Sciences, second edn., Springer, Heidelberg, 1997.

154

References

[91] T. Kohonen, S. Kaski, K. Lagus, J. Salojarvi, J. Honkela, V. Paatero, and A. Saarela ”Self organization of a massive document collection”. ”IEEE Transaction on Neural Networks”, Vol 11, pp. 574–585, 2010. [92] M. H. Beale, M. T. Hagan, and H. B. Demuth ”Handbook of Neural Network Toolbox”.

The MathWorks, Inc., 2014. http://www.mathwork.com

[93] M. Gandhi and L. Mili, ”Robust Kalman filter based on a generalized maximum-likelihood-type estimator”, IEEE Trans. on Signal Processing, vol. 58, no. 5, pp. 2509–2520, 2010. [94] D. Shi, T. Chen, and L. Shi, “Event-triggered maximum likelihood state estimation,” Automatica, vol. 50, no. 1, pp. 247–254, 2014. [95] A. H. Jazwinski, “Limited memory optimal filtering,” IEEE Trans. on Automatic Control, vol. 13, no. 5, pp. 558–563, 1968. [96] H. Michalska and D. Q. Mayne, “Moving horizon observers and observer– based control,” IEEE Trans. on Automatic Control, vol. 6, no. 6, pp. 995– 1006, 1995. [97] C. V. Rao, J. B. Rawlings, and J. H. Lee, “Constrained linear state estimation–a moving horizon approach,” Automatica, vol. 37, no. 10, pp. 1619–1628, 2001. [98] G. Ferrari-Trecate, D. Mignone, and M. Morari, “Moving horizon estimation for hybrid systems,” IEEE Trans. on Automatic Control, vol. 47, no. 10, pp. 1663–1676, 2002. [99] A. Alessandri, M. Baglietto, and G. Battistelli, “Receding-horizon estimation for discrete-time linear systems,” IEEE Transactions on Automatic Control, vol. 48, no. 3, pp. 473–478, 2003. [100] C. V. Rao, J. B. Rawlings, and D. Q. Mayne, “Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximations,” IEEE Trans. on Automatic Control, vol. 48, no. 2, pp. 246–257, 2003.

155

References

[101] A. Alessandri, M. Baglietto, and G. Battistelli, “Moving-horizon state estimation for nonlinear discrete-time systems: New stability results and approximation schemes,” Automatica, vol. 44, no. 7, pp. 1753 – 1765, 2008. [102] A. Alessandri, M. Baglietto, G. Battistelli, and M. Gaggero, “Movinghorizon state estimation for nonlinear systems using neural networks,” IEEE Trans. Neural Networks, vol. 22, no. 5, pp. 768–780, 2011. [103] A. Matasov and V. Samokhvalov, “Guaranteeing parameter estimation with anomalous measurement errors,” Automatica, vol. 32, no. 9, pp. 1317–1322, 1996. [104] A. Alessandri, M. Baglietto, and G. Battistelli, “Robust receding-horizon state estimation for uncertain discrete-time linear systems,” Systems & Control Letters, vol. 54, no. 7, pp. 627–643, 2005. [105] A. Alessandri, M. Baglietto, and G. Battistelli, “Min-max moving-horizon estimation for uncertain discrete-time systems,” SIAM J. Control and Optimization, vol. 50, no. 3, pp. 1439–1465, 2012. [106] A. Alessandri, M. Baglietto, and G. Battistelli, ”Receding-horizon estimation for discrete-time linear systems,” IEEE Trans. on Automatic Control, vol. 48, no. 3, pp. 473–478, 2003. [107] M. Grewal and A. Andrews, ”Kalman Filtering: Theory and Practice Using Matlab”.

Wiley.

[108] A. Alessandri, M. Baglietto, T. Parisini, and R. Zoppoli. ”A neural state estimator with bounded errors for nonlinear systems.” IEEE Transactions on Automatic Control, 44(11):2028–2042, 1999. [109] A. Alessandri, M. Baglietto, and G. Battistelli. ”Receding-horizon estimation for switching discrete-time linear systems.” IEEE Transactions on Automatic Control, 50(11):1736–1748, 2005. [110] P. E. Moraal and J. W. Grizzle. ”Observer design for nonlinear systems with discrete-time measurements.” IEEE Transactions on Automatic Control, 40(3):395–404, 1995.

156

References

[111] M. Alamir and L. A. Cavillo-Corona. ”Further results on nonlinear recedinghorizon observers.” IEEE Transactions on Automatic Control, 47(7):1184– 1188, 2002. [112] C. V. Rao, J. B. Rawlings, and D. Q. Mayne. ”Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximations.” IEEE Transactions on Automatic Control, 48(2):246–257, 2003. [113] C. V. Rao, J. B. Rawlings. ”Constrained process monitoring: A moving horizon approach.” AIChE J., 48, pp. 97–109, 2002. [114] B. Boulkroune M. Darouach M. Zasadzinski. ”Moving horizon state estimation for linear discrete-time singular systems.” IET Control Theory and Applications, Vol. 4, no. 3, pp. 339–350, 2010. [115] K.R. Muske , J.B. Rawlings ”Receding horizon recursive state estimation”. Proc. IEEE American Control Conf.San Francisco, USA, pp. 900–904, 1993. [116] F. Yang, R.W. Wilde ”Observers for linear systems with unknown inputs.” IEEE Trans. Autom. Control, 33(7): pp. 677–681, 1988. [117] M. Darouach, M. Zasadzinski, J.Y. Keller ”State estimation for discrete systems with unknown inputs using state estimation of singular systems”. roc. IEEE American Control Conf., pp. 3014–3015, 1992. [118] A. Bemporad, D. Mmignone,M. Morari ”Moving horizon estimation for hybrid systems and fault detection”. Proc. IEEE American Control Conf., San Diego, Canada, pp. 2471 2475, 1999. [119] L. Pina, M.A. Botto ”Simultaneous state and input estimation of hybrid systems with unknown inputs”. Automatica., 42:(5), pp. 755–762, 2006. [120] J. D. Hedengren. ”Advanced Process Monitoring”. Chapter submitted to Optimization and Analytics in the Oil and Gas Industry, Volume II: The Downstream, Springer-Verlag., 2012. [121] R.K. Pearson. ”Outliers in process modeling and identification.” IEEE Trans. on Control Systems Technolgy, 10(1):55–63, 2002.

References

157

[122] T. Ortmaier, M. Groger, D.H. Boehm, V. Falk, and G. Hirzinger. ”Motion estimation in beating heart surgery.” IEEE Trans. on Biomedical Engineering, 52(10):1729–1740, 2005. [123] J. Zhang, M. Zulkernine, and A. Haque. ”Random-forests-based network intrusion detection systems.” IEEE Trans. on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 38(5):649–659, 2008. [124] T.Ortmaier, M.Groger, D.H. Boehm, V.Falk, and G.Hirzinger. ”Motion estimation in beating heart surgery.” IEEE Trans. on Biomedical Engineering, 52(10):1729–1740, 2005. [125] K. Fallahi, C.-T. Cheng, and M. Fattouche. ”Robust positioning systems in the presence of outliers under weak GPS signal conditions.” IEEE Systems Journal, 6(3):401–413, 2012. [126] S. Meng and L. Liu. ”Enhanced monitoring-as-a-service for effective cloud management”. IEEE Trans. on Computers, 62(9):1705–1720, 2013. [127] H. Ferdowsi, S. Jagannathan, and M. Zawodniok. ”An online outlier identification and removal scheme for improving fault detection performance”. IEEE Trans. on Neural Networks and Learning Systems, 25(5):908–919, 2014. [128] L. Fang, M. Zhi-zhong. ”An online outlier detection method for process control time series”. Control and Decision Conference (CCDC) Chinese, Mianyang, pp. 3263–3267, 2011. [129] RR.D. Cook, and S. Weisberg. ”Residuals and Influence in Regression”. Chapman and Hall, New York, 1982. [130] P.J. Rousseeuw, and B.C. Zomeren. ”Unmasking multivariate outliers and leverage points”. Journal of American Statistical Association, Vol. 85, pp. 623–651, 1990. [131] J. Osborne ”Notes on the use of data transformations”. Practical Assessment, Research Evaluation, Vol. 8(6), 2002.

158

References

[132] J. W. Osborne, and A. Overbay ”The power of outliers and why researchers should always check for them”. Practical Assessment, Research Evaluation, Vol. 9(6), 2004. [133] J. W. Osborne ”Improving your data transformations: Applying the BoxCox transformation”. Practical Assessment, Research

Evaluation, Vol.

15(12), 2010. [134] J. H. McDonald. ”Handbook of Biological Statistics (3rd ed.)”.

Sparky

House Publishing. Baltimore, Maryland, 2014 [135] B. G. Tabachnick, and L. S. Fidell. ”Using multivariate statistics (5th ed.)”. Allyn and Bacon. Boston, 2007 [136] D. C. Howell. ”Statistical methods for psychology (6th ed.)”.

AThomson

Wadsworth. Belmont, CA, 2007. [137] J. W. Tukey. ”The comparative anatomy of transformations”. Annals of Mathematical Statistics, Vol. 28, pp. 602–632, 1957. [138] G. E. P. Box, and D. R. Cox. ”An analysis of transformations”. Journal of the Royal Statistical Society, Vol. 26, pp. 211–234, 1964. [139] R. M. Sakia ”The Box-Cox transformation technique: A review”. The Statistician, Vol. 41, pp. 169–178, 1992. [140] O. Renaud, and M.-P. Victoria-Feser ”A robust coefficient of determination for regression”. Journal of Statistical Planning and Inference, Vol. 140(7), pp. 1852–1862, 2010. [141] W. Greene. ”Econometric Analysis (third ed.)”.

Prentice Hall, 1997.

[142] R. Brown, and P. Hwang. ”Introduction to Random Signals and Applied Kalman Filtering, Third edition”.

Wiley, 1997.

[143] B. Hofmann-Wellenohf, H. Lichtenegger, and J. Collins ”Global Positioning System, Theory and practice, 2nd edition”.

Springer-Verlag 1993.

[144] Institute of navigation ”Global Positioning System”. Vols I,II,III, and IV. The institute of Navigation.1980-86.

References

159

[145] J. C. Rambo ”Receiver Processing Software Design of the Rockwell International DoD Standard GPS Receivers” Proceedings of the 2nd International Technical Meeting of the Satellite Division of the Institute of Navigation, Colorado Springs, pp. 217-225, 1989. [146] R. M. Kalafus, J. Vi lcans , and N. Knable. ”Differential Operation of NAVSTAR GPS.” Navigation, 1. Inst. Navigation, 30(3), pp. 187-204, 1983. [147] E. G. Blackwell ”Overview of Differential GPS Methods,” Navigation, 1. Inst. Navigation, 32(2), pp. 114-125, 1985. [148] X.-W. Chang, Y. Guo ”Huber’s M-estimation in relative GPS positioning: computational aspects” McGill University, Canada. [149] C. Ordonez , J. Martinez, J. R. Hodrfquez-Perez, and A. Reyes. ”Detection of Outliers in GPS Measurements by Using Functional-Data Analysis” JOURNAL OF SURVEYING ENGINE, pp. 150–155, 2010. [150] D. Wright, C. Stephens, and V. P. Rasmussen ”On Target Geospatial Technologies” USU Geospatial Extension Program, 2010. [151] D. Cooksey ”Understanding the Global Positioning System (GPS)” Montana State University-Bozeman. [152] J-M. Zogg ”GPS Basics: Introduction to the system Application overview” u-bloxag, 2002. [153] Y. Bian ”GPS signal selective availability-modelling and simulation for FAA WAAS IVV” proc. Position Location and Navigation Symposium, pp. 515–522, IEEE 1996. [154] Y. Huihai, ”Modeling of selective availability for global positioning system” Journal of Systems Engineering and Electronics, 7(3): pp. 515–522, IEEE 1996. [155] A. Franchois ”Determination of GPS positioning errors due to multi-path in civil aviation” Proceedings of 2nd International Conference on Recent Advances in Space Technologies, pp. 400 - 406, IEEE 2005.

160

References

[156] L. Liu ”Comparison of Average Performance of GPS Discriminators in Multipath” IEEE International Conference on Acoustics, Speech and Signal Processing, pp. III-1285 - III-1288, IEEE 2007. [157] K. Breivik, B. Forsselli, C. Kee, P. Enge and T. Walter ”Estimation of Multipath Error in GPS Pseudorange Measurements” Journal of Navigation, pp. 43–52, 2014. [158] ”Quasi-Zenith Satellite System (QZSS) Service” http://www.qzs.jp/en/ Satellite Positioning Overview, Co. Japan. [159] T.-K. Yeh, C. Hwang, G. Xu, C.-S. Wang, and C.-C. Lee. ”Determination of global positioning system (GPS) receiver clock errors: impact on positioning accuracy” Measurement Science and Technology, 20(7), 2009. [160] F.-c. Chan ”Stochastic modeling of atomic receiver clock for high integrity gps navigation” Transactions on Aerospace and Electronic Systems, IEEE, 50(3), pp.1749–1764, 2014. [161] K. L. Nathan and J. Wang. ”A Comparison of Outlier Detection Procedures and Robust Estimation Methods in GPS Positioning” J. Geodesy. 66(4), pp. 699–709, 2009. [162] E. Gkalp ,O. Gngr, and Y. Boz. ”Evaluation of Different Outlier Detection Methods for GPS Networks ” Sensor, 8, pp. 7344–7358, 2008. [163] Y. Zhang, F. Wu, and H. Isshiki. ”A New Cascade Method for Detecting GPS Multiple Outliers Based on Total Residuals of Observation Equations ” Position Location and Navigation Symposium (PLANS),IEEE, pp. 208–215, 2012. [164] E. D. Kalpan. ”Understanding GPS, Principles and Applications” Artech House, 1996. [165] A. Giremus, E. Grivel, and F. Castani. ”Is H infinity filtering relevant for correlated noises in GPS navigation ?” IEEE proceedings DSP, 8, pp. 1–6, 2009. [166] P. Alxelrad and R.G Brown, ”GPS navigation algorithms” Global Positioning System: Theory and Applications, vol. 1, 1996.

161

References

[167] K. Fallahi, C.-T. Cheng, M. Fattouche ”Robust Positioning Systems in the Presence of Outliers Under Weak GPS Signal Conditions” IEEE Systems Journal, 6(3), pp.401–413, 2012. [168] G. Pulford ”Analysis of a nonlinear least squares procedure used in global positioning systems” IEEE Trans. Signal Process, 58(9), pp.4526–4534, 2010 [169] T.-H. Chang , L.-S. Wang, and F.-R. Chang.

”A solution to the ill-

conditioned GPS positioning problem in an urban environment” IEEE Trans. on Intelligent Transportation Systems, 10(1), pp.135–145, 2009. [170] M. F. Abdel-Hafez ”The autocovariance least-squares technique for GPS measurement noise estimation” IEEE Trans. Vehicle Technology, 59(2), pp.574–588, 2010. [171] Y. Wang ”Position Estimation using Extended Kalman Filter and RTSSmoother in a GPS Receiver” 5th International Congress on Image and Signal Processing (CISP), pp.1718–1721, 2012. [172] P. Misra, and P. Enge. ”Global Positioning System Signals, Measurements, and Performance(Second Edition)”.

Wiley, 2006.

[173] K. Borre and D. M. Akos. ”A software-Defined GPS and Galileo Receiver”. Springer. Birkhauser Basel, Aug. 2006. [174] R. Leonardo, and M. Santos ”Stochastic models for GPS position - An imperial approach”.

GPS world, February 2007