J Geod (2013) 87:935–944 DOI 10.1007/s00190-013-0658-8
ORIGINAL ARTICLE
Application of least squares variance component estimation to errors-in-variables models A. R. Amiri-Simkooei
Received: 28 April 2013 / Accepted: 31 August 2013 / Published online: 8 October 2013 © Springer-Verlag Berlin Heidelberg 2013
Abstract In an earlier work, a simple and flexible formulation for the weighted total least squares (WTLS) problem was presented. The formulation allows one to directly apply the existing body of knowledge of the least squares theory to the errors-in-variables (EIV) models of which the complete description of the covariance matrices of the observation vector and of the design matrix can be employed. This contribution presents one of the well-known theories—least squares variance component estimation (LS-VCE)—to the total least squares problem. LS-VCE is adopted to cope with the estimation of different variance components in an EIV model having a general covariance matrix obtained from the (fully populated) covariance matrices of the functionally independent variables and a proper application of the error propagation law. Two empirical examples using real and simulated data are presented to illustrate the theory. The first example is a linear regression model and the second example is a 2D affine transformation. For each application, two variance components—one for the observation vector and one for the coefficient matrix—are simultaneously estimated. Because the formulation is based on the standard least squares theory, the covariance matrix of the estimates in general and the precision of the estimates in particular can also be presented. Keywords Weighted total least squares · Errors-invariables model · Least squares variance component estimation · 2-D affine transformation A. R. Amiri-Simkooei Section of Geodesy, Department of Surveying Engineering, Faculty of Engineering, University of Isfahan, 81746-73441Isfahan, Iran A. R. Amiri-Simkooei (B) Chair Acoustics, Faculty of Aerospace Engineering, Delft University of Technology, Delft, The Netherlands e-mail:
[email protected]
1 Introduction In many geodetic applications, different observation types usually lead to a model with heterogeneous observations. One reason for such heterogeneity is the use of observations with different precision. In a linear model, to reach the best linear unbiased estimators (BLUE) of the unknown parameters, a proper covariance matrix of the observables is required. Therefore, observations with different precision (weights) should adequately contribute to the ‘best’ solution. This can be achieved by employing the variance component estimation (VCE) methods of which different noise components can be estimated. There exist several VCE methods like Helmert method, minimum norm quadratic unbiased estimation (MINQUE), best invariant quadratic unbiased estimation (BIQUE), restricted maximum likelihood (REML), and the least squares method. We make use of the least-squares variance component estimation (LS-VCE) (Teunissen 1988a; Teunissen and Amiri-Simkooei 2008; Amiri-Simkooei 2007). LS-VCE has been recently applied to a series of geodetic data sets. For applications of LS-VCE to GNSS data series, we may refer to Amiri-Simkooei and Tiberius (2007), Amiri-Simkooei et al. (2007), Amiri-Simkooei (2009) and Amiri-Simkooei et al. (2009, 2013). When the observables are normally distributed, LS-VCE gives identical results with those of the many of the existing VCE methods such as the BIQUE (Koch 1978, 1999; Crocetto et al. 2000; Schaffrin 1983; Caspary 1987), the MINQUE (Rao 1971; Rao and Kleffe 1988; Sjöberg 1983; Xu et al. 2007), and the REML estimator (Koch 1986). The VCE methods have been widely applied to the socalled standard least squares (SLS) problem in which only the observation vector is subject to heterogeneous noise. A significant part of literature on the estimation theory distin-
123
936
guishes between the SLS and the total least squares (TLS). The latter originates from the work of Golub and Van Loan (1980) in mathematical literature in which they introduced the errors-in-variables (EIV) models. An EIV model differs from the standard Gauss–Markov model (GMM) in the sense that the coefficient matrix connecting the parameters to the random observable vector is also affected by random errors. In geodetic literature, an EIV model treated as the 2-D nonlinear symmetric Helmert transformation was introduced by Teunissen (1988b) in which the exact solution was given using a rotational invariant covariance structure. Later, many other researchers contributed to the solution of the EIV models. We may for instance refer to Schaffrin and Wieser (2008), Schaffrin and Felus (2009), Schaffrin and Wieser (2009), Schaffrin and Wieser (2011), Tong et al. (2011) and Shen et al. (2011). Most of the research ongoing in the TLS problem concerns estimation of the parameters of interest in the functional part of the model. In this case, it is usually assumed that the stochastic model of the TLS problem either is completely known or is unknown only for the variance factor of the unit weight. It is well known that the ‘best’ property of the least squares can only be guaranteed when the inverse of a proper covariance matrix is used as the weight matrix. This holds also for the TLS problem. In an earlier study (Amiri-Simkooei and Jazaeri 2012), the weighted total least squares (WTLS) problem was formulated based on the SLS theory in which the existing body of knowledge of the least squares theory can be applied to the EIV models. Based on this formulation, Amiri-Simkooei and Jazaeri (2013) applied the data snooping procedure to the EIV models. Having available the theories of LS-VCE and WTLS, both formulated in the least squares framework, this contribution presents the formulation of the LS-VCE to the EIV models. The method can accordingly be applied to the many existing VCE methods, such as MINQUE, BIQUE, and REML. Two numerical examples are presented to illustrate the theory. This paper is organized as follows. In Sect. 2, the LS-VCE theory is reviewed. Application and implementation of LSVCE to the EIV models are provided in Sect. 3. Section 4 presents the applicability of the algorithm to two numerical examples: (1) a linear regression model of which both x and y coordinates have been observed and having different variances, and (2) a planar linear affine transformation in which the coordinates are observed in both the start and target systems. For this application, we also estimate one variance component for each system. For both examples, estimates for the variance components along with their standard deviations are provided. We make the conclusions in Sect. 5.
123
A. R. Amiri-Simkooei
2 LS-VCE on standard least-squares problem To establish the least squares variance component estimation (LS-VCE), consider the linear(ized) form of the Gauss– Markov model y = Ax + e,
D (y) = Q y ,
(1)
where y is the m-vector of observations, A is the m ×n design matrix, e is the m-vector of residuals, x is the n-vector of unknown parameters, and Q y is the m ×m covariance matrix of the observations. D is the dispersion operator. When the covariance matrix Q y of the observables is not completely known, one may write D (y) = Q y = Q 0 +
p
σk Q k ,
(2)
k=1
where Q k , k = 1, . . . , p are some m ×m symmetric cofactor matrices, and σk , k = 1, . . . , p are the corresponding variance and/or covariance components; for the sake of brevity, we will refer to them as variance components. The cofactor matrices are assumed known, while the variance components are assumed unknown to be estimated. The symmetric matrix Q 0 is assumed to be the known part (if any) of the stochastic model. LS-VCE is adopted to cope with any type of error components in the observations. The least squares estimates for the p-vector of unknown variance components are obtained as (Teunissen and Amiri-Simkooei 2008; Amiri-Simkooei 2007) σˆ = N −1l,
(3)
where the entries of the p × p matrix N and p-vector l are ni j =
1 ⊥ −1 ⊥ tr(Q i Q −1 y PA Q j Q y PA ) 2
(4)
and li =
1 t −1 1 −1 ⊥ −1 ⊥ eˆ Q y Q i Q −1 y eˆ − tr(Q 0 Q y PA Q i Q y PA ) 2 2
(5)
with i, j = 1, . . . , p, eˆ = PA⊥ y the least squares residuals, −1 t −1 and PA⊥ = I − A(At Q −1 y A) A Q y an m × m orthogonal projector. We note that the inverse of the normal matrix N automatically produces the covariance matrix of the variance −1 components, i.e., Q σ = N . Because the covariance matrix Q y and the projector PA⊥ are both functions of the unknown variance components, we may perform iterations to obtain the final estimates. Therefore, we start with an initial guess for the σk s, k = 1, . . . , p. Based on these initial values, Q y , PA⊥ , and eˆ are calculated and hence N and l are obtained by Eqs. (4) and (5), respectively. New update for the variance components is provided by σˆ = N −1 l. They are then considered the improved values
Application of LS-VCE to EIV models
937
for σk s. The procedure is repeated until the estimated values do not change by further iterations. A few remarks on the applicability of LS-VCE to many geodetic applications are highlighted. The LS-VCE method is conceptually a simple method because it is based on the well-known principle of the least-squares. It is also an attractive VCE method because it allows one to apply the existing body of knowledge of least-squares theory to the problem of (co)variance component estimation. We can at least enumerate that this method allows one to (1) obtain measures of discrepancies in the stochastic model, (2) determine the covariance matrix of the variance components, (3) obtain the minimum variance estimator of variance components by choosing the weight matrix as the inverse of the covariance matrix, (4) take the a priori information on the variance components into account, (5) solve for a nonlinear variance component model, (6) apply the idea of robust estimation to variance components (see, Khodabandeh et al. 2012), (7) evaluate the estimability of the variance components, (8) avoid the problem of obtaining negative variance components, and (9) statistically test whether or not the selected stochastic model is an appropriate one. For detailed information on LS-VCE, we may refer to Teunissen and Amiri-Simkooei (2008). 3 LS-VCE on total least-squares problem 3.1 Weighted total least-squares problem In many geodetic applications, we usually assume that only the observables y are contaminated by random noise. In this case, the standard least squares theory can be used to estimate the unknown parameters along with the possible variance components. There are, however, cases that the model itself can also be contaminated by random noise. In this case, the Gauss–Markov model in Eq. (1) is replaced by an EIV model expressed as y = (A − E A ) x + e y
(6)
with its stochastic properties characterized by
ey eA
0 e Qy 0 ∼ , σ02 := y , vec(E A ) 0 0 QA
The weighted total least squares estimate of the unknown parameters x is (Amiri-Simkooei and Jazaeri 2012) t −1 −1 ˜ ˜ xˆ = A − E A Q y˜ A − E A t ˜ A xˆ y − E × A − E˜ A Q −1 y˜
(8)
which resembles the standard least squares estimate in which A˜ = A − E˜ A plays the
role of the design matrix A, Q y˜ = t Q y + xˆ ⊗ Im Q A xˆ ⊗ Im plays the role of the covariance matrix Q y , and y˜ = y − E˜ A xˆ plays the role of the observable vector y. Therefore, we deal with the predicted ˜ the predicted observable vector y˜ , and the design matrix A, covariance matrix Q y˜ of the predicted observables. In the preceding formulation, the error matrix E˜ A is obtained as E˜ A = −ivec Q A (xˆ ⊗ Im )Q −1 (y − A x) ˆ , (9) y˜ where the operator ivec reconstructs an mn-vector to an m ×n matrix. Equation (8) is written as −1 ˜ xˆ = A˜ t Q −1 (10) A A˜ t Q −1 y˜ y˜ y˜ The formulation presented in Eqs. (8) and (10) is also in agreement with the Fang’s formulation (see Fang 2011). Equation (10) was also proved by Jazaeri et al. (2013) without the use of the Lagrange multipliers. As already pointed out by Amiri-Simkooei and Jazaeri (2012), this formulation of WTLS allows one to directly apply the existing body of knowledge of the standard least squares theory to many problems in EIV models. For example, the covariance matrix of the unknown parameters xˆ in the EIV model is approximated as −1 ˜ (11) Q xˆ = A˜ t Q −1 y˜ A which directly provides the second-order moments of the estimated parameters. Also the estimated vectors of the ˜ +e˜ observables and of the total residuals in the model y˜ = Ax were, respectively, yˆ = PA˜ y˜ = A˜ xˆ = (A − E˜ A )xˆ , (12) eˆ = P ⊥˜ y˜ = y˜ − yˆ = y − A xˆ A
(7)
where e y is the m-vector of observational noise, A is the m × ncoefficient matrix of input variables (observed), E A is the corresponding m × n matrix of random noise, x is the n-vector of unknown parameters, D(e y ) = σ02 Q y and D (e A ) = σ02 Q A are the corresponding symmetric and nonnegative dispersion matrices of size m × m and mn × mn, respectively. In both expressions, σ02 is the (un)known variance factor of the unit weight.
−1 ˜ where PA˜ = A˜ A˜ t Q −1 and P ⊥ = I − A˜ t Q −1 y˜ A y˜ A˜ −1 ˜ A˜ A˜ t Q −1 A˜ t Q −1 y˜ A y˜ are two orthogonal projectors (Teunissen 2000). We highlight that eˆ = y − A xˆ is the so-called ‘total residuals’ of the model and hence different
from the y − A xˆ = residuals e˜ y = y − yˆ = y − A˜ xˆ = Q y Q −1 y˜ −1 Q y Q y˜ eˆ of the observations y. We have in fact eˆ = e˜ y − E˜ A x. ˆ In addition, the covariance matrices of the least-squares esti˜ xˆ A˜ t and Q eˆ = mates yˆ and eˆ are Q yˆ = PA˜ Q y˜ = AQ ⊥ P ˜ Q y˜ = Q y˜ − Q yˆ , respectively. Further, the least-squares A estimate of the variance factor of the unit weight is given as
123
938
σˆ 02 =
A. R. Amiri-Simkooei
eˆ T Q −1 y˜ eˆ
(13)
m−n
√ with the (estimated) standard deviation of σˆ σˆ 2 = 2σˆ 02 / 0 √ m − n (see Amiri-Simkooei 2007). Because all matrices involved along with the predicted observation vector y˜ are functions of the unknown vector x, the final estimates can be sought in an iterative procedure. 3.2 LS-VCE on weighted total least squares problem Consider the model of observation equations in an EIV model y = (A − E A ) x + e y
(14)
of which its stochastic properties have a more general form than Eq. (7) ey 0 Qy 0 e ∼ , := y , (15) eA vec(E A ) 0 0 QA where D (y) = Q y =
p1
σk Q yk
(16)
k=1
and D (A) = Q A =
p 1 + p2
σk Q Ak
(17)
k= p1 +1
are the covariance matrices of the observable vector and the coefficient matrix, respectively. In general, both matrices are supposed to have a few unknown variance components to be estimated. This is a common assumption in many geodetic applications where different observation types usually lead to a model with heterogeneous observations. For example, when the observations have different precision, the only variance factor σ02 of unit weight in Eq. (7) is not likely sufficient to express the noise characteristics of the observables. To reach the best linear unbiased estimators (BLUE) of the unknown parameters, a proper covariance matrix should include more sophisticated parameters such as those introduced in Eqs. (16) and (17). t When rewriting Q y˜ = Q 0 + Q y + (x ⊗ Im )Q A (x ⊗ Im ) p in the form of Q y˜ = Q 0 + k=1 σk Q k (similar to the structure presented in Eq. 2), it follows that the cofactor matrices are indeed of the form Q k = Q yk
(18)
for k = 1, . . . , p1 , and
Q k = x t ⊗ Im Q Ak (x ⊗ Im )
(19)
for k = p1 +1, . . . , p1 + p2 . With this structure, we note that there exist p1 number of unknown variance components in the covariance matrix of the observable vector and p2 number of components in the covariance matrix of the coefficient
123
matrix. The total number of unknown components is summed up to p = p1 + p2 . The matrix Q 0 is also included to express the known (if any) part of the stochastic model in Q y , in Q A , or in both, which is assumed to be constant and hence does not change through the iterations. For example, if the covariance matrix Q y of the observables y is known (i.e., Q 0 = Q y and p1 = 0), one may only be interested in estimating the variance components in Q A . Having available all of the matrices involved, we may now apply the LS-VCE theory of Sect. 2 to the EIV model. An iterative algorithm should be used for applying the LSVCE method to the EIV model (Algorithm 1). Two iterative loops are required to apply this algorithm: an internal loop for the weighted total least squares adjustment and an external loop for the variance component estimation. We the cofactor matrices of the Q A , i.e.,
note that Q k = x t ⊗ Im Q Ak (x ⊗ Im ) for k = p1 +1, . . . , p1 + p2 , should also be modified through the iterations.
Application of LS-VCE to EIV models
There are a few issues when dealing with the parameter estimation in nonlinear models. They mainly include the bias induced in the estimated parameters and the convergence (rate) to the optimal solution. Peter Teunissen has intensively studied the problem of nonlinear estimation. We may for example refer to Teunissen (1984,1985,1990) and Teunissen and Knickmeyer (1988). Further explaination goes in the following two notes. Note 1 The covariance matrices Q xˆ and Q σˆ were directly obtained by the error propagation law and based on the assumption that the vector x, of the functional model, and the vector σ , of the stochastic model, are not random. In real cases, both of these vectors are unknown and, therefore, one has to use their estimates xˆ and σˆ . This will then take the risk to have random vectors in the formulation. This indicates that for application of the error propagation law one has to take into account the randomness of xˆ and σˆ , which has been ignored in the formulation. The approximation applied is indeed in conjunction with all nonlinear least squares problems in which the estimated covariance matrices are just approximates of real ones because a linearized form is used for the application of the error propagation law. In addition, the unbiasedness property of the linear least squares problem will also be violated in the TLS formulation. Therefore, the estimators xˆ and σˆ are biased. This bias is proportional to the observations precision and the geometric properties of the manifold such as curvature (see Teunissen 1985; Teunissen and Knickmeyer 1988). Note 2 In Algorithm 1, two iterative loops, namely an internal loop for the WTLS adjustment of the functional model and an external loop for the LS-VCE of the stochastic model, are required. The WTLS problem is to be iteratively solved due to the intrinsic nonlinearity of the problem in the functional model. The LS-VCE is also a nonlinear problem in the stochastic model. The convergence of the nonlinear problems is not always guaranteed. The degree of the nonlinearity of the manifold is a key issue when evaluating the convergence of the nonlinear least squares models (Teunissen 1985, 1990).
4 Numerical results and discussions To verify the efficacy of the presented algorithm, two examples are provided. Both examples have been widely used in many geodetic TLS research papers and are particularly of interest in engineering Surveying and Geomatic applications. The first example is a linear regression model in which a real data set is used. The second example is a 2-D affine transformation for which simulated weighted datasets have been used. In both examples, individual variance components for the observation vector and for the coefficient matrix are simultaneously estimated. In addition, because the variance
939
components are estimated using the LS-VCE, the precision of the estimates can directly be provided. We note that for both examples considered in this contribution, the degree of the nonlinearity of the manifolds is not very severe as they are both bilinear problems. Many simulated results on both examples show that these problems are always converged to the final optimal solution with high convergence rate.
4.1 Linear regression model The first example considers the problem of linear regression model in which variables u and v have been observed: vi − evi = a(u i − eu i ) + b. Therefore, errors in both variables are involved. We aim to estimate the slope a and intercept b of the regression line using the presented WTLS and LS-VCE algorithms. If we define the parameter vector as x = [a, b]T , only the first column of the coefficient matrix in Eq. (6) has random errors, while the values in the second column are fixed. We may also assume that in general the precision of observables vi and u i is different. Let us assume that Q vm and Q um are the cofactor matrices of the v and u observables, respectively. The scales of these two matrices are assumed unknown. We then compose the Q vm and covariance matrices as follows: Q y = σ1 Q y1 = σv2 1 0 Q A = σ2 Q A2 = σu2 (Q n=2 ⊗ Q um ), where Q n=2 = 0 0 (σv2 and σu2 are the variance factors of v and u, respectively).
p Comparing this formulation with Q y˜ = Q 0 + k=1 σk Q k and those provided in Eqs. (18) and (19) indicates that σ1 = σv2 , σ2 = σu2 , Q 0 = 0, Q 1 = Q y1 = Q vm , and Q 2 = (x t ⊗ Im )Q A2 (x ⊗ Im ) = (x t ⊗ Im )(Q n=2 ⊗ Q um )(x ⊗ Im ) = x t Q n=2 x Q um = a 2 Q um . We note that when the matrices Q vm and Q um are linearly dependent, Q 1 and Q 2 become linearly dependent and hence the variances σv2 and σu2 are not simultaneously estimable. We now consider an application of the method when the matrices Q vm and Q um are not linearly dependent and the variance factors σv2 and σu2 are simultaneously estimable. We use the data presented and used by Neri et al. (1989), which later used by Schaffrin and Wieser (2008) and Amiri-Simkooei and Jazaeri (2012) (see Table 1). In the studies by Neri et al. (1989) and Schaffrin and Wieser (2008), only estimates for the unknown parameters x = [a, b]T were provided. In the study by Amiri-Simkooei and Jazaeri (2012), the precision of the estimates along with an estimate for the variance of the unit weight (i.e., σˆ 02 = 1.4832941492) was also provided. The (estimated) √ √ standard deviation of the estimated σˆ 02 is σˆ σˆ 2 = 2σˆ 02 / 10 − 2 = 0.741647074. 0 We now aim to estimate two separate variance factors σv2 and σu2 , simultaneously for the v and u coordinates. We note that Q 1 = Q vm = (diag([Wv1 , . . . , Wv10 ]))−1 and
123
940
A. R. Amiri-Simkooei
Table 1 Observed points and corresponding weights according to Neri et al. (1989)
Table 2 Two variance factors along with standard deviations estimated using Algorithm 1 for linear regression model
Point no.
Parameter/ standard deviation
This paper with two Parameter/ variance factors standard deviation
σˆ v2
1.82131151
σσˆ v2
Observed data
Initial weights
Estimated weights
v
u
Wv
Wv
1
5.9
0.0
1.0 1,000.0
0.55 1,523.52
2
5.4
0.9
1.8 1,000.0
0.99 1,523.52
3
4.4
1.8
4.0
500.0
2.20
4
4.6
2.6
8.0
800.0
4.39 1,218.81
5
3.5
3.3
20.0
200.0
10.98
304.70
6
3.7
4.4
20.0
80.0
10.98
121.88
7
2.8
5.2
70.0
60.0
38.43
91.41
8
2.8
6.1
70.0
20.0
38.43
30.47
9
2.4
6.5
100.0
1.8
54.91
2.74
10
1.5
7.4
500.0
1.0
274.53
1.52
Wu
Wu
761.76
σˆ 02
Variance factor of unit weight
1.48329415
1.08555683
σσˆ 2
0.74164707
σˆ u2
0.65637635
–
–
σσˆ u2
0.90202588
–
–
0
Variance factor of unit weight is also indicated Table 3 Estimated straight-line parameters along with their standard deviations and correlation coefficient using data of Table 1 Parameter/ standard deviation
Exact solution (Neri et al. 1989)
WTLS with only variance of unit weight σˆ 02
WTLS with two variance factors σv2 and σu2
aˆ bˆ
−0.480533407
−0.480533407
−0.489907073
5.479910224
5.479910224
5.527557906
σaˆ
–
0.070620270
0.067155233
σbˆ
–
0.359246523
0.361275441
Exact solution by Neri et al. (1989), WTLS solution with the only variance of unit weight σˆ 02 , and WTLS solution with the two variance factors σv2 and σu2
Fig. 1 Estimates of two variance factors σu2 and σv2 of linear regression model over different iteration steps applied to data set of Neri et al. (1989)
Q 2 = a 2 Q um = a 2 (diag([Wu 1 , . . . , Wu 10 ]))−1 are not linearly dependent and the corresponding variances are simultaneously estimable. Algorithm 1 with the thresholds = 10−10 and δ = 10−8 is used to estimate the two variance factors iteratively. The initial values of the variance factors are σv2 = 1 and σu2 = 1. Figure 1 shows the two variance factors σv2 and σu2 at each iteration steps. The external loop of the algorithm converges after 17 iterations. Table 2 shows the estimated variance factors along with their standard deviations after the last iteration. Estimated line parameters using the two scenarios are presented in Table 3. As can be seen, the WTLS algorithm with the only variance of the unit weight provides the so-called ‘exact solution’, given by Neri et al. (1989). When the two variance factors are introduced, one for the observation vec-
123
tor and one for the coefficient matrix, the results are slightly modified. This is what we would expect because estimating two separate variance components for the u and v axes will affect the contribution of the observations on each axis to the final least squares solution and, therefore, the results deviate with the so-called ‘exact solution’. The second issue as to the estimation of two variance components for this data set may initially originate from the fact that the estimated variance of unit weight is not one, but σˆ 02 = 1.48329415. This may indicate that the weights presented in the Neri’s paper are in fact approximate ones. Therefore, a more realistic estimation of the weights for the observations on these two axes may be performed using the LS-VCE of which the results were presented in Table 2. This will then allow one to modify the initial weights in Table 1 to the estimated weights, presented also in the same table. 4.2 Two-dimensional affine transformation The planar linear affine transformation (six-parameter transformation) is expressed as ⎡
ut vt
=
u s vs 1 0 0 0
0 0 u s vs
a1 ⎢ b1 ⎢ ⎢ c1 0 ⎢ ⎢ a2 1 ⎢ ⎣ b2 c2
⎤ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎦
(20)
Application of LS-VCE to EIV models
941
where the parameters c1 and c2 are the shifts along the u and v axes, and a1 , a2 , b1 and b2 express together four physical parameters of the 2-D linear transformation, i.e., two scales along the u and v axes, one rotation, and one nonperpendicularity parameter. u s and vs are the coordinates of a given point in the start system, while u t and vt are its corresponding coordinates in the target system. When a series of points (i.e. i = 1, . . . , k points) are observed in both the start and the target coordinate systems, the transformation parameters along with their standard deviations can be estimated using the WTLS algorithm provided by Amiri-Simkooei and Jazaeri (2012). Equation (20) makes in total m = 2k number of equations and six number of unknown parameters to be estimated. The observation vector y and the design matrix A are ⎡
u t1 ⎢ vt1 ⎢ ⎢ .. ⎢. y=⎢ ⎢ .. ⎢. ⎢ ⎣ ut k vtk
⎡
⎤ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎦
u s1 ⎢0 ⎢ ⎢ .. ⎢. A=⎢ ⎢. ⎢ .. ⎢ ⎣ us k 0
vs1 0 .. . .. . vsk 0
1 0 .. . .. . 1 0
0 u s1 .. . .. . 0 u sk
0 vs1 .. . .. . 0 vsk
0 1 .. . .. . 0 1
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ (21) ⎥ ⎥ ⎥ ⎦
The observation vector y is subject to observational noise e y and the coefficient matrix A is subject to an error matrix E A (of size m × 6). We then deal with the model y = (A − E A ) x + e y , where x = [a1 , b1 , c1 , a2 , b2 , c2 ]t is the vector of unknown parameters to be estimated. We assume that the covariance matrix of the observation vector y is Q u t vt . Further, it is assumed that the covariance matrix of the functionally independent variables u s1 , vs1 , . . ., u sk , vsk (elements of the coefficient matrix A) is the m × m matrix Q u s vs . One can simply show that ⎡ ⎤ u Q 1 ⎢ s1 ⎢ Q 2 ⎥ ⎢ vs1 ⎥⎢ . ⎢ ⎢ Q 3 ⎥ ⎢ .. ⎥⎢ ⎢ vec(A) = ⎢ ⎥⎢ ⎢ Q 4 ⎥ ⎢ ... ⎣ Q5 ⎦ ⎢ ⎣ us k Q6 vsk ⎡
⎤ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎦
Q 5 = Ik ⊗ Q 6 = Ik ⊗
0 0 0 1 0 0 0 0
, ,
Application of the error propagation law to Eq. (22) gives ⎤ Q1 ⎢ Q2 ⎥ ⎥ ⎢ ⎢ Q3 ⎥ ⎥ QA = ⎢ ⎢ Q 4 ⎥ Q u s vs ⎥ ⎢ ⎣ Q5 ⎦ Q6 ⎡
⎡
⎤t Q1 ⎢ Q2 ⎥ ⎢ ⎥ ⎢ Q3 ⎥ ⎢ ⎥ ⎢ Q4 ⎥ ⎢ ⎥ ⎣ Q5 ⎦ Q6
(23)
Therefore, a proper application of the error propagation law constructs Q A regardless of whether the original variables are fully correlated or not, i.e. whether Q u s vs is fully populated or it is a diagonal matrix. The results for fully populated Q u s vs are already provided by Amiri-Simkooei and Jazaeri (2012). We now consider an application of LS-VCE to a 2-D affine transformation. Figure 2 considers ten data points in the start and target coordinate systems (actual coordinates). The actual transformation parameters are x = [a1 , b1 , c1 , a2 , b2 , c2 ]t = [2, −3, 120, −1, 1, 70]t . Table 4 presents the ten data points simulated in the start and target coordinate systems (m = 20). For the simulation process, the weights of the observations in the start and target systems are required, which are also provided in this table. The cofactor matrices in both coordinate systems −1 and are then Q u t vt = diag Wu t1 Wvt1 . . . Wu t10 Wvt10 −1 , where the Q u s vs = diag Wu s1 Wvs1 . . . Wu s10 Wvs10 weights come from Table 4. We note that when simulating the errors for the observation vector and the design matrix, use is made of the MATLAB built-in function mvnrnd.m of Q u t vt
(22)
where Q 1 = Ik ⊗ Q 2 = Ik ⊗ Q 3 = Ik ⊗
1 0 0 0 0 1 0 0 0 0 0 0
0 0 Q 4 = Ik ⊗ 1 0
, , , ,
Fig. 2 Real coordinates (errorless) of 10 points in start and target system
123
942
A. R. Amiri-Simkooei
Table 4 Observed points and corresponding weights in start and target coordinate systems Point no.
Start system us
Wu s
Target system vs
Wvs
ut
Wu t
vt
Wvt
1
70.08 1,050 49.98 1,500 109.98 1,600 49.99
6,800
2
66.17 1,200 61.84 1,100
67.09 2,000 65.57
2,200
3
56.14 1,000 69.00 1,350
25.30 9,900 82.87
1,600
4
43.76 1,000 69.07 1,000
0.58 1,350 95.20
4,700
5
33.81 2,700 61.66 1,350
2.37 3,650 97.93
1,850
6
30.01 2,700 50.00 3,150
29.99 7,250 90.00 14,150
7
33.95 1,100 38.26 1,200
72.90 1,950 74.44
3,500
8
43.79 6,300 30.97 1,050 114.69 1,650 57.15
1,100
9
56.17 1,450 30.93 1,350 139.42 1,000 44.85
1,300
10
66.17 1,700 38.32 1,050 137.60 1,300 42.07
1,900
and 4Q u s vs , respectively. Let us now assume that the scales of the cofactor matrices Q u t vt and Q u s vs are unknown; the expected values of these two unknown parameters are then 1 and 4, respectively. The covariance matrices are composed as follows: Q y = σ1 Q y1 = σt2 Q u t vt and Q A = σ2 Q A2 = σs2 Q A2 , where Q A2 is expressed in Eq. (23) and σt2 and σs2 are the two unknown variance factors in the target and start systems, respectively.
p=2 Comparing this formulation with Q y˜ = Q 0 + k=1 σk Q k and those provided in Eqs. (18) and (19) indicates that σ1 = σt2 , σ2 = σ s2 , Q 0 = 0, Q 1 = Q y1 = Q u t vt , and Q 2 = x t ⊗ I20 Q A2 (x ⊗ I20 ). Algorithm 1 with the thresholds = 10−10 and δ = 10−8 is used to estimate the two variance factors iteratively. The initial values of the variance factors are σt2 = 1 and σs2 = 1. Figure 3 shows the two variance factors σt2 and σs2 at each iteration steps. The external loop of the algorithm converges after 20 iterations; practically, however, only a few iterations are required. Table 5 shows the estimated variance factors along with their estimated standard deviations after the last iteration. The results for the simplest case, in which only the variance factor of the unit weight is considered, are also presented in this table. Estimated transformation parameters using the algorithm of this contribution are presented in Table 6. It includes the results for the simplest case of the only variance factor of the unit weight and those for the two variance factors σt2 and σs2 . When two variance factors are introduced, one for the target system and one for the start system, the results are slightly modified. They are in fact closer to their actual values provided above. This can be expected because a more realistic stochastic model will result in more precise estimates of the unknown parameters. The simulation process explained above was implemented over 10,000 independent runs. The WTLS and LS-VCE
123
Fig. 3 Estimates of two variance factors σt2 and σs2 of planar linear affine transformation model over different iteration steps applied to data set in Table 4 Table 5 Two variance factors for target and start coordinate systems of planar linear affine transformation model along with standard deviations estimated by Algorithm 1 Parameter/ standard deviation
This paper with two variance factors
Parameter/ standard deviation
Variance factor of unit weight
σˆ t2
0.77622290
σˆ 02
2.70418327
σσˆ 2
0.69497263
σσˆ 2
1.35209164
σˆ s2
4.23998855
–
–
σσˆ s2
2.24738841
–
–
t
0
Variance factor of unit weight is also indicated Table 6 Estimated planar linear affine transformation parameters along with their standard deviations using data of Table 4; WTLS solution with the only variance factor of unit weight σˆ 02 (columns 2 and 3), and WTLS solution with the two variance factors σv2 and σu2 (columns 4 and 5) Estimated linear affine transformation parameter
WTLS with only variance of unit weight σˆ 02
WTLS with two variance factors σt2 and σs2
Estimated value
Estimated value
Standard deviation
Standard deviation
aˆ 1 bˆ1
2.0039
0.0034
2.0036
0.0042
−2.9973
0.0038
−2.9977
0.0047 0.3145
cˆ1
119.6581
0.2572
119.6996
aˆ 2 bˆ2
−1.0015
0.0014
−1.0014
0.0017
0.9983
0.0017
0.9986
0.0018
cˆ2
70.1661
0.1100
70.1418
0.1252
algorithms (Algorithm 1) were applied to these data sets. The average (over 10,000 runs) estimates of variance factors are σˆ t2 = 0.9976 and σˆ s2 = 4.0590, which closely fol-
Application of LS-VCE to EIV models
low their actual values 1 and 4, respectively. Their standard deviations (of 10,000 estimates) are σˆ σˆ 2 = 0.8486 and σˆ σˆ s2 t = 2.2467. Finally, a comment on the application of LS-VCE to the EIV models is in order. For both applications considered in the present contribution, the cofactor matrices were assumed to be diagonal. However, the proposed formulation can be generalized to have fully populated cofactor matrices. For example, the cofactor matrices Q u t vt and Q u s vs in the 2-D affine transformation were assumed to be diagonal. These matrices can in general be any positive definite matrices, although the error propagation law in Eq. (23) may in general result in a non-negative definite cofactor matrix Q A . Because the formulation is based on the matrix Q A , rather than its inverse, the possible singularity of Q A is not a drawback to the results of the EIV model in general and to the LS-VCE results in particular.
5 Concluding remarks The least squares method is a well-known estimation principle in a linear(ized) model. In two previous works, the problems of (1) variance component estimation (VCE) and (2) weighted total least squares (WTLS) were formulated based on the least squares principle. This contribution applied the least squares variance component estimation (LS-VCE) to the EIV models. This application takes advantages of the LS-VCE and WTLS problems, both formulated using the standard least squares theory. An iterative algorithm that performs the WTLS and LS-VCE problems at the same time was presented. Two iterative loops were required to apply this algorithm to an EIV model: an internal loop for the WTLS adjustment and an external loop for the LS-VCE problem. The algorithm uses the complete description of the covariance matrices of the observation vector and of the coefficient matrix, possibly with some unknown components of each. To make the corresponding cofactor matrices involved, a proper application of the error propagation law can be used. The efficacy of the proposed algorithm was demonstrated by solving two commonly used WTLS problems in geodetic literature, namely a linear regression model and a 2-D affine transformation using real and simulated data. For each application, two variance components in the stochastic model, one for the observation vector and one for the coefficient matrix, were simultaneously estimated. The standard deviation of the estimated variance factors was also obtained from the covariance matrix of the estimates. The proposed algorithm was shown to be simple in the concept and easy in the implementation. This application is also attractive and flexible because the theory of LS-VCE can directly be applied to many total least squares problems.
943 Acknowledgments I would like to acknowledge three anonymous reviewers for their valuable comments, which improved the presentation and quality of this paper.
References Amiri-Simkooei AR (2007) Least-squares variance component estimation: theory and GPS applications. PhD thesis, Delft University of Technology, Publication on Geodesy, 64, Netherlands Geodetic Commission, Delft Amiri-Simkooei AR, Tiberius CCJM (2007) Assessing receiver noise using GPS short baseline time series. GPS Solut 11(1):21–35 Amiri-Simkooei AR, Tiberius CCJM, Teunissen PJG (2007) Assessment of noise in GPS coordinate time series: methodology and results. J Geophys Res 112:B07413. doi:10.1029/2006JB004913 Amiri-Simkooei AR (2009) Noise in multivariate GPS position timeseries. J Geod 83(2):175–187 Amiri-Simkooei AR, Jazaeri S (2012) Weighted total least squares formulated by standard least squares theory. J Geod Sci 2(2):113–124 Amiri-Simkooei AR, Jazaeri S (2013) Data-snooping procedure applied to errors-in-variables models. Studia Geophysica et Geodaetica 57(3):426–441 Amiri-Simkooei AR, Teunissen PJG, Tiberius CCJM (2009) Application of least-squares variance component estimation to GPS observables. J Surv Eng 135(4):149–160 Amiri-Simkooei AR, Zangeneh-Nejad F, Asgari J (2013) Least-squares variance component estimation applied to GPS geometry-based observation model. J Surv Eng. doi:10.1061/(ASCE)Su.1943-5428. 0000107 Caspary WF (1987) Concepts of network and deformation analysis. Technical report, School of Surveying, The University of New South Wales, Kensington Crocetto N, Gatti M, Russo P (2000) Simplified formulae for the BIQUE estimation of variance components in disjunctive observation groups. J Geod 74:447–457 Fang X (2011) Weighted total least squares solutions for applications in Geodesy. PhD dissertation, Publ. No. 294, Dept. of Geodesy and Geoinformatics, Leibniz University, Hannover, Germany Golub G, Van Loan C (1980) An analysis of the total least squares problem. SIAM J Numer Anal 17:883–893 Jazaeri S, Amiri-Simkooei AR, Sharifi MA (2013) Iterative algorithm for weighted total least squares adjustment. Surv Rev (in press) Khodabandeh A, Amiri-Simkooei AR, Sharifi MA (2012) GPS position time-series analysis based on asymptotic normality of M-estimation. J Geod 86:15–33 Koch KR (1978) Schätzung von Varianzkomponenten. Allgemeine Vermessungs Nachrichten 85:264–269 Koch KR (1986) Maximum likelihood estimate of variance components. Bull Géod 60:329–338, ideas by A.J. Pope Koch KR (1999) Parameter estimation and hypothesis testing in linear models. Springer, Berlin Neri F, Saitta G, Chiofalo S (1989) An accurate and straightforward approach to line regression analysis of error-affected experimental data. J Phys Ser E Sci Instr 22:215–217 Rao CR (1971) Estimation of variance and covariance components— MINQUE theory. J Multivar Anal 1:257–275 Rao CR, Kleffe J (1988) Estimation of variance components and applications. Series in Statistics and Probability, vol 3. North-Holland, Amsterdam Schaffrin B (1983) Varianz-kovarianz-komponenten-schätzung beider ausgleichung heterogener wiederholungsmessungen C282. Deutsche Geodätische Kommission, München Schaffrin B, Felus Y (2009) An algorithmic approach to the total leastsquares problem with linear and quadratic constraints. Studia Geophysica et Geodaetica 53:1–16
123
944 Schaffrin B, Wieser A (2008) On weighted total least-squares adjustment for linear regression. J Geod 82(7):415–421 Schaffrin B, Wieser A (2009) Empirical affine reference frame transformations by weighted multivariate TLS adjustment. In: Drewes H (ed) International Association of geodesy symposia, vol 134. Geodetic Reference Frames. Springer, Berlin, pp 213–218 Schaffrin B, Wieser A (2011) Total least-squares adjustment of condition equations. Studia Geophysica et Geodaetica 55:529–536 Shen Y, Li B, Chen Y (2011) An iterative solution of weighted total least-squares adjustment. J Geod 85:229–238 Sjöberg LE (1983) Unbiased estimation of variance-covariance components in condition adjustment with unknowns—a MINQUE approach. Zeits ür Vermessungswesen 108(9):382–387 Teunissen PJG (1984) A note on the use of Gauss’ formula in nonlinear geodetic adjustments. Stat Descisions 2:455–466 Teunissen PJG (1985) The geometry of geodetic inverse linear mapping and nonlinear adjustment. Netherlands Geodetic Commission, Publication on Geodesy, New Series, vol 8, No. 1, Delft Teunissen PJG (1988a) Towards a least-squares framework for adjusting and testing of both functional and stochastic model. Geodetic Computing Centre, Delft, MGP Series No. 26, 2004 (Reprint 1988)
123
A. R. Amiri-Simkooei Teunissen PJG (1988b) The nonlinear 2D symmetric Helmert transformation: an exact nonlinear least-squares solution. Bull Geod 62:1–15 Teunissen PJG (1990) Nonlinear least-squares. Manuscr Geod 15(3):137–150 Teunissen PJG, Knickmeyer EH (1988) Nonlinearity and least squares. CIAM J ACSGC 42(4):321–330 Teunissen PJG (2000) Adjustment theory: an introduction. Delft University Press, Delft University of Technology, Series on Mathematical Geodesy and Positioning Teunissen PJG, Amiri-Simkooei AR (2008) Least-squares variance component estimation. J Geod 82(2):65–82 Tong X, Jin Y, Li L (2011) An improved weighted total least squares method with applications in linear fitting and coordinate transformation. J Surv Eng 137(4):120–128 Van Huffel S, Vandewalle J (1991) The total least-squares problem. Computational Aspects and Analysis. SIAM, Philadelphia Xu PL, Liu YM, Shen YZ (2007) Estimability analysis of variance and covariance components. J Geod 81:593–602
