Error Bounds for Conditional Algorithms in Restricted Complexity Set Membership Identi cation A. Garulli
B. Z. Kacewicz y
A. Vicino
G. Zappa z
Abstract Restricted complexity estimation is a major topic in control-oriented identi cation. Conditional algorithms are used to identify linear nite dimensional models of complex systems, the aim being to minimize the worst-case identi cation error. High computational complexity of optimal solutions suggests to employ suboptimal estimation algorithms. This paper studies di erent classes of conditional estimators, and provides results that assess the reliability level of suboptimal algorithms.
Keywords. Control-oriented identi cation, restricted complexity, set membership, linear mod-
els, projection algorithms.
1 Introduction A considerable amount of the recent literature is focused on control-oriented identi cation, either in a `soft bound' or in a `hard bound' setting of the problem (see e.g. [1, 2, 3, 4, 5]). One of the most important topics in this area of research is restricted complexity (or reduced order) model estimation. Although there exist several settings in which this Dipartimento di Ingegneria dell'Informazione, Universita di Siena. Via Roma 56, 53100 Siena, Italy. E-mail:
[email protected],
[email protected]. Tel. (39) 577-263612; Fax: (39) 577-263602. Corresponding author is Andrea Garulli. y Department of Mathematics, Informatics and Mechanics, Warsaw University. Banacha 2, 02-097 Warsaw, Poland. E-mail:
[email protected]. Tel./Fax: (48) 22-6583236. z Dipartimento di Sistemi e Informatica, Universit a di Firenze. Via di S. Marta 3, 50139 Firenze, Italy. E-mail:
[email protected] .it. Tel. (39) 55-4796263; Fax: (39) 55-4796363.
1
problem can be studied, the basic assumption is that the true system lives in the world of real systems, including in nite dimensional and possibly nonlinear systems, while the set of models within which we look for an `optimal' estimate lives in a nite, possibly low dimensional linear world, whose structure and topology is compatible with modern robust control synthesis techniques. The main problem consists in nding the element in the model set whose distance from the true system is minimal. In this paper an information based complexity perspective is taken to deal with the restricted complexity estimation problem in a worst case setting. Several classes of algorithms have been proposed in the literature ([6, 7, 8]), such as conditional central estimators, central projection estimators, restricted projection estimators, interpolatory projection estimators. Conditional central estimators are known to be optimal [6]. Although for speci c choices of norms these estimators are computable in an e ective way, in general their computational complexity is formidable, which prevents their use in practical applications. For this reason, suboptimal estimators have often been adopted, exhibiting lower computational complexity at the price of larger estimation errors. In spite of this, surprisingly little attention has been devoted to derive bounds on the estimation errors of these algorithms ([7, 9]). The aim of this paper is to provide a number of results in this direction. The main result regards the central projection estimator when both the estimation error and the noise are measured according to the Euclidean norm. For this case, it is shown that there exists a tight upper bound on the ratio of the error of the central projection estimator and the optimal error, i.e. the conditional radius of information of the problem. The paper is organized as follows. Restricted complexity estimation and conditional algorithms are introduced in Section 2, together with some preliminary results in a general setting. A rather complete characterization of conditional `2 estimates for energy bounded noise is given in Section 3. Some concluding remarks are reported in Section 4.
2 Problem formulation and general results Let X and Y be linear nite dimensional normed spaces, with dim(X ) dim(Y ), and x be an unknown element in X . Then, suppose that a set of noisy measurements y 2 Y are available and satisfy
y = Fx + e 2
(1)
where e 2 Y is an additive noise vector and F : X ! Y is a known linear mapping, usually referred to as the information operator. It is assumed that F as a matrix is full rank. The noise vector is supposed to be unknown but bounded in some norm:
kekY "
(2)
with given " > 0. Now, consider the following estimation problem.
Problem (Restricted complexity estimation) Let M be a given set in X . Find optimal or suboptimal (in some sense) estimation algorithms : Y ! M, that provide an estimate z = (y ) of the unknown element x 2 X . In the following, these algorithms are called conditional or restricted complexity estimators; the set of all conditional estimators will be denoted by M. In the set membership setting, the quality of the estimate provided by an algorithm is usually evaluated according to the Y -local worst-case estimation error, given by
E ( ) = sup kx ; (y)kX
(3)
F = F (y; ") = fx 2 X : kFx ; ykY "g
(4)
x2F
where is the feasible parameter set, i.e. the set of all elements in X that are compatible with the information provided by the measurements y and the noise bound " (it will be assumed that F 6= ;). The minimum error associated with the class of conditional estimators M is called conditional radius of information and is given by
RM = 2inf E ( ): M If M = X , we denote by R = RX the (unconditional) radius of information, i.e. the minimum error that can be achieved by any estimation algorithm. Obviously, RM R, for any set M. A conditional estimator that achieves the minimum error RM is said to be conditionally optimal. Moreover, if an algorithm satis es the inequality E ( ) k RM 8y; for some xed k 2 IR, k > 1, then is said to be k-almost optimal (or optimal within a factor k). 3
It is useful to recall that the Chebishev center of a set S X is de ned as
c(S ) = arg xinf sup kx ; skX : 2X s2S
(5)
and r(S ) = sups2S kc(S ) ; skX is called Chebishev radius of the set S . The diameter of S is given by d(S ) = maxx;w2S kx ; wkX , and d(S ) 2r(S ). Analogously, given two sets S ; Z X , the conditional Chebishev center of S , with respect to Z , is given by
cZ (S ) = arg zinf sup kz ; skX : 2Z s2S
(6)
and rZ (S ) = sups2S kcZ (S ) ; skX denotes the conditional Chebishev radius of S . Now, let us introduce some algorithms, usually employed in restricted complexity estimation problems.
De nition 1 A conditional central estimator is de ned as cc(y) = zcc where zcc = arg zinf sup kz ; xkX = cM(F ): 2M x2F
(7)
De nition 2 An interpolatory projection estimator is de ned as ip(y) = zip where zip = arg zinf kz ; xikX 2M
(8)
and xi is an element of the set F .
De nition 3 A central projection estimator is given by cp(y) = zcp where zcp = arg zinf kz ; c(F )kX : 2M
(9)
De nition 4 A restricted projection estimator is given by rp(y) = zrp where zrp = arg zinf kFz ; ykY : 2M
(10)
In the following, optimality and almost optimality of the estimation algorithms are studied in a general setting, i.e. without specifying the set F , the estimate set M and the norms adopted in the spaces X and Y . Since the norms are general, they will be omitted when clear from the context. First, let us recall the following property that follows from the de nition of conditional central algorithm [6]. 4
Theorem 1 The conditional central algorithm cc is conditionally optimal in the class of
conditional estimators M
E ( cc) E ( ) 8y ; 8 2 M : Hence, the conditional central algorithm provides the minimum estimation error (3), among all the conditional estimators. Notice that, from the de nitions of conditional and unconditional radius of information, we get respectively RM = rM(F ) = E ( cc) and R = r(F ). The aim of the following theorems is to characterize the quality of the estimates provided by the algorithms introduced by De nitions 2-4, in terms of the quantities R and RM .
Theorem 2 Any interpolatory projection estimator satis es E ( ip) RM + 2R: Proof. Let x 2 F and z 2 M. From De nition 2, one has kzip ;xk kzip ; xi k +kxi ; xk kz ; xi k + d(F ) supw2F kz ; wk +2R. The result is obtained by maximizing over x 2 F and minimizing over z 2 M. 2
Theorem 3 Any central projection estimator satis es E ( cp) RM + 2R: Proof. Let x 2 F and z 2 M. From De nition 3, one has kzcp ; xk kzcp ; c(F )k + kc(F ) ; xk kz ; c(F )k + kc(F ) ; xk kz ; xk + 2kc(F ) ; xk. The result is obtained by maximizing over x 2 F and minimizing over z 2 M. 2 Notice that the previous result cannot be obtained from Theorem 2, since the Chebishev center of a set does not necessarily belong to the set itself (i.e., cp is not an interpolatory projection algorithm, in general). When this occurs, the bound on the worst-case performance of the central projection algorithm can be improved.
Corollary 1 If c(F ) 2 F , then E ( cp) RM + R: 5
Proof. The result immediately follows from the proof of Theorem 3, by noting that kzcp ; c(F )k RM. 2 Since R RM , the above results can be restated in terms of \almost optimality" of the estimation algorithms. In particular, interpolatory projection and central projection algorithms are optimal within a factor 3. Moreover, if c(F ) 2 F , the central projection algorithm is optimal within a factor 2. In general, it is not possible to give a bound on the estimation error provided by the restricted projection algorithm rp. In Sect. 3, it will be shown that it can not be found a k such that this algorithm is k-almost optimal.
3 Conditional `2 estimators with energy bounded noise In several identi cation problems, linear parameterization of the estimated model is required, i.e. the set M is supposed to be a linear subset of X . Let X = IRn, Y = IRm , with m n, and make the following assumptions. (A1). The set M is a p-dimensional linear manifold in IRn, with p < n, i.e. M = fz 2 IRn : z = zo + M g, where M 2 IRnp is full rank. (A2). k kX = k kY = `2 . Assumption (A2) means that energy-bounded noise is considered in (2) and the estimation error (3) is measured in the energy norm. For ease of notation, the l2 norm will be denoted by k k in the following. According to the assumptions (A1) and (A2), the feasible parameter set (4) is given by the ellipsoid
F = fx 2 IRn : x0 F 0Fx ; 2y0Fx + y0y "2g:
(11)
and the estimates zip, zcp, zrp in (8), (9) and (10) can be easily computed as
zip = zo + M (M 0 M );1 M 0 (xi ; zo ) zcp = zo + M (M 0 M );1 M 0 (F 0F );1F 0(y ; Fzo ) zrp = zo + M (M 0 F 0FM );1 M 0 F 0(y ; Fzo ) :
(12) (13) (14)
On the other hand, a dicult min-max optimization problem must be solved in order to compute the conditional central estimate zcc in (7). A complete characterization of the conditional center of F and an ecient procedure for computing it has been recently proposed in [10]. 6
Through a suitable change of coordinates in the space X (see the Appendix), the restricted estimation problem can be reformulated w.l.o.g. as follows. Let F be the ellipsoid
F = E = fx 2 IRn : x0 Qx 1g Q = diagfqigni=1 ; 0 < q1 q2 qn
(15) (16)
and M be the linear manifold
M = fz 2 IRn : z = zo + M ; 2 IRp; p < ng
M 0 M = Ip ;
M 0zo = 0 :
(17) (18)
Then c(E ) = 0, and the estimates (12)-(14) provided by the interpolatory projection, central projection and restricted projection algorithms are given by
zip = zo + MM 0 xi zcp = zo zrp = [In ; M (M 0 QM );1 M 0 Q]zo :
(19) (20) (21)
In the following, the evaluation of the reliability level of the above estimates is addressed. This is done by computing tight upper bounds on the worst-case estimation error of each conditional projection algorithm.
Theorem 4 Let assumptions (A1)-(A2) hold. Then, the central projection algorithm cp
satis es
r
E ( cp) 43 RM :
Moreover, there exist estimation problems in which the upper bound is achieved. Proof. First, let us show that one can assume w.l.o.g. p = dim(M) = n ; 1. In fact, suppose p < n ; 1 and denote by Me the (n ; 1)-dimensional linear manifold containing zcp = zo and orthogonal to zo . By de nition, the estimate provided by cp and the error E ( cp) do not change if we consider Me instead of M. Moreover, M Me and hence RMe RM . Therefore, the maximum ratio between E ( cp) and the conditional radius of information is always achieved for an (n ; 1)-dimensional manifold. Consider the setting (15)-(18) and de ne
xo = arg max kzo ; xk: x2E 7
(22)
Denote by z+ and z; the orthogonal projection of respectively xo and ;xo onto M. Set = arccos(kz+ ; z;k=kxo ; z; k) and zm = z; + 2 cos1 2 (z+ ; z; ): for 0 < 2 (the case = 2 corresponds to z+ = z; = zo and is trivial). It is easy to verify that zm satis es kzm ; xo k = kzm ; z;k. Then, de ne h = kxo ; z+ k, l = kxo ; zm k, d = kxo ; zo k (a sketch of the above situation is depicted in Fig. 1, for the case n = 3, p = 2). Now, let us consider the set E~ = fz; ; xog, containing only two xo 0 -x
o
d
l
h
M α
z-
o
z
zm
z+
Figure 1: The geometric setting of the proof of Theorem 4. points. By construction, the projection of its Chebishev center onto M is zo and hence (by de nition of xo ) the error E ( cp) is the same for both E and E~, and is given by d. Moreover, the conditional Chebishev center of E~ is given by ( z if 0 4 z~cc = m+ z if 4 < 2 and the corresponding conditional radius is ( l if 0 4 R~M = : h if 4 < 2 Then, by noting that ;xo 2 E , one has
RM maxfkzcc ; xo k; kzcc + xo kg maxfkzcc ; xo k; kzcc ; z; kg zmin maxfkz ; xo k; kz ; z; kg = R~M: 2M 8
Hence,
8 >
RM R~M : Finally, standard calculations lead to
d = cos p4 ; 3 cos2 if 0 4 l p 2 : d = 4 tan + 1 if 4 < 2 h 2 tan
max g( ) = 0 2
q
r
4 3
and the maximum is achieved for = arccos 23 . This means that the bound is tight. 2 Remark 1 The above result can be extended to a more general class of sets. It can be observed that Theorem 4 still holds for a compact set E such that (;xo + z o ) 2 E , for some 2 IR. In fact, the projection of (;xo + zo ) onto M is again z ; and the same reasoning follows. In particular, this means that Theorem 4 holds for all compact balanced sets.
Theorem 5 Let assumptions (A1)-(A2) hold. Then, the interpolatory projection algo-
rithm ip satis es
E ( ip) RM + R
and there exist estimation problems in which the upper bound is achieved. Proof. Let us consider again the setting (15)-(18). Denote by e+ and e; the extremum points of d(E ) (the diameter of E ), and for any z 2 M let be the angle between the vectors z and e+. One has max kz ; xk maxfkz ; e+k; kz ; e;kg x2E p = kzk2 + r2(E ) + 2kzkr(E )j cos j p kzo k2 + r2(E ); where r(E ) is the length of the maximal semiaxis of E . Minimizing over z 2 M one obtains
p
RM h2 + R2 where h = kzo k and R = r(E ). On the other hand, from (19) one gets for any x 2 E
kzip ; xk kzo + MM 0 xi k + kxk p = kzo k2 + kMM 0 xik2 + kxk p kzo k2 + kxi k2 + kxk; 9
(23)
and then, recalling that xi 2 E by de nition, and maximizing over x 2 E
p
E ( ip) h2 + R2 + R RM + R:
(24)
Moreover, it can be easily seen that the upper bound is achieved when M contains the diameter of the ellipsoid E , and xi is one of the extremum points of the diameter. 2
Remark 2 Theorem 5 can be extended to a more general class of sets. In particular, the above proof still holds for any balanced set E . Now, the restricted projection algorithm introduced in De nition 4 is considered. The following straightforward proposition gives a simple geometric interpretation of the estimate zrp and provides an upper bound on the estimation error of the restricted projection algorithm, in the case when there are conditional estimates that belong to the feasible set E.
Proposition 1 Let (A1) and (A2) hold. If E T M 6= ;, then T (i) zrp in (21) is the symmetry center of the set E M; (ii) E ( rp ) 2R. However, the following theorem shows that it is not possible to determine a bound on the estimation error of rp in the general case.
Theorem 6 Let (A1) and (A2) hold. For any k > 0, there exists a restricted complexity
estimation problem such that
E ( rp) > k RM :
Proof. Point (i) is just a corollary of Proposition 1. In order to prove point (ii), let us set set n = 2, p = 1. Then consider the ellipsoid E in (15) with matrix Q = Q0 (generally nondiagonal), and the manifold M in (17) with zo = [;1 0]0 and M = [0 1]0. Let 1 = 1 and 2 > 1 be the eigenvalues of Q, and 2 (0; 2 ) be the angle between the maximal semiaxis of E (of length 1) and M (see Fig. 2). Moreover, let 2 = 2 ( ) be parameterized with . Rotating E around its center so that ! 0, for 2( ) ! +1 suciently fast one gets
zcc ! zro p RM ! 12 + kzo k2 = 2 1 10
M
o
z
E
α
λ-1/2 2
1
0
Figure 2: The geometric setting of the proof of Theorem 6. Since by (21) zrp is the projection of 0 onto M in the weighted norm kxk2Q = x0Qx, one 2 also has kzrpk ! +1 and E ( rp) ! +1. Hence lim !0 ER( Mrp) = +1: p Since in Theorem 4 it has been proven that the central projection algorithm is 4=3almost optimal, one could conjecture that the estimate provided by the algorithm cp is always \better" than that of the algorithm rp. This is not true, as it is shown in the following example.
Example 1 Consider the ellipsoid E de ned by (15) with Q = diagf0:05; 0:25; 2:5g, and the linear set M in (17), where zo = [;0:47 ; 0:1 0:94]0, M = v=kvk2 with v = [;0:17 1:269 0:05]0. By computing the conditional central estimate zcc, following
the procedure described in [10], and the estimates zcp and zrp according to (20)-(21), one obtains the following estimation errors
E ( cc) = 25:0691; E ( cp) = 25:3368; E ( rp) = 25:1121: which disproves the conjecture E ( cp) E ( rp).
2
4 Conclusions A rather complete characterization of the reliability level of conditional estimation algorithms in set membership identi cation has been provided. Computing tight upper 11
bounds on the worst-case estimation error is important for evaluating the degree of suboptimality of conditional estimators and allows to assess model quality. For the case when energy-bounded noise is considered, it is shown that central projection and interpolatory p projection algorithms are almost optimal within a factor 4=3 and 2 respectively, while the estimation error of restricted projection algorithms may be unbounded. Pointwise bounded noise and di erent error norms will be the subject of future research.
References [1] A. J. Helmicki, C. A. Jacobson, and C. N. Nett. Control oriented system identi cation: a worst-case/deterministic approach in H1. IEEE Transactions on Automatic Control, 36:1163{1176, 1991. [2] R. L. Kosut, G. C. Goodwin, and M. P. Polis (Guest Eds.). Special issue on system identi cation for robust control design. IEEE Transactions on Automatic Control, 37:899{1008, 1992. [3] G. C. Goodwin, M. Gevers, and B. Ninness. Quantifying the error in estimated transfer functions with application to model order selection. IEEE Transactions on Automatic Control, 37:913{928, 1992. [4] P. M. Makila, J. R. Partington, and T. K. Gustafsson. Worst-case control-relevant identi cation. Automatica, 31(12):1799{1819, 1995. [5] P. M. J. Van den Hof and R. J. P. Schrama. Identi cation and control - closed loop issues. Automatica, 31:1751{1770, 1995. [6] B. Z. Kacewicz, M. Milanese, and A. Vicino. Conditionally optimal algorithms and estimation of reduced order models. Journal of Complexity, 4:73{85, 1988. [7] L. Giarre, B. Z. Kacewicz, and M. Milanese. Model quality evaluation in set membership identi cation. Automatica, 33(6):1133{1139, 1997. [8] A. Garulli, A. Vicino, and G. Zappa. Optimal and suboptimal H2 and H1 estimators for set-membership identi cation. In Proc. of 36th IEEE Conf. on Decision and Control, pages 764{766, San Diego, December 1997.
12
[9] B. Z. Kacewicz. Quality of conditional system identi cation in a general class of norms. Technical Report RW 97-02, Institute of Applied Mathematics and Mechanics, Warsaw University, 1997. [10] A. Garulli, A. Vicino, and G. Zappa. Conditional central algorithms for worst-case estimation and ltering. In Proc. of 36th IEEE Conf. on Decision and Control, pages 2453{2458, San Diego, December 1997.
Appendix For clarity of exposition, denote the new coordinates in (15)-(18) by x~, z~. Let us de ne
P = "2 ; y0y + y0F (F 0F );1F 0y
;1
(F 0 F )
and let P = UQU 0 , where Q is given by (16) and U 0 U = In. Moreover, let xs = (F 0F );1F 0y. Then, through the change of coordinates
x~ = U 0 (x ; xs) ;
x = U x~ + xs
equation (11) becomes F = fx~ 2 IRn : x~0 Qx~ 1g, which is (15). Now assume that M = fz 2 IRn : z = zo + M ; 2 IRp; p < ng, with M 0 M = Ip and M 0 zo = 0. In the new coordinates, one has
M = fz~ 2 IRn : z~ = U 0 (zo ; xs ) + U 0 M = U 0 (zo ; xs + MM 0 xs) + U 0 M ( ; M 0 xs) = z~o + M~ ~ ; ~ 2 IRpg where z~o = U 0 (zo ; xs + MM 0 xs ), M~ = U 0 M and ~ = ; M 0 xs. It is easy to verify that M~ 0 M~ = Ip and M~ 0 z~o = 0, which are the conditions (18).
13
