ThC07.6
43rd IEEE Conference on Decision and Control December 14-17, 2004 Atlantis, Paradise Island, Bahamas
Nonlinear Observers Appearing in Dynamical Machine Vision Hiroshi Inaba, Member, IEEE, Rixat Abdursul and Bijoy K. Ghosh, Fellow, IEEE
Abstract— Perspective dynamical systems arise in dynamic machine vision, in which only perspective observation is available. This paper first develops a perspective linear system in which the number of observing points is more than one, and studies a Luenberger-type nonlinear observer for such a system Then assuming a given perspective linear system to be Lyapunov stable and to satisfy some sort of detestability condition, it is shown that the estimation error converges exponentially to zero.
I. INTRODUCTION The observation obtained by observing a moving rigid body by a camera is essentially the direction vector of a point observed, called a perspective observation, because all the points on a line passing through the center of the camera are projected to a single point of the image plane. And the basic problem in dynamical machine vision is how to determine the position and velocity of a moving body and/or any unknown parameters characterizing the motion and shape of the body from such perspective observation. A perspective dynamical system arises in mathematically describing such a dynamic machine vision problem and has been studied in a number of approaches in the framework of systems theory (See, e.g., [1]-[13]). Some interesting works recently reported are concerned with nonlinear observers for estimating the unknown state of perspective linear systems [1], [10], [11]. In particular, the present authors [1] has proposed a nonlinear observer of the Luenberger-type for such a system without transforming it into an implicit system as proposed in [10],[11], and shown that under some reasonable assumptions on the given system the estimation error of the proposed nonlinear observer converges exponentially to zero.
This work was supported in part by the Japanese Ministry of Education, Science, Sports and Culture under both the Grant-Aid of General Scientific Research C-15560387 and the 21st Century Center of Excellence (COE) Program. Hiroshi Inaba is with the Department of Information Sciences, Tokyo Denki University, Hatoyama-machi, Saitama 350-0394 (corresponding author’s phone: +81-49-296-2911; fax: +81-49-296-6304; e-mail:
[email protected]). Rixat Abdursul is with the Department of Information Sciences, Tokyo Denki University, Japan (e-mail:
[email protected]). Bijoy K. Ghosh is with the Department of Electrical and Systems Engineering, Washington University, USA (e-mail:
[email protected]).
0-7803-8682-5/04/$20.00 ©2004 IEEE
3903
In these previous works it is assumed that only one point on a moving body is observed, but in more realistic situations the number of observing points, even the number of moving bodies, may be more than one. A similar situation occurs in computer vision problems. In fact, a problem of multiple observing points has been discussed from the viewpoint of dynamical computer vision [4], and a nonlinear filter has been studied in order to causally estimate the 3-dimentional shape by integrating noisy visual information of multiple observing points over time. The objective of the present paper is to consider a perspective dynamical system with multiple observing points and/or more than one moving body, and is to investigate a generalization of the result in [1] to such a perspective system. First, in Section II, a perspective linear system considered in this investigation is described. In Section III, a nonlinear observer of the Luenberger-type for perspective linear systems is proposed and then after discussing some important properties on such systems a convergence theorem of the nonlinear observer is presented with its sketch of proof. Section IV gives a proof of this convergence theorem. More precisely, it is shown that, under suitable assumptions on a given perspective linear system, including that it is Lyapunov stable and satisfies some sort of detestability condition, the estimation error converges exponentially to zero. Finally, Section V gives some concluding remarks. II. PERSPECTIVE LINEAR SYSTEMS This section introduces a mathematical model for a perspective dynamical system with more than one observing point. Throughout this investigation, let us denote the fields of real and complex numbers by and , respectively. First denote the number of observing points by p t 1 and assume that each perspective observation obtained from an observing point generates a smooth curve on the image plane of a camera. Then a perspective linear system considered in this investigation is given as ° x (t ) ® °¯ y (t )
Ax(t ) v(t ), x(0) H (Cx(t )) 2 p
x0 n
(1)
where x(t ) n represents the entire state of the moving bodies, v(t ) n the external input, A nun and C (3 p )un . Finally, y (t ) 2 p represents the perspective
observation vector generated on the image plane by observing the p points, and as seen from the argument in the case of one observing point [1] the function H : n o 2 p that defines the perspective observation is expressed in the following form: ªhº ½ ° H : «« # »» ¾ p, h([ ) : «¬ h »¼ °¿
ª [1 º «[ » « 3 », [ « [2 » « » ¬« [3 ¼»
ª [1 º «[ » , [ z 0 . 3 « 2» ¬«[3 ¼»
(2)
y (t )
H (Cx(t ))
ª y (1) (t ) º « » 2p « # » « ( p) » ¬ y (t ) ¼
III. LUENBERGER-TYPE NONLINEAR OBSERVERS
In this section, we propose a Luenberger-type observer for a perspective linear system of the form (1), and present our main theorem. First, notice that, denoting an estimate of the state x(t ) by xˆ(t ) , a full-order state observer for System (1) is expressed generally in the form xˆ (0)
xˆ0 n
Axˆ (t ) v(t ) K ( y (t ), xˆ (t )) u [ y (t ) H (Cxˆ (t ))], xˆ (0)
(5) xˆ0
n
K : 2 p u n o nu(2 p ) , called an observer gain matrix. The next important step is how to choose a gain matrix K ( y, xˆ ) in (5). To do so, first introduce the notations:
(3)
where y ( k ) (t ) h(C ( k ) x(t )) represents the perspective observation generated from the k -th observing point and C [C (1)T " C ( p )T ]T with C ( k ) 3u n . The purpose of the present paper is to investigate a Luenberger-type observer for System (1) without converting it to an implicit system as discussed in [10]. As mentioned in the previous work [1], the advantage of this observer over the one in [10] is that the former does not require a transformation that involves an integration of the input v( s ) (0 d s d t ) , which may cause difficulty in implementation, in particular, for the case that the input v(t ) is generated in a state feedback form.
d xˆ (t ) M ( xˆ (t ), v(t ), y (t )), dt
d xˆ (t ) dt
where K ( y, xˆ ) is a suitable matrix-valued function
Now for future discussions let us decompose the vector y (t ) 2 p into p sub-vectors as follows: ª h(C (1) x(t )) º « » # « » « » ( p) h ( C x ( t )) ¬ ¼
freedom to adjust its characteristics. As such a function, it is possible to choose r ( xˆ , y ) K ( y, xˆ )[ y h(Cxˆ )] where K ( y, xˆ ) is any sufficiently smooth function. Thus, it is possible to consider a nonlinear observer of the Luenberger-type:
(4)
and satisfies the condition that whenever xˆ(0) x(0) the solution xˆ(t ) of (4) coincides completely with the solution x(t ) of System (1) for any choice of v() . Therefore, under the condition that (4) has a unique solution, it is possible to assume that the function M ( xˆ , v, y ) has the form M ( xˆ , v, y ) Axˆ v r ( xˆ , y ) where r ( xˆ , y ) is any sufficiently smooth function satisfying the condition r ( x, h(Cx)) 0 for all x n . Among many functions r ( xˆ , y ) satisfying this condition, it would be wise to choose a function, which is reasonably simple, but has sufficient
3904
Cx
ª [1(1) º « (1) » « [2 » « (1) » (1) ªC º « [3 » « » « » « # »x :« # » « ( p) » «[1( p ) » ¬C ¼ « » «[ 2( p ) » « ( p) » ¬«[3 ¼»
[ 3 p ,
and similarly Cxˆ : [ˆ R 3 p 3 p , and use them to simplify the term y H (Cxˆ ) as follows: y H (Cxˆ ) ª [1(1) « (1) « [3 « (1) « [2 « [ (1) « 3 « « ( p) « [1 « [ ( p) « 3 « [ ( p) « 2 ( p) ¬« [3
H (Cx) H (Cxˆ )
[ˆ1(1) º » [ˆ3(1) »
ª ª « «1 « 1 « « ˆ(1) « « [3 « 0 « « ¬ « « « ª « «1 « « 1 « « [ˆ( p ) « « 3 «0 « « ¬ ¬
[1(1) º ª (1) ˆ(1) º » [1 [1 [3(1) » « (1) ˆ(1) » » «[ 2 [ 2 » [ 2(1) » « (1) ˆ(1) » 1 (1) «[3 [3 » ¼ [3 »¼ ¬
º » » » » [ˆ(1) » 2(1) » » [ˆ3 » » » » » # # » ( p) » ( p) º ˆ [1 » [1 ( p) ( p) º » ˆ ª ( p) » 0 ( p ) » [1 [1 » [ˆ3 » [3 » « ( p ) ˆ ( p ) » » » «[ 2 [ 2 » » ( p) » ˆ [2 [ 2( p ) » « ( p ) ˆ( p ) » » » ( p) 1 ( p ) «[3 [3 » ¼» [ˆ3 ¼» [3 ¼» ¬ ¼ ª 1 ª º (1) º 0 « ˆ (1) ¬ I 2 y ¼ " » [ « 3 » « » ([ [ˆ ) # % # « » (6) 1 « ( p) º » ª " ( p) ¬ I2 y ¼ » 0 « [ˆ3 ¬ ¼ ( ( xˆ ) B ( y )C U where I 2 indicates the 2 u 2 identity matrix and
0
° ° °( ( xˆ ) : ° ° ° ® ° ° ° ° B( y) : ° ° ¯
ª 1 « (1) I 2 " « C3 xˆ « # % « « 0 " « «¬ ª ª I 2 y (1) º ¼ «¬ « # « « 0 ¬
º » » » 2 p # » 1 I2 » C3( p ) xˆ »»¼ " % " ª¬ I 2
(7) T
With the expression (6), one obtains K ( y, xˆ )( ( xˆ ) B( y )C U
(9) (k )
appearing in ( ( xˆ (t )) , one can choose a gain matrix K ( y, xˆ ) of the form: (10)
*
where C indicates the complex conjugate transpose of C and P nun is an appropriately chosen matrix, which is considered to be a free parameter for the gain matrix. And with this choice for K ( y, xˆ ) , the Luenberger-type nonlinear observer (5) becomes d xˆ (t ) dt
Axˆ (t ) v(t ) P 1C * B* ( y (t ))( 1 ( xˆ (t ))
(14)
,
and hence to eliminate from (9) all the denominators C3 xˆ
P 1C * B* ( y )( 1 ( xˆ )
C * B* ( y (t W ))
u B( y (t W ))Ce AW E0 dW t H I r , t t 0.
(8)
K ( y, xˆ )[ y H (Cxˆ )]
* A*W
³0 E0e
º » » \ (2 p )u(3 p ) # » y ( p ) º¼ » ¼
0
U : x xˆ.
K ( y, xˆ )
W , W0 n denote the generalized eigenspaces corresponding to V ( A) and V 0 ( A) , respectively, and choose a basis matrix E0 [[1 " [ r ] for W0 where r : dimW0 . Then, there exist T ! 0 and H ! 0 such that
0
Remark 3.2. All the conditions given in Assumption 3.1 are necessary and/or reasonable requirements from the viewpoint of machine vision. (i) The condition (i) is imposed to ensure that if v(t ) { 0 then the motion of a moving body takes place within a bounded space. (ii) The condition (ii) is imposed to ensure that the motion is smooth enough and takes place inside a conical region centered at the camera to produce a continuous and bounded measurement y (t ) on the image plane. (iii) The condition (iii) is imposed to ensure some sort of detestability for the perspective system (1). In fact, the inequality (14) implies that (C , A) is a detectable pair and the external input v(t ) must not be identically zero. These facts are verified in Proposition 3.3.
,
(11)
The following proposition gives some system theoretical implications of Assumption 3.1 (iii).
and it follows from (1), (11) and (6) that the differential equation for the estimation error U : x xˆ is obtained as
Proposition 3.3. Assume that System (1) is Lyapunov stable, let A0 denote the critically stable part (i.e., the part corresponding to V 0 ( A) ) of A and set C0 : CE0 . If Assumption 3.1 (iii) is satisfied, then the following statements hold true.
u [ y (t ) H (Cxˆ (t ))], xˆ (0)
d U (t ) [ A P 1C * B* ( y (t )) B( y (t ))C ]U (t ), dt U (0) x(0) xˆ (0) n .
xˆ0 n
(12)
Next, we make various assumptions on System (1), which seem to be necessary and/or reasonable from the viewpoint of dynamical machine vision. Assumption 3.1. System (1) is assumed to satisfy the following conditions.
(i) System (1) is Lyapunov stable. (ii) The observation vector y(t ) is a continuous and bounded function of t , that is, y ( ) C m [0, f) Lmf [0, f) .
(i) (C , A) is a detectable pair, that is, the critically stable part (C0 , A0 ) of (C , A) is observable. (ii) The external input v(t ) is never identically zero.
Proof. To prove (i), first recall that the pair (C , A) being detectable is equivalent to its critically stable part (C0 , A0 ) being observable. Therefore, it suffices to show that (14) implies M:
(13)
(iii) Express the set V ( A) of all eigenvalues of A as
T
³0 T
³0
*
E0* e A W C * Ce AW E0 dW *
e A0W C0* C0 e A0W dW t H 0 I r
V ( A) V ( A) V 0 ( A) where V ( A) and V 0 ( A)
for some H 0 ! 0 . To show this, first note that
indicate the sets of eigenvalues with strictly negative real part and zero real part, respectively. Let
Ce At E0
3905
CE0e A0t
C0e A0t
(15)
(16)
from which the second equality in (15) follows. In order to
show the last inequality in (15), it suffices to verify that M is strictly positive-definite. To do so, we show that if M is not strictly positive-definite then (14) is not satisfied. So assume that M is not strictly positive-definite. Then, there exists a nonzero vector [ r such that T
³0 [
* A0*W
e
T
C0*C0e A0W [ dW
³0
C0e
A0W
2
[ dW
0
C0 e A0W [ { 0 2 p .
(17)
Therefore, it follows from (16) and (17) that T
* * A*W * * E0 e C B ( y (t
W )) B( y (t W ))Ce AW E0[ dW T
³0
B ( y (t W ))C0e A0W [
2
dW
0,
(18)
where [ 0 represents the critically stable initial vector corresponding to the initial state x0 n of System (1). Noticing that the solution of (18) is given by [ (t ) e A0t[ 0 , partition the vector C0[ (t ) as follows: ª Pˆ (1) (t ) º « (1) » « P (t ) » « » 3p « # » « Pˆ ( p ) (t ) » « » «¬ P ( p ) (t ) »¼
(19)
where Pˆ ( k ) (t ) 2 , P ( k ) (t ) for k
K (t )
H (C0 [ (t ))
1, " , p . Thus
ª Pˆ (1) (t ) º « (1) » « P (t ) » « # » 2 p , « » « Pˆ ( p ) (t ) » « ( p) » ¬« P (t ) ¼»
(20)
T
* A*W
³0 [ (t ) E0 e T
B( y (t W ))C0 [ (t W ) p
T
¦ k 1 ³0
2
[ (t )
[ 0 z 0 for all t t 0 and P ( k ) (t W ) is bounded
measurement K (t ) in the critically stable subsystem (18) because the contribution from the stable subsystem of (1) to y (t ) approaches zero. Therefore, one sees that
Q([ (t )) H [ (t )
2
H [0
2
,
which implies that if v(t ) { 0 then (14) is never satisfied.
,
The inequality (14) requires that not only the pair (C , A) is detectable, but also that v(t ) must not be identically zero. It is worthwhile to mention that the present authors [6] have already shown that for a very simple perspective linear system the condition that v (t ) is not identically zero is necessary for observability. Further, it should be recalled that, for a general linear system with system matrix A and output matrix C , the detestability is completely determined by the pair (C , A) , and completely independent of the input v(t ) and initial state x (0) . On the other hand, for a general nonlinear system the detestability is generally dependent on the input and initial state or equivalently the trajectory generated by the input and initial state. Therefore in order to check the detestability of a nonlinear system it is inevitable to verify a similar condition like (14) along the trajectory, but such verification may not be an easy task, as seen in verifying if inequality (14) is satisfied for a given trajectory x(t ) . In fact there seems to be impossible to develop a reasonably ease and practical method for checking the inequality (14). Before stating our main theorem, the following lemma is cited from [1].
x (t )
C * B* ( y (t W ))
u B( y (t W ))Ce AW E0 [ (t )dW
³0
Now, consider the measurement y (t ) in System (1) with v(t ) { 0 and x0 z 0 such that [ 0 z 0 . First, note that
Lemma 3.4. Let A nun be a Lyapunov stable matrix, and consider the linear system of differential equations
and using (16), (8) and (20) one obtains Q ([ (t )) :
2
K (t W ) y (t W ) dW .
large t ! 0 such that
d [ (t ) A0 [ (t ), [ (0) [0 r ° ® dW °¯ K (t ) H (C0 [ (t )),
C0 e A0t [0
2
K (t W ) y (t W ) o 0, Q([ (t )) o 0 (t o f) , and hence that for any H ! 0 there exists a sufficiently
showing that (14) is not satisfied. Next, to prove (ii), first consider the critically stable subsystem of (1) given by
C0 [ (t )
p
for all k 1, " , p because all the eigenvalues of A0 are purely imaginary. Further note that y (t ) converges to the
which leads to
³0 [
T
¦ k 1 P (k ) (t W ) 0
d³
dW 2
Pˆ ( k ) (t W ) P ( k ) (t W ) y ( k ) (t W ) dW
Ax(t ),
x(0)
(21)
Further, use the same notations as in Assumption 3.1 (iii) and let S : C n o W , S 0 : C n o W0 denote the matrix representations of the projection operators along W0 , W , respectively. Then, for any a ! 0 the matrix inequality PA A P d aS *S
3906
x0 n .
(22)
has a positive-definite Hermitian solution P nun .
,
Now, it is ready to state our main theorem. However, since its proof requires a number of lengthy and cumbersome technical arguments, only a sketch of the proof is given here.
° U (t ) P U (t ) d U (0) P U (0), t t 0 ° t 2
® a ³0 U ( s ) ds d U (0) P U (0), t t 0 ° t ° 2 B( y ( s ))C U ( s ) 2 ds d U (0) P U (0), t t 0. ¯ ³0
Theorem 3.5 (Nonlinear Observers). Assume that System (1) satisfies Assumption 3.1 and consider a nonlinear observer of the Luenberger-type (11), that is,
Next we note that it easily follows using (7) and Assumption 3.1 (ii) that
d xˆ (t ) dt
B( y ())
Axˆ (t ) v(t ) P 1C * B* ( y (t ))( 1 ( xˆ (t ))
d 1
f
y ()
f
f.
(30)
,
(31)
(23)
Then the following lemma can be proved using Lemma 3.6, but its proof is very long and hence omitted here.
along with the differential equation (12) for the estimation error U (t ) : x(t ) xˆ (t ) , that is,
Lemma 4.2. Consider the solution U (t ) of (24) and U (t ), U0 (t ) defined in (27). Then the following inequalities are satisfied.
u [ y (t ) H (Cxˆ (t ))], xˆ (0)
xˆ0 n
d U (t ) [ A P 1C * B* ( y (t )) B( y (t ))C ]U (t ), dt U (0) x(0) xˆ (0) n .
(i) There exists a constant J ! 0 , independent of U (0) , such that
(24)
f
where E ( xˆ ) and B( y ) are given by (7). Further,
³0
let S : n o W , S 0 : n o W0 denote the matrix representations of the projection operators along W0 , W , re-
(25)
where a ! 0 is a constant.
Then, U (t ) converges exponentially to zero, that is, there exist D ! 0, E ! 0 such that x(t ) xˆ (t ) d E eD t U (0) , t t 0 .
U (t ) :
(26)
IV. SKETCH OF PROOF FOR THEOREM 3.5
U (t ) : S U (t ),
t t 0 .
U0 (t ) : S 0 U (t ),
(27)
Then since C n W W0 one has (28)
U (t ) U (t ) U0 (t ) f
2
U (t ) dt
f
2
U (t ) U0 (t ) dt
³0
d 2³
f
0
2
f
2
2
,
U0 (t ) dt d J 0 U (t0 ) .
To prove Theorem3.5, it suffices to establish by virtue of Lemma 5.1.2 in [14] that there exists some J ! 0 , independent of U (0) , such that the solution U (t ) of (24) satisfies f
³0
2
2
U (t ) dt d J U (0) .
But this task is straightforward by using Lemmas 4.2. In fact, it follows from (29) and Lemmas 3.7 that
³0
2
f
f
2
2
U (t ) dt d 2³ U (t ) dt 2³ U0 (t ) dt 0
0
d 2(J J 0 ) U (0)
2
: J U (0)
2
where J ! 0 is a constant, independent of U (0) . This completes the proof of Theorem 3.5. V. CONCLUDING REMARKS
which easily leads to the inequality
³0
f
³0
f
Using the notations in Theorem 3.5, first let us define
2
(ii) There exists a constant J 0 ! 0 , independent of U (0) , such that
spectively, and P nun be a symmetric positive definite matrix satisfying the Lyapunov inequality A* P PA d aS *S
2
U (t ) dt d J U (0) .
2
(29)
U (t ) dt 2 ³ U0 (t ) dt. 0
Now the following lemma can be easily shown [1]. Lemma 4.1. The following inequalities hold true:
This paper studied a nonlinear observer for perspective linear systems arising in dynamical machine vision. First a Luenberger-type nonlinear observer was proposed, and then under some reasonable assumptions on a given perspective system, it was shown that it is possible to construct such a Luenberger-type nonlinear observer whose estimation error converges exponentially to zero. REFERENCES [1] R. Abdursul, H. Inaba and B. K. Ghosh, Nonlinear observers for perspective time-invariant linear systems, Automatica, vol. 40, Issue 3, pp. 481-490, 2004.
3907
[2] R. Abdursul and H. Inaba, Nonlinear observers for perspective time-varying linear systems, WSEAS Transactions on Systems, vol. 3, pp.182-188, 2004. [3] R. Abdursul, H. Inaba and B. K. Ghosh, Luenberger-type observers for perspective linear systems, Proceedings of the European Control Conference, CD-ROM, Porto, Portugal, September 2001. [4] A. Chiuso, P. Favaro, H. Jin and S. Soatto, Structure from motion causally integrated over time, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 523-535, 2002. [5] B. K. Ghosh and E. P. Loucks, A perspective theory for motion and shape estimation in machine vision, SIAM Journal on Control and Optimization, vol.35, pp.1530-1559, 1995. [6] B. K. Ghosh, M. Jankovic and Y. T. Wu, Perspective problems in system theory and its applications to machine vision, Journal of Mathematical Systems, Estimation and Control, vol.4, pp.3-38, 1994. [7] B. K. Ghosh, H. Inaba and S. Takahashi, Identification of Riccati dynamics under perspective and orthographic observations, IEEE Transactions on Automatic Control, vol. 45, pp. 1267-1278, 2000. [8] W. P. Dayawansa, B. K. Ghosh, C. Martin and X. Wang, A necessary and sufficient condition for the perspective observability problem, Systems and Control Letters, vol.25, pp.159-166, 1993. [9] H. Inaba, A. Yoshida, R. Abdursul and B. K. Ghosh, Observability of perspective dynamical systems, IEEE Proceedings of Conference on Decision and Control, pp.5157-5162, Sydney, Australia, December 2000. [10] A. Matveev, X. Hu, R. Frezza and H. Rehbinder, Observers for systems with implicit output, IEEE Transactions on Automatic Control, vol. 45, pp.168-173, 2000. [11] S. Soatto, R. Frezza and P. Perona, Motion estimation via dynamic vision, IEEE Transactions on Automatic Control, vol. 41, pp.393-413, 1996. [12] M. Jankovic and B. K. Ghosh, Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer, Systems and Control Letters, vol. 25, pp. 63-73, 1995. [13] B. K. Ghosh and J. Rosenthal, A generalized Popov-Belevitch-Hautus test of observability, IEEE Transactions on Automatic Control, vol. 40, pp.176-1180, 1995. [14] R. F. Curtain and H. J. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory, SpringerVerlag, 1995.
3908
