Balancing Matrix Factorization via Gradient Flow Techniques and the

Ofttlo 3olh Cuwemnm ~ ~won andCmmd @ton,EI%YIwJ +OwmMu 1991

&lngs

VV2=8 -1:30

Balancing Matrix Factorization via Gradient Flow Techniques and the Singular Value Decomposition * J.E.

Perkinsl,

U. Helmke2,and

1 Department

of Systems

Research Australian : i)epartment

School

National

of Mathematics,

of Physical University,

‘Ibis paper

explores

various

gradient

1

flows

value

decompositions are a widely Ilse(] special case of tile more general baia.ncit]g factorization task: given a nlat,rix II fitld .\, F SIICI1t]la~ /i,,,xl = ,Y,,,x,.Frx I Illl(lcr fllc l)aiancing constraint, .Y’.Y = YY’, and .Y, Y iuy of full rank. ‘~llere is a cl,ass of dutiotw to this tuk, unique Up to orthogonal transformations. The solution .f = ~~~, ~ = D+ V’, where U, V are appropriate orthogothen gives a nal 111atrice9 and ~ i9 diagonal Singu]ar Va]ue decomposition H = (J DV’.

A parallel situation in systems theory is realizations which that of diagonal ba]anced special members of the class of all balallce~ realizations. It has been shown that

are

*

“work p~tially supported and DSTO Auatrdia.

k%76-7/91/oooo-0551 m: L

by

Boeing

$01 .00 G 1991 IEEE

Box 4, Canberra 8400

ACT

2601.

Regensburg,

F.R. G.

The problem of finding the singular value decomposition of a (rectangular) matrix is conventionally solved using recursive algebraic matrix manipulations. It has been shown [1], [2], [3] that such matrix manipul~ tions can be viewed as samplings of a differential equation (self-equivalent flow) where the limiting solution of the flow gives the desired singular value decomposition. The work [4] [5] gives alternative self eqllivalent, flows wl]ich are gradient flows on unitary matrices. This work can be viewed as a generalization of the gradient flows on orthogonal matrices for diagonalizing a square symmetric matrix of [6]. Also in earlier work [7], balancing matrix factorization are studied along with other balancing methods via exponentially convergent gradient flows on positive definite matrices. Building on this work, related gradient flows on co-ordinate transformation matrices to achieve balanced realizations have been developed in [8], along with gradient flow equations on the system matrices themselves.

being factorized

Introduction

Sillgu]ar

Engineering,

the more familiar diagonal balanced realization may not be best for certain minimum sensitivity applications, but rather some other realization. This suggests to us balanced that the study of balancing matrix factorization tasks as above could also be of a wider interest, although the only specific application we are aware of at this stage, is in connection with finding balanced realizations from Hankel matrices.

on ]nanifolds which converge exponentially to of which the balanced matrix factorization, singular value decomposition is the most well known. Such flows are initialized on trivial nonbalanced factorization. We look at flows f,-lrthe transformation matrix given an initial factorization, as well as flows on the matrix factors themselves. More general flows are given that allow the matrix to be parameter dependent.

and

of Regensburg,

Abstract

Moorel

Engineering,

Sciences GPO

University

J.B.

(BCAC)

=’

Here,

we apply

the technical

[8], to explore gradient

approach

of

lar transformation matrices associated with balanced matrix factorization, and associated gradient flows for factors which converge to satisfy the balancing constraint. The resulting highly nonlinear matrix differential equations with smoothly convergent behaviour seem to us to be of independent interest. Also we build on the technical approach of [9] exploiting the exponential convergence properties of the differential equations to deal with the case of decomposition of parametrized matrices along trajectories in parameter space, as for example, time varying matrices. Our aim is to clemonstrate a tracking capacity by showing that along such parameter space trajectories, we can track the relevant factorization arbitrarily closely. In Section 2, gradient flows on the set of symmetric positive definite matrices of [7] [8] are reviewed namely

F(t) = P-l(i)

2

Flows on P, T

Gradient

flows on nonsingu-

YY’P-l (t) – X’X

for some initial full rank factorization H = XY. The limiting solution ~ = P(m) has the form P = TT’ where T is a transformation matrix such that X = XT, ~ = ~-l Y. As a gradient flow, this equation has the form ~(i) = – v @ where P(i) is a time varying matrix for some scalar function, @(P). The equation resulting from this gradient flow has an exponential convergence rate, and is readily extended to the case where H is parameter dependent. It is the transformation ma trix ~, rather than ~ = ~~, which is of interest and this motivates new results, revised from [8], for evolution of the transformation matrices Z’(t) which converge to ~. These equations can also be extended to the situation where H(a) follows a trajectory in crparameter space, as when it is a time varying quantity. Section 3 gives flows on the factorization matrices X(t) and Y(t), initialized by X, Y and which converge to .~, ~. Flows for the matrices X(i) and Y(t) allow the evolution of the factorization to be observed directly. Section 4 gives some concluding remarks and discussion.

The differential equation we consider here is a gradient flow, with an associated cost function, in terms of factorizing matricea X, Y such that H = XY, and a transformation matrix T. Thus consider, Q(T) = IIX(Z’)112 + IIY(Z’’)II2 = tr[xrr’x’ + T-lYY’T’-1] (2.1) This index when minimized over all T achieve T, x = XT, F = ~- lY and the balancing condition X’X = YY’.

2.1

Flows on Positive Definite Matrices Gradient

matrix T is Given that the transformation only unique up to an orthogonal right factor, it makes sense to first work with the unique ~ = !i?~ to characterize the class of all transformation matrices ~. With the unique ~ determined, then all “square roots” ~ can be determined. Clearly (2.1) can be re-expressed in terms of P = TT’ as O(P) = ir[PX’X Before proceeding rates of variation

+ YY’.P-l]

(2.2)

let us find a bound of F(t).

Lemma 2.1 Given a fime vaying izafion X(i)Y(t) = H(i) such that X’(t)x(t) Y(t) Y’(t)

0,1< ql < C131