MULTIDIMENSIONAL STABILITY TEST USING SUM-OF-SQUARES DECOMPOSITION Bogdan Dumitrescu Tampere International Center for Signal Processing Tampere University of Technology P.O.Box 553, SF-33101 Tampere, Finland e-mail:
[email protected] ABSTRACT A new stability test for d-dimensional systems is presented. It consists of maximizing the minimum eigenvalue of a positive definite Gram matrix associated with a polynomial positive on the unit dcircle. This formulation is based on expressing the polynomial as a sum of squares and leads to a semidefinite programming (SDP) problem, which may be solved reliably. Although the test is based on a sufficient condition, the practical results are accurate. Also, the test has the advantage of not giving false positives. 1. INTRODUCTION The d-dimensional discrete-time system with transfer function H(z1 , . . . , zd ) =
B(z1 , . . . , zd ) A(z1 , . . . , zd )
(1)
of length ni + 1 gathers the canonical base of one variable polynomials of degree at most ni . Similarly, with ⊗ the Kronecker product symbol, the vector p(z) = p(zd ) ⊗ . . . ⊗ p(z1 ), (4) Qd of length N = i=1 (ni + 1), contains the canonical base of d-dimensional polynomials of exponent at most n. For instance, with n1 = 2, n2 = 1, we have p(z) = [1 z1 z12 z2 z1 z2 z12 z2 ]T . We also denote n0i = ni + 1. 2. SUM-OF-SQUARES POLYNOMIALS
is structurally stable if and only if A(z1 , . . . , zd ) 6= 0, for |z1 | ≤ 1, . . . , |zd | ≤ 1.
We denote z −1 = (z1−1 , . . . , zd−1 ). Denoting k = (k1 , . . . , kd ) an ordered set of d integers, a d-dimensional monomial with exP k ponent k is z k = z1k1 . . . zd d ; its degree is |k| = di=1 ki . The vector p(zi ) = [1 zi . . . zini ]T (3)
(2)
Several simplifications of the condition (2) were proposed, e.g. the DeCarlo-Strintzis test [1]. Practical stability tests for the system (1), with d ≥ 3, are based on solving systems of polynomial equations [2, 3], nonconvex optimization [4] or transform methods [5]. None of the proposed algorithms is felt like a generally appropriate solution, due to theoretical or complexity difficulties. We propose here a different approach, based on convex optimization, namely semidefinite programming (SDP). The bulk of the method is to test the strict positivity of a d-variable polynomial on the unit d-circle. This is realized by expressing the polynomial as a sum of squares (which is always possible) and attaching it a Gram matrix. If a positive definite Gram matrix can be found, then the polynomial is strictly positive and the system stable. Although the condition is theoretically only sufficient (necessity would be guaranteed only without bounding the size of the Gram matrix), all our numerical tests gave accurate results. Also, the complexity compares favorably with other methods. The summary of this paper is as follows. In Section 2 we present the relation between SDP and sum-of-squares multivariable polynomials. Section 3 is dedicated to the description of the new stability test. Finally, Section 4 contains the report of some experiments. As a general notation rule, boldface letters denote multidimensional entities. The d-dimensional variable is z = (z1 , . . . , zd ). Work supported by Academy of Finland, project No. 44876 (Finnish Centre of Excellence Program (2000-2005))
We present here a parameterization of d-dimensional polynomials that may be expressed as a sum-of-squares (and thus are positive) on the unit d-circle. This parameterization was proposed in [6], in a very condensed form (see also [7, 8] for sum-of-squares polynomials of real variables). Our purpose is to give an explicit form of the parameterization which makes easier an SDP implementation. A symmetric d-dimensional polynomial is G(z) =
n X
gk z k ,
(5)
k=−n with coefficients symmetric about the origin g−k = gk .
(6)
Q The total number of coefficients in (5) is M = di=1 (2ni +1), but 0 due to (6) there are only M = (M + 1)/2 distinct coefficients. Any polynomial G(z) as defined in (5) may be written in Gram matrix form [7] G(z) = pT (z −1 ) · Q · p(z),
(7)
where Q ∈ RN ×N is a symmetric matrix. We first derive the relation between the coefficients of the polynomial G(z) and the elements of the Gram matrix Q. Proposition 1 Let G(z) be a polynomial of the form (5) and Q ∈ RN ×N such that (7) holds. Then, the relation gk = tr[T k · Q]
(8)
holds, where T k = T kd ⊗ . . . ⊗ T k1 0
(9)
0
and T ki ∈ Rni ×ni are elementary Toeplitz matrices with ones only on the ki -th diagonal; trX is the trace of matrix X. Proof. We may write (7) in the equivalent form G(z) = tr[pT (z −1 ) · Q · p(z)] = tr[P (z)Q],
(10)
where P (z)
p(z) · pT (z −1 ) " 1 # " 1 # O O T −1 p(zi ) · p (zi )
= (4)
=
i=d
i=d
1 h O
=
i p(zi ) · pT (zi−1 ) .
(11)
i=d
(For the last equality above, we have used d − 1 times the identity (A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD), where A, B, C and D are matrices of appropriate sizes.) Taking into account that 1 zi−1 . . . zi−ni −n +1 n zi X 1 . . . zi i p(zi )pT (zi−1 ) = . z ki T ki , = . . . .. .. .. .. k =−n i i i zini . . . . . . 1 we may rewrite (11) as " 1 # n O (9) X k T k zk . P (z) = z T ki = i=d k=−n k=−n n X
(12)
Identifying (5) with (10) and using (12), we obtain (8).
G(ej ω ) =
|R` (ej ω )|2 , ωi ∈ [−π, π],
(13)
`=1
G(z) =
ν X
R` (z −1 )R` (z),
(14)
`=1
(where ν ≤ N ) if and only if there exists a positive semidefinite matrix Q ∈ RN ×N such that (8) holds. Proof. Denoting ν the rank of Q, we can write Q=
ν X
λ2` q ` q T` ,
3. SUM-OF-SQUARES STABILITY TEST One set of conditions for describing the stability of the system (1) is given by the DeCarlo-Strintzis test [1]. According to this test, the system (1) is stable regardless the numerator polynomial B(z) (or structurally stable) if and only if the following monovariable polynomials A(z1 , 1, . . . , 1) A(1, z2 , . . . , 1) (16) .. . A(1, 1, . . . , zd ) have no roots inside and on the unit circle (i.e. they are Schur polynomials), and
Proposition 2 The polynomial (5) may be expressed as a sum-ofsquares of degree n on the unit d-circle, i.e. ν X
same diagonal, generally not all the elements on that diagonal are nonzero. Remark 2. Obviously, any sum-of-squares polynomial G(z) is positive on the unit d-circle. The reciprocal is also true [10], i.e. any polynomial positive on the unit d-circle is sum-of-squares and thus may be written as in (13); however, although the sum is guaranteed to have a finite number ν of terms, there is no guarantee (unless d = 1) that the degree of all terms R` (z) is at most n; accordingly, there is no previously known bound on the size of the positive semidefinite Gram matrix Q from (8). However, taking Q of size N × N provides a good approximation and gives good practical results. Remark 3. If G(z) is a symmetric polynomial positive on the unit d-circle, i.e. G(ej ω ) > 0, then there exists a positive definite Gram matrix Q > 0 such that (8) is true. (Proof: there exists > 0 such that G (ej ω ) = G(ej ω ) − > 0. By Prop.2, there exists a Gram matrix Q ≥ 0 of size m × m associated with G (z), in the sense of (8). Then, Q = Q + (/m)I > 0 is a Gram matrix of G(z).)
(15)
`=1
where q ` are eigenvectors of Q and λ2` the corresponding eigenvalues. Then, denoting R` (z) = λ` q T` p(z), we notice that (7) becomes (14). To obtain (13) it is sufficient to notice that pT (e−j ω ) = pH (ej ω ). Conversely, any sum-of-squares polynomial (14), with degR` (z) ≤ n, may be written like in (7), satisfying (8) for Q ≥ 0 defined as in (15). Remark 1. For Q ≥ 0, we name (8) the generalized trace parameterization, inspired from the 1-D case [9]. Note that although the matrix T k from (9) has all its nonzero elements on the
A(z1 , . . . , zd ) 6= 0, for |z1 | = . . . = |zd | = 1.
(17)
Only condition (17) is difficult to implement, since it involves all d variables, although only on the unit d-circle. From now on we assume that the 1-D conditions are satisfied and so condition (17) decides the stability. One idea of verifying (17) is to compute the minimum value of |A(ejω1 , . . . , ejωd )|, for ω ∈ [−π, π]d . If this minimum value is greater than zero, then the system is stable; if it is zero, the system is unstable. Such an idea was used in [4], where the minimum was found via genetic optimization. Since |A(ej ω )| is not a convex function, the possible danger is to find a nonzero local minimum when the global minimum is zero, and thus to conclude that the system is stable, when actually it is unstable. We propose an approach using convex optimization. Let us define the sum-of-squares polynomial G(z) = A(z −1 )A(z),
(18)
for which G(ej ω ) = |A(ej ω )|2 . Instead of finding the minimum value γ of |A(ej ω )|, we compute γ2 =
min G(ej ω ). ω ∈[−π,π]d
(19)
If γ > 0, then the system is stable; if γ = 0, the system is unstable.
To compute γ, we replace (19) with a slightly relaxed (in the sense of Remark 2 above), but convex, problem. Namely, to determine the largest constant α? such that G(z) − α? is a sum-ofsquares polynomial α? = maxα s.t.
α G(z) − α is sum-of-squares
(20)
If α? > 0, then the system is stable; if α? = 0, we decide that the system is unstable. Using the trace parameterization (8) for the polynomial G(z)− b the problem (20) may be stated as the α, with Gram matrix Q, semidefinite program (note that T 0 = I) α? = max
α
(21)
b α,Q
s.t.
b k 6= 0 gk = tr[T k · Q], b g0 − α = trQ b ≥0 Q
The relaxation appears actually only at the implementation of (21), b is chosen, according to the when the size of the Gram matrix Q chosen maximum degree of the sum-of-squares terms from (14). b to be N × N or even smaller for reasons We normally take Q given below. Another view of the problem is obtained if we notice that, on the unit d-circle, relation (7) becomes G(ej ω ) = pH (ej ω ) · Q · p(ej ω ). (22) It is clear that if Q > 0, then G(ej ω ) > 0. (See also Remark 3 above.) So, we can try to maximize the minimum eigenvalue of Q, in order to make Q strictly positive definite. The corresponding SDP problem is λ? = max λ,Q
s.t.
λ
(23)
gk = tr[T k · Q] Q ≥ λI
where ρz = (ρz1 , . . . , ρzd ); its coefficients are a ˜k = ak ρ|k| . Equivalently, we may say that the stability margin is given by the maximum ρ = 1 + σ for which the DeCarlo-Strintzis conditions ˜ (16)-(17) are satisfied. For a fixed ρ, we decide that A(z) is the ˜ denominator of a stable system by solving (21) or (23) for G(z) = ˜ −1 )A(z). ˜ A(z To find (a possibly conservative estimation of) the maximum value of ρ, we simply use bisection on the values of ρ, as proposed in [11] for 2-D systems. Complexity reduction. The size of the SDP problems (21) or (23) may be quite large, although we could solve in reasonable times all the examples found in the cited papers (and others also). However, we would want to reduce the size of the SDP problems, if possible. Assuming that G(z) is sparse (contains only few monomials), we notice that the Gram matrix representation (7) may be simplified by eliminating from p(z) some of the base monomials and thus eliminating the corresponding rows and columns from Q. (When solving (21) or (23), the same rows and columns should be eliminated from the constant matrices T k ; some of these matrices become zero, showing constraints (8) trivially satisfied.) However, the elimination should be performed with care, in order not to reduce too much the class of Gram matrices associated with the polynomial and thus affecting the results of the SDP problems. For example, if A(z) = 3 + z13 + z22 + z1 z2 , i.e. d = 2, n1 = 3, n2 = 2, then, in writing (7), the length of the vector p(z) is N = (n1 + 1)(n2 + 1) = 12. However, the polynomial (18) may be written via (7) with p1 (z) = [1 z13 z1 z2 z22 ]T instead of p(z); thus, the Gram matrix would have size 4 × 4 instead of 12 × 12. Such a reduction is often too drastic. We propose a different heuristic that gave very satisfactory results: we keep in p(z) only the monomials with exponents belonging to the Newton polytope of A(z). (The Newton polytope of a polynomial is the convex hull of the exponents of monomials with nonzero coefficients in the polynomial.) In our example, we obtain the base pp (z) = [1 z1 z12 z13 z2 z1 z2 z22 ]T of length 7, i.e. we eliminate the monomials z1 z22 , z12 z2 , z12 z22 , z13 z2 , z13 z22 . 4. IMPLEMENTATION AND RESULTS
The relation between problems (21) and (23) is clear once we b = Q−λI. Since trT = 0, for k 6= 0, remark that we can take Q k b = tr[T · Q], for k 6= 0, and also g (23) it results that tr[T k · Q] 0 = k (21) b b trQ = trQ+α. Thus, since trQ = trQ+N λ, the optimal values of (21) and (23) are related by α? = N λ? . The problem (23) has the nice interpretation of finding the ”most interior” (in the cone of positive semidefinite matrices) Gram matrix corresponding to the given sum-of-squares polynomial G(z). Stability test. To conclude, solving (21) or (23) and obtaining α? 6= 0 or λ? 6= 0 give a certificate of stability. How to decide numerically that λ? 6= 0 will be discussed in next section. If α? = λ? = 0, we infer that the system is unstable. This test gives no false positive (i.e. it is not possible to evaluate an unstable system to be stable). A refinement of the test when α? = λ? = 0 (suggested by Remark 2) would be to increase artificially the degree of G(z), by adding appropriate zero coefficients and solve again (21) or (23); however, in our experiments, we never obtained different results by such increase. Stability margin. The stability margin of a d-dimensional system is defined [2] as the maximum value σ for which A(z) 6= 0 for all z with |zi | < 1 + σ, i = 1 : d. For ρ > 0, let us define the polynomial ˜ A(z) = A(ρz), (24)
The solution of (21) and (23) was found using the SDP library SeDuMi [12]. We report here only the implementation and results of (23), the other SDP problem being quite similar. The programs were run on a PC computer with a Pentium processor at 800 MHz. The programs are available by e-mail request from the author. The most important implementation aspect is how to decide numerically that λ? 6= 0. Practically, the SDP program always returns a positive value λ? ; the point is to decide if the value is significantly large or not. The SDP algorithms implemented in [12] and similar libraries have a primal dual nature, in the sense that they solve the specified (primal) SDP problem and its dual; thus, they can return the gap η between the values of the primal and dual optimal (numerical) values. Theoretically, if both primal and dual are feasible, the gap should be zero. Practically, the gap is a measure of the accuracy in solving the SDP problem; in [12], the accuracy is a user parameter, which we chose to be 10−10 . Since in (23) the objective is λ, we should compare the values of λ? and of the gap η. The decision we propose is the following: if λ? > M η, where M is a constant (we took conservatively M = 100, but smaller values could also be used), then we decide that λ? 6= 0. If λ? ≤ M η, then we decide that λ? = 0. For experiments, we used mainly examples drawn from the previous literature on the subject of multidimensional system sta-
bility. On these examples we both tested stability and computed the margin of stability. For all examples, we have obtained results identical to those reported in the original papers. We discuss here only the following examples from [3, 4]: A1 (z) A2 (z) A3 (z)
= = =
5 + z2 + z4 + z1 z4 + z3 z4 , 5 + z12 z33 + z33 z42 + z13 z2 z5 + z1 z2 z3 z4 z5 , 5.6 + 0.8z1 + 1.5z12 z2 + 1.8z23 + 0.2z3 + 1.3z2 z32 .
The polynomial A1 (z) was found to be stable. The complete SDP problem (23), with Q of size 16 × 16, was solved in about 0.5 seconds. The same result was obtained using the Newton polytope heuristic, now with Q of size 5 × 5 (in this case, the base pp (z) contains exactly the monomials present in A1 (z)), in less than one second (the time is greater than in the complete problem due to a rather inefficient computation of the Newton polytope). A margin of stability σ = 0.1583 was computed in 13 seconds using the complete approach and in 10 seconds using the Newton polytope. (Note that an SDP problem is solved at each bisection step, while the Newton polytope has to be computed only once, in the beginning. This aspect makes the Newton polytope approach more efficient for stability margin calculation.) The polynomial A2 (z) was also found to be stable, with a margin of stability of σ = 0.0456 (the margin of stability is not computed in [4]). The Gram matrix of the complete SDP problem has size 192 × 192 and stability was decided in 325 seconds. In the Newton polytope approach, the necessary time was 15 seconds, from which less than half second for the SDP problem (while the rest for the computation of the Newton polytope), with Q of size 6 × 6. In the same approach, the margin of stability was computed in less than 25 seconds. The polynomial A3 (z) was found to be unstable. In the complete problem (size 36 × 36, execution time 2 seconds), we obtained λ? = 3.7 · 10−12 , while the gap was η = 2.0 · 10−12 . In the Newton polytope approach (size 13 × 13, execution time 2.8 seconds), we obtained λ? = 6.1 · 10−13 , while the gap was η = 1.0 · 10−12 . However, a small perturbation of A3 (z) may make it stable. If we modify it as in (24), with ρ = 0.999 (i.e. subunitary), we now obtain (with the complete problem) λ? = 6.1 · 10−6 and η = 3.5 · 10−13 . Moreover, for ρ = 0.9999, we obtain λ? = 6.1 · 10−8 and η = 1.2 · 10−12 . For both values of ρ, we decide that the polynomial is stable. (With the Newton polytope approach, results are similar. For example, with ρ = 0.9999, we obtained λ? = 1.7 · 10−7 and η = 2.2 · 10−15 .) From these and other example we appreciate that the proposed algorithm gives accurate stability decisions in convenient times. For comparison, we remark that in [4] the reported times for deciding the stability of A2 (z) or A3 (z) are of 1-3 minutes, on a faster computer. 5. CONCLUSIONS AND FURTHER WORK We have proposed an algorithm for deciding if the d-dimensional polynomial A(z) has no zeros inside the unit d-circle. The idea is to form the ”squared” polynomial (18) and to express it in Gram matrix form (7). Then, we compute the most positive definite of the Gram matrices associated with the polynomial, via the SDP problem (23). If the minimum eigenvalue of this matrix is positive, then the polynomial is surely stable. If it is zero, then A(z) is most probably unstable. Numerical evidence shows that in the experiments performed this test gives accurate results.
Further work will be directed towards studying the theoretical foundation of the Newton polytope heuristic for reducing the size of the SDP problem and also on speeding up its implementation. Acknowledgement. The author is indebted to Prof. Bruce Reznick for help in understanding the topology of the set of sumof-squares polynomials of real variables. 6. REFERENCES [1] M.G. Strintzis, “Tests of Stability of Multidimensional Filters,” IEEE Trans. Circ. Syst., vol. CAS-24, no. 8, pp. 432– 437, August 1977. [2] E. Walach and E. Zeheb, “N -Dimensional Stability Margins Computation and a Variable Transformation,” IEEE Trans. Acoustics, Speech, Signal Proc., vol. ASSP-30, no. 6, pp. 887–993, Dec. 1982. [3] E. Curtin and S. Saba, “Stability and Margin of Stability Tests for Multidimensional Filters,” IEEE Trans. Circ. Syst. I, vol. 46, no. 7, pp. 806–809, July 1999. [4] N.E. Mastorakis, I.F. Gonos, and M.N.S. Swamy, “Stability of Multidimensional Systems using Genetic Algorithms,” IEEE Trans. Circ. Syst. I, vol. 50, no. 7, pp. 962–965, July 2003. [5] N. Damera-Venkata, M. Venkataraman, M.S. Hrishikesh, and P.S. Reddy, “A New Transform for the Stabilization and Stability Testing of Multidimensional Recursive Digital Filters,” IEEE Trans. Circ. Syst. II, vol. 47, no. 9, pp. 965–968, Sept. 2000. [6] J.W. McLean and H.J. Woerdeman, “Spectral Factorizations and Sums of Squares Representations via Semidefinite Programming,” SIAM J. Matrix Anal. Appl., vol. 23, no. 3, pp. 646–655, 2001. [7] M.D. Choi, T.Y. Lam, and B. Reznick, “Sums of Squares of Real Polynomials,” Proc. Symp. Pure Math., vol. 58, no. 2, pp. 103–126, 1995. [8] J.B. Lasserre, “Global Optimization with Polynomials and the Problem of Moments,” SIAM J.Optim., vol. 11, no. 3, pp. 796–814, 2001. [9] B. Dumitrescu, I. T˘abus¸, and P. Stoica, “On the Parameterization of Positive Real Sequences and MA Parameter Estimation,” IEEE Trans. Signal Proc., vol. 49, no. 11, pp. 2630–2639, Nov. 2001. [10] A. Megretski, “Positivity of Trigonometric Polynomials,” in Conf. Decision Control (CDC, submitted), 2003, (http://web.mit.edu/ameg/www/images/sos cdc.ps). [11] M. Wang, E.B. Lee, and D. Boley, “A Simple Method to Determine the Stability and Margin of Stability of 2-D Recursive Filters,” IEEE Trans. Circ. Syst. I, vol. 39, no. 3, pp. 237–239, March 1992. [12] J.F. Sturm, “Using SeDuMi, a Matlab Toolbox for Optimization over Symmetric Cones,” Optimization Methods and Software, vol. 11-12, pp. 625–653, 1999, http://fewcal.kub.nl/sturm/software/sedumi.html.
