Convergence Criteria for Hierarchical Overlapping

Convergence Criteria for Hierarchical Overlapping Coordination Under Linear Constraints Hyungju A. Park Devadatta Kulkarni

Nestor Michelenay Panos Papalambrosy

Abstract

Decomposition of large engineering design problems into smaller design subproblems is desirable since decreased problem size enhances robustness and speed of numerical solution algorithms. The subproblems are then solved in parallel, using the optimization technique most suitable for the underlying subproblem. This re ects the typical multidisciplinary nature of system design problems and allows better interpretation of results. Hierarchical overlapping coordination (HOC) simultaneously uses two or more design problem decompositions, each of them associated with dierent partitions of the design variables and constraints. Coordination is achieved by the exchange of information between decompositions. This article presents the HOC algorithm and several sucient conditions for convergence of the algorithm to the optimum in case of convex problems with linear constraints. One of these equivalent conditions involves the rank of the constraint matrix and is computationally ecient to verify. Computational results obtained by applying the HOC algorithm to several quadratic programming problems of various sizes are also presented.  Department of Mathematics and Statistics, Oakland University, Rochester, MI 48309. e-mail: fpark, [email protected]. y Department of Mechanical Engineering and Applied Mechanics, University of Michigan, Ann Arbor, MI 48109. e-mail: fnestorm, [email protected]

1

Keywords: decomposition methods, large scale optimization, distributed com-

puting, hierarchical coordination

1 Introduction Engineering design can be viewed as a decision-making process that uses mathematical models to predict design behavior and to select a design whose value is considered satisfactory. A typical approach consists of formulating a design optimization problem using models to estimate design criteria and constraint functions, and applying formal methods to search the design space for an optimum. In this article, we assume that a design problem can be formulated as a convex optimization problem of the form: nd x 2 X such that h(x) = 0; g(x)  0 and f (x) is minimized; where X  R n is a nonempty open convex set, f : X ! R and gi : X ! R are convex functions on X , and hi : X ! R are ane functions on X . We assume that the problem above has a nonempty solution set, and that f and gi are dierentiable functions on X . In the case of large-scale nonlinear design problems that involve hundreds of variables and constraints, numerical optimization algorithms may not be relied upon and decomposition of the design problem into smaller design subproblems becomes necessary. The subproblems can then be solved in parallel, using the optimization technique most suitable for the underlying submodel, gaining in robustness, speed and interpretation of results. Moreover, system design problems typically involve several disciplines and subsystem design teams that eect an explicit problem decomposition. Thus, coordinated solution of design subproblems may be the only way to address the overall system problem in a practical and robust manner. Hierarchical overlapping coordination (HOC) uses two or more design problem decompositions, each of them associated with dierent partitions of the design variables and constraints. This kind of problem decomposition may re ect, for example, matrix-type organizations structured according to product lines and the disciplines involved in the design process. Coordination is achieved by the exchange of information between decompositions, as explained in Section 2.1. 2

The mathematical formulation of HOC was rst proposed in [4], and several criteria for convergence of the coordination algorithm under linear equality and inequality constraints were developed in [4] and [10]. Convergence criteria developed in those articles are computationally dicult to check and possibly incorrect (see Remark 4.4). In this article, we present computationally ecient conditions that ensure the convergence of overlapping coordination under linear equality and inequality constraints.

2 HOC Under Linear Equality Constraints In the case of linear equality constraints only, the original optimization problem can be restated in the following form: Minx f (x) subject to Ax = c; (2.1) where f : R n ! R is convex and dierentiable, A is an m  n constraint matrix with real entries, x 2 R n is the vector of optimization variables, and c 2 Rm is a constant vector. We assume that the above problem has a nonempty solution set. Model-based decomposition methods [6, 7] can be applied to Problem 2.1 to obtain two or more distinct (say, -, -, -, . . . ) decompositions. This involves generating a block-angular matrix A by reordering the columns and rows of A, and components of x and c, as shown in Figure 1. In Figure 1, x is the vector of reordered design variables for the decomposition, and c is the reordered vector c; y is the vector of linking variables for the -decomposition, and n is the number of the linking variables. The linking variables for the -decomposition will be referred to as -linking variables; p denotes the number of subproblems in the decomposition (diagonal blocks in the gure). More explicitly, 0 x(1) 1 0 c1 1 y := @ ... A and c := @ ... A : cp x(n ) xi is the vector of local variables associated with block Ai , i.e., with subproblem i for i = 1; 2; : : : ; p . Although the results that follow can be generalized to three or more decompositions, we will consider only two problem decompositions ( and ) to simplify notations and proofs. 3

x = c

A

y x1

B1 A1 B2

A2

...

c1 c2

... = ...

...

Bp

Ap

xp

cp

Figure 1: Block-decomposition of constraint matrix A, design vector x, and constant vector c. Under the assumption that the objective function f is -additive separable, Problem 2.1 takes the following form: Minx f0 (y) +

p X i=1

fi (y; xi )

subject to Bi y + Ai xi = ci ; i = 1; : : : ; p: (2.2) For a given vector d 2 Rn , xing the -linking variables y = d in (2.2) results in the following Problem : Problem  : For each i = 1; : : : ; p; (2.3) Minxi fi (d; xi ) subject to Ai xi = ci ? Bi d: Problem  can be solved by solving p independent uncoupled subproblems. Similarly, Problem can be de ned and solved for a -decomposition after xing the -linking variables.

2.1 Generic HOC Algorithm

The generic hierarchical overlapping coordination algorithm can be described as following, for the case of two decompositions ( and ): 4

Step 1. Fix linking variables y, and solve Problem  by solving the p

independent subproblems given in (2.3). Step 2. Fix linking variables y to their values determined in Step 1, and solve Problem by solving p independent subproblems. Step 3. Go to Step 1 with the xed values of -linking variables determined in Step 2. Step 4. Repeat these steps until convergence is achieved.

Remark 2.1 The accumulation point achieved in Step 4 is not necessar-

ily an optimal solution of Problem 2.1. A sucient condition guaranteeing the convergence to a solution of the original optimization problem will be developed in the following sections.

2.2 Optimality Conditions

The Lagrange multiplier theorem for linear equality constraints [1, Proposition 3.4.1] states that x 2 R n is a solution to Problem 2.1 if and only if there exists a vector  2 R m such that

rf t(x ) + At = 0:

(2.4)

In contrast to the case of nonlinear constraints, this optimality condition is valid even without the regularity assumption on x . This is a consequence of Farkas' Lemma for polyhedral sets [1, page 292]. Condition 2.4 is equivalent to

rf t(x ) = ?At ; which can be rephrased as \rf t (x) belongs to the row space RS(A)." Let ei 2 R n be the i-th standard row vector whose i-th component is one and all other components are zero. Once an -decomposition and a -decomposition are given, de ne two matrices H and H by 5

0 e (1) 1 0 e(1) 1 B e(2) CC ; H := BB e (2) CC : H := B @ ... A @ ... A

(2.5)

e (n )

e(n)

These are unique n  n and n  n matrices having ones and zeros as their entries such that Hx = y ; H x = y : De ne K, K and K as follows: 0A1 A A K := H ; K := H ; K := @ H A :  H

Problem , with xed values for the -linking variables y = d , can be de ned as Minx f (x) subject to Ax = c and H x = d : (2.6) One notes that x 2 R n is a solution to Problem  if and only if there exists vectors  2 R m and  2 R n() such that rf t(x ) + At  + Ht  = 0: (2.7) This optimality condition can be rephrased as \rf t(x) belongs to the row space RS(K)."

Analogously, x is a solution to Problem if and only if

\ rf t(x ) belongs to the row space RS(K )."

2.3 Properties of HOC

The following properties of HOC were observed in [4], and can be proved in a straightforward manner: 1. If the HOC algorithm is started with a feasible point x0, then at each stage of the process, problem  and problem will have nonempty feasible domains. 6

1 2. If the sequences fxi g1 i=1 and fx i gi=1 result from solving problem  min and problem , respectively, and f := minff (x) j Ax = cg, then (a) f (xi )  f (x i )  f (x(i+1) ) (b) limi!1 f (xi ) = limi!1 f (x i ) = f   f min 1 3. Any accumulation point x of either fxi g1 i=1 or fx i gi=1 solves both problem  and problem .

3 Conditions for Convergence Under Linear Equality Constraints Once - and -decompositions of the optimization problem in (2.1) are ob1 tained, let fxi g1 i=1 and fx i gi=1 be the sequences obtained by applying the generic HOC algorithm to these decompositions as described in Section 2.1. Theorem 3.1 below gives a sucient condition for these sequences to converge to a minimum of Problem 2.1 in terms of the row spaces RS(A), RS(K) and RS(K ). Theorem 3.1 Let x be an accumulation point of fxi g1i=1 or fx i g1i=1. If RS(A) = RS(K) \ RS(K ); then x is a solution to the optimization problem in (2.1). Proof: By Property 3 of Section 2.3, x solves both Problem  and Problem . Therefore, rf t(x ) 2 RS(K ) and rf t(x ) 2 RS(K ): Since RS(A) = RS(K) \ RS(K ), one gets rf t (x) 2 RS(A), which implies x is a solution to the original optimization problem. 2 Although Theorem 3.1 oers a conceptually clear sucient condition for the convergence of HOC, it involves the algorithmic process of computing the intersection of the two vector spaces RS(K) and RS(K ). The computational cost associated with this process can be fairly high. As an attempt to obtain a computationally ecient HOC convergence condition, we prove in Theorem 3.2 that a certain matrix rank condition implies the convergence condition of Theorem 3.1. 7

Theorem 3.2 Let r be the rank of A and A^ be an r  n submatrix of A with

0 A^ 1 K^  := @ H A

full rank. If the matrix

H has full rank, then RS(A) = RS(K ) \ RS(K ):

Proof: Clearly, RS(A)  RS(K) and RS(A)  RS(K ). Therefore, RS(A)  RS(K) \ RS(K ): To show the reverse inclusion, choose an arbitrary v 2 RS(K) \ RS(K ). Let v1; : : : ; vr be the row vectors of A^, and ei 2 R n be the i-th standard row vector. The full rank condition on K^  = (v1 ; : : : ; vr ; e(1) ; : : : ; e(n) ; e (1) ; : : : ; e (n ))t

implies that fv1 ; : : : ; vr ; e(1) ; : : : ; e(n ) g and fv1; : : : ; vr ; e (1) ; : : : ; e (n ) g are bases for RS(K ) and RS(K ), respectively.

v 2 RS(K ) =) v = v 2 RS(K ) =) v = Therefore,

r X i=1

r X i=1 r

X i=1

a i vi + b i vi +

(ai ? bi )vi +

n X i=1

n X i=1 n

X i=1

sie(i) for some ai 's and si's in R . tie (i) for some bi 's and ti's in R .

sie(i) ?

n X i=1

ti e (i) = 0:

Since v1; : : : ; vr ; e(1) ; : : : ; e(n ); e (1) ; : : : ; e (n ) are linearly independent, one concludes

ai = bi ; sj = 0; tk = 0; for all i = 1; : : : ; r; j = 1; : : : ; n; k = 1; : : : ; n : This means v =

Pr a v 2 RS(A). i=1 i i

2 8

Corollary 3.3 Same notations as in Theorem 3.2. For Problem 2.1, sup1 pose - and -decompositions are given. Let fxi g1 i=1 and fx i gi=1 be two sequences obtained by applying HOC to these decompositions, and x be an 1 ^  has full rank, accumulation point of fxi g1 i=1 or fx i gi=1 . If the matrix K  then x is a solution to the optimization problem in (2.1). Remark 3.4 The convergence condition given in Corollary 3.3 oers a very

ecient criterion for the convergence of hierarchical overlapping coordination in terms of rank(K^  ). A model partitioning algorithm as in [7] should just check if rank(K^  ) is equal to r + n + n . The following theorem oers several interpretations of the HOC convergence condition given in Corollary 3.3.

Theorem 3.5 With the same notations as in Theorem 3.2, the following three conditions are equivalent. 1. The matrix K^  has full rank.

2. There exists no (nontrivial) linear relation exclusively among the and -linking variables. 3. The set of -linking variables and the set of -linking variables are disjoint, and the matrix A^ obtained from A^ by deleting columns corresponding to the - and -linking variables has full rank.

Proof: (1) () (2): From Ax = c, one can nd the unique vector c^ = (^c1 ; : : : ; c^r )t such that A^x = ^c. Let v1 ; : : : ; vr be the row vectors of A^ and ei 2 R n be the i-th standard row vector. Note that

vi x = c^i; eix = xi: K^  is rank-de cient. () There exists a nontrivial linear relation among the row vectors of K^  : () There exists a nontrivialn linear relation r n X X X aivi + sie(i) + tie (i) = 0: i=1

i=1

9

i=1

() There exists a nontrivial linear relation n r n X X X aivi x + si e(i) x + tie (i) x = 0: i=1 i=1 i=1 () There exists a nontrivialn linear relation r X i=1

aic^i +

n X i=1

six(i) +

X i=1

tix (i) = 0:

() There exists a nontrivial linear relation

exclusively among - and -linking variables:

H 

(1) =) (3): Since H has full rank, the sets f(1); : : : ; (n)g and f (1); : : : ; (n)g are disjoint. By performing appropriate elementary row operations on K^  , one easily concludes that A^ has full rank. (3) =) (1): Straightforward.

2

Theorem 3.6 below shows that, under a certain additional condition, the two conditions of Theorem 3.2 are actually equivalent.  A^   A^  Theorem 3.6 Suppose that K^  := H and K^ := H have full rank. 0 A^ 1 Then, RS(A) = RS(K) \ RS(K ) if and only if the matrix K^  := @ H A H has full rank.

Proof: ((=): Shown in Theorem 3.2. (=)): Suppose K^  is rank-de cient. Let v1; : : : ; vr be the row vectors of A^, and ei 2 R n be the i-th standard row vector. Since K^  = (v1 ; : : : ; vr ; e(1) ; : : : ; e(n) ; e (1) ; : : : ; e (n ))t is rank-de cient, there exists a nontrivial linear relation among its row vectors: r X i=1

ai vi +

n X i=1

sie(i) + 10

n X i=1

tie (i) = 0;

(3.1)

where the coecients are not identically zero. In this expression, the rst two sums belong to RS(K) while the third sum belongs to RS(K ). Therefore, r X i=1

aivi +

n X i=1

sie(i) = ?

n X i=1

tie (i) 2 RS(K) \ RS(K ) = RS(A):

Since vi's and e(j)'s form a basis for RS(K ) = RS(K^ ), an arbitrary element of RS(K) has a unique expression as a linear combination of these basis vectors. In particular, an element of RS(A)P RS(K ) is expressed only in P r  tie(i) 2 RS(A), we deduce terms of vi 's. Therefore, from i=1 sivi + ni=1 that s1 =


= sn = 0: Thus, (3.1) becomes r X i=1

ai vi +

n X i=1

tie (i) = 0;

(3.2)

where the coecients are not identically zero. This contradicts the full rank condition on K^ , and thus, K^  has to be a full rank matrix. 2 Since the row space RS(B ) of a matrix B can be viewed as the orthogonal complement of its null space NS(B ), the HOC convergence condition of Theorem 3.1 given in terms of the row spaces of A, K and K can be rephrased in terms of their null spaces. For any subspace W of an inner product space V , we denote the orthogonal complement of W by W ?. We need the following lemma. Lemma 3.7 Let W1 and W2 be subspaces of a nite dimensional inner product space. Then (W1 + W2)? = W1? \ W2? (W1 \ W2)? = W1? + W2? Proof: Exercise 11 in [2, page 313]. 2 Theorem 3.8 Let A, K and K be the matrices de ned by two decompositions of the optimization problem in (2.1). Then RS(A) = RS(K ) \ RS(K ) if and only if NS(A) = NS(K) + NS(K ): 11

Proof: For an arbitrary s  t matrix B , identify RS(B ) and NS(B ) as sub-

spaces of R t , and identify a t-dimensional row vector with a t-dimensional column vector. Just note that RS(B )? = NS(B ); and apply Lemma 3.7. 2 In [4] and [10], the null space condition NS(A) = NS(K ) + NS(K ) was developed as a sucient condition for convergence of HOC. A computational procedure to check the convergence of HOC based on this condition will have to compute the sum of two vector spaces, N(K) and N(K ), which is an expensive computational process. Also, this condition is sometimes dicult to work with. For instance, the following apparently incorrect statement appears in [10]: [10, Property 4] If the decision variables corresponding to the interaction (i.e., linking) variables y and y are bounded by common equations, then NS(A) 6= NS(K) + NS(K ): Example 2 in [10] was constructed speci cally to demonstrate the above Property 4. The constraints in this example are x1 + 3x2 + 2x3 = c1 x2 + 2x3 + x4 = c2 x4 + x5 + x6 + 2x7 = c3 x5 + 3x6 + x7 = c4 x6 + x8 + x9 + x10 = c5 x9 + x10 = c6; and the linking variables are y = x6 and y = x4 . Based on the observation that the third constraint equation contains both y and y , the article [10] claims that this example satis es the hypothesis of Property 4 and therefore NS(A) 6= NS(K) + NS(K ). It also presents a computation that results in the erroneous conclusion that NS(A) and NS(K)+NS(K ) are dierent. An explicit computation using Maple [8] actually shows that NS(A) = NS(K )+ NS(K ), and that indeed this example disproves [10, Property 4]. The above example demonstrates how dicult it can be to check the convergence criterion NS(A) = NS(K) + NS(K ) in actual computation. 12

4 HOC Under Mixed Linear Constraints In this section, we extend the results of the preceding sections to the general case of HOC under mixed linear equality and inequality constraints: Minx f (x) subject to AI x  cI and AE x = cE

(4.1)

where f : R n ! R is convex and dierentiable, AI (AE , resp.) is an mI  n (mE  n, resp.) constraint matrix with real entries, x 2 R n is the vector of optimization variables, and cI 2 R mI (cE 2 R mE , resp.) is a constant vector. We assume that the problem has a nonempty solution set. The HOC algorithm described in Section 2.1 applied to Problem 4.1 re1  sults in two sequences fxi g1 i=1 and fx i gi=1 . For an accumulation point x 1 of fxi g1 i=1 or fx i gi=1 , de ne Ja to be the set of the indices corresponding to the active inequality constraints, i.e.,

Ja := fi j (aIi1 ; : : : ; aIin)x = cIi g;

where aIij denotes the (i; j )-entry of the matrix AI . Let AI be the submatrix of AI consisting of the active inequality constraints. De ne the cone C(A) by C(A) := fx j x =

X

i2Ja

aiviI

+

mE X i=1

biviE ; ai  0g;

(4.2)

where viI (viE , resp.) denotes the i-th row vector of AI (AE , resp.). Also, de ne the induced cones C(K ) and C(K ) as follows: C(K ) := fx j x = C(K ) := fx j x =

X

i2Ja

X

i2Ja

ai viI + ai viI +

mE X i=1 mE

X i=1

bi viE + bi viE +

n X i=1 n

X i=1

sie(i) ; ai  0g; tie (i) ; ai  0g:

The Lagrange multiplier theorem for linear constraints [1, Proposition 3.4.1] states that x 2 R n is a solution to Problem 4.1 if and only if there exists a nonnegative vector I  0 and a vector E such that rf t (x) + AI t I + AE tE = 0: (4.3) 13

As in the case of equality constraints, this result is valid even when x is not regular [1, page 292]. Condition 4.3 is equivalent to (4.4) ?rf t (x) = AI t I + AE tE ; I  0; which can be rephrased as \?rf t (x) belongs to the cone C(A)." For xed values of the -linking variables y = d , Problem  can be de ned as Minx f (x) subject to AI x  cI ; AE x = cE and Hx = d: (4.5) Using the above reasoning, one sees that x is a solution to Problem  if and only if \?rf t (x) belongs to the cone C(K )." Analogously, x is a solution to Problem if and only if ?rf t (x ) belongs to the cone C(K ). The following theorem oers an analogue of Theorem 3.1. Theorem 4.1 Let x be an accumulation point of fxi g1i=1 or fx i g1i=1. If C(A) = C(K ) \ C(K ), then x is a solution to the optimization problem in (4.1). Proof: By Property 3 of Section 2.3, x solves both Problem  and Problem . Therefore, ?rf t (x) 2 C(K ) and ? rf t(x) 2 C(K ): Since C(A) = C(K) \ C(K ), one gets ?rf t (x ) 2 C(A), which implies x is a solution to the original optimization problem. 2 The HOC convergence condition stated in Theorem 4.1 cannot be practically used because one has to know a priori the accumulation point x and the set Ja of active constraints in order to compute the cones C(A), C(K ) and C(K ). As an analogue of Theorem 3.2, Theorem 4.2 below xes this problem and provides a new sucient condition for the convergence of HOC. This condition does not rely on the accumulation point x. 14

 AI 

Theorem 4.2 Let r be the rank of AE and A^ be an r  n submatrix of   AI with full rank. If the matrix AE

0 A^ 1 K^  := @ H A H

has full rank, then C(A) = C(K ) \ C(K ).

Proof: Clearly, C(A)  C(K) and C(A)  C(K ). Therefore, C(A)  C(K ) \ C(K ). To show the reverse inclusion, choose an arbitrary v 2 C(K ) \ C(K ). Let v1; : : : ; vr be the row vectors of A^, and ei 2 R n be the i-th standard row vector. Since

v 2 C(K ) =) v = v 2 C(K ) =) v = we have

X

i2Ja

(ai ? di)viI +

X i2Ja

X

i2Ja

aiviI + diviI +

mE X i=1 mE

X i=1

mE X

n X

i=1

i=1

(bi ? ei)viE +

biviE + ei viE +

n X i=1 n

X i=1

sie(i) ?

si e(i) ; ai  0; tie (i) ; di  0;

n X i=1

tie (i) = 0:

 AI 

(4.6)

Since v1; : : : ; vr form a basis for the row space of AE , there exist 1; : : : ; r 2 R such that mE r X X X I E (ai ? di)vi + (bi ? ei)vi = ivi : i2Ja

i=1

Therefore, (4.6) becomes r X i=1

ivi +

n X i=1

i=1

sie(i) ? 15

n X i=1

tie (i) = 0:

Since

K^  = (v1 ; : : : ; vr ; e(1) ; : : : ; e(n) ; e (1) ; : : : ; e (n ))t has full rank, v1; : : : ; vr ; e(1) ; : : : ; e(n ) ; e (1) ; : : : ; e (n ) are linearly independent. Therefore,

i = 0; sj = 0; tk = 0; for all i = 1; : : : ; r; j = 1; : : : ; n; k = 1; : : : ; n ; and thus

v=

X i2Ja

ai viI

This implies that v 2 C(A).

+

mE X i=1

bi viE ; ai  0:

2

Theorem 4.2 combined with Theorem 4.1 immediately implies the following Corollary.

Corollary 4.3 Same notations as in Theorem 4.2. For Problem 4.1, sup1 pose - and -decompositions are given. Let fxi g1 i=1 and fx i gi=1 be two sequences obtained by applying HOC to these decompositions, and x be an 1 ^  has full rank, then x is accumulation point of fxi g1 i=1 or fx i gi=1 . If K a solution to the optimization problem in (4.1).

Remark 4.4 In [10, Property 13], an HOC convergence condition under

inequality constraints was given in terms of null spaces, which does not rely on the accumulation point x , either. Suppose that - and -decompositions are given for the problem Minx f (x) subject to Ax  c; in which A is an m  n matrix. De ne matrices A~, K~  and K~ by







(4.7)



A~ := (A Im ); K~  := HA I0m ; K~ := HA I0m :  It was claimed in [10] that the condition NS(A~) = NS(K~  ) + NS(K~ ) guarantees the appropriate convergence of HOC for the problem in (4.7). 16

However, this assertion does not seem to be correct. First, note that the three matrices A~, K~  and K~ have full rank, and due to Theorem 3.6, the above null space condition is equivalent to the full rank condition on the matrix 0A I 1 @ H 0m A : H 0 H  This matrix has full rank if and only if H has full rank, which is true if and only if the sets of - and -linking variables are disjoint. However, the disjointness of - and -linking variables is not enough to guarantee the convergence of HOC.

5 Computational Results

5.1 Obtaining two distinct decompositions

For a given optimization problem, a matrix called Functional Dependence Table (FDT) can be constructed as a Boolean matrix representing the dependence of design constraint functions on variables. The (i; j )-th entry of the FDT is one if the i-th constraint depends on the j -th variable and zero otherwise. A decomposition of the given optimization problem can be achieved by reordering rows and columns of the FDT corresponding to the constraints and variables, respectively. To solve an optimization problem P by the HOC algorithm, two distinct (-, -) decompositions of P satisfying the sucient condition for convergence of HOC (Corollary 4.3) can be found by the following heuristic: 1. Apply the hypergraph-based model decomposition algorithm developed in [7]1 to problem P to obtain an -decomposition. 2. In the process of obtaining a -decomposition, penalize2 the -linking variables so that the disjointness of the set of -linking variables and An implementation of this decomposition algorithm can be found at the URL: http://arc.engin.umich.edu/graph part.html. 2 A variable is penalized when it is not desirable to have the variable as a linking variable. This can be achieved by assigning a high weight to the corresponding hyperedge in the hypergraph-based model decomposition algorithm described in [7]. 1

17

the set of -linking variables is accomplished, as required by the convergence condition (see part (3) of Theorem 3.5). If the two sets of linking variables are not disjoint, then go back to Step 1 and obtain a new -decomposition after penalizing the common linking variables. 3. Check if the resulting - and -decompositions satisfy the convergence condition in Corollary 4.3. If the convergence condition is not satis ed, then go back to Step 1 and obtain a new -decomposition after penalizing one of the interdependent linking variables (see part (2) of Theorem 3.5).

5.2 Example Problems

The HOC algorithm developed in this paper has been applied to a family of QP problems of various sizes. The smallest QP problem P1 has 25 variables and 21 linear constraints (19 equalities and 2 inequalities) with a strictly convex, additive separable objective function. (Thus, the FDT of problem P1 is a 21  25 table.) The largest QP problem P9 has 500 variables and 420 linear constraints (380 equalities and 40 inequalities). Figure 2 shows the reordered FDTs for the - and -decompositions obtained by applying the above decomposition process to the example problem P1. Maple [8] was used to verify that these two decompositions do satisfy the convergence condition in Corollary 4.3. The -decomposition in Figure 2 has two subproblems and one linking variable (x13 ), whereas the -decomposition has two subproblems and two linking variables (x3 and x9 ). Once the - and -decompositions are determined, the QP subproblems have to be repeatedly solved. QPOPT, the QP solver developed by Gill et al. [3], was used for this purpose. The MATLAB [5] program implementing the HOC algorithm calls a QPOPT MEX function to solve the QP subproblems. To compare the eectiveness of the HOC algorithm with the ordinary AllAt-Once (AAO) algorithm (i.e. one not using decompositions), the problems were solved in both ways. The results for P1 and for the other QP problems of larger sizes are shown in Table 1. Runtime was measured in CPU seconds on a stand-alone Sun UltraSparc 1. Runtimes only include QPOPT function calls, excluding I/O and data transfer between - and -decompositions.

18

DESIGN

VARIABLES

DESIGN

1000000000111111111222222 3123456789012456789012345

D E S I G N R E L A T I O N S

1 2 3 4 5 6 7 8 9 10 11 12 20 21 13 14 15 16 17 18 19

VARIABLES

0001111000000111111222222 3960268124578134579012345

.%%%%%................... ...%..%.................. .%.....%%................ ...%.....%%.............. .......%...%............. ..........%.%............ *%%%..................... *%.........%.%........... .......%......%.......... ..........%....%......... .%.....%......%.%........ ...%......%....%.%....... .....%..%%............... ...%........%............ *.................@...... ..................@@..... *...................@.... ...................@@@... ..................@...@.. ..................@....@. ..................@.....@

2 4 6 10 12 21 1 3 5 7 8 9 11 13 14 15 16 17 18 19 20

(a)

*.%...................... **.%..................... ...%%.................... ...%.%................... *..%.%%.................. *...%.................... *......@@@@.............. .......@...@@............ ...........@.@........... *......@@.....@.......... .......@.....@@@......... ...........@....@........ .......@...@....@@....... ..............@...@...... ..................@@..... ..............@.....@.... ...................@@@... ..................@...@.. ..................@....@. ..................@.....@ .*........@.@............ (b)

Figure 2: Decompositions of Example Problem P1: (a) -decomposition and (b) -decomposition

5.3 Discussion

The implementation of HOC used to solve the nine QP problems shown in Table 1 is set up so that the HOC iteration process stops if the dierence between the values of the objective function for two consecutive iterations is less than a preset tolerance value. The tolerance value used for the computation was 10?5. Table 1 shows that the algorithm terminates after four iterations in all cases. Each runtime shown in Table 1 represents the average of runtimes measured for ve separate runs of the algorithm; the times of the ve runs were consistently close. Serial-runtime is measured for the HOC computation in which the subproblems are solved sequentially, whereas parallel-runtime is measured for the HOC computation in which the subproblems are simulated to be solved in parallel. HOC has lower parallel-runtimes than the ordinary AAO algorithm except for problem P1. HOC has lower serial-runtimes than the AAO algorithm except for problems P1 and P2. Each of the two decompositions for P1 has two subproblems. Each of the two decompositions for P9 has 40 subproblems. Note that HOC for P9 has parallel- and serial-runtimes that are 840 and 45 times faster than the AAO runtime, respectively. This clearly demon19

prob no. no. no. var constr subpr P1 25 21 2 P2 50 42 4 P3 75 63 6 P4 100 84 8 P5 125 105 10 P6 200 168 16 P7 250 210 20 P8 375 315 30 P9 500 420 40

All-At-Once

Hierarchical Overlapping Coordination

Objective Runtimey Objective 165.52777 331.05554 496.58329 662.11103 827.63875 1324.2218 1655.2772 2482.9157 3310.5590

0.026 0.156 0.453 0.980 1.810 7.036 13.66 44.12 103.2

165.52777 331.05555 496.58333 662.11111 827.63889 1324.2222 1655.2777 2482.9166 3310.5555

yRuntime is measured in CPU seconds on a Sun UltraSparc 1.

Serial- Parallelno. Runtimey Runtimey iterations 0.123 0.096 4 0.246 0.100 4 0.340 0.110 4 0.443 0.110 4 0.560 0.103 4 0.896 0.123 4 1.163 0.126 4 1.716 0.113 4 2.260 0.123 4

Table 1: Comparison of CPU-Runtimes for QP problems of various sizes strates the advantage of using HOC for large scale problems with many loosely-linked subproblems.

Acknowledgment This research was supported by the U.S. Army Automotive Research Center for Modeling and Simulation of Ground Vehicle under contract No. DAAE0794-C-R094, and by the Oakland University Foundation. This support is gratefully acknowledged. The authors wish to thank Mr. Hyung Min Kim for his assistance with the MATLAB implementation of the HOC algorithm.

References [1] D. Bertsekas. Nonlinear Programming. Athena Scienti c, Belmont, Massachusetts, 1995. [2] S. Friedberg, A. Insel, and L. Spence. Linear Algebra. Prentice Hall, 1989. [3] P. Gill, W. Murray, and M. Saunders. User's Guide for QPOPT 1.0: A FORTRAN package for quadratic programming, August 1995. [4] D. Macko and Y. Haimes. Overlapping coordination of hierarchical structures. IEEE Transactions on Systems, Man, and Cybernetics, SMC-8:745{751, 1978. 20

[5] The Math Works, Inc., Natick. MA. Using MATLAB, version 5.1 edition, 1997. [6] N. Michelena and P. Papalambros. Optimal model-based decomposition of powertrain system design. Transactions of the ASME, Journal of Mechanical Design, 117(4):499{505, 1995. [7] N. Michelena and P. Papalambros. A hypergraph framework to optimal model-based decomposition of design problems. Computational Optimization and Applications, 8(2):173{196, Sep. 1997. [8] R. Nicolaides and N. Walkington. Maple: A Comprehensive Introduction. Cambridge University Press, New York, 1996. [9] T. Shima. Hierarchical overlapping coordination of large scale systems with multiple objectives. PhD thesis, Case Western Reserve University, 1982. [10] T. Shima and Y. Haimes. The convergence properties of hierarchical overlapping coordination. IEEE Transactions on Systems, Man, and Cybernetics, SMC-14:74{87, Feb. 1984.

21