The Dual Projective Pivot Algorithm for Linear ...

1

The Dual Projective Pivot Algorithm for Linear Programming THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

Ping-Qi Pan Department of Mathematics, Southeast University Abstract. Recently a class of algorithms, based on “deficient basis”, were proposed for solving linear programming problems. Both projectively and pivotally featured, these algorithms performed very satisfactorily with dense problems. This paper presents a sparse implementation of a revised version of the dual algorithm, supported by a dual Phase-1 procedure using an auxiliary row. Reported are computational results with a set of 49 NETLIB standard test problems as well as a set of 10 larger sparse problems with as many as tens of thousands of rows and columns. These results show the unambiguously superiority of the dual projective pivot algorithm to the standard simplex algorithm. Keywords: large-scale linear programming, dual projective pivot algorithm, deficient basis, auxiliary row

1. Introduction We are concerned with the linear programming (LP) problem in the standard form min cT x x ≥ 0.

s.t. Ax = b,

(1.1)

where A ∈ Rm×n with m < n, and b ∈ Rm , c ∈ Rn . Note that nothing is made on the rank of A, except for 1 ≤ rank(A) ≤ m. We shall denote the jth column of A by aj , and the jth component of a vector • by •j . In addition, ∥ • ∥ designates the 2-norm of a vector •, and ei the unit vector with the ith component 1. Basically, there have been two major trends of developments of the art since the emergence of the simplex methodology for solving LP problems in the later forties. In order to reduce the number of iterations required, on one side, great effort has been made to improve pivot criteria used (e.g., Harris [16]; Goldfarb and Reid [13]; Zionts [39]; Terlaky [37]; Chen, Pardalos and Saunders [7]; and Pan [23-30]; for a survey, see Terlaky and Zhang [38]). On the other hand, every endeavor has also been made to reduce computational work involved per iteration (e.g., Dantzig and Orchard-Hayes [9]; Bartels and G.H. Golub [2]; Hellerman and Rarick [19]; and Forrest and Tomlin [10] ). A number of simplex variants have been proposed, and frequently used by practitioners in such areas as business, economics, and etc.(See, e.g., Baumol [3] and Heesterman [18]) Recently, Pan proposes a “dual projective simplex method,” attempting to reduce the number of iterations as well as computational work per iteration [31]. Numerical results 1 Project

19971014 supported by National Natural Science Foundation of China Typeset by AMS-TEX

1

2

PING-QI PAN

demonstrates its promising of success with dense LP problems. Based on QR decomposition, however, the method is not a very suitable choice for solving large and sparse problems. In this paper, we show that it is possible to develop an LU-decompositionbased version of such type, as is important, and deserves a full treatment indeed. Its tableau form produces, in a fresh manner, a sequence of m × (n + 1) tableaus until an optimal one is reached. Its revised form involves less computational work per iteration: only a single s × s system (s ≤ m) is solved, compared with the two m × m systems in the revised primal or dual simplex method. A key feature of the proposed method is the use of deficient basis, one that has fewer columns than rows. It is such doing that could make high primal degeneracy no longer be a nuisance but a favorite. In the next section, we first extend the basis to the deficient case. Then, in Section 3, we develop the proposed approach in a “standard” tableau form. In Section 4, we recast it into a “revised” version. Section 5 addresses the Phase-1 issue of how to get started. Section 6 highlights a crash heuristic for providing a “good” initial basis. Finally, in Section 7, we report numerical results, giving an insight into the behavior of the proposed approach. 2. Generalization of Basis This topic is closely related to a difficult and long-standing problem: the degeneracy, in which case one or more basic variables bear value zero, and consequently a zero-length step could be taken. Adverse effects of this phenomenon were noted as early as nearly at the emergence of the simplex method (See, for example, Charnes [6], Hoffman [20] and Beale [4]). Theoretically, it undermines the applicability of finiteness theorems for almost all popular algorithms. Practically, it often causes stalling for too long a time (although rarely lead to cycling), and greatly degrades algorithm’s performance. While authors make the non-degeneracy assumption to make a detour, degeneracy occurs all the time in practice, unfortunately. Because of involving a great mass of zero-length steps in their solution process, in fact, many problems from practice are thought “difficult” to be solved. In this section, we shall attack the following question: What about the root cause of the trouble, and how to get rid of it? Let us examine the most fundamental concept in the simplex methodology: the basis. Conventionally, basis is restricted to being a square nonsingular submatrix from the coefficient matrix, of order exactly equal to the number, m, of rows of the coefficient matrix. This definition unnaturally imposes an extra full-row-rank precondition on the coefficient matrix. Since a basis has m columns, more seriously, there must be some basic variables bearing value zero, whenever the right-hand side b belongs to a subspace, spanned by some proper subset of the basis’ columns. In other words, the degeneracy comes from rigidly fixing the number of basis’ columns on m. A direct way of remedying this difficulty is therefore to remove those basic columns that do not belong to the proper subset–such action will eliminate degeneracy completely, if the subset is a smallest one of such kind. The above idea leads to the following generalization of basis, which applies to any underdetermined systems: Definition 2.1. A basis is submatrix consisting of any linearly independent set of the coefficient matrix’s columns, whose range space includes the right hand side. Accordingly, the submatrix consisting of remaining columns is termed nonbasis. Other associated items should be defined, as in the conventional case. For example, variables cor-

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

3

responding to columns of the basis are called basic, while those corresponding to columns of the nonbasis called nonbasic; the system’s solution, obtained by setting all nonbasic variables to zero, is called the basic solution, corresponding to the basis, and etc. According to the preceding definition, the number of a basis is no longer necessarily equal to m, and hence all bases now fall into two categories: Definition 2.2. If the number of basis’ columns equals the number of rows of the coefficient matrix, it is a full basis; else, it is a deficient basis. Clearly, the simplex method and existing variants of it merely use full bases. Let B be a basis with s(≤ m) columns and let N be nonbasis, consisting of the remaining n − s columns. Define the ordered basic and nonbasic (index) sets respectively by JB = {j1 , . . . , js }, and JN = {k1 , . . . , kn−s }, (2.1) where ji , ∀i = 1, . . . , s, is the index of the ith column of B, and kj , ∀j = 1, . . . , n − s, the index of the jth column of N . The subscript of a basic index ji is termed row index, and that of a nonbasic index kj column index. Corresponding components of vectors and columns of matrices will be called basic and nonbasic, respectively. For simplicity of exposition, hereafter these components and columns will always be rearranged, and partitioned conformably, as the ordered set {JB , JN } changes in the solution process. For example, we have A = [B, N ] = [aj1 , . . . , ajs ; ak1 , . . . , akn−s ] cT = [cTB , cTN ] = [cj1 , . . . , cjs ; ck1 , . . . , ckn−s ], xT = [xTB ; xTN ] = [xj1 , . . . , xjs ; xk1 , . . . , xkn−s ]. So, program (1.1) can be written min cTB xB + cTN xN

(2.2a)

s.t. BxB + N xN = b xB ≥ 0,

xN ≥ 0.

(2.2b) (2.2c)

The associated dual program is then max bT y [ ] [ ] [ T] zB cB B s.t. y+ = NT zN cN zB ,

zN ≥ 0

(2.3a) (2.3b) (2.3c)

If the basis is full, then a pair of complementary primal and dual basic solutions can be obtained from (2.2b) and (2.3b), respectively, that is, x ¯N = 0

and

x ¯B = B −1 b

(2.4)

and y¯ = B −T cB z¯N = cN − N T y¯ and

(2.5a) z¯B = 0

(2.5b)

When the basis is deficient, however, the preceding formulas, as well as conventional simplex steps will no longer be well-defined. Therefore, new methods have to be developed to handle the generalized basis, just introduced.

4

PING-QI PAN

3. Main Procedure: Tableau Form We call the proposed method dual, since it is an analogy with the dual simplex method: under a rule similar to the conventional dual pivot criteria, it proceeds toward primal feasibility, while maintaining dual feasibility. In this section, we shall first describe it in a tableau form. To do so, we use system’s augmented matrix to represent the system itself. A matrix is termed tableau if it represents a system in this way. Thus, [A | b] ≡ [B, N | b] is the original tableau. A tableau is said to be canonical if its basic part is upper triangular. Note that tableau’s basic part is a basis if and only if its range space includes the right-hand side. Produced will be a sequence of tableaus, whose basic and nonbasic parts are adjusted from iteration to iteration, until optimality is achieved. The m × (n + 1) tableau is similar to that used in the simplex method; however, in contrast with the fixed number, m, of basic columns in existing simplex variants, the number of basic columns here is allowed to be lower than m, and changing dynamically in solution process; Particularly, the procedure can terminate with a deficient optimal basis. Gaussian elimination with row interchanges (or just Gaussian elimination, for short) will be applied to tableaus to upper-triangularize their basic parts. Since triangularized will always be of full column rank, we will go without saying that all triangular matrices encountered have nonzero diagonal entries. (See, for example, Golub and Van Loan [10]). ¯ | ¯b] be current canonical tableau, with the associated set {JB , JN } given. Let [U, N Without loss of generality, assume that the upper triangular matrix U ∈ Rm×s is a deficient basis, that is, s < m, and all the last m − s components of ¯b are zero. The following program is equivalent to (2.2a,b,c), or the original program (1.1): min cTB xB + cTN xN ¯ xN = ¯b s.t. U xB + N xB ≥ 0,

(3.1a) (3.1b)

xN ≥ 0.

Partition the tableau of (3.1b) as follows: [ ¯ | ¯b] ≡ U1 [U, N 0

(3.1c)

] ¯b1 , 0

¯1 N ¯2 N

(3.2)

where U1 ∈ Rs×s is upper triangular. The associated basic solution is x ¯N = 0,

(3.3a)

x ¯B = U1−1¯b1 ,

(3.3b)

f = cTB U1−1¯b1 .

(3.4)

corresponding to the objective value

Then the dual program of (3.1a,b,c) can be written in the tableau form below: 

y′

  ¯T  b  T U ¯T N

zB 0 IB 0

zN 0 0 IN

RHS





y1′

    −  ≡  ¯bT1   T cB U1 ¯T cN N 1

y2′

zB

0 0 ¯T N 2

0 IB 0

zN 0 0 IN

RHS



  − ,  cB cN

(3.5)

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

5

where IB ∈ Rs×s and IN ∈ R(n−s)×(n−s) are the identity matrices. Immediately, a basic solution to (3.5) reads: [ ′ ] [ −T ] y¯ U1 cB ′ y¯ = 1′ = (3.6 a) y¯2 0 ¯1T U −T cB z¯N = cN − N (3.6 b) 1 z¯B = 0

(3.6 c)

Obviously, x ¯ and z¯, respectively featured by (3.3a,b) and (3.6b,c), exhibit complementary slackness. We can look upon (3.6) as a “dual basic solution”, since it is a basic solution to the original dual program (2.3), except for y¯′ being different from the true one only by a nonsingular matrix factor (This will be declosed a little later). Now assume dual feasibility, i.e., z¯N ≥ 0. If, in addition, x ¯B ≥ 0, then primal and dual optimal solutions are already obtained, and hence we are done. Suppose that this is not the case, that is, x ¯B ̸≥ 0. Firstly, we determine a column to leave the basis. To do so, select a row index p such that x ¯jp = min{¯ xji | i = 1, . . . , s} < 0. (3.7) Then bring jp from JB to the end of JN . Suppose that correspondingly moving the pth ¯ converts tableau (3.2) into: column of U to the end of N [ ] ¯b1 ˜1 N ˜1 U ¯ ˜ ˜ [U , N | b] ≡ (3.8) ˜2 0 N 0 ˜ ’s columns: s = s − 1. Note that Set the number of U ˜1 en−s = U1 ep N

and

˜2 en−s = 0 N

(3.9)

˜1 is upper Hessenberg with nonzero subdiagonal In the case of p ≤ s, the (s+1)×s matrix U entries in the its p through sth columns. We apply Gaussian elimination to the tableau to annihilate these unwanted entries from p to sth columns successively; in the process, row interchanges are conducted before annihilating, whenever column’s diagonal entry is less, in module, than the subdiagonal entry. Suppose that this turns (3.8) into the following canonical tableau: ] [ ˆb1 ˆ1 N ˆ1 U ˆ ˆ ˆ , (3.10) [U , N | b] ≡ ˆ2 0 N 0 ˆ1 ∈ R(s+1)×s is upper triangular. where U The dual program associated with (3.10) can be written  ′′  ′′  y2′′ zB y1 y zB zN RHS     ˆT  ˆT  0 0 0 0 −  ≡   b  b1T  ˆT  ˆ  U1 0 IB U IB 0 cB T T T ˆ ˆ ˆ N 0 IN cN N1 N2 0 Consider the following least squares problem:

[ ] [ ]

ˆb1

ˆ1 U

. min − x B

xB 0 0

in the tableau form below:  zN RHS   0 −  (3.11) . 0 cB  IN cN

(3.12)

6

PING-QI PAN

The residual at the solution to the preceding is simply   0 r′′ =  eTs+1ˆb1  . 0

(3.13)

If r′′ is taken to be as the search direction in y ′′ space, the corresponding search direction in z space will be: ] [ T ] [ [ ] ¯B ˆ d 0 U 0 ′′ d¯ = ¯ = − ˆ1T (3.14) ˆ T es+1 . ˜ T r = −(eTs+1ˆb1 )N dN N1 N 1 2 This leads us to taking a look at the following line search form: zN = z¯N + αd¯N ,

(3.15)

where α ≥ 0 is the step-length to be determined. ˜ −1 the product Let us show the validity of the preceding more rigorously. Denote by L of some Gauss transformations and permutation matrices such that ˜ −1 [B, N | b] = [U ˆ, N ˆ | ˆb]. L

(3.16)

It is easy to see that (3.11) can be obtained from (2.3) by making the following change of variables: ˜ −T y ′′ . y=L (3.17) ˜ −T y ′′ , z) is a feasible Clearly, (y ′′ , z) is a feasible solution to (3.11) if and only if (y, z) = (L solution to (2.3). Since it is ˆb’s orthogonal projection onto the complement of the range ˆ , the residual r′′ is an uphill direction in y ′′ space, with respect to ˆbT y ′′ , that space of U is, ˆbT r′′ > 0, (3.18) which along with (3.16) gives

˜ −T r′′ ) > 0. bT ( L

(3.19)

˜ −T r′′ , is uphill, with Therefore, the associated search direction in y space, that is, r = L y respect to the original dual objective b . And it is easy to show that the associated search ¯ featured by (3.14). Further, let us demonstrate direction in z space is just equal to the d, that zkn−s (just leaving from the basic set of variables of z) is never a blocking variable. ˆ1 , N ˆ1 en−s | ˆb1 ] is equivalent From the relation between (3.2) and (3.10), it is evident that [U ˆ1 , N ˆ1 en−s ] to [U1 | ¯b1 ], in the sense of their representing equivalent systems, and hence [U has rank s + 1, implying ˆ1 en−s ̸= 0. (3.20) eTs+1 N Substituting x ¯ featured by (3.3a,b) into the equation, represented by the sth row of (3.10), we obtain ˆ1 en−s )¯ (eTs+1 N xjp = eTs+1ˆb1 , (3.21) which together with (3.20) and x ¯jp < 0 implies eTs+1ˆb1 ̸= 0.

(3.22)

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

7

It follows from (3.14), (3.17) and x ¯jp < 0 that ˆ1T es+1 > 0. (d¯N )n−s = −(eTs+1ˆb1 )eTn−s N

(3.23)

Therefore, (zN )n−s ≥ 0 holds for all α ≥ 0. ˆ1 , eT ˆb1 ) is Computing d¯N via (3.14) requires only n − s multiplications since (eTs+1 N s+1 available–it is just the (s + 1)th row of tableau (3.10). In fact, even further simplification can be achieved if d¯N in (3.15) is replaced by dN , featured below: ˆ1T es+1 , dN = −Sign(eTs+1ˆb1 )N

(3.24)

which agrees with d¯N in direction, and involves less computational effort. Now what should be done is to maximize step-length α in (3.15), and hence the dual function, subject to the non-negativeness condition zN ≥ 0. However, if set J = {j | (dN )j < 0, j = 1, . . . , n − s − 1}.

(3.25)

is empty, then the dual program is unbounded above, and hence program (1.1) has no feasible solution. In the other case, such a maximization of α gives α ¯ = −¯ zkq /(dN )q = min{−¯ zkj /(dN )j | j ∈ J} > 0.

(3.26)

Consequently, we obtain the following formula for updating the dual solution: z¯N := z¯N + α ¯ dN ,

(3.27)

together with z¯B = 0. Formula (3.26) determines not only a step-length to be taken but also a nonbasic index kq to become basic. So, bring kq from JN to the end of JB ; conformably, bring the qth ˆ to the end of U ˆ , and set s := s + 1. Then, applying Gaussian elimination column from N with row exchanges to the resulting tableau to annihilate the m through sth entries of this new basic column completes the iteration. The preceding iteration involves both row and column pivoting and associated operations, and hence does not change the rank (or the number of columns) of the basis finally. Such an iteration is referred to as a full one. It is clear that if some of the last m − s components of ¯b in (3.2) is nonzero (If some special row exchanges are conducted, however, examining whether ¯bs+1 is nonzero is enough–see the Note, following Algorithm 3.1), then only column pivoting is needed. Such an iteration is said to be rank-increasing, as it ends with the basis’ rank increased by one. The related steps can be summarized into the following model: Algorithm 3.1. Tableau version. Let (3.2) be the initial canonical tableau, where U ∈ Rm×s (s ≤ m) is upper triangular. Given the associated set {JB , JN }, and dual feasible solution z¯ ≥ 0, featured by (3.6b,c). 1. Compute x ¯B by (3.3b). 2. Stop if x ¯B ≥ 0. 3. Determine row index p by (3.7). ¯ conformably. 4. Bring jp from JB to the end of JN , and adjust columns of U and N Let (3.8) be the resulting tableau.

8

PING-QI PAN

5. Set s := s − 1. 6. Apply Gaussian elimination with row exchanges to the tableau to annihilate subdi˜ . Let (3.10) be the resulting tableau. agonal entries in the p through sth columns of U 7. Compute dN by (3.24). 8. Stop if set J, defined by (3.25), is empty. 9. Determine step-length α ¯ and column index q by (3.26). 10. Update z¯ by (3.27) together with z¯B = 0. ˆ and U ˆ conformably. 11. Bring kq from of JN to the end of JB , and adjust columns of N 12. Set s := s + 1. ¯ | ¯b] in (3.2), and go to Step 1. 13. If s = m, then set the resulting tableau to [U, N ∗ 14. Apply Gaussian elimination to the tableau to annihilate the (s + 1) through mth ¯ | ¯b]. entries of the sth basic column. Set the resulting tableau to [U, N 15. If ¯bs+1 = 0, go to Step 1; else, go to Step 7. Note∗ . Special row exchanges are conducted here. First, interchange rows to move the largest (in module; the same below) of the (s+1) through mth entries to the (s+1)th place of the column. Then, by using the (s + 1)th entry as a pivot, eliminate all entries below it, so that the corresponding zero components of the right-hand side are not disturbed at all. If the sth (diagonal) entry is no less than the (s + 1)th component, use it as a pivot to zero the (s + 1)th entry; otherwise, interchange s and (s + 1)th rows and then zero the (s + 1)th entry. Consequently, all entries below the diagonal of the column are zero, while so are all ones below the (s + 1)th component of the right-hand side. The next will be a full or a rank-increasing iteration, depending on whether the (s + 1)th component of the new right hand side is zero or not. It is noted that the procedure does not terminate if and only if cycling occurs, that is, several sets of basic columns repeat infinitely. Since the number of basic columns does not decrease in the process, cycling never involves any rank-increasing iteration; in other words, cycling can only occur with full iterations. The dual nondegeneracy assumption on all full iterations guarantees the termination since the dual objective value increases strictly in such iterations. Thus, based on the discussions made priori to Algorithm 3.1, we state the following: Theorem 3.2. Under dual nondegeneracy assumption for all full iterations, Algorithm 3.1 terminates at either (1) Step 2, with primal and dual optimal solutions reached; or (2) Step 8, indicating the infeasibility of program (1.1). 4. Main Procedure: Revised Form The approach described in the foregoing section proceeds by accumulating operations in the tableau. In this section, we recast Algorithm 3.1 into a revised version by maintaining, and updating LU factors of basic columns. Let B = [aj1 , . . . , ajs ] ∈ Rm×s be a basis and let N = [ak1 , . . . , akn−s ] ∈ Rm×(n−s) be nonbasis. Suppose that an LU decomposition of B is available, or more precisely that Gauss transformations M1 , . . . , Ms−1 ∈ Rm×m and permutation matrices P1 , . . . , Ps−1 ∈ Rm×m have been determined such that

is upper triangular, where

L−1 B = U,

(4.1a)

L−1 = Ms−1 Ps−1 · · · M1 P1 .

(4.1b)

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

9

Consider tableau [U, L−1 N | L−1 b], which is equivalent to (3.2). Note that all diagonal entries of U are nonzero, whereas (s + 1) through mth components (if any) of L−1 b are zero. So, we can make the following partition:  [U L−1 N | L−1 b] = 

U1 0

(L−1 N )1 −1

(L

(L−1 b)1

N )2

 ,

(4.2)

0

where U1 ∈ Rs×s . Using the above notation, we rewrite the primal basic solution, featured by (3.3a,b), as follows: x ¯N = 0, x ¯B =

(4.3a)

U1−1 (L−1 b)1

(4.3b)

corresponding to objective value f = cTB U1−1 (L−1 b)1 ,

(4.4)

The associated dual solution, featured by (3.6a,b,c), can be rewritten: [

y¯′ y¯ = 1′ y¯2 ′

]

[

U1−T cB = 0

] (4.5a)

z¯N = cN − (L−1 N )T1 U1−T cB

(4.5b)

z¯B = 0

(4.5c)

Other computations carried out by Algorithm 3.1 can be rewritten similarly with program’s original data and LU factors of basic columns. So, we only derive updating formulas for LU factors. Suppose that the LU decomposition (4.1a,b) is known. (i) Case 1: Deleting a column from the basis. Let column ajp be deleted from basis ˜ . Set B. Assume that conformably removing the pth column from U ∈ Rm×s yields U ˜ is already upper triangular. In the other case of p ≤ s, s := s − 1. If p = s + 1, then U ˜ matrix U is upper Hessenberg with nonzero subdiagonal entries in its p through sth columns. Suppose that these unwanted entries are eliminated by premultiplying Gauss ˜ k ∈ Rm×m , k = p, . . . , s, and permutations P˜k ∈ Rm×m , k = p, . . . , s, transformations M where each P˜k interchanges the k and (k + 1)th rows to have a larger pivot, so that ˜ s P˜s · · · M ˜ p P˜p U ˜ M is upper triangular. Then, it is easy to obtain the following formulas for updating L−1 and U , respectively: ˜ s P˜s · · · M ˜ p P˜p L−1 L−1 := M

and

˜ s P˜s · · · M ˜ p P˜p U ˜ U := M

(4.6)

(ii) Case 2: Appending a column to basic columns. Let akq be the nonbasic column that is moved to the end of the basic part, becoming the (s + 1)th basic column. Set s := s + 1 and define a ˜kq = L−1 akq . (4.7)

10

PING-QI PAN

Without loss of generality, assume that s < m. Let Pˆs+1 be a permutation such that the (s + 1)th component of Pˆs+1 a ˜kq is the largest of its (s + 1) through mth components. And ˆ let Ms+1 be a Gauss transformation such that the (s+2) through mth components (if any) ˆ s+1 Pˆs+1 a ˆ s+1 Pˆs+1 a of M ˜kq are zero. Further, assume that permutation Pˆs makes Pˆs M ˜kq have its sth component larger than (s + 1)th component, and that Gauss transformation ˆ s is determined so that all the (s + 1) through mth components of M ˆ s Pˆs M ˆ s+1 Pˆs+1 a M ˜kq are zero. Then it is easy to obtain the following updating formulas: ˆ s Pˆs M ˆ s+1 Pˆs+1 L−1 L−1 := M

and

ˆ s Pˆs M ˆ s+1 Pˆs+1 a U := [U, M ˜kq ]

(4.8)

Noticed that the (s + 2) through mth components (if any) of the new L−1 b are zero. It might be convenient to arrange computations of maintaining, and updating LU factors in a so-called revised tableau. Let B be an initial basis and let (4.1) be its LU decomposition. At the beginning, a revised tableau can be constructed as follows: [U ; L−1 |¯b],

(4.9)

where U ∈ Rm×s is upper triangular, and ¯b = L−1 b. Then, we can formulate the revised version of Algorithm 3.1 as follows: Algorithm 4.1. Revised version. Assume that (4.9) is the initial revised tableau, and {JB , JN } the associated ordered index set. Given a feasible dual solution z¯, featured by (4.5b,c). 1. Compute x ¯B by (4.3b). 2. Stop if x ¯B ≥ 0. 3. Select row index p by (3.7). 4. Bring the jp from JB to the end of JN , and update the revised tableau by dropping the pth column of U . 5. Set s := s − 1. 6. Apply Gassian elimination to the revised tableau to annihilate subdiagonal entries in p through sth columns of U . 7. Compute dN = −Sign(eTs+1¯b)N T L−T es+1 , where N ∈ Rm×(n−s) consists of nonbasic columns of A. 8. Stop if set J defined by (3.25) is empty. 9. Determine step-length α ¯ and column index q by (3.26). 10. Update z¯ by (3.27) together with z¯B = 0. 11. Bring kq from of JN to the end of JB , and set [U ; L−1 |¯b] := [U, L−1 akq ; L−1 | ¯b]. 12. Set s := s + 1. 13. If s = m, go to Step 1. 14. Apply Gaussian elimination∗ to the tableau to annihilate (s + 1) through mth components of the sth column of U . 15. If ¯bs+1 = 0, go to Step 1; else, go to Step 7. Note∗ . The Gaussian elimination should be conducted in the same manner as was described in the Note, following Algorithm 3.1. The other discussions at the end of Section 3 also apply. In the preceding algorithm, Gauss transformations and permutation matrices are accumulated in the revised tableau to maintain L−1 explicitly. To exploit sparsity, however, it should be favorable to keep L−1 as products of these simple matrices, as has been done

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

11

in the well-known implementation of the simplex method. It is also needed to deal with such typical issues as balancing the aim of maintaining sparsity and the aim of insisting numerical stability, appropriate ordering of matrix’s columns and rows to gain sparser factors, and so on. All these are beyond the scope of this paper, and will be handled separately. 5. Phase-1 Procedure Algorithms established in forgoing Sections require dual feasibility to get themselves started. As have been done in the conventional Phase-1, it is possible to develop a procedure for this purpose, by solving an auxiliary problem with piecewise-linear sums of dual infeasibilities as its objective. In this Section, we shall focus on this issue, only in conjunction with the tableau form. Once again, let us begin with the canonical tableau (3.2). But this time, it is assumed that the associated dual solution z¯, featured by (3.6b,c), is infeasible. Define e˜ ∈ Rn−s by { e˜i =

1,

for z¯ki < 0

0,

for z¯ki ≥ 0

i = 1, . . . , n − s

(5.1)

In stead of the dual objective, ¯bT y ′ , of program (3.5), use the following auxiliary function:

or simply

¯ y ′ ), e˜T zN = e˜T (cN − N

(5.2)

−uT y ′ ,

(5.3a)

¯ ’s columns corresponding to infeasible dual variables, that is, where u is the sum of N ¯ e˜. u=N

(5.3b)

Then, steps of a single iteration of our Phase-1 are similar to those of the main procedure, described in Section 3. We shall not go into details here, but only mention two differing points: vector ¯b is now replaced by −u; and, since there exist some infeasible dual variables, the pivot rule used should be adapted, as in the conventional Phase-1. 6. LU-Based Crash Heuristic Let us now turn to how to provide an initial canonical tableau, or basis. A typical crash, like that used in MINOS, produces a nearly triangular basis by selecting columns from the coefficient matrix; the basis is then factored by another subroutine. Unfortunately, it does so without taking into account whether the resulting basis is close to an optimal one. This is, in our view, a serious flaw since it is well accepted that the quality of an initial basis is of great importance–starting with a basis close to an optimal one should lead to fewer iterations required; in fact, if it happens to be optimal, there will be no need for any iteration at all. Although it seems to be impossible to obtain an optimal basis immediately, every endeavor should be made to provide an initial basis as close to an optimal one as possible. Along this line, we proposed a QR-decomposition- based crash heuristic (Pan [31]), based on the plausible characterization of optimal basis (Pan [23,24]). Its essence, in dual case,

12

PING-QI PAN

is to favor an index j to become basic (or to enter JB ) for which the gradient aj of the constraint aTj y ≤ cj makes the most acute possible angle with the gradient b of the dual objective. In this section, we shall develop an LU-decomposition-based version of it to fit our needs. Firstly, set JB to empty, and set JN := {k1 , . . . , kn } ≡ {1, . . . , n}. (0)

(0)

Conformably, set U (0) to empty, N (0) = N2 := A and b(0) = b2 := b. We shall begin with tableau [A, b] = [U (0) , N (0) , b(0) ], and determine indices one by one to enter JB . Define vector (0) (0) (0) (0) h(0) = [hk1 , . . . , hkn ]T = −(N2 )T b2 . (6.1) It holds that h(0) ̸= 0, since h(0) = 0 and (6.1) together lead to b = 0, a contradiction. If h(0) > 0, then program (2.4) has no optimal solution, and so does program (2.1), since if there exists a feasible solution, say (y, z), then the (y + αb, z + αh) would also be feasible for any arbitrarily large α ≥ 0, and hence (2.1) is unbounded above. Suppose now that (2.1) has an optimal solution. Then, a column index q can be chosen such that (0)

q = Arg min{hkj /∥akj ∥ | j = 1, . . . , n}

(6.2)

(0)

Clearly, hkq < 0, and that the gradient akq of the constraint aTkq y ≤ ckq makes the most acute possible angle with b, among akj , j = 1, . . . , n. Note that akq is nonzero. So, we can have the following tableau: M1 P1 [U (0) , N (0) , b(0) ]P¯1 = [U (1) , N (1) , b(1) ],

(6.3)

where P¯1 is the permutation matrix interchanging A’s first column and qth column, P1 the permutation matrix interchanging the first component of the new first column and its largest component in module, M1 the Gauss transformation annihilating entries below the diagonal of the first column, and U (1) ∈ Rm×1 is upper triangular with the nonzero diagonal (first) entry. Then we rearrange the ordered set {JB , JN } conformably, and complete the first iteration. Assume that for some l − 1 < m we have obtained sets JB = {j1 , . . . , jl−1 },

and

JN = {k1 , . . . , kn−l+1 },

(6.4)

and the associated tableau Ml−1 Pl−1 · · · M1 P1 [U (0) , N (0) , b(0) ]P¯1 · · · P¯l−1 = [U (l−1) , N (l−1) , b(l−1) ] [ (l−1) (l−1) ] (l−1) N1 b1 U1 = (l−1) , (l−1) b2 0 N2 (6.5) (l−1)

(l−1)

where U1 ∈ R(l−1)×(l−1) is nonsingular and upper triangular. If b2 = 0, then (6.5) renders an initial canonical tableau, with the associated basis, U (l−1) , of order l − 1, and hence all is done. In the other case, consider the least squares problem below:



min b(l−1) − U (l−1) xB . (6.6) xB

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

The residual at the solution to the preceding is simply [ ] 0 r = (l−1) b2

13

(6.7)

The above is the orthogonal projection of b(l−1) onto the null space of (U (l−1) )T , and can serve as a search direction in y space. Thereby, the corresponding search direction in zN space is (l−1)

h(l−1) = [hk1

(l−1)

(l−1) T (l−1) ) b2 .

, . . . , hkn−l+1 ]T = −(N (l−1) )T r = −(N2

(6.8)

(l−1)

Under b2 ̸= 0 and the consistency of (2.1b), it is easy to show that h(l−1) is nonzero. Based on the plausible characterization of optimal basis, we determine a column index q such that (l−1) q = Arg min{hkj /∥vj ∥ | j = 1, . . . , n − l + 1}, (6.9) where vj is the jth column of N (l−1) . And the remaining manipulations and discussions are similar to those, made for the first iteration. Using the above notation, we summarize the steps into the following model: Algorithm 6.1. LU-decomposition-based crash heuristic. 1. Set JB to empty and JN := {1, . . . , n}. 2. Set U (0) to empty, N (0) := A and b(0) := b. 3. Compute vector h(0) by (6.1). 4. For l = 1, . . . , m, (1) compute h(l−1) by (6.8); (2) stop if h(l−1) = 0; (3) stop if h(l−1) ≥ 0; (4) select column index q via rule (6.9); (5) compute [U (l) , N (l) , b(l) ] := Ml Pl [U (l−1) , N (l−1) , b(l−1) ]P¯l , where P¯l is the permutation matrix interchanging N (l−1) ’s first column and qth column, Pl the permutation matrix interchanging the lth component of the new first column and the largest component, in magnitude, below the diagonal, Ml the Gauss transformation annihilating entries below the diagonal of the column, and U (l) ∈ Rm×l is upper triangular with nonzero diagonals; (6) Rearrange the ordered set {JB , JN } conformably. 5. Stop. It is noted from the preceding that from iteration to iteration, the number of basic columns grows one by one from 0 up to at most m, while the number of nonbasic columns decreases from n down to at least n−m. The heuristic requires no more than m iterations. It terminates either at Step 4(3), indicating nonexistence of an optimal solution to (2.1), or at Step 4(2) or 5, producing an initial canonical tableau and the associated set {JB , JN }: Step 4(2) yields a deficient basis, while Step 5 provides a full one. Because the former is closely related to the primal degeneracy, in fact, it should occur very frequently in practice, as is confirmed by our computational experiments, reported in the next section. 7. Computational Results To corroborate our theory and give an insight into new approach’s behavior, we have performed some computational experiments. In this section, we report our numerical results obtained in these tests.

14

PING-QI PAN

The following two FORTRAN 77 codes were tested, and compared: Code RSM: The revised two-Phase simplex method, in which the inverse of the basis is updated and maintained explicitly. Code DPSLU: Algorithms 3.1 is used as Phase-2, supported by its adaptation, described in Section 5, as Phase-1. The crash heuristic Algorithm 6.1 is furnished to provide an initial basis. The preceding were coded without exploiting sparsity. Code RSM was a very efficient one available. Dantzig’s original rule and Harris’ idea were implemented in both of them. Compiled using the NDP–FORTRAN–386 VER. 2.1.0. with default options, the codes were run under DOS 6.2 system on an IBM 486/66 DX2 compatible microcomputer with memory 32 Mbytes available. All the CPU time was measured in seconds with utility routine DOSTIM. As such a slow computer waw used specially, the obtained data of CPU time should be very reliable. The machine precision used was about 16 decimal digits. And 10−6 was taken to be as the primal and the dual feasibility tolerance. The test set includes 25 standard test problems from NETLIB that do not have BOUNDS and RANGES sections in their MPS files (Gay [11]), since the current version of our code cannot handle such problems implicitly. In Table 7.1, the columns labeled M and N give, respectively, the number of rows and of columns of the coefficient matrix, before adding slack variables. So, the column labeled N actually indicates the number of nonbasic columns for the simplex method (N = n − m). In terms of N + M , the test set is the largest possible subset of NETLIB problems of such type that can be solved in our computing environment. Results obtained with RSM and DPSLU are displayed in Tables 7.1 and 7.2, respectively. The number of iterations and running time, required for solving a program or for Phase-1 only, is given respectively in columns labeled Iter and Time under Total or Phase-1. The percentage of the number of degenerate, deficient and full iterations is listed in the columns labeled % Dgn , % Dfc % Nfl , respectively. The percentage of the number of the final basis’ columns (against m) is given in the column labeled % M1 . The final objective values are not included, as they were achieved by both codes correctly, as those in NETLIB index file. Table 7.3 compares the performance of the two Codes by giving iteration and time ratios of RSM to DPSLU. The bottom line labeled Total shows that iterations required by DPSLU and by RSM are not far apart from one another, with overall and Phase-1 ratio 1.10 and 0.82, respectively. On the other hand, however, the margin between the running time is quite large, with overall and Phase-1 ratio as high as 7.87 and 6.57, respectively. We stress that running time should be taken to be as the sole index for evaluation of codes’ efficiency. It is noted that computational work, required by DPSLU per iteration is much lower than that required by RSM. Moreover, as expected, the new method is also more efficient than the QR-decomposition-based version (Pan [31]). First of all, the excellent performance of DPSLU might be partially due to the heavy occurrence of basis deficiency, associated with the new method. From the column labeled % Dfc in Table 7.2, it is seen that as high as 91.72% of total iterations involve deficient basis. The column labeled % M1 indicates that DPSLU terminated with a deficient basis with over a half of the tested problems. Clearly, handling of deficient basis is inexpensive, compared with handing full basis. In addition, we point out that a full iteration is more expensive than a rank-increasing one since it involves both dropping from and adding into the basis a column, whereas the latter does adding only (Section 3). The column labeled % Nfl in Table 7.2 indicates

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

15

that full iterations carried out by DPSLU are quite low, with overall 35.16% only. The percentage of degenerate iterations, i.e. those with a zero-length step, would be a good index for anti-degeneracy nature of algorithms. From columns labeled % Dgn in Tables 7.1 and 7.2, it is seen that overall the percentage of such iterations involved by RSM is as high as 27.21%, whereas that by DPSLU is 4.34% only. Therefore, degeneracy seems to be no longer a problem. This might also contribute to DPSLU’s success. The merit of the crash heuristic should not be underestimated. In Table 7.4, the column labeled Ninf and Ndi list the number of primal and dual infeasibilities resulting from the heuristic, respectively. Notably, primal feasibility was completely achieved by it alone for 9 out of the 25 problems. Giving the percentage of primal infeasibilities against m, the column labeled % Ninf indicates that infeasibilities from the heuristic are as low as 4.13% overall. On the other hand, the column labeled % Iter , which may serve as another index for DPSLU’s efficiency, shows that iterations carried out by the heuristic make up about a half of total iterations. We also mention that some crash heuristics and pivot rules based on the plausible characterization of optimality (Pan [23,24]) met similar success in other contexts. (See Pan [25-36]). It would be interesting to see how the method performs with small text problems. We tested with another set of 60 arbitrarily collected such problems, with sizes of up to 15 × 37, and 6 × 13 on average. It is seen from Table 7.5 that DPSLU also outperforms RSM with these problems significantly. Surprisingly, primal feasibility was achieved by the heuristic alone with 58 out of the 60 problems; only one primal infeasibility was left with each of the remaining two problems. Finally, we conclude that the numerical results obtained are very interesting, distinctive and encouraging, showing that the proposed method are full of promise.

16

PING-QI PAN

REFERENCES 1. R.H. Bartels, A stabilization of the simplex method, Numerische Mathematik 16 (1971), 414–434. 2. R.H. Bartels and G.H. Golub, The simplex method of linear programming using LU decomposition, Communication ACM, 12 (1969), 275–278. 3. W.J. Baumol, Economic Theory and Operations Analysis(2nd ed.), Prentice-Hall, Inc., Englewood Cliffs, NJ, 1965. 4. E.M. L. Beale, Cycling in the dual simplex algorithm, Naval Research Logistics Quarterly, 2 (1955) 269–275. 5. M. Benichou, J.M. Gauthier, G. Hentges, G. Ribiere, The efficient solution of linear programming problems–some algorithmic techniques and computational results, Mathematical Programming , 13 (1977), 280–322. 6. A. Charnes, Optimality and degeneracy in linear programming, Econometrica 20 (1952) 160–170. 7. D. Chen, P. Pardalos and M. Saunders, The simplex algorithm with a new primal and dual pivot rule, Operations Research Letters, 16 (1994), 121–127. 8. G.B. Dantzig, Linear Programming and Extensions, Princeton University Press , Princeton , New Jersey, 1963. 9. G.B. Dantzig and W. Orchard-Hayes, The product form for the inverse in the simplex method, Mathematical Tables and Other Aids to Computation 8 (1954), 64–67. 10. J.J.H. Forrest and J.A. Tomlin, Updated triangular factors of the basis to maintain sparsity in the product form simplex method, Mathematical Programming 2 (1972), 263– 278. 11. D.M. Gay, Electronic mail distribution of linear programming test problems, Mathematical Programming Society COAL Newsletter 13 (1985) , 10–12. 12. P.E. Gill, G.H. Golub, W. Murray and M.A. Saunders,Methods for modifying matrix factorization, Math. Comp. 28 (1974), 505–538. 13. D. Goldfarb and J.K. Reid, An practical steepest-edge simplex algorithm, Mathematical Programming 12 (1977), 361-371. 14. G.H. Golub, Numerical methods for solving linear least squares problems, Numerische Mathematik 7(1965), 206–216. 15. G.H. Golub and C.F. Van Loan, Matrix Computations, John Hopkins University Press, Baltimore, Maryland, 1983. 16. P.M.J. Harris, Pivot selection methods of the Devex LP code, Mathematical Programming 5 (1973), 1–28. 17. B. Hattersley and J. Wilson, A dual approach to primal degeneracy, Mathematical Programming 42 (1988), 135–145. 18. A.R.G. Heesterman, Matrices and Simplex Algorithms, D.Reidel Publishing Company, Dordrecht, Holland, 1983. 19. E. Hellerman and D.C. Rarick, The partitioned preassigned pivot procedure, in D.J. Rose and R.A. Willouhby (eds.), Sparse Matrices and Their Applications , Plemum Press, New York, 68–76, 1972. 20. A.J. Hoffman, Cycling in the simplex algorithm, Report No. 2974, National Bureau Standards, Washington, D.C., 1953. 21. I. Maros and G. Mitra, Simplex Algorithms, in Advances in Linear and Integer Programming (Ed. J. Beasley), Oxford University Press, Oxford, 1996.

THE DUAL PROJECTIVE PIVOT ALGORITHM FOR LINEAR PROGRAMMING

17

22. J.L. Nazareth, Computer Solution of Linear Programs, Oxford Science Publications, Oxford, 1987. 23. P.-Q. Pan, Practical finite pivoting rules for the simplex method, OR Spektrum 12 (1990), 219–225. 24. P.-Q. Pan, A simplex-like method with bisection for linear programming, Optimization 22 (1991 ) (5), 717–743. 25. P.-Q. Pan, Ratio-test-free pivoting rules for a dual Phase-1 method, in: S.-T Xiao and F. Wu, eds., Proceedings of the Third Conference of Chinese SIAM (Tsinghua University Press, Beijing, 1994a), 245–249. 26. P.-Q. Pan, Achieving primal feasibility under the dual pivoting rule, Journal of Information & Optimization Sciences 15 (1994b) No. 3, 405–413. 27. P.-Q. Pan, Composite phase-1 methods without measuring infeasibility, in: M.-Y. Yue, eds., Theory of Optimization and its Applications (Xidian University Press, Xian, 1994c), 359–364. 28. P.-Q. Pan, New non-monotone procedures for achieving dual feasibility, Journal of Nanjing University, Mathematics Biquarterly 12 (1995), No.2, 155–162. 29. P.-Q. Pan, A modified bisection simplex method for linear programming, Journal of Computational Mathematics 14 (1996) No.3, 249–255. 30. P.-Q. Pan, The most-obtuse-angle row pivot rule for achieving dual feasibility: a computational study, European Journal of Operational Research 101 (1997), 167–176. 31. P.-Q. Pan, A dual projective simplex method for linear programming, Computers and Mathematics with Applications 35 (1998a), No. 6, 119–135. 32. P.-Q. Pan, A basis-deficiency-allowing variation of the simplex method, Computers and Mathematics with Applications, 36 (1998b), No. 3, 33–53. 33. P.-Q. Pan, A projective simplex method for linear programmingLinear Algebra and its Applications, 292(1999), 99–125. 34. P.-Q. Pan, A projective simplex algorithm using LU decomposition, Computers and Mathematics with Applications 39 (2000), 187–208. 35. P.-Q. Pan, 24. A projective simplex method for linear programming, Linear Algebra and Its Applications , 292 (1999), 99–125. 36. P.-Q. Pan and Y.-P. Pan, A phase-1 approach to the generalized simplex algorithm, computers and Mathematics with Applications 42(2001), No. 10/11, 1455–1464. 37. T. Terlaky, A convergent criss-cross method, Math. Oper. und Stat. ser. Optimization 16 (5) (1985), 683–690. 38. T. Terlaky and S. Zhang,Pivot rules for linear programming: A survey on recent theoretical developments, Annals of Operations Research 46 (1993), 203–233. 39. S. Zionts, The criss-cross method for solving linear programming problems, Management Science 15 (7) (1969), 426–445. Department of Mathematics, Southeast University, Nanjing 210096, P.R. of China