Jul 26, 2000 - The authors gratefully acknowledge the assistance of Charles Johnson, Chi-Kwong Li, and Ilya. Spitkovsky. The second author thanks the ...
Correcting an Inconsistent System of Linear Inequalities by Nonlinear Programming Paula Amaral∗
Michael W. Trosset†
Pedro Barahona‡
July 26, 2000
Abstract We consider the problem of correcting an inconsistent system of linear inequalities, Ax ≤ b, subject to nonnegativity constraints, x ≥ 0. We formulate this problem as a nonlinear program and derive the corresponding Karush-Kuhn-Tucker conditions. These conditions reveal several interesting properties that solutions must satisfy and allow us to derive several equivalent problems that involve fewer decision variables and are more amenable to solution. We propose using a gradient projection method to minimize an objective function φ(x), subject only to x ≥ 0. We also propose a hybrid approach that exploits an interesting relation between the correction problem and the method of total least squares.
Contents 1 Problem Formulation
2
2 Necessary Conditions and Algorithms
3
3 Relation to Total Least Squares
8
4 Discussion
10
∗
Departamento de Matem´ atica, Faculdade de Ciˆencias e Tecnologia, Universidade Nova de Lisboa, 2825-114 Caparica, Portugal. † Department of Mathematics, College of William & Mary, P.O. Box 8795, Williamsburg, VA 23185-8795, USA. ‡ Departamento de Inform´ atica, Faculdade de Ciˆencias e Tecnologia, Universidade Nova de Lisboa, 2825-114 Caparica, Portugal.
1
1
Problem Formulation
We consider the problem of correcting an inconsistent system of linear inequalities, Ax ≤ b x≥0 ,
(1)
where A ∈ σn0 +1 and vn0 +1,n0 +1 6= 0, then Problem (16) has a unique solution given by [H0∗ | p∗0 ] = −σn0 +1 un0 +1 vnT0 +1 . Furthermore, k [H0∗ | p∗0 ] k2 = kH0∗ k2 + kp∗0 k2 = σn2 0 +1 and the corrected linear system (A0 + H0 )x0 = b0 + p0 has a unique solution given by "
x∗0 −1
#
=−
vn0 +1 vn0 +1,n0 +1
.
Thus, if σn0 > σn0 +1 , if vn0 +1,n0 +1 6= 0, and if x∗0 ≥ 0, then we have solved Problem (2). The preceding approach to correcting an inconsistent system of linear inequalities was suggested by Amaral and Barahona [1]. Its practical utility is evidently limited by the difficulty of determining J0 , the set of inactive nonnegativity constraints, and I0 , the set of active inequality constraints. We now propose a practical way to make these determinations. Let us re-examine the approach that we proposed in Section 2, viz. using a gradient projection method to solve Problem (12). This method is guaranteed to converge to a local minimizer of Problem (12); however, the convergence rate is only linear. In practice, gradient projection methods often converge quite slowly. The outstanding virtue of gradient projection methods is that they usually identify the active constraints and an approximate solution after a small number of iterations. These observations suggest a hybrid approach: apply a gradient projection method to Problem (12) to determine which constraints are active; then use the method of total least squares to efficiently compute the solution. After several iterations of a gradient projection method applied to Problem (12), we should have a good sense of which constraints are active. We guess J0 and I0 , then accurately solve Problem (2) by computing one singular value decomposition. To illustrate this approach, we return to the numerical example presented in Section 2. After t = 20 iterations, it seems abundantly clear that x∗ > 0, i.e. J0 = {1, 2}, and that the set of active inequality constraints is I0 = {1, 4, 6}. Guessing that this is the case, we compute the singular value decomposition of
−0.10433318 −0.3349605 −2.2440190 2.5746725 . 1.05241872 −0.4327864 0.24375548 0.5536801 0.4284550 We immediately obtain σ2 = 0.8696041 > 0.4639082 = σ3 , 9
v3,3 = −0.2092186 6= 0, "
v3,1 /v3,3 v3,2 /v3,3
∗
x =−
#
"
=
3.920980 2.543885
#
> 0,
−0.16868379 −0.10944004 0.04302082 ∗ ∗ T [H0 | p0 ] = −σ3 u3 v3 = −0.07738515 −0.05020657 0.01973617 , −0.33224353 −0.21555566 0.08473481 and φ(x∗ ) = k [H0∗ | p∗0 ] k2 = σ32 = 0.2152108. The computational advantages of exploiting the method of total least squares are compelling. Having connected Problem (12) and the method of total least squares, we conclude this section with some additonal observations about the former. Let x∗ denote a global minimizer of Problem (12) and suppose that x∗ > 0. Let C = [A | b] and let y∗ = [xT∗ | − 1]T . Let C0 comprise the rows of C for which (Cy∗ )+ = (Ax∗ − b)+ = 0. Then the global minimum of Problem (12) is
φ (x∗ ) =
2
(Cy∗ )+
ky∗ k
2
=
y∗T C T
(Cy∗ )+
+ y∗T y∗
=
y∗T C0T C0 y∗ . y∗T y∗
Now define ψ :
