Applied Mathematics, 2012, 3, 1334-1350 http://dx.doi.org/10.4236/am.2012.330190 Published Online October 2012 (http://www.SciRP.org/journal/am)
On the Stable Sequential Kuhn-Tucker Theorem and Its Applications* Mikhail I. Sumin Mathematical Department, Nizhnii Novgorod State University, Nizhnii Novgorod, Russia Email:
[email protected],
[email protected] Received July 2, 2012; revised August 2, 2012; accepted August 9, 2012
ABSTRACT The Kuhn-Tucker theorem in nondifferential form is a well-known classical optimality criterion for a convex programming problems which is true for a convex problem in the case when a Kuhn-Tucker vector exists. It is natural to extract two features connected with the classical theorem. The first of them consists in its possible “impracticability” (the Kuhn-Tucker vector does not exist). The second feature is connected with possible “instability” of the classical theorem with respect to the errors in the initial data. The article deals with the so-called regularized Kuhn-Tucker theorem in nondifferential sequential form which contains its classical analogue. A proof of the regularized theorem is based on the dual regularization method. This theorem is an assertion without regularity assumptions in terms of minimizing sequences about possibility of approximation of the solution of the convex programming problem by minimizers of its regular Lagrangian, that are constructively generated by means of the dual regularization method. The major distinctive property of the regularized Kuhn-Tucker theorem consists that it is free from two lacks of its classical analogue specified above. The last circumstance opens possibilities of its application for solving various ill-posed problems of optimization, optimal control, inverse problems. Keywords: Sequential Optimization; Minimizing Sequence; Stable Kuhn-Tucker Theorem in Nondifferential Form; Convex Programming; Duality; Regularization; Optimal Control; Inverse Problems
1. Introduction We consider the convex programming problem (P) f z min, Az = h, g i z 0, i = 1, , m, z D Z , where f :D R1 is a convex continuous functional, A :Z H is a linear continuous operator, h H is a fixed element, gi :Z R1 , i = 1, , m , are convex functionals, D is a convex closed set, and Z and H are Hilbert spaces. It is well-known that the Kuhn-Tucker theorem in nondifferential form (e.g. see [1-3]) is the classical optimality criterion for Problem (P). This theorem is true if Problem (P) has a Kuhn-Tucker vector. It is stated in terms of the solution to the convex programming problem, the corresponding Lagrange multiplier, and the regular Lagrangian of the optimization problem (here, “regular” means that the Lagrange multiplier for the objective functional is unity). Note two fundamental features of the classical Kuhn-Tucker theorem in nondifferential form (e.g. see *
This work was supported by the Russian Foundation for Basic Research (project no. 12-01-00199-a) and by grant of the Ministry of Education and Science of the Russian Federation (project no. 1.1907.2011)
Copyright © 2012 SciRes.
[4-7]). The first feature is that this theorem is far from being always “correct”. If the regularity of the problem is not assumed, then, in general, the classical theorem does not hold even for the simplest finite-dimensional convex programming problems. In particular, the corresponding one-dimensional example can be found in [7] (see Example 1 in [7]). For convex programming problems with infinite- dimensional constraints, the nonvalidity of this classical theorem can be regarded as their characteristic property. In this case a simple meaningful example can be found in [7] (see Example 2 in [7]) also. The second important feature of the classical KuhnTucker theorem is its instability with respect to perturbations of the initial data. This instability occurs even for the simplest finite-dimensional convex programming problems. The following problem can be a particular example. Example 1.1. Consider the minimization of a strongly convex quadratic function of two variables on a set specified by an affine equality constraint: 1 1 1 2 x min, Ax = y, x R 2 , A , y . (1) 0 0 0
The exact normal solution is x = 0,5; 0,5 . The AM
M. I. SUMIN
dual problem for (1) has the form V L x , =
1 AA , y, max, R 2 , 4 L x, x , Ax y 2
where
1 x argmin L x, , x R 2 = A . 2
and
Its solutions are the vectors 1, R1 . It is easy to verify that every vector of this form is a Kuhn-Tucker vector of problem (1). For > 0 consider the following perturbation of problem (1) 2
x min, A x = y , x R 2 , 1 1 1 A , y . 2 0
(2)
The corresponding dual problem
V L x , max, R 2 2 2 2 4 , has the solution = . 3
According to the classical Kuhn-Tucker theorem, the vector
1 1 x argmin L x, , x R 2 = 1 ,
is a solution to perturbed problem (2). At the same time, this vector is an “approximate” solution to original system (1), and no convergence to the unique exact solution occurs as 0 . It is natural to consider the above-mentioned features of the classical Kuhn-Tucker theorem in nondifferential form as a consequence of the classical approach long adopted in optimization theory. According to this approach, optimality conditions are traditionally written in terms of optimal elements. At the same time, it is well-known that optimization problems and their duals are typically unsolvable. The mentioned above examples from [7] show that the unsolvability of dual problems fully reveals itself even in very simple (at first glance) convex programming problems. On the one hand, optimality conditions for such problems cannot be written in terms of optimal elements. On the other hand, even if they can, the optimal elements in these problems are unstable with respect to the errors in the initial data, which is demonstrated by Example 1.1. This fact is a fundamental obstacle preventing the application of the classical optimality conditions to solving practical problems. An effective way of overcoming the two indicated features of the classical Kuhn-Tucker theorem (which Copyright © 2012 SciRes.
1335
can also be regarded as shortcomings of the classical approach based on the conventional concept of optimality) is to replace the language of optimal elements with that of minimizing sequences that is sequential language. In many cases this replacement fundamentally changes the situation: the theorems become more general, absorb the former formulations, and provide an effective tool for solving practical problems. So-called regularized Kuhn-Tucker theorem in nondifferential sequential form was proved for Problem (P) with strongly convex objective functional and with parameters in constraints in [7]. This theorem is an assertion in terms of minimizing sequences (more precisely, in terms of minimizing approximate solutions in the sense of J. Warga) about possibility of approximation of the solution of the problem by minimizers of its regular Lagrangian without any regularity assumptions. It contains its classical analogue and its proof is based on the dual regularization method (see, e.g. [4-7]). The specified above minimizers are constructively generated by means of this dual regularization method. A crucially important advantage of these approximations compared to classical optimal Kuhn-Tucker points (see Example 1.1.) is that the former are stable with respect to the errors in the initial data. This stability makes it possible to effectively use the regularized Kuhn-Tucker theorem for practically solving a broad class of ill-posed problems in optimization and optimal control, operator equations of the first kind, and various inverse problems. In contrast to [7], in this article we prove the regularized Kuhn-Tucker theorem in nondifferential sequential form for nonparametric Problem (P) in case the objective functional is only convex and the set D is bounded. Just as in [7], its proof is based on the dual regularization method. Simultaneously, the dual regularization method is modified here to prove the regularized Kuhn-Tucker theorem in the case of convex objective functional. This article consists of an introduction and four main sections. In Section 2 the convex programming problem in a Hilbert space is formulated. Section 3 contains the formulation of the convergence theorem of dual regularization method for the case of a strongly convex objective functional including its iterated form and the proof of the analogous theorem when the objective functional is only convex. In turn, in Section 4 we give the formulation of the stable sequential Kuhn-Tucker theorem for the case of a strongly convex objective functional. Besides, here we prove the theorem for the same case but in iterated form and in the case of the convex objective functional also. Finally, in Section 5 we discuss possible applications of the stable sequential Kuhn-Tucker theorem in optimal control and in ill-posed inverse problems. AM
M. I. SUMIN
1336
2. Problem Statement
3.1. The Original and Perturbed Problems
Consider the convex programming Problem (P) and suppose that it is solvable. Its solutions we denote by z 0 . We also assume that
Let F be the set formed of all collections of initial data f f , A, h, gi ,i = 1, , m for Problem (P). Each collection consists of a functional f, which is continuous and convex on D, of a linear continuous operator A, an element h and functionals gi ,i = 1, , m , that are continuous and convex on D. Moreover, it holds that
f z1 f z2 LM z1 z2 , g z1 g z2 LM z1 z2 z1 , z2 D S M ,
f z1 f z2 LM z1 z2 ,
where LM > 0 is a constant and SM z Z : z M .
gi z1 gi z2 LM z1 z2
Below we use the notations:
z1 , z2 D S M , i = 1, , m,
L z, , f z , Az h , g z , D z D : Az h , gi z , i = 1, , m , 0.
Define the concave dual functional called the value functional V , inf L z , , , H , R m z D
where the constant LM is independent of the collection. If the objective functional of Problem (P) is strongly convex, then a functional f in each collection is continuous and strongly convex on D and has the constant of strong convexity > 0 that is independent of the collection. Furthermore, we define collections f 0 and f of unperturbed and perturbed data, respectively:
V , sup, , H Rm .
(3)
In what follows the concept of a minimizing approximate solution to Problem (P) plays an important role. Recall that a sequence z k D , k = 1, 2, , is solution to Problem (P) called a minimizing approximate k if f z k , for z k D , k 0 , and k . Here is the generalized infimum:
f
and
If f is a strongly convex functional and also if D is a bounded set, can be written as
f z ,if z exists; otherwise . 0
0
Recall that in this case the Kuhn-Tucker vector of Problem (P) is a pair , H Rm such that f
f z L z, ,
0
2
,
A A0 C , h h0 C ,
g z g 0 z C 1 z
2
(4)
,
where C > 0 is independent of and
g g1 , , g m
.
Denote by (P0) Problem (P) with the collection f 0 of unperturbed initial data. Assume that (P0) is a solvable problem. Since
D 0 z D : A0 z = h 0 , g i0 z 0, i = 1, , m
f z , Az h , g z z Z ,
where z 0 is a solution to (P). It is well-known that every such Kuhn-Tucker vector , is the same as a solution to the dual problem (3), and combined with z 0 constitutes a saddle point of the Lagrangian
L z , , , z , , D H Rm .
3. Dual Regularization Algorithm In this section we consider dual regularization algorithm for solving Problem (P) which is a stable with respect to perturbations of its input data. Copyright © 2012 SciRes.
, A , h , gi
zD
0
f z f 0 z C 1 z
0 lim , inf f z , , if D = .
,i = 1, , m ,
where 0, 0 characterizes the error in initial data and 0 > 0 is a fixed scalar. Assume that
f
f 0 f 0 , A0 , h0 , gi0 ,i = 1, , m
and the dual problem
is a convex and closed set, we denote the unique solution of Problem (P0) in the case of strongly convex f 0 by z 0 . The same notation we leave for solutions of Problem (P0) in the case of convex f 0 also. The construction of the dual algorithm for Problem (P0) relies heavily on the concept of a minimizing approximate solution in the sense of J. Warga [8]. Recall that a minimizing approximate solution for this problem is a sequence z i , i = 1, 2, , such that
f 0 zi 0 i , 0 i
and nonnegative scalar sequences i , i , where z i D i = 1, 2, , converge to zero. Here 0 is the geAM
M. I. SUMIN
1337
neralized infimum for Problem (P0) defined in Section 2, and
D 0 z D : A0 z h0 , gi0 z , i = 1, , m .
It is obvious that, in general , where is the classical value of the problem. However, for Problem (P0) defined above, we have 0 = 00 . Also, we can assert that every minimizing approximate solution z i , i = 1, 2, of Problem (P0) obeys the limit relation 0
0 0
0 0
f 0 z i min f 0 z , i , z D 0
both in the case of convex f 0 and in the case of strongly convex f 0 . Since the initial data are given approximately, instead of (P0) we have the family of problems
0, 0, 0 is fulfilled.
3.2. Dual Regularization in the Case of a Strongly Convex Objective Functional In this subsection we formulate the convergence theorem of dual regularization method for the case of strongly convex objective functional of Problem (P0). The proof of this theorem can be found in [7]. Theorem 3.1. Let the objective functional of Problem (P) is strongly convex and the consistency condition (5) be fulfilled. Then, regardless of whether or not the Kuhn-Tucker vector of Problem (P0) exists, it holds that
, , ,
f z inf , A z = h ,
gi z 0, i = 1, , m, z D, P
,
A z
zD
If the functional f is strongly convex, then due to strong convexity of the continuous Lagrange functional L z , , , z D , for all , H Rm , where
, is attained at a unique element , argminL z, , , z D .
If D is a bounded set, then obviously the dual functional V is defined and finite for all elements , H R m . When the functional f is convex, in the last case the value V , is attained at elements of the non-empty set
Argmin L z , , , z D .
,
, , ,
, ( ) , , h , g z , , ,
Along with the above relations, it holds that
f 0 z 0
,
A z 0 0
,
,
f 0 z 0 , 0,
,
,
h0 , g 0 z 0 , , ,
and, as a consequence, lim V
0
0
=
sup
,
,
,
, H R
m
on the set H R . Assume that the consistency condition
V 0 , = f 0 z0 .
, H Rm 2
0, 0
2
Copyright © 2012 SciRes.
,
R , ,
m
,
, , ,
the unique point that Denote by , furnishes the maximum to the functional
V , ,
0, 0.
R x = x1 , , xm R :xi 0,i = 1, , m , m
,
0, 0, 0,
V , inf L z , , , H , R m .
z
and the concave dual functional (value functional)
0,
, , gi0 z , ,
, g z , z D, H , R m ,
the value V
2
, , A0 z , h0 0,
L z , , f z , A z h
, , f 0 z , f 0 z 0 , 0,
depending on the “error” . Define the Lagrange functional
m
(5)
(6)
If the dual problem is solvable, then it also holds that
,
,
,
, , 0, 0
0
where 0 , 0 H Rm is the solution to the dual problem with minimal norm. If the strongly convex functional f 0 is subdifferAM
M. I. SUMIN
1338
entiable ( in the sense of convex analysis ) on the set D, then it also holds that , , z , z 0 0, 0.
where
k
0, k
, 0,
k
k
,
k
, , k
,k = 0,1, 2,
k 1
, k 1
= Pr
, V k
H Rm
k
2 k k k , k
where
0
k
k
k
,
k
0 k
A z
k lim = 0, k 1 C0 , k
k 3
0, k
k
k 3
0,
k
k
k 6
z 0 k , k z 0 k , k 0, k
k
z k , k z k , k 0.
The specified circumstances allow us to transform Theorem 3.1. into the following statement. Theorem 3.2. Let the objective functional of Problem (P0) is strongly convex, the set D is bounded and the consistency conditions (8) be fulfilled. Then, regardless of whether or not the Kuhn-Tucker vector of Problem (P0) exists, it holds that
k
k
A0 z k , k h0 0,
k
k
0, k k = . k =1
k
k
k
, k , A z k , k h , g
k
z
k
k , k
0, k 0, k .
Along with the above relations, it holds that
f 0 z 0 k , k f 0 z 0 ,
(8)
The existence of sequences k and k , k = 1, 2, , satisfying relations (8) is easy to verify. For example, we 1 5 3 can use k = k 1 6 and k = k . Then, as it is shown in [4], the limit relations Copyright © 2012 SciRes.
k
i = 1, , m, k ,
k 0, k > 0, k > 0,
k
k
gi0 z k , k k , k 0, k 0,
and the sequences k , k , k , k = 1, 2, , obey the consistency conditions
k 1
0 0
k
k
(7)
V , = A z , h , g z , ,
k
(10)
, A z , 0, k
gi0 z k , k gi0 z 0 k , k 0,
, 0 H Rm ,
k
f 0 z [ k , k f 0 z 0 , k 0, k ,
, k = 0,1, 2,,
k
f 0 z k , k f 0 z 0 ,
z 0 k , k z 0 0, k .
(9)
Besides, if the strongly convex functional f 0 is subdifferentiable at the points of the set D, then we have also
k
is the sequence generated by dual regularization algorithm of Theorem 3.1. in the case = 0, , = k . Here k , k = 1, 2, is an arbitrary sequence of positive numbers converging to zero. Suppose that the sequence k , k , k = 1, 2, is constructed according to the iterated rule
0,
hold and, as consequence, we have
In this subsection we formulate the convergence theorem of iterative dual regularization method for the case of strongly convex objective functional of Problem (P0). It is convenient for practical solving similar problems. The proof of this theorem can be found in [4]. We suppose here, that the set D is bounded and use the notation , 0,
k
3.3. Iterative Dual Regularization in the Case of a Strongly Convex Objective Functional
0, k
, k k , k
z k , k z 0 k , k 0, k
In other words, regardless of whether or not the dual problem is solvable, the regularized dual algorithm is regularizing one in the sense of [9].
k
k
, k , A0 z 0 k , k h0 , g 0 z 0 k , k
0, k 0, k
and, as a consequence, lim V
k 0
0
k
,k =
sup
V 0 , = f 0 z0 .
, H Rm
If the dual problem is solvable, then it also holds that AM
M. I. SUMIN
, , , k , , H R is the solution k
0
k
l s ,
0
0 0 m to the dual where problem with minimal norm. If the strongly convex functional f 0 is subdifferentiable (in the sense of convex analysis) on the set D, then it also holds that
z
k
lim
A zsj,i h , g zsj,i i s, w lim j
V ,
= clconv w lim A z h , g i
i
z :z i
i
D,
z D
where CV , is the generalized Clarke gradient of V , at , , and the limit w-lim is understood in the sense of weak convergence in H R m . Further, first of all we can write the inequality
I1 , I 2 2 , , , ,
, '
0 ' , '
,
,
,
H Rm ,
for an element
I1 , I 2 V , , , . Then, taking into account Lemma 3.1. we obtain Copyright © 2012 SciRes.
0
l s ,
i s, = 1,
' , ' H Rm ,
i =1
where zsj,i D, j = 1, 2, is such sequence that
L zsj,i , , , ,
minL z , , , , , j . z D
Suppose without loss of generality that the sequence zsj,i D, j = 1, 2, converges weakly, as j , to an element zs ,i D belonging obviously to the set
M , , ,
z D :L z , , , ,
, Argmin L ( z , , :z D}. = minL z ,
,
,
zD
,
,
Due to weak lower semicontinuity of the convex continuous functionals gi , i = 1, , m and boundedness of D we obtain from (11) the following inequality
n
i
(11)
lim
L z , , inf L z , , , i
'
2 , , , , ' , ' , ( ) , , ( )
l s n ,
= CV ,
i s, 0, i = 1, , l s, ,
In this subsection we prove the convergence theorem of dual regularization method for the case of bounded D and convex objective functional of Problem (P0). Below, an important role is played by the following lemma, which provides an expression for the superdifferential of concave value function V , , , H Rm in the case of a convex objective functional and a bounded set D. Here, the superdifferential of a concave function (in the sense of convex analysis) V is understood as the subdifferential of the convex functional V taken with an opposite sign. The proof is omitted, since it can be found for a more general case in [4]. Lemma 3.1. The superdifferential (in the sense of convex analysis ) of the concave functional V , at a point , H R m is expressed by the formula
s i =1
k , k z 0 0, k .
3.4. Dual Regularization in the Case of a Convex Objective Functional
1339
i =1
2
i s n , A zs n ,i h , g zs n ,i
,
,
,
, , '
'
,
,
,
0 ' , ' H Rm ,
where s n , n = 1, 2, is some subsequence of the sequence s = 1, 2, . To justify this inequality we have to note that in the , case k > 0 for some k the limit relation
j lim g k zs ,i = g k zs ,i j
holds despite the fact that the sequence zsj,i , j = 1, 2, converges only weakly to zs ,i . This circumstance is explained by specific of Lagrangian (it is the weighed sum of functionals) and the fact that the sequence zsj,i , j = 1, 2, is a minimizing one for it. As consequence of the last inequality, we obtain l s n ,
lim
n
i =1
i s n , A zs n ,i h = 2 , , (12) AM
M. I. SUMIN
1340
l s n ,
lim
n
i =1
l s n ,
lim
n
i =1
, , A zs n ,i h , , g zs n ,i
i s n , g j zs ( n ),i = 2 j , , (13)
C1
i s n , g j z s n ,i > 0, if j
i s n ,
lim
i =1
> 0,
,
if j
g j
( z s n ,i ) 0,
i =1
(14)
= 0, j = 1, , m.
l s n ,
i =1
, ( )
i s n , A zs ( n ),i h , g ( zs ( n ),i ,
n
, , ( )
i =1
2 ,
2
i =1
,
C1
,
L z 0 , f
z 0
,
,
,
f
0
,
0
,
,g
, g z s n ,i
,
,
z
Copyright © 2012 SciRes.
,
2
,
z D
, ,
2
z 0
z D
, ,
2
,
C1 f 0 z 0 min f z 0. zD
From the last estimate it follows that 2
, ,
,
g z0 g 0 z0 .
0 2
2
2
2
2C1
, A zs n ,i h
1 z C
or
C1
, 2
0
,
z C 1
z D
or
0
2
2C1
From here, due to the estimates (4), we obtain
,
2
A z 0 h
C1 f 0 z 0 min f z
C1 f 0 z 0 min f z
,A z h
f z s n ,i
,
,
C1
C1 f 0 z 0 min f z
f 0 z 0 f z 0 f 0 z 0
, 2
, A z s n , i h
,
,
or, because of the equality (15)
C1
,
, g z s n ,i
,
i s n , f z s n ,i
, g z s n ,i
,
the following inequalities
i s n , , , A zs n ,i h
s n ,i
, g z s n ,i
or
z 0 Z 0 z D 0 : f 0 z = min f 0 z zD0
f
,
l s n ,
lim
Further, we can write for any
i s n , , , A z s n ,i h
n
(15)
,
li s n ,
0.
z
C1 f 0 z 0 f z s n ,i
C1 , C1 , C1 f 0 z 0
= 2 ,
L z s n ,i ,
,
or
In turn, the limit relations (12)-(14) and boundedness of D lead to the equality lim
C1
l s n ,
l s n , n
,
,
1 z 0 2
2
2C1 2C12 2 8 K 4
,
where
K min f z f 0 z 0 C1 . z D
Thus, we derive the following limit relations AM
M. I. SUMIN
, 0, , 0, 0.
l s n ,
(16)
liminf n
The limit relations (12)-(14), (16) give the possibility to write l s n ,
lim
n
i s n , A zs n ,i h 0, 0,
i =1
l s n ,
i =1
,
,
,
f 0 z
l s n , g j i s n , zs n ,i i =1 l s n , i =1
i =1
or l s n , liminf f 0 i s n , z s n ,i i =1 n
i s n , zs n ,i , n = 1, 2, .
, g z0 .
f 0 z 0 , 0, 0
liminf
i s n , g j zs n ,i
l s n ,
i s n , f 0 z s n ,i
n
Then, because of the limit relations (17) and the obvious inequality
,
i s n , f 0 z s n ,i
n
li s n , i =1
, A z 0 h
l s n ,
any weak limit point of the sequence
,
liminf
Further, let us denote by
i =1
From the last limit relation, the consistency condition (5), the estimate (4) and boundedness of D we obtain
, 0, 0, j = 1, , m. z M
i s n , f ( zs ( n ),i )
f (z0 )
i s n , g j z s ,i
lim
n
(17)
1341
i =1
f 0 z 0 , 0, 0.
Thus, due to boundedness of D and weak lower semicontinuity of f 0 , g 0j we constructed the family of , , depending on elements z M , such that
A0 z h0 0, g 0j z , 0, 0, j = 1, , m,
we obtain A z h 0, g j z lim
l s n ,
n
and simultaneously
i s n , g j z s n ,i
i =1
, 0,
and, as consequence, due to boundedness of D A0 z h0 0, g 0j z , 0, 0.
Simultaneously, since
z s n ,i M
,
,
,
and the inequality
f z s n ,i
,
, g z s n ,i
f z
,
,
,
,g
, A zs n ,i h
, A z h
z
z D,
holds, we can write for any z 0 Z 0 due to the limit relation (15) Copyright © 2012 SciRes.
f 0 z min f 0 z , 0, z D 0
where any weak limit point of any sequence z k , k = 1, 2, , is obviously a solution of Problem (P0). Along with the construction of a minimizing sequence for the original Problem (P0), the dual algorithm under discussion produces a maximizing sequence for the corresponding dual problem. We show that the family
,
,
,
is the maximizing one for the dual problem, i.e. the limit relation
V0
,
,
,
sup V 0 ,
(18)
, H Rm
holds. First of all, note that due to boundedness of D the evident estimate V , V 0 , C 1 ,
(19)
is true with a constant C > 0 which depends on sup z zD
AM
M. I. SUMIN
1342
but not depends on , , H R m . Since
V
,
,
( )
,
,
is fulfilled for some l > 0 . Since
2
sup
2
,
V 0 ,
, H Rm
V , ( ) , H Rm , 2
2
,
,
,
V0
V 0 ,
2
,
2
2
=V
,
,
,
,
,
V 0
V
,
,
,
,
,
,
,
V
,
C 1
,
,
,
,
2
,
,
,
,
,
zD , H R m
f 0 z z
,
V 0 ,
sup , H Rm :
(20)
M , M
,
, A z
h , g z
0, 0.
,
s , s
Theorem 3.3. Let the objective functional of Problem (P0) is convex and the consistency condition (5) be fulfilled. Then, regardless of whether or not the Kuhn-Tucker vector of Problem (P0) exists, it holds for some
z Argmin L z , , , , :z D
M
sup V 0 , l , s = 1, 2,
Copyright © 2012 SciRes.
,
Z Argmin f 0 z :z D 0 ,
holds. Let us, suppose now that the limit relation (18) is not true. Then there exists such convergent to zero a sequence s , s = 1, 2, that the inequality
, H Rm
,
Z , Argmin L z , , :z D .
, H Rm : M , M
min, z D 0 , > 0 ,
So, as a result of this subsection, the following theorem holds. To formulate it, introduce beforehand the notations
,
2
with strongly convex objective functional. Writing this duality relation and passing to the limit as 0 we get because of boundedness of D the duality relation (21). In turn, from the duality relation (21), the estimate (19) and the limit relation (18) we deduce the limit relation
s , s
(21)
0 max L z , ,
= min
2
From here, we deduce, due to the consistency condition (5) and limit relations (16), that for any fixed M > 0 and for any fixed > 0 there exists such > 0 for which the estimate
V0
2
z D 0
C 1 , H Rm .
, l
M , M
= min f 0 z
C 1 , ,
V
for Problem (P0) holds. Indeed, similar duality relation is valid due to Theorem 3.1. (see relation (6)) for the problem
V 0 , , ( )
V0
sup
0
, H Rm
whence we obtain V0
s , s
sup V 0 ,
,
,
,
takes a place. This estimate contradicts to obtained above estimate (20). The last contradiction proves correctness of the limit relation (18). At last, we can assert that the duality relation
C 1 | |2 ,
V0
s , s
, H Rm :
,
M , M
for M , we deduce from the last estimate that for all sufficiently large positive M the inequality
we can write, thanks to (19), the estimates V
V 0 , 0,
sup , H Rm :
,
,
,
that AM
M. I. SUMIN
, , , 0, f 0 z min f 0 z , 0,
(22)
z D 0
,
, A z
h ,g
z
A
k
k
, k , k
k
z k , k h , g
k
z
k
k , k
(24)
are fulfilled, and the sequence
0, 0, 0, ,
0, k ,
A0 z h0 0, gi0 z ,
,
1343
k
0, 0.
z k , k , k = 1, 2, ,
Along with the above relations, it holds that
is bounded. Moreover, the latter sequence is the desired minimizing approximate solution to Problem (P0); that is,
z k , k z 0 , k .
lim V
0
0
=
sup
,
,
,
V 0 , = min f 0 z .
k
(23)
zD0
, H Rm
At the same time, the limit relations
,
,
,
where
0
, , 0,
A z
0
0
, 0 H Rm
At first, in this section we give the formulation of the stable sequential Kuhn-Tucker theorem for the case of strongly convex objective functional. Next we prove the corresponding theorem in a form of iterated process in the same case and, at last, prove the theorem in the case of the convex objective functional.
4.1. The Stable Kuhn-Tucker Theorem in the Case of a Strongly Convex Objective Functional Below the formulation of the stable sequential KuhnTucker theorem for the case of strongly convex objective functional is given. The proof of this theorem can be found in [7]. Theorem 4.1. Assume that f 0 : D R1 is a continuous strongly convex subdifferentiable functional. For a bounded minimizing approximate solution to Problem (P0) to exist (and, hence, to converge strongly to z 0 ), it is necessary that there exists a sequence of dual variables
k
,
k
H R
m
, k = 1, 2, ,
k
, k ,
0 0
is the solution to the dual problem with minimal norm.
4. The Stable Sequential Kuhn-Tucker Theorem
f 0 z 0 k , k f 0 z 0 ,
If the dual problem is solvable, then it also holds that
k , k h0 , g 0 z 0 k , k
(25)
0, k
are also valid; as a consequence,
V 0 k , k
is fulfilled. The points chosen as the points
,
,
,
k
V 0 ' , ' ,
sup
' , ' H Rm
(26)
, k , k = 1, 2, , may be
,
k = 1, 2,
from Theorem 3.1. for = k , where k 0 , k 0, k . Conversely, for a minimizing approximate solution to Problem (P0) to exist, it is sufficient that there exists a sequence of dual variables
k
, k H Rm , k = 1, 2,
such that
k k , k 0 , k , the limit relations (25) are fulfilled, and the sequence k
z k , k , k = 1, 2, ,
is bounded. Moreover, the latter sequence is the desired minimizing approximate solution to Problem (P0); that is, k
such that
k k , k 0 , k , the limit relations k
k
z k , k D 0 , k 0, Copyright © 2012 SciRes.
z k , k z 0 , k .
If in addition the limit relations (25) are fulfilled, then (26) is also valid. Simultaneously, every weak limit point of the sequence
k
, k H Rm , k = 1, 2, , AM
M. I. SUMIN
1344
is a maximizer of the dual problem
k
z k , k z 0 , k .
V 0 , max, , H Rm .
4.2. The Stable Kuhn-Tucker Theorem in a Form of Iterated Process in the Case of a Strongly Convex Objective Functional In this subsection we prove the stable sequential KuhnTucker theorem in a form of iterated process for the case of strongly convex objective functional. Note that the regularizing stopping rule for this iterated process in the case when the input data of the optimization problem are specified with a fixed (finite) error > 0 can be found in [4]. Theorem 4.2. Assume that the set D is bounded and f0: D → R1 is a continuous strongly convex subdifferentiable functional. For a minimizing approximate solution to Problem (P0) to exist ( and, hence, to converge strongly to z 0 ) , it is necessary that for the sequence of dual variables k , k H Rm , k = 1, 2, , generated by iterated process (7) with the consistency conditions (8) the limit relations
z
k
, D
A
k
k
k
k
0 k
, 0, k
the boundedness of the sequence k
z k , k , k = 1, 2, ,
and the conditions imposed on the initial data of Problem (P0). Hence due to asserted in Subsection 3.3 there exists the sequence
k
k
z k , k h , g
k
0, k
, 0,
k
z
k
k , k
(27)
k
,
k
, , k
k
generated by dual regularization algorithm of Subsection 2.2 and, as consequence, the sequence
,k ,
k
z k , k D 0 ,
k
Simultaneously, the limit relation (28) is valid. Proof. To prove the necessity we first observe that Problem (P0) is solvable because of the conditions on its input data and existence of minimizing approximate solution. Now, the limit relations (27), (28) of the present theorem follow from Theorem 3.2. Further, to prove the sufficiency, we first can observe that Problem (P0) is solvable in view of the inclusion
k
, k , k = 1, 2, ,
generated by iterated process (7) with the consistency conditions (8), obey the limit relations (9), (10) and (28). Thus, the sequence k
z k , k , k = 1, 2,
0, k , k
are fulfilled. In this case the sequence z k , k is the desired minimizing approximate solution to Problem (P0); that is, k
z k , k z 0 , k .
Simultaneously, the limit relation
V 0 k ,k
sup
V 0 ' , ' ,
' , ' H Rm
(28)
is fulfilled. Conversely, for a minimizing approximate solution to Problem (P0) to exist, it is sufficient that for the sequence of dual variables
k
, k H Rm , k = 1, 2, ,
generated by iterated process (7) with the consistency conditions (8), the limit relations (P0) are fulfilled. Moreover, the sequence z
k
k , k , k = 1, 2, ,
is the desired minimizing approximate solution to Problem (P0); that is, Copyright © 2012 SciRes.
is the desired minimizing approximate solution to Problem (P0).
4.3. The stable Kuhn-Tucker Theorem in the Case of a Convex Objective Functional In this subsection we prove the stable sequential KuhnTucker theorem in the case of the convex objective functional. Theorem 4.3. Assume that the set D is bounded and f 0 : D R1 is a continuous convex functional. For a minimizing approximate solution to problem (P0) to exist (and, hence, every its weak limit point belongs to Z*), it is necessary and sufficient that there exists a sequence of dual variables
k
, k H Rm , k = 1, 2, ,
such that
k k , k 0 , k , and the relations k
k
z k , k D 0 , k 0, k ,
(29)
AM
M. I. SUMIN
k
V
, k ,
k
1345
k
k
A z k , k h , g
k
k
z k , k
(30)
= f
k
z
k
k
k , k , k = 1, 2, ,
is the desired minimizing approximate solution, and every weak limit point of this sequence is a solution to Problem (P0). As a consequence of the limit relations (29), (30) the limit relation
V 0 ,
sup
(31)
, H Rm
holds. Simultaneously, every weak limit point of the sequence
k
V
and z k respectively. Further, due to the estimate (19) and the limit relation
k k , k 0 , k , we have
, k V 0 k , k 0, k .
Then, taking into account the equality (see (23)) sup
V
0
, H Rm
, = min0 f z , 0
z D
the limit relation (30) and obtained limit relation
k
f 0 z k , k f 0 z 0 , k
(see (22)) we can write Copyright © 2012 SciRes.
k , k
and, as consequence, the limit relation (31) is valid. So, we have shown that the limit relation (31) is a consequence of the limit relations (29), (30). Now, let the sequence
k
, k H Rm , k = 1, 2, ,
be bounded. Then, since the concave continuous functional V0 is weakly upper semicontinuous, every weak limit point of this sequence is a maximizer of the dual problem V 0 , max, , H Rm . Now we prove the sufficiency. We first observe that the set
.
k , k k , k , , k = 1, 2,
k
k
k
is nonempty in view of the inclusion
, max, , H R
z
k
k
k
k
Z k , k D 0 ,
Proof. To prove the necessity, we first observe that Problem (P0) is solvable, i.e. Z , because of the conditions on its input data and existence of minimizing approximate solutions. Now, the first two limit relations (29), (30) of the present theorem follow from Theorem k 3.3. if k , k and z k , k are chosen as the points
V
k
z k , k h , g
m
,
k
, k H Rm , k = 1, 2, ,
is a maximizer of the dual problem 0
k , k
f 0 z0 ,
Moreover, the sequence
V 0 k , k
k
z k , k Z k , k .
z
, k
k
A
hold for some elements k
k
,
0, k .
k
k
k
k
Z k , k D 0 ,
the boundedness of the sequence k
z k , k , k = 1, 2, ,
and the conditions imposed on the initial data of Problem (P0) is solvable. Furthermore, since (P0). Hence, problem k k k the point z k , is a minimizer of the Lagrange functional L , k , k , we can write f
k
z
k
k
A
k
f
k
, k
k
, k , k
k
z k , k h , g
z k , k , A
k
k
z k
k
z h , g
k , k k
z
z D.
By the hypotheses of the theorem, this implies that f f
k
k
z
k
k , k
z k , k , A
k
k
z h , g
k
z k
z D, k 0, k .
We set z = z 0 Z in this inequality and use the AM
M. I. SUMIN
1346
consistency condition
k
z k , k , k = 1, 2,
k k , k 0 , k , to obtain
k
is a minimizing approximate solution of Problem (P0).
5. Possible Applications of the Stable Sequential Kuhn-Tucker Theorem
f 0 z k , k f 0 z 0 k , k 0, k .
Below in this section we consider two illustrative examples connected with possible applications of the results obtained in the previous sections. Its main purpose is to show principal possibilities of using various variants of the stable sequential Kuhn-Tucker theorem for solving optimal control and inverse problems.
In addition we have k
k
z k , k D 0 .
Using the classical properties of the weak compactness of a convex closed bounded set and the weak lower semicontinuity of a convex continuous functional, we easily deduce from the above facts that f
0
z
k
, f k
k
0
5.1. Application in Optimal Control
z , k , 0
First of all we consider the optimal control problem with fixed time and with functional equality and inequality constraints
i.e. the sequence f u
1
0
A0 t x u t , x u t B0 t u t , u t dt min, u D, POC
ai u ci t , x u t dt di t , u t dt = 0, i = 1, , k , gi u 0
1
1
1
0
0
Ai t x u t , x u t Bi t u t , u t dt 0, i = 1, , m,
A u A u a1 u , , ak u ,
g u g1 u , , g m u , 0, 0 . Here and below, 0 is a number characterising an error of initial data, 0 > 0 is a fixed number, f :L2 0,1 R1 is a convex functional, gi :L2 0,1 R1 ,i = 1, , m
and 0, 0 x u t K u D, 0, 0 ,t 0,1 . Assume that Ai Ai0
, Bi Bi0
L2 0,1
ci ci0
L2 0,1
L2 0,1
, i = 0,1, , m,
, di di0 , A A0
are convex functionals, Ai ,i = 0,1, , m, A : 0,1 R
n n
(32)
,
, B B
Bi ,i = 0,1, , m, B : 0,1 R nl
L2 0,1 L2 0,1
0 L2 0,1
,
are Lebesgue measurable and uniformly bounded with respect to 0, 0 matrices,
whence we obtain due to the estimate (32) for some constant L > 0
ci : 0,1 R n , di : 0,1 R l , i = 1, , k
f u f 0 u L ,
are Lebesgue measurable and uniformly bounded with respect to 0, 0 vectors,
A u A0 u L ,
D u L2 0,1 :u t U for a.e. t 0,1 , U R l is a convex compact set, x u t , t 0,1 is a solution to the Cauchy problem x = A t x B t u t , x 0 = x0 R n , t 0,1 .
Obviously, for each control u D this Cauchy problem has a unique solution x u t , t 0,1 and all these solutions are uniformly bounded with respect to u D Copyright © 2012 SciRes.
g u g 0 u L u D. Define the Lagrange functional L u, , f u , A u , g u , u D and the concave dual functional V , inf L u, , , R k , Rm . uD
Define also the notations AM
M. I. SUMIN
U Argmin f 0 u :u D 0 ,
The points points
U , Argmin L u , , :u D .
Here and below, we leave the notation D in Section 1.
Let
,
,
,
V
,
2
2
m
that maximizes on
sup, , R
k
R , m
0, 0, 0.
,
R R , k = 1, 2, , , 0 , k ,
k
k
such that
k
k
m
k
and the relations k
k
u k , k D 0 , k 0, k ,
k
k
k
, k , A u k , k , g
k
u
k
k , k
(33)
(34)
hold for some elements u
, U k
k
k
, . k
k
u k , k , k = 1, 2, ,
is the desired minimizing approximate solution, and every weak limit point of this sequence is a solution to 0 Problem POC . As a consequence of the limit relations (33), (34) the limit relation
V 0 k , k
sup
V 0 ,
, R k Rm
holds. Simultaneously, every limit point of the sequence
k
where k > 0 , k 0, k . In conclusion of this subsection, we note that the Pontryagin maximum principle can be used for finding optimal elements u D in the problem
L u, k , k min, u D.
(35)
Denote , A t x B t u
A0 t x, x B0 t u , u
, C t x D t u
, A1 t x, x B1 t u, u ,
, , Am t x, x Bm t u , u
, k R k Rm , k = 1, 2, ,
is a maximizer of the dual problem V 0 , max, , R k Rm . Copyright © 2012 SciRes.
,
where C t is n n matrix with lines ci t ,i = 1, , n , D t is n m matrix with lines di t ,i = 1, , n . Then, due to convexity of the problem (35) we can assert that any its minimizer u satisfies the following maximum principle. Theorem 5.2. The maximum relation
H t , x u t , u t , t , k , k
for a.e. t 0,1 , k
Moreover, the sequence
, k , k = 1, 2, , may be chosen as the
= maxH t , x u t , v, t , k , k vU
0, k k
k
H t , x, u , , ,
Applying in this situation, for example in the case of convexity of f , Theorem 4.3 we obtain the following result. Theorem 5.1. For a minimizing approximate solution 0 to Problem POC to exist (and, hence, every its weak limit point belongs to U ), it is necessary and sufficient that there exists a sequence of dual variables k
k , k k , k , , k = 1, 2, ,
accepting
denote the unique point in R R this set the functional k
1347
holds for the Lagrange multipliers k , k R k Rm , where t , t 0,1 is the solution of the adjoint problem
= x H t , x u t , u t , , k , k , 1 = 0.
5.2. Application in Ill-Posed Inverse Problems Now we consider the illustrative example of the ill-posed inverse problem of final observation for a linear parabolic equation in the divergent form for recovering a distributed right-hand side of the equation, initial function, and boundary function on the side surface of the cylindrical domain for the third boundary value problem. Here we study the simplified inverse problem with a view of compact presentation. Similar but more general inverse problem may be found in [10]. Let U R m , V R1 and W R1 be convex AM
M. I. SUMIN
1348
fixed-time optimal control problem on finding a minimum-norm control triple with strongly convex objective functional and a semi-state equality constraint
compacts, QT 0, T , S ,
ST x, t :x S , t 0, T ,
f 0 inf , A0 = h0 ,
be a bounded domain in R n , D D1 D2 D3 ,
D1 u L QT :u x, t U a.e. on QT
D L2 QT L2 L2 ST Z ,
h0 H L2 , PIP0
,
D2 v L :v x V a.e. on ,
where
D3 w L ST :w x, t W a.e. on ST .
f 0 u
Let us consider inverse problem of finding a triple π u , v, w D of unknown distributed, initial, and boundary coefficients for the third boundary value problem for the following linear parabolic equation of the divergent form zt
ai , j x, t z x j bi x, t z xi xi
2 2,QT
v
2 2,
w
2 2, ST
A0 z 0 , T .
The input data for the inverse problem (and, hence, for are assumed to meet the following Problem PIP0 conditions: 1) functions
ai , j ,bi , a : 0, T R1 ,i, j = 1, , n ,
(36)
g : 0, T R m
a x, t z g x, t , u x, t = 0,
are Lebesgue measurable; 2) the estimates
z x, 0 = v x , x ,
ai , j x, t i j 2
z x, t z = w x, t , N
2
for a.e. x, t QT , , > 0,
x, t ST ,
bi x, t , a x, t , g x, t K
determined by a final observation
for a.e, x, t QT , x, t K
h0 = z 0 , T L2 ,
for a.e. x, t ST ,
whose value is known approximately, at a certain T . Here, similar to [11], z x, t N
ai , j x, t z x j x, t cos i x, t
and i x, t is the angle between the external normal to S and the xi axis, and 0 is a number characterising an error of initial data, 0 > 0 is a fixed number. The solution z to the boundary value problem (36) corresponding to the desired actions π u , v, w D is a weak solution in the sense of the class V21,0 QT [11]. Clearly, the solution to the inverse problem in such a statement may be not unique. Therefore, we will be interested in finding a normal solution, i.e., a solution with the minimal norm u
2 2,QT
v
2 2,
w
2 2, ST
,
which we denote by 0 . It is easy to see that the above-formulated inverse problem of finding the normal solution by a given observation h0 L2 is equivalent to the following Copyright © 2012 SciRes.
,
hold, where K > 0 is a constant not depending on 0, 0 ; 3) the boundary S is piece-wise smooth. Denote by h L2 approximate final observation (with parameter) and assume that g g 0
,QT
, h h0
2,
.
(37)
From conditions 1) - 3) and the theorem on the existence of a weak (generalized) solution of the third boundary value problem for a linear parabolic equation of the divergent type [11], (Chapter III, Section 5), [12] it follows that the direct problem (36) is uniquely solvable in V21,0 QT for any triple u , v, w Z and any T > 0 and besides a priory estimate z
QT
z
2, ST
C u
2,QT
v
2,
w
2, ST
is true, where a constant C > 0 not depends on 0, 0 . The last facts together with the estimates (37) lead to corresponding necessary estimate for deviation of perturbed linear bounded operator A :Z H , AM
M. I. SUMIN
A z , T ,
from its unperturbed analog (details may be found in [10]) A A0 C 1 z z Z
V p0, r max, H .
L , f 0 , A h , D
with the minimizer and the concave dual functional V inf L , , H = L2 . D
,
Let denotes the unique point in H L2 that maximizes on this set the functional 2
Besides, the limit relation (39) is also valid. Simultaneously, every weak limit point of the sequence k H , k = 1, 2, , is a maximizer of the dual problem The points k , k = 1, 2, , may be chosen as the points
where a constant C > 0 not depends on . Define the Lagrange functional
V
1349
sup, H ,
D k
, 0,
(38) k
k
0, k ,
, k .
At the same time, the limit relation
' H
(39)
is fulfilled. Conversely, for a minimizing approximate solution to problem PIP0 to exist, it is sufficient that there exists a sequence of dual variable k H , k = 1, 2, such that
k k 0 , k , the limit relations (38) are fulfilled. Moreover, the latter sequence is the desired to Problem PIP0 ; that is,
k
k 0 , k . Copyright © 2012 SciRes.
2 ), k
k
k
k
(40)
k
k D 0 , k 0, k
k
k , A k h
k
0, k ,
(41)
are fulfilled. In this case the sequence k
k is the desired minimizing approximate solution to Problem PIP0 ; that is,
k 0 , k . Simultaneously, the limit relation
V 0 k supV 0 ' (42)
0
V 0 k supV 0 ' ,
k
k
are fulfilled. Moreover, the latter sequence is the desired minimizing approximate solution to Problem PIP0 ; that is, k
k
with the consistency conditions (8) the limit relations
k
k , A k h
k
k
k = 0,1, 2, , 0 H ,
0 k
k
k 1 = k k A h
Applying Theorem 4.1. in this situation of strong convexity of f 0 we obtain the following result. Theorem 5.3. For a bounded minimizing approximate solution to Problem PIP0 to exist (and, hence, to converge strongly to 0 as k ), it is necessary that there exists a sequence of dual variable k H , k = 1, 2, , such that k k 0 , k , the limit relations k
k = 1, 2, ,
where k > 0 , k 0, k . Since the set D is bounded in Problem PIP0 , we can apply for solving our inverse problem the regularized Kuhn-Tucker theorem in a form of iterated process also. Thus, Theorem 4.2. leads us to the following theorem. Theorem 5.4. For a minimizing approximate solution to Problem PIP0 to exist (and, hence, to converge strongly to π0 ), it is necessary that for the sequence of dual variable k H , k = 1, 2, , generated by iterated process
0, 0, 0.
k
,
k , k
' H
is fulfilled. Conversely, for a minimizing approximate solution to Problem PIP0 to exist, it is sufficient that for the sequence of dual variable k H , k = 1, 2, , generated by iterated process (40) with the consistency conditions (8), the limit relations (41) are fulfilled. Moreover, the sequence
k
k , k = 1, 2, , is the desired minimizing approximate solution to Problem PIP0 ; that is,
k
k 0 , k .
Simultaneously, the limit relation (42) is valid. AM
M. I. SUMIN
1350
REFERENCES
[7]
M. I. Sumin, “Regularized Parametric Kuhn-Tucker Theorem in a Hilbert Space,” Journal of Computational Mathematics and Mathematical Physics, Vol. 51, No. 9, 2011, pp. 1489-1509. doi:10.1134/S0965542511090156
[1]
V. M. Alekseev, V. M. Tikhomirov and S. V. Fomin, “Optimal Control,” Nauka, Moscow, 1979.
[2]
F. P. Vasil’ev, “Optimization Methods,” Faktorial Press, Moscow, 2002.
[8]
[3]
M. Minoux, “Mathematical Programming. Theory and Algorithms,” Wiley, New York, 1986.
J. Warga, “Optimal Control of Differential and Functional Equations,” Academic Press, New York, 1972.
[9]
[4]
M. I. Sumin, “Duality-Based Regularization in a Linear Convex Mathematical Programming Problem,” Journal of Computational Mathematics and Mathematical Physics, Vol. 47, No. 4, 2007, pp. 579-600. doi:10.1134/S0965542507040045
A. N. Tikhonov and V. Ya Arsenin, “Methods of Solution of Ill-Posed Problems,” Halsted, New York, 1977.
[5]
[6]
M. I. Sumin, “Ill-Posed Problems and Solution Methods. Materials to Lectures for Students of Older Years. TextBook,” Lobachevskii State University of Nizhnii Novgorod, Nizhnii Novgorod, 2009. M. I. Sumin, “Parametric Dual Regularization in a Linear-Convex Mathematical Programming,” In: R. F. Linton and T. B. Carrol, Jr., Eds., Computational Optimization: New Research Developments, Nova Science Publishers Inc., New-York, 2010, pp. 265-311.
Copyright © 2012 SciRes.
[10] M. I. Sumin, “A Regularized Gradient Dual Method for Solving Inverse Problem of Final Observation for a Parabolic Equation,” Journal of Computational Mathematics and Mathematical Physics, Vol. 44, No. 12, 2004, pp. 1903-1921. [11] O. A. Ladyzhenskaya, V. A. Solonnikov and N. N. Ural’tseva, “Linear and Quasilinear Equations of Parabolic Type,” Nauka, Moscow, 1967. [12] V. I. Plotnikov, “Uniqueness and Existence Theorems and a Priori Properties of Generalized Solutions,” Doklady Akademii Nauk SSSR, Vol. 165, No. 1, 1965, pp. 33-35.
AM
