A deep cut ellipsoid algorithm and q uasiconvex ...

A deep cut ellipx

A deep cut ellipsoid algorithm and q uasiconvex ptogramrning J.G.B. Frenk, J. Gromichol , F. Plastria and S. Zhang EconometricInstitute, ErasmusUniversity,Rotterdam,The Netherlands Center for Industrial Location, Universityof Brussels,Belgium Departmentof Econometrics,University of Groningen,The Netherlands

I I I I

t

In this paper we describe a deep cut version of the ellipsoid algorithm and introduce a class of functions and their corresponding finite dimensional optimization problems to which it can be applied. Moreover, we show that an important subset of the quasiconvexfunctions belong to the above mentioned class.

1. fntroduction

l I il

I

2. A deep cut elli

One frequently encountersin engineering,economicsand management science finite dimensional convex optimization problems. A representation of such a problem is given by inf{/e(c):f;(a)(0, f - t,...,m,xeIRn }

(p)

with /d : IRn - IR, i = 0,. . .,m, denoting a set of conuerfunctions. For these conaet programmincproblems a lot of properties are known. Among them we mention the availability of necessaryand sufficient Kuhn-Tucker optimality conditions and duality results (cf. [1, 14]). Exploiting these properties one might presume that many efficient algorithms to solve general instances of (P) should exist in the literature. However, it turns out that a large subclass of these algorithms only deal with special instances of (P). A well-known example is the simplex method for linear programming. On the other hand, efficient algorithms which can be applied to general instances of (P) a.ssume in most casesthat the objectiuefunction /e and the conslraint functions f;, i - L, . . .,rn, are differentiable (cf. [4, 10]). These algorithms can roughly be divided into interior and exterior point methods (cf. [4, 6]). An exception is given by a cutting plane method proposed by Plastria (cf. [13]). Thismethrrd generalizesthe well-known convex cutting plane algorithm of Kelley (cf. [tO]) rAuthor on leave from D.E.I.O. (Universidade de Lisboa, Portugal). This research was supported by J.N.LC,T. (Portugal) under contract number BD/631/90-RM.

I I

and can be applied to a tion problems. Also ba we now propose in this 1 considered in [13] whicl This algorithm, explain, ellipsoid method and ex casediscussedin [f0] an though no theoretical cc versions of this algoritt optimization problems. results in [12], an impor this algorithm can be a result in Section 3 showr the execution ofthe algc the cutting plane algorit above subclass of quasic the well-known steepes

To introduce the set of (P) should belong in orc to introduce the followi center 0 is defined by

..: wrth ll . ll representlng arbitrary function / : ^8 set of level pl is given by L As in [12] the following Definition 2.1 A subdifferentiableal c e

to t for eueryg belonging if f is lower subdiffere calledlowersubgradie

programming deep cut ellipsoid algorithm and quasiconvex

andcanbeappliedtoamuchlargerclassthantheclassofconvexoptimiza.q.to based on coistructing socalled separatinghyperplanes iir" pt"lr"*". the one proposein this paper an algorithm completelydifferent-from ;; problems' ";, optimization of class same the ir, 1rr1which can solve "ofi"r"a in detail in Section2, is a deep cut version of the ifrJrfg*ithrrr, "*pl.i*d extendsa similar method for the unconstrainedconvex ;G;fi;"trt"a ""a f.t the constrainedconvexcasediscussedin [8]' Alin [16] ,*iiir"o.r"d ""a related in no theoreiicut .orru",g""ce propertiesare reported' also [5] ;;dh versionsofthisalgorithmu,""t"'t"aandappliedtoa-setofdifferentiable extending related opii*irotio' probl-"*r. I'i,,utly, in Section i we identify' ,"*tt,in[rz],animportantsubclassofthequasiconvex.functionstowhich and [t3] the main Iti, atgoriit*.un be applied' However,contrary'to [12] neededfor result in Section3 showshow to computeaseparatinghyperplane theexecutionofthealgorithm.Assuch,thiscomputationcanalsobeusedin in [12] and [13]'-Observe'that for the the cutting plane algoiithm discussed reducesto finding abovesubcla* of qousi.onvexfunctions tlis computation the well-knownsteepestdescentdirection'

nds rds

5orithm and ensional ophow that an 'e mentioned

i * ?

2. L deeP cut elliPsoid

(P)

l

B , : = { a € I R " : l l c l l< r }

ns. For these rong them we er optimalitY roperties one I instances of arge subclass \ well-known e other hand, ,f (P) assume

his research was M.

algorithm

and constraints of To introduce the set of functions to which the objective algorithm we need (P) should belongin order to apply the deepcut ellipsoid B" of radius r and to introduce the following ,rotutiott. The Euclidean ball center0 is definedbY

ment science ion of such a

functions f;, m roughly be r exception is This method elley (cf. {101)

63

Moteover'for an with ll 'll representingthe well-knownEuclideannorm' arbitraryfunctionf:"K..EwithdomainKgR'"thestrictlowerlevel set of level P is givenbY L o p ( f: )= { n e K : f ( ' ) < P } ' As in [12] the following classof functions is introduced' g R is calledlower Definition 2.1 A functionf : I{'-* R' with K with IR at n e i{ if there erisls somed' e subilifferentiable

f(y) > /(,) + 4@ -,)

(1)

is colledlower sabdifferenliable

I

'iffor eaergg belongingto L"161U)' Moreoaer, f a € K' If the set of all sof ir"to"r"r ruiatfr"rr"i;;li;'at euerv.point by A- f @) thefunctionf d' of f al a is denoled calleillowersubgra"rli,ents

A dec

J.B.G. Ftenk et aL is called boundedly lower subdifferentiableif f is lower subdifferenliable and there edsts a constant N > 0 such that for eaery a E I{ one cenfind some d, e 0- f(a) with lld,ll < N. To relate the above definition to the (bounded) lower subdifferentiability of a function f : IRn---+ lR on a subset of its domain we introduce the restriction fis with K g n.A function fx : K--* ft is called the restriction of f ; IPc----+ IR on I{ if its domain is given by K and fi6 equals / on /(. We are now able to introduce the classesBL, and,L,. Definition 2.2 A function f : IR---...+IR belongsto (BE,) L, if and onlg if its restriction fB, is (boundedly) lower subdifrerentiable. It turns out that the set .c" describes exactly the class of objective and constraint functions to which the deep cut version of the ellipsoid method can be applied. Observe it is shown in [lZ] thar f : IR.-.R belongs to BLr if .fr, is convexfor some s ) r. Moreover, in Section B it will be proved as a corollary of the main theorem that the same result holds for a subclassof the quasiconvexfunctions. To describe the proposed algorithm we impose the following restrictions on (P). Assumption

2.3

for every gl

for every g

To conclud< ,or r n (r , ; l i € U

Based o mization pr the ellipsoi

with /0, /r For a rn introduce t vector o €

Problern (P) safisfiesthe following condilions.

1. An optimal solution a,. of (P) edsls and an upper bound r on its Euclidean norm is known, i.e. lllc.ll < r. 2 . T h e f u n c t i o n sf ; , i = 0 , . . . , f f i ; n Q j

b e l o n gt o L , w i t h r g i u e nb y I .

Observe, it is easy to verify that the set,C" (BLr) is closed under marcimization over a finite number of functions belonging to L, (BLr). We only give a proof for 4". Lemrna 2.4 The set L, is closed uniler maaimization oaer a finite number of funclions belongingto L,. P r o o f. L e t g ; e L r , i - 1 , . . . , m , b e g i v e n a n d t a k e o € B " f i x e d . B y assumption it follows for each i - 1,.. ., rn that there exists a vector d; such

that theinequalitv

sifu) > g;(a)+ dl(v- ,)

(2)

holds for every gr € B, and.Ci(A) < g;(c). Consider now 9 := ma,xt fo(r*) for every m € IN we may distinguish three different cases.

r.a^€sn,B" Since /e e L, it follows that (1) holds at o- (remember that c* € B, n Llo'^,t (/o)) ana so there exists a nonzero d- such that fo(a.1> fo(a^1+ dt^(a- - o^).

(9)

struction of a va cut as an object

2. a^4. B, h If this subca.se tion h(a) = lle

that h is conve:

equals ffi' ct"

(5) it follows thr r ) h(n.) :

and so we concl to the lower hal applying (12) a

0

For a derivation ofa deep or central valid cut with respect to /e observe the followin g. lf d^(a* - o^) ) l^ - fo(a^) it follows by (9) that - l^ .fo(c-) > fo(a^) * I* - fo(a^) and this is not possible by the definition of o*. Hence c* must belong to the lower halfspaceH-(B^):= {c € IR : dl^a S p^} with B^ := il^a^ *l* - f o(a*1 . We will no* 'n"tify whether ihe hyperpl.tt" H (B^) correspondsto a valid cut. Observeby (g), a* e E(A^;o-) and (b) that

0 1 fs(a^) - l^

S S

"ol

and hence 0(o-:=

3.o^€B"andc If this holds we function fi. Y first subcase (r

fo(a^1- fo(a.1 I d^a^ - d,^r* t*@^ -min {d^a:a € E(A^;o^)}

= ,[t^d^a* t*a^ - fr^

fo(o^) - l^

< 1

and this Yields a valid deeP cr and d :- Vh( smaller volume a* € E(A^iq

(1 1 )

o 2 /r(".) 2

with some nor the second ine

A deep cut ellipsoid algorithm and quasiconvex programming

67

implying that H(0^) is a valid cut. Clearly this is a valid deep cut whenever l* 1 fo(a^) and it can be derived using only one additional computation. Substituting q i= Qmt 0 := 0^ and d := d^ it follows by (7) and (S) that in this case a smaller volume ellipfl soid .E(A-.'riom+r) can be constructedsatisfying u* € E(A^;o-) the condiscussing we are finished and so H-(P^) 9 E(A^+tiam+r) struction of a valid cut for /s. In the remainder we will refer to such a cut as an objectiue cut. 2. a^(8, If this subcaseholds we construct a valid cut with respect to the function h(c) = ll"ll. We shall refer to such cuts as norrn culs. Observe that h is convex and its gradient Vh(o) exists for ever| c I 0 and equals Clearty by the subgradient inequality, Assumption 2.3 and ffi. (5) it follows that r > h(a*) =

Yh(o*)to* > min{Vh(o^)tt:a

e E(A^;a^)}

(r2) and so we conclude by the secondinequality in (12) that o' must belong to the lower halfspaceH-(r) 1= {a € IR :Yh(a,^)', < r}'Moreover, applying (12) again we obtain

0So^

:=

Vh(o^)ta^

@

- r

ll"*ll- "

{Yh(a^)t

A^Yh(a^)

< 1

(13)

and this yields that the hyperplane I1(r) is a valid cut. Clearly this is a valid deep cut whenever lla,-ll > r. Substituting o := a^' B := r and d := Yh(a^) it follows bv (7) and (8) that also in this case a can be constructed satisfying smaller volume ellipsoid E(A^+rian+r) a* € E(A*; a^) fr H- (P^) 9 E(A^+ri am+r). 3.o-€B"anda^45 If this holds we construct a valid deep cut with respect to the constraint function .fr. w" shall refer to such cuts as conslrainl culs. As in the first subcase (remember h € L,) we obtain

0 > "fr(c.)> f{o^) + d*(r' - a^) > h(a^1 -

dl)A^d;

€4)

with some nonzero d- (remember fi(o-) 10 < fv(a^)) and henceby the second inequality in (la) o* belongs to the lower halfspace defined

68

J.B.G.Frenket aI.

A deep cut ellil

\y n-@^) := {a, € IR : dt^aS p^} wirh f- := d^a^ - h(o^1.

Moreover, applying (fa) again we obtain 0(a-:=

=@5r

(15)

yft^t^d^

and this yields that the hyperplane H(B^) is a valid cut. clearly it is a valid deep cu-t since fi_(a_) > 0. Subsiituting e := c.mtg := g^ and d := d^ it follows by (z) and (g) that again ii this case a smaller volume ellipsoid E(A.^+t;o,m+t) can be constructed satisfying c. 6 E(A^; o^) A H- (p^) e E(A^+ri om+r). This concludes the description of the three disjoint subcases and leads to the determination of the smaller volunneellipsoid to be used in the (rn { 1)ra step. The algorithm consists now of the"following steps: S t e p 0 l e t r n : = 0 , A s : = r 2I a n d ,o o : = 0 ; Step 1 if a^ is feasible and optimal then goto Step 4 elsegoto Step 2; Step 2 if o^ f B, then apply a norm cut else if a^ ( S then apply a constraint cut

step3 update rheeuipsoid ,retm,1;'ltit ffi l*:fl: Step 4 stop.

if a^ = 0 for eoeta wheneuera*>0fm

!ll; ,,

This algorithm includesboth the central and the deepcut versions. For the just take a- := 0, for the deep cut evaluatea-centralcut according to the subcasesdiscussedabove. Moreover,it is alsopossibleto replacethe optimality checkin Step 1 by a near-optimalstoppingrulu (cf. tS]j. observe that our slightly modified (regardin! prJvious .,r"rrior* of the algorithm) step 2 has two differentimplications. on one hand it aims to improve the numerical stability of the method by trying to keep the centers of the generatedellipsoidsinsidethe boundedt"giorr-B,l on the other hand, it allowsthe introductionof functionswith ,,coniex-like" properties in some bo'aded set' This point will becomemore clearin the next section. The following convergenceresurt is given in [g] for the case of convex functions-fi ! 0,1, with the extra aszumptionihat a point exists in the interior of .9 n .B". Theorem 2.5 If the deep cut ellipsoid algorithm, wifhout aiplfyiijeither a stoppingrule or an optimaritychick,ewi.utesan infinite numberof iterationsthenlhe sequence of lowestrecorderr asrues,onorrgi, to the optimal solution aalue. Moreoaer,the conaergence is geometricat a rate ("i1:t1i "f

Given that every lower subgradient of that the above resull reader is referred to l

3. The comput

In this section we wi withdomain KgI the remainder that tl int(I{). Moreover, r lower subgradient. Definition

3.1

1. The functior J its lower leoel

pelR' 2. The functiont

if the itequalit

holdsfor eoe4

It is well-known is given by

J

for every n,y e K a In the remainder for which the directi

exists in every poinl Before proving the n of the function d r-

A deep cut ellipsoid algoilthm and quasiconvex programming

69

if a^ = 0 for euery m (central cal uersion) and, at a possibly higher rate wheneuera* ) 0 for some m (deep cut uersion) with a:=

n2-t

-[

and o i=

n+l n-l

Given that every objective cut is performed with an uniformly bounded lower subgradient of /6 one can verify copying the same proof as given in [8] that the above result still holds for /o € BL, as well. For more details the reader is referred to [8] and [7].

3. The computation In this section we will with domain K I m the remainder that the int(I{). Moreover, we lower subgradient. Definition

3.L

of a lower subgradient

discuss an important class of real-valued functions / belonging to BL,. Observe it is always assumed in domain 1( is a closedsubset with a nonempty interior also present for this class a procedure to compute a

Let f : I{-

IR be any giaenfunction.

1. The function f is called quasiconuexif K I IRn is a conr)et set and ils lower leael sel Lr(f) := {z e K : f(a) < p} is conae, for eaerg PEIR. 2. The function f is calledLipschitz conlinuous with Lipschilz constantL if lhe inequalily

lf(") - f(s)l S r,ll, - vll

holdsfor eaerye,g e K. It is well-known(cf. [17]) that an equivalentdefinition of quasiconvexity is givenby f(\n + (t - ))s) < max{/(c), f(s)} for everya,A eK and 0 < ) < 1. In the remainderof this sectionwe will only considerfunctionsf : KIR for whichthe directionalderivativeft(a;d),i.e. (a + td)- f (a) ,f , ( \a : d_. \_, r ._^ f / ; l i i t ., exists in everypoint a belongingto int(K) and in every direction d €.R". Beforeprovingthe main result we list the followingeasyverifiableproperties of the function d r- f'("; d) with a e int(/f ) fixed.

70

J.B.G. Frenk et al.

Lemma 3.2 ties.

The function d,r--+ f'(a;d)

A deep cut eIIfun

satisfiesthe following proper-

Wenow considerfrr id

1. If f : K-+ IR is Lipschitz continuous with Lipschitz constant L then the function d'-ft(a;d) is also Lipschitz continuous with the sarne Lipschitz conslant. 2. If f : K+ R is quasiconaerthen the function d *--+ f,(a;d) quasiconuer 3. The function d, t-+ ft(u;d)

is positiaely homogeneoasandf,(r;O)

is also Deffnitioa - 0.

Proof . To verify 1. we obtain by the definition of a direi:tional derivative that

f@ + tdr) : f(s' + tdz) f' (a; di - f' (r;d2) = lim ,10 t

for any dt,dz €.1R". Since c € int(I() it follows for t sufficiently small that a * tdt and c * f d2 belong to -I( and hence by the Lipschitz continuity of / this implies

lf(" +tdt) - f(" +tdz)l < Ltllq * dzll and so

lf'(';dr) - I'(r;dz)l < Lll& - dzll. To prove 2. we observe for d4,d2 € .R" given and t > 0 suftciently small that f("+t(.\dr +(1-.\)d2)) =

f(\(a+tdi*(l

-.\)(c +td)\

fo1 any 0 < .l < 1. By this inequality the result follows armost immediately. Finally it is easy to verify that f,(a;0) - 0 and f,(u;ld) = \f,(a;d) for every .tr) 0 and so the proof is completed.I Rernark. Observeit is shownin [3] that the function d,+-+ f'(a;'d) can be expressedas the minimum of two convexfunctions. In pariicular,'if c denotesthe convexset C := {d e IR , f,(r;d) < 0} and

e-(d),= { {g'', f,f":#" and

( n

e+@),= t l,(",r):1f":#" then g- and rp* are convexfunctions and ft(a;d)

Clearly by 3. of Iem bounded from aboveby definition.

equals min{g_(d), e+@)}.

3.3

f;K*IRattCin steepest ilescenl dir'r.r.t (P2) and f'(r;t) o weobtain i Po,'forsome ]lldrll=1 ' lliarll= i[[doll+ llldo+ ldrll< 11}aoll+

if d2 Moreover, "=#*' and (20)that

(20)

3'2 andrelations(19) it followsby 3' of Lemma

f,(n;dz)=

t'"

o = tJil'

Moreovet,if the sequenced1

+do+ idl \

(

We now show that o equa} the definition of lim sup one .lim . f- @ u ) - f '( ,'' " )' exist '', -*' - all ft€rc-o llarx

\"' ffi61)

EA+ffi

ldt) f'(a;L'do+

it follows by the Bolzano-Y rC' ( convergentsubsequence we alsoobtain that lldll = I

7'(n;do) and again we have a contradiction'

;

Wecannowprovethefollowingimportantresult. anil quasiconuex IR-i1L.insfitz conlinaoas K: If ilescent 3.5 f Theorem ot ; e 1"ltni then th'e'steepesteqaals of a t ill'""'''on 'es"'ntt Euc.Iiileonnorm and thereis s

untqi"la,rr;rit'"ii'rt ong;o t o L'J )' ofI ,rrz'iir'iii s ilirectiondo @1$ -'\ > 0";; ; ;;'v- s b"t one. Moreoa e',it f oltiwiii "i'a(n

P r o o f . T h e f i r s t p a r t o f t h e a b o v e r e st11"s'i't";;i u l t i s a l r e a d y p r o v eo,y! d i n L e there m m a 3exists .4.

we observ" ^' o ' t to' Bv this To provethe secondparr J.i ' lt"i i"; * 1 ,nli ";"; ro > 0 such it" sorne t Z [*l with 'L* t"' ir'"' belong doesnot "n"tv inequalitythe point c sivenbY

e=

lim 4

&6r'*co

I

By the unicity of the steepe

+ Ly:={s eK: f(s)Sf @ ido)}'

SinceKisclosedand/isLiP s c h i t z l o n t i n u o u so:t:Tii:* i t f o l l o wto s t trr h a twith theset.Lr a;* bv now Denote orthogonal th: iil;qiill is alsocrosed. the"so-called is from T' i""-'-'"* that minimumEu"lid"oniistance e ir it-fottowsdirectlv io" ;;'i;;il;'i on o of projection

= = |,r,-"ll