A method of minimizing an undifferentiable function

4.

S. P. U r y a s ' e v , "A method of adjusting the step size for the E - s u b g r a d i e n t method," in: Behavior of Systems in Random Environments [in Russian], Inst. Kibern. Akad. Nauk UkrSSR, Kiev (1979), pp. 76-84.

A ME T H O D

OF

FUNCTION

WITH

V.

I.

MINIMIZING

AN

UND!FFERENTIABLE

GENERALIZED-GRADIENT

AVERAGING

Norkin

UDC 519.853.6

A method is given here for unconditional minimization of an undifferentiable function (generalized differential function), in which the direction of descent is chosen f r o m a convex shell of generalized (anti)gradients taken on a fixed number of preceding i t e r a t i o n s , while the step size is adjusted by the software. This method r e s e m b l e s the method of [1] taking a position intermediate between relaxation and nonrelaxation methods. Definition [2, 3]. The function F(x), x ~ R m is called generalized differentiable if there exists a s e m i continuous f r o m above point-set mapping G(F) :x ~ R m ~ G(F, x) ~ R m such that the sets G (F, x) are bounded, convex, and closed, and at each point y e R m the following applies: F (x) = F (y) + (g (x), ~ - - x) + o (y, x, g),

(1)

w h e r e o(y, x, g ) / i x - y~ -- 0 uniformly for x - - y and g 6 G(F, x). The elements of set G(F, x) a r e called the g e n e r a l i z e d gradients of F at point x. The c l a s s of g e n e r a l i z e d differentiable functions contains continuously differentiable ones, convex functions, and concave functions, and it is closed under the finite operations of maximum, minimum, and s u p e r position. The gradients of continuously differentiable functions and the subgradients of convex functions a r e g e n e r a l i z e d gradients of these. To calculate the generalized gradients of complicated functions one has r u l e s analogous to the rules for calculating o r d i n a r y gradients. A g e n e r a l i z e d differentiable function satisfies the local Lipshits condition. A n e c e s s a r y condition for a turning point in F at x is 0 ~ G(F, x) [2, 3]. To minimize F we use an a l g o r i t h m : x% x~ . . . . . x q 6 R ' , X k + l = Xk - - Oh " pk (Xk gk . . . . .

(2)

X k-nk, gk--nk), k ~ q,

p~ ~ 0, p~ --,- 0, ~ p~, = oo,

(3)

(4)

~0

p~ =

]~ ~, (x ~, ~

. . . . . x~-'% ~-"~)~,

is)

t=k--tl k

g" 6G(F,x~),

(6)

0 ~< nk ~< rain (n, k), n = const,

(7)

(8)

.....

r--_-~--ak

IPkI~M< ~.

(9)

The following minimizing property of the algorithm of (2)-(9) applies. THEOREM I. Let Ys --Y, 0 ~ G(F, y); for each s w e consider the sequence ix}, k > m a x (0, s - n)} formed in accordance with the following rules: x~ = y~, k < s ; X~-}-I

s

p~ ~-Pk(x~,g~

9

k

= x~9

, X~r- n k

~

p~,

(10)

k

.P~ , k > s , k--nk"

,gs

},

g~EG(P, x~).

(11) (12)

T r a n s l a t e d f r o m Kibernetika, No. 6, pp. 88-89, 102, N o v e m b e r - D e c e m b e r , 1980. Original article submitted September 13, 1978. 890

0011-4235/80/1606-0890507.50

9 1981 Plenum Publishing C o r p o r a t i o n

Then t h e r e e x i s t s a A(y) s u c h t h a t f o r a l l 0, 0 < 5 --_ A(v) and f o r a l l s we have re(s, 6) = rain {kl Ix~ --el F(y)=

l i m F ( y , ) > lirn s~oo

s~r

~6} < ~o,

rain

s ~k~ ~/2 ~,d ~ p, > ~: > 0. r~s

T h e a s s e r t i o n s of the I e m m a o c c u r e s s e n t i a l l y in [2, 4, 5] w i t h s t r o n g e r a s s u m p t i o n s on the n u m b e r s { pk }. T h e g e n e r a l c a s e of (4) r e d u c e s to the p a r t i c u l a r c a s e c o n s i d e r e d in [2] by- m o n o t o n e f r a c t i o n a t i o n of the

891

s t e p Pk = Pkl + P l a + 9 9 - + P k r k and the i n t r o d u c t i o n of the i n t e r m e d i a t e v e c t o r s P ~ 6sk r . A n o t h e r p r o o f is to be found in [7].

= p ks , zsls~w i t h the n u m b e r s

W e c o n t i n u e the p r o o f of T h e o r e m 1. We put

= co {g 6 R" Ig 6G (F, r), I r - - g l ~

5x}.

m~,6)

For P6~/2~ S2 -> S 1 t h e r e a r e

kS r=s

This means

that k s - nks -> k s - n >- s for sufficiently large s. We

substitute for k s in (17) to get

ks

F(~,)