1
Semiconvergence in distribution of random closed sets with application to random optimization problems Silvia Vogel
1
Technische Universit¨at Ilmenau Institut f¨ ur Mathematik Weimarer Straße 25, 98684 Ilmenau Germany
[email protected] Tel.: (+49) 03677 693626
Abstract The paper considers upper semicontinuous behavior in distribution of sequences of random closed sets. Semiconvergence in distribution will be described via convergence in distribution of random variables with values in a suitable topological space. Convergence statements for suitable functions of random sets are proved and the results are employed to derive stability statements for random optimization problems where the objective function and the constraint set are approximated simultaneously. Key words: convergence of random sets, inner approximation in distribution, stability of random optimization problems
1 The
author is grateful to two anonymous referees for helpful suggestions.
2
Statements on convergence in distribution in connection with optimization problems are needed in many situations. Often optimization is done for models which are based on simulation studies. Apart from the special problem the simulation studies are designed for, they usually have some common features. They use an approximate model instead of the true one because of a lack of information or for numerical reasons. They are based on random numbers and hence probability comes into play. Eventually, (an estimation of) the distribution of certain output quantities, say parameters which are optimal in some sense, is asked for. In order to draw conclusions on the distribution of these output quantities in the true problem, one needs conditions ensuring convergence in distribution of optimal values and/or solution sets. Approximations of random functions (random processes, random fields etc.) are dealt with in many papers and books. The behavior of infima of random processes over a fixed constraint set, for example, has been well studied and used in several applications (cf. Billingsley (1968), Embrechts, Kl¨ uppelberg, and Mikosch (1997)). Furthermore, many statistical estimators being solutions to random optimization problems, results on convergence in distribution of solutions to random optimization problems can be employed to derive statements on the asymptotic distribution of statistical estimators (cf. Pollard (1984), van der Vaart and Wellner (1996), van der Vaart (1998), Pflug (1992)). As far as one has to deal with convergence of random variables taking values in a metrizable space, one can rely on the well-established theory of convergence in distribution in metric spaces, including the so-called Portmanteau Theorem and the Continuous Mapping Theorem (cf. Billingsley (1968), Dudley (1989), Lo`eve (1977)). Random closed sets in Rp may also be treated in this framework since they can be regarded as random variables with values in the space of closed subsets of Rp , provided with the σ-field of Borel sets with respect to Fell topology (cf. Matheron (1975), Salinetti and Wets (1986), Pflug (1992)). However, there are several problems where one cannot expect convergence in distribution of a sequence of random sets involved. The solution sets to optimization problems with random objective function and/or random constraint sets are of that nature. Under reasonable conditions one obtains a certain kind of semiconvergence only. Pflug (1992) calls this property ‘asymptotic dominance’. Roughly spoken, asymptotic dominance of a probability measure PΓo , describing the distribution of a random set Γo , over a sequence (PΓn )n∈N of distributions of random sets Γn means that the sequence (Γn )n∈N tends to approach in ‘distribution’ a subset of Γo . In the following, we will call this property ‘inner approximation in distribution’ in comparison to an outer approximation, which means that a superset of Γo is approached. The present paper will investigate the inner approximation only, outer approximations can be dealt with in a similar manner. We give results which offer the possibility to carry over results from the deterministic setting to convergence in distribution. Applying our results to optimization problems we obtain assertions on the convergence in distribution of optimal values and solution sets. We allow for the simultaneous approximation of the objective functions and the constraint sets and improve results by Salinetti and Wets (1986)
3
and Pflug (1992). Furthermore, with our results we also contribute to the existing literature on the asymptotic distribution of statistical estimators, since we can deal with constrained estimation and solution sets which are not single-valued. Estimation problems of that kind gain growing interest (cf. van der Vaart (1998)). Eventually, results can be used in model selection where one is interested in estimating the optimal value rather than the solution set (cf. Dudley (1989) and the literature quoted there). We do not impose differentiability assumptions. If additional differentiability conditions are satisfied, delta theorems can be employed to derive sharper assertions on the asymptotic distribution. Meanwhile results have been proven under rather weak differentiability assumptions and constraints have been taken into account (cf. Shapiro (1991, 2000), King and Rockafellar (1993)). Delta theorems for random sets are considered by Dentcheva (2001, 2002). The distribution of the unique solution to a ‘blown-up empirical program’ with constraints is derived in Pflug (1995), also non-smooth cases are taken into account. Our starting point was a paper by Lachout (2000), who suggested to study a topology which can play a similar role for inner approximations as Fell topology does for convergence in Kuratowski-Painlev´e sense of sequences of closed sets. Unfortunately, this topology can only be described via a so-called quasi-pseudo-metric (cf. Francaviglia, Lechicki and Levy (1985)), hence, when passing over to convergence in distribution, we can not immediately make use of the theory of convergence in distribution in metric spaces. Therefore, as a basis for our considerations, we prove statements which may serve as a surrogate for the the Continuous Mapping Theorem. The author is grateful to one referee for bringing the paper by Hoffmann-Jørgensen (1998) to her knowledge. Hoffmann-Jørgensen (1998) investigates convergence in distribution of random elements with values in topological spaces in a general setting. Some of the results are also applicable to the topology under consideration here, compare section 2. The paper is organized as follows: In Section 1 we summarize the main facts on convergence in distribution of random closed sets. Section 2 investigates inner approximations in distribution and provides several auxiliary results, which will be used to prove stability statements for sequences of random optimization problems in section 3. Application of the results to problems arising in statistics will be dealt with elsewhere.
1
Convergence in distribution of random sets
Random sets could be defined via measurable multifunctions as investigated by Rockafellar and Wets (1998). We follow Matheron (1975), Salinetti and Wets (1986), and Pflug (1992) in regarding random closed sets as random variables with values in the space of closed sets provided with the σ-field of Borel sets with respect to a suitable topology. For the equivalence of the two approaches see for instance Rockafellar and Wets (1998). For the reader’s convenience we repeat the main facts. We will confine ourselves to subsets of Rp , however, more general spaces could be dealt with (cf.
4
Matheron (1975)). Let F be the family of all closed subsets of Rp . F is equipped with the so-called Fell topology τF (cf. Beer (1993), Matheron (1975)). The open sets with respect to τF may be generated from a prebase which contains all ‘missing’ sets M(K) := {F ∈ F : F ∩ K = ∅} with a compact set K ⊂ Rp and all ‘hitting’ sets H(G) := {F ∈ F : F ∩ G 6= ∅} with an open set G ⊂ Rp . For some considerations in connection with distance functions the empty set would require a special treatment without yielding additional insight. Therefore we shall exclude the empty set in these cases and deal with the family of nonempty ˜ closed sets, which is denoted by F. ˜ τ˜F ] enjoy several nice properties, see Beer The topological spaces [F, τF ] and [F, (1993), Matheron (1975), Hu and Papageorgiou (1997), Lucchetti and Torre (1994). Fell topology is of particular importance in stability theory because convergence in Fell topology is equivalent to convergence in Kuratowski-Painlev´e sense (cf. Beer (1993), Theorem 5.2.6). ‘Semilimits’ in the Kuratowski-Painlev´e sense for sequences (Fn )n∈N with Fn ⊂ Rp are defined in the following way: K−lim supFn := {x ∈ Rp : ∃(xn )n∈N with xn → x n→∞
and xn ∈ Fn for infinitely many n}, K−lim inf Fn := {x ∈ Rp : ∃(xn )n∈N with xn → x n→∞
and xn ∈ Fn for all n ≥ no }. These semilimits always exist and belong to F. A set F ∈ F is said to be the Kuratowski-Painlev´e limes to a sequence (Fn )n∈N if K−lim supFn = K−lim inf Fn = F . n→∞
n→∞
˜ τ˜F ] are metrizable (cf. Beer (1993), TheThe topological spaces [F, τF ] and [F, ˜ orem 5.1.5). A suitable metric for [F, τ˜F ] for instance, which will be referred to in section 3, is R∞ d(F1 , F2 ) := dρ (F1 , F2 )e−ρ dρ 0
where dρ (F1 , F2 ) = max |d(x, F1 ) − d(x, F2 )| ||x||≤ρ
and d(·, F ) denotes the usual Euclidean distance to a nonempty closed set F (cf. Rockafellar and Wets (1998)). Let SF denote the σ-field of Borel sets of [F, τF ]. Definition 1.1 A random closed set Γ is a random variable, defined on a given probability space [Ω, Σ, P ] with values in the measurable space [F, SF ]. Γ induces a probability measure PΓ on [F, SF ] in the usual way: PΓ (A) := P {ω : Γ(ω) ∈ A}. For elements of the prebase of Fell topology one obtains PΓ (M(K)) := P {ω : Γ(ω) ∩ K = ∅},
5
PΓ (H(G)) := P {ω : Γ(ω) ∩ G 6= ∅}. Convergence in distribution of sequences of random closed sets (Γn )n∈N can now be defined as convergence of random variables with values in a metric space. We shall give a definition which employs the so-called continuity sets. A PΓ -continuity set A is an element of SF with the property PΓ (∂A) = 0 where ∂A denotes the boundary of A. Definition 1.2 A sequence (Γn )n∈N of random closed sets is said to converge in distribution to a random closed set Γo (abbreviated Γn D→Γo ) if lim PΓn (A) = PΓo (A) for all PΓo -continuity sets A. n→∞
Salinetti and Wets (1986) and Pflug (1992) showed that one may restrict oneself to continuity sets having a special form, for instance finite unions of closed balls with rational centres and rational radii. The following relations are equivalent to Γn D→Γo by the Portmanteau (or Alexandrov’s) Theorem (cf. Billingsley (1968), Lo`eve (1977)): (i) lim inf PΓn (U) ≥ PΓo (U) for all τF -open sets U, n→∞
(ii) lim supPΓn (C) ≤ PΓo (C) for all τF -closed sets C, n→∞
(iii) lim
n→∞
R F
g(F )dPΓn (F ) =
R
g(F )dPΓo (F )
F
for all function g which are bounded and continuous with respect to τF , R R (iv) lim g(F )dPΓn (F ) = g(F )dPΓo (F ) n→∞
F
F
for all function g which are bounded and continuous with respect to τF in PΓo -almost all ‘points’ F . Furthermore, if Γn D→Γo , then h(Γn ) D→h(Γo ) for all functions h which map F into another metric space and are continuous PΓo -almost everywhere. This statement is usually called Continuous Mapping Theorem.
2
Semiconvergence in distribution
When considering the solution sets of optimization problems, results from deterministic parametric programming tell us that, in general, we can only expect that the cluster points of sequences of solutions to the approximate problems belong to the solution set of the limit problem, i.e., we have some kind of upper semicontinuous behavior or, in other words, an ‘inner approximation’. In terms of convergence in distribution, for special sequences of stochastic optimization problems, this topic was dealt with by Pflug (1992) and Vogel (1991). This section will investigate the topological background to semiconvergence in distribution and prove results which pave the way, for instance, for the derivation of stability statements ‘in distribution’ for rather general random optimization problems.
6
In Section 1 we described convergence in distribution of random sets as convergence in distribution of random variables with values in the measurable space [F, SF ], which is appropriate, because convergence in Kuratowski-Painlev´e sense coincides with convergence in Fell topology. Now, in order to describe the ‘upper semicontinuous’ behavior as ‘one half’ of Kuratowski-Painlev´e convergence, we can use a topology which is coarser than Fell topology. The same could be done for lower semicontinuous behavior, which is of importance, for instance, when constraint sets are approximated. This, however, will be done elsewhere. Definition 2.1 A sequence (Fn )n∈N of subsets of Rp is called inner approximai tion to Fo ⊂ Rp (abbreviated Fn →F o ) if K−lim supFn ⊂ Fo . n→∞
Let a topology τM on F be defined by the prebase consisting of all missing sets M(K), K ⊂ Rp compact. An inner approximation in distribution may now be defined in the following way: Definition 2.2 The sequence (Γn )n∈N of closed random sets is called an inner →Γo ), if approximation in distribution to the closed random set Γo (Γn i−D lim inf PΓn (U) ≥ PΓo (U) for all τM -open sets U. n→∞
The following lemma gathers up equivalent characterizations of an inner approximation in distribution. Lemma 2.1 Let {Γn , n ∈ No } be a family of closed random sets. Then the following conditions are equivalent: →Γo , (i) Γn i−D (ii) lim supPΓn (C) ≤ PΓo (C) for all τM -closed sets C, n→∞
(iii) lim supPΓn (∩ki=1 {F : F ∩ Ki 6= ∅}) ≤ PΓo (∩ki=1 {F : F ∩ Ki 6= ∅}) n→∞
for all k ∈ N and all compact sets Ki , (iv) lim supPΓn (∩ki=1 {F : F ∩ Ki 6= ∅}) ≤ PΓo (∩ki=1 {F : F ∩ Ki 6= ∅}) n→∞
for all k ∈ N and all compact sets Ki with the property P (Γo ∩ Ki 6= ∅, Γo ∩ intKi = ∅) = 0.
(1)
Remark 2.1 The property (1) means that the sets {F : F ∩ Ki 6= ∅} are PΓo continuity sets with respect to Fell topology (cf. Salinetti and Wets (1986)). Proof. (ii) is simply another form of (i). (iii) is implied by (ii), because the sets ∩ki=1 {F : F ∩ Ki 6= ∅} are τM -closed.
7
We shall show that (iii) ⇒ (ii). Each τM -closed set C may be written in the form ji C C C = ∩∞ i=1 {F : F ∩ (∪j=1 Ki,j ) 6= ∅} where {Ki,j , i = 1, 2, . . . ; j = 1, . . . , ji } is a i C ˜ i is again a com=: K Ki,j suitable family of compact subsets of Rp and hence ∪jj=1 k ˜ pact set . Thus C = lim Ck with Ck := ∩i=1 {F : F ∩ Ki 6= ∅} and, consequently, k→∞
PΓo (C) = lim PΓo (Ck ). k→∞
Suppose that lim supPΓn (C) > PΓo (C) for a τM -closed set C. Then there are a ko , n→∞
an α > 0 and a sequence (PΓnl (C))l∈N with PΓnl (C) > PΓo (Ck ) + α Since C ⊂ Ck , we have a contradiction to (iii).
∀k ≥ ko .
(iii) implies (iv) by definition. It remains to show that (iv) ⇒ (iii). Let compact sets Ki , i = 1, . . . , k, be given. To each Ki and each > 0 there exists a set Ki, ⊃ Ki with property (1) and PΓo (∩ki=1 {F : F ∩ Ki 6= ∅}) > PΓo (∩ki=1 {F : F ∩ Ki, 6= ∅}) − . Thus lim supPΓn (∩ki=1 {F : F ∩ Ki 6= ∅}) ≤ lim supPΓn (∩ki=1 {F : F ∩ Ki, 6= ∅}) n→∞
n→∞
2
≤ PΓo (∩ki=1 {F : F ∩ Ki, 6= ∅} < PΓo (∩ki=1 {F : F ∩ Ki 6= ∅} + .
The following lemma shows that inner approximations may be characterized by convergence in the topology τM . Lemma 2.2 Let {Fn , n ∈ No } be a family of elements of F. Then i Fn τM →Fo ⇔ Fn →F o. Proof. (i) Suppose that the sequence (Fn )n∈N does not converge to Fo in τM . This implies that Fo can not be equal to Rp . Hence there is an element G˜ of τM with Fo ∈ G˜ and Fn ∈ / G˜ for infinitely many n. Furthermore, there is an element G of the prebase of τM with Fo ∈ G and Fn ∈ / G for infinitely many n. G has the form G = {F ∈ F : F ∩ K = ∅} for a suitable K ∈ Kp . Fo ∈ G then implies Fo ∩ K = ∅, and Fn ∈ / G means Fn ∩ K 6= ∅. Consequently, there is an xo ∈ K−lim supFn ∩ K. n→∞
Thus K−lim supFn 6⊂ Fo . n→∞
(ii) Suppose that K−lim supFn 6⊂ Fo . Hence there is an xo ∈ K−lim supFn n→∞
n→∞
with xo ∈ / Fo . To xo there is a compact set K with xo ∈ intK, but Fo ∩ K = ∅, implying that G = {F ∈ F : F ∩ K = ∅} is an element of the prebase of τM that contains Fo . Because of xo ∈ K−lim supFn , to xo we find a sequence (xnk )k∈N with n→∞
xnk k→∞ →xo and xnk ∈ Fnk ∀k ∈ N . Thus Fnk ∩ K 6= ∅ ∀k ≥ ko which means Fnk ∈ / G ∀k ≥ ko . Consequently, (Fn )n∈N does not converge to Fo in topology τM . 2 Fell topology on F˜ can be generated by several metrics (cf. Rockafellar and Wets (1998), Pflug (1992)), and hence convergence in Fell topology may be described by convergence in these metrics. We will propose some kind of half-sided ‘distance’ which is not a metric, but can play a similar role in our framework.
8 ˜ Let for ρ > 0 and F1 , F2 ∈ F, i dρ (F1 , F2 ) := max (d(x, F1 ) − d(x, F2 ))+ ||x||≤ρ
where (.)+ denotes the positive part. The letter ‘i’ indicates that the term will be used to describe inner approximations. Furthermore, define R∞ di (F1 , F2 ) := 0 diρ (F1 , F2 )e−ρ dρ. Remark 2.2 di (F1 , F2 ) is finite. This may easily be seen if we compare di with the ˜ which is investigated by Rockafellar and Wets metric d mentioned in section 1 on F, (1998). By definition, di (F1 , F2 ) ≤ d(F1 , F2 ). Hence we can employ Lemma 4.41b in Rockafellar and Wets (1998). Obviously, di lacks symmetry. However, it enjoys a triangle inequality and is hence a quasi-pseudo-metric as introduced by Francaviglia, Lechicki and Levy (1985). Lemma 2.3 di (F1 , F2 ) ≤ di (F1 , F3 ) + di (F3 , F2 ),
˜ i = 1, 2, 3. Fi ∈ F,
Proof. We have, for ρ > 0, diρ (F1 , F2 ) = max (d(x, F1 ) − d(x, F2 ))+ ||x||≤ρ
= max (d(x, F1 ) − d(x, F3 ) + d(x, F3 ) − d(x, F2 ))+ ||x||≤ρ
≤ max ((d(x, F1 ) − d(x, F3 ))+ + (d(x, F3 ) − d(x, F2 ))+ ) ||x||≤ρ
≤ diρ (F1 , F3 ) + diρ (F3 , F2 ). Furthermore, R ∞ di (F , F ) := 0 diρ (F1 , F2 )e−ρ dρ R ∞ 1i 2 R∞ ≤ 0 dρ (F1 , F3 )e−ρ dρ + 0 diρ (F3 , F2 )e−ρ dρ.
2
di may be used to describe inner approximations. We start with two auxiliary ˜ results. In the following, {Fn , n ∈ No } denotes a family of elements of F. i i Lemma 2.4 Fn →F o ⇒ lim dρ (Fo , Fn ) = 0 n→∞
∀ρ > 0.
Proof. Suppose that there are a ρ > 0, an α > 0 and a sequence (Fnk )k∈N with diρ (Fo , Fnk ) > α. Hence max (d(x, Fo ) − d(x, Fnk ))+ > α, which implies the ||x||≤ρ
existence of xok , k ∈ N, with ||xok || ≤ ρ and d(xok , Fo ) − d(xok , Fnk ) > α. As the sequence (xok )k∈N contains a convergent subsequence, we find an xo with ˜ ||xo || ≤ ρ and d(xo , Fo ) − d(xo , Fnk ) > α2 for all k belonging to an infinite subset N ˜. of N. In the following we consider k ∈ N Let xnk ∈ Fnk be such that d(xo , Fnk ) = d(xo , xnk ). If the sequence (xnk )k∈N˜ has a cluster point x ˜, we obtain d(xo , Fo ) − d(xo , x ˜) > α2 . Hence, choosing x˜o ∈ Fo with d(˜ x, Fo ) = d(˜ x, x˜o ), the relations d(˜ x, Fo ) ≥ d(xo , x˜o ) − d(xo , x ˜) ≥ d(xo , Fo ) − d(xo , x ˜) > α2 follow.
9
Thus x ˜ cannot belong to Fo and K −lim supFn 6⊂ Fo . n→∞
Now, suppose that the sequence (xnk )k∈N does not have a cluster point. Consequently, lim d(xo , xnk ) = ∞ in contradiction to d(xo , Fo ) − d(xo , xnk ) > α. 2 k→∞
Lemma 2.5 lim diρ (Fo , Fn ) = 0 n→∞
i ∀ρ ≥ ρo ⇒ Fn →F o.
/ Fo . Hence we find Proof. Suppose that there is an xo ∈ K −lim supFn with xo ∈ n→∞
an α > 0 such that d(xo , Fo ) > α. Consequently, for ρo := ||xo ||, diρ (Fo , Fn ) = max (d(x, Fo ) − d(x, Fn ))+ ||x||≤ρ
≥ (d(xo , Fo ) − d(xo , Fn )) > α for infinitely many n. 2 Lemma 2.4 and Lemma 2.5, together with the convergence of the integral over ρ, imply the next statement. ˜ Then Lemma 2.6 Let {Fn , n ∈ No } be a family of elements of F. i i ⇔ lim d (F , F ) = 0. Fn →F o o n n→∞
In the following, we shall need the function diA |F˜ → R+ describing the distance of an element F ∈ F˜ to a Borel subset A of F˜ : diA (F ) := inf di (Fˆ , F ). Fˆ ∈A
A useful property of diA is a kind of upper semicontinuity with respect to the topology τM . Lemma 2.7 Let {Fn , n ∈ No } be a family of elements of F˜ and A be τM -closed . Then Fn τM →Fo ⇒ lim sup diA (Fn ) ≤ diA (Fo ). n→∞
Proof. Let Fn τM →Fo . Then, by Lemma 2.2 and Lemma 2.6, lim di (Fo , Fn ) = 0. n→∞ Furthermore, to each k ∈ N and j ∈ No there is Fˆjk ∈ A with |diA (Fj ) − di (Fˆjk , Fj )| < k1 . Now, making use of the triangle inequality, we obtain for all k ∈ N di (Fˆok , Fn ) ≤ di (Fˆok , Fo ) + di (Fo , Fn ) and diA (Fn ) ≤ diA (Fo ) + di (Fo , Fn ) + k1 . Hence lim sup diA (Fn ) ≤ diA (Fo ). 2 n→∞
Remark 2.3 F ∈ A for a τM -closed set A implies, by di (F, F ) = 0, the equality diA (F ) = 0. Remark 2.4 If one considers the metric d with R∞ d(F1 , F2 ) = dρ (F1 , F2 )e−ρ dρ 0
10
and dρ (F1 , F2 ) = max |d(x, F1 ) − d(x, F2 )|, ||x||≤ρ
which metrizes Fell topology (cf. Rockafellar and Wets (1998)), and defines a distance dA (F ) := inf d(Fˆ , F ) Fˆ ∈A
then dA is continuous with respect to Fell topology. Now we are ready to prove the following sufficient condition for an inner approximation in distribution. Lemma 2.8 Let {Γn , n ∈ No } be a family of nonempty closed random sets and assume that Z Z lim sup g(F )dPΓn (F ) ≤ g(F )dPΓo (F ) n→∞
F
F
for all functions g which are bounded and upper semicontinuous with respect to τM . Then lim supPΓn (A) ≤ PΓo (A) for all τM -closed sets A. n→∞
Proof. For a τM -closed set ARthe indicator function IA is upper semicontinuous with respect to τM . Because of IA (F )dPΓ (F ) = PΓ (A) for a random closed set F
Γ the assertion follows.
2
Hoffmann-Jørgensen (1998) considered convergence in law of random elements in general topological spaces. He used the assumption in Lemma 2.8 as definition of convergence in distribution (in Borel law) and proved equivalent characterizations. Application of his results to the case under consideration yields that the assumption in Lemma 2.8 is also an equivalent characterization of the convergence notion in Definition 2.1. Now we come to the main part of the paper. We shall provide auxiliary results which may be employed to derive assertions on the convergence in distribution of functions of converging sequences. These results will be used in section 3 to derive stability statements ‘in distribution’ for random optimization problems. As we shall have to deal with Euclidean spaces with possibly different dimensions, we shall indicate the dimension r by a subscript: Fr denotes the space of closed subsets of Rr and τ (r) a suitable topology on Fr such that each F ∈ Fr has a countable base of neighbourhoods. The assumption concerning τ (r) enables us to deal with sequences instead of nets when we consider continuity. Note that both Fell topology and the coarser topology τM have this property. Theorem 2.1 Let {Γn , n ∈ No }, Γn |Ω → [Fr , τ (r) ], be a family of closed random sets and Sˆ a continuous mapping from [Fr , τ (r) ] into the space [Fm , τ (m) ]. Then lim supPΓn (A) ≤ PΓo (A) for all τ (r) -closed sets A n→∞
ˆ o ). ˆ n ) i−D implies S(Γ →S(Γ
11 Proof. Let lim supPΓn (A) ≤ PΓo (A) for all τ (r) -closed sets A. Taking into n→∞
account that inverse images of closed sets via continuous mappings are closed, we (m) obtain for a τM -closed set Aˆ ˆ n ) ∈ A) ˆ = lim supP (Γn ∈ Sˆ−1 (A)) ˆ lim supP (S(Γ n→∞
n→∞
2
ˆ = P (S(Γ ˆ o ) ∈ A). ˆ ≤ P (Γo ∈ Sˆ−1 (A)) Furthermore, we have the following result:
Theorem 2.2 Let {Γn , n ∈ No }, Γn |Ω → [Fr , τ (r) ] be a family of closed ran¯1, Σ ¯ 1 ],i.e. dom sets and ϕˆ a lower semicontinuous mapping from [Fr , τ (r) ] into [R (r) lim inf ϕ(F ˆ n ) ≥ ϕ(F ˆ o ) for all sequences (Fn )n∈N with Fn τ →Fo . Then n→∞
lim supPΓn (A) ≤ PΓo (A) for all τ (r) -closed sets A implies n→∞
lim supP (ϕ(Γ ˆ n ) ≤ y) ≤ P (ϕ(Γ ˆ o ) ≤ y)
¯ ∀y ∈ R.
n→∞
Proof. Let lim supP (Γn ∈ A) ≤ P (Γo ∈ A) for all τ (r) -closed sets A. Since the n→∞
¯, level sets of l.s.c. mappings are closed, we have for all y ∈ R −1 lim supP (ϕ(Γ ˆ n ) ≤ y) = lim supP (Γn ∈ ϕˆ ([−∞, y]) n→∞
n→∞
≤ P (Γo ∈ ϕˆ−1 ([−∞, y]) = P (ϕ(Γ ˆ o ) ≤ y).
3
2
Approximation of random optimization problems
Suppose that a random optimization problem (IP o ) min fo (x, ω) x∈Co (ω)
is approximated by a sequence of surrogate problems (IP n ) min fn (x, ω), n ∈ N, ω ∈ Ω. x∈Cn (ω)
The constraint sets Cn | Ω → Fp , n ∈ No , are closed random sets and the 1 1 objective functions fn |Rp × Ω → R , n ∈ No , are supposed to be (Σp ⊗ Σ, Σ )1 measurable. The σ−field Σ is generated by Σ1 and {+∞}, {−∞}. Cn , n ∈ No , may be specified by inequality constraints: (i) Cn (ω) = {x ∈ Rp | gn (x, ω) ≤ 0, i = 1, . . . , io } (i) with random functions gn |Rp ×Ω → R1 , which have to be (Σp ⊗Σ, Σ1 )-measurable. By Φn , n ∈ No , we denote the optimal values and by Ψn , n ∈ No , the solution sets: ( inf fn (x, ω), if Cn (ω) 6= ∅, x∈Cn (ω) Φn (ω) = +∞ otherwise, Ψn (ω) := {x ∈ Cn (ω) : fn (x, ω) = Φn (ω)}. Under our assumptions the necessary measurability conditions are fulfilled, cf. Vogel (1994).
12
We shall show how the results of Section 3 may be employed to derive statements on the asymptotic behavior of the constraint sets, the optimal values and the solution sets. Note, however, that we are dealing with closed-valued multifunctions only. Hence the epigraph Epi f and the graph Graphf of a functionf , which are considered in the following, have to be closed-valued, and semicontinuity assumptions have to be imposed on the objective functions. In order to overcome these assumptions at least partially (namely for the approximate problems), additional considerations are necessary (cf. Vogel (1991)). In order to indicate that we are dealing with the ‘closed-valued’ case only, we shall repeat the corresponding conditions in the assumptions of the following statements. We start by investigating the constraint sets. (i)
Theorem 3.1 Let gn , i = 1, . . . , io , n ∈ No , be l.s.c. for almost all ω. Then (1) (i ) (1) (i ) Epign × . . . × Epign o i−D→Epigo × . . . × Epigo o implies Cn i−D→Co . In order to ensure lower semicontinuous behavior in distribution of the optimal values, we need a compactness condition. Definition 3.1 The sequence (fn , Cn )n∈N is called equi-inf bounded if for each ω ∈ Ω and each y ∈ R1 ∪ {−∞} there is a compact set K(ω, y) such that lim P ({ω ∈ Ω : {x ∈ Cn (ω) : fn (x, ω) ≤ y} ⊂ K(ω, y)}) = 1. n→∞
Theorem 3.2 Let fn (·, ω), n ∈ No , be l.s.c. for almost all ω and Cn be closedvalued for all n ∈ No . Additionally, assume that the sequence (fn , Cn )n∈N is equi-inf bounded. Then Epifn × Cn i−D→Epifo × Co ¯1. implies lim supP (Φn ≤ y) ≤ P (Φo ≤ y) for all y ∈ R n→∞
Theorem 3.3 Let fn (·, ω) be continuous for almost all ω and Cn be closed-valued for all n ∈ No . Then Graphfn × Cn D→Graphfo × Co implies Ψn i−D →Ψo . In the following proofs, let LSC(Rp ) denote the space of lower semicontinuous functions fˆ|Rp → R1 . Proof of Theorem 3.1 Let C| DC ⊂ Fio (p+1) → Fp be defined by C(Epiˆ g (1) , . . . , Epiˆ g (io ) ) := {x ∈ Rp | gˆ(i) (x) ≤ 0, i = 1, . . . , io }, gˆ(i) ∈ LSC(Rp ), i = 1, . . . , io . In order to exploit Theorem 2.1, we have to show that C is a continuous (i (p+1)) (p) mapping from [Fio (p+1) , τMo ] into [Fp , τM ]. Assume that to a family {(ˆ gn ), n ∈ No }, gˆn ∈ LSC(Rp ), there is a sequence (i ) (1) (xnk )k∈N with xnk → xo , xnk ∈ C(Epiˆ gnk , . . . , Epiˆ gnko ) and (i) (io ) (i) xo ∈ / C(Epiˆ go , . . . , Epiˆ go ). Hence gˆnk (xnk ) ≤ 0 ∀i ∈ {1, . . . , io } ∀k ∈ N , but (i) gˆo (xo ) > 0 for at least one i in contradiction to (i) (i) K−lim sup Epiˆ gn ⊂ Epiˆ go ∀i ∈ {1, . . . , io }. 2 n→∞
13 ˆ K | D ˆ ⊂ (Fp+1 × Fp ) → R1 for a compact set K Proof of Theorem 3.2 Let Φ ΦK be defined by ˆ K (Epifˆ, C) ˆ := inf fˆ(x), Cˆ ∈ Fp ; fˆ ∈ LSC(Rp ). Φ ˆ x∈C∩K
(2p+1) ˆ K we have to show that In order to apply Theorem 2.2 with τ (r) = τM to Φ i ˆ ˆ ˆ ˆ ˆ ˆ ˆ lim inf ΦK (Epifn , Cn ) ≥ ΦK (Epifo , Co ) if Epifn × Cˆn →Epi fˆo × Cˆo . n→∞
i Since Epifˆn →Epi fˆo is equivalent to lim inf fˆn (xn ) ≥ fˆo (xo ) for all sequences n→∞
(xn )n∈N tending to xo , we can proceed in the following way: ˆ K (Epifˆn , Cˆn ) < Φ ˆ K (Epifˆo , Cˆo ). Because of the lower Assume that lim inf Φ n→∞ i ˆ Co , there is a sequence (xnk )k∈N with xnk ∈ Cˆnk ∩ K, semicontinuity of fˆn and Cˆn → ˆ K (Epifˆn , Cˆn ), xn k→∞ fˆn (xn ) = Φ → xo ∈ Cˆo ∩ K and lim fˆn (xn ) < fˆo (xo ) k
k
k
k
k
k→∞
k
k
in contradiction to lim inf fˆnk (xnk ) ≥ fˆo (xo ). k→∞
ˆ K is l.s.c. with respect to τ (2p+1) and one has Thus Φ M lim supP ({ω ∈ Ω : inf fn (x, ω) ≤ y}) ≤ P ({ω ∈ Ω : x∈Cn (ω)∩K
n→∞
inf x∈Co (ω)∩K
fo (x, ω) ≤ y}).
Now we are ready to prove the assertion. For y = +∞ there is nothing to show. Let y ∈ R1 ∪ {−∞} and ε > 0 be given. The sequence (Ωk )k∈N with Ωk := {ω ∈ Ω : K(ω, y) is contained in the closed ball with radius k} is increasing and has the limit Ω. Consequently, there is a ko such that P (Ωko ) > 1 − 2ε . This, by the equi-inf boundedness condition, implies lim supP ({ω ∈ Ω : n→∞
inf x∈Cn (ω)
≤ lim supP ({ω ∈ Ω :
inf x∈Cn (ω)∩Bko
n→∞
≤ P ({ω ∈ Ω :
inf x∈Co (ω)∩Bko
≤ P ({ω ∈ Ω :
fn (x, ω) ≤ y}
inf x∈Co (ω)
fn (x, ω) ≤ y}) + ε
fo (x, ω) ≤ y}) + ε 2
fo (x, ω) ≤ y}) + ε.
Proof of Theorem 3.3 Let C(Rp ) denote the space of continuous functions ˆ ˆ D ˆ ⊂ Fp+1 × Fp → Fp be defined by f |Rp → R1 , and let Ψ| Ψ ˆ ˆ := {x ∈ Cˆ : fˆ(x) = inf fˆ(˜ Ψ(Graph fˆ, C) x)}, Cˆ ∈ Fp , fˆ ∈ C(Rp ). We have ˆ x ˜ ∈C
ˆ fˆn , Cˆn ) ⊂ Ψ( ˆ fˆo , Cˆo ) whenever to show that K−lim sup Ψ( n→∞
Graphfˆn × Cˆn →Graphfˆo × Cˆo where convergence is understood with respect to the product of Fell topologies in Fp+1 and Fp . Graphfˆn × Cˆn →Graphfˆo × Cˆo implies K−lim Cˆn = Cˆo and lim fˆn (xn ) = fˆo (xo ) n→∞
for all xo ∈ Rp and all sequences (xn )n∈N tending to xo .
n→∞
ˆ fˆn , Cˆn ) which does not belong to Suppose that there is an xo ∈ K−lim sup Ψ( n→∞
ˆ fˆo , Cˆo ). Hence there is a sequence (xn )k∈N with xn ∈ Cˆn , Ψ( k k k
14 fˆnk (xnk ) = inf fˆnk (x) and xnk k→∞ →xo . The assumption implies xo ∈ K−lim sup Cˆn ˆn x∈C k
n→∞
and hence xo ∈ Cˆo and, furthermore, lim fˆ(xnk ) = fˆ(xo ). k→∞
ˆ fˆo , Cˆo ), we find x Now, if xo ∈ / Ψ( ˆo ∈ Cˆo with fˆo (ˆ xo ) < fˆo (xo ). To x ˆo there is a sen→∞ →ˆ xo . Since lim fˆ(ˆ xn ) = fˆ(ˆ xo ), quence (ˆ xn )n∈N with x ˆn ∈ Cˆn ∀n ≥ no and x ˆn n→∞ we have fˆ(xnk ) > fˆ(ˆ xnk ) ∀k ≥ ko in contradiction to the definition of xnk . It remains to employ Theorem 2.2. 2
References [1] Artstein, Z. (1984). “Distributions of random sets and random selections.” Israel J. Mathematics 46, 313-324. [2] Attouch, H. (1984). Variational Convergence for Functions and Operators. London: Pitman. [3] Bank, B. et al. (1982). Non-Linear Parametric Optimization. Berlin: AkademieVerlag. [4] Beer, G. (1993). Topologies on Closed and Closed Convex Sets. Kluwer Academic Publishers. [5] Billingsley, P. (1968). Convergence of Probability Measures. New York: Wiley. [6] Dentcheva, D. (2000). “Regular Castaing representations with applications to stochastic programming.” SIAM J. Optimization 10, 732 - 749. [7] Dentcheva, D. (2001). “Approximations, extensions, and univalued representations of multifunctions.” Nonlinear Analysis: Theory, Methods, & Applications 45, 85 - 108. [8] Dudley, R.M. (1989). Real Analysis and Probability. Pacific Grove, Ca.: Wadsworth & Brooks/Cole. [9] Dudley, R.M. (1998). “Consistency of M-estimators and one-sided bracketing.” In E. Eberlein, M. Hahn, M. Talagrand (eds.), High Dimensional Probability. 151-189. Basel: Birkh¨ auser. [10] Dupaˇcov´ a, J. and R.J.-B. Wets. (1988). “Asymptotic behavior of statistical estimators and of optimal solutions of stochastic problems.” Annals of Statistics 16, 1517-1549. [11] Embrechts, P., C. Kl¨ uppelberg, and T. Mikosch. (1997). Modelling Extremal Events. Springer-Verlag. [12] Francaviglia, S., A. Lechicki, and S. Levy. (1985). “Quasi-uniformization of hyperspaces and convergence of nets of semicontinuous multifunctions.” J. Math. Anal. Appl. 112 , 347 - 370.
15
[13] Hoffmann-Jørgensen, J. (1998). “Convergence in law of random elements and random sets.” In E. Eberlein, M. Hahn, M. Talagrand (eds.), High Dimensional Probability. 151-189. Basel: Birkh¨auser. [14] Hu, S. and N.S. Papageorgiou. (1997). Handbook of Multivariate Analysis. Kluwer Academic Publishers. [15] King, A.J. (1989). “Generalized delta theorems for multivalued mappings and measurable selections.” Math. Oper. Res. 14, 4, 720-736. [16] King, A.J. and R.T. Rockafellar. (1993). “Asymptotic theory for solutions in statistical estimation and stochastic programming.” Math. Oper. Res. 18, 148 162. [17] Kuratowski, K. (1966). Topology, Vol. I. London, New York: Academic Press. [18] Lachout, P. (2000). “Epi-convergence versus topology: Splitting and localization.” Research Report of the Academy of Sciences of the Czech Republic, UTIA No. 1987. [19] Lo`eve, M., (1997). Probability Theory. 4th edition, Springer-Verlag. [20] Lucchetti, R. and A. Torre. (1994). “Classiscal set convergence and topologies.” Set-Valued Analysis 2, 219-240. [21] Matheron, G. (1975). Random Sets and Integral Geometry. Wiley. [22] Pflug, G.C. (1992). “Asymptotic dominance and confidence for solutions of stochastic programs.” Czechoslovak J. Oper. Res. 1, 1, 21 - 30. [23] Pflug, G.C. (1995). “Asymptotic stochastic programs.” Math. Oper. Res. 20, 769-789. [24] Pollard, D. (1984). Convergence of Stochastic Processes. Springer-Verlag. [25] Robinson, S.M. (1987). “Local epi-continuity and local optimization.” Math. Programming 37, 208-222. [26] Rockafellar, R.T. and R.J.-B. Wets. (1998). Variational Analysis. SpringerVerlag. [27] Salinetti, G. and R.J.-B. Wets. (1986). “On the convergence in distribution of measurable multifunctions (random sets), normal integrands, stochastic processes and stochastic infima,.” Math. Oper. Res. 11, 385-419. [28] Shapiro, A. (1991). “Asymptotic analysis of stochastic programs.” Annals of Operations Research 30, 169-186. [29] Shapiro, A. (2000). “Statistical inference of stochastic optimization problems.” In S. Uryasev (ed.), Probabilistic Constrained Optimization: Methodology and Applications. 282-304. Kluwer Academic Pubishers.
16
[30] van der Vaart, A.W. (1998). Asymptotic Statistics. Cambridge University Press. [31] van der Vaart, A.W. and J.A. Wellner. (1996). Weak Convergence and Empirical Processes. Springer-Verlag. [32] Vogel, S. (1991). Stochastische Stabilit¨ atskonzepte. Habilitation, TH Ilmenau. [33] Vogel, S. (1994). “A stochastic approach to stability in stochastic programming.” J. Comput. and Appl. Mathematics, Series Appl. Analysis and Stochastics. 56, 65-96.
