On (n, m)-Convex Sets - Springer Link

Ukrainian Mathematical Journal, Vol. 53, No. 3, 2001

ON ( n, m )-CONVEX SETS Yu. B. Zelinskii and I. V. Momot

UDC 519.6

We investigate the class of generalized convex sets on Grassmann manifolds, which includes known generalizations of convex sets for Euclidean spaces. We extend duality theorems (of polarity type) to a broad class of subsets of the Euclidean space. We establish that the invariance of a mapping on generalized convex sets is equivalent to its affinity.

In what follows, straight lines, m-planes, and hyperplanes in the Euclidean space Rn (C n ) are understood as affine subspaces of Rn (C n ) of dimension 1, m, and n – 1, respectively [1]. We say that a pair of manifolds ( M, M ∗ ) generates (n, m) -convexity if one of these manifolds is the Euclidean space Rn and the other one is a Grassmann manifold G ′(n, m) of m-planes in Rn [2]. It is clear that there exists a natural correspondence between these manifolds, namely: every point x ∈ Rn is associated with a submanifold l( x ) = G (n, m) ⊂ G ′(n, m), where G ′(n, m) is a Grassmann manifold of m-planes passing through the point x , and every point y ∈ G ′(n, m) is associated with an m-plane l( y) in Rn . (In some cases, for convenience, we assume that Rn is compactified by a single infinitely remote point. In this case, each m-plane is a compact subset of this compactification.) We extend the notion of m-plane from the Euclidean space to subsets of Grassmann manifolds. Namely, we assume that the sets l( x ) = G (n, m) indicated above are the m-planes in this manifold. Definition 1. We say that a set E ⊂ M, where M is one of the spaces Rn o r G ′(n, m), is (n, m) -convex if, for every point of the complement x ∈ M \ E, there exists an m-plane l such that x ∈ l and l I E = ∅. It is easy to verify that all convex domains and compact sets in Rn and m-convex sets in Rn satisfy this definition [1]. Remark 1. If, for a pair of manifolds ( M, M ∗ ) , where one manifold is the Euclidean space C n and the other one is a complex Grassmann manifold C G ′(n, m), one considers an analogous correspondence of submanifolds, then, by analogy with the real case, one can introduce the notion of complex (n, m) C -convexity, which coincides with linear convexity for m = n – 1. Definition 2. For an arbitrary set E ⊂ M, a subset E ∗ ⊂ M ∗ is called dual to the set E if E ∗ = {y ∈ M ∗ the m-plane l( y) does not intersect E}. It is clear that, by definition, E ∗∗ is dual to E ∗ . Consider the many-valued mapping Φ: M → M ∗ that associates a point x with the m-plane Φ( x ) = l ( x ). It is easy to verify that this many-valued mapping with compact images is continuous and there exists an analogous many-valued mapping from M ∗ into M, Φ: M ∗ → M . This mapping is of interest because we obviously have E ∗ = M ∗ \ Φ ( E ) and, therefore, we can obtain information about the dual set by studying its complement Φ( E ) . Institute of Mathematics, Ukrainian Academy of Sciences, Kiev. Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 53, No. 3, pp. 422–427, March, 2001. Original article submitted April 20, 2000. 482

0041–5995/01/5303–0482 $25.00

© 2001 Plenum Publishing Corporation

ON ( n, m ) -CONVEX S ETS

483

Let us investigate the properties of dual sets. Proposition 1. If E1 ⊂ E , then E1∗ ⊃ E ∗ and E1∗∗ ⊂ E ∗∗ . If follows from the relation E1 ⊂ E that Φ ( E1 ) ⊂ Φ ( E ) . Then E1∗ = M ∗ \ Φ ( E1 ) ⊃ M ∗ \ Φ ( E ) = E ∗ . Proposition 2. If E is a compact set, then E ∗ is open. Since the mapping Φ is continuous, Φ( E ) is a compact set. This means that E ∗ = M ∗ \ Φ ( E ) is open. Proposition 3. If E is open, then E ∗ is closed. Proof. Let E be open. If y o ∈ Φ( E ) , then the hyperplane l ( y o ) intersects E. There exists a small neighborhood U of the point y o such that, for any y ∈ U, the hyperplane l ( y) still intersects E. This means that U ⊂ Φ( E ). Therefore, Φ( E ) is open. Consequently, E ∗ = M ∗ \ Φ ( E ) is closed. Corollary 1. If E is open, then E ∗∗ is also open. We say that a set E ⊂ M is bounded if E is a compact set in M and relatively compact if the one-point compactification M U (∗) transforms the set E U (∗) into a compact subset M U (∗) . Remark 2. For bounded closed sets, Proposition 3 can be strengthened, namely, if E is an open bounded set, then E ∗ is a relatively compact set. Let T be a fixed κ -plane in Rn , 0 < κ < m. Under the mapping, the set of points x ∈ T corresponds to m planes in G ′(n, m). Note that the intersection of these planes is a manifold G(n – κ , m – κ ) ⊂ G(n, m). This manifold is called an ( m – κ ) -plane in G ′(n, m). This correspondence T ↔ G(n – κ, m – κ ) is an extension of the correspondence considered at the beginning of the paper for κ = 0 or m = κ because G(n – m, 0) ⊂ G(n, m) is a point of the space G ′(n, m) that defines the m -plane T. Proposition 4. Let f : R n → R l be an affine mapping of the space R n “onto” the space E ⊂ R l be (l, m) -convex. Then the set f –1 ( E ) ⊂ R n is (n, m + n – l ) -convex.

R l and let

Proof. Let x ∈ R n \ f –1 ( E ) . Consider a point f ( x ) and the m -plane L ⊂ R l that passes through the point f ( x ) and does not intersect E. Then f –1 ( L) is an affine submanifold of the space R n and f –1 ( L) I f –1 ( E ) = ∅. Furthermore, under the affine mapping “onto”, the ( m + n – l ) -plane is the preimage of the m -plane. Proposition 5.

( Uα Eα )

∗

=

I α Eα∗ .

Indeed, we have    U Eα  α 

∗

= M∗ \

U Φ ( Eα ) α

=

I ( M ∗ \ Φ ( Eα )) α

=

U Eα∗ . α

484

YU. B. ZELINSKII

Proposition 6. Suppose that a sequence of compact sets Eκ , κ = 1, 2, … , is such that

E=

I κ Eκ .

∗

Then E =

U

E∗. κ κ

Proof. It follows from the relation E κ ⊂ E that E κ∗ ⊂ E ∗ and ∗

U Eκ∗ ⊃ E∗ .

AND

I. V. M OMOT

Eκ +1 ⊂ Eκ a n d

Assume that a point y belongs

E κ∗ .

to E and, hence, the m -plane l ( y) does not intersect E. Assume that y ∉ Then the m-plane l ( y) intersects each of the sets Eκ , κ =1, 2, … . We obtain a system of imbedded compact sets Kκ = E κ I l ( y) , Kκ +1 ⊂ Kκ . Consequently, K = I Kκ ≠ ∅ , which contradicts the equality K = I Kκ = ( I Eκ ) I l ( y) = E I l ( y) = ∅. By virtue of the arbitrariness of the choice of the point y, we obtain the inverse inclusion

U Eκ∗ ⊃ E∗.

Therefore, E ∗ =

Uκ Eκ∗ .

Proposition 7. If a compact set E is approximated from the outside by a sequence of domains Dκ , κ = 1,

2, … , then E ∗ =

Uκ Dκ∗

and E ∗∗ =

I κ Dκ∗∗ .

Proof. By virtue of Proposition 6, we have E ∗ = Dκ +1 ⊂ Dκ . According to Proposition 1, we have

Dκ∗ + 1

U κ Dκ∗ .

⊃

Dκ∗+ 1

Further, we use the chain of inclusions Dκ+1 ⊂ ⊃ Dκ* . This yields

U κ Dκ∗ = U κ Dκ .

Therefore,

E ∗ = U κ Dκ∗ . It now follows from Proposition 5 that E ∗∗ = I κ Dκ∗∗ . The notation E1 ⊂ ⊂ E means the closed imbeddability of a subset, i.e., the closure of E1 ⊂ M , together with a certain open neighborhood of it, lies in E. Proposition 8. If E1 ⊂ ⊂ E , then E ∗ ⊂ ⊂ E1∗ and E1∗∗ ⊂ ⊂ E ∗∗ . Assume that a certain neighborhood U ( E1 ) = E2 ⊃ E1 lies in E. In this case, we get E1 ⊂ E1 ⊂ E2 ⊂ E . By virtue of Proposition 1, we have E1∗ ⊃ E1∗ ⊃ E2∗ ⊃ E ∗ . By virtue of Proposition 3, the set E2∗ is closed and, by virtue of Proposition 2, the set E ∗ is open. Therefore, E1∗ ⊃ ⊃ E ∗ . Repeating this reasoning once again, we complete the proof. Proposition 9. E ∗∗ ⊃ E and E ∗∗∗ = E ∗ . Proof. It follows from the definition of dual set that E ∗∗ ⊃ E and E ∗∗∗ ⊃ E ∗ . Applying Proposition 1 to the inclusion E ∗∗ ⊃ E , we obtain E ∗∗∗ = E ∗ . Theorem 1. In order that a set E be (n, m) -convex, it is necessary and sufficient that E ∗∗ = E . Proof. Assume that a set E ⊂ M is (n, m) -convex. Then, for any point x ∈ M \ E , there exists an m -plane l ( y) that passes through the point x and does not intersect E. Consider a point y ∈ E ∗ that defines this plane. The points of the plane l ( y) define in M ∗ a collection of m -planes that intersect one another at the point y. Each of these planes intersects E ∗ . Therefore, none of the points of the plane l ( y) , including x, can belong to E ∗∗ . Therefore, E ⊃ E ∗∗ and, by virtue of the previous proposition, they coincide. Let us prove the converse statement. Let E ∗∗ = E and let x ∈ M \ E = M \ E ∗∗ . Then there exists a point y ∈ E ∗ such that x ∈ l ( y). The plane l ( y) passes through the point x but does not intersect the set E because none of its points belongs to the set E ∗∗. Therefore, E is an (n, m) -convex set. The theorem is proved. This theorem yields the following obvious statement:


485

Corollary 2. For an arbitrary set E, the dual set E ∗ is (n, m) -convex. Proposition 10. The intersection of an arbitrary collection of (n, m) -convex sets is (n, m) -convex. Proof. Let E = Iα Eα , where Eα are (n, m) -convex sets. If x ∉ E , then there exist Eα such that x ∉ Eα and the m -plane L that contains x and possesses the property L I Eα = ∅. Then L I E = ∅. For a union of sets, even if they are ordered by inclusion, an analogous result is not true. This can easily be verified with the use of Example 2.1 in [1]. We can easily generalize this result to the case of arbitrary (n, m) -convexity. Proposition 11. Let {Eα} , α ∈ A, be a family of (n, m) -convex open sets Eα ⊂ Rn linearly ordered by inclusion, i.e., A is linearly ordered and α ≤ β ⇔ Eα ⊂ Eβ . Then the set E = U α∈A Eα is (n, m) -convex. Proof. Let x ∉ E . Then l ( x ) = G(n, m) is a compact manifold and x ∉ Eα for all α . It is clear that each intersection Eα∗ I l ( x ) of the closed set Eα∗ I l ′( x ) is also a compact set. It follows from the relation Eα ⊂ Eβ that Eα∗ ⊃ Eβ∗ and Eα∗ I l ( x ) ⊃ Eβ∗ I l ( x ). We obtain a system of imbedded compact sets in l ( x ) that has the

nonempty compact intersection A =

( I Eα∗ ) I l( x ) ⊂ E ∗ .

Definition 3. An open domain D in a manifold M is called weakly (n, m) -convex if, for every point x of the boundary ∂ D, there exists an m -plane l that passes through the point x and does not intersect D. It is clear that any (n, m) -convex domain is weakly (n, m) -convex. Proposition 12. Every weakly (n, m) -convex domain is a connected component of a certain (n, m) -convex open set. Proof. Let x be an arbitrary point of the boundary ∂ D. We take one of the m -planes that pass through x and do not intersect D. This m -plane can be written as l ( y) for a certain point y ∈ D ∗ . Therefore, every point of the plane l ( y) , including x, does not belong to D ∗∗ . Thus, none of the points of the boundary ∂ D belongs to D ∗∗ , but D ⊂ D ∗∗. Therefore, D is a connected component of the (n, m) -convex open set D ∗∗ . Definition 4. A closed set F ⊂ M is called weakly (n, m) -convex if it is approximated from the outside by a sequence of weakly (n, m) -convex domains. Proposition 13. Every weakly (n, m) -convex closed set is a connected component of an (n, m) -convex closed set. Proof. Let F be approximated from the outside by a sequence of weakly (n, m) -convex domains F =

I Dκ ,

κ = 1, 2, … , Dκ +1 ⊂ Dκ . By virtue of Proposition 12, Dκ coincides with a connected component of the set Dκ∗∗ . ∗∗

By virtue of Proposition 7, we have F ∗∗ = ( I Dκ ) taining F is necessarily contained in the intersection

=

I Dκ∗∗ .

I Dκ = F .

Therefore, the connected component F ∗∗ conm

If a collection of parallel m -planes that fills the entire space, i.e., a trivial stratification R n → R with the layer γ = R n – m , is given in the space R n , then the projection of R n to the layer R n – m is understood as a mapping that identifies points with m -planes. If a stratification p : G′(n, m) → G(n, m) is given, then we note that each section of the stratification intersects a fixed layer at a unique point. The mapping that associates each section

486

YU. B. ZELINSKII

AND

I. V. M OMOT

by an m -plane with the point of its intersection with a layer is called the projection of the set of sections to the layer γ, which is also homeomorphic to R n – m . The projection of the set E ( E ∗) to a layer is understood as the image of E ( E ∗) tions considered above. Note that if we have a section of an m -convex set E ⊂ M by an arbitrary m -plane E I l is associated with an m -plane in M ∗ that does not intersect E ∗ and, vice versa, is associated with an m -plane in M ∗ that intersects E ∗ . Taking the obvious equality account, we complete the proof of the proposition.

under one of the projecl, then every point x ∈ every point x ∈l \ E I l π M ( E ) = π M ( E ∗∗) into

Proposition 14. For the projection π M : E → γ , the set γ \ π M ( E) is homeomorphic to l I E ∗. The bounded part of the boundary is understood as the points of the boundary different from the compactifying point. Proposition 15. If G is a bounded open set, then a point y belongs to the bounded part of the boundary ∂G if and only if the m -plane l ( y) passes through a certain point of the boundary ∂G but does not intersect the set G. ∗

Proof. If the m -plane l ( yo ) , yo ∈ G∗ , does not intersect G but passes through a certain point x ∈∂G , then yo cannot lie in the interior of the relatively compact set G∗ ; indeed, if a certain neighborhood U ( yo ) of the point yo lies in G∗ , then all m -planes l ( y) , y ∈U ( yo ) , do not intersect G, which contradicts the assumption that the intersection l ( yo ) I ∂G is not empty. On the other hand, if y lies in the bounded part of the boundary ∂G , then y ∈ G∗ by virtue of the fact that G∗ is relatively compact; hence, the m -plane l ( y) does not intersect G. If l ( y) does not intersect the compact set G, then, by virtue of Proposition 2, y belongs to the open set G ∗ ⊂ G∗ and cannot lie on the boundary ∂G∗ . Therefore, l ( y) does not intersect G but necessarily intersects G∗ , which means that l ( y) passes through a certain point of the boundary ∂G . Proposition 16. If E ⊂ R n is an (n, m) -convex set, then its interior int E is also an (n, m) -convex set. Proof. It is necessary to prove that, for an arbitrary point x ∈∂E , there exists an m -plane that contains x and does not intersect int E. For the other points from R n \ E , this follows from the (n, m) -convexity of E. We consider a compact neighborhood U ( x ) of the point x. For a many-valued continuous mapping Φ, the image Φ (U ( x )) is a compact set. We choose a sequence {xn} ∈ (U ( x ) – x ) \ E such that lim n→∞ xn = x . It is clear that ∞  Φ  U xn  ⊂ Φ (U ( x )) .  n=1  For every point xn , there exists an m -plane l ( yn ) that does not intersect E. Therefore, there exists a point yn in the compact set Φ (U ( x )) that defines this plane. We choose a convergent sequence {yn′ } from the sequence {yn} . Then lim yn′ = yo ∈ Φ (U ( x )) . The point yo is associated with the m -plane l ( yo ) , which passes through the point x and does not intersect int E because the sequence of m -planes l ( yn ), each of which does not intersect int E , converges to l ( yo ) . We say that a mapping f is strictly invariant on a family ᑜ of subsets of a topological space if f maps any subset A ∈ ᑜ onto a subset B ∈ ᑜ.


487

Theorem 2. If a homeomorphic mapping f : Rn → Rn , n ≥ 2 , is strictly invariant on (n, m) -convex compact sets, then f is an affine mapping. Proof. If f is not an affine mapping, then, according to [1], there exists an m -plane l such that f ( L) is not an m -plane. Without loss of generality, we can assume that L = {x xm +1 = xm + 2 = K = xn = 0}, where xi is the i th coordinate of the point x. Let Kp be a sequence of cylinders of the form Kp = Qp × Bpm , where Qp =

{x ′ ∈ R

n–m

}⊂

1 1 ≤ x′ ≤ 1 – p p

Bpm = {x ′′ ∈ Rm

Rn – m ,

x ′′ ≤ p} ⊂ Rm ,

that converges to L. It is clear that all Kp are (n, m) -convex. Furthermore, the set A =

U p Kp

=

U p int K p

is

also (n, m) -convex. There is a unique m -plane l that passes through the origin of coordinates and does not intersect A. Since f preserves the (n, m) -convergence of compact sets, all f ( Kp ) are (n, m) -convex compact sets. It

is clear that Kp ⊂ Kp+1 . Therefore, f ( Kp ) ⊂ f ( Kp+1 ) . Furthermore, U f ( Kp ) = U int f ( Kp ) and int f (Kp ) ⊂ int f (Kp+1) (int f ( Kp )) ⊂ (int f ( Kp+1 )) , where (int f ( Kp )) is a closed set. According to Proposition 16, int f (K p ) is an (n, m) -convex set. Then, by virtue of Propositions 5 and 6, we have    U f ( K p )  p 

∗∗

  =  U int f ( K p )  p 

∗∗

 ∗ =  I (int f ( K p ))   p 

∗

=

U (int f ( K p ))

∗∗

=

p

Therefore, by virtue of Theorem 1 and the fact that f is homeomorphic, f ( A) =

U int f ( K p ) p

Up f (K p )

=

U f (K p ). p

is an (n, m) -con-

vex set. For the point f (0) , there exists an m-plane L1 ⊃ f (0) that does not intersect f ( A). The set f ( A) partitions the space Rn . Therefore, the point f (0) and, hence, the m-plane L1 must lie in the set f ( L) homeomorphic to the m-plane. Since a proper subset of the m-dimensional manifold f ( L) cannot be a manifold of the same dimension, we have f ( L) = L1. The contradiction obtained proves the theorem. Note that a different approach to the definition of convexity on Grassmann manifolds was proposed in [3]. REFERENCES 1. Yu. B. Zelinskii, Many-Valued Mappings in Analysis [in Russian], Naukova Dumka, Kiev (1993). 2. V. A. Rokhlin and D. B. Fuks, Beginner's Course in Topology [in Russian], Nauka, Moscow (1977). 3. F. E. Goodmen, “When is a set of lines in space convex?,” Notic. Amer. Math. Soc., 45, No. 2, 222–232 (1998).