Optimizing condition numbers

Optimizing condition numbers P IERRE M ARÉCHAL Toulouse (France) Joint work with Jane Ye Victoria (Canada)

Introduction p Some convexity properties of κ, κp and κp Basic facts in nonsmooth analysis Convergence analysis

The problem

Pierre Maréchal

Optimizing condition numbers


The problem (P)

Minimize κ(A) s.t. A ∈ Ω

Pierre Maréchal



The problem (P)


Ω compact convex subset of Sn+ (not containing the null matrix)

Pierre Maréchal



The problem (P)


Ω compact convex subset of Sn+ (not containing the null matrix) κ(A) condition number of A

Pierre Maréchal



The problem (P)


Ω compact convex subset of Sn+ (not containing the null matrix) κ(A) condition number of A    λ1 (A)/λn (A) if λn (A) > 0 ∞ if λn (A) = 0 and λ1 (A) > 0 κ(A) =   0 if A = 0 Pierre Maréchal



The problem (P)


Ω compact convex subset of Sn+ (not containing the null matrix) κ(A) condition number of A    λ1 (A)/λn (A) if λn (A) > 0 ∞ if λn (A) = 0 and λ1 (A) > 0 κ(A) =   0 if A = 0 λ1 (A) ≥ . . . ≥ λn (A) eigenvalues of A Pierre Maréchal



The problem (P)


Ω compact convex subset of Sn+ (not containing the null matrix) κ(A) condition number of A    λ1 (A)/λn (A) if λn (A) > 0 ∞ if λn (A) = 0 and λ1 (A) > 0 κ(A) =   0 if A = 0 λ1 (A) ≥ . . . ≥ λn (A) eigenvalues of A Pierre Maréchal



Properties of κ

Pierre Maréchal



Properties of κ

◮

κ is nonconvex

Pierre Maréchal



Properties of κ

◮

κ is nonconvex

◮

κ is not everywhere differentiable

Pierre Maréchal



Properties of κ

◮

κ is nonconvex

◮


◮

κ is quasiconvex (i.e. has convex level sets)

Pierre Maréchal



Properties of κ

◮

κ is nonconvex

◮


◮


◮

κ is difference convex

Pierre Maréchal



Properties of κ

◮

κ is nonconvex

◮


◮


◮

κ is difference convex

◮

κ is pseudoconvex

Pierre Maréchal



κ as a difference-convex function (from JBHU)

Pierre Maréchal




λ1 1 κ= = λn 2

"

1 λ1 + λn

Pierre Maréchal

2

# 1 2 − λ1 + 2 λn




λ1 1 κ= = λn 2

"

1 λ1 + λn

2

# 1 2 − λ1 + 2 λn

1/λn is log-convex (thus convex): ln

1 = − ln λn λn

Pierre Maréchal




λ1 1 κ= = λn 2

"

1 λ1 + λn

2

# 1 2 − λ1 + 2 λn


1 = − ln λn λn

λn is concave

Pierre Maréchal




λ1 1 κ= = λn 2

"

1 λ1 + λn

2

# 1 2 − λ1 + 2 λn


1 = − ln λn λn

λn is concave − ln is convex decreasing

Pierre Maréchal



Example: Markovitz model for portfolio selection

Pierre Maréchal




(M )

Minimize hx, Qxi s.t. x ∈ ∆n , hc, xi ≥ b

Pierre Maréchal




(M )


Q is a covariance matrix (to be inferred)

Pierre Maréchal




(M )


Q is a covariance matrix (to be inferred) o n P ∆n = x ∈ Rn+ | j xj ≤ 1 , c ∈ Rn and b ∈ R

Pierre Maréchal




(M )



Q is constrained to belong to some polytope P

Pierre Maréchal




(M )



Q is constrained to belong to some polytope P Minimize κ(Q) s.t. Q ∈ Sn+ ∩ P Pierre Maréchal



An auxiliary function

Pierre Maréchal



An auxiliary function κ(A) =

Pierre Maréchal

λ1 (A) λn (A)




λ1 (A) λn (A) p+1

λ p (A) κp (A) = 1 λn (A)

Pierre Maréchal




λ1 (A) λn (A) p+1

λ p (A) κp (A) = 1 λn (A) κp (A) → κ(A) pointwise as p → ∞

Pierre Maréchal




λ1 (A) λn (A) p+1

λ p (A) κp (A) = 1 λn (A) κp (A) → κ(A) pointwise as p → ∞ κpp (A) =

λp+1 1 (A) λpn (A)

Pierre Maréchal

is convex !




λ1 (A) λn (A) p+1

λ p (A) κp (A) = 1 λn (A) κp (A) → κ(A) pointwise as p → ∞ κpp (A) =

λp+1 1 (A) λpn (A)

is convex !

κp and κpp have the same minimizers Pierre Maréchal



An optimization strategy

Pierre Maréchal



An optimization strategy Replace the original quasiconvex problem Minimize κ(A) (P) s.t. A ∈ Ω

by the surrogate convex problem Minimize κpp (A) (Pp ) s.t. A ∈ Ω

and let p → ∞.

Pierre Maréchal



An optimization strategy Replace the original quasiconvex problem Minimize κ(A) (P) s.t. A ∈ Ω

by the surrogate convex problem Minimize κpp (A) (Pp ) s.t. A ∈ Ω

and let p → ∞.

Question: can we approach a solution to Problem (P) with solutions to the surrogate problems (Pp )? Pierre Maréchal



Main results

Pierre Maréchal



Main results Theorem (Exact approximation) ¯ p be a solution to Problem (Pp ). Then Let pk ↑ ∞, and let A k k ¯ the sequence (Apk ) has a subsequence which converges to a ¯ of Problem (P). global solution A

Pierre Maréchal




Theorem (Inexact approximation) ¯ εpk an εk -solution to ¯ k := A Let pk ↑ ∞, and let εk ↓ 0. Let A k ¯ k ) has a subsequence Problem (Ppk ). Then the sequence (A ¯ of Problem (P). which converges to a global solution A

Pierre Maréchal




Theorem (Inexact approximation) ¯ εpk an εk -solution to ¯ k := A Let pk ↑ ∞, and let εk ↓ 0. Let A k ¯ k ) has a subsequence Problem (Ppk ). Then the sequence (A ¯ of Problem (P). which converges to a global solution A ∀A ∈ Ω,

¯ k ) − εk κ(A) ≥ κ(A

Pierre Maréchal



Quasi-convexity

Pierre Maréchal



Quasi-convexity

Definition A function f : Rn → R is said to be quasi-convex if it has convex level sets.

Pierre Maréchal



Quasi-convexity

Definition A function f : Rn → R is said to be quasi-convex if it has convex level sets. levα (f ) := {x |f (x) ≤ α}

Pierre Maréchal



Quasi-convexity

Definition A function f : Rn → R is said to be quasi-convex if it has convex level sets. levα (f ) := {x |f (x) ≤ α}

Proposition κ, κp and κpp are quasi-convex.

Pierre Maréchal



Proof

Pierre Maréchal



Proof

=

=

lev0 (λ1 − αλn )

levα (κ) :=

A ∈ Sn++ |κ(A) ≤ α {0} ∪ A ∈ Sn++ |λ1 (A) − αλn (A) ≤ 0

Pierre Maréchal



Proof

=

=


levα (κ) :=


λ1 convex λn concave

=⇒ λ1 − αλn convex

Pierre Maréchal



Proof

=

=


levα (κ) :=




p+1 levα (κp ) = lev0 λ1 p − αλn

Pierre Maréchal



Proof

=

=


levα (κ) :=




p+1 levα (κp ) = lev0 λ1 p − αλn

κpp (A) ≤ α ⇔ κp (A) ≤ α1/p levα (κpp ) = levα1/p (κp ) Pierre Maréchal



Convexity of κpp

Proposition Suppose f : Rn → [0, ∞] is • quasiconvex, • lower semi-continuous, • positively homogeneous of degree p ≥ 1 (∀t > 0, ∀x ∈ Rn , f (tx) = t p f (x)). Then f is convex.

Pierre Maréchal



Building a convex set

Lemma The set-valued mapping r 7→ r · C is increasing on R+ if and only if C ⊂ Rn is a convex set containing the origin. ¯ is concave and nonnegative on its Consequently, if g : Rm → R domain dom g, then the set [ (g(y ) · C × {y }) y ∈dom g

is a convex subset of Rn × Rm .

Pierre Maréchal



Illustration y

←−g(y )·C×{y }

←−g(0)·C×{0} g(y )

building

[

(g(y ) · C × {y })

y ∈dom g Pierre Maréchal



Proof of the proposition

Pierre Maréchal



Proof of the proposition We can assume

WLOG

that ∃x0 : f (x0 ) < ∞.

Pierre Maréchal



Proof of the proposition We can assume WLOG that ∃x0 : f (x0 ) < ∞. f (0) = 0: Since f is lower semi-continuous, one has: f (0) = f (lim tx0 ) ≤ lim f (tx0 ) = lim t p f (x0 ) = 0. t↓0

t↓0

t↓0

Since f takes its values in [0, ∞], one must have f (0) = 0.

Pierre Maréchal



Proof of the proposition We can assume WLOG that ∃x0 : f (x0 ) < ∞. f (0) = 0: Since f is lower semi-continuous, one has: f (0) = f (lim tx0 ) ≤ lim f (tx0 ) = lim t p f (x0 ) = 0. t↓0

t↓0

t↓0

Since f takes its values in [0, ∞], one must have f (0) = 0. For all r > 0, levr (f ) = r 1/p · lev1 (f ): n o levr (f ) = {x |f (x) ≤ r } = x |r −1 f (x) ≤ 1 n o = x |f (r −1/p x) ≤ 1 n o = r 1/p x ′ |f (x ′ ) ≤ 1 = r 1/p lev1 (f ) Pierre Maréchal



Proof of the proposition 0 · lev1 (f ) := 0+ (lev1 (f )) = lev0 (f ): 0+ (lev1 (f )) = 0+ {x ∈ Rn |f (x) ≤ 1} \ = {βx ∈ Rn |f (x) ≤ 1} β>0

=

\

x ′ ∈ Rn |f (x ′ /β) ≤ 1

β>0

=

\

levβ p (f ) = lev0 (f )

β>0

Pierre Maréchal



Proof of the proposition 0 · lev1 (f ) := 0+ (lev1 (f )) = lev0 (f ): 0+ (lev1 (f )) = 0+ {x ∈ Rn |f (x) ≤ 1} \ = {βx ∈ Rn |f (x) ≤ 1} β>0

=

\

x ′ ∈ Rn |f (x ′ /β) ≤ 1

β>0

=

\

levβ p (f ) = lev0 (f )

β>0

Hence, the formula levr (f ) = r 1/p · lev1 (f ) holds for every r ≥ 0. Pierre Maréchal




epi f is convex epi f

= {(x, r ) ∈ Rn × R+ |f (x) ≤ r } [ (levr (f ) × {r }) = r ∈R+

=

[

r ∈R+

Pierre Maréchal

r 1/p · lev1 (f ) × {r } .




epi f is convex epi f

= {(x, r ) ∈ Rn × R+ |f (x) ≤ r } [ (levr (f ) × {r }) = r ∈R+

=

[

r ∈R+

r 1/p · lev1 (f ) × {r } .

The conclusion then follows by the Lemma.

Pierre Maréchal



The Clarke directional derivative

Pierre Maréchal



The Clarke directional derivative Suppose f : Rn → R is Lipschitz near x0 . Given v , the ratio f (x + tv ) − f (x) t

(t > 0)

is bounded for (x, t) sufficiently close to (x0 , 0). The Clarke directional derivative is then defined as f ◦ (x0 ; v ) = lim sup x→x0 t↓0

Pierre Maréchal

f (x + tv ) − f (x) . t



The Clarke subdifferential

Pierre Maréchal



The Clarke subdifferential The Clarke subdifferential of f at x0 is the subset of Rn defined by ∂f (x0 ) = ξ ∈ Rn ∀v ∈ Rn , hξ, v i ≤ f ◦ (x0 ; v ) .

Pierre Maréchal



The Clarke subdifferential The Clarke subdifferential of f at x0 is the subset of Rn defined by ∂f (x0 ) = ξ ∈ Rn ∀v ∈ Rn , hξ, v i ≤ f ◦ (x0 ; v ) .

Theorem Let f : Rn → R be locally Lipschitz and x0 ∈ Rn . Then • ∂f (x0 ) is convex compact; • for every v ∈ Rn , f ◦ (x0 ; v ) = max {hξ, v i|ξ ∈ ∂f (x0 )}.

Pierre Maréchal



A fundamental result

Pierre Maréchal




Let f : Rn → R be locally Lipschitz. Rademacher’s theorem then says that f is differentiable almost everywhere.

Pierre Maréchal




Let f : Rn → R be locally Lipschitz. Rademacher’s theorem then says that f is differentiable almost everywhere.

Theorem Let Ωf := {x ∈ Rn |f is not differentiable at x }. Then ∂f (x) is the convex hull of the set lim ∇f (xk ) xk ∈ Ωf , xk → x, lim ∇f (xk ) exists . k →∞

k →∞

Pierre Maréchal



Example 0.010 0.008

f (x) = x 2 sin

0.006 0.004 0.002 0.000 −0.002 −0.004 −0.006 −0.008 −0.010 −0.15

−0.10

−0.05

0.00

0.05

0.10

Pierre Maréchal

0.15


1 x


Example 0.010 0.008

f (x) = x 2 sin

0.006 0.004 0.002 0.000 −0.002 −0.004 −0.006 −0.008 −0.010 −0.15

−0.10

−0.05

0.00

0.05

0.10

0.15

• f is differentiable on R; • f ′ (0) = 0 and ∂f (0) = [−1, 1].

Pierre Maréchal


1 x


Clarke regularity

Pierre Maréchal



Clarke regularity Recall that the standard directional derivative is f ′ (x0 ; v ) = lim t↓0

f (x0 + tv ) − f (x0 ) t

whenever the limit exists.

Pierre Maréchal




f (x0 + tv ) − f (x0 ) t

whenever the limit exists. The function f is said to be Clarke regular at x0 (or merely regular at x0 ) if, for every v ∈ Rn , f ′ (x0 ; v ) exists and f ′ (x0 ; v ) = f ◦ (x0 ; v ).

Pierre Maréchal




f (x0 + tv ) − f (x0 ) t

whenever the limit exists. The function f is said to be Clarke regular at x0 (or merely regular at x0 ) if, for every v ∈ Rn , f ′ (x0 ; v ) exists and f ′ (x0 ; v ) = f ◦ (x0 ; v ).

Theorem Let f : Rn → R be continuously differentiable. Then f is Clarke regular and, for every x, ∂f (x) = {∇f (x)}. Pierre Maréchal



Example

f (x) = −|x|

Pierre Maréchal



Example

f (x) = −|x|

• ∂f (0) = [−1, 1], thus f ◦ (0, 1) = sup {ξ |ξ ∈ [−1, 1]} = 1; • f ′ (0, 1) = −1.

Pierre Maréchal



Example

f (x) = −|x|

• ∂f (0) = [−1, 1], thus f ◦ (0, 1) = sup {ξ |ξ ∈ [−1, 1]} = 1; • f ′ (0, 1) = −1. f is not regular at 0

Pierre Maréchal



Quotient rule Proposition ¯ be Lipschitz near x0 . Assume that Let f1 , f2 : Rn → R • f1 (x0 ) ≥ 0 and f2 (x0 ) > 0; • f1 and −f2 are Clarke regular at x0 . Then, f1 /f2 is Clarke regular at x0 and f1 f2 (x0 )∂f1 (x0 ) − f1 (x0 )∂f2 (x0 ) ∂ . (x0 ) = f2 f22 (x0 )

Pierre Maréchal



Composition rule Proposition Let S be a subset of Rn and x0 ∈ int S. Let f = g ◦ h, where h : S → R and g : R → R. Assume that • g is continuously differentiable at h(x0 ); • h is Lipschitz near x0 . Then ∂f (x0 ) = g ′ (h(x0 ))∂h(x0 ). Moreover, if g is continuously differentiable in a neighborhood of h(x0 ) and h is Clarke regular at x0 , then f is also Clarke regular at x0 . Pierre Maréchal



A regularity trick Lemma Let f : Rn → R be locally Lipschitz, x0 ∈ Rn and ϕ : R → R. Assume that • −f is Clarke regular at x0 ; • ϕ is continuously differentiable and non-decreasing at f (x0 ). Then −ϕ ◦ f is Clarke regular at x0 and ∂(−ϕ ◦ f )(x0 ) = −ϕ′ (f (x0 ))∂f (x0 ) = ϕ′ (f (x0 ))∂(−f )(x0 ).

Pierre Maréchal



Proof The formula is an immediate consequence of the chain rule (since f is Lipschitz near x0 ).

Pierre Maréchal



Proof The formula is an immediate consequence of the chain rule (since f is Lipschitz near x0 ). By the Mean Value Theorem applied to ϕ, −ϕ f (x0 + tv ) + ϕ f (x0 ) = ϕ′ (u) −f (x0 + tv ) + f (x0 ) for some u ∈ [f (x0 ), f (x0 + tv )].

Pierre Maréchal



Proof The formula is an immediate consequence of the chain rule (since f is Lipschitz near x0 ). By the Mean Value Theorem applied to ϕ, −ϕ f (x0 + tv ) + ϕ f (x0 ) = ϕ′ (u) −f (x0 + tv ) + f (x0 ) for some u ∈ [f (x0 ), f (x0 + tv )]. Now,

−ϕ(f (x0 + tv )) + ϕ(f (x0 )) t −f (x0 + tv ) + f (x0 ) t↓0 ′ −→ ϕ (f (x0 ))(−f )′ (x0 ; v ) = ϕ′ (u) t (the regularity of −f ensures existence of the limit).

Pierre Maréchal



Proof The formula is an immediate consequence of the chain rule (since f is Lipschitz near x0 ). By the Mean Value Theorem applied to ϕ, −ϕ f (x0 + tv ) + ϕ f (x0 ) = ϕ′ (u) −f (x0 + tv ) + f (x0 ) for some u ∈ [f (x0 ), f (x0 + tv )]. Now,

−ϕ(f (x0 + tv )) + ϕ(f (x0 )) t −f (x0 + tv ) + f (x0 ) t↓0 ′ −→ ϕ (f (x0 ))(−f )′ (x0 ; v ) = ϕ′ (u) t (the regularity of −f ensures existence of the limit). Thus, (−ϕ ◦ f )′ (x0 ; v ) = ϕ′ (f (x0 ))(−f )′ (x0 ; v ). Pierre Maréchal



Proof (end) Since

∂(−ϕ ◦ f )(x0 ) = ϕ′ (f (x0 ))∂(−f )(x0 ),

(−ϕ ◦ f )◦ (x0 ; v ) = =

max

s∈∂(−ϕ◦f )(x0 )

max

s′ ∈∂(−f )(x0 )

= ϕ′ (f (x0 ))

hs, v i

ϕ′ (f (x0 ))hs′ , v i max

hs′ , v i

s′ ∈∂(−f )(x0 )

= ϕ′ (f (x0 ))(−f )◦ (x0 ; v ) = ϕ′ (f (x0 ))(−f )′ (x0 ; v ) [ regularity of − f ] = (−ϕ ◦ f )′ (x0 ; v )

Pierre Maréchal

[ previous step ]



Subgradients

Proposition Let A ∈ Sn++ . Then: ∂κ(A) = λ1 (A)−1 κ(A) ∂λ1 (A) − κ(A)∂λn (A) ∂κpp (A) = κ(A)p (p + 1)∂λ1 (A) − pκ(A)∂λn (A) 1−p p + 1 p ∂λ1 (A) − κ(A)∂λn (A) ∂κp (A) = λ1 (A) p κ(A) p

Pierre Maréchal



Pseudoconvexity

Pierre Maréchal



Pseudoconvexity ¯ lsc and Lipschitz near x¯ . Let Ω ⊂ Rn , x¯ ∈ Ω and f : Rn → R,

Pierre Maréchal



Pseudoconvexity ¯ lsc and Lipschitz near x¯ . Let Ω ⊂ Rn , x¯ ∈ Ω and f : Rn → R, (1) We say that f is pseudoconvex at x¯ on Ω if ∀x ∈ Ω,

f ◦ (x¯ ; x − x¯ ) ≥ 0 =⇒ f (x) ≥ f (x¯ ).

We say that f is pseudoconvex on Ω if f is pseudoconvex at every x¯ ∈ Ω on Ω.

Pierre Maréchal



Pseudoconvexity ¯ lsc and Lipschitz near x¯ . Let Ω ⊂ Rn , x¯ ∈ Ω and f : Rn → R, (1) We say that f is pseudoconvex at x¯ on Ω if ∀x ∈ Ω,

f ◦ (x¯ ; x − x¯ ) ≥ 0 =⇒ f (x) ≥ f (x¯ ).

We say that f is pseudoconvex on Ω if f is pseudoconvex at every x¯ ∈ Ω on Ω. (2) We say that f is strongly pseudoconvex at x¯ on Ω if ∀ξ ∈ ∂f (x¯ ),

∀x ∈ Ω,

hξ, x − x¯ i ≥ 0 =⇒ f (x) ≥ f (x¯ ).

We say that f is strongly pseudoconvex on Ω if f is strongly pseudoconvex at every x¯ ∈ Ω on Ω. Pierre Maréchal



Optimality condition for pseudoconvex functions

Theorem

¯ be Let x¯ ∈ Ω ⊂ Rn , where Ω is closed convex. Let f : Rn → R lower semicontinuous and Lipschitz near x¯ . If f is pseudoconvex at x¯ on Ω, then the following are equivalent: (a) x¯ is a global minimizer of f on Ω; (b) 0 ∈ ∂f (x¯ ) + NΩ (x¯ ).

Pierre Maréchal



Proof of (b) ⇒ (a)

Pierre Maréchal



Proof of (b) ⇒ (a) Assume that 0 ∈ ∂f (x¯ ) + NΩ (x¯ ):

Pierre Maréchal



Proof of (b) ⇒ (a) Assume that 0 ∈ ∂f (x¯ ) + NΩ (x¯ ): There exists ξ0 ∈ ∂f (x¯ ) such that −ξ0 ∈ NΩ (x¯ )

Pierre Maréchal



Proof of (b) ⇒ (a) Assume that 0 ∈ ∂f (x¯ ) + NΩ (x¯ ): There exists ξ0 ∈ ∂f (x¯ ) such that −ξ0 ∈ NΩ (x¯ ) ∀x ∈ Ω,

h−ξ0 , x − x¯ i ≤ 0

Pierre Maréchal



Proof of (b) ⇒ (a) Assume that 0 ∈ ∂f (x¯ ) + NΩ (x¯ ): There exists ξ0 ∈ ∂f (x¯ ) such that −ξ0 ∈ NΩ (x¯ ) ∀x ∈ Ω, ∀x ∈ Ω,

h−ξ0 , x − x¯ i ≤ 0

max {hξ, x − x¯ i|ξ ∈ ∂f (x¯ )} ≥ 0 {z } | f ◦ (x¯ ;x−x¯ )

Pierre Maréchal



Proof of (b) ⇒ (a) Assume that 0 ∈ ∂f (x¯ ) + NΩ (x¯ ): There exists ξ0 ∈ ∂f (x¯ ) such that −ξ0 ∈ NΩ (x¯ ) ∀x ∈ Ω, ∀x ∈ Ω,

h−ξ0 , x − x¯ i ≤ 0

max {hξ, x − x¯ i|ξ ∈ ∂f (x¯ )} ≥ 0 {z } | f ◦ (x¯ ;x−x¯ )

∀x ∈ Ω,

f (x) ≥ f (x¯ )

Pierre Maréchal

[pseudoconvexity]



Optimality condition for κ

Pierre Maréchal



Optimality condition for κ

Proposition ¯ The function κ is strongly pseudoconvex. Consequently, A ¯ + NΩ (x¯ ). minimizes κ over Ω is and only if 0 ∈ ∂κ(A)

Pierre Maréchal



Proof

Pierre Maréchal



Proof ¯ = λ1 (A) ¯ −1 κ(A) ¯ ∂λ1 (A) ¯ − κ(A)∂λ ¯ ¯ ∂κ(A) n (A)

Pierre Maréchal




¯ ⇔ V = λ1 (A) ¯ −1 κ(A) ¯ V ∈ ∂κ(A)

¯ Vn V1 −κ(A) |{z} |{z}

¯ ∈∂λ1 (A)

Pierre Maréchal

¯ ∂λn (A)




¯ ⇔ V = λ1 (A) ¯ −1 κ(A) ¯ V ∈ ∂κ(A)

¯ Vn V1 −κ(A) |{z} |{z}

¯ ∈∂λ1 (A)

¯ ∂λn (A)

¯ n (A) = λ1 (A)−λ1 (A) ¯ +κ(A) ¯ −λn (A)+λn (A) ¯ λ1 (A) − κ(A)λ | {z } | {z } ¯ ≥hV1 ,A−Ai

¯ ≥h−Vn ,A−Ai

[λ1 convex]

[−λn convex]

Pierre Maréchal




¯ ⇔ V = λ1 (A) ¯ −1 κ(A) ¯ V ∈ ∂κ(A)

¯ Vn V1 −κ(A) |{z} |{z}

¯ ∈∂λ1 (A)

¯ ∂λn (A)



[λ1 convex]

[−λn convex]

¯ = λ1 (A)κ( ¯ A) ¯ −1 hV , A − Ai ¯ ¯ n (A) ≥ hV1 − κ(A)V ¯ n , A − Ai λ1 (A) − κ(A)λ {z } | ¯ ¯ −1 V λ1 (A)κ( A)

Pierre Maréchal




¯ ⇔ V = λ1 (A) ¯ −1 κ(A) ¯ V ∈ ∂κ(A)

¯ Vn V1 −κ(A) |{z} |{z}

¯ ∈∂λ1 (A)

¯ ∂λn (A)



[λ1 convex]

[−λn convex]

¯ = λ1 (A)κ( ¯ A) ¯ −1 hV , A − Ai ¯ ¯ n (A) ≥ hV1 − κ(A)V ¯ n , A − Ai λ1 (A) − κ(A)λ {z } | ¯ ¯ −1 V λ1 (A)κ( A)

¯ ≥ 0 =⇒ λ1 (A) − κ(A)λ ¯ n (A) ≥ 0 ⇐⇒ κ(A) ≥ κ(A) ¯ hV , A − Ai Pierre Maréchal



Our main convergence result

Pierre Maréchal



Our main convergence result

Theorem (Exact approximation) ¯ p be a solution to Problem (Pp ). Then Let pk ↑ ∞, and let A k k ¯ the sequence (Apk ) has a subsequence which converges to a ¯ of Problem (P). global solution A

Pierre Maréchal



Proof

Pierre Maréchal



Proof ¯p ) Since Ω is compact, taking a subsequence, we can assume that (A k ¯ ∈ Ω. We shall prove that 0 ∈ ∂κ(A). ¯ The conclusion will converges to some A follow by the pseudoconvexity argument.

Pierre Maréchal



Proof ¯p ) Since Ω is compact, taking a subsequence, we can assume that (A k ¯ ∈ Ω. We shall prove that 0 ∈ ∂κ(A). ¯ The conclusion will converges to some A follow by the pseudoconvexity argument. ¯p : Optimality condition for A k

¯p ) 0 ∈ λ1 (A k |

1−pk pk

¯p ) κ(A k

pk + 1 ¯ p ) +NΩ (A ¯ p )∂λn (A ¯ p ) − κ(A ¯p ) ∂λ1 (A k k k k pk {z } ¯p ) ∂κpk (A k

Pierre Maréchal



Proof ¯p ) Since Ω is compact, taking a subsequence, we can assume that (A k ¯ ∈ Ω. We shall prove that 0 ∈ ∂κ(A). ¯ The conclusion will converges to some A follow by the pseudoconvexity argument. ¯p : Optimality condition for A k

¯p ) 0 ∈ λ1 (A k |

1−pk pk

¯p ) κ(A k

pk + 1 ¯ p ) − κ(A ¯ p )∂λn (A ¯ p ) +NΩ (A ¯p ) ∂λ1 (A k k k k pk {z } ¯p ) ∂κpk (A k

¯ p ) and Vn(k ) ∈ ∂λn (A ¯ p ) such that ∈ ∂λ1 (A k k 1−pk ¯ p ) pk κ(A ¯ p )Vn(k ) + NΩ (A ¯ p ) pk + 1 V (k ) − κ(A ¯p ) 0 ∈ λ1 (A k k k k 1 pk (k )

Thus, there exist V1

Pierre Maréchal



Proof (end) Reminder: ∂λk (A) = co xx ⊤ |x ∈ Rn , kx k = 1, Ax = λk (A) · x (Cox, Overton, Lewis).

Pierre Maréchal



Proof (end) Reminder: ∂λk (A) = co xx ⊤ |x ∈ Rn , kx k = 1, Ax = λk (A) · x (Cox, Overton, Lewis). Thus ∂λk (A) is compact, and it follows that, taking a subsequence, we can assume that (k )

V1

¯1 ∈ ∂λ1 (A) ¯ →V

(k )

and Vn

¯n ∈ ∂λ1 (A), ¯ →V

by closedness of the multifunctions ∂λ1 and ∂λn .

Pierre Maréchal




V1

¯1 ∈ ∂λ1 (A) ¯ →V

(k )

and Vn

¯n ∈ ∂λ1 (A), ¯ →V

by closedness of the multifunctions ∂λ1 and ∂λn . Since NΩ is also a closed multifunction, we can pass to the limit in 1−pk ¯ p ) pk + 1 V (k ) − κ(A ¯ p ) pk κ(A ¯ p )Vn(k ) + NΩ (A ¯ p ). 0 ∈ λ1 (A k k k k 1 pk

Pierre Maréchal




V1

¯1 ∈ ∂λ1 (A) ¯ →V

(k )

and Vn

¯n ∈ ∂λ1 (A), ¯ →V

by closedness of the multifunctions ∂λ1 and ∂λn . Since NΩ is also a closed multifunction, we can pass to the limit in 1−pk ¯ p ) pk + 1 V (k ) − κ(A ¯ p ) pk κ(A ¯ p )Vn(k ) + NΩ (A ¯ p ). 0 ∈ λ1 (A k k k k 1 pk and obtain ¯ −1 κ(A) ¯ 0 ∈ λ1 (A)

¯ ¯n ¯1 −κ(A) V V |{z} |{z}

¯ ∈∂λ1 (A)

|

{z

¯ ∈∂κ(A)

Pierre Maréchal

¯ ∈∂λn (A)

¯ ⊂ ∂κ(A) ¯ + NΩ (A). ¯ +NΩ (A)

}


Optimizing condition numbers

Optimizing condition numbers

Suggest Documents

ON CONDITION NUMBERS FOR THE CANONICAL GENERALIZED ...

The generalized condition numbers - Australian Mathematical Society

Condition numbers and scale free graphs

Distributions of Demmel and Related Condition Numbers

Unifying Condition Numbers for Linear ... - Optimization Online

componentwise condition numbers for generalized matrix inversion ...

Computing least squares condition numbers on ...

Structured Hölder condition numbers for multiple ... - Infoscience

Structured Hölder condition numbers for multiple ... - Infoscience

condition numbers for structured least squares problems

Integration of Condition Monitoring in Self-optimizing ... - ThinkMind

Optimizing Lifetime Condition and Reliability of Deteriorating ... - RUN

Error Estimates and Condition Numbers for Radial Basis Function ...

Bounds for the Condition Numbers of Spatially ... - OSA Publishing

Condition Numbers and Backward Error of a Matrix ... - Springer Link

Computation of condition numbers for linear ... - Maths, NUS

The distribution of condition numbers of rational ... - Semantic Scholar

Condition numbers of stochastic mean payoff games and what they ...

on the probability distribution of condition numbers ... - Semantic Scholar

On the Explanatory Value of Condition Numbers for

1 Smoothed Analysis of Condition Numbers - Semantic Scholar

A comparison of eigenvalue condition numbers for ... - UCSB Math

Perturbation analysis and condition numbers of scaled total least ...

on the probability distribution of condition numbers of ... - Google Sites