The Finite Dimensional Normed Linear Space Theorem ... - Oswego

The Finite Dimensional Normed Linear Space Theorem Richard DiSalvo Dr. Elmer Mathematical Foundations of Economics Fall/Spring, 2011-2012

The claim that follows, which I have called the nite-dimensional normed linear space theorem, essentially says that all such spaces are topologically cases the intuition we obtain in

2

R, R ,

and

carry over into not only higher dimension

R

Rn

3

Rn

with the Euclidean norm. This means that in many

by imagining intervals, circles, and spheres, respectively, will

but also any vector space that has nite dimension. Through-

out this discussion I assume that all scalars for forming linear combinations in this nite dimensional linear space are real numbers (I have heard of complex vector spaces, but in this mathematical foundations for economics independent study we are solely concerned with real vector spaces). By nite-dimensional normed linear space I mean a set

X

of vectors along with addition of vectors and

X is closed under arbitrary linear || · || : X → R satisfying the usual axioms of a norm (e.g., the triangle inequality), with the induced distance function on X dened in the usual way: ρX (x, y) = ||x−y||. Finally, by nite dimensional, I mean that there exists a nite set of vectors {x1 , x2 , ..., xn } ⊆ X that are linearly independent and span X (every vector in X can be written as a linear combination of vectors in {x1 , x2 , ..., xn }, and none of the xi are redundant). A standard example of such a vector space is X = R3 with Euclidean norm and basis {e1 , e2 , e3 } = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}. These assumptions force X to be highly structured. In particular, it can be shown that every open ball in X is simply a translation and rescaling of the unit open ball centered around the origin. So the metric on X is, in this context, determined entirely by the shape of its unit ball. For example, the Euclidean norm

scalar multiplication satisfying the usual axioms for a vector space (e.g., combinations of its elements), and a norm function

has this unit ball be the normal sphere; the taxicab norm leads to a diamond-shape; the sup norm has its unit ball a cube. Within this context, we have the following.

Claim (The Finite-Dimensional Normed Linear Space Theorem ):1 1. Any nite-dimensional normed linear space is complete (Cauchy

⇐⇒

convergent for sequences in that

space). 2. In any nite-dimensional normed linear space, a set is compact if and only if it is closed and bounded (Compact

⇐⇒

Closed & Bounded.

space will always be

Rn )

Interestingly, in some applied contexts (when e.g.

the linear

this is used as the denition for compact set.).

3. All norms on a nite dimensional normed linear space induce the same topology on that space; that is, all norms dene (through their induced metric) the same open sets on that space.

(I.e., it does

not matter what norms are used for purposes of answering the question what are the open sets, and consequently answering questions of what are the closed sets, bounded sets, what sequences convergent and to what limiting vectors, and all other questions that are dened in terms of open sets.)

1I

followed the exercises on pages 122 through 125 of the following in order to develop these arguments:

Carter, Michael.

Foundations of Mathematical Economics.

Cambridge: MIT Press, 2001.

1

Lemma (the ratio of the total representation magnitude and the magnitude of any vector in a normed linear space is bounded):2 Let

S = {x1 , x2 , ..., xn } be a linearly independent c > 0 such that for every x ∈linS

set of vectors in a normed linear space. Then there is

a real number

||x|| ≥ c (|α1 | + |α2 | + · · · + |αn |) where

x

Proof:

is expressed as

x = α1 x1 + α2 x2 + · · · + αn xn .

∀c > 0, ∃w ∈lin{x1 , x2 , ..., xn } such that ||w|| < w = α1 x1 + α2 x2 + · · · + αn xn . 1 Step 1. Since the above is true for all positive c, it is true for all c in the descending set { m |m ∈ N}. 1 ∞ So let us consider the sequence { }m=1 . For every term in this aforementioned sequence we are guaranteed m m m |αm m 1 |+|α2 |+···+|αn | where the αi s are taken from (by the supposition) a wm ∈linS such that ||wm || < m 3 the basis representation for wm . Choose any particular list of these wm to dene the sequence {wm }. I seek to push the sequence of norms {||wm ||} to tend to 0, but it is not clear that I can do that with m m m the sum |α1 | + |α2 | + · · · + |αn | in the numerator of each norm. But I can dene another sequence {ym }, to be: given the chosen {wm } sequence, let ym = |αm |+|αm1|+···+|αm | wm (∀m), where the αi s are n 1 2 taken from the basis representation for wm . Then the norm of each term in the sequence {ym } is ||ym || = |+···+|αn | 1 1 1 m · |α1 |+|α2m = m , again where the ai s are from the basis ||wm || < |α1 |+|α2 |+···+|α |α1 |+|α2 |+···+|α n| n| representation for wm . Consequently, {yn } denes a sequence of vectors with corresponding sequence of 1 norms ||ym || < m → 0 (as m → ∞). m m m Step 2. My expression for {ym } can be written as ym = β1 x1 + β2 x2 + · · · + βn xn where the sum of Pn m 4 the absolute values of the betas satises For each m, for each i = 1, 2, 3, ..., n, we must i=1 |βi | = 1. m then have |βi | ≤ 1 (simply because the sum of these numbers is 1 and they are all positive, so none of Suppose for the sake of contradiction that

c (|α1 | + |α2 | + · · · + |αn |),

2 Comment:

where

My statement here is an expression of a rewriting the claim as:

the claim) such that for any

∃M > 0

(where

M =

x 6= 0, |α1 | + |α2 | + · · · + |αn | ||x|| there is a real number c > 0

1 in the statement of c

M ≥ Of course another way to express the thing is that

c≤

small enough so that for any

x 6= 0

||x|| |α1 | + |α2 | + · · · + |αn |

it is clear that there must be a small enough choice of

c

(or large enough choice of

M)

that works

globally

in a particular

given linear space using a particular basis. -

{(1, 0), (0, 1)} for p R2 . Then (x, y) ∈ (x, y) = x(1, 0)+y(0, 1) by this basis. I used the Euclidean norm of (x, y), that is ||(x, y)|| = x2 + y 2 . Given any xed k = ||(x, y)||, then it is intuitively clear that to maximize |x| + |y| we must choose (x, y) to be on a 45 degree ray emanating from the origin. Since the same situation occurs in all four directions, we might as well suppose that x and y √ are on the positive 45 degree ray, and so x = y ≥ 0. In such a case then ||(x, y)|| = x 2 and x + y = 2x. We want to choose c so as to ensure that ||(x, y)|| ≥ c(|x| + |y|), and this argument shows that in the worst case this expression will look like (for a √ √ given (x, y)) ||(x, y)|| = x 2 ≥ c(2x) =⇒ 2 ≥ 2c (the inequality is denitely true if x = 0), and so it would suce to take c √ Comment: To build my intuition, I tried nding such a

c

in

R2 .

Take, for example, the basis

R2 is expressed as

to be

2 1 (or √ ). 2 2

In three space (but same context), my intuition leads me to believe that a worst case would be again all coordinates being the same, resting upon the 45 degree ray in the positive octant. In such a case the expression boils down to

c=

√1 n

√

3 ≥ 3c,

that is, we can take

c=

3 3

√ x 3 ≥ 3cx,

which

√1 . The pattern is interesting enough to make a conjecturethat 3 will work in the statement of the lemma in Rn normed with Euclidean measurement using the standard basis

would be satised if taking

√

=

vectors as our linearly independent spanning set. The lemma, of course, is much more general, since it deals with arbitrary normed linear spaces and arbitrary spanning sets.

3 That

is,

{xm }

1 {m } in that each term xm in the {xm } sequence lists vectors 1 c is taken to be m . term wm by dividing wm by the exact necessary scalar for this

is a sequence that corresponds to the sequence

that are assumed to contradict the lemma we're trying to show when

4 They

satisfy this sum because I dened each

ym

from the

to happen. To be more precise, let

m m wm = αm 1 x1 + α 2 x2 + · · · + α n xn

be the basis representation for the

2

mth

term in the

w

sequence.

{ym }

them can be bigger than 1). The weights on the basis vectors used to write each term in the sequence individually dene sequences of real numbers, which we may denote coordinate sequences is bounded (by

1

{βik }∞ k=1 , i = 1, 2, 3, ..., n.

Each of these

in particular).

{β1k }, is bounded, the Bolzano-Weierstrass Then let p(m) be the strictly increasing sequence

We proceed as follows. Since the rst coordinate sequence, theorem guarantees that it has a convergence subsequence. of natural numbers such that the subsequence the limit of this sequence by

β¯1 .

Apply

p(·)

p(m)

{β1

}

of the rst coordinate sequence converges. Denote

to form sequence:

p(m)

yp(m) = β1

p(m)

x 1 + β2

x2 + · · · + βnp(m) xn

we know that the rst coordinate sequence of this new sequence converges to

β¯1 , but we know little about

the others so far. Yet, they are subsequences of their original coordinate sequences, and so they are bounded sequences of real numbers! So we are now guaranteed, again by the Bolzano-Weierstrass theorem, that the

p(m)

β2

q be the function from the range of p(m) q(p(m)) ¯ into the positive integers such that β2 converges to β2 . Now, apply this this function q to each of the coordinate sequences of yp(m) 's basis representation, obtaining a new sequence yq(p(m)) dened by: sequence of real numbers

has a convergent subsequence. Let

q(p(m))

yq(p(m)) = β1 q(p(m))

β1

Note that now

q(p(m))

x1 + β2

x2 + · · · + βnq(p(m)) xn

, being a subsequence of the convergent sequence

p(m)

β1

Also, now have have pinned down the second coordinate, so that it converges to Continue this process. That is, the third coordinate sequence

q(p(m))

{β3

, must still converge to

β¯2 .

β¯1 .

} in the above sequence of vectors

is a bounded sequence of real numbers and therefore has a convergent subsequence. Use the mapping which gives this convergent subsequence, which is from the range of

q(p(m))

into

N,

in order to restrict every

coordinate sequence so that the third coordinate sequence converges. Note that rst and second coordinate sequences still converge.

We can repeat this argument,

n

times in total, until every coordinate sequence

converges. Let

∗ ym = β1sm x1 + β2sm x2 + · · · + βnsm xn be the sequence of vectors thus obtained.

∗ ym

ym had the property that the 1, then every term in the sequence sm sm ∗ s has this property as well. The sequence {|β1 | + |β2 | + · · · + |βnm |} = {1} converges to 1. We know ym sm sm ¯ that if the sequence {β1 } converges to β1 , then the sequence {|β1 |} converges to |β¯1 |. We also know that Step 3. Now

is a subsequence of

ym .

Since every term in the sequence

sum of the absolute values of the weights in the basis representation was

if we have any set of sequences all of which converge individually, then their sum converges to the sum of the

sm {|β1sm | + |β2s m | + · · · + |β |} must converge to β¯1 + β¯2 + · · · + β¯n . n ∗ Since the limit of a sequence is unique, then in fact β¯1 + β¯2 + · · · + β¯n = 1. So the sequence ym converges ∗ ¯ ¯ ¯ ¯ to the vector y = β1 x1 + β2 x2 + · · · + βn xn where not all of the βi can be zero, since the sum of the absolute values of these coecients is 1. ∗ ∗ ∗ Now we have it. We know ||ym || → 0. Therefore, ||ym || → 0. Hence ym → 0. Yet, {ym } converges to

individual limits. So then the sequence

a vector which is a weighing of the basis vectors such that at least one weight must be nonzero. Therefore,

∗ {ym }

actually does not converge to the zero vector

claimed.

Then we dened

ym

So there must always exist such a constant

c,

as

by

ym = In the

⇒⇐.

♦

n=2

1 1 m m w = m (αm 1 x1 + α2 x2 + · · · + αn xn ) m | + · · · + |αm | m m | + · · · + |αm | |αm | + |α |α | + |α n n 1 2 1 2

case, saying that the absolute values of the betas sum to 1 is the same as saying in the expression for

It's clear that this argument can be

α1 α2 |α1 | + |α2 | |α | + |α | + |α | + |α | = ||α | + |α || = 1 1 2 1 2 1 2 generalized for arbitrary n ∈ N.

3

ym

above:

Claim (1):5 Every nite-dimensional normed linear space is complete. Proof: Let X be a nite dimensional normed linear space of dimension n.

X converges X . Then each

arbitrary Cauchy sequence in

{x1 , x2 , ..., xn }

be a basis for

We must show that any

X . Let {xm } be an arbitrary Cauchy m term x has a unique representation: in

sequence in

X.

Let

xm = α1m x1 + α2m x2 + · · · + αnm xn Let

x ∈ X,

c

be the constant guaranteed by the previous lemma. That is, let

the product of

c

c

be that constant so that for any

times the sum of the absolute value of the weights in the basis representation of

(by the basis I specied in the rst paragraph above) is no greater than the norm of Since

{xm }

is a Cauchy sequence in

X,

c > 0

for any

there exists an

N ∈N

x

x.

such that, for all integers

r, m ≥ N , ||xm − xr || < c Substituting the representations for

xm

xr

and

by the basis vectors, we obtain:

|| (α1m x1 + α2m x2 + · · · + αnm xn ) − (α1r x1 + α2r x2 + · · · + αnr xn ) || < c some manipulation, and we obtain:

|| [α1m − α1r ] x1 + [α2m − α2r ] x2 + · · · + [αnm − αnr ] xn || < c By the lemma we know that the constant

c

satises:

c (|α1m − α1r | + |α2m − α2r | + · · · + |αnm − αnr |) < || [α1m − α1r ] x1 + [α2m − α2r ] x2 + · · · + [αnm − αnr ] xn || < c so that

|α1m − α1r | + |α2m − α2r | + · · · + |αnm − αnr | < c/c =  Now consider

|αim − αir |

for any

i = 1, 2, ..., n.

It is clear that this, being a term in the sum of all positive

terms of the left hand side, is at least as small as the right hand side of the above inequality. Therefore, we can say for any

i, |αim − αir | ≤ |α1m − α1r | + |α2m − α2r | + · · · + |αnm − αnr | < 

This shows that given any between the

xr

ith

>0

N so that for all m, r ≥ N the distance xm and the ith weight in the basis representation for any  > 0, ∃N ∈ N such that, for all m, r ≥ N ,

we can nd a positive integer

weight in the basis representation for

is strictly smaller than

.

That is, for each

i,

for

|αim − αir | <  Therefore each sequences of weights

{αim }∞ m=1 , i = 1, 2, 3, ..., n,

(which we called the coordinate sequences

R is complete, for each i, each such {αim } converges in R to some αi ∈ R. m Dene x = α1 x1 + α2 x2 + · · · + αn xn . Then x ∈ X . It is clear that x → x, since each of the coordinates m in the basis representation for x converges to the corresponding coordinate of x. This shows that every Cauchy sequence in X converges. Therefore, X is complete. ♦ in the lemma) are themselves Cauchy sequences of real numbers. Since sequence

5 Comment:

This argument rests on the fact that

that we've been taking our scalars from

R.

R

is complete. In order to get this into the game, we heavily rely on

Given that we can target any point in our linear space by weighting a nite basis

for that space, we can in fact think of our points as, essentially, these weights. Which is to say, our points are eectively in The argument just expresses these thoughts more formally.

4

Rn .

Claim (2):

In a nite dimensional normed linear space, a set is compact if and only if it is closed and

bounded.

Proof:

The proof that any compact set is both closed and bounded is on page 61 of Carter; I do not

repeat the argument. That the converse holds in any nite dimensional normed linear space is the result I seek to show here. Let S be a closed and bounded subset of a nite-dimensional normed linear space X . Let X have the basis {x1 , x2 , ..., xn }. Let {xm } be a sequence in S . We want to show that {xm } has a convergent subsequence m (sequential compactness). Let x = α1m x1 + α2m x2 + · · · + αnm xn denote the basis representation of xm . Since S is bounded, then it is contained in an open ball. But in a normed linear space, every open ball is merely a translation and rescaling of the unit ball. So say S is contained in the open ball x0 + rB , where r > 0 is a real number representing the scaling of the unit ball B , and x0 is the translation of the unit ball. m Then the magnitude of any vector in S can be no greater than M = ||x0 || + |r|. Therefore any term x ∈S m has ||x || ≤ M , and so by the lemma there is a constant c such that

c|αim | ≤ c (|α1m | + |α2m | + · · · + |αnm |) < ||xm || ≤ M i = 1, 2, 3, ..., n; that is, |αim | < M/c. Since M/c {αim } of the sequence {xm } is bounded.

for any sequence

is a constant, this shows that each coordinate

We are going to apply the same argument as in the lemma. Since the coordinate sequence

p(m) bounded sequence of real numbers it has a convergent subsequence {αi } that converges to the subsequence formed by selecting coordinates: p(m)

xp(m) = α1 such that

p(m)

→ α1 ,

α1

p(m)

x1 + α 2

α1 .

{αim }

is a

Construct

x2 + · · · + αnp(m) xn

and who knows what the other coordinate sequences doexcept that they are

bounded sequences of real numbers and therefore have convergent subsequences. So repeat the argument

p(m)

q(p(m))

{α2 }, so that for some function q(·), α2 → α2 . Continuing in this way we obtain m m a subsequence of {x }, call it {x∗ } whose coordinate scalars converge to the vector (α1 , ..., αn ). Let x∗ = α1 x1 + α2 x2 + · · · + αn xn . Then xm ∗ → x∗ because every coordinate in the former converges to the corresponding coordinate of the latter. Since S is closed, every convergent sequence whose terms all m come from S converges to an element of S , so x∗ ∈ S . Therefore the arbitrary sequence {x } in S has a 6 subsequence which converges in S . Hence, S is compact. ♦

for the sequence

6 Speaking

of convergence, the following conjecture arose when I was experimenting with some norms on

R2 .

I have not

proven it. Conjecture: the sequence of norms

{||x||2n }∞ n=1

converges to

||x||p =

q p

||x||∞ ,

where

xp1 + xp2 + · · · + xpn

||x||∞ = max{|x1 |, |x2 |, ..., |xn |} Comment: Looking at this conjecture a half of a year later, it seems even better to conjecture that

lim

p→∞

p p |x1 |p + |x2 |p + · · · + |xn |p = max{|x1 |, |x2 |, ..., |xn |}

Well, the CES production function looks like:

q p α1 xp1 + α2 xρ2 + · · · + αn xρn where the

xi ≥ 0

αi > 0. As p → −∞, this function converges to min{x1 , x2 , ..., xn } n = 2). But the p-norm is just this production function with p > 1.

and the

in February for the case

5

(this was proven by Dr. Elmer

Lemma (exercise 1.212):7 are positive constants

A

and

B

Let

|| · ||a

and

so that for any

|| · ||b be x ∈ X,

two norms on a given linear space

X.

Assume there

||x||a ≤ A||x||b ||x||b ≤ B||x||a A as a number that will always stretch the vector x to greater magnitude when measured b than it is when measured by norm A; and similarly for the number B .) Then the two norms || · ||a and || · ||b are topologically equivalent, by which I mean (not having studied any formal topology) that any open ball around a vector x that we construct by using norm a contains within it an open ball around x constructed using norm b, and the converse. Consequently every open set in X under || · ||a is open when X is analyzed under || · ||b , and the converse. Hence, all things dened in terms of the open sets on X , will (I'm thinking of

in norm

result in the same notions, regardless of which norm we use.

Proof:

a.

Let

Bra (x)

be an arbitrary open ball of radius

r

around the point

x∈X

constructed using norm

Since every open ball in a normed linear space is the same as a rescaling and translation of the unit open

Bra (x) = x + rB a for unit ball B a = {x| ||x||a < 1}. 1 1 b 1 b Consider A · B = {x| ||x||b < A } = {x| A||x||b < 1}. Now whenever x ∈ A B , then A||x||b < 1, and 1 b a a we know by the given equations that ||x||a ≤ A||x||b < 1, hence x ∈ B . Therefore A B ⊆ B . Then 1 b a x + r A B ⊆ x + rB . Therefore any open ball under norm a has within it an open ball under norm b. 1 b a On the other hand, if Br (x) is an arbitrary open ball around x using norm b, then B · B = {x| ||x||a < 1 1 a b b B } = {x| B||x||a < 1} ⊆ B , and so x + r B B works as a suitable open ball contained within Br (x), as desired. ♦ ball, we may write

7 Comment:

I recall once seeing the hypotheses of this lemma taken as the

sometimes phrased as:

∃C, D,

positive constants, so that

denition

of two norms being equivalent. It is

∀x ∈ X :

C||x||a ≤ ||x||b ≤ D||x||a but dividing over and breaking these inequality into two parts leads to the assumption I have made.

x, if you measure the magnitude of x using b, then, D you can stretch x by to make it at least as large as its magnitude under b, under norm a; and there is a constant quantity C you can squash x by in order to make its magnitude smaller under norm a than it is under norm b. Because C and D are constants, we can do this globally in the linear space; and so, any open ball under b will both be contained in, and contain within it, and open ball under norm a. This is enough for us to conclude that the norms are topologically equivalent, since then all the open sets under norm a will be the same under norm b, and that is all we need to dene the topological properties of X . [Note that these intuitive characterizations rest on the norm axiom: ||cx|| = |c|||x|| = c||x|| for c > 0, and cx is a rescaling of x.] Intuitively, the phrasing above leads to this characterization: given any

there is a constant quantity

6

Claim (exercise 1.213):

In a nite dimensional linear space, any two norms are topologically equivalent.

Proof:

be two arbitrary norms in a nite dimensional linear space

Let

||·||a

and

||·||b

X

with

dim X = n.

I will show that the previous lemma holds between these two norms using the lemma proven at the beginning of this paper, thus showing that these two norms are in fact topologically equivalent (as discussed). Let unique

x ∈ X be arbitrary. Let {x1 , x2 , x3 , ..., xn } be a basis for X . Then x = α1 x1 + α2 x2 + · · · + αn xn αi 's.8 We know by our initial lemma that there are constants ca and cb such that

for

||x||a ≥ ca [|α1 | + |α2 | + · · · + |αn |] ||x||b ≥ cb [|α1 | + |α2 | + · · · + |αn |] on the other hand by the triangle inequality

||x||a = ||α1 x1 + α2 x2 + · · · + αn xn ||a ≤ |α1 |||x1 ||a + |α2 |||x2 ||a + · · · + |αn |||xn ||a ≤ max {||xi ||a } 1≤i≤n

1≤i≤n

ka = max1≤i≤n {||xi ||a }

and

kb = max1≤i≤n {||xi ||b }.9

For simplicity of notation, let

|αi |

i=1

||x||b = ||α1 x1 + α2 x2 + · · · + αn xn ||b ≤ |α1 |||x1 ||b + |α2 |||x2 ||b + · · · + |αn |||xn ||b ≤ max {||xi ||b } Let

n X

n X

|αi |

i=1

Pn

i=1

|αi | = M .

Then the above four statements say:

ka M ≥ ||x||a ≥ ca M kb M ≥ ||x||b ≥ cb M By using the diagonals of these inequalities we obtain the result we need. Since the above implies:

ka M ≥ ||x||a ||x||b ≥ cb M then rearranging we obtain:

ka ||x||b ≥ ||x||a cb through similar logic we may obtain:

kb ||x||a ≥ ||x||b ca Let

ka cb

=A

and

kb ca

= B.

Then we have:

||x||a ≤ A||x||b ||x||b ≤ B||x||a these

A, B

are the constants we sought.

8 Comment:

♦

x and basis set for X , the (1, 2) = 1(1, 0) + 2(0, 1) in R2 , regardless of the constants ca and cb are clearly not (we

A hurdle (maybe) for understanding this argument is to realize that, given any

representation-by-basis for

x

is completely unrelated to the norm on

whether our norm is Euclidean, taxicab, sup, etc.

This

X.

The point

is invariant to norm, but

xed the norm in the lemma where we obtained these constants, and we showed that they must exist for any norm, but they are denitely functions of the norm we choose).

9 Comment:

The maximum norms in the basis set plays a pivotal role in this proof. They exist because we are in

dimensional space.

7

nite