Chapter 6: Cauchy sequences and completeness

It’s worth noting that if A is closed in X, and if f : X ! Y is continuous, then f (A) need not be closed in Y . For example, consider the map f: R ! R x 7 ! e x This is continuous, but the image of the closed set [0, 1) is (0, 1], which is not closed in R.

6

Cauchy Sequences and Completeness.

There is a problem with the definition of convergence that we met earlier. We don’t know any way to test for convergence without first guessing a limit, and then verifying Definition 3.1. It may not be possible to guess exactly what the limit must be, and it may only be possible to give an approximation. Example 6.1. Define f : R2 ! R2 by f (x, y) =

✓

cos y sin x + 1 , 3 2

◆

,

define (x0 , y0 ) = (0, 0), and let (xn+1 , yn+1 ) = f (xn , yn ). Then (x1 , y1 ) = (0.3333, 0.5000), (x2 , y2 ) = (0.2925, 0.6636),. . . , (x7 , y7 ) = (0.2687, 0.6329), (x8 , y8 ) = (0.2688, 0.6328),. . . . It certainly looks like it converges to a limit (0.269, 0.633) (to 3 decimal places). But we haven’t actually checked the definition of convergence – to do this, we need to give the limit a, and then test the definition. Example 6.2. Before we go on, here’s a warning example! This example looks like it ought to converge – but then it doesn’t! We’ll use the iteration defined by xn+1 =

x2n

0.01 , 2xn

and take the initial value as x0 = 10, say. Here are the first few values of the iteration: x1 = 4.9995, x2 = 2.4987, x3 = 1.2474, x4 = 0.6197, x5 = 0.3018, x6 = 0.1343, x7 = 0.0299, x8 = 0.1521, x9 = 0.0432, x10 = 0.0943. Although the first six or seven terms clearly seemed to tend to 0, something funny began to happen after that. Indeed, if x were to be a limit of the iteration, it would have to satisfy x=

x2

0.01 , 2x

and, rearranging, we find that x2 = 0.01, which has no real solution. So we have a sequence which looks like it converges, and then it doesn’t. So there is no limit. We could have replaced 0.01 by 0.000001, say, and the sequence would have looked even more like it converged (as long as we only worked to 4 decimal places again). These examples hopefully show that looking at the first few terms of the sequence may not be enough to check the definition, and to decide whether the sequence converges or not. What we’d really like is a test for convergence which didn’t need us to guess a limit. If we are given a sequence, is it possible to tell just by looking at its terms whether it converges or not, without needing prior knowledge of the limit? MAS331

33

Autumn 2011-12

6.1

Cauchy Sequences.

To say that a sequence converges to a limit means, roughly, that the terms of the sequence get closer and closer to the limit. One consequence of this is that the terms will get closer and closer to each other. Maybe this statement — that the terms get closer together — is enough to guarantee that the sequence converges to a limit. This is the idea behind a Cauchy sequence. Definition 6.3. We say that a sequence x1 , x2 , . . . in a metric space X is a Cauchy sequence if: For all ✏ > 0, we can find N such that d(xm , xn ) < ✏ whenever m, n > N . To see that this is a good internal test for convergence, we’d need to check that sequences converge precisely when they satisfy this Cauchy condition. Our first indication that this is a good test comes with the following result, which shows that convergent sequences are Cauchy. Lemma 6.4. Convergent sequences are always Cauchy. Proof. Suppose that the sequence (xn ) converges to x. Let ✏ > 0. Then there is some N such that d(xn , x) < 2✏ for all n > N . It follows that, for n, m > N , we have d(xn , xm ) 6 d(xn , x) + d(x, xm )
1. This says that the distance between consecutive terms xm , xm+1 halves every time m increases. Then (xn ) is Cauchy. To show this, first note that by the assumption we have d(xn , xn 1 ) 6 2n1 2 d(x2 , x1 ), so that if n > m we have d(xm , xn ) 6 d(xm , xm+1 ) + · · · + d(xn 1 , xn )  1 1 6 d(x1 , x2 ) ⇥ m 1 + · · · + n 2 2 2  1 1 6 d(x1 , x2 ) ⇥ m 1 + m + · · · 2 2 d(x2 , x1 ) = , 2m 2 where we used the formula for geometric series on the last line. This last quantity con2 ,x1 ) verges to 0 as m ! 1. So for ✏ > 0 let N be large enough that d(x < ✏ for m > N . 2m 2 Then d(xn , xm ) < ✏ whenever n, m > N . This proves that (xn ) is Cauchy. MAS331

34

Autumn 2011-12

Example 6.6. Consider the space C[0, 1] of continuous functions on the closed interval [0, 1], with the metric d1 . Define the sequence (fn ) by 8 if x 6 12 , < 0 n(x 12 ) if 12 6 x 6 12 + n1 , fn (x) = : 1 if 12 + n1 6 x. 1

⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ 1 2

1

Then (fn ) is a Cauchy sequence. To prove this, first note that ✓ ◆ 1 1 1 1 d1 (fn , fm ) = < 2 m n 2m for m < n. Then, given ✏ > 0, set N =

1 . 2✏

d1 (fn , fm )
N , then assume that n > m, so that 1 1 < = ✏, 2m 2N

and therefore (fn ) is Cauchy.

6.2

Completeness.

In this section we’ll single out and study the following important property of a metric space. Definition 6.7. The metric space X is complete if every Cauchy sequence (xn ) in X converges to a limit x 2 X. More generally, a subset A ✓ X is complete if every Cauchy sequence in A converges to a limit in A. Example 6.8. Q is not complete. In this subsection we are going to prove that R is complete, and see some general results relating complete subsets to closed subsets. Then in the next subsection we’ll see which of our other favourite metric spaces are also complete. These results are important to our study of whether iterative processes converge. To prove that R is complete is slightly tricky, and to prove it we’ll need this preliminary result: Theorem 6.9 (Bolzano-Weierstrass). Let (xn ) be a sequence of real numbers which is bounded, i.e. there are some ↵, such that ↵ 6 xn 6 for all n. Then the sequence (xn ) has a convergent subsequence.

MAS331

35

Autumn 2011-12

Proof. Let ↵1 = ↵, 1 = . Let = ↵+2 . As there are infinitely many xn in [↵, ], there are either infinitely many xn in [↵1 , ] or in [ , 1 ] (or both). If there are infinitely many in [↵1 , ], set ↵2 = ↵1 , 2 = ; otherwise set ↵2 = , 2 = 1 . Repeat, halving the length of the interval each time. Then ↵1 6 ↵2 6 ↵3 6 . . . 6

3

6

2

6

1.

Since (↵n ) is increasing, and bounded above by any n , it converges to a limit, and similarly ( n ) converges to a limit. These limits are the same, since n ↵n halves at each step. Write ` for this limit. Choose xn1 lying in [↵1 , 1 ]. Choose n2 > n1 such that xn2 2 [↵2 , 2 ]. Choose n3 > n2 such that xn3 2 [↵3 , 3 ]. Continue in this way to get a subsequence (xni ) that converges to `. Theorem 6.10. R is complete. Proof. Let x1 , x2 , . . . be a Cauchy sequence in R. We have to check the condition for convergence. This means we have to conjure up some limit. We’ll use the BolzanoWeierstrass Theorem 6.9. First note that Cauchy sequences are bounded. Indeed, by the definition, we can find N such that |xm xn | < 1 whenever m, n > N . Then |xm | 6 |xN | + |xm xN | 6 |xN | + 1 whenever m > N , so that |xm | 6 max{|x1 |, . . . , |xN

1 |, |xN |

+ 1},

so that (xm ) is indeed bounded. The Bolzano-Weierstrass Theorem tells us that there is a subsequence (xni ) tending to a limit `. Let’s prove that the whole sequence tends to `. Let ✏ > 0, and let N be large enough that |xm xn | < 2✏ for n, m > N , and let K be large enough that |xnk `| < 2✏ for k > K. Then, whenever n > max{N, nK }, we have |xn

`| 6 |xn

xnk | + |xnk

`|
N . Thus xn ! `. Proposition 6.11. Let X be a complete metric space, and let A be a closed subset of X. Then A is complete. Proof. Let a1 , a2 , . . . be a Cauchy sequence in A. Then it is also a Cauchy sequence in X, and therefore converges to some x 2 X. Since A is a closed subset of X, the limit x actually lies in A, so A is complete. Proposition 6.12. Let A be a complete subset of a metric space X. Then A is closed. Proof. Let (xn ) be a sequence in A that converges to a point x 2 X. We need to show that x is in A. By 6.4, (xn ) is a Cauchy sequence and so — since A is complete — converges to some a 2 A. Since limits are unique, a = x and x 2 A as required.

MAS331

36

Autumn 2011-12

6.3

Examples.

Theorem 6.13. Rk , with the Euclidean metric d2 , is complete. Proof. Start by noting the following simple property of the metric d2 on Rk . Let a = (a1 , . . . , ak ) and b = (b1 , . . . , bk ) be any elements of Rk . Then for 1 6 i 6 k we have p |ai bi | 6 (a1 b1 )2 + · · · + (ak bk )2 = d2 (a, b). Now let (xn ) be a Cauchy sequence in Rn . Write the individual terms of the sequence (1) (k) as xn = (xn , . . . , xn ). Let ✏ > 0, and let N be such that d2 (xn , xm ) < ✏ for all n, m > N . Then for each 1 6 i 6 k we have |x(i) n

x(i) m | 6 d2 (xn , xm ) < ✏ (i)

for all n, m > N . This means that the sequence (xn ) of ith coordinates is also Cauchy, and therefore it converges to some limit x(i) 2 R. This means that each coordinate of the sequence (xn ) converges, and so it follows from Proposition 3.8 that (xn ) converges to x = (x(1) , . . . , x(k) ). Thus Rk is complete. Example 6.14. C[a, b], with the metric d1 , is not complete. We’ll demonstrate this in the case of C[0, 1]. Remember from Example 6.6 that we had a Cauchy sequence of functions (fn ) defined by 8 if x 6 12 , < 0 n(x 12 ) if 12 6 x 6 12 + n1 , fn (x) = : 1 if 12 + n1 6 x. Suppose for a contradiction that this Cauchy sequence does converge, i.e. that fn ! f for some continuous function f . Then d1 (fn , f ) ! 0 as n ! 1. But d1 (fn , f ) =

Z1

1

|fn (t)

f (t)|dt >

0

Z2

1

|fn (t)

f (t)|dt =

0

Z2

|f (t)|dt > 0,

0

R1 so that we must have 02 |f (t)|dt = 0. Since f is continuous, this implies that f (t) = 0 for R1 0 6 t 6 12 . The same argument shows that d1 (fn , f ) > 1 |f (t) 1|dt > 0, and since f is 2

continuous, then f (t) = 1 for all 12 6 t 6 1. No such f can exist, since we have f ( 12 ) = 0 and f ( 12 ) = 1. Thus C[0, 1] is not complete with respect to the metric d1 . The last example showed that (C[a, b], d1 ) is not complete. However, we know that the two metrics on C[a, b] behave very di↵erently; indeed, they behave di↵erently enough that with one metric the space is not complete but with the other it is: Theorem 6.15. (C[a, b], d1 ) is complete. Proof. We’ll assume for simplicity that [a, b] = [0, 1]. Let f1 , f2 , . . . be a Cauchy sequence in C[0, 1]. We need to show that the sequence converges to some limit f , and we also have to see that f 2 C[0, 1], i.e., prove that it is also continuous. Let’s first try to find a limit function f . If x0 2 [0, 1], we need to say what f (x0 ) will be. A good guess will be to consider the values of the functions at x0 : MAS331

37

Autumn 2011-12

f1 (x0 ), f2 (x0 ), f3 (x0 ), . . ., and to define f (x0 ) as the limit of these values. But we don’t even know that this sequence converges for any value of x0 ! However, we can use the fact that f1 , f2 , . . . is a Cauchy sequence with respect to the metric d1 to see that f1 (x0 ), f2 (x0 ), . . . is a Cauchy sequence of real numbers: |fm (x0 )

fn (x0 )| 6 sup{|fm (t)

fn (t)| : t 2 [0, 1]} = d1 (fn , fm ).

We know that f1 , f2 , . . . is Cauchy, so that for ✏ > 0 we can find N such that d1 (fn , fm ) < ✏ for n, m > N . It then follows that |fm (x0 ) fn (x0 )| < ✏ for m, n > N , and therefore (fn (x0 )) is a Cauchy sequence in R. But R is complete, so (fn (x0 )) converges to a limit that we call f (x0 ). At this point, we have defined f (x0 ) for all x0 2 [0, 1], and so we have defined the function f on all of [0, 1]. But we still need to prove that f is continuous, and that fn ! f in (C[0, 1], d1 ). First, we claim that, given ✏ > 0, we can find N such that |fn (x) f (x)| < ✏ for all n > N and x 2 [0, 1]. To prove this, choose N large enough that d1 (fn , fm ) < 2✏ for all n, m > N . Then for n > N and x 2 [0, 1] we have |fn (x)

f (x)| 6 |fn (x) fm (x)| + |fm (x) ✏ < + |fm (x) f (x)| 2

f (x)|

and |fm (x) f (x)| ! 0 as m ! 1, so that by a familiar rule for limits and inequalities we have |fn (x) f (x)| 6 2✏ < ✏ as claimed. Now we can prove that f is continuous. Let x 2 [0, 1] and let ✏ > 0. By the claim above, we can choose n large enough that |fn (x) f (x)| < 3✏ for all x, and since fn is continuous at x we can find > 0 so that |fn (x) fn (y)| < 3✏ whenever |x y| < . Then for |x y| < we have |f (x)

f (y)| 6 |f (x)

fn (x)| + |fn (x)

fn (y)| + |fn (y)

f (y)|