F = R or C. Definition 6.4.1. Given a subspace M â V , fix an orthonormal basis {v1, . . . , vm} for M. We define the
SWILA NOTES
67
Moreover, for all x ∈ V , ||x||2 = ||hx, v1 iv1 + · · · + hx, vn ivn ||2
(by Lemma 6.3.7)
= ||hx, v1 iv1 ||2 + · · · + ||hx, vn ivn ||2
(by the Pythagorean Theorem)
= |hx, v1 i|2 + · · · + |hx, vn i|2 = ||T x||2 . By the previous lemma, it follows that V is isometrically isomorphic to Fn .
6.4. Orthogonal complements and projections. Recall in Section 3.2 we defined complements of subspaces and projections. Given a subspace M ⊆ V , a complement of W is a subspace N ⊆ V such that V = M ⊕ N . A linear operator E : V → V is called a projection if E 2 = E. In this section, we will take a more refined look at them in the context of inner product spaces. This perspective wasn’t available to us before because we didn’t have the power of an inner product on our vector spaces to be able to talk about orthogonality. Throughout this section, we will assume V is a finite-dimensional inner product space and F = R or C. Definition 6.4.1. Given a subspace M ⊆ V , fix an orthonormal basis {v1 , . . . , vm } for M . We define the orthogonal projection onto M to be the linear operator projM : V → V defined by projM (x) = hx, v1 iv1 + · · · + hx, vm ivm . Remark 6.4.2. It’s a fact that the projection onto M is well-defined, i.e., independent of the orthonormal basis chosen for M . Moreover, one can somewhat easily show that proj2M = projM , i.e., projM is a projection. Definition 6.4.3. Fix a subspace M ⊆ V . We define the orthogonal complement of M to be M ⊥ := {x ∈ V : hx, yi = 0 for all y ∈ M }. Proposition 6.4.4. Fix a subspace M ⊆ V . Then ker(projM ) = M ⊥ and V = M ⊕ M ⊥ . In particular, (M ⊥ )⊥ = M and dim(V ) = dim(M ) + dim(M ⊥ ). Proof. Fix an orthonormal basis {v1 , . . . , vm } for M . For x ∈ M , observe projM (x) = 0
⇐⇒
hx, vj i = 0,
for all j = 1, . . . , m.
(This is due to the linear independence of {v1 , . . . , vm }.) Note that these are further equivalent to hx, yi = 0 for all y ∈ M, using the linearity of the inner product and that {v1 , . . . , vm } is a basis for M . Thus, we have x ∈ ker(projM ) ⇐⇒ x ∈ M ⊥, which proves ker(projM ) = M ⊥ . It isn’t too hard to see Im(projM ) = M . Then Proposition 3.2.8 implies V = M ⊕ M ⊥ . For any x ∈ M , note x ∈ (M ⊥ )⊥ . This implies M ⊆ (M ⊥ )⊥ . On the other hand, since M ⊥ ⊕ (M ⊥ )⊥ = M ⊕ M ⊥ , dim((M ⊥ )⊥ ) = dim(V ) − dim(M ⊥ ) = dim(M ).
68
BY DEREK JUNG, ADAPTED FROM NOTES BY UCLA PROFESSOR PETER PETERSEN
It follows that M = (M ⊥ )⊥ .
Random Thought 6.4.5. Have you heard the one about the towel and the desert? Actually nevermind... It’s pretty dry humor. Proposition 6.4.6. For any x ∈ V and a subspace M ⊆ V , projM (x) is the closest element to x in M in the following sense: For any z ∈ M , ||x − projM (x)|| ≤ ||x − z||. Proof. Fix x ∈ V . Observe x − projM (x) ∈ ker projM = M ⊥ and projM (x) − z ∈ M . Hence, by the Pythagorean Theorem, ||x − z||2 = ||x − projM (x)||2 + ||projM (x) − z||2 ≥ ||x − projM (x)||2 . Theorem 6.4.7. (Bessel’s inequality) Fix a subspace M ⊆ V . If {e1 , . . . , em } is an orthonormal basis for M , then for all x ∈ V , m X
|hx, ej i|2 ≤ ||x||2
j=1
with equality if and only if x ∈ M . Proof. Write x = xM + xM ⊥ for xM ∈ M and xM ⊥ ∈ M ⊥ . As xM = hx, e1 ie1 + · · · + hx, em iem , 2
2
2
2
||x|| = ||xM || + ||xM ⊥ || ≥ ||xM || =
m X
|hx, ej i|2 .
j=1
Note we have equality if and only if xM ⊥ = 0, which is equivalent to x ∈ M .
Remark 6.4.8. This section would take a fair bit longer to write if we allowed for V to be infinite-dimensional. We would have to assume that the subspace M is closed, and then prove that minimizers of distance exist. Then we would be able to prove the existence of the orthogonal complement. Also, a more general statement of Bessel’s inequality holds for infinite-dimensional product spaces and is a vital tool for studying Hilbert spaces. See Terence Tao’s An Epsilon of Room, I: Real Analysis [7] for a more thorough treatment. 6.5. The adjoint of a linear transformation. In this section, we are interested in defining the adjoint of a linear transformation. As always, V and W will always be inner product spaces with the same field of scalars (either F = R or C). It will typically be enough to do the theory for when F = C, so we will often assume F = C. This section will be vital in setting up the next chapter. For motivation, we begin by defining the adjoint of a matrix. Definition 6.5.1. Let A ∈ Ml×m (F). In the case F = R, we define the adjoint of A to be the transpose At ∈ Mm×l (R). In the case F = C, we define the adjoint of A to be A∗ := At ∈ Mm×l (C). Here, if A = (aij ), At = (aji ).
SWILA NOTES
69
Example 6.5.2. If A= then
1 0 −i 2 + 3i 2 4i
∈ M2×3 (C),
1 2 − 3i 2 ∈ M3×2 (C). A∗ = 0 i −4i
Remark 6.5.3. Recall the inner product on Fn defined in Examples 6.1.3 or Example 6.1.4. We may naturally identify a vector x ∈ Fn with an (n × 1)-matrix whose elements are the coordinates of x. For any A ∈ Ml×m (F), x ∈ Fm , and y ∈ Fl , hAx, yi = (Ax)t y¯ = xt At y¯ = x(A¯t y) = hx, A∗ yi. As linear transformations between finite-dimensional vector can be identified with matrices (see Proposition 2.2.3), we should be able to define the adjoint A∗ : W → V of a linear transformation A : V → W . The map should satisfy hAx, yiW = hx, A∗ yiV
for all x ∈ V, y ∈ W.
We now prove the existence (and uniqueness) of the adjoint map. Theorem 6.5.4. Given a linear map T : V → W , there exists a unique linear map T ∗ : W → V satisfying hT x, yiW = hx, T ∗ yiV
for all x ∈ V, y ∈ W.
We call T ∗ : W → V the adjoint of T . Proof. Fix an orthonormal basis e1 , . . . , en for V . Fix y ∈ W . By Corollary 6.3.8, we should have T ∗ y = hT ∗ y, e1 iV e1 + hT ∗ y, e2 iV e2 + · · · + hT ∗ y, en iV en Thus, we define T ∗ y = hy, T e1 iW e1 + · · · + hy, T en iW en . It is easy to see T ∗ is linear as the inner product is linear in the first coordinate. Using the orthonormality of {e1 , . . . , en }, hT ej , yiW = hej , T ∗ yiV
for all j = 1, . . . , n and y ∈ W.
By linearity of T and of the inner product, we obtain hT x, yiW = hx, T ∗ yiV
for all x ∈ V, y ∈ W.
It is an easy exercise to show that T ∗ is linear using the linearity of T and that of the inner product. For uniqueness, suppose there is another linear transformation S : W → V satisfying hT x, yiW = hx, SyiV
for all x ∈ V, y ∈ W.
By taking a difference, it follows hx, T ∗ y − Syi
for all x ∈ V, y ∈ W.
Letting x = T ∗ y − Sy for y ∈ W , we have T ∗ y = Sy by non-degeneracy. We may conclude the adjoint exists and is unique.
70
BY DEREK JUNG, ADAPTED FROM NOTES BY UCLA PROFESSOR PETER PETERSEN
Remark 6.5.5. Recall that every linear transformation T : Fm → Fl is of the form T (x) = Ax for some A ∈ Ml×m (F) (see Proposition 2.2.3). By the discussion at the beginning of this section, the adjoint map T ∗ : Fl → Fm is given by T ∗ (y) = A∗ y, where A∗ is the conjugate transpose matrix of A. Random Thought 6.5.6. You have probably noticed that nearly all comedians have really large knees. Now that’s because all of them eat a lot of pho noodles, which causes them to get what is commonly referred to as the “pho knee”. This also explains why most comedians are either funny or phony; it’s due to confusion in pronunciation. The adjoint satisfies a few other important properties (which we leave as an exercise). One uses strongly the uniqueness of the adjoint to prove these properties. Proposition 6.5.7. Let S, T : V → W and T1 : V1 → V2 , T2 : V2 → V3 be linear transformations. Then (1) (S + T )∗ = S ∗ + T ∗ . (2) T ∗∗ = T . ¯ V. (3) (λ1V )∗ = λ1 ∗ (4) (T2 T1 ) = T1∗ T2∗ . (5) If T is invertible, then (T −1 )∗ = (T ∗ )−1 . Theorem 6.5.8. (The Fredholm Alternative) Let T : V → W be a linear transformation. Then (1) ker(T ) = im(T ∗ )⊥ . (2) ker(T ∗ ) = im(T )⊥ . (3) ker(T )⊥ = im(T ∗ ). (4) ker(T ∗ )⊥ = im(T ). Proof. As T ∗∗ = T and M ⊥⊥ = M , all four statements are equivalent. Thus, it suffices to prove the first statement. If x ∈ ker(T ), hx, T ∗ yi = hT (x), yi = 0 for all y ∈ W. If x ∈ im(T ∗ )⊥ , 0 = hx, T ∗ yi = hT (x), yi for all y ∈ W. Letting y = T x, it follows from non-degeneracy that x ∈ ker(T ). This proves ker T = im(T ∗ )⊥ . Corollary 6.5.9. Let T : V → W be a linear map between finite-dimensional inner product spaces. If T is surjective, then T ∗ is injective. If T is injective, then T ∗ is surjective. In particular, if T is an isomorphism, then T ∗ is an isomorphism. Proof. This follows from the Fredholm Alternative and Proposition 6.4.4.
Corollary 6.5.10. Let T : V → W be a linear map between finite-dimensional inner product spaces. Then rank(T ) = rank(T ∗ ). Proof. By the rank-nullity theorem and the Fredholm Alternative, dim(V ) = dim(ker(T )) + dim(im(T )) = dim(im(T ∗ )⊥ ) + dim(im(T )) = dim(V ) − dim(im(T ∗ )) + dim(im(T )).
SWILA NOTES
The exercise follows by the rank-nullity theorem.
71
We conclude this chapter with an interesting corollary. Corollary 6.5.11. Let T : V → V be a linear operator. Then λ is an eigenvalue for T if and ¯ is an eigenvalue for T ∗ . Moreover, their eigenvalue pairs have the same geometric only if λ multiplicity: ¯ V )). dim(ker(T − λ1V )) = dim(ker(T ∗ − λ1 Proof. Note that λ is an eigenvalue of T if and only if dim(ker(T − λ1V )) > 0. By the previous corollary and the Rank-Nullity Theorem, ¯ V )) dim(ker(T − λ1V )) = dim(ker(T ∗ − λ1 for all λ ∈ F. The result follows.
