Lecture 19: Compact Subsets of a Hilbert Space and Finite-Rank Operators

April 29, 2021 We’ll continue discussing compactness today – recall that a subset K of a metric space X is compact if every sequence {xn }n in K has a subsequence that is convergent in K. While being closed and bounded is equivalent to being compact in Rn , this is not true in general Hilbert spaces (for example, take the orthonormal basis vectors in `2 ). So we need an additional condition – last time, we proved that if we have a convergent sequence {vn }n in H converging to v , then the subset K = {vn : n ∈ N} ∪ {v } is compact, and it has equi-small tails with respect to any orthonormal subset. Here, the definition is that if {ek }k is a countable orthonormal subset of H, then for all ε > 0, there exists some N ∈ N such that for all ṽ ∈ K (either an element of the sequence or v ), we have X |hṽ , ek i|2 < ε2 . k>N (We know that this sum over all k is bounded, and thus convergent, by Bessel’s inequality for any individual ṽ , so we can always find an N that makes this work for a fixed ṽ , but the condition requires it simultaneously for all ṽ ∈ K. So we can think of “equi-small tails” as really meaning “uniformly small tails.”) It turns out that this condition suffices (and is necessary) for compactness: Theorem 197 Let H be a separable Hilbert space, and let {ek }k be an orthonormal basis of H. Then a subset K ⊂ H is compact if and only if K is closed, bounded, and has equi-small tails with respect to {ek }. Proof. For the forward direction, first suppose that K is compact. We know by general metric space theory that K is then closed and bounded, and we’ll show that K has equi-small tails with respect to {ek } by contradiction. Suppose otherwise: then there exists some ε0 such that for each natural N, there is some uN ∈ K such that X |huN , ek i|2 ≥ ε20 . k>N This then gives us a sequence {un } (by picking such a uN for every natural number N), and thus by the assumption of compactness, there is some subsequence {vm } = {unm } and some v ∈ K such that vm → v . But we also know that P for all n ∈ N, k>n |hvn , ek i| ≥ ε20 , because vm = unm is the nth or later term of the original sequence (so summing over k > nm is at most the value we get summing over k > n). That means that the subset {vn : n ∈ N} ∪ {v } does not have equi-small tails, which is a contradiction of our previous theorem. So if K is compact, then it must have equi-small tails, as desired. On the other hand, suppose K is closed, bounded, and has equi-small tails. We wish to show that any sequence {un } has a convergent subsequence in K. Because K is closed, any sequence that converges will converge in K, so we just need to show that there is some convergent subsequence. We know that any bounded sequence of complex numbers has a convergent subsequence (showing convergence of the real and imaginary parts by Bolzano-Weierstrass), so the plan is to expand {un } in terms of the orthonormal basis of H and think about the coefficients along each basis vector. Since K is bounded, there is some C ≥ 0 (only depending on K) so that for all n, ||un || ≤ C. Therefore, for all k and for all n, the “Fourier coefficient” |hun , ek i| ≤ ||un || · ||ek || ≤ C, and thus for each fixed k, we get a bounded sequence of coefficients along the kth basis vector: specifically, we have the bounded sequence of numbers {hun , ek i}n in C. Thus, by Bolzano-Weierstrass (fixing k = 1), there is some subsequence {hun1 (j) , e1 i of {hun , e1 i}n which converges in C (in other words, we have a subset of the original {un }s in which the first entry converges). And now {hun1 (j) , e2 i is still a bounded sequence, and thus by Bolzano-Weierstrass again we have a further subsequence {hun2 (j) , e2 i which converges. So we now have a subset of the original {un }s in which the first and second entries both converge (since the first subsequence converges in the first entry, and thus any subsequence of it will also converge in the first entry). We can repeat this argument arbitrarily many times: further subsequences of the un2 (j) s gives us a subsequence un` (j) such that {hun` (j) , e` i converges, meaning that we have convergence along our sequence in the first ` entries. If we now define v` = un` (`) ∀` ∈ N, then the {v` }` form a subsequence of the {un }n s with convergence in the kth entry (for any fixed k) as ` → ∞. This on its own doesn’t mean that the sequence converges, but here is where we will use the fact that K has equi-small tails. It suffices to show that {v` }` is Cauchy (because H is complete and K is closed). For any ε > 0, having equi-small tails tells us that there is some N such that X ε2 16 |hv` , ek i|2 < k>n for all ` ∈ N. Now because the N sequences {hv` , e1 i}` through {hv` , eN i}` each converge, we can then find an M such that for all `, m ≥ M, N X |hv` , ek i − hvm , ek i|2 < k=1 ε2 . 4 We claim that this M is the one that we want for our sequence {v` }: indeed, ||v` − vm || = N X #1/2 " X |hv` − vm , ek i|2 + k=1 |hv` − vm , ek i|2 k>N (because we have an orthonormal basis, the norm squared is the sum of the Fourier coefficients). Now using the fact √ √ √ that a + b ≤ a + b, we can bound this as " #1/2 N X X   1/2 ≤ |hv` − vm , ek i|2 + |hv` − vm , ek i|2 . k=1 k>N By our choice of M, the first term is at most 2ε , and we can use the `2 triangle inequality for the second term, thinking of that second term as the difference of the sequences {hv` , ek i}k and {hvm , ek i}k . Thus we have the bound " #1/2 " #1/2 X X ε ε ε ε 2 2 < + |hv` , ek i| + |hvm , ek i| < + + = ε, 2 2 4 4 k>N k>N where the last inequality comes from how we chose N. So our subsequence is Cauchy, thus convergent, and thus K is compact. Example 198 Let K be the set (not subspace) of sequences {ak }k in `2 satisfying |ak | ≤ 2−k – this set is known as the Hilbert cube, and it is compact. It may seem unwieldy that we make this definition with respect to an orthonormal basis, but we can characterize compact sets in another way as well: Theorem 199 A subset K ⊂ H is compact if and only if K is closed, bounded, and for all ε > 0, there exists a finite-dimensional subspace W ⊂ H so that for all u ∈ K, inf w ∈W ||u − w || < ε. In other words, our additional condition is that we can approximate the points in K by a finite-dimensional subspace. This proof also involves a similar “diagonal argument,” and notably it works for non-separable Hilbert spaces as well, but we can read about the proof on our own. This should be a believable result, because the equi-small tail condition we worked with in our previous proof was basically saying that we can approximate points in K by the first N vectors in our orthonormal basis (since the contribution from the other basis vectors is small). We’ll now start to talk about various classes of operators, and we’ll start with the simplest ones. From linear algebra, we know that matrices are operators in finite-dimensional vector spaces, and we can represent them with an array of numbers. We can now generalize that definition to our current setting. Fact 200 From here on, H will be a Hilbert space, and we’ll denote B(H, H) by B(H). Definition 201 A bounded linear operator T ∈ B(H) is a finite rank operator if the range of T (a subspace of H) is finitedimensional. We denote this as T ∈ R(H). Example 202 If H is a finite-dimensional Hilbert space, then every linear operator is of finite rank. For a more interesting example, for any positive integer n, the operator Ta = na o a2 an , ·, , 0, · · · 1 2 n 1 , is a finite rank operator (because the image is spanned by the first n standard basis vectors). Proposition 203 The set R(H) is a subspace of B(H). Proof. The range of a scalar multiple of an operator is the same as the original range, and the sum of two finite rank operators has range contained in the direct sum of the individual ranges (which is also finite-dimensional). We’ll now prove that these finite rank operators are really like matrices: Theorem 204 An operator T ∈ B(H) is in R(H) if and only if there exists an orthonormal set {ek }Lk=1 and an array of constants {cij }Li,j=1 ⊂ C, such that Tu = L X i,j=1 cij hu, ej iei . Proof. The backwards direction is clear: if T has such a representation, then the range of T is contained in the span of the L vectors {e1 , · · · , eL } and is thus finite-dimensional. Now suppose that T is a finite rank operator. Then we can find an orthonormal basis {e k }N k=1 of the range of T , such that Tu = N X hT u, e k ie k k=1 (since T u is in the range, it must be this particular combination of the orthonormal basis vectors). Now by the definition of the adjoint operator, we can rewrite this sum as n N X X ∗ = hu, T e k ie k = hu, vk ie k , k=1 k=1 where we’ve define vk = T ∗ e k . If we now apply the Gram-Schmidt process to the vectors {e 1 , · · · , e N , v1 , · · · , vN }, we get an orthonormal subset {e1 , · · · , eL } with the same span as our original e i s and vi s. Thus, there exist constants aki , bkj so that (expanding in terms of the new orthonormal subset) ek = L X aki ei , L X vk = i=1 bkj ej . j=1 Thus, substituting back in, Tu = L X N X i,j=1 k=1 ! hu, ej iei , aki bkj and now the term in the inner parentheses is our desired cij . And with this characterization, we can now describe our finite rank linear operators more explicitly: for example, the nullspace of T contains the set of vectors orthogonal to all of the ek s. Theorem 205 If T ∈ R(H), then T ∗ ∈ R(H), and for any A, B ∈ B(H), AT B ∈ R(H). In other words, R(H) is a “star-closed, two-sided ideal in the space of bounded linear operators” – it’s closed under two-sided multiplication and adjoints. Proof. We’ll leave the closure under multiplication as an exercise: the main point is that if T has a finite-dimensional range, the range of AT is also finite-dimensional, and whatever happens with B doesn’t really matter. For closure under adjoints, if T is a finite rank operator, then we can write Tu = L X cij hu, ej iei , i,j=1 and thus * ∗ hu, T v i = hT u, v i = L X + cij hu, ej iei , v i,j=1 By linearity in the first entry, we can rewrite this inner product as = X i,j cij hu, ej ihei , v i, . and we can now use linearity to pull things into the second component instead: * + * + X X = u, cij hei , v iej = u, cij hv , ei iej . i,j i,j But since this is true for all u ∈ H, we’ve shown that * ∗ u, T v − + X cij hv , ei iej =0 i,j for all u, v ∈ H, and thus we must have T ∗ v = PL i,j=1 c ij hv , ei iej for all v ∈ H, and thus T ∗ ∈ R(H) as well: in fact, we can recover the coefficients in terms of the coefficients for T by reindexing as = L X cji hv , ej iei . i,j=1 Thus, the coefficients of the matrix governing T ∗ are obtained by taking the conjugate transpose of the ones for T. Since the set of finite rank linear operators is a subspace of the Banach space of bounded linear operators, which come with a norm, it makes sense to ask if the subspace of finite rank operators is closed (under the norm). In other words, if Tn ∈ R(H), and ||Tn − T || → 0 as n → ∞, we want to know whether T ∈ R(H). It turns out the answer is no: Example 206 Let Tn : `2 → `2 be a sequence of operators defined as na o an 1 Tn a = , · · · , , 0, · · · . 1 n We can imagine that the limit T of these operators is the infinite-dimensional “diagonal matrix” with entries (1, 12 , 13 , · · · ): specifically, defining a1 a2 a3 , , , · · · }, 1 2 3 1 , but T is not of finite rank (since T (kek ) = ek is in the range for each standard we can check that ||T − Tn || ≤ n+1 Ta = { basis vector ek ). In other words, the space of finite rank linear operators (which are nice because we can solve linear equations involving them using matrices) is not closed. But we still want to know about the closure of R(H), and the hope is that we still have a useful characterization: Definition 207 An operator K ∈ B(H) is a compact operator if K({u ∈ H : ||u|| ≤ 1}), the closure of the image of the unit ball under K, is compact. We’ll show next time that K is a compact operator if and only if it is in the closure of R(H), meaning that there is a sequence of finite rank operators converging to K. These compact operators will come up in useful problems – for example, T in our example above is compact, and the inverse of many differential operators will turn out to be compact as well. And as a sanity check before we do the proof next time, finite rank operators are indeed compact operators, because the image of the unit ball will be a bounded subset of a finite-dimensional subspace, and thus the closure of that image is a closed and bounded subset of a finite-dimensional subspace, which is compact by Heine-Borel.