Lecture 14: Basic Hilbert Space Theory

April 8, 2021 Last time, we introduced the concept of a pre-Hilbert space (a vector space that comes equipped with a Hermitian inner product). This inner product is positive definite, linear in the first argument, and becomes complex conjugated when we swap the two arguments, and we can use this quantity to define ||v || = hv , v i1/2 for any v in the pre-Hilbert space. And we want to show that this is actually a norm – towards that goal, recall that last time, we showed the Cauchy-Schwarz inequality |hu, v i| ≤ ||u|| ||v || for all u, v in the pre-Hilbert space. We’ll now put that to use: Theorem 143 If H is a pre-Hilbert space, then || · || is a norm on H. Proof. We need to prove the three properties of the norm. For positive definiteness, note that we do have ||v || ≥ 0 for all v , and ||v || = 0 ⇐⇒ hv , v i = 0 ⇐⇒ v = 0 because the Hermitian inner product is (defined to be) positive definite. Furthermore, for any v ∈ H, we have hλv , λv i = λλhv , v i =⇒ ||λv || = |λ| ||v || by taking square roots of both sides, which shows homogeneity. So we just need to show the triangle inequality: indeed, if we have u, v ∈ H, then ||u + v ||2 = hu + v , u + v i = ||u||2 + ||v ||2 + 2Re(hu, v i). Because Re(z) ≤ ||z|| and using the Cauchy-Schwarz inequality, this can be bounded by ≤ ||u||2 + ||v ||2 + 2|hu, v i| ≤ ||u||2 + ||v ||2 + 2||u|| ||v || = (||u|| + ||v ||)2 , and now taking square roots of both sides yields the desired inequality. The Cauchy-Schwarz inequality can also help us in other ways: Theorem 144 (Continuity of the inner product) If un → u and vn → v in a pre-Hilbert space equipped with the norm || · ||, then hun , vn i → hu, v i. Proof. Notice that if un → u and vn → v , that means that ||un − u|| → 0 and ||vn − v || → 0 as n → ∞. Therefore, we can bound |hun , vn i − hu, v i| = |hun , vn i − hu, vn i + hu, vn i − hu, v i| and factoring and using the triangle inequality for C gives us = |hun − u, vn i + hu, vn − v i| ≤ |hun − u, vn i| + |hu, vn − v i|. The Cauchy-Schwarz inequality then allows us to bound this by ≤ ||un − u|| · ||vn || + ||u|| · ||vn − v ||, and now because vn → v we know that ||vn || → ||v ||, and this convergent sequence of real numbers must be bounded. Thus, our new bound is ≤ ||un − u|| · sup ||vn || + ||u|| · ||vn − v ||, n and now because ||un − u||, ||vn − v || → 0, the linear combination of them given above also converges to 0, and we’re done. Thus hun , vn i indeed converges to hu, v i (by the squeeze theorem). Definition 145 A Hilbert space is a pre-Hilbert space that is complete with respect to the norm || · || = h·, ·i1/2 . Example 146 The space of n-tuples of complex numbers Cn with inner product hz, w i = Pn j=1 zj wj is a (finite-dimensional) Hilbert space. Example 147 The space `2 = {a : P n |an |2 < ∞} is a Hilbert space, where we define ha, bi = ∞ X ak bk . k=1 In this latter example, we can check that ha, ai1/2 coincides with the `2 norm ||a||2 . And it turns out that every separable Hilbert space (which are the ones that we’ll primarily care about) can be mapped in an isometric way to one of these two examples, so the examples above are basically the two main types of Hilbert spaces we’ll often be seeing! But here’s another one that we’ll see often: Example 148 Let E ⊂ R be measurable. Then L2 (E), the space of measurable functions f : E → C with Hilbert space with inner product R E |f |2 < ∞, is a Z hf , gi = f g. E We might notice that we focused on `2 and L2 , and that’s because the inner product only induces the `2 norm in the way that it’s written right now. But we might want to ask whether there’s an inner product that we could put on the other `p or Lp so that they are also Hilbert spaces (so that we get out the appropriate norm), and the answer turns out to be no. We’ll see that through the following result: Proposition 149 (Parallelogram law) Let H be a pre-Hilbert space. Then for any u, v ∈ H, we have
||u + v ||2 + ||u − v ||2 = 2 ||u||2 + ||v ||2 . In addition, if H is a normed vector space satisfying this equality, then H is a pre-Hilbert space. We can use this result (which can be verified by computation) to see that there are always u, v which make this inequality not satisfied if p 6= 2 for the `p and Lp spaces. And now that we have this inner product, we can start doing more work in the “linear algebra” flavor: Definition 150 Let H be a pre-Hilbert space. Two elements u, v ∈ H are orthogonal if hu, v i = 0 (also denoted u ⊥ v ), and a subset {eλ }λ∈Λ ⊂ H is orthonormal if ||eλ || = 1 for all λ ∈ Λ and for all λ1 6= λ2 , heλ1 , eλ2 i = 0. Remark 151. We may notice that the index set we’re using is some arbitrary set Λ instead of N: we’ll mainly be interested in the case where we have a finite or countably infinite orthonormal set, but the definition makes sense more generally. We’ll see some examples corresponding to each of the examples of Hilbert spaces Cn , `2 , L2 that we described above: Example 152 The set {(0, 1), (1, 0)} is an orthonormal set in C2 , and {(0, 0, 1), (0, 1, 0)} is an orthonormal set in C3 . Example 153 Let e n be the sequence which is 1 in the nth entry and 0 everywhere else, we find that {en }n≥1 is an orthonormal subset of `2 . Example 154 √1 e inx (as elements of L2 ([−π, π])) form an orthonormal subset of L2 ([−π, π])). (This Rπ is because the integral −π e imx e inx dx = −π e i(m−n)x dx is zero unless m = n; if we’re uncomfortable integrating The functions fn (x) = R π2π a complex exponential, we can break it up into its real and imaginary parts by Euler’s formula.) Notice that we haven’t talked about whether the spaces `2 and L2 are separable, but it was an exercise for us to show that the continuous functions are dense in Lp (for all p < ∞) and the Weierstrass approximation tells us that continuous functions on a closed and bounded interval can be uniformly approximated by a polynomial. So the polynomials are dense in Lp , and to get to a countable dense subset, we only consider the polynomials with rational coefficients, and there are indeed countably many of those. So for all Lp with p finite, the polynomials with coefficients of the form q1 + i q2 for rational q1 + q2 form a countable dense subset of Lp ([a, b]), and thus those Lp spaces are separable. And the set of sequences which terminate after some point form a dense subset in `p for any finite p as well, so we can get our countable dense subset of `p by looking at the set of sequences of rationals that terminate eventually! Theorem 155 (Bessel) Let {en } be a countable (finite or countably infinite) orthonormal subset of a pre-Hilbert space H. Then for all u ∈ H, we have X n |hu, en i|2 ≤ ||u||2 . Proof. First, we do the finite case. If {en }N n=1 is a finite collection of orthonormal vectors in H, we can verify that 2 N X hu, en ien * + X X = hu, en ien , hu, em iem , n n=1 m and we can pull out some numbers to write this as = X hu, en ihu, em ihen , em i, n,m By orthonormality, the inner product hen , em i is only nonzero when n = m (in which case it’s equal to 1), so that we P 2 end up with N n=1 hu, en i . We can also say by linearity that * N X u, hu, en i.en + = n=1 N X hu, en ihu, en i = N X |hu, en i|2 . n=1 n=1 From here, note that 2 N X hu, en ien 0≤ u− , n=1 where the term inside the parentheses can be thought of as the projection of u onto the orthogonal space to the ei s. We can then rewrite this by expanding in the same way we previously did for ||u + v ||2 , and we get 2 N X 2 0 ≤ ||u|| + hu, en ien * N X − 2 Re u, hu, en ien n=1 N X |hu, en i|2 − 2 n=1 . n=1 Both of the last two terms now just give us multiples of 0 ≤ ||u||2 + + PN N X n=1 |hu, en i|2 by our work above, and we end up with |hu, en i|2 = ||u||2 − n=1 N X |hu, en i|2 , n=1 and this is exactly what we want to show for the finite case. And the infinite case follows by taking N → ∞: more formally, if {en } is an orthonormal subset of H, then N X n=1 |hu, en i|2 ≤ ||u||2 =⇒ lim N→∞ N X |hu, en i|2 ≤ ||u||2 , n=1 and this proves the result that we want for all countable orthonormal subsets of H. Orthonormal sets are not the most useful thing on their own for studying a pre-Hilbert space H, since we might leave out some vectors in our span. That motivates this next definition: Definition 156 An orthonormal subset {eλ }λ∈Λ of a pre-Hilbert space H is maximal if the only vector u ∈ H satisfying hu, eλ i = 0 for all λ ∈ Λ is u = 0. Example 157 The n standard basis vectors in Cn form a maximal orthonormal subset. (A non-example would be any proper subset of that set.) Example 158 Our example {e n } of sequences from above is a maximal orthonormal subset of `2 . We’ll soon see that a countably infinite maximal orthonormal subset basically serves the same purpose as an orthonormal basis does in linear algebra, but not every element will be able to be written as a finite linear combination of the orthonormal subset elements (like was possible with a Hamel basis). Theorem 159 Every nontrivial pre-Hilbert space has a maximal orthonormal subset. We can prove this result by using Zorn’s lemma and thinking of subsets as being ordered by inclusion. But if that scares us (because of the use of the Axiom of Choice), we can do a slightly less strong proof by hand: Theorem 160 Every nontrivial separable pre-Hilbert space H has a countable maximal orthonormal subset. (Recall that a space is separable if it has a countable dense subset.) Proof. We’ll use the Gram-Schmidt process from linear algebra as follows. Because H is separable, we can let {vj }∞ j=1 be a countable dense subset of H such that ||v1 || = 6 0. We claim that for all n ∈ N, there exists a natural number m(n) and an orthonormal subset {e1 , · · · , em(n) } so that the span of this subset is the span of {v1 , · · · , vn }, and {e1 , · · · , em(n+1) } is the union of {e1 , · · · , em(n) } and either the empty set (if vn+1 is already in the span) or some vector em(n+1) (otherwise). In other words, we can come up with a finite orthonormal subset that has the same span as the first n vectors of our countable dense subsets, and we can keep constructing this iteratively by adding at most one element. We’ll prove this claim by induction. For the base case n = 1, we can take e1 = v1 ||v1 || , which indeed satisfies all of the properties we want. Now for the inductive step, suppose that our claim holds for n = k, and now we want to span v1 through vk+1 instead of just v1 through vk . If vk+1 is already in the span of {v1 , · · · , vk }, then the span of {e1 , · · · , em(k) } is the same as the span of {v1 , · · · , vk }, which is the same as the span of {v1 , · · · , vk+1 }. So in this case, we don’t need to add anything, and all of our conditions are still satisfied. Otherwise, vk+1 is not in the span of {v1 , · · · , vk }, and we’ll define m(k) wk+1 − X hvk+1 , ej iej j=1 to be vk+1 with components along the other vi s subtracted off. This vector is not zero, or else vk+1 would be in the span of the existing ei s and thus in the span of the existing vi s. We then define the normalized version em(k+1) = wk+1 ||wk+1 || to add to our orthonormal subset: this is a unit vector by construction, and for any 1 ≤ ` ≤ k we can indeed check orthogonality: hem(k+1) , e` i = 1 ||mk+1 || * vk+1 − m(k) X + hvk+1 , ej iej , e` j=1 now simplifies because the first m(k) e’s are already orthonormal: we just pick out j = ` from the sum and we have = 1 (hvk+1 , v` i − hvk+1 , e` i) = 0. ||mk+1 || Therefore, {e1 , · · · , em(k) , em(k+1) } is indeed an orthonormal subset, and that proves the desired claim. It now remains to show that the collection of all e` s forms a maximal orthonormal subset. We define the set ∞ [ S= {e1 , · · · , em(n) }; n=1 this is an orthonormal subset of H which can be finite or countably infinite, and we want to show that S is maximal. And here is where we use the fact that the vi s are dense in H: suppose that we have some u ∈ H so that hu, e` i = 0 for all `. Then we can find a sequence of elements {vj(k) }k such that lim vj(k) → u. k→∞ Because the span of the vi s and the ei s are the same, we know that each vj(k) is in the span of {e1 , · · · , em(j(k)) }, so now m(j(k)) 2 ||vj(k) || = X |hvj(k) , e` i|2 `=1 (we have equality for a finite set of such orthonormal elements), and now we can rewrite this as m(j(k)) = X |hvj(k) − u, e` i|2 ≤ ||vj(k) − u||2 `=1 by Bessel’s inequality. But because vj(k) → u by construction, this means that ||vj(k) || → 0, and thus the limit of the vj(k) s, which is u, must be zero. That proves that our orthonormal basis is indeed maximal. Next time, we’ll understand more specifically what it means for these maximal orthonormal subsets to serve as replacements for bases from linear algebra!