Lecture 15: Orthonormal Bases and Fourier Series

April 13, 2021 We’ll discuss orthonormal bases of a Hilbert space today. Last time, we defined an orthonormal set {eλ }λ∈Λ of elements to be maximal if whenever hu, eλ i = 0 for all λ, we have u = 0. We proved that if we have a separable Hilbert space, then it has a countable maximal orthonormal subset (and we showed this using the Gram-Schmidt process and Bessel’s inequality). Such subsets are important in our study here: Definition 161 Let H be a Hilbert space. An orthonormal basis of H is a countable maximal orthonormal subset {en } of H. Many of the examples we’ve encountered so far, like Cn , `2 , and L2 , are indeed countable and thus have an orthonormal basis. And the reason that we call such sets bases, like in linear algebra, is that we can draw an analogy between the two definitions: Theorem 162 Let {en } be an orthonormal basis in a Hilbert space H. Then for all u ∈ H, we have convergence of the FourierBessel series lim m→∞ m ∞ X X hu, en ien = hu, en ien = u. n=1 n=1 So just like in finite-dimensional linear algebra, we can write any element as a linear combination of the basis elements, but we may need an infinite number of elements to do so here. Pm Proof. First, we will show that the sequence of partial sums n=1 hu, en ien is a Cauchy sequence. Since we know P 2 2 that n=1 |hu, en i| converges by Bessel’s inequality (it’s bounded by ||u|| ), the partial sums must be a Cauchy sequence of nonnegative real numbers. Thus for any ε > 0, there exists some M such that for all N ≥ M, ∞ X |hu, en i|2 < ε2 . m=N+1 Thus, for any m > ` ≥ M, we can compute m ` X X hu, en ien − hu, en ien n=1 n=1 2 = m X |hu, en i|2 n=`+1 by expanding out the square ||v ||2 = hv , v i and using orthonormality, and now this is bounded by ≤ ∞ X |hu, en i|2 < ε2 . n=`+1 So for any ε, the squared norm of the difference between partial sums goes to 0 as we go far enough into the sequence, which proves that we do have a Cauchy sequence in our Hilbert space. Since H is complete, there then exists some u 0 ∈ H so that m X u = lim hu, en ien . 0 m→∞ n=1 0 We want to show that u = u, and we will do this by showing that hu 0 − u, en i = 0 for all n. By continuity of the inner 77 product, we know that for all ` ∈ N, we have * m X u− hu, en ien , e` 0 hu − u , e` i = lim n→∞ + , n=1 and this simplifies by linearity to = lim hu, e` i − n→∞ m X hu, en ihen , e` i, n=1 but by orthonormality the last term only exists for n = `, so this simplifies to hu, e` i − hu, e` · 1i = 0, which proves the result because hu − u 0 , e` i = 0 for all ` if and only if u − u 0 = 0 by maximality. So if we have an orthonormal basis, every element can be expanded in this series in terms of the orthonormal basis elements. And thus every separable Hilbert space H has an orthonormal basis, and the converse is also true: Corollary 163 If a Hilbert space H has an orthonormal basis, then H is separable. Proof. Suppose that {en }n is an orthonormal basis for H. Define the set ( m ) [ X S= qn en : q1 , · · · , qm ∈ Q + i Q . m∈N n=1 This is a countable subset of H, because the elements in each component indexed by m are in bijection with Q2m , which is countable, and then we take a countable union over m. So now by Theorem 162, S is dense in H, because every element u can be expanded in the Fourier-Bessel series above, so the partial sums converge to u, and thus for any ε > 0, we can take a sufficiently long partial sum of length L and get within ε 2 of u, and then approximate each coefficient with a rational number that is sufficiently close, and that eventual finite-length partial sum will indeed be in one of the parts of the S we defined. So S is indeed a countable dense subset of H, and we’re done. We can now strengthen Bessel’s inequality, which held for any orthonormal subset, with our new definition: Theorem 164 (Parseval’s identity) Let H be a Hilbert space, and let {en } be a countable orthonormal basis of H. Then for all u ∈ H, X |hu, en i|2 = ||u||2 . n (In Bessel’s inequality, we only had an inequality ≤ in the expression above!) Proof. We know that u= X hu, en ien , n so if the sum over n is a finite sum, the result follows immediately by expanding out the inner product ||u||2 = hu, ui. Otherwise, by continuity of the inner product, we can write * m + m X X 2 ||u|| = lim hu, en ien , hu, e` ie` , m→∞ n=1 `=1 78 and we can move the constants out (with a complex conjugate for one of them) and rearrange sums to get = lim m→∞ m X hu, en ihu, e` ihen , e` i. n,`=1 Again, orthonormality only picks up the term where n = `, so we’re left with = lim m→∞ m m X X hu, en ihu, en i = lim |hu, en i|2 , m→∞ n=1 n=1 and this last expression is the left-hand side of Parseval’s identity. We now actually have a way to identify every separable Hilbert space with the one that was introduced to us at the beginning of class: Theorem 165 If H is an infinite-dimensional separable Hilbert space, then H is isometrically isomorphic to `2 . In other words, there exists a bijective (bounded) linear operator T : H → `2 so that for all u, v ∈ H, ||T u||`2 = ||u||H and hT u, T v i`2 = hu, v iH . (The finite-dimensional case is easier to deal with – we can show that those Hilbert spaces are isometrically isomorphic to Cn for some n.) Proof sketch. Since H is a separable Hilbert space, it has an orthonormal basis {en }n∈N , and by Theorem 162, we must have u= ∞ X hu, en ien n=1 for all u ∈ H, which implies that ||u|| = ∞ X !1/2 2 |hu, en i| . n=1 So we’ll define our map T via T u = {hu, en i}n : in other words, T u is the sequence of coefficients showing up in the expansion by orthonormal basis, and this sequence is in `2 by the inequality we wrote down above. We can check that T indeed satisfies all of the necessary conditions P – it’s linear in u, it’s surjective because every such sum ∞ n=1 cn en is Cauchy in H, and it’s one-to-one because every u is expanded in this way, meaning that if two expansions are the same the evaluations of the infinite sums must also be the same. We can now use this theory that we’ve been discussing in a more concrete setting, focusing on the specific example of Fourier series. Proposition 166  The subset of functions e inx √ 2π  is an orthonormal subset of L2 ([−π, π]). n∈Z (If we’re uncomfortable working with complex exponentials, we can define e ix = cos x + i sin x and work out all of the necessary properties – everything that we expect for exponentials remains true.) 79 Proof. Notice that he inx , e imx i = Z π e inx e imx dx = −π Z π e i(n−m)x dx −π is equal to 2π when n = m (since the integrand is 1) and otherwise it is π 1 i(n−m)x i(n−m) e −π = 0 because the exponential is always 2π-periodic x. So normalizing by 2π indeed gives us the desired   inx   1 m=n e e imx √ ,√ =  2π 2π 0 m 6= n. Definition 167 For a function f ∈ L2 ([−π, π]), the Fourier coefficient fˆ(n) of f is given by Z π 1 fˆ(n) = f (t)e −int dt, 2π −π and the Nth partial Fourier sum is SN f (x) = X fˆ(n)e inx = |n|≤N X  e inx  e inx √ . f, √ 2π 2π |n|≤N We can then look at the limit of the partial sums, but we’re not going to make any claims about convergence here yet: Definition 168 The Fourier series of f is the formal series P n∈Z fˆ(n)e inx . The motivating question for Fourier when first studying these objects was whether or not all continuous functions could be expanded in this Fourier series manner. Trying to study things on a pointwise convergence level is difficult, but the space we should really be viewing this setup within is the L2 space, and there we’ll be able to get some results. The problem we’re trying to resolve is as follows: Problem 169 Does the convergence (in L2 norm) P∞ n=1 fˆe inx → f hold for all f ∈ L2 ([−π, π])? In other words, does Z π |f (x) − SN f (x)|2 dx ||f − SN f ||2 = 1/2 −π converge to 0 as N → ∞? We’ll rephrase this equivalent as follows: we want to know whether n e inx √ 2π o is a maximal subset in L2 ([−π, π]), which is equivalent to showing that fˆ(n) = 0 ∀n ∈ Z =⇒ f = 0. We already know that if we have an orthonormal basis, then we can indeed make this infinite expansion for any element of the space L2 . So this rephrasing in terms of the language of Hilbert spaces will help us out here (and we should remember that we require the completeness of L2 to get to this rephrased problem statement). The answer to our problem turns out to be yes, but it’ll take us a bit of work to get there. 80 Proposition 170 Rπ For all f ∈ L2 ([−π, π]) and all N ∈ Z≥0 , we have SN f (x) = DN (x) = −π   2N+1 2π 1  sin((N+ 2x)x ) 2π sin 2 DN (x − t)f (t)dt, where x =0 . x 6= 0 We can check that the function DN is continuous (and in fact smooth), and it is called the Dirichlet kernel. The proof of this will be basically a warm-up calculation in preparation for some other calculations to come: Proof. For any f ∈ L2 ([−π, π]), we know that    Z π X X  1 Z π 1 SN f (x) = f (t)e −int dt e inx = e in(x−t)  dt. f (t)  2π −π 2π −π |n|≤N |n|≤N by linearity of the Lebesgue integral (even though we’re using the Riemann notation, integrals are always Lebesgue here). The term in parentheses is the function DN (x − t), where DN (x) = 2N 1 −iNx X inx 1 X inx e = e e . 2π 2π n=0 |n|≤N This is a geometric series with ratio e ix , so this evaluates to = 1 −iNx 1 − e i(2N+1)x e 2π 1 − e ix whenever e ix 6= 1. That happens whenever x 6= 0 (this is the only value within the range (−2π, π) that it needs to be defined on), and when x = 0 the original geometric series is clearly 2N+1 2π . So now for the x 6= 0 case, we can simplify this expression some more to = and now because sin x = e ix −e −ix 2i 1 e i(N+1/2)x − e −i(N+1/2)x , 2π e ix/2 − e −ix/2 , we can rewrite this as

1 2i sin N + 12 x = , 2π 2i sin x2 and canceling out the 2i s gives us the desired expression for DN above. Definition 171 Let f ∈ L2 ([−π, π]). The Nth Cesaro-Fourier mean of f is N 1 X σN f (x) = Sk f (x). N +1 k=0 We’ve rephrase our convergence of Fourier series to the statement that “if the Fourier coefficients are all zero, then the function is zero.” And the direction we’re going with this definition here is that if we can show the partial sums SN f converge to f , then f must be a sum of zeros, but trying to do this with SN directly is our original problem statement! So this “averaged” Cesaro-Fourier mean will be an easier thing to work with, and we’ll try to show that σN f → f instead. 81 Remark 172. We do know from real analysis that the Cesaro means of a sequence of real numbers behave better than the original sequence, but we don’t lose any information, so we have some expectation of getting better behavior here as well. In particular, sequences like {1, −1, 1, −1, · · · } do not converge, but their Cesaro means do. So next time, we’ll discuss more why this convergence works: we’ll show that for every f ∈ L2 , σN f converges to f in L2 . That would then show the desired result, because if all of the Fourier coefficients are zero, then σN f is zero for each N, and thus the limit of those functions is also the zero function.