Lecture 17: Minimizers, Orthogonal Complements and the Riesz Representation Theorem

April 22, 2021 Last time, we discussed orthonormal bases, considering the concrete question of whether complex exponentials formed an orthonormal basis for L2 ([−π, π]). Today, we’ll go back to a general discussion of Hilbert spaces, and the rest of the course from here on will be general theory and some concrete applications to particular problems. Our first topic today will be length minimizers: recall that we can describe a norm on V /W for subspaces of a normed vector space, and we did so via an infimum. It makes sense to ask whether this minimal distance is actually achieved: Theorem 178 Let C be a nonempty closed subset of a Hilbert space H which is convex, meaning that for all v1 , v2 ∈ C, we have tv1 + (1 − t)v2 ∈ C for all t ∈ [0, 1]. Then there exists a unique element v ∈ C with ||v || = inf u∈C ||u|| (this is a length minimizer). The convexity condition can alternatively be stated as “the line segment between any two elements of C is contained in C.” And to connect this with our discussion earlier, one such example of a set would be v + W for some closed subspace W of C and some v ∈ H. Remark 179. The condition that C is closed is required: for example, we can let C be an open disk outside the origin, in which case the minimum norm is not achieved (because it’s on the boundary). And convexity is also required – for example, otherwise we could take the complement of an open disk centered at the origin, in which case the minimum norm is achieved on the entire boundary. Proof. We should recall that a = inf S if and only if a is a lower bound for S, and there exists a sequence {sn } in S with sn → a. If we let d = inf u∈C ||u||, this is some finite number because norms are always bounded from below by 0, and C is nonempty. So there exists some sequence {un } in C such that ||un || → d. We claim that this sequence is actually Cauchy. To see that, let ε > 0 – because of convergence of ||un || to d, there exists some N so that for all n ≥ N, we have 2||un ||2 < 2d 2 + ε2 . 2 Then for all n, m ≥ N, the parallelogram law tells us that ||um − un ||2 = 2||um ||2 + 2||un ||2 − 4 un +um lies on the line segment 2 un +um 2 ≥ d 2 , and thus 2 and now because C. Therefore, un + um 2 2 , between un and um (taking t = 21 ), convexity tells us that it is also in ||um − un ||2 ≤ 2||um ||2 − 2d 2 + 2||un ||2 − 2d 2 < ε2 ε2 + = ε2 2 2 by our choice of N, and taking a square root shows that the sequence {un } is indeed Cauchy. Because our Hilbert space is complete, this means that the sequence also converges, and thus there is some v ∈ H such that un → v , and v ∈ C as well because our subset C is closed. So now continuity of the norm tells us that ||v || = lim ||un || = d, n→∞ and thus we’ve found our minimizer v ∈ C. To show uniqueness, suppose that v , v are both in C and have norm d. Then the parallelogram law tells us that ||v − v ||2 = 2||v ||2 + 2||v ||2 − 4 again using that v +v 2 2 v +v 2 ≤ 2d 2 + 2d 2 − 4d 2 = 0, is also in C by convexity, and thus we must have v − v = 0 =⇒ v = v . We’ll obtain some important consequences from this result – the first one is how to decompose our Hilbert space using a closed linear subspace, much like we usually like to do in Rn and Cn . Theorem 180 Let H be a Hilbert space, and let W ⊂ H be a subspace. Then the orthogonal complement W ⊥ = {u ∈ H : hu, w i = 0 ∀w ∈ W } is a closed linear subspace of H. Furthermore, if W is closed, then H = W ⊕ W ⊥ ; in other words, for all u ∈ H, we can write u = w + w ⊥ for some unique w ∈ W and w ⊥ ∈ W ⊥ .) A picture to keep in mind is the case where H is R2 and W is the x-axis – then W ⊥ would be the y -axis, and we’re saying that all elements can be broken up into a component along the x-axis and a component along the y -axis. Proof. Showing that W ⊥ is a subspace is clear, because if hu1 , w i = 0 and hu2 , w i = 0 for all w ∈ W , any linear combination of u1 and u2 will also be orthogonal to all w ∈ W by linearity of the inner product. Furthermore, W ∩ W ⊥ = {0}, because any element w ∈ W that is also in W ⊥ must satisfy hw , w i = 0 =⇒ w = 0. To show that W ⊥ is closed, let {un } be a sequence in W ⊥ converging to u ∈ H. We wish to show that hu, w i = 0 for all w ∈ W , so that u ∈ W ⊥ as well. Indeed, by continuity of the inner product, we have hu, w i = lim hun , w i = lim 0 = 0, n→∞ n→∞ so that our sequential limit is also in our subspace W ⊥ . It remains to show that H = W ⊕ W ⊥ if W is closed. The result is clear for W = H, since W ⊥ = {0} and H = H ⊕ {0} is a trivial decomposition. Otherwise, if W 6= H, then let u ∈ H \ W (that is, u is in H but not W ), and define the set C = u + W = {u + w : w ∈ W }. This set C is nonempty because it contains u, and it is convex because for any two elements u + w1 , u + w2 ∈ C (for w1 , w2 ∈ W ) and for any t ∈ [0, 1], we have t(u + w1 ) + (1 − t)(u + w2 ) = (t + (1 − t))u + tw1 + (1 − t)w2 = u + (tw1 + (1 − t)w2 ) and the last term is in W because subspaces are closed under linear combinations. So we now need to show that C is closed: indeed, if u + wn is a sequence of elements in C that converge to some element v ∈ H, we know that u + wn → v =⇒ wn → v − u, and because W is closed, wn must converge to some element in W . Thus v − u ∈ W , and thus v = u + w for some w ∈ W , which is exactly the definition of being in C. So C is indeed closed. So returning to the problem, if we want to write an element u of H as a sum of a part in W and a part in W ⊥ , it makes sense that our component in W ⊥ will be the minimizer to C (keeping the R2 example from above in mind). So applying Theorem 178, because C is closed and convex, there is some unique v ∈ C with ||v || = inf ||c|| = inf ||u + w ||. c∈C w ∈W . Since v ∈ C, we know that u − v ∈ W , so we will write u = (u − v ) + v . Our goal is to show that v ∈ W ⊥ , and we do this with a variational argument (in physics, this is the Euler-Lagrange equations, and it is another way of phrasing properties of the infimum). If w ∈ W , define the function f (t) = ||v + tw ||2 = ||v ||2 + t 2 ||w ||2 + 2tRehv , w i, which is a polynomial in t. We know that f (t) has a minimum at t = 0, because all elements of the form v + tw are in C, and we know the minimizer of norm uniquely occurs at v . So f 0 (0) = 0, and thus 2Rehv , w i = 0. So the real part of the inner product is zero, and now we can repeat this argument but with ||v + i tw || instead of ||v + tw ||, which will show us that Rehv , i w i = Imhv , w i = 0. Therefore, hv , w i = 0, and since this argument was true for all w ∈ W , we must have v ∈ W ⊥ . It remains to show that this decomposition is unique, and this is true because W ∩ W ⊥ = {0}: more specifically, if u = w1 + w1⊥ = w2 + w2⊥ , that means that w1 − w2 = w2⊥ − w1⊥ is in both W and W ⊥ , and thus both sides of this equation are 0. So w1 = w2 and w1⊥ = w2⊥ , showing uniqueness. The following result is left as an exercise for us: Theorem 181 If W ⊂ H is a subspace, then (W ⊥ )⊥ is the closure W of W . In particular, if W is closed, then (W ⊥ )⊥ = W . Now that we have this decomposition u = w + w ⊥ for our subspace W , we can construct a map which takes in u and outputs w . If we use the R2 example from above, we can see that this map is a projection onto the x-axis, and more generally we can make the following definition: Definition 182 Let P : H → H be a bounded linear operator. Then P is a projection if P 2 = P . Proposition 183 Let H be a Hilbert space, and let W ⊂ H be a closed subspace. Then the map ΠW : H → H sending v = w + w ⊥ (for w ∈ W, w ⊥ = W ⊥ ) to w is a projection operator. Proof. First, we show that ΠW is linear. Indeed, if v1 = w1 + w1⊥ and v2 = w2 + w2⊥ , and we have λ1 , λ2 ∈ C, then λ1 v1 + λ2 v2 = (λ1 w1 + λ2 w2 ) + (λ1 w1⊥ + λ2 w2⊥ ). The two terms on the right-hand side are in W and W ⊥ , respectively, by closure of subspaces under linear combinations. So ΠW (λ1 v1 + λ2 v2 ) = λ1 w1 + λ2 w2 , which is indeed λ1 ΠW (v1 ) + λ2 ΠW (v2 ), as desired. We can also see that ΠW is bounded, because when v = w + w ⊥ , ||v ||2 = ||w + w ⊥ ||2 = ||w ||2 + ||w⊥ ||2 ≥ ||w ||2 (since the inner product cross term is zero when hw , w ⊥ i = 0). Therefore, ||ΠW (v )|| ≤ ||v ||, and the operator norm is at most 1. And now we just need to check that Π2W = ΠW : if v = w + w ⊥ , then ΠW (v ) = w , and then ΠW (ΠW (v )) = ΠW (w ) = w = ΠW (v ), and since this is true for all v , we have Π2W = ΠW , as desired. Our next application of length minimizers will be the following important result: Theorem 184 (Riesz Representation Theorem) Let H be a Hilbert space. Then for all f ∈ H 0 , there exists a unique v ∈ H so that f (u) = hu, v i for all u ∈ H. In other words, every element of the dual can be realized as an inner product with a fixed vector. We’ve seen something similar before when we proved that the dual of `p is identified with `q (for 1 p + 1 q = 2) via a pairing, and the p = q = 2 case is the example relevant to Hilbert spaces. Proof. If such a v exists, it is unique, because f (u) = hu, v i = hu, ṽ i = 0, then hu, v − v i = 0 for all u ∈ H. Setting u = v − v tells us that v − v = 0. So we just need to construct such a v that works. The easiest case is f = 0, because in that case, we take v = 0. Otherwise, there exists some u1 ∈ H so that f (u1 ) 6= 0, and we take u0 = u1 f (u1 ) so that f (u0 ) = 1. We can then define the nonempty set C = {u ∈ H : f (u) = 1} = f −1 ({1}), which is closed because f is a continuous function, {1} only has one element so is closed, and the preimage of a closed set by a continuous function is a closed set. We claim that C is convex: indeed, if u1 , u2 ∈ C and t ∈ [0, 1], then f (tu1 + (1 − t)u2 ) = tf (u1 ) + (1 − t)f (u2 ) = t · 1 + (1 − t) · 1 = 1, so that tu1 + (1 − t)u2 is also in C. So now by Theorem 178, there exists v0 ∈ C so that v0 = inf u∈C ||u||, and we define v = v0 ||v0 ||2 (noting that v0 6= 0 because the infimum is not 0). We claim that this is the v that we want; in other words, let’s check that f (u) = hu, v i. Indeed, if we let N = f −1 ({0}) = {w ∈ H : f (w ) = 0} be the nullspace of f , then we can check that C = {v0 + w : w ∈ N} and that ||v0 || = inf w ∈N ||v0 + w ||. So by the argument that we made earlier in Theorem 180 using ||v0 + tw ||2 , v0 ∈ N ⊥ , and now for any w ∈ H, f (u − f (u)v0 ) = f (u) − f (u)f (v0 ) = 0 by linearity of f , and thus u = (u − f (u)v0 ) + f (u)v0 is a sum of a component in N and a component in N ⊥ . hu, v i = 1 1 hu, v0 i = [h(u − f (u)v0 ), v0 i + f (u)hv0 , v0 i] , ||v0 ||2 ||v0 ||2 The first term here has u − f (u)v0 ∈ N and v0 ∈ N ⊥ , so that inner product is zero, and we’re left with = f (u) hv0 , v0 i = f (u), ||v0 ||2 as desired. So we’ve found v (a scaled version of the minimizer) so that f (u) = hu, v i for all u, concluding the proof. We’ll study adjoint operators next time – we defined it as a map from dual spaces to dual spaces, but because we can identify dual spaces of Hilbert spaces with themselves, adjoint operators will be essentially regular operators, and we’ll soon see how they relate to solving equations on Hilbert spaces and why they are the analogs of the transpose matrix in finite-dimensional linear algebra as well.