Lecture 27: Euclidean and Hermitian Spaces

Lecture 27: Euclidean and Hermitian Spaces 27 27.1 Euclidean and Hermitian Spaces Review: Orthogonal Projection Last time, we ended by talking about orthogonal projections and splittings such that V = W ⊕ W ⊥ . Specifcally, suppose we have a vector space V with a bilinear form ⟨·, ·⟩ as well as a subspace W such that the restriction ⟨·, ·⟩|W is non-degenerate. Then, given a orthogonal basis of W = Span{w1 , . . . , wk }, we were able to write a formula for the projection onto W : proj(v) = ⟨w1 , v⟩ ⟨wk , v⟩ wk . w1 + · · · + ⟨wk , wk ⟩ ⟨w1 , w1 ⟩ and we had that v = proj(v) + u, where proj(v) ∈ W and u ∈ W ⊥ . 27.2 Euclidean and Hermitian Spaces In order to evaluate the projection formula, we need an orthogonal basis of a subspace. How do we calculate this? To start, we will be talking about Euclidean and Hermitian spaces. Recall that a pairing ⟨·, ·⟩ is positive defnite if ⟨v, v⟩ > 0 for all v ̸= 0. Defnition 27.1 A Euclidean space is a real vector space V and a symmetric bilinear form ⟨·, ·⟩ such that ⟨·, ·⟩ is positive defnite. Analogously, a Hermitian space is a complex vector space V and a Hermitian form ⟨·, ·⟩ such that ⟨·, ·⟩ is positive defnite. These spaces have the following nice property. Theorem 27.2 If V is Euclidean or Hermitian, then there exists an orthonormal basis {v1 , · · · , vn } for V such that ⟨vi , vj ⟩ = 0 and ⟨vi , vi ⟩ = 1. In particular, the pairing ⟨·, ·⟩ looks like the dot product or the standard Hermitian product in this basis. Proof. From last time, we saw that for any pairing, there exists an orthogonal basis where all of the self-pairings were either 1, 0, or −1. By defnition, all of the self-pairings must be 1 when the form is positive defnite. Furthermore, we no longer have the case where a form can be degenerate on a subspace. Claim 27.3. For any W ⊆ V and ⟨·, ·⟩|W , the restriction ⟨·, ·⟩|W is always nondegenerate. Proof. For each w ∈ W, we need to fnd a w′ ∈ W such that ⟨w, w′ ⟩ = ̸ 0. By the positive defniteness of the pairing, we can just take w′ = w. This means that we can inherit all the properties that we showed last time about non-degenerate subspaces. In particular, we can always perform the orthogonal projection on any subspace of a Euclidean/Hermitian space. 27.3 Gram-Schmidt Algorithm As an application, we have the Gram-Schmidt algorithm for fnding an orthonormal basis. Take a Euclidean or Hermitian vector space V and any basis {v1 , · · · , vn } a basis for V. In order to build an orthonormal basis {u1 , · · · , un }, we inductively build {ui } such that Span{u1 } = Span{v1 }, Span{u1 , u2 } = Span{v1 , v2 }, and so on. Let Vk = Span{v1 , · · · , vk }. • Step 0. Our frst vector must just be a scaled version of v1 . We have to scale it such that ⟨u1 , u1 ⟩ = 1. Let 1 u1 := p v1 . ⟨v1 , v1 ⟩ Then {u1 } is a basis for V1 and ⟨u1 , u1 ⟩ = 1. Lecture 27: Euclidean and Hermitian Spaces • Step 1. Set x2 = projV1 v2 , which is the projection of v2 onto V1 , and let y2 = v2 − x2 , which is the orthogonal portion of the vector v2 . Then, let 1 y2 u2 = p ⟨y2 , y2 ⟩ be the result of normalizing y2 . Then ⟨u1 , u2 ⟩ = 0, since the portion of v2 spanned by u1 has been subtracted of, and ⟨u2 , u2 ⟩ = 1. Thus {u1 , u2 } is an orthonormal basis of V2 . • Step k. Assume that {u1 , · · · , uk } is a basis for Vk . Then, let xk+1 = projVk (vk+1 )yk , the projection onto Vk , and let yk+1 = vk+1 − xk+1 , the orthogonal portion of vk+1 . Then let uk+1 = 1 . ⟨yk+1 , yk+1 ⟩ By induction, {u1 , . . . , uk+1 } is an orthonormal basis for Vk+1 . At every step, the projection formula is used, until a basis is found. Note that whenever we are calculating the projection, we conveniently have an orthogonal (even more, orthonormal) basis of the previous subspace to use the projection formula. Specifcally, on step k the projection formula simplifes to: projvk (vk+1 ) = ⟨u1 , vk+1 ⟩u1 + · · · + ⟨uk , vk+1 ⟩uk . We can interpret the algorithm in matrix form as well. Take M ∈ GLn (R). The columns of M , {v1 , · · · , vn }, are a basis for Rn . The Gram-Schmidt algorithm says that we can take this basis and turn it into an orthonormal basis: u1 = a11 v1 u2 = a12 v1 + a22 v2 u3 = a13 v1 + a23 v2 + a33 v3 , and so on, where {u1 , · · · , un } is an orthonormal basis of Rn . The upshot is that there exists an upper triangular matrix A and an orthogonal matrix B such that M = AB. A is the matrix derived from putting ui as columns, and B is the inverse of the matrix derived from the aij values. Student Question. What sense of orthogonal are we using in the algorithm? Answer. This algorithm refers to orthogonal bases with respect to the standard dot product. One could consider more general kinds of symmetric bilinear forms, and what results are diferent kinds of groups. We will look at this later. For Euclidean spaces, we can consider all pairings as the dot product, so orthogonality always is just the normal defnition. When the pairing is not positive defnite, then we start getting weirder things. Lecture 27: Euclidean and Hermitian Spaces 27.4 Complex Linear Operators Now, we want to focus on linear operators in a Hermitian space (V, ⟨·, ·⟩). Consider a linear operator T : V −→ V. We can the defne the analogue of the adjoint operation on matrices. Defnition 27.4 Pick an orthonormal basis {u1 , · · · , un } of our Hermitian space. By doing so, we can map our vector space into coordinate vectors. We have the correspondence V ⇝ Cn , and the form ⟨·, ·⟩ maps to the standard Hermitian product. Then, T ⇝ M ∈ Matn×n (C). Then let the adjoint T ∗ be the linear operator T ∗ : V −→ V such that, with respect to the basis {u1 , · · · , un }, T ∗ is the matrix M ∗ = M T . The following claim gives the key property of the adjoint linear operator and another way to characterize the adjoint operator. Claim 27.5. For v, w ∈ V, ⟨T v, w⟩ = ⟨v, T ∗ w⟩. This property means that T ∗ is uniquely determined, by putting v = ui and w = uj . Proof. Using the correspondence given by taking a basis, v ⇝ x ∈ Cn w ⇝ y ∈ Cn T v ⇝ M x ∈ Cn . Then, ⟨T v, w⟩ = (M x)∗ y = x∗ M ∗ y = x∗ (M ∗ y) = ⟨v, T ∗ w⟩. This also means that T ∗ is independent of the choice of a basis. Just as we defned adjoint for both linear operators and matrices, we can do the same for the defnition of Hermitian. Defnition 27.6 A linear operator T : V −→ V is a Hermitian operator if T ∗ = T, which is equivalent to ⟨T v, w⟩ = ⟨v, T w⟩. Also, a unitary matrix is a matrix such that U ∗ U = In and U ∗ = U −1 . The following defnition is analogous. Defnition 27.7 A linear operator T : V −→ V is a unitary operator if T ∗ T = Id, or equivalently if ⟨T v, T w⟩ = ⟨v, w⟩ for all v, w ∈ V. The following defnition is harder to motivate, but encapsulates the previous two. Defnition 27.8 A linear operator T : V −→ V is normal if T T ∗ = T ∗ T , which is equivalent to ⟨v, T ∗ T w⟩ = ⟨T v, T w⟩ = ⟨T ∗ v, T ∗ w⟩ = ⟨v, T T ∗ w⟩. Lecture 27: Euclidean and Hermitian Spaces The set of unitary matrices and the set of Hermitian matrices are both subsets of the set of normal matrices.   1 1 0 However, there are normal matrices that are neither Hermitian nor unitary. For example, A = 0 1 1 is 1 0 1 normal but neither Hermitian nor unitary. Next class, we will discuss the Spectral Theorem, which is the main important property of normal matrices. Theorem 27.9 (Spectral Theorem) For a Hermitian space V, and a normal linear operator T : V −→ V , V has an orthonormal basis {u1 , · · · , un } where each ui is an eigenvector of T. In other words, we both diagonalize T as well as fnd an orthonormal basis for V . We don’t need to mess around with Jordan forms. The matrix version states that for a normal matrix M ∈ Matn×n (C),95 it is possible to fnd a unitary matrix P 96 such that   λ1   .. P ∗ M P = P −1 M P =  . . λn The columns of P form our eigenbasis. What about Euclidean spaces? In full generality, this theorem is false. However, for a Euclidean space V and a symmetric linear operator T : V −→ V , T does have an orthonormal eigenbasis. Generalizing from symmetric to orthogonal, which is the analogous version of unitary, or some condition similar to “normal," does not work. For example, we saw that it is not true for orthogonal matrices, as rotation matrices in the plane do not have any eigenvectors. 95 M ∗ M 96 P ∗ P = MM∗ =I