Lecture 25: Orthogonality

Lecture 25: Orthogonality 25 25.1 Orthogonality Review: Bilinear Forms We discussed bilinear forms last time, which was a function that took two vectors as input, and gave a scalar as an output. It was linear in both of the inputs. For now, we will only be interested in symmetric bilinear forms because they model after the dot product. On real vectors, every bilinear form can be written as: ⟨ #x», #» y ⟩ = #» x T A #y». The form is symmetric if and only if A is symmetric. We may also refer to a bilinear form as a ‘pairing’. 25.2 Hermitian Forms When we worked over a complex vector space, we discussed the Hermitian form, the complex version of a symmetric bilinear form. However, symmetry did not work as normal. They had a complex conjugate in #» = ⟨ w, #» #» their relation: ⟨ #v», w⟩ v ⟩ The standard Hermitian form was defned to be ⟨ #x», #» y ⟩ = #» x ∗ #y». In particular, Hermitian forms are not exactly linear. The second term is linear, but when we scale the frst term, we are scaling the output by the complex conjugate. The following chart summarizes the comparison between symmetric bilinear forms over the reals and Hermitian forms. Although they have subtle diferences, we will study them together. Field R C Canonical Example dot product Symmetry #» = ⟨ w, #» #» ⟨ #» v , w⟩ v⟩ Matrix AT = A Change of basis P T AP Standard Hermitian form #» = ⟨ w, #» #» ⟨ #» v , w⟩ v⟩ ? ? Now we will fgure out what goes in the two remaining entries of the table. In order to fnd the matrix for a Hermitian form on V, a vector space over C, the process is analogous to fnding the matrix for a symmetric form on a vector space over R. First, pick a basis v#»1 , . . . , v#» n of V. Then, set A = (aij )i,j=1,··· ,n , where aij = ⟨ #» v i , #» v j ⟩. If #» v = x1 #» v 1 + · · · + xn #» vn and #» = y #» #» w 1 v 1 + · · · + yn v n , by using the almost-bilinearity of the Hermitian form and expanding the Hermitian form in the same way that bilinearity was used to expand the bilinear form, we get #» = #» ⟨ #v», w⟩ x ∗ A #y», where there is a conjugate transpose instead of a transpose. Then, for every entry of the matrix A, aij = ⟨ #v»i , #» v j ⟩ = ⟨ #v»j , #» v i ⟩ = aji , and so A∗ = A. Defnition 25.1 A matrix A is called a Hermitian matrix if A∗ = A. The upshot is that giving a Hermitian form is essentially equivalent to providing a Hermitian matrix on Cn : Hermitian form on Cn ← → Hermitian matrix A90 Similarly, one can show that the change of basis formula is given by A′ = P ∗ AP. 90 Use the standard basis of Cn , e1 , · · · , en . Lecture 25: Orthogonality Example 25.2 (n = 2) For a Hermitian matrix  A=  5 2 + 2i , 2 − 2i 3 the associated Hermitian form is ⟨ #» x , #» y ⟩ = ⃗x∗ A⃗y = 5x1 y1 + 3x2 y2 + (2 + 2i)x1 y2 + (2 − 2i)x2 y1 , simply by evaluating the matrix product. In particular, when ⃗x = ⃗y , then the Hermitian inner product is actually a real number! The Hermitian inner product of ⃗x with itself is ⟨⃗x, ⃗x⟩ = 5|x1 |2 + 3|x2 |2 + Re((2 + 2i)(x1 x2 )) ∈ R. It turns out Hermitian matrices have very nice properties compared to random complex matrices. Let’s see one of them now. Claim 25.3. A Hermitian matrix always has real eigenvalues. Proof. An eigenvalue λ ∈ C of a Hermitian matrix A satisfes A⃗v = λ⃗v for some ⃗v ∈ Cn . By the Hermitian property, v ∗ Av ∈ R, but because it is an eigenvector, this is equal to v ∗ λv = λ(v ∗ v), where v ∗ v is a nonzero real number. Thus, λ= v ∗ Av ∈ R. v∗ v Not only is the eigenvalue real, λ can be obtained by comparing the value of the Hermitian form to the value of the standard Hermitian form. From now on, we will study symmetric bilinear forms on the real numbers and Hermitian forms on the complex numbers in parallel. They have very similar properties. One idea that carries over is orthogonal matrices. Example 25.4 Consider R equipped with the standard dot product. Let M ∈ Matn×n (R). Recall that we had several ways of describing that M was orthogonal. The following properties are all equivalent: M is orthogonal ⇐⇒ M⃗x · M ⃗y = ⃗x · ⃗y ⇐⇒ M T M = In ⇐⇒ for column vectors v#»i , v#»j of M , ⟨v#»i , v#»j ⟩ = ( 1 if i = j 0 otherwise. The last condition says that the columns of M are orthonormal. A similar type of matrix can be defned for C with the standard Hermitian form. Lecture 25: Orthogonality Defnition 25.5 Let V = Cn . The matrix M is called unitary if it satisfes any of the following equivalent conditions. M is unitary ⇐⇒ ⟨M⃗x, M ⃗y ⟩ = ⟨⃗x, ⃗y ⟩ ⇐⇒ M ∗ M = In ; M −1 = M ∗ ⇐⇒ for column vectors v#»i , v#»j of M , v#»i ∗ v#»j = ( 1 if i = j 0 otherwise. What you can do in one world is very parallel to what you can do in the other world. 25.3 Orthogonality Consider V real with ⟨·, ·⟩ symmetric, or V complex with ⟨·, ·⟩ Hermitian. Defnition 25.6 ⃗ ⟩ = 0 for all w ⃗ ∈ W. A vector ⃗v is orthogonal to w ⃗ if ⟨⃗v , w ⃗ ⟩ = 0. Also, for W ⊂ V, a vector ⃗v ⊥ a W if ⟨⃗v , w a This is the symbol representing orthogonality If ⟨·, ·⟩ is not the standard inner product, this idea of "orthogonality" does not necessarily correspond with geometric intuition. Example25.7    −1 0 0 0 1 0  0 1 0 0    Let A =   0 0 1 0 , and ⃗v = 0. Then ⟨⃗v , ⃗v ⟩ = 0, so ⃗v is orthogonal to itself. 0 0 0 1 1 This form comes up a lot when studying special relativity, but does not necessarily correspond to our geometric intuition of what "orthogonality" means. One thing that we do a lot of in the geometric world is that we take a subspace W and then look at the vectors that are orthogonal to it. Defnition 25.8 For a subspace W ⊂ V, the orthogonal complement is W ⊥ = {⃗v ∈ V such that ⃗v ⊥ W }. Example 25.9 Consider V = R3 , and W as a plane. Then W ⊥ is a line perpendicular to W . In this case, W ⊥ is a complement to W, and R3 = W ⊕ W ⊥ . In general, there are many possible complements, or ways to extend a basis of a subspace to the whole vector space, but the dot product picks out a specifc one. Guiding Question For a general bilinear form, when can we decompose V into the sum of a subpace and its orthogonal complement? It is possible to some extent, but we need to be careful. For example, ⃗v = ̸ 0 can be perpendicular to all of V. In particular, taking A = [0], ⃗v ⊥ ⃗v ′ for any ⃗v , ⃗v ′ ∈ V. Thus, for any ⃗v , ⃗v ⊥ = V. Lecture 25: Orthogonality Defnition 25.10 The null space is N = {⃗v ∈ V : ⃗v ⊥ = V } ⊆ V. If A = In , the null space is N = {⃗0}, but when A = 0, the null space is N = V. Defnition 25.11 Given a vector space V and a bilinear form ⟨·, ·⟩, if N = {⃗0}, (V, ⟨·, ·⟩) is called non-degenerate. Given the matrix of a form, how is it possible to tell whether the bilinear form is non-degenerate? Proposition 25.12 A form on a vector space (V, ⟨·, ·⟩) is non-degenerate if and only if the matrix of the form, A, is invertible, which is when det A ̸= 0. So when A = In , it is non-degenerate, but when A = 0, it is extremely degenerate. In matrix form, v ∈ N if and only if w ⃗ ∗/T A #» v = 091 for all w ⃗ ∈ V , which is equivalent to saying that A⃗v = ⃗0, and so ⃗v ∈ ker A. Consider the restriction of the form to W, ⟨·, ·⟩|W : W ×W → R or C. It can happen that ⟨·, ·⟩|W can be degenerate, even if ⟨·, ·⟩ is non-degenerate. Example 25.13   −1 0 0 0  0 1 0 0  Let V = R4 . The matrix from before, A =   0 0 1 0 , is non-degenerate but a bit weird since it had 0 0 0 1   1 0  that property that a vector could be orthogonal to itself. However, let v =  0 and consider 1 W = Span(v). Then, W ×W −→ R ⟨a⃗v , a⃗v ⟩ = 0, and the restriction of the form to W is identically zero and is degenerate. Given a vector space V, a form ⟨·, ·⟩, and a subspace W ⊂ V, the restriction of the form ⟨·, ·⟩|W is non-degenerate if and only if for all non-zero w ⃗ ∈ W, there exists some w ⃗ ′ ̸= w ⃗ such that ⟨w, ⃗ w ⃗ ′ ⟩ ̸= 0. In particular, this is ⊥ ⊥ ⃗ equivalent to saying that W ∩ W = {0}, by the defnition of W . If there were a vector both in W and W ⊥ , it would not be possible to fnd such a w ⃗ ′ , since the inner product with w ⃗ would always be zero since it is in W ⊥ . Theorem 25.14 If ⟨·, ·⟩|W is non-degenerate, then V = W ⊕ W ⊥ is a direct sum of W and its orthogonal space. As a reminder, there are several equivalent ways of thinking about the direct sum. If V = W ⊕ W ⊥ , then the following are all true 1. If w1 , · · · , wk is a basis for W, and w1′ , · · · , wj′ is a basis for W ⊥ , then gluing them together gets a basis {w1 , · · · , wk , w1′ , · · · , wj′ } for V. 2. Every ⃗v ∈ V can be written uniquely as ⃗v = w ⃗ + ⃗u where w ⃗ ∈ W and ⃗u ∈ W ⊥ . 91 Depending on whether we consider R or C, we take either the conjugate transpose or the transpose. Lecture 25: Orthogonality 3. The intersection is W ∩ W ⊥ = {⃗0} and V = W + W ⊥ . These are all diferent ways of talking about the way V has been split here. It is not always the case that W and W ⊥ direct sum to V. Once the non-degeneracy condition has been encoded into the restriction of the form to W, a splitting can be found.