Lecture 32: Exam 3 Review

Exam 3 review The exam will cover the material through the singular value decomposition. Linear transformations and change of basis will be covered on the final. The main topics on this exam are: • Eigenvalues and eigenvectors • Differential equations du dt = Au and exponentials e At • Symmetric matrices A = A T : These always have real eigenvalues, and they always have “enough” eigenvectors. The eigenvector matrix Q can be an orthogonal matrix, with A = QΛQ T . • Positive definite matrices • Similar matrices B = M−1 AM. Matrices A and B have the same eigen­ values; powers of A will “look like” powers of B. • Singular value decomposition Sample problems 1. This is a question about a differential equation with a skew symmetric matrix. Suppose ⎡ 0 du = Au = ⎣ 1 dt 0 ⎤ −1 0 0 −1 ⎦ u. 1 0 The general solution to this equation will look like u ( t ) = c 1 e λ1 t x 1 + c 2 e λ2 t x 2 + c e e λ3 t x 3 . a) What are the eigenvalues of A? The matrix A is singular; the first and third rows are dependent, so one eigenvalue is λ1 = 0. We might also notice that A is antisymmet­ ric (A T = − A) and realize that its eigenvalues will be imaginary. To find the other two eigenvalues, we’ll solve the equation | A − λI | = 0. � � � − λ −1 0 �� � � 1 −λ −1 � = −λ3 − 2λ = 0. � � � 0 1 −λ � √ √ We conclude λ2 = 2 i and λ3 = − 2 i. At this point we know that our solution will look like: √ u ( t ) = c1 x1 + c2 e 2 it x2 + c e e − √ 2 it x3 . We can now see that the solution doesn’t increase without bound or de­ cay to zero. The size of eiθ is the same for any θ; the exponentials here correspond to points on the unit circle. b) The solution is periodic. When does it return to its original value? (What is its period?) This is not likely to be on the exam, but we can quickly remark that √ √ √ e 2 it = e0 when 2t = 2π, or when t = π 2. c) Show that two eigenvectors of A are orthogonal. The eigenvectors of a symmetric matrix or a skew symmetric matrix are always orthogonal. One choice of eigenvectors of A is: ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 √−1 √1 x 1 = ⎣ 0 ⎦ , x 2 = ⎣ 2 i ⎦ , x 3 = ⎣ 2i ⎦ . 1 −1 1 Don’t forget to conjugate the first vector when computing the inner product of vectors with complex number entries. d) The solution to this differential equation is u(t) = e At u(0). How would we compute e At ? If A = SΛS−1 then e At = SeΛt S−1 where ⎡ λt ⎤ e 1 ⎢ ⎥ .. eΛt = ⎣ ⎦. . eλn t So e At comes from the eigenvalues in Λ and the eigenvectors in S. Fact: A matrix has orthogonal eigenvectors exactly when AA T = A T A; i.e. when A commutes with its transpose. This is true of symmetric, skew symmetric and orthogonal matrices. 2. We’re told that a three by three matrix A has eigenvalues λ1 = 0, λ2 = c and λ3 = 2 and eigenvectors ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 1 1 x1 = ⎣ 1 ⎦ , x2 = ⎣ −1 ⎦ , x3 = ⎣ 1 ⎦ . −2 1 0 a) For which c is the matrix diagonalizable? The matrix is diagonalizable if it has 3 independent eigenvectors. Not only are x1 , x2 and x3 independent, they’re orthogonal. So the matrix is diagonalizable for all values of c. b) For which values of c is the matrix symmetric? If A = QΛQ T is symmetric its eigenvalues (the entries of Λ) are real. On the other hand, if c is real, then A T = Q T Λ T Q = A is symmetric. The matrix is symmetric for all real numbers c. c) For which values of c is the matrix positive definite? All positive definite matrices are symmetric, so c must be real. The eigenvalues of a positive definite matrix must be positive. The eigen­ value 0 is not positive, so this matrix is not positive definite for any values of c. (If c ≥ 0 then the matrix is positive semidefinite.) d) Is it a Markov matrix? In a Markov matrix, one eigenvalue is 1 and the other eigenvalues are smaller than 1. Because λ3 = 2, this cannot be a Markov matrix for any value of c. e) Could P = 12 A be a projection matrix? Projection matrices are real and symmetric so their eigenvalues are real. In addition, we know that their eigenvalues are 1 and 0 because P2 = P implies λ2 = λ. So 12 A could be a projection matrix if c = 0 or c = 2. Note that it was the fact that the eigenvectors were orthogonal that made it possible to answer many of these questions. Singular value decomposition (SVD) is a factorization A = (orthogonal)(diagonal)(orthogonal) = UΣV T . We can do this for any matrix A. The key is to look at the symmetric matrix A T A = VΣ T ΣV T ; here V is the eigenvector matrix for A T A and Σ T Σ is the matrix of eigenvalues σi2 of A T A. Similarly, AA T = UΣΣ T U T and U is the eigenvector matrix for AA T . (Note that we can introduce a sign error if we’re unlucky in choosing eigenvectors for the columns of U. To avoid this, use the formula Avi = σi ui to calculate U from V.) On the exam, you might be asked to find the SVD of a matrix A or you might be given information on U, Σ and V and asked about A. � � 3 0 3. Suppose Σ = and U and V each have two columns. 0 2 a) What can we say about A? We know A is a two by two matrix, and because U, V and Σ are all invertible we know A is nonsingular. � � 3 0 b) What if Σ = ? 0 −5 This is not a valid possibility for Σ. The singular values – the diagonal entries of Σ – are never negative in a singular value decomposition. c) What if Σ = � 3 0 0 0 � ? Then A is a singular matrix of rank 1 and its nullspace has dimension 1. The four fundamental subspaces associated with A are spanned by orthonormal bases made up of selected columns of U and V. In this example, the second column of V is a basis for the nullspace of A. 4. We’re told that A is symmetric and orthogonal. a) What can we say about its eigenvalues? The eigenvalues of symmetric matrices are real. The eigenvalues of orthogonal matrices Q have |λ| = 1; multiplication by an orthogonal matrix doesn’t change the length of a vector. So the eigenvalues of A can only be 1 or −1. b) True or false: A is sure to be positive definite. � False – it could have an eigenvalue of −1, as in 1 0 0 −1 � . c) True or false: A has no repeated eigenvalues. False – if A is a three by three matrix or larger, it’s guaranteed to have repeated eigenvalues because every λ is 1 or −1. d) Is A diagonalizable? Yes, because all symmetric and all orthogonal matrices can be diago­ nalized. In fact, we can choose the eigenvectors of A to be orthogonal. e) Is A nonsingular? Yes; orthogonal matrices are all nonsingular. f) Show P = 12 ( A + I ) is a projection matrix. We could check that P is symmetric and that P2 = P: P2 = � 1 (A + I) 2 �2 = 1 2 ( A + 2A + I ). 4 Because A is orthogonal and symmetric, A2 = A T A = I, so P2 = 1 2 1 ( A + 2A + I ) = ( A + I ) = P. 4 2 Or we could note that since the eigenvalues of A are 1 and −1 then the eigenvalues of 12 ( A + I ) must be 1 and 0. These questions all dealt with eigenvalues and special matrices; that’s what the exam is about.