Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices

Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices 9 Dimension Formula 9.1 Review Last time, we discussed linear transformations between two vector spaces. By picking a basis cleverly, it is possible to write the matrix of the linear transformation in a very nice form. For example, given a linear transformation M : F n −→ F m , by changing the bases for F n and F m with the invertible matrices P and Q, the matrix A will have a very simple form. Fn M Q P Fn A With appropriate bases, −1 A=Q Fm Fm  MP = Ik 0  0 , 0 where A is an m×n matrix with the identity in the top left corner. The frst k columns correspond to the image and the last n − k columns correspond to the kernel. As a corollary, it is possible to see the dimension formula dim im(A) + dim ker(A) = n. Corollary 9.1 Given a matrix M ∈ Matm×n (F ), we have rank(M ) = rank(M T ). Essentially, this corollary states that the dimension of the span of the columns is the same as the dimension of the span of the rows, which is surprising! The frst is a subspace of F m , while the second is a subspace of F n , but they still have the same dimension. Sketch of Proof. This theorem is clearly true for A, since the row-rank and the column-rank are both just k. However, since A and M difer by isomorphisms P and Q, the rank of A is the same as the rank of M. Similarly, the rank of AT is also equal to the rank of M T . Therefore, rank(M ) = rank(M T ). We are not going to use this often in this course, but this is a fact emphasized in traditional linear algebra classes. 9.2 Linear Operators Today, we will specialize the discussion on arbitrary linear transformations to linear operators, which go from a vector space to itself. Defnition 9.2 A linear operator is a linear transformation T : V −→ V. Let’s see some examples. Example 9.3 Let V = R2 . Then, T is the linear transformation that is rotation by angle θ counterclockwise. This goes from the vector space to itself. Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices Example 9.4 Let V = {polynomials of degree ≤ 2}. Then the derivative T (f (t)) = f ′ (t) is a linear operator. The frst natural question to ask about linear operators is working out the matrix of the linear transformation upon picking a basis. The only diference between this discussion and the discussion on linear transformations is that here, the transformation is from a vector space to itself, so once a basis has been picked, both sides have a fxed basis. For general linear transformations from a vector space to a diferent vector space, two diferent bases can be picked. Guiding Question What is the matrix of a linear operator on a vector space with a chosen basis? Consider a basis B : F n −→ V. Then T becomes a square matrix A ∈ Matn×n (F ). For Example 9.3, picking the basis standard basis gives a rotation matrix:       1 0 cos θ − sin θ , ⇝A= . 0 1 sin θ cos θ This is determined by fguring out where the ith basis vector is mapped, which is the ith column. For Example 9.4, it is also possible to write down a matrix:   0 1 0 {1, t, t2 } ⇝ A = 0 0 2 . 0 0 0 This should all be reminiscent of the previous section. Proposition 9.5 When working with linear operators T : V −→ V , for V fnite-dimensionala , then T is injective ↔ T is surjective ↔ is an isomorphism. a In this class, the implicit assumption will always be that we are working with fnite-dimensional vector spaces In fact, this fact is true for maps from a fnite set to itself. Finite-dimensional vector spaces can be infnite, but have the same property. Proof. Using the dimension formula, dim ker T + dim im T = dim V. If T is injective, then dim ker T = 0, which is true if and only if dim im T = dim V, which means that T is surjective. So fnite-dimensional vector spaces behave a lot like fnite sets. 9.3 Change of Basis Now, the next natural question to ask is about changing bases. Guiding Question What happens to a matrix for T : V −→ V upon changing basis for V ? Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices Specifying a basis for V determines a diagram T V V B . B A Fn Fm A new basis comes from an invertible matrix P ∈ GLn (F ), where the new basis is B ′ = B · P, and determines an extended diagram T V V B B ′ F B n A n A′ P F F n B′ . P Fn There is a new matrix A′ that represents the same linear transformation T. The diference between this case and the general case is that the bases are the same on either side of the transformation, and there is no longer the freedom to choose diferent bases for the domain and the codomain. The new matrix, by following the arrows on the diagram, is A′ = P −1 AP. The matrix A′ is related to A by conjugation by P. Defnition 9.6 A matrix A′ is similar to A if there exists some P ∈ GLn (F ) such that A′ = P −1 AP. Similar matrices arise from the same linear operator from a vector space to itself, but with diferent bases picked. Again, to emphasize, the diference between today’s case, T : V −→ V , and the case in the last section, T : V −→ W , is that in the frst case there is only one base change matrix P, instead of P and Q, since the matrix must operate the same on the left and right sides. As an result, given a vector space V and an operator T : V −→ V, it is possible to defne the determinant of T without having to specify a basis. The vector space V might be a vector space without a canonical basis, but it is still possible to defne the determinant. Picking any basis of V produces a square matrix A, and the determinant would then be det(T ) = det(A). In fact, from the base change formula, it is clear that the determinant does not depend on which basis is used! From a diferent basis, det(A′ ) = det(P −1 AP ) = det(P )−1 det(A) det(P ) = det(A), since the determinant is multiplicative. As a result, it is possible to defne the determinant of T independently of the choice of basis29 , and so det(T ) has a meaning outside of a particular basis. For example, on Rn , the determinant represents a "volume," which is independent of the particular choice of basis. Here, we are saying that even for felds like fnite felds, where "volume" may not make sense, the determinant still has some intrinsic meaning. 9.4 Eigenvectors, Eigenvalues, and Diagonalizable Matrices Our discussion leads to the following question, which is the same as last class, but for linear operators. Guiding Question How nice can we make A by changing basis of V ? Last class, it was possible to make the matrix extremely nice, since we could pick a basis for the domain and for the codomain. Now, let’s see an example for when the domain is the same vector space as the codomain. 29 It doesn’t depend on which basis was chosen, so any basis works Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices Example 9.7 Let V = R2 . Consider  2 A= 3 We see that A and A  3 . 2       1 5 1 = =5 1 5 1       −1 1 −1 = = −1 . 1 −1 1 The operator is scaling the vector (1, 1) and fipping the vector (−1, 1), and the transformation on any other vector will be a combination of these two moves, scaling and fipping. In particular, taking   1 −1 P = , 1 1 which has the frst vector as the frst column and the second vector as the second column, gives   5 0 A′ = P −1 AP = . 0 −1 In the new basis, it is possible to make the matrix diagonal! Making the matrix diagonal makes it possible to see how it operates, which is stretching by 5 in one direction, and fipping in the other direction (both independently of the other direction). In general, we will really want to be able to make matrices diagonal, since it allows us to see what it is doing in the direction of each basis vector, independently of the other directions (as there are 0s in the matrix elsewhere). This gold standard type of vector will be called an eigenvector. Defnition 9.8 A vector v ̸= 0 is an eigenvector if T v = λv for some λ ∈ F, and λ is called an eigenvalue. When an operator is applied to a vector, the result is proportional to the vector. The operator maintains the direction of the vector, and just scales it. Obviously, scaling an eigenvector by some nonzero scalar also results Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices in another eigenvector. Example 9.9     1 −1 For Example 9.7, the vector is an eigenvector with eigenvalue 5, and is an eigenvector with 1 1 eigenvalue -1. This example is special because not only are there eigenvectors, there are enough to form a basis. Defnition 9.10 A basis {v1 , · · · , vn } of V where each vi is an eigenvector; that is, T vi = λi vi , then the basis is called an eigenbasis. In an eigenbasis, the matrix for T is  λ1  .. . 0 ··· .. . ···  0 ..  , .  λn which is diagonal with λi in the (i, i)th entry. Diagonal matrices are extremely nice. In general, it is very hard to take matrices to high powers, but for diagonal matrices, each entry is simply raised to that power. Defnition 9.11 If a linear operator has an eigenbasis T, it is called diagonalizable. An equivalent defnition holds for matrices. Defnition 9.12 Given a matrix A, if there exists some invertible P such that P −1 AP = D for some diagonal matrix D, then A is called diagonalizable. That is, A is diagonalizable if it is similar to a diagonal matrix. The key concept is that an eigenbasis provides the directions in which the operator T behaves nicely by simply scaling or fipping a vector in that direction. 9.5 Finding Eigenvalues and Eigenvectors Unfortunately, not every matrix is diagonalizable, but the focus for the next few classes will be fnding eigenvectors, eigenvalues, and eigenbases, assuming that a matrix is diagonalizable. Guiding Question How do we fnd eigenvectors, eigenvalues, and eigenbases? • Step 1. Perhaps unintuitively, the frst step is to fnd possible eigenvalues! Given a matrix A ∈ Matn×n (F ) in some less good basis, we want to fnd eigenvectors that form a better basis that is an eigenbasis, so that A will be diagonal and have a nicer form. Suppose λ is an eigenvalue for A. Then there exists some nonzero v such that Av = λv, Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices by defnition. (There may be lots of v, and in fact scaling any v will produce another eigenvector, but for a given λ we just want to know if there is a v at all.) We know that λv = λIn v, so this is equivalent to (λIn − A)v = 0 for some v. That is, the kernel is nontrivial: ker(λIn − A) ̸= {⃗0}. This is equivalent to λIn − A is not invertible, which happens if and only if det(λIn − A) = 0. This is not bad at all! The determinant is a formula that we can just calculate. So in fact, we want to look for λ such that the equation det(λIn − A) = 0 . holds. It is customary in this context to replace t with λ, and so with t as the variable, we have p(t) := det(tIn − A), which is a polynomial of degree n in t called the characteristic polynomial. Example 9.13 Given A=  2 3  3 , 2 we have the characteristic polynomial   t − 2 −3 = (t − 2)(t − 2) − (−3)(−3) = t2 − 4t − 5. pA (t) = det −3 t − 2 In general, where A = (aij ), we have  t − a11  .. pA (t) = det  . ⋆ ··· .. . ··· ⋆ .. .   n  = t + ··· , t − ann which is a degree n polynomial in t. If A′ is similar to A, then they have the same characteristic polynomial, since the determinant is basisinvariant. Proposition 9.14 Given λ ∈ F, λ is an eigenvalue for A if and only if pA (λ) = 0; that is, if and only if λ is a root of pA (t). For example, for our earlier example, the eigenvalues would be −1 and 5, since t2 − 4t − 5 = (t + 1)(t − 5). As a caveat, if F is an arbitrary feld, there may not be any roots. For example, a rotation matrix over R does not have any real eigenvalues. However, if F = C, there will always be n roots (not necessarily distinct), and so there will always be eigenvalues. • Step 2. For each eigenvalue, fnd the associated eigenvectors. For each λ, we want to take a vector in ker(λIn − A), which by assumption is a nonzero subspace. Using Gaussian elimination or row operations, we can mechanically compute a basis for ker(λIn − A), although we will not spend a lot of time on this. Lecture 9: Eigenvectors, Eigenvalues, and Diagonalizable Matrices Example 9.15 For λ = 5, we have  5 · I2 − A = and the kernel is We will say more about this next lecture!  3 −3 , −3 3   1 ker = Span . 1