Coordinate Change Tutorial Materials

Math 217: Summary of Change of Basis and All That I. Coordinates. Let V be a vector space with basis B = {~v1 , . . . , ~vn }. Every element ~x in V can be written uniquely as a linear combination of the basis elements: ~x = a1~v1 + a2~v2 + · · · + an~vn . The scalars ai ’s can be recorded in a column vector, called the coordinate column vector of ~x with respect to the basis B:   a1  a2   . .  ..  an We use the notation [~x]B to denote this column. For short, we also call this column the “B-coordinates” of ~x. Every vector ~x corresponds to exactly one such column vector in Rn , and vice versa. That is, for all intents and purposes, we have just identified the vector space V with the more familiar space Rn . E XAMPLE I: The vector space P2 of polynomials of degree ≤ 2 consists of all expressions of the form a + bx + cx2 . Here, the choice of the basis {1, x, x2 } gives an isomorphism identifying P2 with R3 :   a 3 2  P2 → R a + bx + cx 7→ b  . c The formal mathematical statement is that the mapping V → Rn ~x 7→ [~x]B 2 is an isomorphism of vector spaces. This is deeper than just matching up the elements of V and Rn . More is true: the way we add and scalar multiply also match up: [~x + ~y ]B = [~x]B + [~y ]B and [c~x]B = c[~x]B . Summary of I: B Y CHOOSING A BASIS , A VECTOR SPACE OF DIMENSION n CAN BE IDENTIFIED WITH T HIS IS ONE IMPORTANT REASON BASES ARE USEFUL ! 2 To practice defintions and easy proofs, you should recall the formal definition of isomorphism and verify this! 1 Rn . II. Comparing Coordinates in Different Bases. What if we choose a different basis A = {~u1 , . . . , ~un } for V ? Of course, this gives a different identification of V with Rn , namely V → Rn ~x 7→ [~x]A. Now there are two different ways of writing each vector ~x in V as a column vector, namely [~x]A and [~x]B . How are these columns related? The secret is that there is an n × n matrix S such that S [~x]B = [~x]A . You should think of the matrix S as a machine that takes the B-coordinate column of each vector ~x and converts it (by multiplication) into the A-coordinate column of ~x. This special matrix S is called the change of basis matrix3 from B to A. It can also be denoted SB→A to emphasize that S operates on B-coordinates to produce A-coordinates. But how do we find SB→A ? I recommend carefully memorizing the following definition: Definition II: The change of basis matrix from B to A is the n × n matrix SB→A whose columns are the elements of B expressed in A. That is, SB→A = [[~v1 ]A [~v2 ]A · · · [~vn ]A ]. E XAMPLE II. Another basis for P2 is A = {x + 1, x − 1, 2x2 }. The change of basis matrix from B = {1, x, x2 } to A is   1/2 1/2 0 S = −1/2 1/2 0  . 0 0 1/2   a 2  Consider the element f = a + bx + cx . Then [f ]B = b  and c      1/2 1/2 0 a (a + b)/2 [f ]A = S[f ]B = −1/2 1/2 0   b  = (b − a)/2 . 0 0 1/2 c c/2 This represents the fact that f can also be written as 3 a+b (x 2 + 1) + (b−a) (x 2 − 1) + 2c (2x2 ). “change of coordinates matrix” might be a better name, since it is the coordinates we actually change, but we stick the book’s name. Caution: Do not confuse the roles of the bases A and B in Definition II. Of course, we could also convert from A-coordinates to B-coordinates in the same way, but the matrix that does this, denoted SA→B , is not the same matrix as SB→A . To check your understanding, be sure you see why −1 SA→B = SB→A . T HE SECRET REVEALED : Why is the change of coordinates mapping Rn → Rn [~x]B 7→ [~x]A given by matrix multiplication? Put differently, how do we know that it is a linear transformation? The point is that the change-of-coordinates map is the composition of the maps from Section I: Rn → V → Rn [~x]B 7→ x 7→ [~x]A . Both these identifications are isomorphisms, so their composition is also an isomorphism.4 In particular it is a linear transformation Rn → Rn . As you know, every linear transformation Rn → Rn is given by matrix multiplication. C HECK YOUR UNDERSTANDING II: To better understand Definition II, we compute the columns of S directly. Remember that SB→A “eats” B-coordinates and “spits out” A-coordinates via the matrix multiplication SB→A [~x]B = [~x]A . So in particular, for each basis element ~vj in B, we have SB→A [~vj ]B = [~vj ]A . But what is [~vj ]B ? It is of course the standard unit column vector ~ej ! This says SB→A ~ej = [~vj ]A . But for any matrix, including SB→A , multiplying by ~ej will produce its j-th column. This confirms that j-th column of SB→A is precisely ~vj expressed in A-coordinates. Summary of II: T HE CHANGE OF BASIS MATRIX SB→A IS THE MATRIX WHOSE j- TH COLUMN THE j- TH BASIS ELEMENT OF B. F OR EVERY VECTOR ~ x IN V , WE HAVE SB→A [~x]B = [~x]A . 4 Be sure you can prove this easy fact! IS [~vj ]A , WHERE ~ vj IS III. Using Bases to Represent Transformations. Let T : V → V be a linear transformation.5 The choice of basis B for V identifies both the source and target of T with Rn . Thus T gets identified with a linear transformation Rn → Rn , and hence with a matrix multiplication. This matrix is called the matrix of T with respect to the basis B. It is easy to write down directly: Definition III: The matrix of T in the basis B = {~v1 , . . . , ~vn } is the n × n matrix [T ]B = [[T (~v1)]B [T (~v2 )]B . . . [T (~vn )]B ], whose columns are the vectors T (~vi ) expressed in the basis B. This matrix is helpful for computation. Take any ~x ∈ V . To compute T (~x), we simply convert ~x to the column vector [~x]B , then multiply by the n × n matrix [T ]B . This gives a column vector which represents the element T (x) in the basis B. That is: [T (~x)]B = [T ]B [~x]B . df E XAMPLE III: Consider the linear transformation D : P2 → P2 that sends f to dx . The matrix of D in the basis B = {1, x, x2 } is   0 1 0 [D]B = 0 0 2 . 0 0 0 So differentiation is identified with the matrix multiplication:      b 0 1 0 a [D(a + bx + cx2 )]B = [D]B [a + bx + cx2 ]B = 0 0 2  b  = 2c . 0 c 0 0 0 This represents the fact that C HECK YOUR d (a dx UNDERSTANDING : + bx + cx2 ) = b + 2cx. For [T ]B defined as in Definition III, you should verify the formula [T (~x)]B = [T ]B [~x]B for all vectors ~x in V . A good first step is to check this formula for the elements ~vi in the basis B. Do you see why knowing the formula for the basis elements implies it for all ~x? T Summary of III: L ET V → V BE A LINEAR TRANSFORMATION . T HE CHOICE OF BASIS B IDEN TIFIES BOTH THE SOURCE AND TARGET WITH Rn , AND THEREFORE THE MAPPING T WITH MATRIX MULTIPLICATION BY [T ]B . T HE MATRIX [T ]B IS EASY TO REMEMBER : ITS j- TH COLUMN IS [T (~ vj )]B . IV. Comparing Matrices of a Transformation in Different Bases. How are the matrices of the same transformation in two different bases related? T Fix a linear transformation V → V . The choice of a basis B identifies this map with the matrix multiplication Rn → Rn [~x]B 7→ [T ]B [~x]B . A different basis A gives a different representation of T : Rn → Rn [~x]A 7→ [T ]A [~x]A . What is the relationship between the two n × n matrices [T ]A and [T ]B ? The next theorem answers this question. I recommend memorizing it carefully. Theorem IV: Let SB→A be the change of basis matrix from B to A. Then we have a matrix equality: −1 [T ]B = SB→A [T ]A SB→A or equivalently SB→A [T ]B = [T ]A SB→A . E XAMPLE IV: We continue the example from III. Using the basis A = {x + 1, x − 1, 2x2 } instead of B, the matrix of the differentiation map D is   1/2 1/2 2 [D ]A = −1/2 −1/2 2 . 0 0 0 The matrices [D]A and [D]B are related by −1 [D]B = SB→A [D]A SB→A −1 where SB→A is the change of basis matrix from before. Note that SB→A is the change of basis matrix from A to B so its columns are easy to find:   1 −1 0 −1 SB→A = 1 1 0 . 0 0 2 P ROOF OF T HEOREM IV: We want to prove SB→A [T ]B = [T ]A SB→A . These are two n × n matrices we want to show are equal. We do this column by column, by multiplying each matrix by the standard unit column vector ~ej (on the right). We first compute SB→A [T ]B~ej . Remember that ~ej = [~vj ]B . By definition of the matrix of T with respect to B, [T ]B~ej = [T ]B [~vj ]B = [T (~vj )]B . But now multiplying by the change of basis matrix gives SB→A [T (~vj )]B = [T (~vj )]A . This is the j-th column of the matrix SB→A [T ]B . We now compare this column to the j-th column of [T ]A S, or [T ]A SB→A~ej . Again, since ~ej = [~vj ]B , S~ej = SB→A [~vj ]B = [~vj ]A , by definition of the change of basis matrix S. Multiplying by the matrix of T with respect to A, we will get T (~vj ) expressed in the basis A. That is, [T ]A SB→A~ej = [T ]A [~vj ]A = [T (~vj )]A . This is the j-th column of the matrix [T ]A SB→A . We conclude the matrices SB→A [T ]B = [T ]A SB→A must be equal, since the corresponding columns are equal. D ONE ! C HECK YOUR UNDERSTANDING : It is good exercise to multiply each of the products SB→A [T ]B and [T ]A SB→A . by any column on the right. You should think of the column as representing some vector in the basis B, since both S and [T ]B “act on” B-columns. What happens when each matrix product acts? 6 Summary of Sections I, II, III, and IV on Change of Basis and All That: L ET V BE AN n- DIMENSIONAL VECTOR SPACE . A BASIS B, WE IDENTIFY V WITH COLUMN VECTOR [~ x]B OF B- COORDINATES . I. C HOOSING Rn . E ACH VECTOR ~x IS IDENTIFIED WITH ITS T TRANSFORMATION V → V GETS IDENTIFIED WITH MATRIX MULTIPLICATION BY THE BMATRIX . T HE B- MATRIX IS DENOTED [T ]B . I T IS EASY TO FIND : THE j- TH COLUMN IS T (~ vj ) EXPRESSED IN B. II. T HE III. I F WE INSTEAD CHOOSE A DIFFERENT BASIS A, WE GET A DIFFERENT IDENTIFICATION WITH Rn . T HESE IDENTIFICATIONS ARE RELATED TO EACH OTHER AS FOLLOWS : OF V SB→A [~x]B = [~x]A WHERE SB→A IS THE CHANGE OF BASIS MATRIX FROM B TO A. T HE MATRIX FIND : ITS j- TH COLUMN IS THE j TH ELEMENT OF B EXPRESSED IN A. SB→A IV. L IKEWISE , THE A- MATRIX AND B- MATRIX OF A LINEAR TRANSFORMATION T IS EASY TO ARE RELATED BY SB→A [T ]B = [T ]A SB→A . 6 Here is the answer if you got stuck: We first see what SB→A [T ]B does to columns by first multiplying by [T ]B , then by SB→A . Since [T ]B “eats” B-columns, we think of the column as the B-coordinates [~x]B of some vector ~x in V . The whole point of the B-matrix of T is that the matrix product [T ]B [~x]B represents T (~x) again, but as a column, expressed in the basis B, namely [T (~x)]B . Multiplying now by SB→A changes the B-coordinates to A-coordinates, so that S[T (~x)]B = [T (~x)]A . Putting these together, we have SB→A [T ]B [~x]B = [T (~x)]A . You should similarly think through the meaning of the matrix product [T ]A SB→A [~x]B : what each matrix does to columns and what it means. We get the same result: [T ]A SB→A [~x]B = [T ]A [~x]A = [T (~x)]A . V. Choosing Bases Wisely. Why choose one basis over another? Depending on your problem or data, one basis may be more convenient or illuminating. For example, working with an orthonormal basis is often simpler.7 If A = {~u1 , ~u2 , . . . , ~un } is orthonormal, then it is especially easy to write vectors in the basis A. The coordinates of ~x in {~u1 , ~u2 , . . . , ~un } will be   ~x · ~u1 ~x · ~u2   [~x]A =   ...  . ~x · ~un By contrast, if we try to write ~x in a non-orthnormal basis, we might have to work hard to find the coordinates, perhaps by solving a big linear system. An orthonormal basis is also helpful for computing projections. The change of basis matrix SB→A from any basis B = {~v1 , . . . , ~vn } to an orthonormal basis is likewise easy to find: the ij-th entry will be ~ui · ~vj . And more: if we used Gram Schmidt process to get the new basis, the change of basis matrix will be upper triangular.     3 1 3    E XAMPLE V: Let V be the subspace of R spanned by ~v1 = 0 and ~v2 = 1 . Using the Gram Schmidt 4 1     3/5 4/5 orthogonalization process, we find a new basis {~u1 , ~u2 } = { 0  , √126  5 }, which is orthonormal. 4/5 −3/5 The change of basis matrix from {~v1 , ~v2 } to {~u1 , ~u2 } is easy to find:     5 √7/5 ~v1 · ~u1 ~v2 · ~u1 S= = . ~v1 · ~u2 ~v2 · ~u2 0 26/5   2  We can write the vector ~v = −1 , for example, as a column using {~v1 , ~v2 } or {~u1 , ~u2 }. In the basis 3   1 {~v1 , ~v2 }, this is easy: [~v ]{~v1 ,~v2 } = . In the basis {~u1 , ~u2 }, we can use −1      1/5 √7/5 −6/5 1 √ [~v ]{~u1 ,~u2 } = S[~v ]{~v1 ,~v2 } = = . 0 26/5 −1 − 26/5 This represents the fact ~v can be written as 7 −8 ~u 5 1 + √ − 26 ~u2 . 5 Check it! Note: orthonormality only makes sense in Rn , or some other inner product space. T Another nice basis is an eigenbasis. An eigenbasis for a linear transformation V → V is one in which the matrix is diagonal.8 If the matrix of T is diagonal in a basis B = {~v1 , ~v2 , . . . ~vn }, say   λ1 0 0 . . . 0  0 λ2 0 . . . 0  [T ]B =  , ..   ... ··· . ]0 0 ... 0 λn then this says exactly that T (~vi ) = λi~vi . That is, that the basis B consists of eigenvectors for T [Make sure you see why!] Geometrically, we see T is stretching (or contracting) ~vi by the scalar λi . The directions of stretching are the eigenvectors and the stretching factors λi are eigenvalues. T E XAMPLE V B : Let Rn −→ Rn be multiplication by the n × n matrix A. Suppose that B = (~v1 , . . . , ~vn ) is an eigenbasis for TA , with eigenvalues λ1 , . . . , λn (possibly repeated). Then   λ1 0 0 . . . 0  0 λ2 0 . . . 0  [T ]B =  ..   ... ··· . ]0 0 ... 0 λn is a diagonal matrix. Comparing the matrices of this transformation in the standard basis A = (~e1 , . . . , ~en ) and the eigenbasis B = (~v1 , . . . , ~vn ), we have [T ]B = SA→B [T ]A SB→A .   On the other hand, it is easy to check (do it!) that SB→A = ~v1 . . . ~vn and that SA→B is its inverse. And of course, [T ]A = A [be sure you see why!]. So this statement can be written S −1 AS = D where D is the diagonal matrix formed by the eigenvalues of A and S is the matrix formed by the corresponding (linearly independent) eigenvectors. Writing an expression like this is called diagonalizing the matrix A. Caution: Not every matrix can be diagonalized because not even transformation has an eigenbasis! Summary of V. D EPENDING ON THE PROBLEM WE ARE TRYING TO SOLVE , THE CHOICE OF A CON VENIENT BASIS CAN MAKE THINGS MUCH CLEARER . O RTHONORMAL BASES ( WHICH ALWAYS EXIST ) ARE NICE BECAUSE IT IS EASY TO FIND THE COORDINATES OF ANY VECTOR IN AN ORTHONORMAL BASIS ( BY USING DOT PRODUCT ). A N EIGENBASIS IS NICE FOR ANALYZING A GIVEN TRANSFORMATION T , BECAUSE THE MATRIX OF T IN AN EIGENBASIS WILL BE DIAGONAL . H OWEVER , SOME TRANSFORMATIONS DO NOT HAVE EIGENBASES . 8 Not every linear transformation has an eigenbasis, but as you can imagine, it is much easier to understand a linear transformation that does. VI. Another Use of the change of basis Matrix. The change of basis matrix has another use in the special case where V is a subspace of Rd . Suppose V is a subspace of Rd with basis B = {~v1 , . . . , ~vn }. Each ~vj is a column vector, so we can line them up into a d × n matrix [~v1 . . . ~vn ]. Another basis for V , say A = {~u1 , . . . , ~un }, will give another d × n matrix [~u1 . . . ~un ]. What is the relationship between these two d × n matrices? The following theorem gives the answer. I recommend you memorize it. Theorem VI: Let S be the change of basis matrix from B to A. Then we have a matrix equality [~v1 . . . ~vn ] = [~u1 . . . ~un ]SB→A . On the right side of the equation, we are taking the product of a d × n and an n × n matrix.     1 3 3    E XAMPLE VI: We continue with Example V. The space V ⊂ R has basis {~v1 , ~v2 } = { 0 , 1} and 1 4     3/5 4/5 1    5 }. So we have a matrix equation also {~u1 , ~u2 } = { 0 , √26 4/5 −3/5     3/5 5√426   3 1 5 7/5 5 √  √ 0 1 =  0 . 26 0 26/5 −3 4 1 4/5 5√26 Because the basis {~u1 , ~u2 } is obtained from the {~v1 , ~v2 } by Gram Schmidt, this is the QR factorization. Note that the change of basis matrix R is upper triangular. (We called this S previously, but in this context it is traditional to call it R.) P LEASE N OTE : This is a different use of the change of basis matrix, valid only for subspaces of Rd . P ROOF OF T HEOREM VI: We use “block multiplication.” The “row” [~u1 . . . ~un ] times the j-th column of S   a1j  a2j   is the j-th column of the product. Say the j-th column of S is   ...  . Then the the j-th column of the product anj is a1j ~u1 + a2j ~u2 + · · · + anj ~un . But this is exactly ~vj , by the definition of S! Remember that by definition, the j-th column of S is the coordinate column of ~vj expressed in A. This completes the proof. D ONE !