Linear Algebra- Final Exam Review 2011

Linear Algebra- Final Exam Review 1. Show that Row(A) ⊥ Null(A). SOLUTION: We can write matrix-vector multiplication in terms of the rows of the matrix A. If A is m × n, then:     Row1 (A)x Row1 (A)  Row2 (A)   Row2 (A)x      Ax =  ..  x =  ..    .  .  Rowm (A) Rowm (A)x Each of these products is the “dot product” of a row of A with the vector x. To show the desired result, let x ∈ Null(A). Then each of the products shown in the equation above must be zero, since Ax = 0, so that x is orthogonal to each row of A. Since the rows form a spanning set for the row space, x is orthogonal to every vector in the row space. 2. Let A be invertible. Show that, if v1 , v2 , v3 are linearly independent vectors, so are Av1 , Av2 , Av3 . NOTE: It should be clear from your answer that you know the definition. SOLUTION: We need to show that the only solution to: c1 Av1 + c2 Av2 + c3 Av3 = 0 is the trivial solution. Factoring out the matrix A, A(c1 v1 + c2 v2 + c3 v3 ) = 0 Think of the form Ax̂ = 0. Since A is invertible, the only solution to this is x̂ = 0, which implies that the only solution to the equation above is the solution to c1 v1 + c2 v2 + c3 v3 = 0 Which is (only) the trivial solution, since the vectors are linearly independent. (NOTE: Notice that if the original vectors had been linearly dependent, this last equation would have non-trivial solutions). 3. Find the line of best first for the data: x 0 1 2 3 y 1 1 2 2 Let A be the matrix formed by a column from x column of ones, then we form the normal equations AT Aβ = AT y and solve:     14 6 11 T T A Ax = A y x̂ = 6 4 6 The solution is x̂ = (AT A)−1 AT y = 9/10. 1 [4, 9]T , 10 1 so the slope is 2/5 and the intercept is 4. Let v1 , . . . , vp be orthonormal. If x = c1 v1 + c2 v2 + · · · + cp vp then show that kxk2 = |c1 |2 + · · · + |cp |2 . (Hint: Write the norm squared as the dot product). SOLUTION: Compute x · x, and use the property that the vectors vi are orthonormal: (c1 v1 + c2 v2 + · · · + cp vp ) · (c1 v1 + c2 v2 + · · · + cp vp ) = Since all dot products of the form ci ck vi ·vk = 0 for i 6= k, then the dot product simplifies to: c21 v1 · v1 + c22 v2 · v2 + · · · c2p vp · vp And since the vectors are normalized, this gives the result: kxk2 = c21 + · · · + c2p (Don’t need the magnitudes here, since we’re working with real numbers). 5. Short answer: (a) If ku + vk2 = kuk2 + kvk2 , then u, v are orthogonal. SOLUTION: True- This is the Pythagorean Theorem. (b) Let H be the subset of vectors in IR3 consisting of those vectors whose first element is the sum of the second and third elements. Is H a subspace? SOLUTION: One way of showing that a subset is a subspace is to show that the subspace can be represented by the span of some set of vectors. In this case,       a+b 1 1      a =a 1 +b 0  b 0 1 Because H is the span of the given vectors, it is a subspace. (c) Explain why the image of a linear transformation T : V → W is a subspace of W SOLUTION: Maybe “Prove” would have been better than “Explain”, since we want to go through the three parts: i. 0 ∈ T (V ) since 0 ∈ V and T (0) = 0. ii. Let u, v be in T (V ). Then there is an x, y in V so that T (x) = u and T (y) = v. Since V is a subspace, x + y ∈ V , and therefore T (x + y) = T (x) + T (y) = u + v so that u + v ∈ T (V ). iii. Let u ∈ T (V ). Show that cu ∈ T (V ) for all scalars c. If u ∈ T (V ), there is an x in V so that T (x) = u. Since V is a subspace, cu ∈ V , and T (cu) ∈ T (V ). By linearity, this means cT (u) ∈ T (V ). (OK, that probably should not have been in the short answer section) 2  1  (d) Is the following matrix diagonalizable? Explain. A = 0 0 SOLUTION: Yes. The eigenvalues are all distinct, so the tors are linearly independent.  2 3 5 8  0 13 corresponding eigenvec- (e) If the column space of an 8 × 4 matrix A is 3 dimensional, give the dimensions of the other three fundamental subspaces. Given these numbers, is it possible that the mapping x → Ax is one to one? onto? SOLUTION: If the column space is 3-d, so is the row space. Therefore the null space (as a subspace of IR4 ) is 1 dimensional and the null space of AT is 5 dimensional. Since the null space has more than the zero vector, Ax = 0 has non-trivial solutions, so the matrix mapping will not be 1-1. Since the column space is a three dimensional subspace of IR8 , the mapping cannot be onto. (f) Let Q be a matrix with 3 non-zero orthonormal columns. i. Can Q have fewer than 2 rows? Why or why not? SOLUTION: No. Since the columns are orthogonal, they are linearly independent, so the matrix must have 3 pivot columns, and so we must have (at least) 3 pivot rows. ii. True or False: If Q is m × n with m > n, then QQT = I. SOLUTION: False. While QT Q = I, QQT is a projection to the column space of Q in the sense that QQT x = ProjCol(Q) (x)   1 1 2 2 6. Find a basis for the null space, row space and column space of A, if A =  2 2 5 5  0 0 3 3 The basis for the column space is the set containing the first and third columns of A. A basis for the row space is the set of vectors [1, 1, 0, 0]T , [0, 0, 1, 1]T . A basis for the null space of A is [−1, 1, 0, 0]T , [0, 0, −1, 1]T . 7. Find an orthonormal basis for W = Span {x1 , x2 , x3 } using wait until the very end to normalize all vectors at once):      0 0  0   1       x1 =   1  , x2 =  1  , x3 =  1 2 Gram-Schmidt (you might  1 1  , 1  1 SOLUTION: Using Gram Schmidt (before normalization, which is OK if doing by hand), we get       0 3 0  2   1   0       v1 =   1  , v2 =  −1  , v3 =  1  1 −1 1 3 8. Let Pn be the vector space of polynomials of degree n or less. Let W1 be the subset of Pn consisting of p(t) so that p(0)p(1) = 0. Let W2 be the subset of Pn consisting of p(t) so that p(2) = 0. Which of the two is a subspace of Pn ? SOLUTION: We check the properties• Is 0 in the subspace? For both W1 and W2 , the zero polynomial satisfies both. • Is the subspace closed under addition? For W1 , let p(t) be a polynomial such that p(0)p(1) = 0, and let h(t) be another polynomial with that property, h(0)h(1) = 0. Does that imply that g(t) = p(t) + h(t) has the desired property? g(0)g(1) = (p(0)+h(0))(p(1)+h(1)) = p(0)p(1)+p(0)h(1)+h(0)p(1)+h(0)h(1) = p(0)h(1)+h(0)p(1) For example, if h(0) = 1, h(1) = 0, p(0) = 0, p(1) = 1, then this quantity is not zero. Therefore, W1 is not closed under addition. However, you can check that W2 is closed under addition: Let p(t), h(t) be two functions in W2 . Then g(t) = p(t) + h(t) satisfies the property that g(2) = p(2) + h(2) = 0 + 0 = 0 • Similarly, W2 is closed under scalar multiplication- If g(t) = cp(t), then g(2) = cp(2) = c · 0 = 0. Therefore, W2 is a subspace, and W1 is not. 9. For each of the following matrices, find the characteristic equation, a basis for each eigenspace:      1 0 7 2 3 −1  A= B= C= 0 2 −4 1 1 3 1 0 the eigenvalues and  1 0  1 SOLUTION: For matrix A, λ = 3, 5. Eigenvectors are [1, 1]T and [2, 1]T , respectively. For matrix B, for λ = 3+i, an eigenvector is [1, i]T . The other eigenvalue and eigenvector are the complex conjugates. For matrix C, expand along the 2d row. λ = 2 is a double eigenvalue with eigenvectors [0, 1, 0]T and [1, 0, 1]T . The third eigenvalue is λ = 0 with eigenvector [−1, 0, 1]T .   p(−1) 10. Define T : P2 → IR3 by: T (p) =  p(0)  p(1) (a) Find the image under T of p(t) = 5 + 3t. SOLUTION: [2, 5, 8]T 4 (b) Show that T is a linear transformation. SOLUTION: We show it using the definition. i. Show that T (p + q) = T (p) + T (q):       p(−1) + q(−1) p(−1) q(−1) T (p + q) =  p(0) + q(0)  =  p(0)  +  q(0)  = T (p) + T (q) p(1) + q(1) p(1) q(1) ii. Show that T (cp) = cT (p) for all scalars c.     cp(−1) p(−1) T (cp) =  cp(0)  = c  p(0)  = cT (p) cp(1) p(1) (c) Find the kernel of T . Does your answer imply that T is 1 − 1? Onto? (Review the meaning of these words: kernel, one-to-one, onto) SOLUTION: Since the kernel is the set of elements in the domain that map to zero, let’s see what what the action of T is on an arbitrary polynomial. An arbitrary vector in P2 is: p(t) = at2 + bt + c, and:   a−b+c  c T (at2 + bt + c) =  a+b+c For this to be the zero vector, c = 0. Then a − b = 0 and a + b = 0, so a = 0, b = 0. Therefore, the only vector mapped to zero is the zero vector. Side Remark: Recall that for any linear function T , if we are solving T (x) = y, then the solution can be written as x = xp + xh , where xp is the particular solution (it solves T (xp ) = y), and T (xh ) = 0 (we said xh is the homogeneous part of the solution). So the equation T (x) = y has at most one solution iff the kernel is only the zero vector (if T was realized as a matrix, we get our familiar setting). Therefore, T is 1 − 1. The mapping T will also be onto (see the next part). (d) Find the matrix for T relative to the basis {1, t, t2 } for P2 . (This means that the matrix will act on the coordinates of p). You can verify that the matrix representation for T (using the basis in the order at2 + bt + c) is:   1 −1 1  0 0 1  1 1 1 since multiplying this matrix by [a, b, c]T will give the previous result. From this we see that the matrix is invertible, so the mapping is both 1-1 and onto (as a mapping from IR3 to IR3 ). 5 11. Let v be a vector in IRn so that kvk = 1, and let Q = I − 2vvT . Show (by direct computation) that Q2 = I. SOLUTION: This problem is to practice matrix algebra: Q2 = (I −2vvT )(I −2vvT ) = I 2 −2IvvT −2vvT I +4vvT vvT = I −4vvT +4v(1)vT = I 12. Let A be m × n and suppose there is a matrix C so that AC = Im . Show that the equation Ax = b is consistent for every b. Hint: Consider ACb. SOLUTION: Using the hint, we see that ACb = b. Therefore, given an arbitrary vector b, the solution to Ax = b is x = Cb. 13. If B has linearly dependent columns, show that AB has linearly dependent columns. Hint: Consider the null space. SOLUTION: If B has linearly dependent columns, then the equation Bx = 0 has nontrivial solutions. Therefore, the equation ABx = 0 has (the same) non-trivial solutions, and the columns of AB must be linearly dependent. 14. If λ is an eigenvalue of A, then show that it is an eigenvalue of AT . SOLUTION: Use the properties of determinants. Given |A − λI| = |(A − λI)T | = |AT − λI T | = |AT − λI| the solutions to |A − λI| = 0 and |AT − λI| = 0 are exactly the same.     2 −2 15. Let u = and v = , Let S be the parallelogram with vertices at 0, u, v, and 1 1 u + v. Compute the area of S. SOLUTION: The area of the parallelogram formed by two vectors in IR2 is the determinant of the matrix whose columns are those vectors. In this case, that would be 4.       a b c a + 2g b + 2h c + 2i g h i 16. Let A =  d e f , B =  d + 3g e + 3h f + 3i , and C =  2d 2e 2f . g h i g h i a b c If det(A) = 5, find det(B), det(C), det(BC). SOLUTION: This question reviews the relationship between the determinant and row operations. The determinant of B is 5. The determinant of C is −10. The determinant of BC is −50. 17. Let 1, t be two vectors in C[−1, 1]. Find the length between the two vectors and the cosine of the angle between them using the standard inner product (the integral). Find the orthogonal projection of t2 onto the set spanned by {1, t}. SOLUTION: 6 • The length between the vectors is: s sZ 1 p −1 (1 − t)2 dt = h(1 − t), (1 − t)i = (1 − t)3 3 −1 1 r = −1 8 3 18. Define an isomorphism: A one-to-one linear transformation between vector spaces (see p. 251) NOTE: An isomorphism was the critical piece to understanding when two vector spaces had the same “form”- For example, a plane through the origin in IR3 and the plane IR2 are not equal, but they are isomorphic; the isomorphism takes a point of the plane and returns its coordinates- That is, the plane in IR3 is the span of two vectors in IR3 , so every point on the plane is a linear combination of those two. The point in IR2 that we refer to is the ordered pair of weights from the linear combination. As another example, if the plan is spanned by u and v in vector space V , and x is on the plane so that x = c1 u + c2 v, then the isomorphism takes x ∈ V and gives (c1 , c2 ) ∈ IR2 . 19. Let  B= 1 −3   , 2 −8    −3 , 7 Find at least two B−coordinate vectors for x = [1, 1]T . SOLUTION: Row reduce to find x as a linear combination of the vectors in B:     1 2 −3 1 1 0 −5 5 → −3 −8 7 1 1 1 1 −2 If we label the weights as c1 , c2 and c3 , then c1 = 5 + 5t c2 = −2 − t c3 = t Therefore, we could form x using the weights (5, −2, 0), or (10, −3, 1) or any combination given. NOTE: The columns do NOT form a basis for IR2 , but they do form a spanning set for IR2 . If the columns formed a basis, the weights for the linear combination would be unique (no free variables), but in this case, the expansion of x in this basis was not unique. 20. Let U , V be orthogonal matrices. Show that U V is an orthogonal matrix. SOLUTION: This question deals with the definition of an orthogonal matrix: A square matrix such that U T = U −1 . First, if the product U V is defined, then U and V are both n × n for some n. Secondly, since U, V are each invertible, then so is U V . Furthermore, (U V )−1 = V −1 U −1 = V T U T = (U V )T Therefore, U V is an orthogonal matrix. 7 21. Find the volume of the parallelepiped formed by 0, a,b, c, a + b, c + b, c + a, and the sum of all three.       0 −2 −1 a= 3  b= 0  c= 0  −1 1 1 SOLUTION: The volume is the absolute value of the determinant. In this case, expand along the second row, and get 3. 22. Let T be a one-to-one linear transformation for a vector space V into IRn . Show that for u, v in V , the formula: hu, vi = T (u) · T (v) defines an inner product on V . SOLUTION: This was a homework problem from 6.7. We want to check the properties of the inner product, which are: (i) Symmetry, (ii) and (iii) Linear in the first coordinate, and (iv) Inner product of a vector with itself is non-negative (and the special case of 0). (a) hu, vi = T (u) · T (v) = T (v) · T (u) = hu, vi, where the second equality is true because the regular dot product is symmetric. (b) hu + w, vi = T (u + w) · T (v) = (T (u) + T (w)) · T (v) = T (u) · T (v) + T (w) · T (v) (c) hcu, vi = T (cu) · T (v) = cT (u) · T (v) (d) hu, ui = T (u) · T (u) = kT (u)k2 The dot product of a vector with itself is always non-negative. Furthermore, by the same equation, if hu, ui = 0, then u must be the zero vector. 23. Describe all least squares solutions to x+y =2 x+y =4 SOLUTION: Interesting to think about- In the plane, these are two parallel lines (each has a slope of −1, one has an intercept at 2, the other at 4). Using linear algebra, we have      1 1 x 2 = 1 1 y 4 ⇔ Ax = b We cannot use the normal equations, because A does not have full rank (a rank of 2). However, if we project b into the column space of A (which is the span of [1, 1]T ), then 8 we can solve the sytem (and in fact, we’ll have an infinite number of solutions since there will be a free variable):     2+4 1 3 ProjCol(A) (b) = = 1 3 1+1 Now solve the system Ax̂ = [3, 3]T , which is:     1 1 3 1 1 3 → 1 1 3 0 0 0 If we let the free variable be ŷ = t, then x̂ = 3 − t. Notice that this set of points represents the line ŷ = −x̂ + 3, which is the line right down the middle between the other two lines! 24. Let u = [5, −6, 7]T . Let W be the set of all vectors orthogonal to u. (i) Geometrically, what is W ? (ii) Find the projection of x = [1, 2, 3]T onto W . (iii) Find the distance from the vector x = [1, 2, 3]T to the subspace W . SOLUTIONS: • W is the plane in IR3 going through the origin whose normal vector (in the sense of Calc 3) is u. • We can write x = x̂ + z, where x̂ is the projection onto u, then z will be the desired vector in W :         7/11 0.6364 0.36 5 x · u 5 − 12 + 21  −6  =  −42/55  ≈  −0.7636  ⇒ z ≈  2.76  u= x̂ = u·u 25 + 36 + 49 49/55 0.8909 2.11 7 • The distance is then kx − zk = kx̂k ≈ 1.33 9