18.06 Linear Algebra, Problem Set 6 Solutions

18.06 Problem Set 6 Solutions Total: 100 points Section 4.3. Problem 4: Write down E = �Ax − b�2 as a sum of four squares– the last one is (C + 4D − 20)2 . Find the derivative equations ∂E/∂C = 0 and ∂E/∂D = 0. Divide by 2 to obtain the normal equations AT Ax � = AT b. Solution (4 points) Observe ⎛ 1 ⎜1 A=⎜ ⎝1 1 ⎞ ⎛ ⎞ 0 0 � � ⎜8⎟ 1⎟ ⎟ , b = ⎜ ⎟ , and define x = C . ⎝8⎠ D 3⎠ 4 20 Then ⎛ ⎞ C ⎜ C +D−8 ⎟ ⎟ Ax − b = ⎜ ⎝ C + 3D − 8 ⎠ , C + 4D − 20 and �Ax − b�2 = C 2 + (C + D − 8)2 + (C + 3D − 8)2 + (C + 4D − 20)2 . The partial derivatives are ∂E/∂C = 2C + 2(C + D − 8) + 2(C + 3D − 8) + 2(C + 4D − 20) = 8C + 16D − 72, ∂E/∂D = 2(C + D − 8) + 6(C + 3D − 8) + 8(C + 4D − 20) = 16C + 52D − 224. On the other hand, T A A= � � � � 4 8 36 T , A b= . 8 26 112 Thus, AT Ax = AT b yields the equations 4C + 8D = 36, 8C + 26D = 112. Multiply­ ing by 2 and looking back, we see that these are precisely the equations ∂E/∂C = 0 and ∂E/∂D = 0. Section 4.3. Problem 7: Find the closest line b = Dt, through the origin, to the same four points. An exact fit would solve D · 0 = 0, D · 1 = 8, D · 3 = 8, D · 4 = 20. 1 Find the 4 by 1 matrix A and solve AT Ax � = AT b. Redraw figure 4.9a showing the best line b = Dt and the e’s. Solution (4 points) Observe ⎛ ⎞ ⎛ ⎞ 0 0 ⎜1⎟ ⎜8⎟ T T ⎟ ⎜ ⎟ A=⎜ ⎝3⎠ , b = ⎝ 8 ⎠ , A A = (26), A b = (112). 4 20 Thus, solving AT Ax = AT b, we arrive at D = 56/13. Here is the diagram analogous to figure 4.9a. Section 4.3. Problem 9: Form the closest parabola b = C + Dt + Et2 to the same four points, and write down the unsolvable equations Ax = b in three unknowns 2 x = (C, D, E). Set up the three normal equations AT Ax � = AT b (solution not required). In figure 4.9a you are now fitting a parabola to 4 points–what is happening in Figure 4.9b? Solution (4 points) Note ⎛ 1 ⎜1 A=⎜ ⎝1 1 ⎞ ⎛ ⎞ ⎛ ⎞ 0 0 0 C ⎜ ⎟ 1 1⎟ ⎟ , b = ⎜ 8 ⎟ , x = ⎝D⎠ . ⎝8⎠ 3 9⎠ E 4 16 20 Then multiplying out Ax = b yields the equations C = 0, C + D + E = 8, C + 3D + 9E = 8, C + 4D + 16E = 20. Take the sum of the fourth equation and twice the second equation and subtract the sum of the first equation and two times the third equation. One gets 0 = 20. Hence, these equations are not simultaneously solvable. Computing, we get ⎛ ⎞ ⎛ ⎞ 4 8 26 36 AT A = ⎝ 8 26 92 ⎠ , AT b = ⎝112⎠ . 26 92 338 400 Thus, solving this problem is the same as solving the system ⎛ ⎞⎛ ⎞ ⎛ ⎞ 4 8 26 C 36 ⎝ 8 26 92 ⎠ ⎝D⎠ = ⎝112⎠ . 26 92 338 E 400 The analogue of diagram 4.9(b) in this case would show three vectors a1 = (1, 1, 1, 1), a2 = (0, 1, 3, 4), a3 = (0, 1, 9, 16) spanning a three dimensional vector subspace of R4 . It would also show the vector b = (0, 8, 8, 20), and the projection p = Ca1 +Da2 +Ea3 of b into the three dimensional subspace. Section 4.3. Problem 26: Find the plane that gives the best fit to the 4 values b = (0, 1, 3, 4) at the corners (1, 0) and (0, 1) and (−1, 0) and (0, −1) of a square. The equations C + Dx + Ey = b at those 4 points are Ax = b with 3 unknowns x = (C, D, E). What is A? At the center (0, 0) of the square, show that C +Dx+Ey is the average of the b’s. Solution (12 points) 3 Note ⎛ ⎞ 1 1 0 ⎜1 0 1⎟ ⎟ A=⎜ ⎝1 −1 0 ⎠ . 1 0 −1 To find the best fit plane, we must find x such that Ax − b is in the left nullspace of A. Observe ⎛ ⎞ C +D ⎜C + E − 1⎟ ⎟ Ax − b = ⎜ ⎝C − D − 3⎠ . C −E−4 Computing, we find that the first entry of AT (Ax − b) is 4C − 8. This is zero when C = 2, the average of the entries of b. Plugging in the point (0, 0), we get C + D(0) + E(0) = C = 2 as desired. Section 4.3. Problem 29: Usually there will be exactly one hyperplane in Rn that contains the n given points x = 0, a1 , . . . , an−1 . (Example for n=3: There will be exactly one plane containing 0, a1 , a2 unless .) What is the test to have n exactly one hyperplane in R ? Solution (12 points) The sentence in paranthesis can be completed a couple of different ways. One could write “There will be exactly one plane containing 0, a1 , a2 unless these three points are colinear”. Another acceptable answer is “There will be exactly one plane containing 0, a1 , a2 unless the vectors a1 and a2 are linearly dependent”. In general, 0, a1 , . . . , an−1 will be contained in an unique hyperplane unless all of the points 0, a1 , . . . , an−1 are contained in an n − 2 dimensional subspace. Said another way, 0, a1 , . . . , an−1 will be contained in an unique hyperplane unless the vectors a1 , . . . , an−1 are linearly dependent. Section 4.4. Problem 10: Orthonormal vectors are automatically linearly inde­ pendent. (a) Vector proof: When c1 q1 + c2 q2 + c3 q3 = 0, what dot product leads to c1 = 0? Similarly c2 = 0 and c3 = 0. Thus, the q’s are independent. (b) Matrix proof: Show that Qx = 0 leads to x = 0. Since Q may be rectangular, you can use QT but not Q−1 . 4 Solution (4 points) For part (a): Dotting the expression c1 q1 + c2 q2 + c3 q3 with q1 , we get c1 = 0 since q1 · q1 = 1, q1 · q2 = q1 · q3 = 0. Similarly, dotting the expression with q2 yields c2 = 0 and dotting the expression with q3 yields c3 = 0. Thus, {q1 , q2 , q3 } is a linearly independent set. For part (b): Let Q be the matrix whose columns are q1 , q2 , q3 . Since Q has orthonormal columns, QT Q is the three by three identity matrix. Now, multiplying the equation Qx = 0 on the left by QT yields x = 0. Thus, the nullspace of Q is the zero vector and its columns are linearly independent. Section 4.4. Problem 18: Find the orthonormal vectors A, B, C by GramSchmidt from a, b, c: a = (1, −1, 0, 0) b = (0, 1, −1, 0) c = (0, 0, 1, −1). Show {A, B, C} and {a, b, c} are bases for the space of vectors perpendicular to d = (1, 1, 1, 1). Solution (4 points) We apply Gram-Schmidt to a, b, c. We have � � a 1 −1 A= = √ , √ , 0, 0 . �a� 2 2 Next, ( 1 , 1 , −1, 0) b − (b · A)A B= = 21 21 = �b − (b · A)A� �( 2 , 2 , −1, 0)� � 1 1 √ , √ ,− 6 6 � � 2 ,0 . 3 Finally, c − (c · A)A − (c · B)B C= = �c − (c · A)A − (c · B)B� � √ � 1 1 1 3 √ , √ , √ ,− . 2 2 3 2 3 2 3 Note that {a, b, c} is a linearly independent set. Indeed, x1 a + x2 b + x3 c = (x1 , x2 − x1 , x3 − x2 , −x3 ) = (0, 0, 0, 0) implies that x1 = x2 = x3 = 0. We check a · (1, 1, 1, 1) = b · (1, 1, 1, 1) = c · (1, 1, 1, 1) = 0. Hence, all three vectors are in the nullspace of (1, 1, 1, 1). Moreover, the dimension of the column space of the transpose and the dimension of the nullspace sum to the dimension of R4 . Thus, the space of vectors perpendicular to (1, 1, 1, 1) is three dimensional. Since {a, b, c} is a linearly independent set in this space, it is a basis. 5 Since {A, B, C} is an orthonormal set, it is a linearly independent set by problem 10. Thus, it must also span the space of vectors perpendicular to (1, 1, 1, 1), and it is also a basis of this space. Section 4.4. Problem 35: Factor [Q, R] = qr(A) for A = eye(4)−diag([111], −1). You are orthogonalizing the columns (1, −1, 0, 0), (0, 1, −1, 0), (0, 0, 1, −1), and (0, 0, 0, 1) of A. Can you scale the orthogonal columns of Q to get nice integer components? Solution (12 points) Here is a copy of the matlab code √ √ Note that scaling the first column by 2, the second column by 6, the third √ column by 2 3, and the fourth column by 2 yields ⎛ ⎞ 1 1 1 1 ⎜−1 1 1 1⎟ ⎜ ⎟ ⎝ 0 −2 1 1⎠ . 0 0 −3 1 6 Section 4.4. Problem 36: If A is m by n, qr(A) produces a square A and zeroes below R: The factors from MATLAB are (m by m)(m by n) � � � � R A = Q1 Q2 . 0 The n columns of Q1 are an orthonormal basis for which fundamental subspace? The m−n columns of Q2 are an orthonormal basis for which fundamental subspace? Solution (12 points) The n columns of Q1 form an orthonormal basis for the column space of A. The m−n columns of Q2 form an orthonormal basis for the left nullspace of A. Section 5.1. Problem 10: If the entries in every row of A add to zero, solve Ax = 0 to prove det A = 0. If those entries add to one, show that det(A − I) = 0. Does this mean det A = I? Solution (4 points) If x = (1, 1, . . . , 1), then the components of Ax are the sums of the rows of A. Thus, Ax = 0. Since A has non-zero nullspace, it is not invertible and det A = 0. If the entries in every row of A sum to one, then the entries in every row of A − I sum to zero. Hence, A − I has a non-zero nullspace and det(A − I) = 0. This does not mean that det A = I. For example if � � 0 1 A= , 1 0 then the entries of every row of A sum to one. However, det A = −1. Section 5.1. Problem 18: monde determinant” is ⎡ 1 ⎣ det 1 1 Solution ⎛ 1 det ⎝1 1 Use row operations to show that the 3 by 3 “Vander­ ⎤ a a2 b b2 ⎦ = (b − a)(c − a)(c − b). c c2 (4 points) Doing elimination, we get ⎞ ⎛ ⎞ ⎛ ⎞ a a2 1 a a2 1 a a2 b b2 ⎠ = det ⎝0 b − a b2 − a2 ⎠ = (b − a) det ⎝0 1 b+a ⎠= 2 2 2 0 c−a c −a 0 c − a c 2 − a2 c c 7 ⎛ ⎞ 1 a a2 ⎠ = (b − a)(c − a)(c − b). b+a = (b − a) det ⎝0 1 0 0 (c − a)(c − b) Section 5.1. Problem 31: (MATLAB) The Hilbert matrix hilb(n) has i, j entry equal to 1/(i + j − 1). Print the determinants of hilb(1), hilb(2), . . . , hilb(10). Hilbert matrices are hard to work with! What are the pivots of hilb(5)? Solution (12 points) Here is the relevant matlab code. Note that the determinants of the 4th through 10th Hilbert matrices differ from zero by less than one ten thousandth. The pivots of the fifth Hilbert matrix are 1, .0833, −.0056, .0007, .0000 up to four significant figures. Thus, we see that there is even a pivot of the fifth Hilbert matrix that differs from zero by less than one ten thousandth. 8 Section 5.1. Problem 32: (MATLAB) What is the typical determinant (exper­ imentally) of rand(n) and randn(n) for n = 50, 100, 200, 400? (And what does “Inf” mean in MATLAB?) Solution (12 points) This matlab code computes some examples for rand. As you can see, rand(50) is around 105 , rand(100) is around 1025 , rand(200) is around 1079 , and rand(400) is around 10219 . This matlab code computes some examples for randn. 9 Note that randn(50) is around 1031 , randn(100) is around 1078 , randn(200) is around 10186 , and randn(400) is just too big for matlab. 10