Factorization into A=LU

Factorization into A = LU One goal of today’s lecture is to understand Gaussian elimination in terms of matrices; to find a matrix L such that A = LU. We start with some useful facts about matrix multiplication. Inverse of a product The inverse of a matrix product AB is B−1 A−1 . Transpose of a product We obtain the transpose of a matrix by exchanging its rows and columns. In other words, the entry in row i column j of A is the entry in row j column i of AT . The transpose of a matrix product AB is B T A T . For any invertible matrix � �T A, the inverse of A T is A−1 . A = LU We’ve seen how to use elimination to convert a suitable matrix A into an upper triangular matrix U. This leads to the factorization A = LU, which is very helpful in understanding the matrix A. Recall that (when there are no row exchanges) we can describe the elimi­ nation of the entries of matrix A in terms of multiplication by a succession of elimination matrices Eij , so that A → E21 A → E31 E21 A → · · · → U. In the two by two case this looks like: E21 � 1 0 −4 1 � A � 2 8 1 7 � � = U � 2 1 . 0 3 We can convert this to a factorization A = LU by “canceling” the matrix E21 ; −1 −1 E21 A = E21 U. multiply by its inverse to get E21 A � 2 8 1 7 � � = L � 1 0 4 1 � U � 2 1 . 0 3 The matrix U is upper triangular with pivots on the diagonal. The matrix L is lower triangular and has ones on the diagonal. Sometimes we will also want to factor out a diagonal matrix whose entries are the pivots: A � 2 8 L 1 7 � � = 1 0 4 1 D � � 2 0 0 3 � � U� � 1 1/2 . 0 1 −1 −1 −1 In the three dimensional case, if E32 E31 E21 A = U then A = E21 E31 E32 U = LU. For example, suppose E31 is the identity matrix and E32 and E21 are as shown below: ⎡ 1 ⎣ 0 0 E32 ⎤ 0 0 1 0 ⎦ −5 1 E21 ⎤ 1 0 0 ⎣ −2 1 0 ⎦ 0 0 1 E ⎡ ⎡ = 1 ⎣ −2 10 ⎤ 0 0 1 0 ⎦. −5 1 The 10 in the lower left corner arises because we subtracted twice the first row from the second row, then subtracted five times the new second row from the third. The factorization A = LU is preferable to the statement EA = U because the combination of row subtractions does not have the effect on L that it did on −1 −1 E. Here L = E−1 = E21 E32 : −1 E21 ⎤ 1 0 0 ⎣ 2 1 0 ⎦ 0 0 1 ⎡ −1 E32 ⎤ 1 0 0 ⎣ 0 1 0 ⎦ 0 5 1 ⎡ L ⎤ 1 0 0 ⎣ 2 1 0 ⎦. 0 5 1 ⎡ = Notice the 0 in row three column one of L = E−1 , where E had a 10. If there are no row exchanges, the multipliers from the elimination matrices are copied directly into L. How expensive is elimination? Some applications require inverting very large matrices. This is done using a computer, of course. How hard will the computer have to work? How long will it take? When using elimination to find the factorization A = LU we just saw that we can build L as we go by keeping track of row subtractions. We have to remember L and (the matrix which will become) U; we don’t have to store A or Eij in the computer’s memory. How many operations does the computer perform during the elimination process for an n × n matrix? A typical operation is to multiply one row and then subtract it from another, which requires on the order of n operations. There are n rows, so the total number of operations used in eliminating en­ tries in the first column is about n2 . The second row and column are shorter; that product costs about (n − 1)2 operations, and so on. The total number of operations needed to factor A into LU is on the order of n3 : 12 + 22 + · · · + (n − 1)2 + n2 = n � n i =1 0 ∑ i2 ≈ x2 dx = 1 3 n . 3 While we’re factoring A we’re also operating on b. That costs about n2 operations, which is hardly worth counting compared to 13 n3 . Row exchanges What if there are row exchanges? In other words, what happens if there’s a zero in a pivot position? To swap two rows, we multiply on the left by a permutation matrix. For example, ⎡ ⎤ 0 1 0 P12 = ⎣ 1 0 0 ⎦ 0 0 1 swaps the first and second rows of a 3 × 3 matrix. The inverse of any permu­ tation matrix P is P−1 = P T . There are n! different ways to permute the rows of an n × n matrix (in­ cluding the permutation that leaves all rows fixed) so there are n! permutation matrices. These matrices form a multiplicative group.