Miscellaneous Results, Solving Equations, and Generalized Inverses. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

Transcription

1 Miscellaneous Results, Solving Equations, and Generalized Inverses opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

2 Result A.7: Suppose S and T are vector spaces. If S T and dim(s) = dim(t ), then S = T. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

3 Result A.8: Suppose m n A and b satisfy m 1 Then m n A = m n 0 and b = 0. m 1 m 1 Ax + b = 0 x R n. m n m 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

4 Corollary A.1: If m n B and m n C satisfy Bx = Cx x R n, then B = C. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

5 Corollary A.2: Suppose m n A has full column rank. Then AB = AC = B = C. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

6 Lemma A.1: C C = 0 = C = 0. Proof of Lemma A.1: Let c i denote the i th column of C. Then the i th diagonal element of C C is c i c i. C C = 0, c i c i = 0 i = 1,..., n. Now c i c i = 0 i = c i = 0 i = C = 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

7 Another Result on the Rank of a Product Suppose rank( m n A ) = n and rank( C ) = k. Then k l rank( B ) = rank(abc). n k Proof: HW problem. Corollaries: If A is full-column rank, rank(ab) = rank(b). If C is full-row rank, rank(bc) = rank(b). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

8 Solving Equations: Consider a system of linear equations Ax = c, where m n A is a known matrix and c is a known vector. m 1 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

9 We seek a solution vector x that satisfies Ax = c. If m = n so that m n A is a square and if m n A is nonsingular, then A 1 Ax = A 1 c = x = A 1 c is the unique solution to Ax = c. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

10 If m n A is singular or not square, Ax = c may have no solution or infinitely many solutions or a unique solution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

11 A system of equations Ax = c is consistent if there exists a solution x such that Ax = c. A systems of equation Ax = c is inconsistent if Ax c x R n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

12 Result A.9: A system of equations Ax = c is consistent iff c C(A). Provide an example m n A, c Ax = c is inconsistent. m 1 Provide an example m n A, c Ax = c is consistent. m 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

13 Generalized Inverse A matrix G is a generalized inverse (GI) of a matrix A iff AGA = A. Every matrix has at least one GI. We will use A to denote a GI of a matrix A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

14 If A is nonsingular, then A 1 is the unique GI of A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

15 Result A.10: Suppose rank( m n A) = r. If A can be partitioned as A = where rank( r r C) = r, then C r r E (m r) r G = n m D r (n r) F (m r) (n r) [ ] C , is a GI of m n A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

16 Proof of Result A.10: [ ] [ ] C D C 1 0 AG = E F 0 0 [ ] I 0 =. EC 1 0 [ ] [ ] [ ] I 0 C D C D AGA = =. EC 1 0 E F E EC 1 D We need to show EC 1 D = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

17 First note that rank( r r C) = r = rank([c, D]) = r [C, D] has at least r LI columns and at most r LI rows. (rank([c, D]) r and rank([c, D]) r = rank([c, D]) = r.) Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

18 ([ ]) C D Now rank = r = each row of [E, F] is a LC of the rows of E F [C, D]. Thus a matrix K K[C, D] = [E, F] [KC, KD] = [E, F] KC = E, KD = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

19 Now KC = E K = EC 1. Together with KD = F, this implies EC 1 D = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

20 Permutation Matrix A matrix n n P is a permutation matrix if the rows of P are the same as the rows of n n I but not necessarily in the same order. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

21 Example: P = If A = , then PA = Order of rows of PA are permuted relative to order of rows of A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

22 Example: [ ] P = , B = [ ] Then BP = Order of columns of BP are permuted relative to order of columns of B. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

23 A Permutation Matrix is Nonsingular The rows (and columns) of a permutation matrix are the same as those of the identity matrix. Thus, a permutation matrix has full rank and is therefore nonsingular. Furthermore, if P = [p 1,..., p n ] is a permutation matrix, P 1 = P p 0 if i j ip j = 1 if i = j. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

24 Result A.11: Suppose rank( m n A) = r. There exist permutation matrices m m P and Q such n n that [ ] C D PAQ =, E F where rank( r r C) = r. Furthermore, is a GI of A. [ ] C 1 0 G = Q P 0 0 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

25 Proof of Result A.11: Because rank(a) = r, there exists a set of r rows of A that are LI. Let P be a permutation matrix, the first r rows of PA are LI. Let H be the matrix consisting of the first r rows of PA. Then rank(h) = r. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

26 This implies that a set of r columns of H that are LI. Let Q be a permutation matrix the first r columns of HQ are LI. Then the submatrix consisting of the first r rows and first r columns of PAQ has rank r. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

27 [ ] C D Thus we can partition PAQ as, where rank( E F r r C) = r. [ ] [ ] C 1 0 C D By Result A.10, is a GI for PAQ =. 0 0 E F Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

28 [ ] [ ] C D C D PAQ = A = P 1 Q 1 E F E F [ ] C 1 0 AQ PA = 0 0 [ ] [ ] [ C D C = P 1 Q C Q PP 1 E F 0 0 E [ ] [ ] [ ] C D C = P C D Q 1 E F 0 0 E F [ ] C D = P 1 Q 1 = A. E F ] D F Q 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

29 Use Result A.11 to find a GI for A = Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

30 Algorithm for finding a GI of A: 1. Find an r r nonsingular submatrix of A, where r = rank(a). Call this matrix W. 2. Compute (W 1 ). 3. Replace each element of W in A with the corresponding elements of (W 1 ). 4. Replace all other elements in A with zeros. 5. Transpose resulting matrix to get A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

31 Result A.12: Let Ax = c be a consistent system of equations, and let G be any GI of A. Then Gc is a solution to Ax = c, i.e., AGc = c. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

32 Result A.13: Let Ax = c be a consistent system of equations, and let G be any GI of A. Then x is a solution to Ax = c iff z x = Gc + (I GA)z. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

33 Prove that a consistent system of equations Ax = c has a unique solution if and only if A is full-column rank and infinitely many solutions if and only if A is less than full-column rank. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51