Miscellaneous Results, Solving Equations, and Generalized Inverses opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 51
Result A.7: Suppose S and T are vector spaces. If S T and dim(s) = dim(t ), then S = T. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 2 / 51
Result A.8: Suppose m n A and b satisfy m 1 Then m n A = m n 0 and b = 0. m 1 m 1 Ax + b = 0 x R n. m n m 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 4 / 51
Corollary A.1: If m n B and m n C satisfy Bx = Cx x R n, then B = C. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 6 / 51
Corollary A.2: Suppose m n A has full column rank. Then AB = AC = B = C. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 8 / 51
Lemma A.1: C C = 0 = C = 0. Proof of Lemma A.1: Let c i denote the i th column of C. Then the i th diagonal element of C C is c i c i. C C = 0, c i c i = 0 i = 1,..., n. Now c i c i = 0 i = c i = 0 i = C = 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 10 / 51
Another Result on the Rank of a Product Suppose rank( m n A ) = n and rank( C ) = k. Then k l rank( B ) = rank(abc). n k Proof: HW problem. Corollaries: If A is full-column rank, rank(ab) = rank(b). If C is full-row rank, rank(bc) = rank(b). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 11 / 51
Solving Equations: Consider a system of linear equations Ax = c, where m n A is a known matrix and c is a known vector. m 1 opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 12 / 51
We seek a solution vector x that satisfies Ax = c. If m = n so that m n A is a square and if m n A is nonsingular, then A 1 Ax = A 1 c = x = A 1 c is the unique solution to Ax = c. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 13 / 51
If m n A is singular or not square, Ax = c may have no solution or infinitely many solutions or a unique solution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 14 / 51
A system of equations Ax = c is consistent if there exists a solution x such that Ax = c. A systems of equation Ax = c is inconsistent if Ax c x R n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 15 / 51
Result A.9: A system of equations Ax = c is consistent iff c C(A). Provide an example m n A, c Ax = c is inconsistent. m 1 Provide an example m n A, c Ax = c is consistent. m 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 16 / 51
Generalized Inverse A matrix G is a generalized inverse (GI) of a matrix A iff AGA = A. Every matrix has at least one GI. We will use A to denote a GI of a matrix A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 19 / 51
If A is nonsingular, then A 1 is the unique GI of A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 20 / 51
Result A.10: Suppose rank( m n A) = r. If A can be partitioned as A = where rank( r r C) = r, then C r r E (m r) r G = n m D r (n r) F (m r) (n r) [ ] C 1 0 0 0, is a GI of m n A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 22 / 51
Proof of Result A.10: [ ] [ ] C D C 1 0 AG = E F 0 0 [ ] I 0 =. EC 1 0 [ ] [ ] [ ] I 0 C D C D AGA = =. EC 1 0 E F E EC 1 D We need to show EC 1 D = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 23 / 51
First note that rank( r r C) = r = rank([c, D]) = r [C, D] has at least r LI columns and at most r LI rows. (rank([c, D]) r and rank([c, D]) r = rank([c, D]) = r.) Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 24 / 51
([ ]) C D Now rank = r = each row of [E, F] is a LC of the rows of E F [C, D]. Thus a matrix K K[C, D] = [E, F] [KC, KD] = [E, F] KC = E, KD = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 25 / 51
Now KC = E K = EC 1. Together with KD = F, this implies EC 1 D = F. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 26 / 51
Permutation Matrix A matrix n n P is a permutation matrix if the rows of P are the same as the rows of n n I but not necessarily in the same order. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 27 / 51
Example: 0 1 0 P = 1 0 0. 0 0 1 1 2 3 4 5 6 7 8 If A = 5 6 7 8, then PA = 1 2 3 4. 9 10 11 12 9 10 11 12 Order of rows of PA are permuted relative to order of rows of A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 28 / 51
Example: 0 1 0 [ ] P = 1 0 0 1 2 3, B =. 4 5 6 0 0 1 [ ] 2 1 3 Then BP =. 5 4 6 Order of columns of BP are permuted relative to order of columns of B. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 29 / 51
A Permutation Matrix is Nonsingular The rows (and columns) of a permutation matrix are the same as those of the identity matrix. Thus, a permutation matrix has full rank and is therefore nonsingular. Furthermore, if P = [p 1,..., p n ] is a permutation matrix, P 1 = P p 0 if i j ip j = 1 if i = j. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 30 / 51
Result A.11: Suppose rank( m n A) = r. There exist permutation matrices m m P and Q such n n that [ ] C D PAQ =, E F where rank( r r C) = r. Furthermore, is a GI of A. [ ] C 1 0 G = Q P 0 0 opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 31 / 51
Proof of Result A.11: Because rank(a) = r, there exists a set of r rows of A that are LI. Let P be a permutation matrix, the first r rows of PA are LI. Let H be the matrix consisting of the first r rows of PA. Then rank(h) = r. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 32 / 51
This implies that a set of r columns of H that are LI. Let Q be a permutation matrix the first r columns of HQ are LI. Then the submatrix consisting of the first r rows and first r columns of PAQ has rank r. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 33 / 51
[ ] C D Thus we can partition PAQ as, where rank( E F r r C) = r. [ ] [ ] C 1 0 C D By Result A.10, is a GI for PAQ =. 0 0 E F Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 34 / 51
[ ] [ ] C D C D PAQ = A = P 1 Q 1 E F E F [ ] C 1 0 AQ PA = 0 0 [ ] [ ] [ C D C = P 1 Q 1 1 0 C Q PP 1 E F 0 0 E [ ] [ ] [ ] C D C = P 1 1 0 C D Q 1 E F 0 0 E F [ ] C D = P 1 Q 1 = A. E F ] D F Q 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 35 / 51
Use Result A.11 to find a GI for 1 1 3 4 A = 2 2 6 8. 3 3 0 12 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 36 / 51
Algorithm for finding a GI of A: 1. Find an r r nonsingular submatrix of A, where r = rank(a). Call this matrix W. 2. Compute (W 1 ). 3. Replace each element of W in A with the corresponding elements of (W 1 ). 4. Replace all other elements in A with zeros. 5. Transpose resulting matrix to get A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 42 / 51
Result A.12: Let Ax = c be a consistent system of equations, and let G be any GI of A. Then Gc is a solution to Ax = c, i.e., AGc = c. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 43 / 51
Result A.13: Let Ax = c be a consistent system of equations, and let G be any GI of A. Then x is a solution to Ax = c iff z x = Gc + (I GA)z. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 45 / 51
Prove that a consistent system of equations Ax = c has a unique solution if and only if A is full-column rank and infinitely many solutions if and only if A is less than full-column rank. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 48 / 51