Submatrices and Partitioned Matrices

2 Submatrices and Partitioned Matrices Two very important (and closely related) concepts are introduced in this chapter that of a submatrix and that of a partitioned matrix. These concepts arise very naturally in statistics (especially in multivariate analysis and linear models) and in many other disciplines that involve probabilistic ideas. And results on submatrices and partitioned matrices, which can be found in hapters 8, 9, 3, and 4 (and other of the subsequent chapters), have proved to be very useful. In particular, such results are almost indispensable in work involving the multivariate normal distribution refer, for example, to Searle (97, sec. 2.4f). 2. Some Terminology and Basic Results A submatrix of a matrix A is a matrix that can be obtained by striking out rows and/or columns of A. For example, if we strike out the second row of the matrix 2 4 3 6 5 7 9A ; 2 2 we obtain the 2 4 submatrix 2 4 3 6 2 2 Alternatively, if we strike out the first and third columns, we obtain the 3 2 submatrix 4 6 5 9A I 2 or, if we strike out the second row and the first and third columns, we obtain the 2 2 submatrix

4 2 Submatrices and Partitioned Matrices 4 6 2 Note that any matrix is a submatrix of itself; it is the submatrix obtained by striking out zero rows and zero columns. Submatrices of a row or column vector, that is, of a matrix having one row or column, are themselves row or column vectors and are customarily referred to as subvectors. Let A represent an r s submatrix of an m n matrix A obtained by striking out the i ;;i m r th rows and j ;;j n s th columns (of A), and let B represent the s r submatrix of A obtained by striking out the j ;; j n s th rows and i ;;i m r th columns (of A ). Then, B D A ; (.) as is easily verified. A submatrix of an n n matrix is called a principal submatrix if it can be obtained by striking out the same rows as columns (so that the ith row is struck out whenever the ith column is struck out, and vice versa). The r r (principal) submatrix of an n n matrix obtained by striking out its last n r rows and columns is referred to as a leading principal submatrix (r D ;;n). A principal submatrix of a symmetric matrix is symmetric, a principal submatrix of a diagonal matrix is diagonal, and a principal submatrix of an upper or lower triangular matrix is respectively upper or lower triangular, as is easily verified. A matrix can be divided or partitioned into submatrices by drawing horizontal or vertical lines between various of its rows or columns, in which case the matrix is called a partitioned matrix and the submatrices are sometimes referred to as blocks (as in blocks of elements). For example, 2 4 3 6 2 4 3 6 2 4 3 6 5 7 9 2 2 A ; 5 7 9 2 2 A ; 5 7 9 2 2 are various partitionings of the same matrix. In effect, a partitioned m n matrix is an m n matrix A Dfa ij g that has been reexpressed in the general form A A 2 A c A 2 A 22 A 2c B A A r A r2 A rc Here, A ij is an m i n j matrix (i D ;;r; j D ;;c), where m, ;m r and n, ;n c are positive integers such that m m r D m and n n c D n. Or, more explicitly, A

2. Some Terminology and Basic Results 5 a m m i ; n n j a m m i ; n n j B A ij D A a m m i ;n n j a m m i ;n n j (When i D or j D, interpret the degenerate sum m m i or n n j as zero.) Thus, a partitioned matrix can be regarded as an array or matrix of matrices. Note that a matrix that has been divided by staggered lines, for example, 2 4 3 6 5 7 9A ; 2 2 does not satisfy our definition of a partitioned matrix. Thus, if a matrix, say A A 2 A c A 2 A 22 A 2c B A ; A r A r2 A rc is introduced as a partitioned matrix, there is an implicit assumption that each of the submatrices A i ; A i2 ;;A ic in the ith row of submatrices contains the same number of rows (i D ;2;;r) and similarly that each of the submatrices A j ; A 2j ;;A rj in the j th column of submatrices contains the same number of columns. It is customary to identify each of the blocks in a partitioned matrix by referring to the row of blocks and the column of blocks in which it appears. Thus, the submatrix A ij is referred to as the ij th block of the partitioned matrix A A 2 A c A 2 A 22 A 2c B A A r A r2 A rc In the case of a partitioned m n matrix A of the form A A 2 A r A 2 A 22 A 2r A A r A r2 A rr (.2) (for which the number of rows of blocks equals the number of columns of blocks), the ij th block A ij of A is called a diagonal block if j D i and an off-diagonal block if j i. If all of the off-diagonal blocks of A are null matrices, that is, if

6 2 Submatrices and Partitioned Matrices A A 22 A ; A rr then A is called a block-diagonal matrix, and sometimes diag.a ; A 22 ;;A rr / is written for A.IfA ij D for j<id;;r, that is, if A A 2 A r A 22 A 2r A ; A rr then A is called an upper block-triangular matrix. Similarly, if A ij D for j> i D ;;r, that is, if A A 2 A 22 A ; A r A r2 A rr then A is called a lower block-triangular matrix. To indicate that A is either upper or lower block-triangular (without being more specific), A isreferredtosimplyas block-triangular. Note that a partitioned m n matrix A of the form (.2) is block-diagonal if and only if it is both upper block-triangular and lower block-triangular. Note also that, if m D n D r (in which case each block of A consists of a single element), saying that A is block diagonal or upper or lower block triangular is equivalent to saying that A is diagonal or upper or lower triangular. Partitioned matrices having one row or one column are customarily referred to as partitioned (row or column) vectors. Thus, a partitioned m-dimensional column vector is an m vector a Dfa t g that has been reexpressed in the general form a a 2 B A a r Here, a i is an m i vector with elements a m m i ;;a m m i m i, respectively (i D ;;r), where m ;;m r are positive integers such that m m r D m. Similarly, a partitioned m-dimensional row vector is a m vector a Dfa t g that has been reexpressed in the general form.a ; a 2 ;;a r /.

2.2 Scalar Multiples, Transposes, Sums, and Products of Partitioned Matrices 7 2.2 Scalar Multiples, Transposes, Sums, and Products of Partitioned Matrices Let A A 2 A c A 2 A 22 A 2c A A r A r2 A rc represent a partitioned mn matrix whose ij th block A ij is of dimensions m i n j. learly, for any scalar k, ka ka 2 ka c ka 2 ka 22 ka 2c k A (2.) ka r ka r2 ka rc In particular, A A 2 A c A 2 A 22 A 2c A (2.2) A r A r2 A rc Further, it is a simple exercise to show that A A 2 A r A A 2 A 22 A r2 D B A I (2.3) A c A 2c A rc that is, A can be expressed as a partitioned matrix, comprising c rows and r columns of blocks, the ij th of which is the transpose Aji of the jith block A ji of A. Now, let B B 2 B v B 2 B 22 B 2v B D B A B u B u2 B uv represent a partitioned p q matrix whose ij th block B ij is of dimensions p i q j. The matrices A and B are conformal (for addition) provided that p D m and q D n. Ifu D r, v D c, p i D m i (i D ;;r), and q j D n j (j D ;;c), that is, if (besides A and B being conformal for addition) the rows and columns of B are partitioned in the same way as those of A, then

8 2 Submatrices and Partitioned Matrices A B A 2 B 2 A c B c A 2 B 2 A 22 B 22 A 2c B 2c A B D B A ; (2.4) A r B r A r2 B r2 A rc B rc and the partitioning of A and B is said to be conformal (for addition). This result and terminology extend in an obvious way to the addition of any finite number of partitioned matrices. If A and B are conformal (for addition), then A B A 2 B 2 A c B c A 2 B 2 A 22 B 22 A 2c B 2c A B D B A ; (2.5) A r B r A r2 B r2 A rc B rc as is evident from results (2.4) and (2.2). The matrix product AB is defined provided that n D p.ifc D u and n k D p k (k D ;;c) [in which case all of the products A ik B kj (i D ;;r; j D ;;v; k D ;;c), as well as the product AB, exist], then F F 2 F v F 2 F 22 F 2v AB D B A ; (2.6) F r F r2 F rv where F ij D P c kd A ikb kj D A i B j A i2 B 2j A ic B cj, and the partitioning of A and B is said to be conformal (for the premultiplication of B by A). In the special case where r D c D u D v D 2, that is, where A A A D 2 B B and B D 2 ; A 22 B 22 A 2 result (2.6) simplifies to A B AB D A 2 B 2 A B 2 A 2 B 22 (2.7) A 2 B A 22 B 2 A 2 B 2 A 22 B 22 If A D.A ; A 2 ;;A c / is an m n matrix that has been partitioned only by columns (for emphasis, we sometimes insert commas between the submatrices of such a partitioned matrix), then A A A 2 D B A ; (2.8) A c B 2

and further if 2.3 Some Results on the Product of a Matrix and a olumn Vector 9 B B 2 B D B A B c is an n q partitioned matrix that has been partitioned only by rows (in a way that is conformal for its premultiplication by A), then AB D cx A k B k D A B A 2 B 2 A c B c (2.9) kd Similarly, if A A 2 A is an m n matrix that has been partitioned only by rows, then A r A D.A ; A 2 ;;A r /; (2.) and further if B D.B ; B 2 ;;B v / is an n q matrix that has been partitioned only by columns, then A B A B 2 A B v A 2 B A 2 B 2 A 2 B v AB D B A (2.) A r B A r B 2 A r B v 2.3 Some Results on the Product of a Matrix and a olumn Vector Let A represent an m n matrix and x an n vector. Writing A as A D.a ; a 2 ;;a n /, where a ; a 2 ;;a n are the columns of A, and x as x D.x ;x 2 ;;x n /, where x ;x 2 ;;x n are the elements of x, we find, as a special case of result (2.9), that Ax D nx x k a k D x a x 2 a 2 x n a n (3.) kd Thus, the effect of postmultiplying a matrix by a column vector is to form a linear combination of the columns of the matrix, where the coefficients in the linear combination are the elements of the column vector. Similarly, the effect of

2 2 Submatrices and Partitioned Matrices premultiplying a matrix by a row vector is to form a linear combination of the rows of the matrix, where the coefficients in the linear combination are the elements of the row vector. Representation (3.) is helpful in establishing the elementary results expressed in the following two lemmas. Lemma 2.3.. For any column vector y and nonnull column vector x, there exists a matrix A such that y D Ax. Proof. Since x is nonnull, one of its elements, say x j, is nonzero. Take A to be the matrix whose j th column is.=x j /y and whose other columns are null. Then, y D Ax, as is evident from result (3.). Q.E.D. Lemma 2.3.2. For any two m n matrices A and B, if and only if Ax D Bx for every n vector x. Proof. It is obvious that, if, then Ax D Bx for every vector x. To prove the converse, suppose that Ax D Bx for every x. Taking x to be the n vector whose ith element is and whose other elements are, and letting a i and b i represent the ith columns of A and B, respectively, it is clear from result (3.) that a i D Ax D Bx D b i (i D ;;n). We conclude that. Q.E.D. Note that Lemma 2.3.2 implies, in particular, that A D if and only if Ax D for every x. 2.4 Expansion of a Matrix in Terms of Its Rows, olumns, or Elements An m n matrix A Dfa ij g can be expanded in terms of its rows, columns, or elements by making use of formula (2.9). Denote the ith row of A by ri and the ith column of I m by e i (i D ;2;;m). Then, writing I m as I m D.e ; e 2 ;;e m / and A as r r 2 A and applying formula (2.9) to the product I m A, we obtain the expansion A D r m mx e i ri D e r e 2 r 2 e m r m (4.) id Similarly, denote the j th column of A by a j and the j th row of I n by u j (j D ;2;;n). Then, writing A as A D.a ; a 2 ;;a n / and I n as

u u 2 I n D B A u n Exercises 2 and applying formula (2.9) to the product AI n, we obtain the alternative expansion A D nx a j uj D a u a 2 u 2 a n u n (4.2) j D Moreover, the application of formula (3.) to the product I m a j gives the expansion mx a j D a ij e i id Upon substituting this expansion into expansion (4.2), we obtain the further expansion mx nx A D a ij U ij ; (4.3) id j D where U ij D e i uj is an m n matrix whose ij th element equals and whose remaining mn elements equal. In the special case where n D m (i.e., where A is square), u j D e j and hence U ij D e i ej, and in the further special case where A D I m, expansion (4.3) reduces to I m D mx e i e i (4.4) Note that, as a consequence of result (4.3), we have that! mx nx mx nx e i Au j D e i a ks e k u s u j D a ks e i e k u s u j ; kd sd id kd sd which (since e i e k equals,ifk D i, and equals,ifk i, and since u s u j equals, ifs D j, and equals, ifs j ) simplifies to e i Au j D a ij (4.5) Exercises Section 2.

22 2 Submatrices and Partitioned Matrices. Verify result (.). 2. Verify (a) that a principal submatrix of a symmetric matrix is symmetric, (b) that a principal submatrix of a diagonal matrix is diagonal, and (c) that a principal submatrix of an upper triangular matrix is upper triangular. 3. Let A A 2 A r A 22 A 2r A A rr represent an n n upper block-triangular matrix whose ij th block A ij is of dimensions n i n j (j i D ;;r). Show that A is upper triangular if and only if each of its diagonal blocks A ; A 22 ;;A rr is upper triangular. Section 2.2 4. Verify results (2.3) and (2.6).