THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS DEMETRES CHRISTOFIDES Abstract. Cosider a ivertible matrix over some field. The Gauss-Jorda elimiatio reduces this matrix to the idetity matrix usig at most 2 row operatios ad i geeral that may operatios might be eeded. I [1] the authors cosidered matrices i GL(, q), the set of ivertible matrices i the fiite field of q elemets, ad provided a algorithm usig oly row operatios which performs asymptotically better tha the Gauss-Jorda elimiatio. More specifically their striped elimiatio algorithm has asymptotic complexity 2 log q. Furthermore they proved that up to a costat factor this algorithm is best possible as almost all matrices i GL(, q) eed asymptotically at least 2 2 log q operatios. I this short ote we show that the striped elimiatio algorithm is asymptotically optimal by provig that almost all matrices i GL(, q) eed asymptotically at least 2 log q operatios. 1. Itroductio Let A be a matrix with etries i some field. Our aim is to compute the iverse of A. The well-kow Gaussia elimiatio does this i O( 3 ) steps. There are eve faster algorithms tha this. For example, Strasse s [3] fast matrix multiplicatio computes the product of two matrices i O( log 2 7 ) steps ad this ca be used (see e.g. [2]) to compute the iverse of a matrix i O( log 2 7 ) steps as well. I [1] the authors cosidered the complexity of matrix iversio from a differet poit of view. More specifically they cosidered methods based oly o row operatios ad measured the complexity of matrix iversio based o the umber of such operatios eeded. The ratioale for this approach was that row operatios ca be implemeted o existig processors far more efficietly tha straight lie programs. With this approach it is Date: May 20, 2014. 2010 Mathematics Subject Classificatio. 05A16, 15A09. Key words ad phrases. Matrix Reductio, Complexity, Fiite Fields. Demetres Christofides, School of Scieces, UCLa Cyprus, 7080 Pyla, Laraka, Cyprus, dchristofides@ucla.ac.uk. This work was doe durig a visit to the Istitut Mittag-Leffler (Djursholm, Swede). 1

2 DEMETRES CHRISTOFIDES also the case that the problem becomes more combiatorial i ature as it is equivalet with determiig the diameter of a specific Cayley graph. Recall that if we apply some row operatios to a matrix A i order to reduce it to the idetity matrix, the applyig the same row operatios to the idetity matrix produces the iverse of A. Sice our measure of complexity here is the umber of row operatios performed, the problems of ivertig a matrix ad of reducig it to the idetity matrix have the same complexity. Therefore from ow o we will be thikig i terms of ivertig a ivertible matrix or reducig it to the idetity iterchageably. The Gauss-Jorda algorithm ca reduce a ivertible matrix to the idetity i at most 2 row operatios (oe operatio per matrix elemet). It is easy to see that we caot expect to improve this i geeral sice if we have a matrix over the reals ad we take all elemets of the matrix to be algebraically idepedet, the we really do eed at least 2 row operatios. I [1] the authors showed that if we restrict the elemets of the matrix to lie i a fiite field the oe ca improve o the Gauss-Jorda algorithm sigificatly. Let GL(, q) deote the set of all ivertible matrices with etries i the field of q elemets. The striped elimiatio algorithm of [1] reduces a matrix i GL(, q) to the idetity i asymptotically at most 2 log q row operatios. Furthermore, it is also show that this algorithm is optimal i the sese that for almost every matrix i GL(, q) we eed asymptotically at least 2 row operatios i order to reduce it to the 2 log q idetity. Our aim i this short ote is to show that the striped elimiatio algorithm is optimal i a much stroger sese: Almost every matrix i GL(, q) row operatios i order to be reduced eeds asymptotically at least 2 log q to the idetity. More specifically, we show that followig result: Theorem 1.1. Let be a positive iteger, q a prime power, ad 0 < α < 1. The usig at most 2 2 log q log q 2 1 q 1 log q e log q 1 a log q + log q 8qe row operatios, we caot reduce more tha a α proportio of all matrices i GL(, q) to the idetity matrix. We have decided to write the boud i Theorem 1 i a explicit rather tha a asymptotic format. We have ot tried to optimise the lower order terms i the statemets eve though some of them could clearly be improved at the expese of more calculatios. I ay case sice the striped elimiatio algorithm of [1] rus i a little bit more tha 2 / log q steps, we have o hope to match the secod order asymptotics of the algorithm usig our approach.

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS3 I Sectio 2 we recall some elemetary liear algebra facts. These are very basic facts appearig i almost every udergraduate liear algebra module. We the show that ay product of elemetary matrices ca be writte as a product of elemetary matrices i a caoical way. We will use these caoical products i Sectio 3 i order to prove Theorem. 2. Caoical products of elemetary matrices Let A be a ivertible matrix with etries i some field F. We are allowed to perform the followig row operatios: (1) For 1 i, j with i j, iterchage the i-th row of A with its j-th row. (2) For 1 i ad λ F with λ 0, multiply all elemets of the i-th row by λ. (3) For 1 i, j with i j ad λ F with λ 0, add λ times the i-th row to the j-th row The result of each row operatio o a matrix A, is exactly the same as the multiplicatio of A from the left by a elemetary row matrix. We deote these matrices by E ij, E i (λ) ad E ij (λ) correspodig to the operatios (1), (2) ad (3) respectively. We perform these operatios oe by oe util we reduce A to the idetity matrix. A crucial fact that will eable us to improve o the lower boud of [1] is that eve though the elemetary matrices do ot i geeral commute, i may istaces they do commute pairwise. The ovelty of our argumet is ot this trivial observatio per se, but how to make a good use of it. I fact we will ot make full use of the commutativity, but oly of the fact that if a set of row operatios affect pairwise differet rows, the these operatios pairwise commute. So for example, eve thought E ij (λ) ad E ik (µ) do commute, we will ot use this fact. The other crucial fact is that eve though two elemetary matrices E, E might ot commute we ca sometimes fid aother elemetary matrix E such that E E = EE. We will use the followig istaces of this observatio: E ij (µ)e i (λ) = E i (λ)e ij (λµ) (1) E ji (µ)e i (λ) = E i (λ)e ji (µ/λ) (2) E kl (λ)e ij = E ij E πij (k)π ij (l)(λ) (3) E k (λ)e ij = E ij E πij (k)(λ) (4) where i (3) ad (4), π ij is the traspositio iterchagig i ad j. All of these equalities follow trivially if we cosider the effects of those elemetary matrices o aother matrix B.

4 DEMETRES CHRISTOFIDES Suppose ow that we have a product of k elemetary matrices. Usig the above facts ad observatios, we will rewrite this ito a ew product of at most k elemetary matrices as follows: Usig (3),(4) ad commutativity where ecessary we ca move all appearaces of elemetary matrices of the form E ij to the left. Now we look at the product of the remaiig matrices ad usig (1),(2) ad commutativity we ca move all appearaces of elemetary matrices of the form E i (λ) to the left. We ca also assume by commutativity that for i < j, every appearace of E i (λ) is to the left of every appearace of E j (µ). Furthermore for each i, all appearaces of E i (λ) for λ R ow appear cosecutively i the product ad we ca replace them with their product which is agai a elemetary matrix of that form. Now we look at the product of the remaiig matrices which are all of the form E ij (λ). Give a elemetary matrix of the form E ij (λ), we will call the set {i, j} its idex set. We begi by partitioig these matrices ito blocks as follows: We start from the left by puttig each matrix ito the first block oe by oe for as log as their idex sets are disjoit. Oce we reach a matrix whose idex set meets the idex set of a matrix i the first block, the we put this ito the secod block. We ow repeat by puttig matrices ito the secod block, the create a ew block as we reach a matrix whose idex set meets the idex set of a matrix i the secod block ad so o. Observe that the matrices i each block commute ad so we ca if we wish permute the matrices i the same block at will without chagig their product. We ow do the followig modificatios: Iitially we do o modificatio i the first block. We the look at the first (from the left) matrix of the secod block, if it exists, whose idex set does ot meet the idex set of ay matrix of the first block. If o such matrix exists the we do o modificatio to the secod block either. Otherwise we move this matrix from the secod block to the first, say to the last positio of the first block. By repeatig this for as log as it is ecessary, we will ed up with the situatio tha every matrix of the secod block meets every matrix of the first block. We ow move o to the third block ad i the same way move matrices back to the secod block for as log as they do ot meet the idex sets of matrices of the secod block. Each time we move a matrix oto the secod block we also check to see whether its idex set meets a idex set of a matrix of the first block. If it does ot the we move it ito the first block. By repeatig this procedure we will ed up with the situatio that we will have several blocks of matrices, such that withi each block the idex set of matrices are disjoit while for every matrix from the secod block owards its idex set will meeet the idex set of at least oe matrix from the previous block. This procedure is guarateed to fiish i a fiite umber of steps. For example, givig to each matrix as value the umber

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS5 of the block i which it appears, the the sum of the values of the matrices reduces after each step of the procedure. Furthermore this umber is a o-egative iteger ad so the procedure caot go o forever. We have ow fiished i rewritig the iitial product of elemetary matrices as a ew product with some specific properties. We call ay such product a caoical product of elemetary matrices. More specifically, we say that a product E 1 E 2 E k of elemetary matrices matrices is caoical if there exist o-egative itegers r, r 0, r 1,..., r s such that (a) Each E 1,..., E r is equal to E ij for some i, j. (b) Each E r+1,..., E r+r0 is equal to E i (λ) for some i, λ. Furthermore, if E t = E i (λ) ad E t = E i (λ ) where r + 1 t < t r + r 0 the i < i. (c) Each E k with k > r + r 0 is equal to E ij (λ) for some i, j, λ. Furthermore, if we write I k = {i, j} for the idex set of this elemetary matrix ad defie r i = r + r 0 + r 1 + + r i for each 0 i s the the followig holds: (i) For each 1 i s, we have that the idex sets I r i 1 +1,..., I r i are pairwise disjoit. (i) For each 2 i s, ad each t [r i 1 + 1, r i] there is a t [r i 2 + 1, r i 1] with I t I t. 3. Proof of Theorem 1 Suppose that every matrix i GL(, q) ca be reduced to the idetity matrix usig at most k row-operatios. From our results i Sectio 2, it follows that every such matrix ca be writte as a caoical product of at most k elemetary matrices. This caoical product starts with a product of matrices of the form E ij. Their product is a permutatio matrix, so there are at most! = q log q differet product that ca be obtaied so far. The caoical product cotiues with a product of matrices of the form E i (λ). There are 2 ways to choose which idices i appear i the matrices of this product. For each such matrix, there is a total of q 1 ways to choose λ. So i total the product of those matrices ca be formed i at most 2 (q 1) q log q 2+ ways. Fially, there are at most k more matrices to cosider, all of the form E ij (λ). These matrices will appear ito blocks of r 1, r 2,..., r s matrices for some o-egative iteger s ad some positive itegers r 1,..., r s with r 1 + + r s k. Withi each block the idex sets of the matrices used are

6 DEMETRES CHRISTOFIDES all disjoit, while the idex set of every matrix of a block meets the idex set of at least oe matrix of the previous block. There are exactly 2 k ways i order to choose the umbers r 1,..., r s. To see this observe that give positive itegers r 1,..., r s such that r 1 + + r s k, these determie the subset {r 1, r 1 + r 2,..., r 1 + + r s } of {1, 2,..., k}. Coversely, give ay subset {x 1,..., x s } of {1, 2,..., k} with x 1 < x 2 < < x s the r 1 = x 1, r 2 = x 2 x 1,..., r s = x s x s 1 are positive itegers with r 1 + + r s k. Furthermore these two maps betwee tuples of positive itegers summig up to at most k, ad subsets of {1, 2,..., k} are iverses of each other ad so ideed the umber of ways to choose r 1,..., r s is equal to 2 k = q k log q 2. Suppose ow that r 1,..., r s have bee chose. There are at most q r 1 2r 1 ways to choose the first r 1 matrices. Here the q r 1 is for the choose of λ s ad the 2r 1 for the choice of idices. Havig chose those, there are at most q r 2 (4r 1 ) r 2 ways to choose the secod r matrices. This is because whe choosig each of the r 2 matrices of this block, there are 2 ways to choose which elemet of its idex set will meet the idex set of a matrix from the previous block, there are at most 2r 1 ways to choose that elemet, ad there are at most ways to choose the other elemet. Similarly, there are q r 3 (4r 2 ) r 3 ways to choose the matrices of the third block ad so o. So i total for fixed r 1, r 2,..., r s there are q r 1 2r 1 (4qr 1 ) r2 (4qr s 1 ) rs 2r 1+r 2 +r s (4q) r 1+ +r s r r 2 1 r rs s 1r r 1 s +k (4q) k r r 2 1 r rs s 1r r 1 s = q log q +k log q +2k log q 2+k r r 2 1 r rs s 1r r 1 s. ways to form this product. However may of those products give rise to the same matrix. I particular, the order i which we pick the matrices of the first block does ot matter as it will give up the same product. The same holds for the order of the matrices withi each block. So for each r 1,..., r s, each possible product has bee appeared i the above calculatio at least r 1! r s! times. We ow observe that r 1! r s! ( ) r 1e r1 ( r se ) rs r r 1 1 rs rs. e k Sice the fuctio x log q x is a icreasig fuctio of x, the rearragemet iequality shows that ad so r 1 log q r 1 r s log q r s r 2 log q r 1 + + r s log q r s 1 + r 1 log q r s

THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS7 r r 1 1 rs rs r r 2 1 rs 1r r 1 So i total, for each r 1,..., r s there are at most q log q +k log q +2k log q 2+k+k log q e differet products that ca be formed. So puttig everythig together, there is a total of at most q (k+2) log q +(3k+) log q 2++k+k log q e (5) distict matrices that ca arise from caoical products of at most k matrices from GL(, q). Fially, it is ot difficult to see that 1 1 GL(, q) = (q q k ) = q 2 (1 q k ) = q (1 2 1q ). r But k=0 (1 1q ) r r=1 ad so there are at least k=0 s. r=1 = e r=1 log(1 q r) e r=1 q r e 1 q 1 e 1 q 1 q 2 = q 2 1 q 1 log q e ivertible matrices with etries i F q. This, together with (5), complete the proof of Theorem 1. Refereces [1] D. Adré, L. Hellström ad K. Markström, O the complexity of matrix reductio over fiite fields, Adv. i Appl. Math. 39 (2007), 428 452. [2] J. R. Buch ad J. E. Hopcroft, Triagular factorizatio ad iversio by fast matrix multiplicatio, Math. Comp. 28 (1974), 231 236. [3] V. Strasse, Gaussia elimiatio is ot optimal, Numer. Math. 13 (1969), 354 356.