arxiv: v3 [math.na] 27 Mar 2017

Transcription

1 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS arxiv: v3 [mathna] 27 Mar 2017 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ Abstract The paper studies the goba convergence of the bock Jacobi method for symmetric matrices Given a symmetric matrix A of order n, the method generates a sequence of matrices by the rue A (k+1) = U T k A (k) U k, k 0, where U k are orthogona eementary bock matrices A cass of generaized seria pivot strategies is introduced, significanty enarging the known cass of weak wavefront strategies, and appropriate goba convergence proofs are obtained The resuts are phrased in the stronger form: S(A ) cs(a), where A is the matrix obtained from A after one fu cyce, c < 1 is a constant and S(A) is the off-norm of A Hence, using the theory of bock Jacobi operators, one can appy the obtained resuts to prove convergence of bock Jacobi methods for other eigenvaue probems, such as the generaized eigenvaue probem As an exampe, the resuts are appied to the bock J-Jacobi method Finay, a resuts are extended to the corresponding quasi-cycic strategies 1 Introduction The main incentive for writing this paper was a need to expand the cass of convergent strategies for the bock Jacobi method for symmetric matrices With a arge choice of casses at our disposa we can prove goba convergence of other bock-wise or eement-wise Jacobi-type methods and even appy it to reated probems, eg, to the generaized eigenvaue or singuar vaue probem (see [29]) The techniques we are about to empoy use the theory of bock Jacobi operators, which was described in [18] Over the ast two decades the Jacobi method has emerged as a method of choice for the eigenvaue computation for dense symmetric matrices This is mosty due to its inherent paraeism and high reative accuracy on we-behaved matrices Athough the origina method is very od [23] and it had been one of the first methods to be impemented on computers, it was forgotten in the 1970s after appearance of the QR and Divide and conquer method Aready in 1971 Sameh [31] showed how to adapt the seria Jacobi method to parae processing Later, in 1992, Demme and Veseić [4] proved high reative accuracy of the method on we-behaved symmetric positive definite matrices Foowing their breakthrough, the method came back to the focus of the current researcher Drmač and Veseić [7, 8] showed that, even on standard oneprocessor computers, the method can be modified to become faster than the QR method whie sti retaining its distinguished property: high reative accuracy Nowadays, the Jacobi method is we understood On the one hand, its asymptotic convergence was considered in [35, 15, 30] and Date: 19 February Mathematics Subject Cassification 65F15 Key words and phrases Eigenvaues, bock Jacobi method, pivot strategies, goba convergence Vjeran Hari, Department of Mathematics, Facuty of Science, University of Zagreb, Bijenička 30, Zagreb, Croatia Erna Begović Kovač, Facuty of Chemica Engineering and Technoogy, University of Zagreb, Maruićev trg 19, Zagreb, Croatia This work has been fuy supported by Croatian Science Foundation under the project

2 2 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ its goba convergence was studied in [10, 11, 22, 28, 21, 9, 24, 32, 25, 16] On the other hand, its high reative accuracy was considered in [34, 4, 27, 26], whie its efficiency was investigated in [7, 8] The method has aso been impemented as a standard LAPACK routine With thedeveopment of CPU andgpu parae computingpatforms, it hasbeenfoundthat a sensibe way of increasing numerica efficiency of the method invoves using a one-sided agorithm, together with BLAS 3 subroutines, which can nicey expoit cache memory hierarchy The matrix description of such a method is caed the bock Jacobi method This bock method is aways impemented as a one-sided bock (Jacobi or J-Jacobi) agorithm because high efficiency and high reative accuracy are warranted then However, in the goba and asymptotic convergence considerations the resuts are cast in terms of a two-sided bock Jacobi method The first goba convergence resuts for the bock Jacobi methods were given in [5, 3, 19, 20, 18] These papers considered the most common seria pivot strategies and the strategies equivaent to them The aim of this paper is to further deveop the goba convergence theory for the bock Jacobi method and to provide a arge cass of usabe pivot strategies for which the convergence can be estabished rigorousy In genera, our cass consists of more than 4 2! 3! m! cycic strategies, where m is the number of bock-coumns in the bock-matrix partition of a symmetric matrix of order n These strategies incude the weak-wavefront ones from [32] and many others As a byproduct of this research, we can now prove that every cycic (eement-wise or bock) Jacobi method for symmetric matrices of order 4 is gobay convergent (see [1, 2]) In addition, we consider a simiar cass of quasi-cycic strategies and derive the corresponding convergence resuts The bock anaogue of the strategy that is used in the LAPACK impementation of the Jacobi method ies in that cass The convergence resuts are given in the stronger form, S(A ) cs(a), 0 c < 1 Here, A is the initia symmetric matrix of order n, A is obtained from A after appying one sweep of some cycic or quasi-cycic bock Jacobi method, S( ) is departure from the diagona form, and c is a constant depending on n and the bock-matrix partition, but not on A Such a resut aows for the use of the theory of bock Jacobi operators Hence, it can be utiized to prove the goba convergence of other Jacobi-type methods, designed for different eigenvaue probems As an appication, we wi appy it to the bock J-Jacobi method from [20] Some of the resuts presented here can be found in the unpubished thesis [1] The paper is divided into six sections of the main text and an appendix In Section 2 we present the basic concepts inked with a bock Jacobi method for symmetric matrices Specia attention is paid to cycic and quasi-cycic pivot strategies, and to the ways of enarging significanty the number of convergent strategies The concepts of equivaent, weak-equivaent and permutation equivaent strategies are used Another usefu too is introduced, the so-caed bock Jacobi annihiators and operators for symmetric matrices, and some basic resuts reated to them are proved In Section 3 we introduce a cass of generaized seria strategies and prove the corresponding goba convergence resuts In Section 4 we briefy introduce a simiar cass of quasi-cycic pivot strategies and prove the appropriate convergence resuts As an appication, in Section 5 we prove the goba convergence of the bock J-Jacobi method under the strategies from the newy introduced casses Section 6 announces the future work Finay, to make the paper easier to read, we move a engthy and technica proofs to the Appendix

3 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 3 2 Basic concepts and notation We introduce the basic definitions inked with the bock Jacobi method for symmetric matrices Specia attention is paid to the cycic and quasi-cycic pivot strategies Later we dea with more advanced concepts ike the bock Jacobi annihiators and operators 21 Bock Jacobi method Let A be a square matrix of order n and et π be an integer partition of n, π = (n 1,n 2,,n m ), n i 1, 1 i m; n 1 +n 2 + +n m = n (21) Then π determines the bock-matrix partition of A, A 11 A 12 A 1m A 21 A 22 A 2m A = A m1 A m2 A mm n 1 n 2 n m, (22) where the diagona bocks A 11,,A mm are square matrices of order n 1,,n m, respectivey Reation (22) wi be schematicay denoted by A = (A rs ) Since we consider the goba convergence of the bock Jacobi method for symmetric matrices, we assume that A is symmetric A bock Jacobi method is determined by the partition π, some pivot strategy and the agorithm The partition is chosen in accordance with the capacity of the hierarchica cache memory of the computer Typicay, the code presumes n 1 = n 2 = = n m 1, n m = n (m 1)n 1 n 1 In our anaysis we consider it arbitrary but unchanged over the iterations Actuay, there are situations when it makes sense to change π during the process, but these are inked with the asymptotic convergence of the method The bock Jacobi method uses orthogona eementary bock matrices as transformation matrices An orthogona eementary bock matrix U ij has the form (see [18]) I U U ij n i U ij = I if i < j, or U ij = I U n i if i = j, (23) U ji U jj n j I I where it is presumed that the bock-matrix partition is determined by π from the reation (21) Since i and j address the bocks, they can be caed bock pivot indices, but for brevity we simpy ca them pivot indices Simiary, (i,j) is the pivot pair and [ ] U U Û ij = ij if i < j, or U ji U Ûij = U if i = j, (24) jj isthepivot submatrix ofu ij When(i,j) isunderstoodwewiasowriteû insteadofûij Wecan buid an orthogona eementary bock matrix using the function E which imbeds any orthogona matrix Ũ of order n i + n j (or n i if i = j) into the identity matrix I n, so that U ij = E(i,j,Ũ) impies Ûij = Ũ The mapping E depends on the partition π Each bock Jacobi method is an iterative processes of the form A (k+1) = U T k A(k) U k, k 0; A (0) = A, (25) whereu k, k 0, areorthogonaeementarybockmatrices LetA (k) = (A (k) rs ) WesaythatA (k+1) is obtained or generated from A (k) at step k via the recursion (25) Let U k = E(i(k),j(k),Ûk)

4 4 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ Then i(k), j(k) are the pivot indices and (i(k),j(k)) is the pivot pair at step k For brevity, we wi often omit k and denote the pivot indices simpy by i, j and the pivot pair by (i,j) The way of seecting the pivot pair at each step is caed a pivot strategy At step k the bock Jacobi method diagonaizes the pivot submatrix of A (k) Thus, if i < j, the pivot bocks A (k) ij and A (k) ji are annihiated and the affected diagona bocks A (k) and A (k) jj are diagonaized If Â(k) denotes the pivot submatrix of order n i +n j at step k, it is transformed as foows: [ Λ (k+1) 0 0 Λ (k+1) jj ] = [ U (k) U (k) ji U (k) ij U (k) jj ] T [ A (k) A (k) ij (A (k) ij )T A (k) jj ][ U (k) U (k) ji U (k) ij U (k) jj ], (26) where Λ (k+1) and Λ (k+1) jj are diagona If i = j, then just A (k) is diagonaized For the diagonaization of the pivot submatrix, one can choose any method for soving the symmetric eigenvaue probem Typicay, one appies a standard (eement-wise) Jacobi method for its high reative accuracy [26] and efficiency on neary diagona matrices As has been expained in [18], it is preferabe to preprocess the initia matrix by m bock Jacobi steps with pivot pairs (1,1),,(m,m), so that in the starting matrix the diagona bocks are actuay diagona submatrices This preprocessing is depicted beow for the case π = (3, 2, 1, 2): A = x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x A (0) = x 0 0 x x x x x 0 x 0 x x x x x 0 0 x x x x x x x x x x 0 x x x x x x 0 x x x x x x x x x x x x x x x x x x x 0 x x x x x x 0 x Once the diagona bocks are diagonaized, a ater steps wi preserve that property It means that at each step the pivot indices wi satisfy i < j, which unifies and simpifies the agorithm In this regard the bocks A and A jj on the right side of the reation (26) can be repaced by Λ and Λ jj, respectivey Therefore, in the seque it is presumed that the diagona bocks of each A (k) are diagona and for the pivot indices i < j hods 22 Pivot strategies Each pivot strategy can beidentified withafunctioni : N 0 P m, where N 0 = {0,1,2,3,} and P m = {(r,s) 1 r < s m} If I is a periodic function with period T, we say that I is a periodic pivot strategy In this paper we consider two types of periodic strategies: cycic and quasi-cycic ones If T = M m(m 1) 2 and {(i(0),j(0)),(i(1),j(1)),,(i(t 1),j(T 1))} = P m, then we say that the pivot strategy is cycic It immediatey foows that, during any M successive steps of the method, a off-diagona bocks are annihiated exacty once Such bock Jacobi method is aso said to be cycic and the transition from A ((r 1)M) to A (rm) is caed the rth cyce or sweep of the method If T M and {(i(0),j(0)),(i(1),j(1)),,(i(t 1),j(T 1))} = P m, then the strategy is caed quasi-cycic Thus, during any T successive steps of the method, each off-diagona bock is annihiated at east once The corresponding bock Jacobi method is caed quasi-cycic and the transition from A ((r 1)T) to A (rt) is caed the rth quasi-cyce or sweep of the method Let us examine cycic and quasi-cycic strategies more cosey For S P m we denote by O(S) the set of a finite sequences containing the eements of S, assuming that each pair from S appears at east once in each sequence If I is a cycic or

5 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 5 quasi-cycic strategy with period T, then O I stands for the sequence I(0),I(1),,I(T 1) O(P m ), generated by the first T steps (ie, by the first sweep) of the method Conversey, if O O(P m ), O = (i 0,j 0 ),(i 1,j 1 ),,(i T 1,j T 1 ), then the periodic strategy I O is defined by I O (k) = (i τ(k),j τ(k) ), where τ(k) is the unique integer satisfying 0 τ(k) T 1 and k τ(k)(mod T), k 0 These two functions, O I O and I O I, enabe us to investigate the cycic and quasi-cycic strategies by studying the sequences fromo(p m ) Note that, if I is cycic, then O I is simpy an ordering of P m We wi aso use the term pivot ordering in this case, whie if I is quasi-cycic, we wi use the term pivot sequence An admissibe transposition on O O(S), S P m, is any transposition of two adjacent terms in O, (i r,j r ),(i r+1,j r+1 ) (i r+1,j r+1 ),(i r,j r ), provided that the sets {i r,j r } and {i r+1,j r+1 } are disjoint We aso say that such pairs (i r,j r ) and (i r+1,j r+1 ) commute The number of pairs in O is denoted by O and it is caed the ength of O Definition 21 Two sequences O,O O(S), S P m, are said to be (i) equivaent (we write O O ) if one can be obtained from the other by a finite set of admissibe transpositions; () shift-equivaent (O s O ) if O = [O 1,O 2 ] and O = [O 2,O 1 ], where [, ] stands for concatenation and the ength of O 1 is caed shift ength; (i) weak equivaent (O w O ) if there exist O i O(S), 0 i r, such that every two adjacent terms in the sequence O = O 0,O 1,,O r = O are equivaent or shift-equivaent One can check that, s and w are equivaence reations ono(s) If three or more sequences are connected by or s one can omit the mid terms because of the transitivity property of equivaence reation Hence, if O w O, then there is a sequence O = O 0,O 1,,O r = O such that either O 0 O 1 s O2 O 3 s O4 or O 0 s O1 O 2 s O3 O 4 (27) Two pivot strategies I O and I O are equivaent (resp shift-equivaent, weak equivaent) if the corresponding sequences O and O are equivaent (resp shift-equivaent, weak equivaent) The most common cycic pivot strategies are the row-cycic one, I row = I Orow, and the coumncycic one, I co = I Oco, which are defined by the row-wise and coumn-wise orderings of P m : O row = (1,2),(1,3),,(1,m),(2,3),,(2,m),,(m 1,m) O co = (1,2),(1,3),(2,3),,(1,m),(2,m),,,(m 1,m) The common name for them is seria strategies The cycic pivot strategies which are equivaent (resp weak-equivaent) to the seria ones are aso caed wavefront (resp weak-wavefront) strategies (see [32]) Definition 22 Let O O(S), S P m, O = (i 0,j 0 ),(i 1,j 1 ),,(i r,j r ) Then O = (i r,j r ),,(i 1,j 1 ),(i 0,j 0 ) O(S) is the reverse (or inverse) sequence to O If S = P m, we say that the pivot strategy I O = I O is reverse (inverse) to I O Obviousy, we have O = O and hence I O = I O for O O(P m ) and

6 6 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ Lemma 23 Let O,O O(S), S P m Then O w O if and ony if O w O Proof From formua (27), we see that it is sufficient to prove the assertion for reations and s Let O and O be as in Definition 22 Suppose O is obtained from O by appying one admissibe transposition Then for some 0 t < r we have {i t,j t } {i t+1,j t+1 } = If t 1, then O = (i 0,j 0 ),,(i t 1,j t 1 ),(i t+1,j t+1 ),(i t,j t ),,(i r,j r ), O = (ir,j r ),,(i t,j t ),(i t+1,j t+1 ),(i t 1,j t 1 ),,(i 0,j 0 ), and obviousy O O If t = 0, O = (i 1,j 1 ),(i 0,j 0 ),,(i r,j r ) and O = (i r,j r ),, (i 0,j 0 ),(i 1,j 1 ), so we aso have O O If O is the resut of appying more than one admissibe transposition to O, then the proof proceeds by appying the above argument severa times Suppose that O O hods By the impication proved in the preceding paragraph we have O O, and this is the same as O O Now, et O s O Suppose O = (i t+1,j t+1 ),,(i r,j r ),(i 0,j 0 ),,(i t,j t ) Then O = (i t,j t ),,(i 0,j 0 ),(i r,j r ),,(i t+1,j t+1 ), which is shift-equivaent to O = (i r,j r ),,(i 1,j 1 ), (i 0,j 0 ) with shift equa to r t Now we know that O s O impies O s O and this is the same as O s O To visuay depict an ordering O of P m we make use of the symmetric matrix M O = (m rt ) of order m, defined by the rue m i(k)j(k) = m j(k)i(k) = k, k = 0,1,,M 1; M = m(m 1)/2 We set m rr = 1, 1 r m, but since the pairs (r,r) do not appear in O, we wi rather use to represent 1 Exampe 24 As an iustration, we depict the matrices M O row and M O co for m = M O r = , M Oc = These two matrices give us information on the order in which the off-diagona bocks in the bock-matrix from (22) are annihiated during each cyce 221 Permutation equivaent strategies Let us introduce yet another equivaence reation on O(P m ) and on the set on cycic and quasi-cycic pivot strategies Two pivot orderings O,O O(P m ) are permutation equivaent if M O = PM O P T hods for some permutation matrix P In that case we write O p p O and IO I O Reca that each permutation matrix P of order m is defined by some permutation p of the set S m = {1,2,,m} via the reation Pe r = e p(r), 1 r m (28) Here I m = [e 1,,e m ] The mapping p P is an isomorphism between the symmetric group on the set S m and the group of permutation matrices of order m If X = (x rt ) is any square matrix of order m, then PXP T = (x p 1 (r),p 1 (t)) Hence, if Õ is permutation equivaent to O and M O = (m rt ), MÕ = ( m rt ), then m p(r)p(t) = m rt hods for a r, t This reation shows

7 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 7 that the (r,t)-eement of M O becomes the (p(r),p(t))-eement of MÕ Therefore, if Õ p O and O = (i 0,j 0 ),,(i M 1,j M 1 ), then Õ = (p(i 0),p(j 0 )),, (p(i M 1 ),p(j M 1 )) Here it is presumed that in the case p(i) > p(j), the pair (p(i),p(j)) in the ordering Õ is repaced by (p(j),p(i)) Exampe 25 Let m = 4, S( m = {1,2,3,4}, ) O = (1,2),(2,3),(2,4),(3,4),(1,3), ( ) (1,4) and P = [e 2,e 4,e 3,e 1 ], so that p =, p =, P T = [e 4,e 1,e 3,e 2 ] Let Õ be such that MÕ = PM O P T Since MÕ = e T 4 e T 1 e T 3 e T [e 4,e 1,e 3,e 2 ] = we have Õ = (2,4),(3,4),(1,4),(1,3),(2,3),(1,2) On the other hand, we have O(p) (p(1),p(2)),(p(2),p(3)),(p(2),p(4)),(p(3),p(4)),(p(1),p(3)),(p(1),p(4)) = (2,4),(3,4),(1,4),(1,3),(2,3),(1,2) = Õ Now it is easy to extend the notion of permutation equivaence from pivot orderings to pivot sequences from O(P m ) Definition 26 Let O = (i 0,j 0 ),(i 1,j 1 ),,(i T 1,j T 1 ) O(P m ), T M The sequence O O(P m ) is permutation equivaent to O, and we write O p O, if there is a permutation q of the set S m such that O = (q(i 0 ),q(j 0 )),(q(i 1 ),q(j 1 )),, (q(i T 1 ),q(j T 1 )) Then O is denoted by O(q) Since O = O(e), where e is the identity permutation, we have O p O If O = O(q) then O = O (q 1 ) Aso if O = O(q) and O = O (q ) then O = O(q q) where denotes the binary operation in the permutation group, which is simpy the composition of functions We concude that p is an equivaence reation on the seto(p m ) Note that O r = O r 1 (q r ), 1 r t, impies O t = O 0 (q) with q = q t q t 1 q 1 Lemma 27 Let O,O 1,O 2,O 3,O 4 O(P m ) (i) If O w O 1 p O2, then there is O O(P m ) such that O p O w O2 () If O p O 3 w O4, then there is Õ O(P m) such that O w Õ p O 4 Proof (i) It is sufficient to prove this assertion for the two cases: (a) w is reduced to and (b) w is reduced to s (a) It is sufficient to assume that O 1 is obtained from O by appying one admissibe transposition Let O = (i 0,j 0 ),,(i r,j r ),(i r+1,j r+1 ),,(i T 1,j T 1 ) with {i r,j r } {i r+1,j r+1 } =, so that O 1 = (i 0,j 0 ),,(i r+1,j r+1 ),(i r,j r ),,(i T 1,j T 1 ) If O 2 = O 1 (q), then O 2 = (q(i 0 ),q(j 0 )),,(q(i r+1 ),q(j r+1 )),(q(i r ),q(j r )),, (q(i T 1 ), q(j T 1 )) Since q is a bijection from S m onto itsef, we have {q(i r ),q(j r )} {q(i r+1 ), q(j r+1 )} = Therefore, we can set O = O(q), ie, O = (q(i 0 ),q(j 0 )),,(q(i r ),q(j r )),(q(i r+1 ),q(j r+1 )),,(q(i T 1 ),q(j T 1 )) (b) Let O be as in case (a) and assume that the cut has been made behind the term (i r,j r ), so that O 1 = (i r+1,j r+1 ),,(i T 1,j T 1 ),(i 0,j 0 ),,(i r,j r ) Let O 2 = O 1 (q) = (q(i r+1 ),q(j r+1 )),,

8 8 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ,(q(i T 1 ),q(j T 1 )),(q(i 0 ),q(j 0 )),,(q(i r ),q(j r )) Obviousy, wecandefineo tobethesame as in case (a) To obtain O 2 from O, one has to make the cut just behind the term (q(i r ),q(j r )), ie, to use the shift r+1 () The proof is quite simiar to the proof of (i) First, consider the case when w is reduced to, and is given by one admissibe transposition that interchanges the terms at positions r and r +1, as in case (a) above Let O be as in case (i) and denote O 3 = O( q) Then O 4 is the same as O 2, provided that q is repaced by q and Õ is ike O 1 Next, consider the case when w is reduced to s and O 3 = O( q) Assume that the cut has been made as in case (b), behind the term ( q(i r ), q(j r )) Then O 4 is the same as O 2 (from case (b)), provided that q is repaced by q and Õ is ike O 1 Proposition 28 Let O,O,O 1,O 2,O 3,,O 2t O(P m ), t 1 If O w O 1 p O2 w O3 p O4 w w O2t 1 p O2t w O, (29) then there exist O 0,Õ0 O(P m ) such that O p O 0 w O and O w Õ0 p O If O 2r = O 2r 1 (q r ), 1 r t, then O 0 = O(q) and O = Õ0(q) with q = q t q t 1 q 1 Proof Using assertion (i) (resp assertion ()) of the previous emma, one can graduay move a appearances of p to the eft (resp right) end of the chain (29) First, the eftmost (resp rightmost) p is moved The eftmost (resp rightmost) part of the chain takes the form O p O 1 w O 2 w O3 p O4 (resp O 2t 3 p O2t 2 w O2t 1 w Õ 2t p O ) Note that two consecutive w can be repaced by one, so we can remove O 2 (resp O 2t 1 ) from the p obtained chain Next, that inks O 3 and O 4 (resp O 2t 3 and O 2t 2 ) is moved The eftmost (resp rightmost) part of the chain takes the form O p O 1 p O 3 w w w w p p O 4 O5 (resp O 2t 4 O2t 3 Õ 2t 2 Õ 2t O ) Continuing this way one utimatey obtains O p O 1 p O 3 p p O 2t 1 w O 2t w O (resp O w O 1 w Õ 2 p p Õ 2t 2 p Õ 2t p O ) (210) Here O 2t (resp O 1 ) can be removed Note that p is an equivaence reation Hence by the transitivity property, the eftmost part of the chain O p O 1 p O 3 p p O 2t 1 can be repaced by O p O 2t 1 (and simiary for the rightmost part of the chain) To compete the proof of the first assertion one has to rename O 2t 1 as O 0 (resp Õ 2 as Õ0) The proof of the second assertion is the same, but one can use more information Now, from the proof of the preceding emma we know that in the fina chain (210) we have O 1 = O(q 1) and O 2r 1 = O 2r 3 (q r), 2 r t (resp O = Õ(q t) and Õ2r = Õ2r 2(q r 1 ), 2 r t) Hence O 0 = O 2t 1 = O(q) (resp O = Õ2(q) = Õ0(q)) with q = q t q t 1 q 1 Obviousy, the first or the ast (or both) appearance of w in the chain (29) can be omitted provided that O = O 1 and O = O 2t (resp O = O 1 and O = O 2t ) Two sequences O,O O(P m ) can be inked via a ong chain ike (29), which may incude s w a equivaence reations,, and p introduced so far Proposition 28 shows that each such chain can be reduced to a short chain that uses just one p and one w Furthermore, weak equivaence can be written in the most compact form (27)

9 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 9 Definition 29 Two sequences O,O O(P m ) are connected by a chain of equivaence reations if there exist O 1,,O r O(P m ), r 0, such that in the sequence O, O 1,,O r,o each two s neighboring terms are inked with,, w or p The chain is in canonica form if it ooks ike (27) with one p paced in front of it or behind it We concude that every chain of equivaence reations can be reduced to the canonica form Let us return to the bock matrix A from reation (22) To a partition π = (n 1,,n m ) we associate the n-tupe (s 1,,s m ), s i = s i 1 +n i, 2 i m; s 1 = n 1 Note that the sequence of the first n natura numbers, 1,2,,n, can be written as 1,,s 1, s 1 +1,,s 2,,s m 1 +1,,s m, which is the same as s 1 n 1 +1,,s 1,s 2 n 2 +1,,s 2,,s m n m +1,,s m Here, s r n r +1,,s r are indices of the coumns (resp rows) which make the rth bock-coumn (resp bock-row) of A Let O P m and Õ = O(p), so that Õ = (p(i 0),p(j 0 )), (p(i 1 ),p(j 1 )),, (p(i T 1 ),p(j T 1 )) The permutation p of the set S n = {1,2,,n} associated with p is defined by: ( ) s1 n p = 1 +1, s 1, s 2 n 2 +1, s m n m +1, s m s p(1) n p(1) +1, s p(1), s p(2) n p(2) +1, s p n p +1, s p Using the same rue (28), we obtain the permutation matrix P of order n, associated with p It satisfies Pe t = e p(t), 1 t n The matrix P has the form P = [E p(1) E p ], where each E p(k) is an n n p(k) matrix (ie, a singe bock-coumn) that differs from the zero matrix ony in its p(k)th bock-row, E p(k) = [ 0 0 I p(k) 0 0 ] T, 1 k m n 1 n p(k) n m Let A (T) be the matrix obtained from A by appying one sweep of the quasi-cycic bock Jacobi method defined by I O The iterative process has the form (25), hence using P and P T we can write where PA (T) P T = P(U T T 1 UT T 2 UT 0 PT )PAP T (PU 0 U T 2 U T 1 )P T (211) = (PU T T 1P T ) (PU T 0 P T )PAP T (PU 0 P T ) (PU T 1 P T ) = ŨT T 1 ŨT 0 (PAPT )Ũ0 ŨT 1, Ũ k = PU k P T, 0 k T 1 (212) Each Ũk is an orthogona eementary bock matrix whose pivot pair is (p(i k ),p(j k )) We can interpret the process (211) as a quasi-cycic bock Jacobi method defined by IÕ When it is appied to à = PAPT, after one sweep it resuts in Ã(T) = PA (T) P T Indeed, at step k of that process we have à (k+1) = ŨT k Ã(k) Ũ k, 0 k T 1, (213) where Ã(k) = PA (k) P T for any k For the matrix Ã(k+1) we know that its (p(i k ),p(j k ))-pivot submatrix of order n p(ik ) + n p(jk ) is diagona Hence, it is a quasi-cycic bock Jacobi method Moreover, if U k is in the cass UBCE π ( ) from Section 23, the same wi be true for Ũk The process (213) is a bock Jacobi method for the matrix à which carries bock-matrix partition defined by π p = (n p(1),,n p ) Thus, if A is repaced by à and π by π p, then the

10 10 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ bock method (213) is defined by the pivot sequence Õ = O(p) O(P m) We can formay write (π,o,a) p (π p,o(p),pap T ) This is equivaent to (π p 1, O(p 1 ),P T AP) p (π,o,a) We end this subsection with two remarks First, the reverse ordering O is not the same as O(ẽ), where ( ) 1 2 m ẽ = (214) m m 1 1 Exampes which confirm this caim are those from Section 32 Second, if in Lemma 23 the equivaence reation w is repaced by p, the assertion wi remain to hod The proof is trivia Indeed, if in the chain (27), which may now incude p, ony one sequence O t is repaced by O t, then a the sequences have to be repaced by their inverses Otherwise, the chain can be broken into two chains that are not mutuay connected 23 Goba convergence A bock Jacobi method is convergent on A if the obtained sequence of matrices (A (k) ) converges to some diagona matrix Λ The method is gobay convergent if it is convergent on every symmetric matrix A This definition assumes that the partition π is arbitrary In particuar one can take π = (1,1,,1), which means that it is thepropergeneraization of the standard notion of goba convergence The words goba and gobay are often omitted For exampe, if one says that the bock method converges for some pivot strategy, this means that the method converges for every initia symmetric matrix For the goba convergence considerations, it is irreevant whether the diagona bocks of the initia matrix are diagona submatrices Namey, after some iteration (within the first sweep) this property wi be fufied and it wi remain to hod unti convergence To measure how much the method has converged, we use the quantity S(A) = 2 2 A diag(a) F = [ n 1 n s=1 t=s+1 a st 2 ]1 2, where X F = trace(x T X) stands for the Frobenius norm of X In the definition of S(A) we coud have used bocks instead of eements, but since the diagona bocks are diagona submatrices, this reduces to the same quantity Obviousy, the convergence of a bock Jacobi method on A impies that S(A (k) ) 0 as k The converse is true provided that the diagona eements of diag(λ (k+1),λ (k+1) jj ) from (26) are aways ordered in some prescribed order, typicay nonincreasingy Theorem 210 Let A be a symmetric matrix and A (k), k 0, be the sequence obtained by appying the bock Jacobi method to A Let the pivot strategy be cycic or quasi-cycic and assume that im k S(A (k) ) = 0 (i) If the agorithm that diagonaizes the pivot submatrix aways deivers diag(λ (k+1), Λ (k+1) jj ) with nonincreasingy (resp nondecreasingy) ordered diagona eements, then Λ = im k A (k) and the diagona eements of Λ are nonincreasingy (resp nondecreasingy) ordered () If the agorithm that diagonaizes the pivot submatrix is any standard (ie, eement-wise) gobay convergent Jacobi method, then Λ = im k A (k) Proof The proof has been moved to the Appendix Theorem 210 impies that the goba convergence probem of the bock Jacobi method reduces to the convergence of the sequence S(A (k) ), k 0, to zero

11 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 11 By inspecting the proofs of the resuts reated to the goba convergence of the standard cycic Jacobi method [32, 11] one finds out that they hod for bock methods, too We summarize those resuts as foows Theorem 211 If a bock Jacobi method converges for some cycic strategy, then it converges for a strategies that are weak equivaent to it The bock methods defined by equivaent cycic strategies generate the same matrices after each fu cyce and within the same cyce they produce the same sets of orthogona eementary matrices Indeed, the proof for the standard Jacobi method essentiay uses the fact that commuting pivot pairs resuts in commuting the Jacobi rotations Simiary, the proof for the bock method uses the fact that commuting pivot pairs(i, j) and(p, q) impy commuting orthogona eementary matrices U ij and U pq For the convergence of the diagona eements one shoud presume conditions ike those in Theorem 210 for the kerne agorithms The second part of the theorem hods because it presumes that the bock Jacobi method uses the same kerne agorithm Theorem 211 aso hods for the quasi-cycic methods provided that the care is taken for the bocks that are annihiated more than once within a sweep A sufficient condition for the goba convergence of the seria standard Jacobi methods is the existence of a stricty positive uniform ower bound for the cosines of the rotation anges (see [10]) For the seria bock Jacobi methods a sufficient condition for the goba convergence is that the transformation matrices U k from reation (25) have a stricty positive uniform ower bound for the singuar vaues of the diagona bocks [5] That condition aso appears in the goba convergence anaysis of more genera seria Jacobi-type methods [18] Unitary eementary bock matrices which satisfy such property are caed UBC (uniformy bounded cosine) transformation matrices in [5] In the same paper it was shown that for every unitary matrix of order n and every partition ς = (n 1,n 2 ) of n, there exists a permutation matrix J such that for the eading n 1 n 1 bock of Ũ = UJ one has σ min (Ũ11) γ ς > γ n > 0, γ ς = 3 (4 n 1 +6n2 1)(n 2 +1), γ n = n +26 The second inequaity, which invoves γ n, has been proved in [18] Hence, every unitary eementary bock matrix can be made UBC by an appropriate permutation of its nontrivia coumns In this paper we wi use UBC transformation matrices Therefore, for each 0 < 1 we introduce the cass UBCE π ( ) of eementary unitary UBC bock matrices as foows The unitary eementary bock matrix U ij from reation (23) beongs to the cass UBCE π ( ) if hods, where σ min (U ) = σ min (U jj ) γ ij > γ ni +n j γ n (215) γ ij = 3 (4 n i +6nj 1)(n j +1) (216) If π is understood, we wi write UBCE( ), whie if = 1, the wi aso be omitted from the notation In definitions, statements and ordinary text, at every appearance of we wi automaticay assume that 0 < 1 We wi use the same notation when U ij is rea, ie, orthogona Note that U k from reation (25) is an orthogona eementary matrix defined by the pivot pair (i,j) where i = i(k), j = j(k) To make U k a UBCE transformation, one has to find the permutation J k and then compute U k J k This can be accompished (see [5]) by performing the QR factorization with coumn pivoting of [U (k) ] from reation (26) That QR factorization U (k) ij

12 12 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ yieds Ĵk, which then defines J k as J k = E(i,j,Ĵk) Then Ûk aso beongs to the cass UBCE ςij, where ς ij = (n i,n j ) If (n i,n j ) is understood, ς ij wi be omitted One easiy verifies that U k J k diagonaizes the pivot submatrix Â(k) and the simiarity transformation with J k does not change the Frobenius norm of the affected bocks of U T k A(k) U k In addition, one can show that once S(A (k) ) is sufficienty sma, and the diagona eements affiiated with the same eigenvaue occupy successive positions aong the diagona, the permutations J k are no onger needed (see [5]), ie, J k can be taken to be the identity If π = (1,1,,1), then one can repace γ ij by 2/2 The uniform bound 2/2 is the one from the known Forsythe-Henrici condition [10] Remark 212 The parameter has been introduced for severa reasons First, it simpifies the convergence anaysis of the more genera iterative process described in Section 5 Second, as wi be shown in Sections 3 and 4, the convergence proofs for the symmetric bock Jacobi method hod for any 0 < 1 Finay, for the case = 1, the determination of the permutation J k requires the QR factorization with coumn pivoting of an n i (n i +n j ) matrix Possiby, for some smaer, an appropriate permutation matrix coud be obtained at a smaer cost 24 Bock Jacobi annihiators and operators The Jacobi annihiators and operators have been introduced in [22] as a too for proving the goba and quadratic convergence of the coumncycic Jacobi method Later they have been used for proving the goba convergence of some norm-reducing Jacobi-type methods for genera matrices [12, 14] In [17, 18, 1] they have been generaized to cope with the bock Jacobi methods Here we define a cass of the Jacobi annihiators and operators designed precisey for the bock Jacobi method for symmetric matrices They wi be referred to as bock Jacobi annihiators and operators This wi move us to a more genera point of view of the bock Jacobi methods, which can be used in the convergence considerations First et us introduce some notation For an arbitrary p q matrix X, we define the coumn vector comprising the coumns of X, co(x) = [x 11,x 21,,x p1,,x 1q,,x pq ] T Let π = (n 1,,n m ) be a partition of n Let S n denote the rea vector space of symmetric matrices of order n Let A = (A rs ) S n be as in reation (22) Its bock-matrix partition is determined by π We define the vector-vaued function vec π as foows (see [17, 18]), Then vec π (A) = c 2 c 3 c m vec π : S n R K,, where c j = K = N m i=1 co(a 1j ) co(a 2j ) co(a j 1,j ) n i (n i 1) 2, 2 j m (217), N = n(n 1), (218) 2 is a inear operator Note that vec π (A) contains a off-diagona eements from the upper bock trianguar part of A They are arranged in vec π (A) using doube coumn-wise ordering, one with respect to the bocks in A, the other with respect to the eements within each bock The function vec π is a surjection, but not an injection In order to make it bijection, we restrict it to the vector subspace S 0,n of S n, consisting of a matrices from S n whose diagona bocks (with respect to the bock-matrix partition defined by π) are zero Let vec π,0 = vec π S0,n Obviousy, the function vec π,0 is an invertibe inear operator from S 0,n to R K In the foowing text we wi often assume that the

13 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 13 partition π is known and it wi be omitted from the notation However, it wi be denoted whenever an additiona partition is aso considered If a R K and A = vec 1 0 (a) then A is obtained from a usingthe bock-matrix partition defined by π and the doube coumn-wise ordering, as is described in reation (217) The diagona bocks are set to zero and the whoe matrix is set to be symmetric Obviousy, A is uniquey determined by a Beside the inear operators vec and vec 0, we wi make use of the inear operator N ij : R n n R n n, which aso uses the bock-matrix partition defined by π and sets the pivot submatrix of the argument matrix to zero When appied to A S n, N ij (A) sets the bocks A ij, A ji, A and A jj to zero Definition 213 Let π = (n 1,,n m ) be a partition of n, et [ ] U11 U Û = 12 ni U12 T, U 22 n j be an orthogona matrix of order n i +n j and et U = E(i,j,Û) be the corresponding eementary bock matrix The transformation R ij (Û) determined by R ij (Û)(vec(A)) = vec(n ij(u T AU)), A S n, is caed the ij-bock Jacobi annihiator For each pair 1 i < j m, R ij = {R ij (Û) } Û is orthogona matrix of order ni +n j is the ij-cass of the bock Jacobi annihiators If a Û are restricted to the cass UBCE ςij ( ), then the resuting ij-cass is denoted by R UBCEπ( ) ij Given i, j, Û, the foowing agorithm computes the vector a = R ij (Û)a for a RK It is based on the formua R ij (Û)a = vec(n ij(u T vec 1 0 (a)u)), which can be taken as an equivaent definition of R ij (Û) Agorithm 214 Computing R ij (Û)a a R K A = vec 1 0 (a) for r = 1,,m do A ri = A riu +A rj U ji A rj = A riu ij +A rj U jj end for for r = 1,,m do A ir = UT A ir +Uji TA jr A jr = UT ij A ir +Ujj TA jr end for A ij = 0, A ji = 0, A = 0, A jj = 0 a = vec(a ) % an arbitrary vector % this invokes the modue which computes % the symmetric matrix A = vec 1 0 (a) % this part of code computes % A = U T AU, U = E(i,j,Û) % Û is partitioned as in reation (24) % this part updates A, A N ij (A ) % the modue which computes vec(a ) Matrix A from Agorithm 214 has the same partition as A Note that the mapping a a is a composition of inear transformations Therefore, given a basis in R K, R ij (Û) can be represented

14 14 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ by some square matrix of order K We wi choose the canonic basis (e), which consists of the coumns of I K, and denote the obtained matrix by the bod symbo Hence, R ij (Û)(a) = R ij(û)a, a RK (219) We wi ca the matrix R ij (Û) by the same name, the ij-bock Jacobi annihiator, and the appropriate cass of matrices wi be denoted by R ij = {R ij (Û) Û is an orthogona matrix of order ni +n j } If a Û are restricted to UBCE( ), the obtained cass of bock Jacobi annihiators is denoted by R UBCE( ) ij In the seque, every mention of the bock Jacobi annihiator wi refer to the matrix R ij (Û) from reation (219) The foowing theorem reveas the structure of a bock Jacobi annihiator It is a simpification of [17, Theorem 21] and its proof can be found in [1] The theorem utiizes the function τ(i,j) = (j 1)(j 2)/2 + i, 1 i < j m, and the Kronecker product of matrices The vectors of ength K and the bock Jacobi annihiators of order K carry the bock-partition determined by (n 1 n 2,n 1 n 3,n 2 n 3,,n m 1 n m ) The spectra norm is denoted by 2 Theorem 215 ([17, 1]) Let π = (n 1,,n m ) be the partition of n and et K, N be integers defined in reation (218) Let (i,j) P m, R R ij, R = R(Û), where Û is an orthogona matrix of order n i +n j Then R differs from the identity matrix I K in exacty m 1 principa submatrices, which are given by the foowing reations: R τ(i,j),τ(i,j) = 0, [ ] Rτ(r,i),τ(r,i) R τ(r,i),τ(r,j) R τ(r,j),τ(r,i) R τ(r,j),τ(r,j) [ ] Rτ(i,r),τ(i,r) R τ(i,r),τ(r,j) R τ(r,j),τ(i,r) R τ(r,j),τ(r,j) [ ] Rτ(i,r),τ(i,r) R τ(i,r),τ(j,r) R τ(j,r),τ(i,r) R τ(j,r),τ(j,r) [ U T = I nr Uji T I n r [ = U T ij I n r U T jj I n r ], 1 r i 1, I nr U T S(Uji T I ] n r ) S(I nr Uij T) UT jj I, i+1 r j 1, n r Inr U =[ T I nr Uji T I nr Uij T I nr Ujj T ], j +1 r m, where I ni e T 1 S = = [ I nr ē 1 I nr ē ni ], I ni e T n r I nr ẽ T 1 S = I nr ẽ T n j = [ I nj e 1 I nj e nr ] Here, e i, ē i and ẽ i denote the ith coumn of I nr, I ni and I nj, respectivey The matrix R satisfies R 2 = 1, except in the case m = 2, (i,j) = (1,2), when R = 0

15 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 15 Exampe 216 Let A R 8 8, π = (2,2,2,2), i = 1, j = 2 Then K = 24 and R = R 12 (Û) = U11 T U T 21 U11 T U T 21 U12 T U T 22 U12 T U T 22 U T 11 U21 T U11 T U21 T U 12 T U22 T U12 T U22 T where U 11, U 12, U 21, U 22 are the bocks of order 2 of Û R4 4 and Û is orthogona We see that R is, up to simiarity transformation with permutation, a direct sum of an orthogona matrix and the zero matrix Therefore, R 2 = 1, except in the case m = 2, (i,j) = (1,2) when it is the zero matrix Coroary 217 Let π, (i,j) P m and R R ij be as in Theorem 215 Then R T R ij Moreover, if R R UBCE( ) ij, then R T R UBCE( ) ij Proof The proof has been moved to the Appendix The bock Jacobi annihiators are used to define the bock Jacobi operators, which make up our too for proving the goba convergence of the bock Jacobi methods Definition 218 Let π = (n 1,,n m ) be a partition of n and et Then O = (i 0,j 0 ),(i 1,j 1 ),,(i T 1,j T 1 ) O(P m ), T M = m(m 1) 2 J O = {J J = RiT 1 j T 1 R i1 j 1 R i0 j 0, R ik j k R ik j k, 0 k T 1} is caed the cass of bock Jacobi operators associated with the sequence O The matrices J of order K from J O are the bock Jacobi operators If each R ij in J O is repaced by R UBCE( ) ij, then the notation J UBCE( ) O wi be used An eement J J O wi sometimes be written J O The foowing emma reveas some properties of the bock Jacobi operators The spectra radius of a square matrix X is denoted by spr(x) Lemma 219 ([18, 1]) Let π = (n 1,,n m ) be a partition of n, O,O O(P m ) and O w O Take J O J O and et J O be comprised of the same bock Jacobi annihiators as J O Then spr(j O ) = spr(j O ) If O O, then J O = J O,

16 16 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ Proof The proof is the same as the proof of [18, Lemma 44] If the spectra norm is used instead of the spectra radius, then we have the foowing resut Proposition 220 Let π = (n 1,,n m ) be a partition of n, O,O O(P m ) and O w O Let O and O be inked by the chain O = O 0,O 1,,O r = O, as in reation (27) Suppose that in the chain there are exacty d pairs of neighboring terms that are shift equivaent If J 2 µ π, for a J J UBCE( ) O, then for any d+1 bock Jacobi operators from J UBCE( ) O J 1J 2 J d+1 2 µ π,, The constant µ π, may depend ony on π and one has J 1,,J d+1 J UBCE( ) O Proof The proof is the same as the proof of [18, Lemma 48()] The roe of the set Ψ π from [18, Lemma 48()] is payed by the set i<jr UBCE( ) ij Proposition 221 Let π = (n 1,,n m ) be a partition of n, O O(P m ) and suppose J 2 µ π, for a J J UBCE( ) O, where µ depends on π and Then J 2 µ π, for a J J UBCE( ) O The assertion hods provided that in both appearances the spectra norm is repaced by the spectra radius be arbi- Proof Suppose O = (i 0,j 0 ),(i 1,j 1 ),,(i T 1,j T 1 ) O(P m ) and et J J UBCE( ) O trary Then J = R i(0),j(0) (Û0)R i(1),j(1) (Û1) R i(t 1),j(T 1) (ÛT 1), for some orthogona UBCE( ) matrices Ûk, 0 k T 1, of appropriate sizes If we show that J T J UBCE( ) O, first caim wi foow from J 2 = J T 2 µ π,, whie the part about the spectra radius wi be a consequence of spr( J) = spr( J T ) µ π, Note that [ ] T J T = R i(0),j(0) (Û0)R i(1),j(1) (Û1) R i(t 1),j(T 1) (ÛT 1) = [R i(t 1),j(T 1) (Û T 1 )] T [R i(1),j(1) (Û 1 )] T [R i(0),j(0) (Û 0 )] T By Coroary 217 we know that [R ij (Ûk)] T R UBCE( ) ij, hence J T J UBCE( ) O 241 Permutation equivaence and the bock Jacobi operators Here we derive a simiar resut for the bock Jacobi operators J O and J O(p) Theorem 222 Let π = (n 1,,n m ) be a partition of n and take O O(P m ) Let p be a permutation of S m and set Õ = O(p) (i) If J 2 µ π, for a J J UBCEπ( ) O, where µ π, ony depends on π and, then J 2 µ π, for any J J UBCEπp ( ) Õ () If J 1 J 2 J d+1 2 µ π,, for a J 1,,J d+1 J UBCEπ( ) O, where µ π, ony depends on π and, then J 1 J2 J d+1 2 µ π, for any J 1,, J d+1 J UBCEπp ( ) Õ

17 CONVERGENCE OF THE CYCLIC AND QUASI-CYCLIC BLOCK JACOBI METHODS 17 The assertions aso hod provided that every appearance of the spectra norm is repaced by the spectra radius Proof (i) Let O = (i 0,j 0 ),(i 1,j 1 ),,(i T 1,j T 1 ), so that Õ = (p(i 0),p(j 0 )),(p(i 1 ),p(j 1 )),, (p(i T 1 ),p(j T 1 )) Let J J UBCEπp ( ) be arbitrary Then Õ J = R p(it 1 ),p(j T 1 )(ÛT 1) R p(i1 ),p(j 1 )(Û1)R p(i0 ),p(j 0 )(Û0), (220) where Ûk is an orthogona UBCE πp ( ) matrix of order n p(ik ) +n p(jk ) for each 0 k T 1 Let a R K be an arbitrary nonzero vector, and consider the computation of a = Ja Using Agorithm 214, the vector a can be obtained by the foowing procedure: Compute the symmetric matrix A (0) A = vec 1 π p,0 (a) Recursivey compute: A (k+1) = N p(i)p(j) (U T k A(k) U k ), k = 0,1,,T 1 Compute the vector a = vec πp (A (T) ) Here U k = E(p(i k ),p(j k ),Ûk), 0 k T 1, and the matrices A and A (k) carry the matrix bock-partition defined by π p Let P be the matrix from reation (212), which is defined by Pe t = e p(t), 1 t n, where p is defined as in Section 221, and p is from the statement of this emma Let X be a square matrix of order n, partitioned in accordance with π p Then for any 1 s,t m the transformation X P T XP changes the partition from π p to π and moves the bock X p(s)p(t) to the (s,t) position Therefore, we have ( ) ) P T A (k+1) P = P T N p(i)p(j) Uk T A(k) U k P = N ij (P T Uk T A(k) U k P (221) ) = N ij ((P T U k P) T (P T A (k) P)(P T U k P), k = 0,1,,T 1 If we set Ā(k) = P T A (k) P, Ā = P T AP and Ū(k) = P T U (k) P, then recurrence (221) takes the form Ā (k+1) ([Ū(k) = N ij ] T Ā (k) Ū (k)), k = 0,1,,T 1; Ā (0) = Ā (222) Obviousy, we have vec π (Ā) = vec π(p T AP) = Pa for some permutation matrix P of order K Appying the vec function to reation (222), one obtains where R ij ( Ū k ) R UBCEπ( ) fina form ā (k+1) = R i(k),j(k) ( Ū k )ā (k), k = 0,1,,T 1; ā (0) = Pa, (223) ij This process is associated with the sequence O and resuts in the Pa = J O Pa, JO J UBCEπ( ) O Because a is an arbitrary vector, we have J = P T JO P This impies J 2 = J O 2 µ π, Since, J and JO are simiar, their spectra radius is the same ()Theproofissimiartotheproofof(i) Westartourconsiderationwithanarbitrarynonzero vector a R K and consider the computation of a = J 1 J2 J d+1 a Since we have d+1 bock Jacobi operators, we wi use atogether (d+ 1)T bock Jacobi annihiators A we have to change in the proof of (i) is the the range of the index k in the reations (220) (223): instead of T 1 its argest vaue wi be (d+1)t 1 At the end we wi have J 1 J2 J d+1 = P T J 1 J 2 J d+1 P and the concusion wi foow

18 18 VJERAN HARI AND ERNA BEGOVIĆ KOVAČ 3 Generaized seria strategies The aim of this section is to significanty enarge the cass of the known convergent cycic pivot strategies, namey the seria ones and those that are weak equivaent to them (the so-caed weak wavefront strategies [32]) We study severa casses of cycic pivot strategies, which are generaizations of the seria ones The first (resp second) of those casses is defined by the set B c (resp B r ) of pivot orderings which arise from coumn-wise (resp row-wise) orderings of P m The other two are defined by the first two using reverse orderings Once the goba convergence of the bock Jacobi method under these strategies is proved, one can easiy expand the obtained set of pivot strategies using the theory of equivaent strategies 31 The cass B c We start with the cass of cycic strategies that choose (1,2)-bock as the first pivot bock, then choose a bocks from the second bock-coumn in some order, etc At the ast stage they choose a bocks from the ast bock-coumn in some order For the precise definition of that cass, we denote the set of a permutations of the set { 1, 1 + 1,, 2 } by Π ( 1, 2 ) Let B c = { O O(P m ) O = (1,2),(τ3 (1),3),(τ 3 (2),3),,(τ m (1),m),, (31) (τ m (m 1),m), τ j Π (1,j 1), 3 j m } The set B c is a part of the cass of coumn-wise orderings with permutations of the set P m, which wi be described in Definition 34 A typica ordering O B c (6) is represented by M O beow The second matrix MÕ is defined by some Õ w O Its purpose is to see how far from the seria structure this equivaence can push O M O = , MÕ = From M O we can see that the permutation τ j from (31) is inked with the jth bock-coumn of the matrix The next theorem proves the goba convergence of the bock Jacobi method under the cycic strategies I O defined by the orderings O B c Theorem 31 Let π = (n 1,,n m ) be a partition of n and et O B c Let A S n be partitioned as in reation (22) Suppose that A is obtained from A by appying one sweep of the cycic bock Jacobi method defined by the strategy I O If a transformation matrices are from the cass UBCE( ), then there are constants η π, (depending ony on π and ) and η n, (depending ony on n and ) such that S 2 (A ) η π, S 2 (A), 0 η π, < η n, < 1 Proof The proof is engthy and has been moved to the Appendix We have to expain why we use the two bounds satisfying µ π, < µ n, < 1 Reca that each bock Jacobi method is defined by some partition π of n Different partitions define different bock Jacobi methods, even in the case when the pivot orderings are the same The second bound µ n, can be used in the goba convergence statements for the bock Jacobi method, when the order n of the initia matrix is known, whie about the pivot ordering it is ony known that it beongs