Centro de Processamento de Dados, Universidade Federal do Rio Grande do Sul,

Transcription

1 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD RUDNEI DIAS DA CUNHA Computing Laboratory, University of Kent at Canterbury, U.K. Centro de Processamento de Dados, Universidade Federal do Rio Grande do Sul, Brasil and TIM HOPKINS Computing Laboratory, University of Kent at Canterbury, U.K. Abstract. In this paper we investigate the performance of four dierent acceleration techniques on a variety of linear systems. Two of these techniques have been proposed by Dancis [] who uses a polynomial acceleration together with a sub-optimal. The two other techniques discussed are vector accelerations the " algorithm proposed by Wynn [9] and a generalisation of Aitken's algorithm, proposed by Graves-Morris [3]. The experimental results show that these accelerations can reduce the amount ofwork required to obtain a solution and that their rates of convergence are generally less sensitive to the value of than the straightforward method. However a poor choice of can result in particularly inecient solutions and more work is required to enable cheap estimates of a eective parameter to be obtained. Necessary conditions for the reduction in the computational work required for convergence are given for each of the accelerations, based on the number of oating-point operations. It is shown experimentally that the reduction in the number of iterations is related to the separation between the two largest eigenvalues of the iteration matrix for a given. This separation inuences the convergence of all the acceleration techniques above. Another important characteristic exhibited by these accelerations is that even if the number of iterations is not reduced signicantly compared to the method, they are competitive in terms of number of oating-point operations used and thus they reduce the overall computational workload. Key words:, Acceleration. Introduction Consider the solution of the non-singular, symmetric, positive-denite system of n linear equations Ax = b () by the iteration (I + D ; A L )x (k+) = ; ( ; )I ; D ; A U x (k) + D ; b k =0 ::: () where D = diag(a), A L and A U are the strictly lower and upper triangular parts of A and is the relaxation parameter. Convergence of the method is guaranteed for 0 <<. The iterative method is commonly used for the solution of large, sparse, linear systems that arise from the approximation of partial dierential equations. Its

2 RUDNEI DIAS DA CUNHA AND TIM HOPKINS rate of convergence is dependent on the value chosen for the iteration parameter. Following Young [5], the minimum number of iterations required for convergence is obtained when this parameter has an optimal value, b,which minimises the spectral radius of the iteration matrix, L =(I + D ; A L ) ; ; ( ; )I ; D ; AU : The value of b may be obtained from the largest eigenvalue,, of the Jacobi iteration matrix B = ;D ; (A L + A U ) using b = p + ; However computing b is relatively expensive in most cases. Adaptive procedures exist that can be used to update some initial approximation to b,asintheitpack C package [4], but some of the initial iterations have large error vectors (when compared to using some >). In this case some of the initial estimates of the solution are \wasted" during the iterative process and this may be undesirable. An alternative is to use an acceleration technique that may not be so sensitive to the choice of. The rst two acceleration techniques, detailed in x were proposed by Dancis [] and follow the usual approach of trying to select some for the iteration. We show that for one of his techniques the selection of is not as sensitive as expected. In section 3 we look at an extension of the " algorithm of Wynn applicable to vector and matrices iterations ([8][9]) and in x4 a generalisation of Aitken's algorithm as proposed by Graves-Morris [3] is described. These are vector accelerations and thus do not give a prescription for selecting. Our investigation is aimed at establishing how sensitive these accelerations are with respect to the value chosen for. Section x6 describes the test problems used in the investigation and the results obtained from the experiments. In section x7 we summarise the results and draw some conclusions. (3). Dancis's Acceleration Dancis proposes the use of a polynomial accelerator with, being chosen such that the coordinate of the error vector, corresponding to the largest eigenvalue of L, is annihilated. Dancis recommends that = +, where is the second largest eigenvalue of L. By computing the second largest eigenvalue,, of the Jacobi iteration matrix we can obtain the value of using Equation (3), with replaced by. Two dierent accelerations are proposed. The rst, which we refer to as P, is as follows. Perform r ; iterations and then apply x (r) =~x = ; r x (r;) ; r ; r x (0) (4) continuing with the iterations thereafter. The second acceleration (referred to as P) is obtained by ^x (i+) = ; ai x(i+) ; ai+ + ai ( ; a) ; a i+ x(0) i = ::: r (5)

3 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD 3 where a = ;. After r steps of the above iteration have been performed, we set x (r+) =^x(r+) and the iteration is used from then on. Dancis propose that the value of r be chosen according to a spectral radius analysis (see [, page 87]) its value is given by min =( ; ) m;r (( ; ) r + r r ) =( ; r )( b ; ) m r = ::: (6) where m is some arbitrarily chosen number of iterations. 3. Wynn's " Algorithm In 96 Wynn proposed an extension of the " algorithm ([8]) for vector and matrix iterations ([9]). Consider a sequence S = fs k g k=0 which isslowly convergent. If we dene the sequences " (k) ; = f0g k= and "(k) 0 = S then a new sequence " (k) i+ is generated by " (k) i+ =("(k+) i ; " (k) i ) ; + " (k+) i =0 ::: k =0 ::: (7) i; In certain circumstances, the sequences " (k) i, i = ::: converge faster to the limit of S. We investigate the behaviour of the " (k) sequence obtained from vectors generated by. We can express the new vector iterates, generated by two successive applications of (7) to three vectors, x (k;) " (k+) =, x(k) and x(k+),as (x (k+) ; x(k) ); ; (x (k) ; ); ; x(k;) (k) + x (8) where u ; =u=(j u j ) is the Moore-Penrose generalised inverse and u denotes the complex conjugate. In [3] it is shown that the value of should be taken as + i when using a sequence f" (k) i g i. For the acceleration of shown above, with "(k) as the sequence to be used, then =+, which isthevalue proposed by Dancis. 4. Graves-Morris's Acceleration Graves-Morris suggests, [3, page 5], that the sequence of vectors x (k) generated by () may be accelerated using t (k) = x (k;) (x (k;) ) ; x (k;) x (k;) x(k;) k = 3 ::: (9) where x (k) = x (k+) ; x (k), which is a generalisation of Aitken's process [6]. We will refer to (9) as the G-M iteration. Experimental results on a few model problems given in [3] show that, for the G-M iteration, using the value of which maximises the separation between the largest eigenvalue of the iteration matrix and the other eigenvalues, a reduction in the number of iterations needed for convergence occurs. We investigate whether this reduction is also observed in larger, practical problems and, if so, how critical the eigenvalue separation is to the rate of convergence. The value of is chosen in a similar way as for the " algorithm, following [3].

4 4 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 5. Conditions for the Eectiveness of the Acceleration Techniques Our aim is to discover whether we can reduce the amount of computation needed by to solve a system of the form () by using one of the accelerations techniques. For instance, using the G-M acceleration one might expect (intuitively) that if at least one iteration is saved, the computing workload is reduced by a factor of roughly n multiplications compared to the iteration. We present below necessary conditions for the techniques discussed to require a smaller number of oating point multiplications (ops) than. These conditions are obtained in terms of the number of iterations (k) and the number of ops per iteration. For and each of the accelerations we consider that the total number of ops is ops = k (n + n)o ops P = ; k P (n + n)+n O ops P = ; k P (n + n)+rn O ops " = ; k " (n +7n) ; 6n O ops G-M = k G-M (n +3n)O where O represents a oating-point multiplication. It is easy to show that for k >k accel where accel denotes any of the acceleration techniques, we have ops > ops accel if and only if the following conditions are satised P: n>( + k P ; k )=(k ; k P ) P: n>(r + k P ; k )=(k ; k P ) " : n>(7k " ; k ; 6)=(k ; k " ) G-M: n>(3k G-M ; k )=(k ; k G-M )

5 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD 5 6. Description of the Test Problems and Experiments In this section we present the results obtained from solving a set of problems using MATLAB implementations of the above methods. Three of the problems are taken from the Boeing-Harwell library [] and for these we used Fortran77 and BLAS routines to implement the methods and LAPACK subroutines to compute the eigenvalues. The problems solved present dierent characteristics with respect to the distribution of eigenvalues and are of interest to the analysis of the acceleration techniques discussed. In each experiment, we describe the system of linear equations used, the value of b computed using (3) and the convergence criteria. The results are tabulated for each method in terms of number of iterations to achieve convergence and the ops counting of and the ratios between the ops counting of each acceleration with respect to. For the analysis of the G-M iteration, we provide a graph showing the two largest eigenvalues of the iteration matrix, and,such that j j > j j.for these test problems, the value of r for the Dancis's accelerations was found to be. 6.. Varga's Problem This is a system of order n = 6 described in [7, Appendix B] derived from the ve-point nite-dierence discretisation y)u + (x y)u = S(x in the region R =(0 :)(0 :) subject to the boundary is the outward normal. The functions D, and S are as shown in Figure. The condition number is (A) =3: Thevalue of b is :977.. Fig.. Region for Varga's problem. D(x,y)= σ(x,y)=0.03 D(x,y)= σ(x,y)=0.0 except where marked D(x,y)=3 σ(x,y)=0.05 S(x,y)=0 everywhere 0.

6 6 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 6... Experiment In this problem, we are solving Ax =0 () which has the zero vector as the unique solution. The initial vector x (0) was set to ( ::: ) T and we iterated until the -norm of the solution vector was less than 0 ;4 (this stopping criterion was used in order to reproduce the behaviour of presented in [7, Appendix B, page 304]). A maximum of 000 iterations was allowed. An impressive reduction in the number of iterations is achieved by the G-M, " and P accelerations. Note that while the minimum number of iterations for and P is obtained when = b, for G-M, " and P this minimum occurs at some < b. TABLE I Varga's problem: number of iterations. G-M " P P : : : : : TABLE II Varga's problem: ops counting. Mops ratio to (%) G-M " P P :09 0:54 3:35 6:07 3:0 00:0 :3059 0:54 :30 4:06 :06 00:0 :5098 0:35 :0 3:4 :80 00:0 :737 0:9 5:84 9:89 5:4 00:0 :977 0:04 9:0 40:6 8:59 00:08 ratio to minimum ops(%) :09 369:86 45:93 83:6 4:8 369:94 : :38 3:39 55:36 8:6 364:46 : :08 7:6 8:49 5:83 878:6 : :93 8:3 47:95 5:4 485:0 :977 00:00 9:0 40:6 8:59 00:08

7 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD 7 Fig.. Varga's problem: Eigenvalue separation and number of iterations of the methods. Varga s problem Varga s problem G-M " P P k

8 8 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 6.. Problem TRIDIAG This problem of order n = 00 is the system dened by A = b= () The condition number is (A) =3:9964 and the value of b is : Experiment The starting vector used is x (0) =(0 0 ::: 0) T and the iterations proceed until the -norm of the residual of the solution vector is less than 0 ;0 or the number of iterations exceeded 00. This example shows a situation where b is close to. The experiment shows that in this case little, if any, gain is achieved by the accelerations. Nonetheless even if a single iteration is spared a reduction in the computational eort is veried. TABLE III Problem TRIDIAG: number of iterations. G-M " P P : : :066 3 : : TABLE IV Problem TRIDIAG: ops counting. Mops ratio to (%) G-M " P P :03 0:4 93:48 9:45 9:75 00:08 :0369 0:3 93: 0:08 95:74 00:09 :066 0: 0:98 0:49 00:09 00:09 :0863 0: 97:34 00:86 04:64 00:09 :09 0:3 93: 96:47 04:43 00:09 ratio to minimum ops(%) :03 09:09 0:98 00:86 00:09 09:8 : :55 97:34 05:67 00:09 04:64 :066 00:00 0:98 0:49 00:09 00:09 : :00 97:34 00:86 04:64 00:09 :09 04:55 97:34 00:86 09:8 04:64

9 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD 9 Fig. 3. Problem TRIDIAG: Eigenvalue separation and number of iterations of the methods. Problem TRIDIAG Problem TRIDIAG G-M " P P k

10 0 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 6.3. Problem NOS4 This problem is taken from the Harwell-Boeing sparse matrix collection [, pp ]. It is derived from a nite-element approximation to a structural engineering problem. The system has order n = 00 and its condition number is (A) =: The RHS vector was chosen as ( 0 0 ::: 0) T. The value of b is : Experiment In this problem we iterated until the -norm of the residual of the solution vector was less than 0 ;4 or the number of iterations exceeded 000. The initial estimate of x was (0 0 ::: 0) T. This example shows a similar behaviour to that of Varga's problem. However in this case the " acceleration is worse than and P and P fail to produce any acceleration. TABLE V Problem NOS4: number of iterations. G-M " P P : : : : : TABLE VI Problem NOS4: ops counting. Mops ratio to (%) G-M " P P :0978 :0 64:86 60:69 8:99 8:99 :933 7:55 8:90 60:74 67:38 67:38 :4889 4:94 6:8 57:9 409:00 409:00 :6845 :86 :34 53:09 706:7 706:7 :880 0:96 0:57 7:03 05:8 05:8 ratio to minimum ops(%) : :84 750:36 858:9 05:8 05:8 : :37 7:58 65:65 05:8 05:8 : :74 35:6 8:89 05:8 05:8 : :89 66:56 456:04 05:8 05:8 :880 00:00 0:57 7:03 05:8 05:8

11 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD Fig. 4. Problem NOS4: Eigenvalue separation and number of iterations of the methods. Problem NOS NOS4 problem G-M " P P k

12 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 6.4. Problem 685 BUS This problem is taken from the Harwell-Boeing sparse matrix collection [, pp. 7]. The coecient matrix is derived from the modelling of a power system network. The system has order n = 685 and its condition number is (A) =4: The RHS vector was chosen as ( 0 0 ::: 0) T. The value of b is : Experiment In this experiment we used the same stopping criteria as in problem NOS4. It shows a behaviour similar to that exhibited in Varga's and NOS4 problems except for the " acceleration which was always worse than except at = b. TABLE VII Problem 685 BUS: number of iterations. G-M " P P : : : : : TABLE VIII Problem 685 BUS: ops counting. Mops ratio to (%) G-M " P P : :8 30:79 00:87 00:00 00:00 : :00 6:04 36:4 35:3 35:3 : :98 3:66 89:6 87:97 87:97 :855 57:04 :45 75:56 365:63 365:63 : :35 95:60 97:33 69:59 69:59 ratio to minimum ops(%) :066 69:59 360: 79:8 69:59 69:59 : :9 5: 79:8 69:59 69:59 :6394 6: 47: 79:8 69:59 69:59 :855 39:88 68:6 56:59 69:59 69:59 : :00 95:60 97:33 69:59 69:59

13 A COMPARISON OF ACCELERATION TECHNIQUES APPLIED TO THE METHOD 3 Fig. 5. Problem 685 BUS: Eigenvalue separation and number of iterations of the methods. Problem 685 BUS Problem 685 BUS G-M " P P k

14 4 RUDNEI DIAS DA CUNHA AND TIM HOPKINS 7. Summary We summarise the results presented in x6. The main points are as follows. The P iteration performed poorly in the test problems used in this paper. The other three methods generally outperformed the basic method with the G-M acceleration showing the most consistent improvements.. The G-M, " and P iterations almost invariably reduced the number of iterations required to obtain a specied accuracy when < b. In some cases this reduction was observed for = b. 3. As with the basic method, the G-M, " and P iterations are poor if chosen is greater than b. 4. The reduction in the number of iterations using the G-M method is proportional to the separation of and. Since the " and P iterations produce similar behaviour to G-M, we believe this separation also has an inuence on the convergence properties of these methods. Note that as the separation between and decreases all three iterations exhibit similar behaviour to. The experimental results presented show that the Graves-Morris's acceleration technique is the most attractive of the techniques discussed here, from the point of view both of the overall amount of computational work required and the range of the parameter for which the rate of convergence is improved. Though the number of experiments performed was small we believe the results indicate that these accelerations of the method are eective and may be applicable to other systems. Acknowledgements We would like to thank Peter Graves-Morris for his valuable comments which helped to improve this paper. References. J. Dancis. The optimal is not best for the iteration method. Linear Algebra and its Applications, 54-56:89{845, 99.. I.S. Du, R.G. Grimes, and J.G. Lewis. Users' guide for the Harwell-Boeing sparse matrix collection. Report TR/PA/9/86, CERFACS, October P.R. Graves-Morris. A review of Pade methods for the acceleration of convergence of a sequence of vectors. Report No. NA 9-3, Department of Mathematics, University of Bradford, October R.G. Grimes, D.R. Kincaid, and D.M. Young. ITPACK.0 user's guide. Report No. CNA-50, Center for Numerical Analysis, University of Texas at Austin, August L.A. Hageman and D.M. Young. Applied Iterative Methods. Academic Press, New York, D.R. Kincaid and W. Cheney. Numerical Analysis. Brooks/Cole, Pacic Grove, R.S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Clis, P. Wynn. On a device for computing the em(sn) transformation. Mathematical Tables and Aids of Computation, 0:9{96, P. Wynn. Acceleration techniques for iterated vector and matrix problems. Mathematics of Computation, 6:30{3, 96.