An Implicit Multi-Step Diagonal Secant-Type Method for Solving Large-Scale Systems of Nonlinear Equations

Transcription

1 Applied Mathematical Sciences, Vol. 6, 2012, no. 114, An Implicit Multi-Step Diagonal Secant-Type Method for Solving Large-Scale Systems of Nonlinear Equations 1 M. Y. Waziri, 2 Z. A. Majid and 3 H. Aisha 2 Institute for Mathematical Research, Universiti Putra Malaysia Serdang, Selangor, Malaysia 1,2 Department of Mathematics, Faculty of Science Universiti Putra Malaysia Serdang, Malaysia 1,3 Department of Mathematical Sciences Faculty of Science, Bayero University Kano, Nigeria mywaziri@gmail.com Abstract This paper presents an improved diagonal Secant-like method using two-step approach for solving large scale systems of nonlinear equations. In this scheme, instead of using direct updating matrix in every iteration to construct the interpolation curves, we chose to use an implicit updating approach to obtain an enhanced approximation of the Jacobian matrix which only requires a vector storage. The fact that the proposed method solves systems of nonlinear equations without the cost of storing the weighted matrix can be considered as a clear advantage of this method over some variants of Newton s methods. Mathematics Subject Classification: 65H11, 65K05 Keywords: Diagonal, Newton s Method 1 Introduction Let us consider the systems of nonlinear equations F (x) =0, (1) where F : R n R n is a nonlinear mapping. Often, the mapping, F is assumed to satisfying the following assumptions: A1. There exists an x R n s.t F (x )=0;

2 5668 M. Y. Waziri, Z. A. Majid and H. Aisha A2. F is a continuously differentiable mapping in a neighborhood of x ; A3. F (x ) is invertible. The prominent method for finding the solution to (1), is the Newton s method which generates a sequence of iterates {x k } from a given initial guess x 0 via x k+1 = x k (F (x k )) 1 F (x k ), (2) where k = 0, 1, 2... Nevertheless, Newton s method requires the computation of the matrix which entails the first-order derivatives of the systems. In practice, computations of some functions derivatives are quite costly and sometime they are not available or could not be done precisely. In this case Newton s method cannot be applied directly. Moreover, some significant efforts have been made in order to eliminate the well known shortcomings of Newton s method for solving systems of nonlinear equations, particularly large-scale systems. For example Chord s Newton s method, inexact Newton s method, quasi-newton s method, e.t.c. (see for e.g.[1],[3],[16],[6],[9]). On the other hand, most of these variants of Newton s method still have some shortcomings as Newton s counterpart. For example Broyden s method and Chord s Newton s method need to store an n n matrix and their floating points operations are O(n 2 ). To do away with these disadvantages, a single step diagonally Newton s method has been suggested by Leong et al.[5] and showed that their approach is significantly cheaper than Newton s method and some of its variants. To incorporate the higher order accuracy in approximating the Jacobian matrix, Waziri et al. [7] proposed a two step approach to develop the diagonal updating scheme D k+1 that will satisfy the weak secant equation ρ T k D k+1 ρ k = ρ T k μ k, (3) where ρ k = s k α k s k 1 and μ k = y k α k y k 1. For this purpose, equation 3 is obtained by means of constructing the interpolation quadratic curves x(ν) and y(ν), where x(ν) interpolates the iterates x k 1 x k and x k+1, and y(ν) interpolates the functions F k 1, F k and F k+1. A diagonal updating matrix D k+1 that satisfies 3 is obtained via the following relation D k+1 = D k + (ρt k μ k ρ T k D kρ k ) Tr(Hk 2) H k, (4) where H k = diag((ρ (1) k )2, (ρ (2) k )2,...,(ρ (n) k )2 ), n i=1 (ρ(i) k )4 = Tr(Hk 2 ) and Tr is the trace operation. In order to find the value of ρ, let us consider the following approach by defining the metric as Z m {Z T MZ}. (5)

3 Implicit multi-step diagonal secant-type method 5669 By letting ν 2 = 0 and M = D k, ν 0 and ν 1 can be by computed as follows [7]: and ν 1 = ν 2 ν 1 = x(ν 2 ) x(ν 1 ) Dk = x k+1 x k Dk = s k Dk =(s T k D k s k ) 1 2, (6) ν 0 = ν 2 ν 0 = x(ν 2 ) x(ν 0 ) Dk = x k+1 x k 1 Dk = s k + s k 1 Dk =((s k + s k 1 ) T D k (s k + s k 1 )) 1 2. (7) Waziri et al. [7] incorporated the attribute of the multi-step approach to improve over the single step method. Moreover, they use diagonal updating matrix D k to represent the weighted matrix that is used to parameterize the interpolating polynomials in every iteration. The numerical performance of the multi-step approach is very encouraging. Based on this fact, it is interesting to present an approach which will improve further the Jacobian approximation in diagonal form. This is what lead to the idea of this paper which is organized as follows: the next section present the details of the proposed method. Some numerical results are reported in Section 3. Finally, conclusions are made in Section 4. 2 Derivation process of IMDS update A new two-step diagonal secant-like method for solving large-scale systems of nonlinear equations has been presented in this section. The main idea is to employ an implicit strategy to obtain the weighted matrix than the one proposed by Waziri et al.[7]. The anticipation has been to improve the accuracy of the Jacobian approximation, this is made possible by choosing a new approach for interpolating the curve while maintaining storage and time requirements of the proposed method to only linear order (see [13] for more details). The performance of this approach is highly dependent on how to obtain the {v j } 2 j=0, hence, a diagonal weighted matrix say D k is considered to be used in obtaining more precisely the values of ρ k, μ k and β respectively. It is important to note that, D k is quit different from the D k. Therefore, the weighted diagonal

4 5670 M. Y. Waziri, Z. A. Majid and H. Aisha matrix of this scheme is given as: D k 1 = Diag(D k,ρ k 1,μ k 1 ). (8) Note that, in [7] the weighted matrix is considered as D k. The multi-step implicit updating scheme is given as ν 1 = D k 1 s k }, (9) ν 0 = {s T k {(s k +2s k 1 ) T D k 1 s k +(s T k 1 D k 1 s k 1 )}. (10) where β is denoted by ρ k 1 = s k β ( β +2)s k 1, (11) μ k 1 = y k β ( β +2)y k 1, (12) β = ν 2 ν 0 ν 1, (13) ν 0 we require that { D }}{ k 1 satisfies the secant equation, i.e. ρ k 1 D k 1 = μ k 1. (14) Since it is difficult in letting a diagonal matrix to satisfy the secant equation, in particular, because Jacobian approximations are not usually done in element wise, we consider to use the weak secant condition [4] instead: ρ T k 1 D k 1 ρ k 1 = μ k 1. (15) ρ T k 1 Using the same approach as in [7], the new weighted matrix is obtained as D k 1 = D k + ( ρ T k 1 μ k 1 ρ T k 1 D k ρ k 1 ) G k 1, (16) Tr( G 2 k 1 ) where { G }}{ k 1 = diag((ρ (1) k 1 )2, (ρ (2) k 1 )2...(ρ (n) k 1 )2. Since D k 1 is a diagonal matrix, is clearly that one can obtained it very cheaply. Hence, by using the new

5 Implicit multi-step diagonal secant-type method 5671 weighted matrix, the values of the {v j } 2 j=0 following: can be obtain which lead to the ρ k = s k β2 1+2β s k 1, (17) μ k = y k β2 1+2β y k 1, (18) where β is denoted by β = ν 2 ν 0 ν 1 ν 0. (19) To this end, the updating formula for D is given as follow : D k+1 = D k + (ρt k μ k ρ T k D kρ k ) Tr(G 2 k ) G k, (20) where G k = diag((ρ (1) k )2, (ρ (2) k )2,...,(ρ (n) k )2 ), n i=1 (ρ(i) k )4 = Tr(G 2 k ) and Tr is the trace operation. To safeguard on the possibility of generating undefined D k+1, the proposed updating scheme for D k+1 is used whenever : D k+1 = D k + (ρt k μ k ρ T k Q kρ k) Tr(G 2 k ) G k ; ρ k > 10 4, D k ; otherwise. Hence, the stages of the proposed method are given as follows: Algorithm (IMDS) Step 1 : Choose an initial guess x 0 and D 0 = I, let k := 0. Step 2 : Compute F (x k ). If F (x k ) ɛ 1 stop, where ɛ 1 =10 4. Step 3:Ifk := 0 define x 1 = x 0 D0 1 F (x 0). Else if k := 1 set ρ k = s k and μ k = y k and goto 5. Step 4: Ifk 2 compute ν 1, ν 0 and β via (9), (10) and (13), respectively and find ρ k 1 and μ k 1 using (11) and (12), respectively. If ρ T k 1 μ k ρ k 1 2 μ k 1 2 set ρ k 1 = s k and μ k 1 = y k. Else retains the computed (11) and (12) and compute weighted matrix D k 1 by using (16), Step 5 : Compute ρ k and μ k using (17) and (18). Check If

6 5672 M. Y. Waziri, Z. A. Majid and H. Aisha ρ T k μ k 10 4 ρ k 2 μ k 2, set ρ k = s k and μ k = y k. Else retain the (17) and (18), Step 6 : Check if ρ k 2 ɛ 1 if yes retain D k+1 that is computed by step 5. Else set, D k+1 = D k Step 7 : Set k := k + 1 and goto 2. 3 Numerical results In this section, the performance of IMDS update with that of the Newton s method (NM), Broyden s method (BM), MFDN method proposed by Leong et al.[5] and 2-MFDN update presented by Waziri et al.[7] has been compared. The comparison is based upon the following criterion: Number of iterations and CPU time in seconds. The computations are performed in MATLAB 7.0 using double precision computer. The stopping criterion used is ρ k + F (x k ) (21) The identity matrix has been chosen as an initial approximate Jacobian. Seven (7) different dimensions are performed on the benchmark problems ranging from 50 to The codes also terminates when ever one of the following happens; (i) The number of iteration is at least 600 but no point of x k that satisfies (21) is obtained; (ii) CPU time in second reaches 600, (iii) Insufficient memory to initiate the run. All the results are presented using performance profiles indices in terms of robustness and efficiency as proposed in [2]. In the following, some details on the benchmarks test problems are given as. Problem 1 Trigonometric System of Byeong [14] : f i (x) = cos(x i ) 1 i =1, 2,...,n and x 0 =(0.87, 0.87,..., 0.87) Problem 2 [7]: f i (x) = ln(x i ) cos((1 (1 + (x T x) 2 ) 1 )) exp((1 (1 + (x T x) 2 ) 1 )) i =1, 2,...,n, and x 0 =(2.5, 2.5,.., 2.5). Problem 3 Spares System of Byeong [14] : f i (x) =x i x i+1 1 f n (x) =x n x 1 1, i =1, 2,...,n 1 and x 0 =(0.5, 0.5,..,.5).

7 Implicit multi-step diagonal secant-type method 5673 Problem 4 [7]: f i (x) =n(x i 3) 2 + cos(x i 3) 2 x i 2 exp(x i 3) + log(x 2 i +1) i =1, 2,...,n and x 0 =( 3, 3, 3,..., 3) Problem 5 [15] : f 1 (x) =x 1 f i (x) = cos x i+1 + x i 1, i =2, 3,...n and x 0 =(0.5, 0.5,....5) MFDN IMDS Efficiency indices BM MFDN NM System s Dimensions Figure 1. Efficiency Profile of NM, BM, MFDN, 2-MFDN and IMDS methods as the dimensions increases (in term of CPU time)

8 5674 M. Y. Waziri, Z. A. Majid and H. Aisha MFDN IMDS Robustness index BM MFDN NM System s Dimensions Figure 2. Robustness profile of NM, BM, MFDN, 2-MFDN and IMDS methods as the dimensions increases (in term of CPU time)

9 Implicit multi-step diagonal secant-type method 5675 As expected, Figure 1 and 2 imply that diagonal updating proposed in this paper improves notably over the performance of NM, BM, MFDN and 2-MFDN methods, as having the greatest efficiency and robustness indices. This is not surprising that any method that required storage of full elements of Jacobian or its inverse, particularly when solving large-scale systems, NM, and BM methods are necessary inferior. Moreover, Figure 1 and 2 detailed the growth of the NM, BM, MFDN, 2-MFDN and IDMS methods s CPU time as the dimension gets higher, which shows that IMDS method CPU time increases linearly as apposed by some variants of Newton s method. Therefore, we declare that IDMS method has significantly improved the performance of 2-MFDN method in terms of CPU time for handling large-scale systems of nonlinear equations. 4 Conclusions A new two-step diagonal variant of secant method for solving large-scale system of nonlinear equations has been presented. The fundamental idea behind this approach is to use an implicit updating approach to obtain an enhanced approximation of the Jacobian matrix in diagonal form which only require a vector storage. Numerical testing provides strong indication that the IMDS method exhibits enhanced performance in all the tested problems ( as measured by the CPU time, floating points operations and matrix storage requirement) by comparison with the other variants of Newton s methods, an attribute which becomes more obvious as the dimension of the problems increases. References [1] Dennis, J, E., 1983, Numerical methods for unconstrained optimization and nonlinear equations, Prince-Hall, Inc., Englewood Cliffs, New Jersey [2] Bogle,I.andJ.D.Perkins,1990. A new sparsity preservingquasi-newton updatefor solving nonlinear equations.siam J. Sci. Statist,11: [3] Natasa K. and Zorna L., 2001, Newton-like method with modification of the right-hand vector, J. Maths. Compt. 71, [4] Dennis J.E and Wolkowicz, H.Sizing and least change secant methods, SIAM J. Numer. Anal., 30 (1993), [5] Leong, W. J., Hassan, M. A and Waziri, M. Y., 2011, A matrix-free quasi- Newton method for solving large-scale nonlinear systems, Comput. Math. App. 62 (5):

10 5676 M. Y. Waziri, Z. A. Majid and H. Aisha [6] M. Y. Waziri, W.J. Leong, M. A. Hassan,M. Monsi, A New Newton method with diagonal Jacobian approximation for systems of Non-Linear equations. Journal of Mathematics and Statistics Science Publication. 6 :(3) (2010). [7] M.Y. Waziri, W.J. Leong, M. Mamat, A Two-Step Matrix-Free Secant Method for Solving Large-scale Systems of Nonlinear Equations. Journal of Applied Mathematics, vol. 2012, Article ID , 9 pages, doi: /2012/ [8] Kelley, C. T Iterative Methods for Linear and Nonlinear Equations. PA: SIAM, Philadelphia [9] M.Y. Waziri, Leong, W. J. and Hassan, M. A, 2011, Diagonal Broyden- Like Method for Large-Scale Systems of Nonlinear Equations,Malaysian Journal of Mathematical Sciences 6(1): (2012) [10] Ford, J. A. and Moghrabi, L. A., Alternating multi-step quasi-newton methods for unconstrained optimization, J. Comput. Appl. Math (1997). [11] Ford, J. A. and Moghrabi, L. A., Multi-step quasi-newton methods for optimization, J. Comput. Appl. Math (1994). [12] Farid, M., Leong, W. J. and Hassan, M. A., A new two-step gradienttype method for large scale unconstrained optimization. J. Comput. Math. Appl. 59: [13] Ford, J. A. and S. Thrmlikit, S., New implicite updates in multi-step quasi-newton methods for unconstrained optimization, J. Comput. Appl. Math (2003). [14] Byeong, C. S. Darvishi, M. T. and Chang, H. K A comparison of the Newton-Krylov method with high order Newton-like methods to solve nonlinear systems. Appl. Math. Comput. 217: [15] Roose, A. Kulla, V. L. M. and Meressoo, T., Test examples of systems of nonlinear equations. Tallin: Estonian Software and Computer Service Company. [16] Waziri, M.Y., Leong W. J., Hassan M. A. and Monsi M., Mathematical Problems in Engineering Volume 2011, Article ID , doi: /2011/467017, 12 pages. Received: June, 2012