A modified BFGS quasi-newton iterative formula

Transcription

1 A modified BFGS quasi-newton iterative formula W. Chen Present mail address (as a SPS Postdoctoral Research Fellow): Apt.4, West st floor, Himawari-so, 36-2, Waasato-itaichi, Nagano-city, Nagano-en, , APAN Permanent affiliation and mail address: Dr. Wen CHEN, P. O. Box , iangshu University of Science & echnology, Zhenjiang City, iangsu Province 2203, P. R. China Present chenw@homer.shinshu-u.ac.jp Permanent chenwwhy@hotmail.com Abstract he quasi-newton equation is the very basis of a variety of the quasi-newton methods. By using a relationship formula between nonlinear polynomial equations and the corresponding acobian matrix. presented recently by the present author, we established an exact alternative of the approximate quasi-newton equation and consequently derived an modified BFGS updating formulas. Key words. acobian matrix, polynomial-only nonlinear problems, BFGS quasi- Newton formula.

2 . Introduction Recently, the present author [,2] proved a relationship theorem as stated below heorem. If N ( m ) ( x ) and m x ( ) ( ) are defined as m-order nonlinear polynomial function vector and its corresponding acobian matrix, respectively, then N x m m ( )= ( x ) x ( m) ( ) is always satisfied. In this paper, we will apply this theorem to establish an exact alternative of the traditional approximate quasi-newton equation. It is well nown that the quasi-newton equation is the very basis of various quasi-newton methods [3,4]. herefore, by using the presented alternative equation, we derive the modified BFGS quasi-newton updating formulas. It is noted that the present BFGS formulas are also different from the ones previously given in Chen []. 2. Modified BFGS quasi-newton formula o avoid time-consuming evaluation and inversion of the acobian matrix in each iterative step of the standard Newton method, the quasi-newton method was developed with maintaining a superlinear convergence rate. his ey of such methods is a matrixupdating procedure, of which the BFGS method is the most successful and widely used. he so-called quasi-newton equation is the very fundamental of various quasi-newton 2

3 methods, namely, ( )= ( ) ( ) x x f x f x. () he acobian matrix is updated by adding a ran-one matrix to the previous - in satisfying equation () and the following relations: r= r, when xi xi r 0, (2) i i ( ) = ( ) ( )= where x x = p, f x f x q. It is emphasized that here is the acobian matrix of total system. It is noted that equation () is an approximate one. For the polynomial-only problems, we can gain an exact alternative of equation () by using theorem. Without loss of generality, Let us consider the following nonlinear polynomial equations ( ) ( ) ( )= + ( )+ ( )+ = f x Lx N 2 x N 3 x b 0, (3) where Lx, N (2) (x) and N (3) (x) represent the linear, quadratic and cubic terms of the system of equations. he acobian matrix of the system is given by ( 2) ( 3) f( x) N ( x) N ( x) = = L + +. (4) x x x By using theorem, we have ( ) ( ) = + ( )+ ( )= ( ) 2 3 x Lx 2N x 3N x f x. (5) herefore, we can exactly establish x x = f( x ) f( x )= y (6) After some simple deductions, we get ( x x )= x + x + y. (7) It is worth stressing that equation (7) differs from equation () in that it is exactly 3

4 constructed and forms the basis of this paper. By analogy with the deduction of the BFGS formula, let = + up. (8) Substituting the above equation (8) into equation (7) yields u = y p ( ). (9) px herefore, we have = + y p p ( ). (0) px Note that the left second term of the above equation (0) is a ran-one matrix. By using the nown Shermann-Morrison formula, we can derive ( s )( p ) =, () + p s where s = y p ( ). Note that y can be evaluated easily without. he px updating formulas (0) and () are a modified version of the following original BFGS formulas: = ( ) p q p p p (2) and = ( ) q p p p, (3) q 3. Remars One can find that the computing effort of both Eqs. () and (3) are nearly the same, only about 3n 2 operations. heoretically, the present updating formulas improve the 4

5 accuracy of the solution by establishing themselves on the exact equation (7) instead of the approximate quasi-newton equation (). he basic idea of the quasi-newton method is a successive update of ran one or two. herefore, it is noted that equation (7) is not actually exact due to the approximate acobian matrix yielded in the previous iterative steps. It may be better to initialize acobian matrix via an exact approach. In addition, it is a puzzle for a long time why one-ran BFGS updating formula performs much better compared to the other two-ran updating schemes such as DFP method [5]. In our understanding, the most possible culprit is due to the inexactness of quasi-newton equation (). herefore, this suggests that the updating formulas of higher order based on exact equation (7) may be more attractive, which will include more additional curvature information to accelerate convergence. It is noted that in one dimension, the present equations (0) and () degenerates into the original Newton method by comparing with the fact that the traditional quasi-newton method becomes the secant method. he performances of the present methodology need be examined in solving various benchmar problems. Finally, it is stressed again that the present modified BFGS updating formulas (0) and () are different from the previously given ones in Chen [], although they were derived based on the same equation (7). Also, the present BFGS formulas are only applicable to the nonlinear polynomial equations. 5

6 Reference. W. Chen, acobian matrix: a bridge between linear and nonlinear polynomial-only problems, (submitted). Also published in Computing Research Repository (CoRR): (Abstract and full text can be found there). 2. W. Chen, Relationship formula between nonlinear polynomial equations and the corresponding acobian matrix, (submitted). Also published in arxiv.org e-print archive: (Abstract and full text can be found there). 3.. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New Yor, D. Wang, Solution of Nonlinear Equations and the Methods of Optimization (in Chinese), People Education Press, Beijing, K. W. Brodlie, Unconstrained minimization, in he State of the Art in Numerical Analysis, D. acobs Ed., Academic Press, New Yor, 977, pp