New Variants of Newton s Method for Nonlinear Unconstrained Optimization Problems

Transcription

1 Itelliget Iformatio Maagemet, 00,, doi:0.436/iim Pulished Olie Jauary 00 ( ew Variats of ewto s Method for oliear Ucostraied Optimizatio Prolems Astract V. KAWAR, Kapil K. SHARMA, Ramadeep BEHL Uiversity Istitute of Egieerig ad Techology, Paja Uiversity, Chadigarh60 04, Idia Departmet of Mathematics, Paja Uiversity, Chadigarh60 04, Idia vmithil@yahoo.co.i, kapilks@pu.ac.i, ramaehl87@yahoo.i I this paper, we propose ew variats of ewto s method ased o quadrature formula ad power mea for solvig oliear ucostraied optimizatio prolems. It is proved that the order of covergece of the proposed family is three. umerical comparisos are made to show the performace of the preseted methods. Furthermore, umerical eperimets demostrate that the logarithmic mea ewto s method outperform the classical ewto s ad other variats of ewto s method. MSC: 65H05. Keywords: Ucostraied Optimizatio, ewto s Method, Order of Covergece, Power Meas, Iitial Guess. Itroductio The celerated ewto s method f () f used to approimate the optimum of a fuctio is oe of the most fudametal tools i computatioal mathematics, operatio research, optimizatio ad cotrol theory. It has may applicatios i maagemet sciece, idustrial ad fiacial research, chaos ad fractals, dyamical systems, staility aalysis, variatioal iequalities ad eve to equilirium type prolems. Its role i optimizatio theory ca ot e overestimated as the method is the asis for the most effective procedures i liear ad oliear programmig. For a more detailed survey, oe ca refer [ 4] ad the refereces cited therei. The idea ehid the ewto s method is to approimate the ojective fuctio locally y a quadratic fuctio which agrees with the fuctio at a poit. The process ca e repeated at the poit that optimizes the approimate fuctio. Recetly, may ew modified ewto-type methods ad their variats are reported i the literature [5 8]. Oe of the reasos for discussig oe dimesioal optimizatio is that some of the iterative methods for higher dimesioal prolems ivolve steps of searchig etrema alog certai directios i [8]. Fidig the step size,, alog the directio vector d ivolves solvig the su prolem to miimize f f d, which is a oe dimesioal search prolem i [9]. Hece the oe dimesioal methods are most idispesale ad the efficiecy of ay method partly depeds o them [0]. The purpose of this work is to provide some alterative derivatios through power mea ad to revisit some well-kow methods for solvig oliear ucostraied optimizatio prolems.. Review of Defiitio of Various Meas For a give fiite real umer, the th power mea m of positive scalars a ad, is defied as follows [] m a () It is easy to see that a For, m (Harmoic mea) a, (3) a For, m, (4) a For, m (Arithmetic mea). (5) For 0, we have lim m a, (6) 0 which is the so-called geometric mea of a, ad may e deoted y m. g Copyright 00 SciRes

2 For give positive scalars a ad, some other well-kow meas are defied as a a (Heroia mea), (7) 3 C (Cotra-harmoic mea) T ad L (Cetroidal mea) (Logarithmic mea) a a a a 3 a log a a log 3. Variats of ewto s Method for oliear Equatios V. KAWAR ET AL. 4, (8), (9). (0) Recetly, some modified ewto s method with cuic covergece have ee developed y cosiderig differet quadrature formula for computig itegral, arisig i the ewto s theorem [] f f f t dt () Weerakoo ad Ferado [3] re-derived the classical ewto s method y approimatig the idefiite itegral i () y rectagular rule. Suppose that is the root of the equatio f, we the put the left side f 0 i the idetity () ad approimate the itegral y the rectagular rule accordig to which 0 f t dt f () Therefore, from () ad (), they otai the wellkow ewto s method. Weerakoo ad Ferado [3] further used trapezoidal approimatio to the defiite itegral accordig to which f t dt f f (3) to otai modified ewto s method give y where f (4) f f f f (5) is the ewto s iterate. I cotrast to classical ewto s method, this method uses the arithmetic mea of f, istead of * ad f f. Therefore, it is called arithmetic mea ewto s method [3]. By usig differet approimatios to the idefiite itegral i the ewto s theorem (), differet iterative formulas ca e otaied for solvig oliear equatios [4]. 4. Variats of ewto s Method for Ucostraied Optimizatio Prolems ow we shall eted this idea for the case of ucostraied optimizatio prolems. Suppose that the fuctio f is a sufficietly differetiale fuctio. Let is a etremum poit of root of f (6) 0 f, the is a Etedig ewto s theorem (), we have f f f t dt (7) Approimatig the idefiite itegral i (7) y the rectagular rule accordig to which f t dt f (8) ad usig f 0, we get f f, f. (9) 0 This is a well-kow quadratically coverget ewto s method for ucostraied optimizatio prolems. Approimatig the itegral i (7) y the trapezoidal approimatio f t dt f f i comiatio with the approimatio (0) f f f f ad f 0, f we get the followig arithmetic mea ewto s method give y f () f f for ucostraied optimizatio prolems. This formula is also derived idepedetly y Kahya [5]. If we use the midpoit rule of itegratio i (0) (Gaussia-Legedre formula with oe kot) [5], the we otai a ew formula give y Copyright 00 SciRes

3 4 V. KAWAR ET AL. f mula (3) as follows: () ) For (arithmetic mea), Formula (3) correspods to cuically coverget arithmetic mea ew- f to s method This formula may e called the midpoit ewto s f formula for ucostraied optimizatio prolems. ow (4) for geeralizatio, approimatig the fuctios i cor- f f rectio factor i () with a power meas of a f ) For (harmoic mea), Formula (3) correspods to a cuically coverget harmoic mea ad f, we have ewto s method f f (5) f f f f sigf 0 3) For 0 (geometric mea), Formula (3) correspods to a ew cuically coverget geometric mea (3) ewto s method f This family may e called the th power mea iterative family of ewto s method for ucostraied op- sigf 0 f f timizatio prolems. (6) Special cases: 4) For, Formula (3) correspods to a ew It is iterestig to ote that for differet specific values cuically coverget method of, various ew methods ca e deduced from For- 4 f 0 f f sig f f f (7) 5) For (root mea square), Formula (3) correspods to a ew cuically coverget root mea square ewto s method f (8) sig f 0 f f Some other ew third-order iterative methods ased o heroia mea, cotra-harmoic mea, cetrodial mea, logarithmic mea etc. ca also e otaied from Formula () respectively. 6) ew cuically coverget iteratio method ased o heroia mea is 3 f 0 f f sig f f f (9) 7) ew cuically coverget iteratio method ased o cotra-harmoic mea is 3 ff f (30) f f 8) ew cuically coverget iteratio method ased o cetroidal mea is 3f f f (3) f f f f 9) ew cuically coverget iteratio method ased o logarithmic mea is f log f log f (3) f f 5. Covergece Aalysis Theorem 5. Let I e a optimum poit of a sufficietly differetiale fuctio f : I for a ope iterval I. If the iitial guess 0 is suffi- Copyright 00 SciRes

4 V. KAWAR ET AL. 43 cietly close to, the for, the family of methods defied y (3) has cuic covergece with the followig error equatio e 9 c c3 e Oe (33) Proof: Let e a etremum of the fuctio f (i.e. f 0 ad 0 f is k f where e ad ck, k 3, 4,.... k! f f ). Sice sufficietly differetiale fuctio, epadig f, ad f aout y meas of Tay- f lor s epasio, we have f f f f e O e! 3! 3 4 e 4 f f e c e c e O e 3 f f c e c e O e Usig (35) ad (36) i (9), we get [ f f c e c c c e O e Case ) For as 3 4 ] (34) (35) (36) (37) \ 0, Formula (3) may e writte f f f f (38) Usig iomial theorem ad the Formulae (35), (36) ad (37) i (38), we fially otai e c3 c4e Oe (39) Case ) For 0, Formula (3) ca e writte as f 0 sig f f f Upo usig (35), (36) ad (37), we otai (40) f 0 sig f f f 9 e c c e Oe From (40) ad (4), we otai e c c e O (4) e (4) Therefore, it ca e cocluded that for all, the th power mea iterative family (3) for ucostraied optimizatio prolems coverges cuically. O similar lies, we ca prove the covergece of the remaiig Formulae (9)-(3) respectively. 6. Further Modificatios of the Family (3) The two mai practical deficiecies of ewto s method, the eed for aalytic derivatives ad the possile failure to coverge to the solutios from poor startig poits are the key issues i ucostraied optimizatio prolems. Family (3) ad other methods (9)-(3) are also variats of ewto s method ad will fail miseraly if at ay stage of computatio, the secod order derivative of the fuctio is either zero or very close to zero. The defect of ewto s method ca easily e elimiated y the simple modificatio of iteratio process. Applyig the ewto s method (9) to a modified fuctio: p f e f (43) where p. This fuctio has etter ehavior as well as the same optimum poit as f ; we shall get the modified ewto s method give y f (44) f p f This is a oe parameter family of ewto s iteratio method for ucostraied optimizatio prolems ad do ot fail if f 0 like ewto s method. I order to otai the quadratic covergece, the sig of etity p should e chose so that deomiator is largest i magitude. O similar lies, we ca also modify some of the aove-metioed cuically coverget variats of ewto s methods for ucostraied optimizatio prolems. Kahya [5] has also derived similar formula y usig the differet approach ased o the ideas of Mamta et al. [6]. Similar approaches for fidig the simple root of a oliear equatio or system of equatios have ee used y Be-Israel [7] ad, Kawar ad Tomar [8]. 7. umerical Results I this sectio, we shall preset the umerical results Copyright 00 SciRes

5 44 V. KAWAR ET AL. Tale. Test prolems o. Eamples Iitial guess Optimum poit e cos , , Tale. Compariso tale Eamples M AM HM GM Method (7) RMSM HERM CM TMM LMM otaied y employig the iterative methods amely ewto s method ( M ), arithmetic mea ewto s method ( AM ), harmoic mea ewto s method ( HM ), geometric mea ewto s method ( GM ), method (7), root mea square ewto s method ( RMSM ), heroia mea ewto s method ( HEM ), cotra-harmoic mea ewto s method ( CM ), cetroidal mea ewto s method ( TM ), logarithmic mea ewto s method (LMM) respectively to compute the etrememum of the fuctio give i Tale. We use the same fuctios as Kahya [6, 7]. The results are summarized i Tale. We use 0 5 as tolerace. Computatios have ee performed usig C i doule precisio arithmetic. The followig stoppig criteria are used for computer programs: ), ) f. 8. Coclusios It is well-kow that the quadrature formulas play a importat ad sigificat role i the evaluatios of defiite itegrals. We have show that these quadrature formulas ca also e used to develop some iterative methods for solvig ucostraied optimizatio prolems. Further, this work proposes a family of ewto-type methods ased o oliear meas ad ca e used to solve efficietly the ucostraied optimizatio prolems. Proposed family (3) uifies some of the most kow third-order methods for ucostraied optimizatio prolems. It also provides may more ukow processes. All of the proposed third-order methods require three fuctio evaluatios per iteratios so that their efficiecy ide is.44, which is etter tha the oe of ewto s method.4. Fially eperimetal results showed that the harmoic mea ad logarithmic mea ewto s method are the est amog the proposed iteratio methods. 9. Ackowledgemet We would like to thak the aoymous referee for his or her valuale commets o improvig the origial mauscript. We are also thakful to Professor S.K. Tomar, Departmet of Mathematics, Paja Uiversity, Chadigarh for several useful suggestios. Ramadeep Behl further ackowledges the fiacial support of CSIR, ew Delhi. 0. Refereces [] B. T. Ployak, ewto s method ad its use i optimiza- Copyright 00 SciRes

6 V. KAWAR ET AL. 45 tio, Europea Joural of Operatio Research, Vol. 8, pp , 007. [] Y. P. Lapti, A approach to the solutio of oliear ucostraied optimizatio prolems (rief commuicatios), Cyeretics ad System Aalysis, Vol. 45, o. 3, pp , 009. [3] G. Guderse & T. Steihaug, O large-scale ucostraied optimizatio prolems ad higher order methods, Optimizatio methods & Software, DOI: 0.080/ , o., 009. [4] H. B. Zhag, O the Halley class of methods for ucostraied optimizatio prolems, Optimizatio Methods & Software, DOI: 0.080/ , o. 0, 009. [5] E. Kahya, Modified secat-type methods for ucostraied optimizatio, Applied Mathematics ad Computatio, Vol. 8, o., pp , 007. [6] E. Kahya & J. Che, A modified Secat method for ucostraied optimizatio, Applied Mathematics ad Computatio, Vol. 86, o., pp , 007. [7] E. Kahya, A class of epoetial quadratically coverget iterative formulae for ucostraied optimizatio, Applied Mathematics ad Computatio, Vol. 86, pp , 007. [8] C. L. Tseg, A ewto-type uivariate optimizatio algorithm for locatig earest etremum, Europea Joural of Operatio Research, Vol. 05, pp , 998. [9] E. Kahya, A ew uidimesioal search method for optimizatio: Liear iterpolatio method, Applied Mathematics ad Computatio, Vol. 7, o., pp. 9 96, 005. [0] M. V. C. Rao &. D. Bhat, A ew uidimesioal search scheme for optimizatio, Computer & Chemical Egieerig, Vol. 5, o. 9, pp , 99. [] P. S. Bulle, The power meas, had ook of meas ad their iequalities, Kluwer Dordrecht, etherlads [] J. E. Deis & R. B. Schale, umerical methods for ucostraied optimizatio ad oliear equatios, Pretice-Hall, ew York, 983. [3] S. Weerakoo & T. G. I. Ferado, A variat of ewto s method with accelerated third-order covergece, Applied Mathematics Letter, Vol. 3, pp , 000. [4] M. Frotii & E. Sormai, Some variats of ewto s method with third-order covergece, Applied Mathematics ad Computatio, Vol. 40, pp , 003. [5] W. Gautschi, umerical aalysis: a itroductio, Birkhäuser, Bosto, Ic., Bosto, 997. [6] Mamta, V. Kawar, V. K. Kukreja & S. Sigh, O a class of quadratically coverget iteratio formulae, Applied Mathematics Computatio, Vol. 66, pp , 005. [7] B. I. Adi, ewto s method with modified fuctios, Cotemporary Mathematics, Vol. 04, pp , 997. [8] V. Kawar & S. K. Tomar, Epoetially fitted variats of ewto s method with quadratic ad cuic covergece, Iteratioal Joural of Computer Mathematics, Vol. 86, o. 9, pp , 009. Copyright 00 SciRes