A MODIFIED REGULARIZED NEWTON METHOD FOR UNCONSTRAINED NONCONVEX OPTIMIZATION

Transcription

1 JRRAS 9 Ma 4 wwwarpapresscom/volumes/vol9ssue/jrras_9 6p A MODFED REGULARZED NEWON MEOD FOR UNCONSRANED NONCONVEX OPMZAON e Wa Me Q & abo Wa Collee o Scece Uverst o Shaha or Scece a echolo Shaha Cha 93 Emal: waheusst@6com ABSRAC ths paper we preset a moe reularze Newto metho or the ucostrae ocove optmzato b us trust reo techque We show that the raet a essa o the objectve ucto are Lpschtz cotuous the the moe reularze Newto metho M-RNM has a lobal coverece propert Numercal results show that the alorthm s ver ecet Kewors: Reulare Newto metho; rust reo metho; Ucostrae ocove optmzato; Global coverece MSC: M5 NRODUCON We coser the ucostrae optmzato problem m R where R : R s twce cotuousl eretable whose raet a essa are eote b a respectvel rouhout ths paper we assume that the soluto set o s oempt a eote b S a all cases reers to the -orm t s well ow that s cove a ol s smmetrc postve semete or all R Moreover s cove the S a ol s a soluto o the sstem o olear equatos ece we coul et the mmzer o b solv []-[3]he Newto metho s oe o a ecet soluto metho At ever terato t computes the tral step N 3 where a As we ow s Lpschtz cotuous a osular at the soluto the the Newto metho has quaratc coverece owever ths metho has a obvous savatae whe the s sular or ear sular o overcome the cult cause b the possble sulart o metho where the tral step s the soluto o the lear equatos where s the ett matr parameter [4] propose a reularze Newto 4 s a postve parameter whch s upate rom trato to terato he s trouce to overcome the cultes cause b the sulart or ear sulart o the essa Now we ee to coser aother questo "how to choose the moe reularze parameter? " whch wll pla mportat roles ot ol theoretcal aalss but also umercal epermets Yamashta a Fuushma [5] chose a showe that the reularze Newto metho has quaratc coverece uer the local error bou 77

2 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho coto whch s weaer tha osulart Fa a Yua [6] too wth [] a prove that the Leveber-Marqulart metho preserves the quaratc coverece uer the same cotos ths paper we wll choose the parameter as where s a postve costat ma > s upate rom terato to terato b a trust reo techque m most past stues [7]-[9] or the reularze Newto metho the coverece propertes have bee scusse ol whe s cove ths paper we propose a moe Newto metho or whose objectve ucto s ocove whch s mal motvate [] Do-ul Masao Fuushma Lqu Q a Nobuo Yamashta have propose that reularze Newto a eact Newto methos are possble etee to ocove mmzato problems [] he chose to sats where C > s a costat a ma C 5 s the mmum eevalue o ma m Base o the better perormace o the moe reularze metho wth we wll coser the choce o ma 6 m ths paper We ete the reularze Newto metho 4 to the ucostrae ocove optmzato At the -th terato o the M-RNM we set reularze parameter as > From the eto o the matr where a s postve semete eve s ocove hereore the we ca use reularze Newto metho to solve the problem o he ma scheme o the moe reularze Newto metho or ucostrae ocove optmzato s ve as ollows At ever terato t solves the lear equatos to obta the Newto step where to obta the appromate Newto step 7 a the solves the lear equatos wth 8 he purpose o ths paper s to vestate whether the propose metho or ucostrae ocove optmzato has lobal coverece he paper s oraze as ollows secto we preset a ew moe reularze Newto alorthm b us trust reo techque the prove the lobal coverece secto 3 Numercal results are ve Fall we coclue the paper the secto 4 E ALGORM AND GLOBAL CONVERGENCE ths secto we rst preset the ew moe reularze Newto alorthm b us trust reo techque the prove the lobal coverece Frst we ve the moe reularze Newto alorthm Let ete a techque be ve b 7 a 8 respectvel Sce the matr s a escet recto o at but s smmetrc a postve ma ot be ece we preer to use a trust reo 78

3 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho 79 Dee the actual reucto o at the -th terato as Are Note that the Newto step s the mmzer o the problem: m R we let the t ca be prove [] that s also a soluto o the trust reo problem: m R s t B the amous result ve b Powell [] we ow that m Smlar to s ot ol the mmzer o the problem: m R but also the soluto o the ollow trust reo problem: m R s t where hereore we also have m 3 Base o the equaltes a 3 t s reasoable or us to ee the ew precte reucto as e 4 whch satses m m e 5 he rato o the actual reucto to the precte reucto e Are r 6 plas a e role ec whether to accept the tral step a how to ajust the reularze parameter

4 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho he reularze Newto alorthm wth correcto or ucostrae ocove optmzato problems s state as ollows 3; Alorthm Step Gve R > m > < c c c < < p << p : Step the stop Otherwse o to step 3 Step 3 Compute ma m Solve 7 Step 4 Compute to obta Set Solve to obta Set r Set Step 5 Upate as Are e 8 s s r c 9 otherwse Set : a o step 3; p r < c r c c map m r > c Alorthm the ve postve costat m s the lowerbou o t plas the role prevet the step rom be too lare whe the sequece s ear the mmzer set Beore scuss the lobal coverece o the alorthm above we mae the ollow assumpto Assumpto a are both Lpschtz cotuous that s there ests a costat > such that L L > 8

5 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho a t ollows rom that L R L R L R 3 he ollow lemma ve below shows the relatoshp betwee the postve semete matr a smmetrc postve semete matr Lemma3 A real-value matr s postve semete a ol A / oo See [] Net we ve the bous o a postve ete matr a ts verse Lemma4 Suppose A s smmetrc postve semete he a hol or a > oo A A t ollows rom Lemma 3 a the eto o the -orm that A ma ma ma A A A A A A A s postve semete where A A meas the larest eevalue o A A Smlarl we have ma hs completes the proo So we have the ollow results A ma ma m A A A A A A A A heorem 5 Uer the cotos o Assumpto s boue below the Alorthm termates te teratos or satses oo that lm We prove b cotracto the theorem s ot true the there ests a postve a a teer 4 such 5 8

6 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho 8 Wthout loss o eeralt we ca suppose Set he Now we wll aalss two cases whether s te or ot Case : s te he there ests a teer such that B 9 we have < c r hereore b a 5 we euce 6 Sce we et rom 7 a 6 that 7 From 8 we obta L 8 where s a postve costat t ollows rom a 4 that o o e Are 9 Moreover rom 55 a 7 we have m L e or sucetl lare Duo to 9 a we et

7 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho 83 m o o L e e Are r whch mples that r ece there ests postve costat such that hols or all lare whch cotracts to 6 Case : s te he we have rom 5 a 5 that m m m lm > L c c e c whch mples that lm 3 he above equalt toether wth the upat rule o meas 4 Smlar to 8 t ollows rom 3 a 4 that 3 or some postve costat 3 he we have 3 s hs equalt toether wth els < s whch mples that 5

8 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho t ollows rom a 8 that Sce whch meas 6 rom 7 we have rom 5 a 6 that B the same aalss as we ow that ece there ests a postve costat cotracto to 7 he proo s complete L > m such that r 8 hols or all sucetl lare whch ves a 3 NUMERCAL EXPERMENS ths secto we test the perormace o the moe reularze Newto metho a the reularze Newto metho wthout correcto he ucto to be mmze s where are costats t s clear that ucto s mmzer set o S R s ocove a the t ca also be prove that proves a local error bou ear the mmzer o s ve b he essa 3 where 5 a a b 3 5 b 4 c c 84

9 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho Matr s ete or all but sular as the sum o ever colum s zero Sce the essa s alwas sular the Newto metho caot be use to solve olear equatos But b us the reularzato techque both reularze Newto metho a Alorthm wor qute well We teste the moe reularze Newto metho or problem where s ve b 3 wth eret values o a We set c c 5 c 75 p 5 p 4 5 m or Alorthm a able reports the orms o Star rom at ever terato whe a the Alorthm ol taes ve teratos to obta the mmzer o at each terato able lsts the orm o e- 633e- 94e-4 663e- able : Results o M-RNM o 3 he results able relect the coverece propert o he eerate teratos are We ma observe that the whole sequece coveres to We also ra the reularze Newto alorthm RNA [] wthout correcto that s we o ot solve the lear equatos 8 a just set the soluto o 7 to be the tral step s he we teste the reularze Newto alorthm [] wthout correcto a moe reularze Newto alorthm or varous o a eret choces o the start pot he results are lste able a able 3 : the selecte value o : the selecte value o ; Dm: the meso o the problem; : the th elemet ; m: the umber o teratos requre; : the al value o ; the stopp crtero : the al value o We use ; 5 as 85

10 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho Dm m 3/ 63e-7 55 / / 85e / 48e-6 55 / 9/ 48e-6 9 6/ 6e-6 55 / 5/ 5e /6 63e-7 55 / 8/8 8e / 66e-6 55 / 8/ 56e /3 384e- 55 / 5/4 486e-9 9 8/ 495e-6 55 / /5 3e /9 345e-7 55 / /5 548e /4 956e-8 55 / 5/ 9e /6 77e-7 55 / 9/ 3e /35 485e-7 55 / 9/ 8e / 4e-7 55 / 9/ 56e-6 36 able : Results o RNA a Alorthm 86

11 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho Dm m 4/6 43e-6 55 / 5/ 35e / e-7 55 / 6/ 6e-7 9 /55 74e-7 55 / 5/ 45e /8 685e-7 55 / 6/ 589e / 45e-8 55 / 3/ 543e /3 33e-8 55 / /5 756e /4 48e-7 55 / 5/7 7e /6 98e-7 55 / 5/7 454e /4 537e-8 55 / /4 379e /6 4e-9 55 / 7/ 33e-7 9 6/35 4e-7 55 / 8/9 8e /3 75e-7 55 / 38/ 47e-7 36 able 3: Results o RNA a Alorthm Moreover we ca see that or the same a able a able 3 show that the correcto term oes help to mprove reularze Newto alorthm whe the tal pot s ar awa rom the mmzer hese acts cate that the troucto o correcto s reall useul a coul accelerate the coverece o the reularze Newto metho 4 CONCLUSON We propose a moe reularze Newto metho or ucostrae ocove optmzato a show that the raet a essa o the objectve ucto are Lpschtz cotuous the the M-RNM has a lobal coverece he presete umercal results show that the practcal o the propose metho We ma raw the cocluso that our metho s better tha the reularze Newto alorthm RNM[] wthout correcto 5 REFERENCES [] W Su Y Yua Optmzato heor a Methos Sprer Scece a Busess Mea LLC New Yor 6 [] W Zhou D L A loball coveret BFGS metho or olear mootoe equatos wthout a mert uctos Math Comp [3] CKelle teratve Methos or Optmzato : Froters Apple Mathematcs vol 8 SAM Phlaelpha 999 [4] D Su A reularzato Newto metho or solv olear complemetart problems Appl Math 87

12 JRRAS 9 Ma 4 Wa et al A Moe Reularze Newto Metho Optm4 999 pp [5] D L M Fuushma L Q a N Yamashta Reularze Newto methos or cove mmzato problems wth sular solutos Comput OptmAppl 8 4 pp 3-47 [6] JYFa a YXYua O the quaratc coverece o the Leveber-Marquart metho wthout osulart assumpto Comput 74 5 pp 3-39 [7] JaFa YaaYua A reularze Newto metho or mootoe olear equatos a ts applcato Optmzato Methos a Sotware 9 4 pp -9 [8] Yamashta N Fuushma O the rate o coverece o the Leveber-Maruart metho Comput SupplWe 5 pp 7-38 [9] Pola RA Reularze Newto metho or ucostrae cove optmzato Math oram Ser B 9 pp 5-45 [] Do-huL Masao Fuushma Lqu Q Nobuo Yamashta Reularze Newto methos or cove mmzato problems wth sular solutos Computatoal Optmzato a Applcatos 8 4 pp 3-47 [] MJD Powell Coverece propertes o a class o mmzato alorthms : OL Maasara RR Meer SM Robso Es : Nolear oramm vol Acaemc ess New Yor 975 pp -7 88