ISSN 746-7659, Eglad, UK Joual of Ifoatio ad Coputig Sciece Vol. 9, No. 4, 04, pp. 67-76 A soothig Newto ethod fo the iiu o solutio of liea poga Lia Zhag, Zhesheg Yu, Yaya Zhu Uivesity of Shaghai fo Sciece ad echology, Mobile: +86 88784966; E-ail addess: zhaglia9880806@6.co. (Received Mach 4, 04, accepted Octobe, 04 Abstact. I this pape, we popose a soothig Newto ethod to fid the iiu o solutio of liea poga pobles. By usig the soothig techique, we efoulate the poble as a ucostaied iiizatio poble with a twice cotiuous diffeetiable objective fuctio. he iiizatio of this objective fuctio ca be caied out by the classical Newto-type ethod which is show to be globally covegece. Keywods: Liea poga, Miiu o solutio, Ucostaied iiizatio efoulatio, Soothig fuctio, Newto-type ethod.. Itoductio Coside the liea poga i pial fo i c x st.. Ax b, x 0 ( ogethe with its dual ax b λ st.. A λ c ( Whee A R, c R, ad b R ae the give data, ad A is assued to have full ow a. Let us deote the optial value of the pial poble ( by if(p : {c x Ax b, x 0} houghout this auscipt, we assue if ( P R (3 his is equivalet to sayig that the pial (ad hece also the dual liea poga has a oepty solutio set. he ai of this pape is to fid the iiu o solutio of the pial poga (, i.e., we wat to * fid the solutio x of the poga i x st.. Ax b, c x if(p, x 0 (4 Note that this poble has a uique solutio ude the assuptio (3.Sice the iiu o solutio could be a vetex as well as a poit belogig to the elative iteio of the solutio set, eithe the siplex ethod [] o the class of iteio-poit ethods [] will be assued to fid the iiu o solutio of (. he stadad ethod fo fidig a iiu o solutio of a covex poga is based o the ihoov egulaizatio[3]. Specialized to ou liea poga (, the ihoov egulaizatio geeates a sequece of iteates{ x } with x beig the uique solutio of the egulaized poga ε i cx+ x st.. Ax bx, 0 (5 whee ε > 0 is a positive paaete ad the sequece { ε } teds to zeo. Howeve, the ihoov egulaizatio is, i geeal, quite costly sice it has to solve a sequece of quadatic pogas. O the othe had, due to special popeties of liea pogas, it is ow that a solutio of a sigle quadatic poga (5 Published by Wold Acadeic Pess, Wold Acadeic Uio
68 Lia Zhag et.al. : A soothig Newto ethod fo the iiu o solutio of liea poga with a sufficietly sall but positive paaete aleady gives a solutio of (4. his follows fo Magasaia ad Meye[[4],Coollay]. o ovecoe this dawbac, Kazow, Qi ad Qi [5] descibes a ew techique to solve poble (4, thei ai idea is to efoulate the poble as a ucostaied iiizatio poble with a covex ad sooth objective fuctio, a Newto-type ethod with global covegece was poposed. We ote that although the objective fuctio i [5] is sooth ad covex, it does ot twice cotiuous diffeetiable, ad theefoe the classical Newto ethod ca t be used. Based o the ecetly developet of soothig fuctio fo copleetaity pobles, see, fo exaples [8,5,6,9], i this pape, we popose a soothig ucostaied optiizatio efoulatio ad a Newto ethod to fid the solutio of poble (. his efoulatio povides a twice cotiuous diffeetiable objective fuctio of the efoulatio i [5], we obtai the positive defiite of the Hessia atix of the objective fuctio ude cetai coditios ad hece the classical Newto ethod ca be used to solve the poble diectly. he ogaizatio of this pape is as follows. I Sectio, we give the soothig ucostaied optiizatio efoulatio ad soe popeties of the efoulatio icludig the Newto ethod. he ueical esults ae epoted i Sectio 3. A few wods about ou otatio: We deote the -diesioal eal space by R. Fo a vecto x R, we wite x fo its Euclidea o, ad x + o [ x ] + fo the vecto ax{ 0, x }, whee the axiu is tae copoetwise, i.e., x + is the pojectio of x oto the oegative othat. he ows of a atix B will be deoted by B i, whee as we wite bij fo the ( i, j th eleet of the atix B.. Ucostaied Miiizatio Refoulatio he followig esult is give i [6], which is essetially based o soe elated esults give by Magasaia [7] ad Magasaia ad Meye[4]. * heoe. A vecto x R is the iiu o solutio of the pial liea poga ( if ad oly if thee exists a positive ube R>0 such that, fo each R, we have * * x A λ c + Whee λ * deotes a solutio of the oliea syste A A λ c b + Motivated by the chaacteizatio stated i heoe., Kazow, Qi ad Qi [5] gave a ucostaied iiizatio efoulatio of poble (4 as follows: i f ( λ A λ c b λ, λ R (6 + he objective fuctio i (6 has the followig popety, the poof ca be foud i [5]. Lea. he fuctio f fo (6 is covex ad cotiuously diffeetiable with gadiet: f ( λ A A λ c b (7 + * * he lea shows that x is the iiu o solutio of ( if ad oly if λ is the statioay poit of poble (6. he objective fuctio f is oce but ot twice cotiuously diffeetiable, [5] eployed a geealized Newto ethod to fid the solutio of poble (6. I what follows, we coside efoulatig the poble ito a soothig ucostaied optiizatio ad the eploy the classical Newto to solve it. Fistly, we itoduce the soothig fuctio used i this pape. Defie the step fuctio if x > 0 σ ( x 0 if x 0 I the extesive eual etwo liteatue, the step fuctio is vey effectively appoxiated by the sigoid fuctio s( x,, > 0 x + e JIC eail fo cotibutio: edito@jic.og.u
Joual of Ifoatio ad Coputig Sciece, Vol. 9 (04 No. 4, pp 67-76 69 ( x + I this wo we utilize the itegal of the sigoid fuctio as a appoxiatio to the plus fuctio as follows: x x ( x p( x, s( y, dy x log( e + + + (8 Fo eve odeate value of, the fuctio p( x, is a good appoxiatio to the plus fuctio. As appoaches ifiity, p( x, appoaches ( x + fo above ad eais cotiuously diffeetiable as ay ties as we wish. Hece fist ode ad secod ode gadiet ethods ca be used to solve the efoulated poble ivolvig the p fuctio. We teat as a paaete i the fuctio p (., x. Hece - whe we say p o p, we ea the deivative o ivese of p with espect to the fist vaiable with the paaete fixed. he basic popeties of p(x, ca be foud i [8]. p x,, > 0has the followig popeties: Lea. he fuctio of (. p( x, is -ties cotiuously diffeetiable fo ay positive itege, with x e ad p (x, x (+ e.. p( x, is stictly covex ad stictly iceasig o R. 3. p( x, > x+, fo all x R. log 4. ax{ p( x, x+ } p( 0, x R 5. li p( x, x+ 0, fo all > 0. x 6. li p( x, x+, fo all x R. 7. p( x, ( o, fo all x R, 0 p(x, + >.he ivese fuctio p is well defied fo x (0,. 8. p(x, > p(x, β, fo < β, x R. his p fuctio with a soothig paaete is used hee to eplace the plus fuctio of (6. he we ca ewite the ucostaied iiizatio poble as (9, whose objective fuctio is twice cotiuously diffeetiable. f( λ : p( A λ c, b λ, λ R (9 Lea.3 Cobiig lea. ad the popety of lea., we ow that the objective fuctio (9 is also covex. I ode to descibe ou Newto-type ethod we have to give the gadiet ad Hessia atix of (9. Fo the sae of witig coveietly i ou calculatig, we use x istead of λ, let A R, c R, ad b R, the objective fuctio (9 is the followig fo: ( f ( x A x c+ log( + e b x, x R he gadiet of f ( x as well as its Hessia atix is descibed as follows: t t f ( x A I b O t A x c x e JIC eail fo subsciptio: publishig@wau.og.u
70 Lia Zhag et.al. : A soothig Newto ethod fo the iiu o solutio of liea poga f ( x i i i + ( A Whee ( log( x c x A x c e + + ad tj, j,, L. ( ax ij i ci + + i e he calculate pocess is descibed as follows: ( A Let ( log( x c x A x c e + +, ( x is a diesioal vecto ( x ( x ( x (0 M ( x hus the objective fuctio of (9 ca be wote as f ( x ( x ( x b x ( the f ( x b, we eed to calculate fist. ( A log( x c A e + + ( ( A x c log( + e a a L a ( aixi ci+ ( aixi ci+ ( aixi ci+ i i i e e e M M M (3 a a L a ( aixi c+ ( aixi c+ ( aixi c+ i i i e e e t t A O t with tj, j,, L.Coside both ( ad (3, we ca get ( ax ij i ci + + i e t t A A (4 O t So the gadiet of f ( x is JIC eail fo cotibutio: edito@jic.og.u
Joual of Ifoatio ad Coputig Sciece, Vol. 9 (04 No. 4, pp 67-76 7 t t f ( x A A b O t t t A I b O t Now we copute the Hessia atix of f ( x. ( x ( x ( x L x x x Sice ( x M M M ( x ( x ( x L x x x so its taspositio ca be oted as ( x ( x ( x L x x x ( x M M M ( x ( x ( x L x x x Usig (6 ad (7 we wo i i i i i i i i i ( + i ( ( + i L + i i x x x i x x xx i x x xx ( M M M i i i i i i i i i ( + i ( + i L ( + i i x x xx i x x xx i x x x i i i + ajixj c log j i ajixj c i e + +. j As we ow f ( x b, so the Hessia atix of f ( x is f ( x i i i whee +. Sice we get the Hessia atix of the objective fuctio (9, ou ext theoe is supposed to pove that it s positive defiite. heoe. Let assue the atix A has ow full a, the the Hessia atix of f (x is positive defiite. Poof he Hessia atix of f ( x is the fo of (5 (6 (7 JIC eail fo subsciptio: publishig@wau.og.u
7 Lia Zhag et.al. : A soothig Newto ethod fo the iiu o solutio of liea poga f ( x i i i ajixj c j + whee i ajixj ci log + + e. j It is obviously that seidefiite. We calculate i is positive defiite, thus we oly eed to pove which is the fo as we show ext. ai ai ( a ji x j c i ai j ai i + ( e M ( ajixj ci M j a + e i ai a i ( ajixj ci j e ai ( ajixj ci M j + e ai ai a i ( ajixj ci M j + e a i Now we eed to copute i, which is ecessay fo ou poof. a i ( ajixj ci j + e ai ( ajixj ci i j + e M ai ( ajixj ci j + e aa ( i i aa i i aa i i ajixj ci j e aia i aiai aiai ( ajixj ci j + e aiai aiai aiai i i is at least positive i JIC eail fo cotibutio: edito@jic.og.u
Joual of Ifoatio ad Coputig Sciece, Vol. 9 (04 No. 4, pp 67-76 73 a i ( ajixj ci j e ai ( ajixj ci M j + e ai ( a a L a i i i We ca easily fid i 0, coside the desciptio of lea. fo all x R, p( x, > x, so + p( x, > 0. Now we ca coclude i i 0, i.e. i i is positive seidefiite. he the i i Hessia atix of f ( x is positive defiite. Sice the Hessia atix of f ( x is positive defiite, we ca use a Newto-type algoith with a Aijo stepsize. We ext descibe ou Newto-type algoith fo the objective fuctio f( λ, which is efe to the algoith i [9]. Algoith.(Newto-type ethod 0 (S.0 choose paaetes 0 ε, δ (0,, σ (0,0.5,,, ad a statig poit λ R. Set :0. * (S. calculate g f( λ, if g ε, stop, set λ λ. (S. coputeg f( λ, ad solvig the followig liea syste of equatios to get the solutio d : Gd g (S.3 let is the sallest o-egative iteges satisfyig the ude iequatio. f ( λ + δ d f( λ + σδ gd (S.4 let, + β δ λ λ + βd, : +, go to (S. Cosideig the twice diffeetiability of the objective fuctio of poble (9 as well as a Aijo stepsize, ou algoith ca be descibed as global quadatic covegece. he ext wo we ll do is to show this popety of Algoith.. heoe.3 Let { ** λ } be a sequece geeated by Algoith. ad λ be the uique solutio of poble (9. (i he sequece { ** 0 λ } coveges to the uique solutio λ fo ay iitial poit λ i R. 0 (ii Fo ay iitial poit λ, thee exists a itege such that the stepsize β of Algoith. equals fo ad the sequece { ** λ } coveges to λ quadatically. he poof is siila to the heoe 3. i [0], we oit it hee. 3. Nueical esults I this sectio we coside soe ueical esults obtaied with the appoach descibed i the pevious sectio. he Algoith. wee coded i MALAB03b ad u o a PC with.50 GHz CPU pocesso. We ipleet MALABV03b to solve the followig two exaples. I the ueical expeiets, we * choose paaetes δ0.55 σ, 0.4, x is the iiu o solutio of the pial liea poga (, g deotes the Euclidea o of the gadiet. Exaple. JIC eail fo subsciptio: publishig@wau.og.u
74 Lia Zhag et.al. : A soothig Newto ethod fo the iiu o solutio of liea poga ax z x + x 3x+ 5x 5 st.. 6x+ x 4 x, x 0 0 We choose the iitial poit λ (0,0 ad letε e 0. ab. Results fo Exaple Exaple. ax z 3x+ x+.9x3 8x+ x + 0x3 300 0x+ 5x + 8x3 400 st.. x+ 3x+ 0x3 40 x, x, x3 0 0 We choose the iitial poit λ (0,0, 0 ad letε e 0. JIC eail fo cotibutio: edito@jic.og.u
Joual of Ifoatio ad Coputig Sciece, Vol. 9 (04 No. 4, pp 67-76 75 ab. Results fo Exaple I suay, the esults fo table ad table show that the pefoace of ou soothig ethod is quite effective. he paaete ad the soothig paaete effects ou esults as they chaged. 4. Refeeces [] D Betsias, J N sitsilis. Itoductio to Liea Optiizatio, Athea Scietific, Belot, MA, 997. [] L. Qi, J. Su. A Nosooth Vesio of Newto's Method, Matheatical Pogaig, 58(993, pp.353-367. [3] A. Fische, C. Kazow. O Fiite eiatio of a Iteate Method fo Liea Copleetaity Pobles, Matheatical Pogaig,,74(996, pp. 79-9. [4] O. L. Magasaia, R.R.Meye. Noliea Petubatio of Liea Pogas, SIAM Joual o Cotol ad Optiizatio, 7(979, pp. 745-757. [5] C. Kazow, H. Qi, ad L. Qi. O the Miiu No Solutio of Liea Pogas, Joual of Optiizatio heoy ad Applicatios, 6(003, pp.333-345. [6]. Deluca, F. Facchiei, C. Kazow. A Seisooth Equatio Appoach to the Solutio of Noliea Copleetaity Pobles, Matheatical Pogaig, 75(996, pp. 407-439. [7] O. L. Magasaia. Noal Solutios of Liea Pogas, Matheatical Pogaig, (984, pp. 06-6. [8] C. Che, O. L. Magasaia. A Class of Soothig Fuctios fo Noliea ad Mixed Copleetaity Pobles. Coputatio Optiizatio with Applicatios. 5(996, pp. 97-38. [9] C. F. Ma. Optiizatio Method ad Its Matlab Pogaig, Sciece Pess, 0, pp. 9-40. [0] Y. J. Lee, O. L. Magasaia. A Sooth Suppot Vecto Machie fo Classificatio, Coputatioal Optiizatio ad Applicatios, 0(00, pp. 5-. [] B. Che, P.. Hae. A No-iteio-poit Cotiuatio Method fo Liea Copleetaity Pobles. SIAM Joual o Matix Aalysis ad Applicatios, 4(993, pp.68-90. [] C. Kazow. Soe Noiteio Cotiuatio Methods fo Liea Copleetaity Pobles. SIAM Joual o Matix Aalysis ad Applicatios. 4(996, pp. 85-868. [3] J. Bue, S. Xu. he Global Liea Covegece of a Noiteio Path-followig Algoith fo Liea Copleetaity Pobles. Matheatics of Opeatioal Reseach. 3(998, pp. 79-734. [4] W. Y. Su, Y. X. Yua. Optiizatio heoy ad Methods, Sciece Pess, 00, pp. -330. [5] H. D. Qi, L. Z. Liao. A Soothig Newto Method fo Geeal Noliea Copleetaity Pobles, JIC eail fo subsciptio: publishig@wau.og.u
76 Lia Zhag et.al. : A soothig Newto ethod fo the iiu o solutio of liea poga Coputatioal Optiizatio ad Applicatios, 7(000, pp. 3-53. [6] L. Qi, D. Su, G. Zhou. A New Loo at Soothig Newto Methods fo Noliea Copleetaity Pobles ad Box Costaied Vaiatioal Iequalities. Math Poga, A87(000, pp. -35. [7] S. H. Pa, Y. X. Jiag. Soothig Newto Method fo Miiizig the Su of p-nos, Joual of Optiizatio heoy ad Applicatios, 37(008, pp. 55-75. [8] C. F. Ma. A New Soothig ad Regulaizatio Newto ethod fo P0- NCP, Joual of Global Optiizatio, 48(00, pp. 4-6. [9] J. G. Zhu, B. B. Hao. A New Class of Soothig Fuctios ad a Soothig Newto Method fo Copleetaity Pobles, Optiizatio Lettes, 7(03, pp. 48-497. [0] L. A. Vese,. F. Cha. A ultiphase level set faewo fo iage segetatio usig the Mufod ad Shah odel. Iteatioal Joual of Copute Visio,, 50(3 (00,pp.7-93 [] Weiguo Lu. Fast fee-fo defoable egistatio via calculus of vaiatios.phys. Med. Biol., 49(004, pp.3067 3087. JIC eail fo cotibutio: edito@jic.og.u