A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008
Outle: Secod-order coe: defto ad propertes Hstory ad recet wors Secod-order coe cuttg surface method Optmalty codtos he updatg drecto: lear cut he updatg drecto: multple SOCC Complety results Lear-quadratc-semdefte programmg Computatoal eperece
Secod-order coe: : ( 0; ), 0 s a closed cove coe : 0 Notatos: y ff y y ff y
Some fuctos ( ; ), s ( s ; s ) 0 0 : ( ; ), 0 0 0 0 ad are egevalues of 1 0 2 0 0, 0, 1 2 s ( s; s s ) If ad the s vertable ad 1 e : (1; 0) tr( ) : 2 2 1 2 0 det( ) : F : 2 1 2 0 2 2 1 2 : ma, 1 2 2 Q 0 2 2 2 det( ) I 2 0 he aalogous operator to Q ay symmetrc matr S to XSX symmetrc matr algebra s the oe that maps
Bloc Cases Let ( ; ; ) ad s ( s ; ; s ), where, s 1 1 1 2 e ( e; ; e ) 1 Q Q Q 1 det() det( ) tr() 2e tr( ) 2 F 2 F 2 ma
Hstory of ACCPM Soeved (1988), New algorthms cove programmg based o a otato of ceter ad o ratoal etrapolatos. Goff, Haure, ad Val (1992), Decomposto ad odfferetable optmzato wth the projectve algorthm. Ye (1992), A potetal reducto algorthm allowg colum geerato. Ye (1997), Complety aalyss of the aalytc ceter cuttg plae method that uses multple cuts. Goff ad Val (2000), Multple cuts the aalytc ceter cuttg plae methods. Luo ad Su (2000), A polyomal cuttg surfaces algorthm for the cove feasblty problem defed by self-cocordat equaltes. oh, Zhao, ad Su (2002), A multple-cut aalytc ceter cuttg plae method for semdefte feasblty problems. Osoorouch ad Goff (2003), he aalytc ceter cuttg plae method wth semdefte cuts. Osoorouch ad Goff (2005), A teror pot cuttg plae method for the cove feasblty problem wth secod-order coe equaltes.
Recet Wors o ACCPM Svaramarsha (2007), A parallel dual decomposto algorthm for polyomal optmzato problems wth structured sparsty Osoorouch ad Goff (2007), A matr geerato approach for egevalue optmzato Osoorouch ad Mtchell (2008), A secod-order coe cuttg surface method: complety ad applcato Svaramarsha, Martez, ad erlay (2006), A coc teror pot decomposto approach for large scale semdefte programmg Baboeau ad Val (2006), ACCPM wth a olear costrat ad a actve set strategy to solve olear multcommodty flow problems Baboeau, Merle ad Val (2006), Solvg large scale lear multcommodty flow problems wth a actve set strategy ad Promal-ACCPM
Secod-order coe cuttg surface method Let y m : Α y c, ad A y c w here A A, A,, A, c c ; c ; ; c, A 1 2 1 2 1 2 q be a compact cove set. Assume: 1) cotas a full dmesoal ball wth radus. 2) here s a oracle that returs cuts. q q m l We are terested fdg a pot ths ball
Aalytc Ceter Defe the dual barrer fucto where ( s, s) 1 log det s 2 log s, s : c A y 0, s : c A y 0. It s easy to verfy that s a strctly cocave fucto o the teror of. herefore the mamzer of ths fucto ests ad s uque. hs mamzer s called the aalytc ceter. From the KK optmalty codtos y s the aalytc ceter of f ad oly f there ests s 0 ad s 0 such that 1 1 A A 0
Aalytc Ceter he prmal barrer fucto: over 1 (, ) : c log det( ) c log 2 q l, : A A 0 s also strctly cocave. herefore has a uque mamzer over. he Cartesa product of ad gves the prmal-dual set of localzato. he correspodg barrer fucto s defed va (,, s, s) (, ) ( s, s)
Optmalty Codtos: he aalytc ceter s uquely defed: Optmalty Codtos: A A 0 s : c - A 1 1 y 0 s : c A y 0 s s
Optmalty Codtos: he aalytc ceter s uquely defed: Appromate aalytc ceter A A 0 s c A y 0 s c A y 0 Q 1/2 2 2 s e s 1 F 1
Newto drecto: Let a strctly feasble pot be gve. Sce s strctly cocave o, mplemetg Newto's method to mamze over yelds d 2 d X s he aalytc ceter of the dual the reads y where Q s, 1 G g 2 G AQA AX A 2 g AQc AX c,.
So far: We defed the dual set of localzato ad the prmal set of localzato m y : Α y c, ad A y c q l, : A A 0 ad dscussed how to compute the aalytc ceter.
he algorthm: 0 0 Let (, ),a strctly feasble pot of (Prmal Set of Localzato) be gve q l, : A A 0 Step 1. Compute (, ), a appromate aalytc ceter of ad y, a appromate ceter of Step 2. Call the oracle. If y s the -ball stop. Step 3. If the oracle returs a sgle lear cut b y d, update q l,, 0 : A A b 0 Step 4. If the oracle returs multple secod-order coe cut B y p d, update q, u p l, : A Bu A 0 Step 5. Fd a strctly feasble pot of ad retur to Step 1.
he updatg drectos: a sgle LC q l,, 0: A A b 0 Mtchell-odd drecto: ma st.. log A d Ad b 0 Result 1: Q s 1/2 2 2 d Sd 1 1.,,, where d, d, ad 1- s a strctly feasble pot of. 2. Startg from ths pot, O(1) Newto steps suffces to obta a appromate aalytc ceter of F
he updatg drectos: multple SOCC s p l, u, : A Au A 0 ma st.. 1 log det u 2 A d Bu Ad 0 Q s 1/2 2 2 d Sd 1 F Result 2: 1., u,, where d, d, ad 1- s a strctly feasble pot of. 2. Startg from ths pot, O( p log( p 1) ) Newto steps suffces to obta a appromate aalytc ceter of
Complety: Result 3: he aalytc ceter cuttg surface algorthm fds a pot the -ball, whe the total umber of lear ad secod-order coe cuts reaches the boud O mp 2 3, where p s the mamum umber of secod-order coe cuts added at the same tme ad 0 s a codto umber of a feld of cu ts.
Lear-quadratc-semdefte programmg problem: Prmal m st.. C X c c X A A b, I X 1, X 0, 0, 0 Dual ma st.. b y z y zi C, A y c, A y c Parameters: C A s q l, c, c, q l, A, b, s m :, ( X ) A X, s :, m m m m y m 1 y A Varables: m s q l X,,, y
Lear-quadratc-semdefte programmg problem: ma st.. b y z y zi C, A y c, A y c ma b y ( C y) st.. A A y m y c, c
M-egevalue fucto: f ( y) b y ( C y) m he mmum egevalue of a symmetrc matr ca be cast as a SDP: f ( y) b y m ( C y) V tr( V ) 1, V 0 Usg the Clar geeralzed gradet of m ( C - y) ad a cahr rule: f ( y) b v v ( Q AQ) V, tr( V ) 1, V m pˆ where the orthoormal colums of Q pˆ are the egevectors corresposg to m ( C - y) p. wth multplcty ˆ
From optmzato to feasblty: Let ( y; z) : A y c, A y c, b y z, 0 m 1 0 where s large eough to esure cotas the optmal soluto. 0 0 0 0 Let y, z be a tal query pot. If f s dfferetable at y, the pˆ 1 ad Q 0 reduces to a colum vector q, ad s updated by ( y ˆ ˆ where bˆ m, wth bˆ q A q, 1,... m, ad dˆ q Cq. 0 0 ; z) : b y z d, b y z ma(, f ( y )), 0 0 If f s odfferetable at y, the p 1 ad s updated by ˆ 0 0 ( y; z) : ˆ y zi Dˆ, b y z ma(, f ( y )), where ˆ y zi Dˆ s a pˆ dmesoal semdefte equalty, ad ˆ y y Bˆ ad Bˆ Q A Q, 1,..., m, Dˆ Q CQ.
Secod-order coe relaato a b a c a b 0 a b a b c b 2c 2c 3 O the other had, every 2 semdefte matr s postve semdefte 2 prcple submatr of a postve Now cosder the semdefte equalty. Cosder the 2 2 prcple submatr locatos ad j, j ˆ y zi Dˆ
Secod-order coe relaato ˆ y zi Dˆ Dˆ Dˆ Bˆ Bˆ 1 0 z Dˆ Dˆ Bˆ Bˆ 0 1 m j j y j jj 1 j jj 0 ˆ ˆ ˆ ˆ D y B z Dj y Bj ˆ ˆ ˆ ˆ D y B D y B z j j jj jj 0 ˆ ˆ ˆ ˆ D D y B B 2z jj jj ˆ ˆ ˆ D D y B Bˆ jj jj 2Dˆ 2 y Bˆ j j 3
Secod-order coe relaato herefore a pˆ dmesoal semdefte cut ca be relaed pˆ( pˆ 1) to p secod-order coe cuts. herefore at each 2 terato, the set of localzato has the form m 1 ˆ ˆ ˆ y : A y 2 ze c, ( A ) y 1z cˆ, ad b y z, where Aˆ cotas q 2 ˆ m A l, cotas ma, f y l 1 1 1 p blocs of SOCC's, lear cuts, ad
A smple llustrato: f ( y) m{ a y b : 1,..., m} y y y m 1 (, ) :, ad
A smple llustrato: f ( y) m{ a y b : 1,..., m} y y y m 1 (, ) :, ad
A smple llustrato: f ( y) m{ a y b : 1,..., m} y y y m 1 (, ) :, ad
Stoppg crtero: he lower boud s updated at each terato by costructo. Gve the defto of, at the th terato the followg relaato of the dual s formed: ma st.. b y ˆ A y 2ze cˆ ( Aˆ ) y 1z cˆ b y z z
Stoppg crtero: A upper boud for ths problem s obtaed by evaluatg the objectve fucto of the restrcted prmal problem at a feasble pot: m ( cˆ ) ( cˆ ) st.. A ˆ Aˆ b b 2e 1 1 0, 0, 0 O the other had from optmalty codtos of the aalytc ceter we have ˆ ˆ A A b 0 2e 1 0 herefore ( /, /, 0) s feasble for restrcted prmal problem ad 1 : ( cˆ ) ( cˆ ) s the updated upper bou d.
Covergece behavor:
, m, s l lc socc p dm gap SOCCSM SDPLR SDP3 SeDuM 300, 50, 100 14 46 3 109 9.2e-4 16 29 69 352 300, 100, 600 4 31 4 121 7.8e-4 56 78 225 843 300, 200, 1100 4 44 4 169 8.6e-4 191 146 503 1834 300, 300, 800 11 79 8 518 9.2e-4 739 -- 656 -- 300, 300, 1000 5 62 7 335 9.2e-4 561 -- 747 -- 500, 50, 200 4 33 4 128 7.6e-4 77 144 247 1245 500, 100, 1000 9 21 3 69 7.3e-4 67 358 1227 2504 500, 200, 500 11 81 5 384 9.1e-4 643 -- 1555 -- 500, 200, 2000 9 30 3 94 8.6e-4 398 -- 1791 -- 500, 300, 1000 1 44 7 221 4.7 646 -- 2558 -- 800, 10, 800 8 1 2 10 4.2e-4 7 159 553 2798 800, 50, 500 1 22 3 58 9.1e-4 77 433 1453 3449 800, 100, 800 2 32 4 114 9.4e-4 313 -- 2811 -- 800, 200, 1000 1 28 4 107 3.4e-3 467 -- 5814 -- 800, 200, 1500 1 24 5 98 4.5e-3 424 -- 6332 --
, m, s l lc socc p dm gap SOCCSM SDPLR SDP3 SeDuM 1000, 10, 500 2 7 2 16 7.6e-4 14 227 986 3995 1000, 10, 400 11 17 2 45 8.4e-4 110 -- 2097 -- 1000, 50, 900 1 19 2 39 8.8e-4 69 -- 2553 -- 1000, 100, 500 3 46 3 137 3.9e-3 223 -- -- -- 1000, 100, 1000 0 20 3 49 4.3e-3 134 -- -- -- 2000, 10, 100 1 12 2 25 7.3e-4 112 -- 4929 -- 2000, 10, 500 1 8 2 17 7.1e-4 65 -- 5996 -- 2000, 10, 1000 0 7 2 14 4.8e-4 59 -- 6617 -- 2000, 20, 100 1 13 3 34 2.6e-3 177 -- 8136 -- 2000, 20, 800 1 7 2 15 3.7e-3 70 -- 9050 -- 2500, 10, 100 2 6 2 14 4.5e-3 104 -- 9999 -- 2500, 20, 50 4 18 3 41 4.3e-3 368 -- 9999 -- 2500, 20, 500 6 2 2 10 4.8e-3 87 -- 9999 -- 3000, 10, 100 1 9 2 19 4.3e-3 177 -- -- -- 3000, 10, 500 0 6 2 12 3.2e-3 106 -- -- --