Recent Progresses on the Simplex Method

Transcription

1 Reet Progresses o the Smple Method K.T. L Professor of Egeerg Stford Uversty d Itertol Ceter of Mgemet See d Egeerg Ng Uversty

2 Outles Ler Progrmmg (LP) d the Smple Method Mrkov Deso Proess (MDP) d ts LP Formulto Smple d poly-terto methods for MDP d Zero-Sum Gme wth fed dsouts Smple method for geerl o-degeerte LP (ludg the ubouded se) Ope Problems

3 Ler Progrmmg strted

4 wth the smple method

5 LP Model Dmeso d 0,, 0 0, b b b s.t. m m m m m b b b d d d d d d d d s.t. m The fesble rego s polyhedro defed by equltes d dmesos.

6 LP Geometry d Theorems Optmze ler obetve futo over ove polyhedro, d there s lwys verte optml soluto.

7 The Smple Method Strt wth y verte, d move to det verte wth mproved obetve vlue. Cotue ths proess tll o mprovemet.

8 Pvotg rules The smple method s govered by pvot rule,.e. method of hoosg det vertes wth better obetve futo vlue. Dtzg's orgl greedy pvot rule. The lowest de pvot rule. The rdom edge pvot rule hooses, from mog ll mprovg pvotg steps (or edges) from the urret bs fesble soluto (or verte), oe uformly t rdom.

9 Mrkov Deso Proess Mrkov deso proess provdes mthemtl frmework for modelg sequetl desomkg stutos where outomes re prtly rdom d prtly uder the otrol of deso mker. MDPs re useful for studyg wde rge of optmzto problems solved v dym progrmmg, where t ws kow t lest s erly s the 950s (f. Shpley 953, Bellm 957). Moder ppltos lude dym plg, reforemet lerg, sol etworkg, d lmost ll other dym/sequetl deso mkg problems Mthemtl, Physl, Mgemet, Eooms, d Sol Sees.

10 Sttes d Atos At eh tme step, the proess s some stte =,...,m, d the deso mker hooses to A tht s vlble for stte, sy of totl tos. The proess respods t the et tme step by rdomly movg to ew stte, d gvg the deso mker mmedte orrespodg ost. The probblty tht the proess eters s ts ew stte s flueed by the hose to. Speflly, t s gve by the stte trsto probblty dstrbuto P. But gve to, the probblty s odtolly depedet of ll prevous sttes d tos; other words, the stte trstos of MDP possess the Mrkov property.

11 A Smple MDP Problem I

12 Poly d Dsout Ftor A poly of MDP s set futo π = {,,, m } tht spefes oe to A tht the deso mker wll hoose for eh stte. The MDP s to fd optml (sttory) poly to mmze the epeted dsouted sum over fte horzo wth dsout ftor 0 γ <. Oe obt LP tht models the MDP problem suh wy tht there s oe-to-oe orrespodee betwee poles of the MDP d bs fesble solutos of the (dul) LP, d betwee mprovg swthes d mprovg pvots. de Ghellk (960), D Epeou (960) d Me (960)

13 Cost-to-Go-Vlues 7/8 3/4 / 0 Chose tos Red

14 Cost-to-Go vlues d LP formulto Let y R m represet the epeted preset ostto-go vlues of the m sttes, respetvely, for gve poly. The, the ost-to-go vetor of the optml poly s Fed Pot of Suh fed pot omputto be formulted s LP. },, rg m{, },, m{ A y p A y p y T T = = γ γ. ;, s.t. m A y p y y T m = γ

15 The dul of the MDP-LP m s.t. = ( e = ) =,, 0,. where e = f A d 0 otherwse. γ p Dul vrble represets the epeted to flow or vst-frequey, tht s, the epeted preset vlue of the umber of tmes to s used.

16 Greedy Smple Rule Chose tos Red

17 Lowest-Ide Smple Rule Chose tos Red

18 Poly Iterto Rule (Howrd 960) Chose tos Red

19 Epoetlly bd emples Klee d Mty (97) showed tht Dtzg's orgl greedy pvot rule my requre epoetlly my steps for LP emple. Melekopoglou d Codo (990) showed tht the smple method wth the smllest de pvot rule eeds epoetl umber of tertos for MDP emple regrdless of dsout ftors. Ferley (00) showed tht the poly-terto method eeds epoetl umber of tertos for udsouted fte-horzo MDP emple. Fredm, Hse d Zwk (0) gve udsouted MDP emple tht the rdom edge pvot rule eeds sub-epoetlly my steps.

20 Ay Good News? I prte, the poly-terto method, ludg the smple method wth greedy pvot rule, hs bee remrkbly suessful d show to be most effetve d wdely used. Ay good ews theory?

21 Boud o the smple/poly methods Y (0): The lss smple d poly terto methods, wth the greedy pvotg rule, termte o more th m γ log( pvot steps, where s the totl umber of tos m-stte MDP wth dsout ftor γ. Ths s strogly polyoml-tme upper boud whe γ s bouded bove by ostt less th oe. m γ )

22 Rodmp of proof Defe ombtorl evet tht ot repets more th tmes. More presely, t y step of the pvot proess, there ests o-optml to tht wll ever re-eter future poles or bses fter m γ m log( γ pvot steps There re t most ( - m) suh o-optml to to elmte from ppere y future poles geerted by the smple or poly-terto method. The proof reles o the dulty, the redued-ost vetor t the urret poly d the optml reduedost vetor to provde lower d upper boud for o-optml to whe the greedy rule s used. )

23 Improvemet d eteso Hse, Mlterse d Zwk (0): For the poly terto method termtes o more steps. γ m log( γ The smple d poly terto methods, wth the greedy pvotg rule, re strogly polyomltme lgorthms for Tur-Bsed Two-Perso Zero-Sum Stohst Gme wth y fed dsout ftor, whh problem ot eve be formulted s LP. )

24 A Tur-Bsed Zero-Sum Gme

25 Determst MDP wth dsouts Dstrbuto vetor p R m ots etly oe d 0 everywhere else. },, rg m{, },, m{ A y p A y p y T T = = γ γ. ;, s.t. m A y p y y T m = γ It hs uform dsouts f ll γ re detl.

26 The dul resembles geerlzed flow m s.t. = ( e = ) =,, 0,. where e = f A d 0 otherwse. γ p Dul vrble represets the epeted to flow or frequey, tht s, the epeted preset vlue of the umber of tmes to s hose.

27 Effey of smple/poly methods They re ot kow to be polyoml-tme lgorthms for determst MDP eve wth uform dsouts. There re qudrt lower bouds o these methods for solvg MDP wth uform dsouts. I Post d Y (0): The Smple method wth the greedy pvot rule termtes t most 3 0( m log m) pvot steps whe dsout ftors re uform, or t most 0( m 5 3 log pvot steps wth o-uform dsouts. Hse, Mlterse d Zwk (03) redued the boud by ftor of m. Not yet ble to prove suh results for the poly terto method. m)

28 Poly strutures wth uform ftors Eh hose to be ether pth-edge or yle-edge. [, m ] f t s pth-to, [ /(-γ), m/(-γ) ] f t s yle-to, so tht they form two possble polyoml lyers.

29 Rodmp of proof There two types of pvots: the ewly hose to s ether o pth or o yle of the ew poly. I every m log(m ) oseutve pvot steps, there must be t lest oe step tht s yle pvot. After every m log(m ) yle pvot steps, there s to tht would ever re-eter s yle or pth to. There re t most to for suh dowgrde. Item result rems true whe dsouts re ot uform, but others do ot hold.

30 Geerl o-degeerte LP Kthr d Mzuo (0) eteded the boud to solvg geerl o-degeerte d bouded LPs: m s.t. = =, ; The smple method termtes t most = b 0,. m log σ m σ ( ) pvot steps, whe the rto of the mmum vlue over the mmum vlue, ll bs fesble soluto etres, s bouded below by σ.

31 Geerl o-degeerte LP Wht bout for geerl o-degeerte LPs wth possble uboudedess: m s.t. = = b, ; The smple method termtes t most = 0,. m log σ m σ ( ) pvot steps, ether fds optml bs fesble soluto or detets the uboudedess.

32 Proof sketh I Let the obetve vlue of the lst bs fesble soluto be z*, d osder the shdow LP problem Obvously, the shdow LP s bouded wth mml vlue z*.. 0, *, ;, s.t. m z b = = = = =

33 m s.t. = = = b, Proof sketh II ; 0, The smple method wth the greedy pvotg rule, ppled to the orgl LP, would geerte the detl soluto d redued ost sequee s t s ppled to the shdow LP whh X rems bs vrble before detets uboudedess. I the shdow LP, the bs vrble vlues (eludg X ) stsfy the σ property. m m ( log ) I t most σ σ pvotg steps, the shdow LP fd the optml bs fesble soluto tht s the lst bs fesble soluto m of the orgl LP before detetg uboudedess.. s.t. = = = = b, = ; z*, 0,.

34 Remrks d Ope Problems Other pvotg rules? Is the poly terto method strogly polyoml tme lgorthm for determst MDP? Is there strogly polyoml tme lgorthm for MDP wth vrble dsouts or eve geerl LP? Solve LPs wth huge sze (bllo-dmeso) prte? The Smple Method Story Cotues