Algorthms Research 4, 3: -5 DOI:.593/j.algorthms.43. T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng Gang Lu Technology Research Department, acroronter, Elmhurst, ew York Abstract We present a close-orm soluton or conve onlnear Programmng LP. It s close-orm soluton all the constrants are lnear, quaratc, or homogeneous. It s polynomal when apple to conve LP. It gves eact optmal soluton when apple to LP. The T-orwar metho moves orwar nse the easble regon wth a T-shape path towar the ncreasng recton o the objectve uncton. Each T-orwar move reuces the resual easble regon at least by hal. The T-orwar metho can solve an LP wth, varables wthn secons, whle other estng LP algorthms wll take mllons o years to solve the same problem run on a computer wth GHz processor. Keywors onlnear programmng, Lnear programmng, Quaratc programmng, Interor pont, Smple, Polyheron, polynomal, Close-orm soluton, Basc easble soluton. Introucton A stanar nonlnear optmzaton problem LP can be ormulate as: amze : Stanar LP: subject to : h =,,, subject to : k k =,,, Whch nvolves varables an constrants. Generc LP s har to solve. It has been ve nto several sub-set problems. Lnear programmng LP s one o the smplest sub-set o LP, an has got concentratons or researchers. Durng Worl War II, Dantzg an otzkn rst evelope the smple metho, whch has become one o the most amous algorthm snce then []. The Journal Computng n Scence an Engneerng lste the smple algorthm as one o the top algorthms o the twenteth century [5]. Khachyan propose ellpso metho n 979[, ]. In 984, Karmarkar starte the so calle nteror-pont revoluton See e.g. [9, 7]. There are also other successul algorthms or LP, such as prmal-ual nteror pont metho [3], logarthmc metho [4], etc. ost o the algorthms that are successul n LP have been * Corresponng author: gang.lu.989@gmal.com Gang Lu Publshe onlne at http://journal.sapub.org/algorthms Copyrght 4 Scentc & Acaemc Publshng. All Rghts Reserve etene to LP, especally the nteror-pont metho [3], prmal-ual metho [6, 8], barrer methos[7, 5], all have oun ther roles or solvng LP. Currently, most o the LP or LP algorthms are varetes o smple or nteror-pont methos [4, 6]. ost o the current estng LP algorthms as well as LP algorthms are path orente an matr orente [, ]. Path orente algorthm tres to search the optmal soluton n the set o Basc Feasble Soluton BFS, whch s the set o all easble regon ene by all o the constrants. However, the BFS s not obvous, thus nees to be calculate or mantane repeately. To mantan or to calculate BFS s one o the major cultes an takes most o the runnng tme or those algorthms. Also, the path orente algorthm nees to search verte by verte along the eges. It coul result a long path or ens up wth a loop. Another common property or those estng algorthms s matr orente, whch nclues a lot o matr operatons n nng the basc easble solutons. Convergence an eactness are two o the most mportant actors or an LP or LP algorthm. Smple metho can gve eact optmal soluton n general. However, t s not guarantee to gve soluton at polynomal tme. Ellpso metho guarantees a polynomal upper boun on 6 convergence at the orer O L ln L ln ln L [], an nteror pont metho mprove the convergence tme to 3.5 O L ln L ln ln L [9]. However, these polynomal tme algorthms may gve ε -appromate soluton, nstea o eact optmal soluton. People are stll lookng or new methos wth better convergence an can gve eact optmal
Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng soluton. We present a new approach, T-orwar metho, whch can solve LP n polynomal tme wth convergence spee O L ln L, an can gve eact optmal soluton when apple to LP. We rst construct the close-orm soluton or the easble regon ene by all the constrants. Havng the close-orm basc easble soluton n han, there s no nee to search paths an no nee to o any matr operatons, an there s no nee to save any easble solutons n computer system. We can always get t through the close-orm ormula. The close-orm soluton or the BFS plays key role n the methos propose by ths paper. In act, once we have the close-orm soluton or BFS, we can transorm the orgnal optmzaton problem nto an unconstrane problem or an equaton problem. The rest o the paper s organze as ollows. Secton scusses how to convert a generc LP to an LP wth lnear objectve uncton. The constrants n an LP ene a easble regon. Secton 3 analyzes basc concepts an propertes relate to a regon. Secton 4 analyzes the easble regon ene by a sngle constrant. Secton 5 urther analyzes the easble regon ene by all constrants an gves the close-orm soluton or the easble regon or conve LP. Secton 6 gves the graent uncton o the easble regon. The close-orm ormulae or the easble regon an neasble regon are summarze n secton 7. Secton 8 ntrouces the concepts o the ual-pont an the ual-recton. Secton 9 ntrouces the T-Forwar metho, whch gves close-orm soluton or conve LP. Secton ntrouces the greey T-Forwar metho, whch s a smple verson o the T-Forwar metho. Secton ntrouces the Facet-Forwar metho, whch can solve LP n polynomal tme. Secton gves close-orm soluton or LP, Quaratc Programmng, an LP wth homogeneous constrants. Secton 3 presents metho or solvng nonconve LP. Secton 4 gves the generc algorthm or T-Forwar metho. In Secton 5, we apply the T-orwar metho to solve some eample optmzaton problems. In Secton 6, we analyze the complety o the T-orwar metho an compare t wth some estng LP algorthms. Conclusons are presente n Secton 7.. Convertng Generc LP nto an LP wth Lnear Objectve A stanar LP can always be converte nto an LP wth lnear objectve uncton through the ollowng transorm: LP: In act, there shoul have another constrant: +. However, ths constrant s reunant when we try to mamze +. The LP problem lste n Equaton s a stanar LP wth Lnear LPL objectve uncton. So, we only nee to eal wth LP n the ollowng ormat: amze : = c a lnear uncton LPL: or R 3 Subject to : g or =,,, Here, we have roppe the constrants k to make the problem more generc. I we want to nclue the constrants, we can replace R wth R, where R s the set o all mensonal real numbers, an R + s k amze : Subject to : Subject to : + = + + h or =,,, the set o all mensonal real postve numbers. We may also use the ollowng notatons: R+ = { : k, or k =,,, } R = : <, or k =,,, R { } ± = R+ R k 4 5 6 3. Dentons Regarng a Regon The basc theory an ormula n ths paper are manly base on the easble an neasble regons ene by the constrants wthn an LPL. We rst ene some useul concepts or notatons regarng a regon A R.
Algorthms Research 4, 3: -5 3 DEFIITIO: Val path: Gven a path C R, path C s calle val path C A. DEFIITIO: Val straght path: A val path an the path s a straght lne. DEFIITIO: Connecte regon: A regon A R s calle connecte regon all ponts n that regon can be connecte through a val path. DEFIITIO: Conve regon: A regon A R s calle a conve regon any two ponts n regon A can be connecte through a val straght path. A conve regon can be epresse n mathematcs ormat as: A, A, µ R+, µ [,] + A Startng rom any pont n a conve regon, we can reach any other ponts n that regon through a val straght path. DEFIITIO: athematc ormulaton or bounary curve: A uncton G s calle a mathematc ormulaton or regon A R t can be use to ormulate regon A n the ollowng ormat: A ={ : G, R } 8 In the rest o ths secton, we assume G s a mathematc ormulaton or regon A. DEFIITIO: Bounary: I A s ormulate as the above equaton, the ollowng set s calle the bounary o A : ={ : G =, R } 9 DEFIITIO: Shne regon o a pont: Gven a pont A, there s a regon A A n whch all ponts can be connecte to through val straght path. A s calle the shne regon o pont. DEFIITIO: Sun shne set: Gven a pont set k A or k =,,, K the set { k } s calle a sun-shne set : K A = A k= That means the shne regons o a sun-shne set can cover all ponts n regon A. DEFIITIO: Orer o rect connecteness: The smallest number o ponts o a sun-set. For eample, any conve regon can be shne by any pont n the regon, an thus the orer o rect connecteness s. The orer o rect connecteness s or a crcle regon, an nnty or a crcle lne curve. DEFIITIO: Ray set: Gven a pont R, a unt vector z ˆ R wth z ˆ =, the ray lne startng rom pont along the recton ẑ s calle an ray set an enote as, z ˆ Ξ. Ξ, z ˆ can be ormulate as:, ˆ { : ˆ Ξ z = = + λz or λ R } Ξ DEFIITIO: Ray-bounary pont: gven a ray set, z, the shortest orere by λ ntersecton pont o Ξ, zˆ an the bounary s calle the ray-bounary pont o the par, ẑ an enote as, z ˆ. Wthn the ray-bounary pont uncton, s calle the start pont, ẑ s calle the movng recton., z ˆ can be ormulate as:, ˆ ˆ z = + zg, zˆ ˆ Where G, z + t oesn t have a postve soluton. It can be epresse n mathematcs ormat as: G, zˆ = mn +, mn{ t : G + tzˆ = or t R+ an t > } 3 DEFIITIO: Lne regon: The ntersecton set o Ξ, z ˆ an A s calle the lne regon assocate to the par o ˆ enotes the smallest non-zero postve root o λ n equaton G + λ z ˆ =, or µ µ + 7
4 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng, ẑ, an enote as A, z ˆ. A, z ˆ can be ormulate as: ˆ A, zˆ = { : = + λz or λ R+ an λ G, zˆ} 4 The ray-bounary pont, z ˆ s the bounary pont that can be reache an connecte rom pont an along the recton ẑ., z ˆ s the basc uncton an representaton or the bounary. We wll try to bul the, z ˆ or LPL n ollowng sectons. 4. Regons Dene by a Constrant The th constrant n the LPL as shown n Equaton 3 enes a easble regon n R. Let enote the easble enote the neasble regon wth the bounary nclue even they regon ene by the th constrant n Equaton 3, are easble ene by the same constrant, an enote the common bounary o an. ote that, as a conventon, we allow the bounary to be nclue n both the easble regon an neasble regon. We may requently use to represent the th constrant. By enton,,, an can be ormulate as: = = { : g, or R } { : g, or R } { : g =, or R } 9 Gven a pont R, g = an thus, s calle a bounary pont o constrant, an the constrant s calle a bounary constrant at pont. I g < an thus, s calle a easble pont to constrant, an the constrant s calle a val constrant at. I g > an thus, s calle an neasble pont to constrant, an the constrant s calle an nval constrant at. = = R = 5 6 7 8 Fgure. The easble regon an bounary ene by a constrant n ellpso type. p s the graent vector o at pont
Algorthms Research 4, 3: -5 5 A constrant s calle a smple constrant t has only one easble regon OR only one neasble regon. In ths paper, we assume all constrants are smple constrants. The ray-bounary pont, zˆ can be ormulate as:, The above equaton gves the easble bounary pont lnke to pont an along the recton ẑ. Fgure. llustrates the easble regon an easble bounary ene by a constrant n ellpso type. Smlarly, gven a pont, there shoul est a regon aroun n whch all ponts are neasble an can be connecte to through straght lne val to. Ths regon s calle neasble regon connecte to ene by constrant. Let enote ths regon an enote ts bounary. The ray-bounary pont can be ormulate as:, zˆ =, zˆ = + zg ˆ + zg ˆ, zˆ, zˆ,, z ˆ 5. Feasble Regon Dene by All the Constrants n LPL All the constrants n an LPL as shown n Equaton 3 ene an mensonal polytope n some pece-wse curves as ts acets, an each pece-wse curve must be a part o one o the easble bounary LPL. All the ponts n the easble regon are also calle Basc Feasble Soluton BFS. R, whch contans ene n Let enote the easble regon ene by all the constrants n LPL, enote the neasble regon ene by all the constrants n the same LPL, enote the common bounary o an. s the set n R that have all the constrants satse. By enton, we shoul have: = = 3 = =. = 4 = 5, there shoul have a regon that all ponts can be connecte to through a straght lne val to Gven a pont. Let enote ths regon an enote ts bounary. Smlarly, gven a pont, there shoul have a regon that all ponts can be connecte to through a straght lne val to. Let enote ths regon an enote ts bounary. Base on our enton, both an are conve, an 6 7 The lne-regon, z ˆ can be ormulate as:, zˆ = Ξ, zˆ =, zˆ 8 R =
6 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng, zˆ =, zˆ ={ : ˆ + z mn g, zˆ = Thus, the ray-bounary pont, z can be ormulate as: where ˆ, zˆ zg ˆ = +, zˆ = } 9 3 g, zˆ mn g, zˆ = 3 ote that all tems wthn the mn operaton n the above equaton are postve an none-zeros. Then the above equaton can be equvalently transorme nto: Where { } g, zˆ = zˆ Φ s the ollowng uncton: An δ s an Functon { } = δ { } δ Φ = = R R uncton ene as: Φ has the ollowng property: g, zˆ Φ ma ma { g, zˆ } 3 ; > or =,,, 33 34 Φ { } = 35 = Then, Φ { } can be nterprete as a probablty partton uncton o the set { } = ma{ }, an probablty or other cases. { } wth non-zero probablty when Φ can also be epresse n the ollowng ormat: ep Φ { } = lm β + ep = β β To make t easy or use, we can smply rop the lm operaton by replacng β wth a large number. Then, the above equaton can be wrtten as: { } Φ { } Φ, ep Φ { } = ep = β ; = β ma β 36 β 37 β s a large number such that epβ s acceptable or computer system wthout ata over low. ormally, β = 5 s goo enough or numercal calculatons. Φ { } s calle eact-partton uncton, whereas { } δ = = otherwse Φ s calle smooth-partton uncton. Here we use
Algorthms Research 4, 3: -5 7 { } Φ to enote the smooth ormat partton uncton, so that t can be stngushe rom the eact ormat. In act, they gve almost the same values once β s bg enough. In the rest o the paper, as long as there s no conuson, we wll use the same notaton { } Φ or both smooth an eact ormat partton unctons. Usng the partton uncton, the ray-bounary pont, z ˆ n both eact an smooth ormats becomes: { g, zˆ } z = + zˆ, ˆ g Φ, zˆ, = 38 Suppose g, z ˆ has been solve rom the equaton g + λ zˆ = or all =,,,, then both Equaton 3 an 38 gve close-orm soluton or a easble pont, z ˆ, whch s the ray-bounary pont o, zˆ., z ˆ wll gve all ponts o the shne-regon once pont ẑ goes through all possble rectons. Further, once pont, z ˆ wll gve all the ponts on easble bounary. goes through all ponts o a sun set, Thereore, Equaton 3, or ts smooth ormat Equaton 38, s a close-orm soluton or the easble bounary or BFS ene by all constrants n an LPL. 6. Graent Functon o the Feasble Bounary In real applcatons, we may requently nee the graent o the easble bounary. Snce we alreay have the close-orm soluton or the easble bounary, especally ts smooth ormat, the graent uncton can be calculate by erentatng the bounary uncton. However, here we gve another smple metho to calculate the graent uncton. Keepng n mn that partton uncton Φ { g, zˆ } s the probablty strbuton uncton or any constrant to be on the bounary. Then, we can use the partton uncton as a weght or probablty strbuton to calculate the graent uncton. Let, z ˆ Λ enote the graent o the easble bounary curve at the pont, z ˆ. Then t can be ormulate as: Λ, zˆ = Λ, zˆ Φ { g, zˆ } 39 = Where, z ˆ Λ s the unt graent vector o the constrant curve at the pont, z ˆ. Λ, z ˆ can be ormulate as: Λ, zˆ = p p, where p, zˆ = g, z = ˆ, 4 Λ, zˆ gves the average graent o all the curves passng through the pont, z ˆ on the easble bounary. It coul be slghtly erent rom the graent calculate by erentatng the bounary uncton. However, they wll be the same as long as the corresponng bounary pont contans only one constrant, whch s most o the case. Although the smooth ormat mght be slghtly erent rom the eact ormat, t always gves a goo appromaton when β s bg enough. The eact ormat graent uncton may not be contnuous, whle the smooth ormat s always smooth an contnuous. When calculatng the graent, t woul be better to use the smooth ormat partton uncton. 7. Summary o Equatons or Bouares Dene by All Constrants We have ormulate the close-orm soluton or the easble bounary an ts graent. Smlarly, we can construct the neasble bounary an ts graent by ormulatng the ray pont, z ˆ startng rom an neasble pont
8 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng an along the movng recton ẑ. As a summary, let us lst the bounary unctons an the graent unctons or an as ollowng: Supplementary unctons: { } Feasble bounary:, zˆ = + zˆ Φ = t t Ineasble bounary:, zˆ = + zˆ Φ { } = { } Feasble graent: Λ, zˆ = Λ Φ t = Ineasble graent: Λ, zˆ = Λ Φ { t } = Postve root o constrant: t g, z Eact partton uncton: { t} t t, 4, 4, 43, 44 = ˆ = mn{ t : g + tzˆ =, t > } δ ma t t Φ = δ ma t t = 45 ; t > ; or =,,, 46 Fgure. The rght se shows the easble bounary, one o ts ray-bounary pont, z ˆ, an ts graent Λ, z ˆ the neasble bounary, one o ts ray-bounary pont, z ˆ, an ts graent Λ, z ˆ. The let se shows
Algorthms Research 4, 3: -5 9 Smooth partton uncton: Graent o constrant : βt ep Φ { t} 47 ep βt = Λ = p p, p, zˆ = g, z = ˆ 48 These are the major unctons we are gong to use n the rest o ths paper. Fgure. llustrates some vectors or ponts relate to the bounary curve an can be calculate by the above equatons. 8. Dual-pont an Dual-recton Usng the bounary unctons gven n the prevous secton, we can start rom a pont to n a easble ray-bounary pont. I s an neasble pont, we can apply uncton, z ˆ to get a easble pont. Suppose s a easble bounary pont we have oun. Let B enote the bounary pont an the enng pont o as shown n Fgure 3. Π enote the contour plane o the objectve uncton passng through pont B, whch can be ormulate as Let =. Let n enote the unt graent vector o Π, enote the tangent plane o the bounary at pont B, an n enote the unt graent vector o at pont B. Let vector v be the projecte vector o n onto Π, pont D be the ntersecton pont o Fgure 3. Illustrates the ponts an curves mentone above. v wth, C be the m pont between B an D. to Fgure 3. A easble bounary pont DEFIITIO: ual-recton. Vector n., ts ual-pont m D, the m-nteror-pont m, an the ual-recton = v v s calle the ual-recton. I v = m D, we set the ual-recton
Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng DEFIITIO: ual-pont. Gven a bounary pont pont o, whch s the pont D or as shown n Fgure 3. DEFIITIO: m-nteror-pont. The mle pont between a bounary pont., ts ray-bounary pont along the ual-recton s calle the ual an ts ual pont s calle the m-nteror-pont o Here we borrow the wors nteror pont an ual pont to ncate some specal ponts relate to a bounary pont an the bounary curves. However, both o them are very erent orm the nteror-pont metho an the ual metho that are requently use n the optmzaton worl. Usng the bounary graent uncton, the graent vector n can be calculate as: n = n n The ual-recton can be easly ormulate as: m The ual-pont can be ormulate as: D n = v v, The m-nteror-pont can be ormulate as: { }, where n = Λ, zˆ = Λ Φ t m v = n = m + n = n n v = v 49 5 D =, m 5 D = + 5 9. T-Forwar etho or onlnear Programmng Problems Fgure 4. Illustraton o the T-Forwar vector T, n, an how t relate to the easble bounary pont m-nteror-pont common ege o m, the ual-recton an D m, the ual-pont, the T, an the T-orwar recton m. Pont T, n s the T-orwar pont. Pont K represents the
Algorthms Research 4, 3: -5 Bulng on the close-orm soluton or the easble bounary an or the bounary graent, we can gve a near-close-orm soluton or LP problems. Fgure 4. llustrates some o the notatons we are gong to use n ths paper an ther relatonshps. The ollowng notatons wll be use n the rest o the paper: : A easble pont,. : An neasble pont,. : A easble bounary pont,. n : The graent o the objectve uncton. : The ual pont o m,. : The m-nteror pont o ; : The tangent curve o at the pont ; : The tangent curve o at the ual-pont ; K : The common ege o an ; n : The graent o ; n : The graent o ; Π : The objectve contour plane passng through pont m = ; n n n D m D : Dual pont recton, T m : T-orwar recton, m T an ; =, n + n sgn n + n n = S : The pont that gves optmal soluton to the LPL problem; We wll use other varables epresse as a uncton o the par, n. ;, n as the basc varables an have Fgure 5. Illustraton o the weghte-nteror-pont m-nteror-pont m W an the m-nteror-pont s at the mle o the lne AD. C s the weghte-nteror-pont as the weght actors. Startng rom the pont C the en pont o o the tangent curve an the ual tangent curve W, the T-orwar recton m. A s the ntal pont. D s the ual pont. The W, whch ves the lne AD wth µ an µ T m wll pont to the pont G, whch s the common ege
Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng W Fgure 5. Illustrates weghte-nteror-pont an how to calculate the weght µ. DEFIITIO: T-orwar-recton. T-orwar recton s ene as: m T = n n, n = n + n sgn n + n n 53 W DEFIITIO: Weghte-nteror-pont. The weghte-nteror-pont s ene as: W = µ + µ = + µ 54 where µ = ; = n α π arccos n ; α arcsn n cos θ + α cosα cosθ θ 55 = DEFIITIO: T-orwar-pont. The ray-bounary pont startng rom the weghte-nteror-pont T T-orwar recton m s calle the T-orwar-pont, enote as T, n. n W an along the Usng the weghte-nteror-pont as startng pont an the T-orwar-recton as the orwar move recton, the ray-bounary-pont W, m T wll pont to the pont K, whch s the common ege o the tangent curve an the ual tangent curve. The T-orwar-pont wll try to reach the ege or verte whenever possble. DEFIITIO: T-orwar-uncton: The uncton maps rom the par, n to the T-orwar-pont T, n s calle T-orwar-uncton, whch s also enote as T, n. By enton, the T-orwar-pont T, n can be ormulate as: W T T D T T, n =, m = µ + µ, m = µ + µ, m, m Thus, the T-orwar-pont an the T-orwar-uncton T, n can be wrtten as: D T T, n = µ + µ, m, m 56 The T-orwar pont T, n s also a bounary pont. We can apply the T-orwar-uncton to t to get another T-orwar-pont. By applyng the T-orwar-uncton repeately, we wll eventually reach a pont where the T-orwar-pont cannot move urther. That must be a pont where the objectve contour plane an the easble bounary have just one common contact pont. That s the optmal soluton or the LP n the shne-regon. an the objectve graent DEFIITIO: T-orwar-soluton: Gven a easble bounary pont n, let S, n enote the optmal soluton or the LPL problem n the shne-regon wth as ts bounary. By repeately applyng the T-orwar-uncton, we can reach a soluton or the LPL. Ths soluton s calle T T-orwar-soluton an enote as S, n. The T-orwar soluton can be ormulate as: S Or epresse n teratve ormat: T S T k, n = T 57, n = T k = T, n, T T, n, n,, n K, n or k =,,, K DEFIITIO: T-orwar pont metho: Usng the T-orwar-uncton to solve LPL problem s calle T-orwar pont metho, or smply T-orwar metho. DEFIITIO: T-orwar path: The seres o the T-orwar ponts startng rom the ntal pont an enng at the optmal pont S, n make up a path, whch s calle T-orwar path. 58
Algorthms Research 4, 3: -5 3 T DEFIITIO: T-orwar accuracy: The relatve erence between S, n T-orwar accuracy, enote as T, n. That s: an S, n s calle T T, n = S, n S, n S, n 59 Gven a easble bounary pont, we can n ts ual-pont. Both an are on the same contour plane Π, an thus gve the same objectve value. I the easble regon s conve an contnuous, all the ponts between an shoul be nteror ponts an thus have some ponts between the contour plane an the easble bounary. For those ponts that are outse the contour plane Π an along the recton wll gve better objectve values. Partcularly, the T-orwar-pont between W T the easble bounary we move rom along the recton m. As long as the pont the ray-bounary-pont W, m T wll entely gve a larger objectve value than the pont W n, whch s the normal recton o Π,, wll possbly gve the longest move wthn W. s a strct nteror pont, ormally, T-orwar-pont metho gves ɛ-appromate soluton or LP. For LP problems, the T-orwar-pont metho s esgne to move to the ege or verte possble. The nal soluton s always on the optmal ege or verte. As long as the LP has a soluton, the T-orwar-pont metho guarantees to gve the eact optmal soluton.. Greey T-Forwar etho or onlnear Programmng Problems The Greey T-orwar metho a smple verson o the T-orwar metho. Everythng s the same as the T-orwar m metho ecept t takes the m-nteror-pont W as the startng pont n each T-orwar step. So, we replace wth m, the T-orwar metho wll become greey T-orwar metho. Compare wth the T-orwar metho, the greey T-orwar metho oesn t have any avantages ecept ts smplcty. It may work as ecent as the T-orwar metho. However, t may not gve eact optmal soluton when apple to LP problems. Everythng n the prevous secton s applcable to the greey T-orwar metho. There s no nee to repeat them. Here we just lst the greey T-orwar uncton. By enton, the greey T-orwar-pont G, n can be ormulate as: m T D T G, n =, m = +, m, m Thus, the greey T-orwar-pont an the greey T-orwar-uncton G, n can be wrtten as: D T G, n = +, m, m 6 The greey T-orwar soluton can be ormulate as: S G k, n = G k = G, n,. Facet-orwar etho or Lnear Programmng K, n or k =,,, K The T-orwar metho gves ɛ-appromate soluton or LP, an gves eact optmal soluton or LP, both n polynomal tme the easble regon s conve. However, the complety or the T-orwar metho s lttle bt cult to prove. In ths secton, we ntrouce acet-orwar metho, whch smply apples the T-orwar metho on acet. Gven a easble bounary pont, that s:, we can n ts constrant curve Π =. Let The acet-orwar metho rst ns the local optmal soluton o the LP problem restrane n the acet Π 6 be the acet on an contans the Π, let t be 6 S.
4 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng S. Ths can be Then we cut the easble bounary usng the objectve contour plane Π, whch passes the pont easly one by ang the uncton S as an etra constrant to the orgnal LP lste n Equaton 3. Let us call the easble bounary o the LP problem as the orgnal easble bounary. Once we a an objectve contour plane S to the orgnal problem LP, the orgnal easble bounary wll be cut by the contour plane nto two parts. DEFIITIO: LPL on resual easble regon. The LPL problem o the orgnal problem restrane n the resual easble bounary. In other wors, t s the ollowng LPL problem: Resual LPL: amze : Subject to : Subject to : = c g S or =,,, DEFIITIO: Resual Feasble Bounary. The easble bounary o the resual LPL problem. Actually, t s the set o the orgnal easble bounary wth objectve value greater than S. Snce S s the largest value on the acet Π, then the constrant becomes completely reunant to the LP on resual easble bounary. The constrant can be completely remove rom the problem by substtutng wth the new constrant S. The new constrant oesn t have any mpact when we search on another acet an try to n objectve values greater than S. So, the problem becomes a LP wth the number o constrant reuce by at least. There are constrants n total, so, at most o steps o such acet orwar search, we wll n the optmal soluton. One metho or nng the local optmal soluton wthn a acet s calle on-bounary T-orwar metho. The on-bounary T-orwar metho smply apples the T-orwar metho n a acet, whch means the ual pont, the ual recton an the T-orwar recton are all restrane n the acet. Another metho s calle ege-orwar metho. Gven a starng easble pont, we can apply the ray-bounary pont uncton, z ˆ repeately to the startng pont, but wth the movng recton vector ẑ constrane n the acet Π, sometmes may be along the ege o that acet. By repeatng ths process, we can eventually reach the local optmal soluton o the spece acet. Π 63. Close-orm Solutons T Everythng n the optmal solutons S, n as escrbe n the prevous sectons are n close-orm ormat ecept the root value g, z ˆ. I g, z ˆ can also be epresse n close-orm ormat, then all these solutons S, n gve close-orm soluton or LPL or LP n the regon. Actually, we can gve close-orm soluton or, z ˆ or some specal cases. In ths secton, we gve close orm soluton or LP, QCQP, an LP wth g Homogeneous LPH constrants... Close-Form Soluton or Lnear Programmng I the problem s LP, then all constrants are lnear unctons. The constrant can be epresse as: ˆ Then g + λ z = becomes: From whch we can easly solve λ as: g + g = b c 64 g = b + zˆ + c = λ 65
Algorthms Research 4, 3: -5 5 c b = b zˆ Base on the enton or g, z ˆ, we have: b zˆ = g, zˆ = λ else λ > + else λ 66 It s n close-orm ormat, an thus Equaton 58 gves close-orm soluton or LP. Also, the easble bounary or LP must be conve an have just one regon. Then, Equaton 58 gves global optmal soluton or LP... Close-Form Soluton or Quaratcally Constrane Quaratc Programmng In the case o QCQP, all the constrants are quaratc. The constrant g can be epresse as: T T g = A + B + C 68 Then g + λ zˆ = becomes: 69 where a A jk zˆ j zˆ k k= j= g + λz ˆ = aλ + bλ + c = jk ˆ j k kj ˆk j j ˆ j k= j= j= ; = ; b= A z + A z + Bz λ can be solve through the well known close-orm soluton or quaratc equaton: g c A + C = k = j= jk k jk 67 7 b + b λ =, λ =, = b 4ac 7 a a By enton,, z ˆ can be ormulate as: g +, zˆ = λ λ else else Then Equaton 58 gves close-orm soluton or QCQP. < or λ an λ λ > an λ λ or λ λ > 7.3. Close-Form Soluton or LP wth Homogeneous Constrants DEFIITIO: Homogeneous uncton. A uncton s calle homogeneous wth egree γ : γ µ = µ, µ R Here γ s a postve nteger number. DEFIITIO: LP wth homogeneous constrant LPH. A nonlnear programmng n whch all the constrants are homogeneous. For an LPH, the th constrant can be epresse as: g or =,,, 74 An has the ollowng property: g γ g µ = µ g or,,, 73 = 75
6 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng For an LPH, the orgn pont s always a easble pont. The g, z ˆ s the root o the equaton g + λ zˆ =. We can choose =. Then, z ˆ s the root o λ z ˆ =. Wth the homogeneous property, λ can be solve as: By enton,, z ˆ can be ormulate as: g g g λ = g, zˆ = ˆ 76 g z γ g zˆ + Thereore, Equaton 58 gves the close-orm soluton or LPH. γ g zˆ > else 77 3. Fn the Global Optmal Soluton or LPL Gven a easble pont, we can n the easble bounary pont by applyng, ˆ =. Then, Equaton 58 an 6 gve the optmal soluton or LPL n the shne regon. I happene to be T the whole easble regon, then the soluton S, n gven by Equaton 58 s the global optmal soluton or LPL. For eample, when s conve, we shoul have T =. Then S, n s the global optmal soluton or LPL. Partcularly, the easble regon o an LP s always conve, we can always gve global optmal soluton through the T close-orm soluton S, n or LP. However, T cannot cover all the easble regon, S, n mght be a local optmal soluton. In that case, we nee to n a sun-shne-set that can cover all ponts n the easble regon through rect path connecton. The neasble bounary uncton, z ˆ gven n Equaton 4 can be useul n nng the sun-shne set. Suppose, we can apply, z ˆ wth some ranomly generate rectons ẑ untl we n a easble pont that s not n the shne regons that have been searche. 4. Algorthm or T-Forwar etho Base on the prevous sectons, the algorthm or T-Forwar metho can be summarze as ollows. an Proceure: T-orwar-metho Input: ε : requeste precson; : number o the constrants; : umber o varables; c : graent o the objectve : an easble nteror pont; an all the coecents n each o the constrant g. Output: A easble bounary pont that mamzes the objectve uncton. uncton;. Formulate the orgnal LP n the ollowng LPL as shown n Equaton 3: amze : = c a lnear uncton LPL: or Subject to : g or =,,,. Intalzaton.. Set n c c... Set z ˆ n..3. Calculate ray-bounary pont : R
Algorthms Research 4, 3: -5 7 get-ray-bounary-pont, ẑ.4. Set /. 3. Whle > ε o loop 3.. Set. 3.. Set get-t-orwar-pont, ẑ. 3.3. En Whle-o loop. 4. Return En o T-orwar-metho as the optmal soluton an as the optmal objectve value. Functon get-t-orwar-pont, ẑ Input: A bounary pont, an all the nput normaton or the LPL. T Output: A bounary pont, whch s the T-orwar pont wth as startng pont.. Set ẑ ;. Calculate { t } Φ { } Φ or the pont : t get-partton-uncton, ẑ ; 3. Calculate graent vector n : n get-graent-vector, ẑ 4. Calculate ual pont through Equaton 5: get-ual-pont, ẑ 5. Calculate graent vector : n n get-graent-vector, ẑ T m through Equaton 53: = n + n sgn n + n n, m T = n n 6. Calculate T-orwar recton n W 7. Calculate weghte-nteror pont through Equaton 54: cos θ + α µ = ; n n cosα cosθ = α π arccos ; W = + = + µ µ T 8. Calculate T-orwar pont : W get-ray-bounary-pont T, m En o Functon get-t-orwar -pont µ Functon get-ray-bounary-pont, ẑ Input: A easble nteror or bounary pont, a unt vector ẑ, an all the nput normaton or the LPL. Output: A bounary pont. Φ or the par, ẑ :. Calculate { t } Φ { } t get-partton-uncton, ẑ ;. Calculate ray-bounary pont through Equaton 4:, zˆ = + zˆ t Φ { t } = En o Functon get-ray-bounary-pont
8 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng Functon get-graent-vector, ẑ Input: A bounary pont, a unt vector ẑ, an all the nput normaton or LPL. Output: n, whch s the graent vector o the easble bounary at. Φ or the par, ẑ ::. Calculate { t } Φ { } t get-partton-uncton, ẑ ;. Calculate graent vector Λ or each constrant at the pont Λ = p p, p, zˆ = g, z = ˆ 3. Calculate graent vector n through Equaton 43: n Λ, zˆ = { } Λ Φ t = En o Functon get-graent-vector Functon get-ual-pont, ẑ Input: A bounary pont, an all the nput normaton or the LPL. Output: A bounary pont, whch s the ual pont o.. Calculate graent vector n : n get-graent-vector, ẑ. Calculate ual recton m D n = v v, D m through Equaton 5: v = n n 3. Calculate ual pont : get-ray-bounary-pont D, m En o Functon get-ual-pont n n through Equaton 48: v = v Functon get-partton-uncton, ẑ Input: A easble pont, a unt vector ẑ, an all the nput normaton or the LPL. Output: Φ { t }, whch s the partton uncton at the easble bounary pont.. Calculate t g, zˆ. For lnear constrants, t can be calculate through Equaton 66 an 67. For quaratc = t constrants, can be calculate through Equaton 7, 7, an 7. { }. Calculate Φ through Equaton 45, 46, or 47. t En o Functon get-partton-uncton 5. Solvng Lnear Programmng Eamples A stanar LP can be epresse as: amze = c or R LP: 78 Subject to : g = a + b or =,,, DFIITIO: egatve Constrant. A constrant n the ormat s calle a negatve constrant g, an a postve constrant otherwse. g
Algorthms Research 4, 3: -5 9 A negatve constrant s a val constrant at the orgn pont. I all constrants are negatve constrants, then the orgn pont wll be a easble pont. I the orgn pont O s a easble pont, then, all constrants must be negatve constrants. Suppose s a easble pont we have oun, or eample, we can apply smple phase-i to n a easble pont. We can transorm all constrants nto negatve constrants through the ollowng transorm: = y + 79 Then the orgnal problem lste n Equaton 78 becomes: amze y = c y + c or R LP ater varable transorm: 8 Subject to : g y = a y + or =,,, where = a + b or =,,, 8 Wthout losng generalty, we can assume that the LP problem has been transorme nto an LP wth all negatve constrants. In other wors, we assume the LP lste n Equaton 78 has the ollowng property: 5.. Eample : An LP wth Varables an Constrants b or =,,, 8 Our rst eample s an LP wth varables an constrants. The nput ata are ranomly generate numbers n the range [-, ] or a, an n the range [-,-] or b. Table lsts the nput ata n table ormat. k We rst apply the close-orm soluton, as lste Equaton 4, to raw the easble bounary, as shown n Fgure 6. It can be observe that the easble bounary contans only 7 straght lnes. Then we know that there are about 93 constrants are reunant or ths problem! However, there s no nee or us to gure out whch constrant s reunant an whch s not. The close-orm bounary uncton can o the nce job or us an can lter out all the reunant constrants automatcally. = Table lsts step-by-step results or T-orwar metho to solve LP Eample- wth objectve. Table. Input ata or LP Eample- n table ormat
Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng = Table. T-orwar metho takes orwar steps to gve eact optmal soluton or LP Eample- objectve Table 3 lsts step-by-step results or T-orwar metho to solve LP Eample- wth objectve =. Fgure 6. also shows the rst ew T-orwar steps or the results lste n Table an Table 3. T-orwar metho takes only two orwar steps to solve LP Eample- an gve eact optmal soluton. = Table 3. T-orwar metho takes orwar steps to gve eact optmal soluton or LP Eample-. objectve Fgure 6. Fnng optmal solutons or LP Eample- through T-orwar metho. The blue lnes on the rght se llustrate the T-orwar path or mamzng = see Table. The re lnes on the let se llustrate the T-orwar path or mamzng = see Table 3. The vertcal ashe lnes represent the ual rectons. T-orwar metho takes orwar steps to n eact optmal soluton or both objectve unctons
Algorthms Research 4, 3: -5 5.. Eample : An LP wth Varables an Constrants Our secon eample s an LP problem that s use to escrbe the nteror-pont metho at the webste: http://en.wkpea.org/wk/karmarkar's_algorthm. Ths problem can be ormulate as: LP Eample-: amze : + Subject to : p + p, p = or =,,, Table 4 lsts the nput ata n table ormat or LP Eample-. Fgure 7. shows the easble bounary o ths problem. By applyng Equaton 4, we can raw the easble bounary as shown n Fgure 8. It can be observe that the bounary curve rawn through our close-orm Equaton 4 s eactly the same as the easble bounary as shown n Fgure 7. Ths gves another valaton or our close-orm soluton. = + Table 5 lsts step-by-step results or T-orwar metho to solve LP Eample- wth objectve. T-orwar metho takes orwar steps to gve eact optmal soluton or ths problem as shown n Fgure 8., whle nteror metho takes many steps to reach an ɛ-appromate soluton as shown n Fgure 7. = + Table 4. Input ata or LP Eample- wth objectve 83 Fgure 7. The easble bounary curve or LP Eample-. The re lne represents the path searche by the nteror metho. Ths gure s use to escrbe the nteror pont metho at the webste http://en.wkpea.org/wk/karmarkar's_algorthm
Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng = + Table 5. T-orwar metho gves eact optmal soluton n orwar steps or LP Eample- objectve Fgure 8. Startng rom a easble bounary pont, T-orwar metho takes two steps to n the eact optmal soluton or LP Eample-. The re arrow lne represents the T-orwar move. The ashe lnes represent the ual recton 6. Complety Analyss Let us analyze the complety or the T-orwar-pont metho an the Facet-orwar metho ntrouce n prevous sectons. The T-orwar-pont metho s smply to call the T-orwar-uncton T, n repeately. Each T-orwar-uncton T, n, as shown n Equaton 56, nclues two calls o the bounary uncton, z ˆ an two calls o the graent uncton, z ˆ Λ. Each o these unctons nees to calculate the roots g or =,,,. The complety or calculatng one g through Equaton 67 s O L, where L represents the mamum arthmetc calculatons or calculatng one g or the length o the nput or one constrant. For LP problems, L =. The complety or calculatng all the roots g or =,,, s O L. Once g s calculate, the bounary uncton, z ˆ an the graent uncton Λ, z ˆ can be calculate n O. Thus, the T-orwar-uncton T, n can be calculate n O L.
Algorthms Research 4, 3: -5 3 ow let us estmate how many T-orwar steps are neee to n the optmal soluton through T-orwar metho. Here we only conser conve LP, whch means the easble regon s conve. Fgure 9. Intally the resual easble regon s, the resual easble regon s wth E as ther upper boun ellpso. Ater one T-orwar a move, the resual easble regon becomes wth E as ther upper boun ellpso. E s an ellpso contane n E an centere at T a an an upper boun or E. The curve GH bsects the regon AGDCA. The volume o E s less than hal o a than hal o E. Then, ater one T-orwar step, the resual easble regon s reuce at least be a actor o 4 As llustrate n Fgure 9., at the ntal easble pont, we use the objectve contour plane E, an the volume o E s less Π to cut the easble regon. The resual easble regon must be n the regon boune by the two plane curves an, where s the tangent plane curve o passng the pont, an s the tangent plane curve o passng through the ual. Pont G represents the common ege o the two plane curves an. The lne GH represents the pont bsector o the two curves an. Beore we start the T-Forwar step, the upper boun ellpso o the resual easble regon s n the regon GAD. Here we only conser the ellpso that s cut by the contour plane Π. Ater the rst T-orwar step, the easble regon s cut at the T-orwar pont T. Snce an t s a bounary pont o the easble regon. Then the resual regon ater, wth T one o ts bounary pont. Thus must be ether n the regon GBT or GET, ether one s hal o the regon GBE. We a raw another ellpso E, whch wth T a as ts center an upper boune by E, an wth E to be contane n E. We shoul have: Vol E Vol E a, Vol E a Vol E T s esgne to be along the recton GH, T becomes the regon as, Vol E Vol E 84 4 k Thereore, each T-orwar step moves the cuttng plane Π close to the target urther an reuces the volume o the resual easble regon at least by a actor o 4. Ater K tmes o T-orwar steps, the volume o the resual easble regon wll be reuce by K 4. The relatve erence between the T-orwar soluton S T, z ˆ an the optmal soluton S
4 Gang Lu: T-Forwar etho: A Close-Form Soluton an Polynomal Tme Approach or Conve onlnear Programmng wll be reuce to the orer o ln ε tmes. K 4. In other wors, the requeste precson s ε, we nee to call T-orwar uncton Ellpso metho s proven to be polynomal through smlar metho as we use above [, ]. In summary, the runnng tme or T-orwar metho to gve an ε -appromate soluton or LP s L ln ε I the problem s LP, the T-orwar metho can gve eact optmal soluton wth complety O L ln. O. The acet-orwar metho smply calls, z ˆ an Λ, z ˆ at most tmes to gve local optmal soluton n a acet. It takes L O tme to get a soluton or one acet an have the number o constrant reuce at least by. Then, t nees to run at most tmes to reuce all constrants an gve nal optmal soluton or LP problems. Thereore, the runnng O 3 L. tme or acet-orwar metho s Table 6 compares the methos propose n ths paper wth the current best known estng LP algorthms, nclung smple metho, the ellpso metho, an the nteror pont metho. Table 6. Comparson o the T-orwar metho wth smple, ellpso, an the nteror pont metho Suppose we run a computer wth Ghz processor, Table 7 lsts the estmate runnng tme n secons or varous methos an varous an L. For smplcty, here we assume = L. Table 6. Estmate runnng tme n secons on a computer wth GHz processor 4 For eample, we want to solve an LP wth = L =, our propose T-orwar metho wll take couple o secons 6 to solve t, whle the nteror pont metho wll take years to solve the same problem, whch s a msson mpossble or most o the current estng LP algorthms. 7. Conclusons The man contrbutons o ths paper nclue the close-orm solutons or LP, QCQP, an the near-close-orm soluton or LP. Another contrbuton s the T-orwar metho or LP an the acet-orwar metho or LP. Both methos are polynomal tme. The T-orwar metho an the acet-orwar metho are not only aster than any estng LP or LP algorthms, but also gve eact optmal soluton when apple to LP. REFERECES [] [AG93] E. L. Allgower an K. Georg. Contnuaton an path ollowng. Acta umerca, 993, Cambrge Unversty Press, Cambrge, UK, 993, pp. 64. [] [AG97] E. L. Allgower an K. Georg. umercal path ollowng. Hanbook o umercal Analyss, Vol. 5, P. G. Carlet an J. L. Lons, es., orth-hollan, Amsteram, 997, pp. 3 7. [3] [AGJ] P. Arman, J. C. Glbert, an S. Jan-J egou. A easble BFGS nteror pont algorthm or solvng conve mnmzaton problems. SIA J. Optm.,, pp. 99.
Algorthms Research 4, 3: -5 5 [4] [Bam86] E.R. Barnes. A varaton on Karmarkar s algorthm or solvng lnear programmng problems. athematcal Programmng, 36:74 8, 986. [5] [CSE] Computng n Scence an Engneerng, volume, no.,. [6] [CO] A. R. Conn,. I.. Goul, D. Orban, an P. L. Tont. A prmal-ual trust-regon algorthm or non-conve nonlnear programmng. ath. Program., 87, pp. 5 49. [7] [CG97] A. R. Conn,. I.. Goul, an P. L. Tont. A globally convergent Lagrangan barrer algorthm or optmzaton wth general nequalty constrants an smple bouns. ath. Comp., 66 997, pp. 6 88, S S. [8] [FG98] A. Forsgren an P. E. Gll. Prmal-ual nteror methos or nonconve nonlnear programmng. SIA J. Optm., 8 998, pp. 3 5. [9] [Kar84]. K. Karmarkar. A new polynomal-tme algorthm or lnear programmng. Combnatorca, 4:373 395, 984. [] [Kha79] L. G. Khachyan, A Polynomal Algorthm n Lnear Programmng. Doklay Akaem auk SSSR 44:S 979, 93-96, translate n Sovet athematcs - Doklay : 979, 9-94. [] [KK79] Kozlov,. K.; S. P. Tarasov an Leon G. Khachyan 979. Polynomal solvablty o conve quaratc programmng. Doklay Akaem auk SSSR 48: 49 5, translate n Sovet athematcs - Doklay : 8 [] [K7] V. Klee, G. nty. How goo s the smple algorthm? Inequaltes III, ew York-Lonon: Acaemc Press: pp. 59-75. [3] [eh9] ehrotra, Sanjay. "On the Implementaton o a Prmal-Dual Interor Pont etho". SIA Journal on Optmzaton 4: 575, 99. [4] [W] G. P. ccormck an C. Wtzgall. Logarthmc SUT lmts n conve programmng. ath. Program., 9, pp. 3 45. [5] [Pol9] R. Polyak. oe barrer unctons theory an methos. ath. Program., 54 99, pp. 77. [6] [V86] R. J. Vanerbe,. S. eketon, an B. A. Freeman. A mocaton o Karmarkar s lnear programmng algorthm. Algorthmca, :395 47, 986. [7] [Wr4]. H. Wrght, The nteror-pont revoluton n optmzaton: Hstory, recent evelopments, an lastng consequences. Bulletn o the Amercan athematcal Socety 4: 39, 4.