Ellipsoid Method for Liear Programmig made simple Sajeev Saxea Dept. of Computer Sciece ad Egieerig, Idia Istitute of Techology, Kapur, INDIA-08 06 December 3, 07 Abstract I this paper, ellipsoid method for liear programmig is derived usig oly miimal kowledge of algebra ad matrices. Ufortuately, most authors first describe the algorithm, the later prove its correctess, which requires a good kowledge of liear algebra. Itroductio Ellipsoid method was perhaps the first polyomial time method for liear programmig[4]. However, it is hardly ever covered i Computer Sciece courses. I fact, may of the existig descriptios[, 6, 3] first describe the algorithm, the later prove its correctess. Moreover, to uderstad oe require a good kowledge of liear algebra like properties of semi-defiite matrices ad Jacobea[, 6, 3, ]. I this paper, ellipsoid method for liear programmig is derived usig oly miimal kowledge of algebra ad matrices. We are give a set of liear equatios Ax B ad have to fid a feasible poit. Ellipsoid method ca check whether the system Ax B has a solutio or ot, ad fid a solutio if oe is preset. The algorithm geerates a sequece of ellipsoids[], E 0, E,..., with cetres x 0, x,... such that the solutio space if there is a feasible solutio is is iside each of these ellipsoids. If the cetre x i of the curret ellipsoid is ot feasible, the some costrait say a T x b is violated for some E-mail: ssax@iitk.ac.i
row a of A, i.e., a T x i < b. As all poits of solutio space satisfy the costrait a T x b, we may add a ew costrait a T x > a T x i, without chagig the solutio space. Observe that this added half-plae will also pass through x i, the cetre of the ellipsoid. Thus, the solutio space is cotaied i half-ellipsoid itersectio of Ellipsoid E t with the half-plae. The ext ellipsoid E i+ will cover this half-ellipsoid ad its volume will be a fractio of the volume of ellipsoid E i. The process is repeated, util we fid a cetre x k for which Ax k B or util the volume of ellipsoid becomes so small that we ca coclude that there is o feasible solutio. If the set has a solutio, the there is a umber U [, 3, 5, 7], such that each x i < U, as a result, x i < U. If we scale each x i, x i x i/ U, the we kow that feasible poit x will be iside uit sphere cetred at origi x i. Thus, we ca take the iitial ellipsoid to be uit sphere cetred at origi. Special Case: x 0 Let us first assume that the added costrait is just x 0. The ellipsoid see figure, will pass through the poit, 0,..., 0, 0 ad also poits o lower dimesioal sphere x + x 3 +... + x itersectio of uit sphere with hyper-plae x 0. By symmetry, the equatio of the ellipsoid should be: αx c + βx + x 3 +... + x As, 0,..., 0 lies o the ellipsoid, α c or α c. Agai, the ellipsoid should
also pass through lower dimesioal sphere i x i with x, we have αc + β or β αc c c c c. Thus, the equatio becomes c x c + c c x + x 3 +... + x or equivaletly, c c For this to be ellipsoid, 0 c <. i x i + x + c x c We wat our ellipsoid to cotai half-sphere. If poit is iside the half-sphere the, x i. Moreover, the term x x x x will be egative, hece, c c x i + x + c x c i c c + x + c x c c c + c c Or x is also o the ellipsoid. Thus, etire half-sphere is iside our ellipsoid. We have certai freedom i choosig the ellipsoid uit sphere is also a ellipsoid which satisfies these coditios with c 0. We will choose c so as to reduce volume by at least a costat factor. If we scale the coordiates as follows:, x x c ad x i c c x i for i The the coordiates x is will lie o a sphere of uit radius. Thus, the volume of our ellipsoid will be a fractio c c c of the volume of a uit sphere. Or the ratio of volumes We wat V 0 V < α V 0 V c c c > α > for some costat α or equivaletly, c c c / c c c / c usig biomial theorem c c c c + c c + c c e.g. the poit 0,, 0,..., 0 lies o ellipsoid 3
We should have c < c or c Θ/. We choose c / +, as this maximises the ratio. With this choice, V 0 c / V c c + / Or equivaletly, usig α < e α ad + α < e α V V 0 + / + + < exp exp + exp + / Thus, volume decreases by a costat factor. 3 Geeral Case Let us choose our coordiate system such that the ellipsoid is aliged with our coordiate axes. This ivolves rotatio of axes ad possibly traslatio of origi. Next, we scale the axes, such that the scaled ellipsoid is a uit sphere with cetre as origi. Costrait a T x > a T x i which will be modified as axes are rotated ca agai be trasformed by rotatio to x > 0. Uder the ew scales, ratio of the two volumes, will agai be exp +. But, as both V 0 ad V will scale by the same amout, the ratio will be idepedet of the scale. Thus after k iteratios, the ratio of fial ellipsoid to iitial sphere will be at most exp If we are to stop as soo as volume of ellipsoid becomes less tha ɛ, the we wat exp ɛ V 0, or takig logs, l V 0 ɛ k + or k O l V 0 ɛ. k + k +. 4 Details ad Algorithm Geeral axes-aliged ellipsoid with cetre as c,..., c is described by Let fc c c c c / The f c c + c / + c / c 3/ c + c 3/ c c, puttig f c 0, we get c c c c or c c or c / + 4
x c a + x c a +... + x c a This ca be writte as x c T D x c where D is a diagoal matrix with d ii a i. If we chage the origi to c, usig the trasformatio x x c, the equatio becomes, x T D x. If we rotate the axes, ad if R is the rotatio matrix x Rx, we get x T R T D Rx. This is the equatio of geeral ellipsoid with cetre as origi. If we ow wish to further rotate the axes say to make a particular directio as x -axis, say with rotatio matrix S, x Sx, the equatio becomes x T S T R T D RSx. If we wat the cetre to be c c, c,..., c, usig the trasform x c x, the the equatio becomes droppig all primes x c T SR T D SRx c. This is the the geeral equatio of ellipsoid with arbitrary cetre ad arbitrary directio as x -axis. Coversely, if we are give a geeral ellipsoid x c T K x c with K R T S T D SR havig cetre c, we ca trasform it to uit sphere cetred at origi by proceedig i reverse directio. I other words, we first traslate the coordiate system chage the origi usig trasform x x c to make the cetre as the origi. Now the ellipsoid is of the form x T R T S T D SRx. Next, we rotate the coordiates twice, x SRx, ot just to make it axes-aliged, but also to make a particular directio as x -axis. The equatio becomes, x T D x. We the scale the coordiates, x x /a i or equivaletly, x D x ad we are left with uit sphere x T x with cetre as origi. Thus to summarise, x D x D SRx D SRx c Remark: Observe that as R ad S are rotatio matrices 3 R T R ad S T S K K R T S T D RS R S D S T R T R T S T D SR Let us ow look at our costrait a T x > a T x i. As x i c the costrait is actually, a T x c > 0. Let e, 0,..., 0 T. The, i the ew coordiate system, the costrait a T x c > 0 becomes e T x > 0 or e T D SRx c 0. Thus, we choose the directio or rotatio matrix S so that a T αe T D SR for some costat α. Or equivaletly, αe T at R T S T D, or αe DSRa. To determie the costat α, observe that α αe T αe a T R T S T DDSRa a T R T S T D SRa a T Ka, thus α a T Ka We kow that the ext ellipsoid i the ew coordiate system will be droppig all primes c x c + c c x + x 3 +... + x, or puttig c / +, ad simplifyig, + x + + x + x 3 +... + x Thus, the K ew diag +,,..., 3 If S is a rotatio matrix, the as rotatio preserves distace, x yx y T x ysx ys T x yss T x y T x yss T x y T, it follows that SS T I. 5
Ad the ew ellipsoid i ew coordiate system ca be writte as with primes x e T K + ew x e + Now, as x D SRx c x e + D SRx c e + D SR x c R S D e + But, as DSRa αe, e αdsra, ad recallig that R ad S are rotatio matrices, we have x e D SR x c R S D e + + D SR x c R S D + α DSRa D SR x c α + RT S T D SRa D SR x c α + Ka Or the ellipsoid is x e D SR x c + T Kew x e + x c α + Ka T K ew α + Ka T R T S T D K ewd SR D SR x c x c α + Ka α + Ka Thus, i the origial coordiate system, the ew cetre is: c + α + Ka c + + Ka a T Ka Ad the ew matrix K is: K R T S T D KewD SR, or equivaletly, K K R S DKewDS T R T R T S T DKewDSR Now, matrix Kew diag, +,..., I + E + Here I is a idetity matrix, ad E diag, 0,..., 0 is a square matrix with oly first etry as ad rest as 0. Observe that E e e T α DSRa α DSRa T α DSRaaT R T S T D 6
Ad + + Thus, K R T S T D I + α DSRaaT R T S T D DSR RT S T DIDSR α + RT S T DDSRaa T R T S T DDSR K α + RT S T D SRaa T R T S T D SR K K α + KaaT K α + KaaT K T Or, K K α + KaaT K T K Kaa T K T + a T Ka 5 Formal Algorithm Observe that computatio of ew cetre ad the ew K matrix does ot require kowledge of S, R or D matrices. I fact eve the matrix K is ot required. After possible scalig, we ca assume that the solutio, if preset, is i the uit sphere cetred at origi. Thus, we iitialise K I ad c 0 We repeat followig two steps, till we either get a solutio, or volume of solutio space becomes sufficietly small:. Let ax b be the first costrait for which ac > b. If there is o such costrait, the c is a feasible poit, ad we ca retur.. Make ad K c c + Ka + a T Ka K Kaa T K T + a T Ka Clearly, each iteratio takes O 3 time. Note that we do ot explicitly costruct ay ellipsoids. 7
Refereces [] D.Bertsimas ad J.N.Tsitsiklis, Itroductio to liear optimizatio, Athea Scietific, 997. [] R.M.Freud ad C.Roos, The ellipsoid method, WI 48, March 007 www.utdallas.edu/~dzdu/cs730/ellipsoid-4.pdf [3] H.Karloff, Liear Programmig, Birkhauser, 99. [4] L.G. Khachiya, Polyomial algorithms i liear programmig USSR Comput. Maths. Math. Phys., Vol 0, o., pp 53-7, 980 [5] K.Mehlhor ad S.Saxea, A still simpler way of itroducig iterior-poit method for liear programmig. Computer Sciece Review : - 06, CoRR abs/50.03339 05 ad vixra:50.0473 [6] C.H.Papadimitriou ad K.Steiglitz, Combiatorial Optimizatio-Algorithms ad Complexity, 98, PHI [7] R.Saigal, Liear Programmig, A Moder Itegrated Aalysis, Kluwer, 995. 8