Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b, ad a are costat coefficiets. 11
Before startig, we eed to commet o several issues: The problem is formulated as miimizatio. If you wat to maximize, simply miimize the egative of the objective fuctio. All decisio variables are positive or zero (oegative). This meas the search space is i the upper right quadrat i the case of two variables. This decisio is made because may egieerig problems have oegative oly real-life meaig. If a variable ca be zero, positive, or egative, this variable is replaced by two oegative variables: x k = x +1 x +2 x +1 0 x +2 0 Cosider the case of a iequality costrait such as, a ki x i b k This iequality costrait ca be coverted ito a equality costrait by addig a ew variable such that, ( a ki x i ) + x +1 = b k Similarly, i the case of a iequality costrait such as, a ki x i b k This iequality costrait ca be coverted ito a equality costrait by addig a ew variable such that, x +1 is labeled surplus variable. ( a ki x i ) x +1 = b k Note: I the case of liear programmig, we are iterested i the cases where m <. If m =, there will be a uique solutio ad there is o eed for optimizatio. If m >, there will be redudat solutios that ca be elimiated. This ca be easily see i the case of two variables. Homework: 3.13, 3.18 (Formulate usig the stadard form ad solve graphically) 12
Defiitio #1: Lie Segmet While you are accustomed to the straight lie equatio that goes from - to +, i liear programmig we eed to thik i terms of lie segmets which are bouded betwee two poits i space. Cosider the case of a lie segmet bouded betwee poits I ad j, the followig formulatio will create the lie segmet. where, Defiitio #2: Vertex (Extreme Poit) A vertex is the ed poit of a lie segmet. L = λx i + (1 λ)x j 0 λ 1 Defiitio #3: Hyperplae A set of poits that satisfies the equatio: a i x i = a T x = b The hyperplae has -1 dimesio i a -dimesioal space. A hyperplae splits the space ito two closed half-spaces such that, Defiitio #4: Covex Set H + = {x a T x b} H = {x a T x b} A covex set is a group of poits where ay two poits ca be coected by a lie segmet whose all poits are withi the covex set. Examples: Ellipse or rectagle. Defiitio #5: Covex Polyhedro ad Covex polytope A covex polyhedro is a set of poits commo to oe or more half-spaces. A bouded covex polyhedro is labeled covex polytope. 13
Defiitio #6: Feasible Solutio A poit that satisfies: ax = b x 0 Defiitio #7: Basic Solutio A solutio that is obtaied whe -m variables are set to zero i order to obtai a solutio for ax = b Defiitio #8: Basis The value of the remaiig variable that came out of the basic solutio Defiitio #9: Optimal Solutio A feasible solutio that optimizes the objective fuctio. 14
Theorem #1: The itersectio of covex sets is covex. Theorem #2: The feasible regio of liear programmig problem is covex. Proof ca be see by formig a lie segmet betwee ay two poits satisfyig, ax = b. Ay poit o this segmet satisfies for ax = b Theorem #3: A local miimum of the liear programmig problem is the global miimum. This ca be easily see i the case of 2 variables where oly poits to be cosidered are the vertices. The covex ature of the feasible space esure that the miimum is at a extreme vertex. Exceptio: x 2 15
Solutio of a System of Simultaeous Liear Equatios It may help to start by rememberig Gauss Elimiatio techique to solve a system of liear equatios. If we express the system i a matrix form, Carl Friedrich Gauss (1777 1855), Wikipedia a 11 a 1 b 1 [ ] { } = { } a 1 a x b We ca elimiate the all compoets i the first colum ad replace a 11 by 1 ad adjust the row accordigly. If we progressively keep doig this, the above coefficiet matrix will become triagular. 16
a 11 a 12 a 1 [ a 22 a x b 1 1 2] { } = { b 2 } 0 x 0 0 a b This process of covertig a ik is labeled pivotig, which is obtaied by the determiat of four elemet icludig the pivot. a jj a jk a ij a ik a ik = (a jja ik a jk a ij ) a jj This process ca start from ay row ad ay colum. 17
Pivotal Reductio of a Solutio of a System of Simultaeous Liear Equatios The process ca be exteded to a system of m equatios with variables where m If we express the system i a matrix form, It is possible to reduce this system to become, I is m x m idetity matrix. a 11 a 1 b 1 [ ] { } = { } a m1 a m x b m a 1,m+1 a 1 I { } + [ ] { x m a m,m+1 a m The variables through x m are labeled pivotal variables. x m+1 b 1 } = { } x b m The variables +1 through x are labeled o-pivotal or idepedet variables. It is easy to see that the basic solutio is, b 1 { } = { } x m b m x m+1 { } = 0 x This process ca be repeated usig the coefficiet of aother pivotal variable to obtai aother solutio. Evetually we ca obtai all the basic solutios of the system. 18
Example 3.3: Fid all basic solutios of this system. This system has 3 equatios with 5 variables. 2 3 2 7 1 x 2 1 [ 1 1 1 3 0] x 3 = { 6} 1 1 1 5 1 x 4 4 { x 5 } Usig Gauss elimiatio, the solutio of the system is, 2 3 2 7 1 x 2 1 [ 1 1 1 3 0] x 3 = { 6} 1 1 1 5 1 x 4 4 { x 5 } 1 1.5 1 3.5 0.5 [ 0 0. 5 2 6.5 0.5] 0 2.5 2 8.5 0.5 x 2 x 3 x 4 { x 5 } 0.5 = { 5.5} 3.5 1 0 5 16 1 x 2 17 [ 0 1 4 13 1 ] x 3 = { 11} 0 0 8 24 3 x 4 24 { x 5 } The above is equivalet to, 1 0 0 1 7/8 x 2 2 [ 0 1 0 1 0.5 ] x 3 = { 1} 0 0 1 3 3/8 x 4 3 { x 5 } 1 0 0 1 7/8 [ 0 1 0] { x 2 } + [ 1 0.5 ] { x 2 4 x } = { 1} 0 0 1 x 5 3 3 3/8 3 Or, 1 7/8 { x 2 } = [ 1 0.5 ] { x 2 4 x } + { 1} x 5 3 3 3/8 3 If we assig the idepedet variables zero variables. 2 { x 2 } = { 1} x 3 3 19
We ca repeat the process for other combiatios of variables. { x 2 }, { x 2 }, { x 2 }, { x 3 }, { x 3 }, { x 3 x 4 x 5 x 4 x 5 The total umber of basic solutios (combiatios) is, I this case, x 4! ( m)! m! x 2 x 3 5! (5 3)! 3! = 10 x 2 x 3 x 2 x 4 x 3 x 4 }, { }, { }, { }, { } x 5 x 4 x 5 x 5 x 5 You ca easily see that as ad m icrease, the umber of possible solutios become icreasigly impossible to evaluate all of them, eve whe we have access to fast computers. This is the reaso, the simplex algorithm became a crucial tool. Homework: 3.5, 3.44 20
Simplex Algorithm The goals of the simplex algorithm, is to miimize the objective fuctio while satisfyig the costraits. The objective fuctio is added to the costraits. The system is reduced as i the previous sectio. I { x m } + f a 1,m+1 a 1 a m [ c ] a m,m+1 c m+1 A basic solutio is ca be obtaied as previously show. Theorem #4: x m+1 { } = { x b 1 b } m f 0 A basic feasible solutio is a optimal solutio with a miimum objective fuctio if all cost coefficiets c j, j = m+1,, are oegative. Improvig a No-Optimal Basic Feasible Solutio The objective fuctio ca be writte as, x m+1 f = f 0 + [c 1 c m ] { } + [c m+1 c ] { } x m x The objective fuctio is equal to f 0 at the case of basic solutio. To idetify the first move, idetify We will use x s as pivot. c s = miimum c j < 0 If all [a 1s a ms ] 0, the solutio is ubouded. If ot, fid (b i /a is ) for a is > 0 Fid r such that, Obtai the caoical form by pivotig aroud a rs b r = mi a rs a is > 0 ( b i ) a is 21
Example 3.4: Maximize, P = 4 + 16x 2 Subject to, 2 + 3x 2 16 4 + x 2 24 x 2 2.5 x i 0 i = 1, 2 We chage the problem to miimizatio by multiplyig the objective fuctio by -1. Miimize, f = 4 16x 2 Subject to, 2 + 3x 2 16 4 + x 2 24 x 2 2.5 x i 0 i = 1, 2 We itroduce three slack variables to have the equatios i the caoical form. x i 0 i = 1, 2, 3, 4, 5 22
The caoical form is, 2 + 3x 2 + x 3 = 16 4 + x 2 + x 4 = 24 x 2 + x 5 = 2.5 4 16x 2 f = 0 x 3, x 4, x 5, f are the basic variables. The basic solutio of this system is, x 3 = 16 x 4 = 24 x 5 = 2.5 f = 0 The solutio is ot optimal, sice the coefficiets of the o-basic variables i the objective fuctio equatio are egative, [c 1 c 2 ] = [ 4 16] 23
Solutio Steps: To idetify the first move, idetify I this case, we will select c 2 = 16 c s = miimum c j < 0 Fid r such that, b r = mi a rs a i2 > 0 (16 3, 24 1, 2.5 1 ) The pivot is a 3,2 To keep solutio orgaized ito a tableau form. Basic Variables b i variable x 2 x 3 x 4 x 5 f b i a is x 3 2 3 1 0 0 0 16 16/3 x 4 4 1 0 1 0 0 24 24/1 x 5 0 1 0 0 1 0 2.5 2.5/1 f -4-16 0 0 0 1 0 Basic Variables b i variable x 2 x 3 x 4 x 5 f b i a is x 3 2 0 1 0-3 0 8.5 8.5/2 x 4 4 0 0 1-1 0 21.5 21.5/4 x 2 0 1 0 0 1 0 2.5 f -4 0 0 0 16 1 40 Basic Variables variable x 2 x 3 x 4 x 5 f b i 1 0 0.5 0-1.5 0 4.25 x 4 0 0-2 1 5 0 4.5 x 2 0 1 0 0 1 0 2.5 f 0 0 2 0 10 1 57 b i a is Sice all, c i 0, termiate the search. The results show that the o-basic variables, x 3 = x 5 = 0 The basic variables are, = 4.25 x 2 = 2.5 f = 57 24
Special Cases: Ubouded Solutio: If all the coefficiets, a is are egative or zero, the problem is ubouded ad o solutio is available. x 2 Ifiite Number of Solutios: If all cost coefficiets of the o-basic variables, c is, are zeros, there are ifiite umber ad the solutio is lie segmet bouded by the results of the last two tableaus. where, L = λx i + (1 λ)x j 0 λ 1 x 2 Homework: Solve 3.47, 3.53 usig the simplex algorithm 25