Linear & Integer programming

Transcription

1 ELL 894 Performance Evaluation on Communication Networks Standard form I Lecture 5 Linear & Integer programming subject to where b is a vector of length m c T A = b (decision variables) and c are vectors of length n A is an m n matri called the constraint matri (m < n) Any LP can be epressed as a standard form Jun B. Seo Standard form II subject to = , 4, free where free means a variable without specified lower or upper bounds For the third constraint, introducing ecess variable e 1, e 1 = 4 and e 1 Standard form III Treating the upperbound of 4 as a constraint,e subject to = e 1 = 4 4 1,, free, e 1 Introducing slack variable s 1, the second constraint becomes s 1 = 5 and s 1 The fourth constraint is also changed into 4 + s = 4 and s 5-5-4

2 Standard form IV Standard form V,s,e subject to = s 1 = e 1 = 4 + s = 4 1,, free, e 1, s 1, s Free variable is epressed as, The 1 is transformed into = 1 = 1 + 1,s,e subject to = s 1 = e 1 = 1 + s = 4 1,,,, e 1, s 1, s The constant term, 1 in the objective function can be ignored With four constraints, we have decision variables as = [ 1,,,, s 1, s, e 1 ] T Standard form VI 1,, subject to = 1,, are free Let 1 = + 1 1, + 1, 1 and 1 = Then, the above problem is cast into standard form as,y,z,s subject to s 1 = = + 1, 1, +,, +,, s 1 Fractional linear progamming c T + α d T + β subject to A b, Mostly, it holds that d T + β > for A b If we let y = t and t = 1 d T +β y,t c T y + tα subject to Ay tb d T y + βt = 1 y, t

3 Basic solution & Basic feasible solution I A point R n is called a basic solution if All equality constraints are active The columns of the constraint matri corresponding to the nonzero elements of are linearly independent For a full row rank matri A of m n, we can write ] A = [B N ] [ B N = B B + N N = b B : (nonzero) m basic variables whose constraint coefficients correspond to an invertible m m basis matri B of A m columns of B need to be linearly independent (for the inverse) If one or more elements of B are zero, LP is called degenerate Degeneracy arises when the LP has a redundant constraint Iterations of simple method can t improve the objective value 5-9 Basic solution & Basic feasible solution II N : n m nonbasic variables which are all zeros Choosing n m variables i to be zero is the same as choosing n m of the equality constraints to be active The set of basic variables is called the basis Consider an LP 1 subject to s 1 = s = 1 + s = 1,, s 1, s, s 5-1 Basic solution & Basic feasible solution III The constraint matri is 1 1 A = b = 1 1 If 1 and s are chosen as nonbasic variables (set to zero), we have the basic variables, s 1, and s as s 1 = B 1 b = 1 = 1 s 1 This is basic but not feasible since s 1 < 5- Basic solution & Basic feasible solution IV If 1 and are chosen as nonbasic variables (set to zero), we have the basic variables s 1, s 1, and s as 1 s 1 1 s = 1 = s 1 This is basic feasible solution since i and s i If we choose the basic variables 1,, and s 1 as = 1 1 = 6 s 1 1 This is also basic feasible solution since i and s i 5-1

4 Etreme point A vector P is an etreme point of P if there do not eist two vectors y, z P, both different from such that = λy + (1 λ)z for any λ (, 1) is an etreme point if = λy + (1 λ)z and λ [, 1], then either y / P or z / P, or = y, or = z Verte A vector P is a verte of P if there eists some c such that c T < c T y for all y satisfying y P and y The line at the bottom touches P at a single point and is a verte w is not an etreme point because it is a conve combination of v and u 5-1 w is not a verte because there is no hyperplane that meets P only at w 5-14 Verte etreme point Let be a verte of P. Then there eists a c such that c T < c T for all P and Then, for all y, z P and y and z, we also have c T < c T y and c T < c T z For λ [, 1], we can get c T < c T (λy + (1 λ)z) and λy + (1 λ)z: thus, is an etreme point Basic feasible solution etreme point I A point is an etreme point of the set { A = b} if and only if it is a basic feasible solution 1. Basic feasible solution Etreme point ] ] For basic feasible solution = not a etreme point. [ B N = [ B we assume that is Then, there eist two distint feasible points y and z such that for < α < 1 where y and z are epressed as ] y = = αy + (1 α)z [ yb y N and z = zb z N

5 Basic feasible solution etreme point II Note that for nonbasic variables we have y N = z N =, since N = = αy N + (1 α)z N for < α < 1 Because, y, and z are feasible, we must have B B = By B = Bz N = b Since B is invertible, we have B = y B = z B = B 1 b This implies that is an etreme point Basic feasible solution etreme point III. Not basic feasible solution not etreme point An etreme point must be feasible so that A = [B N ] = b and If the columns of B are linearly independent, then is a basic feasible solution etreme point If the columns of B are linearly dependent, is not a basic feasible solution Let B i be the i th column of B; if they are linearly dependent, then there eist real numbers d 1,..., d k, not all of which are zero, such that B 1 d 1 + B d + + B k d k = Basic feasible solution etreme point IV Define a vector d = [d 1, d,..., d k ] T. Then, B( B ± αd) = B B ± αbd = B B = b Since B >, for small positive ɛ, we will have B + ɛd > and B ɛd > Then, we can have two distinct points y and z, i.e., B + ɛd B ɛd y = and z = which are feasible and distinct Since = 1 y + 1 z, is not an etreme point. This completes the proof 5-19 Basic feasible solution verte Suppose that P is a basic feasible solution and that a T i = b i for i A and a T i < b i for i / A Define c = i A a i and take any P. For each i A a T i b i = a T i Summing these over i A, we have c T = a T i b i = a T i = c T i A i A i A with equality only if a T i = b i for i A. Since {a i i A} are linearly independent, that holds only when = Thus, c T < c T for all P,, so is a verte 5-

6 Unbounded directions P contains a half-line if there eists d for P such that + αd P for all α Equivalently, for P = { A b} we have A b and Ad Note that if P is unbounded, then P constains a half-line, or vice versa P contains a line if there eists d such that + αd P for all α Equivalently, for P = { A b} we have A b and Ad = Note that if P has no etreme points, then P constains a line, or vice versa 5-1 Consider the set Representation theorem S = { : A = b, } representing the feasible region for a LP in standard form Let V = {v 1, v,..., v k } be the set of etreme points of S If S is nonempty, then V is nonempty and every feasible point S can be epressed as k = d + α i v i where a direction of unboundedness d of S satisfies Ad =, d and k α i = 1 and α i, i = 1..., k i=1 i=1 5- Eistence of optimal points I Theorem If LP in standard form has a finite optimal solution, then it has an optimal basic feasible solution Proof Based on the representation theorem, at the optimal, the objective function has the value k f = c T = c T d + α i c T v i For LP to have a finite optimal solution, we must have c T d = If c T d <, then f is unbounded below: To see this, consider a feasible point γ for any γ > i=1 k γ = γd + α i v i i=1 Due to c T d <, c T γ will be unbounded below as γ increases 5- Proof continued Eistence of optimal points II If c T d >, then for y = k i=1 α i v i we have c T > c T y which implies that is not the optimal point. Thus, we must have c T d = such that c T = c T y Now pick up an inde j for which c T v j = min i {c T v i }. Then, for any conve combination of the v i s, we have k k c T y = α i c T v i α i c T v j i=1 i=1 k = c T v j α i = c T v j i=1 For y to be optimal, we must have c T y = c T v j, showing that there is an optimal etreme point v j (basic feasible) 5-4

7 Simple method I Simple method II Assumptions: c T subject to A = b, A has full row rank (no redundant rows), and LP is feasible All basic feasible solution (etreme points) are nondegenerate Feasible directions P and A = b and, it must be satisfied that A( + αd) = A + αad = b + αad where the feasible direction must satisfy Ad = for all α For basic feasible solution, we have ] = Ad = [B N ] [ db d N = Bd B + N d N d B = B 1 N d N Let d N = e k for k / A and e k has all zeros ecept the k th position. Then, d B is epressed as d B = B 1 a k where a k is the column of N and k / A This constructs a search direction by moving a single nonbasic variable k / A Simple method III Suppose that B = [ 4 5 ] T and N = [ 1 ] T, and 1 1 A = 1 1 and b = so that we have B = 1, B 1 = 1, N = Thus, B is 1 B = B 1 b = 1 7 = 1 Simple method IV If we let d N = [ 1 ] T, then d B is 1 1 d B = B 1 N d N = If 1 and {, 1}, the possible B is 1 B = B 1 b + αd B = α where α is a step size If either 1 or takes 1, it becomes a basic variable If an element of B is set to, it becomes nonbasic variable

8 Simple method V Which basic variable should be set to? For a basic feasible solution and f = c T, we can see that if we move along d, the objective function changes as f = c T ( + αd) = c T + αc T d = f + α c T B c T d B N = f + α(c T B d B + c T N d N ) d N = f + α(c T N c T B B 1 N )d N }{{} reduced costs,z = f + α(c T N y T N )d N where d N = e k and only the k th nonbasic variable moves y = (c T B B 1 ) T = B T c B : simple multiplier 5-9 Simple method VI Change in objective value by moving the k th nonbasic variable If c = [ 1 ] T, then f = f + αz k c c1 1 c B = c 4 = and c N = = c c 5 The reduced cost z is z = (c T N c T B B 1 N ) = [ 5 ] If d N = [1 ] T ( 1 becomes basic variable), then the objective value is reduced 5- Simple method VII For z k < we choose α > as large as possible α = ma{α + αd } Case 1: if d, then it is unbounded feasible descent direction: + αd for all α Case : if d j < for some j, then + αd only if α j /d j for every d j < Thus, choose α using ratio test: Simple method VIII c T subject to A = b, B 1 b Suppose that we have a basic feasible solution whose objective value z is B 1 b B 1 b z = c = [c B, c N ] = c B B 1 b Since A = B B + N N = b, we have B = B 1 b B 1 N N α = min j {j B d j <} d j = B 1 b j N B 1 a j j where B is the inde set of basic variables 5-1 = b j N y j j 5-

9 Simple method IX The objective function is epressed as z = c = c B B + c N N = c B B 1 b B 1 a j j + c j j j N j N = z j N(z j c j ) j where z j = c B B 1 a j for each nonbasic variable. Then, we can get z j N(z j c j ) j subject to B + j N y j j = b, Consider c = [ 1 ] T, Numerical eample I 1 1 A = 1 1 and b = iteration : B = {, 4, 5} (inde set of basic variables) N = {1, } (inde set of nonbasic variables) Note that we want Only the nonbasic variables to appear in the objective The coefficient matri for the basic variables in the equality constraints to be the identity matri Numerical eample II Numerical eample III iteration 1: B = I = B 1 B B = b B = [ 7 ] T simple multiplier y = B T c B = [ ] T reduced cost z = c T N yt N = [ 1 ] Decide which nonbasic variable becomes basic: enters basis (entering variable) due to Decide which basic variable becomes nonbasic: Find the search direction: Bd B = N d N = a d B = [ 1 ] T ratio test: arg min j B:dj < j /d j leaves basis 1 1 B = 1 and B 1 = iteration : B = {, 4, 5}, N = {1, } B B = b B = [ ] T simple multiplier y = B T c B = [ ] T reduced cost z = c T N yt N = [ 5 ] 1 enters basis (entering variable) search direction: Bd B = a 1 d B = [ 1] T ratio test: arg min j B:dj < j /d j 4 leaves basis

10 Numerical eample IV Numerical eample V 1 B = 1 and B 1 = iteration : B = {1,, 5}, N = {, 4} B B = b B = [1 4 ] T simple multiplier y = B T c B = [5/ 4/ ] T reduced cost z = c T N yt N = [ 5/ 4/] enters basis (entering variable) search direction: Bd B = a d B = [ 1 5 leaves basis ]T B = 1 and B = iteration 4: B = {1,, }, N = {4, 5} B B = b B = [ 5 ] T simple multiplier y = B T c B = [ 1 ] T reduced cost z = c T N yt N = [1 ] z : basis is optimal Using Tableau I z = 1 subject to = Construct the following initial tableau: = = 1,,, 4, 5 basic rhs z is the entering variable and is the leaving variable 5-9 Using Tableau II basic rhs z The above tableau is interpreted as basic rhs z 5 c T B B 1 b B = B 1 b Reduced cost, z = c T N ct B B 1 N = [ 5 ] T New basic variables B = B 1 b B 1 N N 5-4

11 Using Tableau III basic rhs z is the leaving variable and 5 is the entering variable basic rhs z The solution is 1 =, = 5, =, and the optimal value z = Using Tableau: Two-phase method I How can we choose an initial basis to get an initial basic feasible solution? z = 1 + subject to 1 + = Using slack and ecess variables, we have 1, z = 1 + subject to 1 + = = =19 1,,, Using Tableau: Two-phase method II By introducing artificial variables into the constraints not having a slack variable, we can have an identity matri as an initial basis z = a 1 + a subject to a 1 = a = = 19 1,,, 4, a 1, a Phase I: The objective value must be zero if the LP is feasible basic 1 4 a 1 a rhs z 1 1 a a Using Tableau: Two-phase method III Setting coefficient of a 1 and a to zero in the first row: basic 1 4 a 1 a rhs z a a Let artificial variables a 1 and a leave one by one: basic 1 4 a 1 a rhs z 8 5 a

12 Using Tableau: Two-phase method IV Deleting a completely when it becomes the leaving variable After pivoting: basic 1 4 a 1 rhs z 8 a basic 1 4 a 1 rhs z a End of phase I: Using Tableau: Two-phase method V basic 1 4 a 1 rhs z Phase II with the objective function: basic 1 4 rhs z Using Tableau: Two-phase method VI Write LP in terms of the current basis: After pivoting: basic 1 4 rhs z basic 1 4 rhs z Using Tableau: Big-M method I z = 1 + subject to 1 + = , Introducing a cost M (large positive number) z = Ma 1 +Ma subject to a 1 = a = = 19 1,,, 4, a 1, a 5-47 Make a 1 and a leave at some basic feasible solution 5-48

13 Using Tableau: Big-M method II Relation between two-phase method and Big-M method: z = c T + M i When dividing M and let M : c T ẑ = lim M M + a i = a i i i The tableaus for the phase-i problem will only differ from the big-m method tableaus in the top row basic 1 4 a 1 a rhs z M M a a a i Using Tableau: Big-M method III basic 1 4 a 1 a rhs z 5M + M + M 16M a a After pivoting and deleting a, we have basic 1 4 a 1 rhs z 8M + 7 M + 1 M a Using Tableau: Big-M method IV basic 1 4 a 1 rhs z M+6 8M 7 M+17 a is the entering variable and a 1 is the leaving variable basic 1 4 rhs z Dual simple method I Using the tableau of the primal simple method, we want to solve dual problem z = 1 + subject to , maimize The tableau of the primal simple method is w = 4y 1 + y subject to y 1 + y y 1 + y y 1, y basic 1 4 rhs z where and 4 are ecess variables, and A T = 5-5

14 Dual simple method II The objective value z of the basic feasible solution = [ B N ] corresponding to B is equal to the objective value of the (possibly infeasible) dual solution: Using y T = c T B B 1, we can see that b T y = b T B T c B = (B 1 b) T c B = T B c B + T N c N = z The simple multiplier y = B T c B is indeed the dual solution corresponding to the basis matri B We can rearrange the first row of the tableau as z T }{{} basic variables c T N c T B B 1 N }{{} non-basic variables c T B B 1 b Dual simple method III Rewrite the basic- and nonbasic variables of the first row as [ }{{} T c T N c T B B 1 ] N }{{} basic var. non-basic var. = [ c T B c T B B 1 Bc T N c T N c T B B 1 N ] = [ c T B c T N ] c T B B 1[ B N ] = [ c T T ] c T B B 1[ A I ] = [ c T T ] y T [ A I ] where in the third equality, T appears due to slack variables. Note that subject to c T A b maimize subject to b T y A T y c Dual simple method IV Multiplying 1, we can see primal infeasiblity and B = I basic 1 4 rhs z Note that we should have, 4. Make primal feasible Referring to the dual problem, maimize w = 4y 1 + y subject to y 1 + y y 1 + y y 1, y The ratio test is eamined with respect to the columns 5-55 Dual simple method V Suppose that a varaible ( B ) s is infeasible so that its right-hand side entry ˆb s < In terms of the current basis, the s th constraint is ( B ) s + â s,j j = ˆb s < j N where â s,j are the entries in row s of B 1 A If some entry â s,j < and nonbasic variable j were to replace ( B ) s in the basis, the new value of j would be ˆb s â s,j > j is the entering variable, the new reduced cost will be c l = ĉ l ĉ j â s,l â s,j For c l to be nonnegative, the ratio ĉ j /â s,j for â s,j < should be selected 5-56

15 Dual simple method VI basic 1 4 rhs 1 z Dual feasible solution is y 1 =, and y = basic 1 4 rhs 1 1 z Since all primal variables are feasible, dual optimal solution is y 1 = 1 8, and y = Dual simple method VII Step 1: construct the basis matri B and N Step : Feasibility test and choosing the leaving variable If the rightmost vector B 1 b is elementwise positive, then stop Otherwise, select the most negative entry, say j Step : Dual ratio-test and choosing an entering variable For the row selected, choose the inde i based on { c T i = arg min N c T } B B 1 N (B 1 (B 1 N ) j < N ) j Step 4: Replace j with i and go to step Dual simple method VIII A baker makes and sells two types of cakes: one simple and one elaborate The price of each type of cakes is 4 and 14 (dollars), resp. Let 1 and be the number of batches of the elaborate and simple cakes produced per day To make each cake, we need basic, fancier ingradients with labour maimize subject to z = (daily limit of basic ingradient in kg) 4y (daily limit of fancier ingradient in kg) (daily limit of labour in hours) 1, e.g., it takes two hours to produce 1 while kg of basic and 4 kg of fancier ingradients are consumed; z = 888, 1 = 16 and = Dual problem of Baker s LP is Dual simple method IX w = 1y 1 + 1y + 7y subject to y 1 + 4y + y 4 y 1 + y + y 14 y 1, y How much should the baker pay for each indgradient, if he is willing to buy some etra? y 1 = 6.4 (basic), y = 1. (fancier) and y = If some other company wants to take over the baker s shop, how much should he pay? 5-6

16 Integer programming Branch and Bound I Mied integer linear programming is in general epressed as maimize subject to c T c T A A b 1 A 1 is an (m 1, n 1 ) and A is an (m, n ) 1 R n 1 and R n, and integer Consider an integer programming: M 1 : maimize subject to ,, and integer The solution of relaed LP: ( 1, ) = (.5, 5.5) Objective value z: 95 One of the non-integer valued variables is chosen to branch on: e.g., 5 or M : maimize Branch and Bound II subject to ( 1, ) = (.75, 5) and z = 95 M : maimize , subject to , Infeasible 5-6 Branch on 1 : M 4 : maimize Branch and Bound III subject to ( 1, ) = (, 5) and z = 85 M 5 : maimize 51 1, 5, 1, subject to ( 1, ) = (1, 4.5) and z = , 5, 1, 5-64

17 Branch and Bound IV Relaed LP infeasible infeasible Branch and Bound V Maimization problem of integer programming N : the set of nodes that have not been selected z L : the current lower bound on the optimal solution of the original model F R : the feasible region of the relaed LP of the original model f : the objective function Step : Initialization. Define F = F R, N = {}, and z L = Step 1: If N, go to Step Else, the current best solution is optimal. Stop * If there is no current best solution, i.e., z L =, then the original model has no optimal solution Step : Node selection phase. Branch and Bound VI Select a node k N and N := N \{k} Step : Bounding phase. Let z U be the optimal objective value of the above node k (if k =, z U = ) If z U z L, then the model corresponding to node k cannot lead to a solution that is better than the current best solution; go to Step 1 If z U > z L, then let ma{f () F k } with F k F R be the submodel M k ; go to Step 4 Branch and Bound VII Step 4: Solution phase. Determine an optimal solution (z k, k ) of M k. Take Step 5 If M k has no optimal solution z k z L (z k is not better than the current best solution). Submodel M k can be ecluded from further consideration, because branching may only lead to nonoptimal solutions or alternative optimal z k > z L and k F (z k is better than the current best solution). Step 5: Branching phase. Partition F k into two or more new subsets, say F k1, F k,... Set N := N {k 1, k, } Backtracking or LIFO: Branch on the most recently created unselected node FIFO: Branch on the unselected node that was created earliest