ORF 307: Lecture 4. Linear Programming: Chapter 3 Degeneracy

Transcription

1 ORF 307: Lecture 4 Linear Programming: Chapter 3 Degeneracy Robert Vanderbei February 15, 2018 Slides last edited on February 16, rvdb

2 Solve This... maximize 2x 1 + 3x 2 subject to x 1 + 2x 2 2 x 1 x 2 1 x 1 + x 2 1 x 1, x

3 Solution Note: The horizontal axis, which one might call the x 1 -axis, is where x 2 = 0 and is labeled as such. Enter: x 2, Leave: w 3 Enter: x 1, Leave: w 1 Enter: w 3, Leave: w 2 In (x 1, x 2 ) coordinates, the pivots visit the following vertices: (0, 0) = (0, 1) = (0, 1) = (4/3, 1/3). Note that the second pivot went nowhere. 2

4 Degeneracy Definitions. A dictionary is degenerate if one or more rhs -value vanishes. Example: ζ = 6 + w 3 + 5x 2 + 4w 1 x 3 = 1 2w 3 2x 2 + 3w 1 w 2 = 4 + w 3 + x 2 3w 1 x 1 = 3 2w 3 w 4 = 2 + w 3 w 1 w 5 = 0 x 2 + w 1 A pivot is degenerate if the objective function value does not change. Examples (based on above dictionary): 1. If x 2 enters, then w 5 must leave, pivot is degenerate. 2. If w 1 enters, then w 2 must leave, pivot is not degenerate. 3

5 Cycling A cycle is a sequence of pivots that returns to the dictionary from which the cycle began. Note: Every pivot in a cycle must be degenerate. Why? Pivot Rules A pivot rule is an explicit statement for how one chooses entering and leaving variables (when a choice exists). Some Examples: Largest-Coefficient Rule. (most common pivot rule for entering variable) Choose the variable with the largest coefficient in the objective function. Random Positive-Coefficient Rule. Among all nonbasic variables having a positive coefficient, choose one at random. First Encountered Rule. In scanning the nonbasic variables, stop with the first one whose coefficient is positive. 4

6 Hope Some pivot rule, such as the largest coefficient rule, will be proven never to cycle. An example that cycles using the following pivot rules: entering variable: largest-coefficient rule. leaving variable: smallest-index rule. ζ = x 1 2x 2 2x 4 w 1 = 0.5x x 2 + 2x 3 4x 4 w 2 = 0.5x 1 + x x 3 0.5x 4 w 3 = 1 x 1. Here s a demo of cycling (ignoring the last constraint)... 5

7 Hope Some pivot rule, such as the largest coefficient rule, will be proven never to cycle. Hope Fades An example that cycles using the following pivot rules: entering variable: largest-coefficient rule. leaving variable: smallest-index rule. ζ = x 1 2x 2 2x 4 w 1 = 0.5x x 2 + 2x 3 4x 4 w 2 = 0.5x 1 + x x 3 0.5x 4 w 3 = 1 x 1. Here s a demo of cycling (ignoring the last constraint)... 6

8 Enter: x 1, Leave: w 1 Enter: x 2, Leave: w 2 Enter: x 3, Leave: x 1 Enter: x 4, Leave: x 2 7

9 Enter: w 1, Leave: x 3 Enter: w 2, Leave: x 4 Cycling is rare for small problems! A program that generates random 2 4 fully degenerate problems was run more than one billion times and did not find one example! However, for larger problems with lots of zeros, cycling is common and can be a real problem. 8

10 Algebra of a Pivot b a pivot b a 1 a d c d bc a c a 9

11 Python Code Create Random Problem m = 2 n = 2 x0 = 4*random.rand() y0 = 4*random.rand() r = 5 theta = (3.*random.rand(m)-1.)*pi/2. x1 = x0 + r*cos(theta) y1 = y0 + r*sin(theta) A = zeros(m*n).reshape(m,n) A[:,0] = x1-x0 A[:,1] = y1-y0 b = (x1-x0)*x1 + (y1-y0)*y1 A = ceil(2*a) b = ceil(2*b-0.5) b = matrix(b) b = b.t c = matrix(ceil(9*random.rand(n,1))) 19

12 Python Code Algorithm while ( (max(c) > eps) ): print(iter) col = argmax(c) Acol = A[:,col].reshape(m,1) tmp = -Acol/b row = argmax(tmp) if (tmp[row] < eps): print('unbounded') break j = nonbasics[col] i = basics[row] # find entering variable # the associate entering column # vector of ratios # the leaving variable # the entering var's subscript # the leaving var's subscript # update A matrix. See section 5.4. Arow = A[row,:] # the row in A of the entering variable a = A[row,col] # the pivot element A = A - Acol*Arow/a # the out-of-row/out-of-col update formula A[row,:] = -Arow/a # update formula for the row A[:,col] = Acol.reshape(1,m)/a # update formula for the col A[row,col] = 1/a # update formula for the pivot element # update the right-hand side brow = b[row,0] b = b - brow*acol/a b[row] = -brow/a # update the objective function ccol = c[col,0] c = c - ccol*(arow.reshape(n,1))/a c[col] = ccol/a # swapping variables x_j and x_i position in the dictionary basics[row] = j nonbasics[col] = i iter = iter+1 21

13 AMPL Code param m := 2; param n := 4; param c {1..n}; param A {1..m, 1..n}; param nonbasics {1..n}; param basics {1..m}; param row; param col; param ii; param jj; param Arow {1..n}; param Acol {1..m}; param cj; param bi; param a; param ccol; param iter; for {k in } { let {i in 1..m, j in 1..n} A[i,j] := Normal01(); let {j in 1..n} c[j] := Normal01(); let {j in 1..n} nonbasics[j] := j; let {i in 1..m} basics[i] := n+i; display k; let iter := 1; repeat while (max {j in 1..n} c[j] > 0) { let cj := 0; for {j in 1..n} { if (c[j] > cj) then { let col := j; let cj := c[j]; } } let jj := nonbasics[col]; let bi := m+n+1; for {i in 1..m: A[i,jj] < -1e-8} { if (basics[i] < bi) then { let bi := basics[i]; let row := i; } } if bi > m+n then {break;} # unbounded polytope let ii := basics[row]; } } let {j in 1..n} Arow[j] := A[row,j]; let {i in 1..m} Acol[i] := A[i,col]; let a := A[row,col]; let {i in 1..m, j in 1..n} A[i,j] := A[i,j] - Acol[i]*Arow[j]/a; let {j in 1..n} A[row,j] := -Arow[j]/a; let {i in 1..m} A[i,col] := Acol[i]/a; let A[row,col] := 1/a; let ccol := c[col]; let {j in 1..n} c[j] := c[j] - ccol*arow[j]/a; let c[col] := ccol/a; let basics[row] := jj; let nonbasics[col] := ii; if iter > 15 then { display "found a cycling example"; break; } let iter := iter+1; 23

14 Perturbation Method Whenever a vanishing rhs appears perturb it. If there are lots of them, say k, perturb them all. Make the perturbations at different scales: An Example. other data ɛ 1 ɛ 2 ɛ k > 0. Entering variable: x 2 Leaving variable: w 2 24

15 Perturbation Method Example Con t. Recall current dictionary: Entering variable: x 1 Leaving variable: w 3 DONE! 25

16 Perturbation Method Applied to Cycling Example x 1 enters, w 2 leaves x 2 enters, w 1 leaves x 3 enters, x 2 leaves w 2 enters, problem unbounded! Note: objective function increases with every pivot: 0 < 2ε 2 < 2ε 1 < 8 3 ε ε 2 26

17 Other Pivot Rules Smallest Index Rule. Choose the variable with the smallest index (the x variables are assumed to be before the w variables). Note: Also known as Bland s rule. No cycling (it s been proved). Random Selection Rule. Select at random from the set of possibilities. No infinite cycles. Greatest Increase Rule. Pick the entering/leaving pair so as to maximize the increase of the objective function over all other possibilities. Note: Too much computation. Needs a tie-breaking rule. 27

18 Theoretical Results Cycling Theorem. If the simplex method fails to terminate, then it must cycle. Why? Fundamental Theorem of Linear Programming. For an arbitrary linear program in standard form, the following statements are true: 1. If there is no optimal solution, then the problem is either infeasible or unbounded. 2. If a feasible solution exists, then a basic feasible solution exists. 3. If an optimal solution exists, then a basic optimal solution exists. 28

19 Geometry maximize x 1 + 2x 2 + 3x 3 subject to x 1 + 2x 3 3 x 2 + 2x 3 2 x 1, x 2, x 3 0 maximize x 1 + 2x 2 + 3x 3 subject to x 1 + 2x 3 2 x 2 + 2x 3 2 x 1, x 2, x 3 0 x 3 x 3 x 2 +2x 3 =2 x 2 =0 1 x 2 +2x 3 =22 x 2 1 x 2 x 2 =0 x 1 +2x 3 =3 x 1 x 1 x 1 +2x 3 =2 29