Overview of course. Introduction to Optimization, DIKU Monday 12 November David Pisinger

Transcription

1 Introduction to Optimization, DIKU Monday November David Pisinger Lecture What is OR, linear models, standard form, slack form, simplex repetition, graphical interpretation, extreme points, basic solution, CPLEX introduction Taha sections,, (briefly), 4 (briefly),,, 4, 6 all examples can be read briefly Overview of course What is OR, linear models, standard form, slack form, simplex repetition, graphical interpretation, extreme points, basic solution, CPLEX introduction Revised simplex algorithm, bounded variables Duality, shadow prices, sensitivity analysis, post-optimal analysis, complementary slackness, KKT optimality constraints, CPLEX sensitivity 4 Network problems, transportation model, total unimodular (perfect matrices, interval matrices, property P), max-flow min-cut duality 5 Interior point methods, Simplex vs interior point 6 External speaker: Erling Andersen, MOSEK Projektopgaver Sensitivity analysis Multicommodity flow problem Assumption Knowledge of matrix notation (appendix D in Taha) Overview of course Linear Programming: VA simplex, equation duality theorem proof of termination INT-OPT simplex, tabular complementary slack condition revised simplex interpretation of tabular sensitivity analysis post-optimal analysis worst-case complexity of simplex interior-point methods What is Operations Research The application of scientific methods By interdisciplinary teams To problems involving the control of organized manmachine systems so as to provide solutions which best serve the purpose of the organization as a whole (RLAckoff) OR is concerned with scientifically deciding how to best design and operate man-machine systems, usually under conditions requiring the allocation of scarce resources (ORSA definition, 976) OR is the activity carried on by members of the OR society; its methods are those reported in our journal (PM Morse, 95) Operations Research: The science of better Time-starved executives are making bolder decisions with less risk and better outcomes Their secret: operations research (INFORMS, definition 004) OR is the discipline of applying advanced analytical methods to help make better decisions (INFORMS, definition 004) OR is an expensive way of being insulted by someone half your age (Stafford Beer) 4

2 Operations Research problem formulation conceptual model model building Tons of raw per ton of Max availability Ext Paint (EP) Int Paint (IP) (tons) Raw, M Raw, M 6 Profit/ton 5 4 real world system symbolic model demand of IP cannot exceed that of EP by more than ton maximum daily demand of IP is tons interpretation solution solution of model Any OR model has three basic components Decision variables that we seek to determine Objective (goal) that we aim to optimize Constraints that we need to satisfy Model building is an art 5 6 Tons of raw per ton of Max availability Ext Paint (EP) Int Paint (IP) (tons) Raw, M Raw, M 6 Profit/ton 5 4 demand of IP cannot exceed that of EP by more than ton maximum daily demand of IP is tons Decision variables x tons produced daily of EP x tons produced daily of IP Objective maximize z = 5x + 4x Constraints 6x + 4x 4 (raw M) x + x 6 (raw M) x + x (x x + ) x Nonnegativity constraints (implicitly given) x 0 x 0 Proportionality and additivity linearity Two decision variables, solution space can be drawn in D Convex solution space 7 8

3 Graphical solution Sensitivity Analysis, objective z = 5x + 4x Solution remains optimal between Isoprofit line Optimal solution in corner point (or side) x = (, ) z = 5 +4 = z = x + x z = 6x + 4x 9 0 Sensitivity Analysis, constraint M Sensitivity Analysis, constraint M range 0 M 6 profit of increasing M one unit z G z C 0 = = dual variable corresponding to M range 4 M 6 profit of increasing M one unit z C z B 6 4 = 0 dual variable corresponding to M =

4 Standard form and Slack form Matrix representation LP problem max c x + c x +c n x n st a x + a x ++a n x n b a x + a x ++a n x n b a m x + a m x ++a mn x n b m x,x,,x n 0 Can be written in standard form Slack form max n j= c jx j st n j= a i jx j b i i =,,m x j 0 j =,,n z = n j= c jx j n j= a i jx j + x n+i = b i i =,,m omit max omit x j 0 introduce slack variables x n+i set z equal to objective, and treat as ordinary constraint LP in standard form max c x + c x +c n x n st a x + a x ++a n x n b a x + a x ++a n x n b a m x + a m x ++a mn x n b m x,x,,x n 0 Written in matrix form max (c,c,,c n ) st x x x n a a a n a a a n a m a m a mn x 0 x 0 x n 0 Short matrix representation max cx st Ax b x 0 x x x n b b b m 4 Standard form and Slack form Any LP can be written as standard form max{cx : Ax b,x 0} slack form max{z = cx : Ax+s = b,x,s 0} min max minz = max z p q p q = p = q p q, p q = p q p+x s = q,x s 0 (slack var) = p q p x s = q,x s 0 (surplus var) free nonneg x j free x j = x j x j,x j,x j 0 Linear Programming (Taha example ) maximize x + x subject to x + x 4 x + x 5 x,x 0 F B A C D Add slack variables maximize x + x subject to x + x + x = 4 x + x + x 4 = 5 x,x,x,x 4 0 The set of constraints form a polyhedral Optimal solution is found at corner point Corner point Name Feasible Objective (0,0,4,5) A yes 0 (0,4,0, ) F no (0, 5, 5, 0) B yes 75 (,0,0,) D yes 4 (5,0, 6,0) E no (,,0,0) C yes 8 E 5 6

5 Linear Programming Basis, basis feasible solution Corner points are found by setting n m variables to 0, and solving remaining m equations with m variables Number of corner points, when m < n (exponential in n) F B A C n m = n! m!(n m)! Corner points are basic solutions C D Non- Basic Basic Corner Feasible Objective basic Solution Point (x,x ) (x,x 4 ) (4,5) A yes 0 (x,x ) (x,x 4 ) (4, ) F no (x,x 4 ) (x,x ) (5,5) B yes 75 (x,x ) (x,x 4 ) (,) D yes 4 (x,x 4 ) (x,x ) (5, 6) E no (x,x 4 ) (x,x ) (,) C yes 8 E Since we have added slack variables, the number of variables n is larger than the number of constraints m maximize cx subject to Ax = b x 0 Choose m linearly independent columns from A The corresponding set B = {i,i,,i m } is called a basis A B columns in A corresponding to basis variables B A N columns in A corresponding to non-basis variables N maximize c B x B + c N x N subject to A B x B + A N x N = b x 0 A Basis feasible solution is obtained by setting x N = 0 A B x B + A N 0 = b x B = A B b x B is well defined: A B is an m m matrix columns must be linearly independent 7 8 Basis, basis feasible solution Example: maximize c B x B + c N x N subject to A B x B + A N x N = b x 0 n = 4 variables, m constraints Assume B = {,4}, N = {,} x x max (c,c,c,c 4 ) x x 4 st is equivalent to max (c,c 4 ) a a 4 a a 4 st a m a a a a 4 a a a a 4 a m a m a m a m4 a m4 ( x ) x 4 ( x ( x +(c,c ) ) + x 4 x x x x 4 ) x a a a a a m a m ( x b b b m ) x b b b m Corner points and basis feasible solutions Theorem A feasible solution x to maximize cx subject to Ax = b x 0 is a corner point if and only if x is a basis feasible solution Trivial algorithm Search through all corner points Basis can be chosen in Cm n = n! m!(n m)! different ways Each time invert A B in time O(m ) 9 0

6 Adjacent basis feasible solutions Simplex algorithm Two basis feasible solutions x and x are adjacent if B and B have m common elements F B A C D maximize x + x subject to x + x + x = 4 x + x + x 4 = 5 x,x,x,x 4 0 E Trivial algorithm: Search through all corner points Basis can be chosen in Cm n = n! m!(n m)! different ways Each time invert A B in time O(m ) Better algorithm: Search through corner points Greedy approach (hopefully not all basic solutions) Consider adjacent solutions (faster to invert A B ) A table can be used to maintain information objective is increased most possible in each step Corner point Name Feasible Objective (0,0,4,5) A yes 0 (0,4,0, ) F no (0, 5, 5, 0) B yes 75 (,0,0,) D yes 4 (5,0, 6,0) E no (,,0,0) C yes 8 Simplex, equation form (Cormen) Iteration 0: z = x + x x = 4 x x x 4 = 5 x x Most promising variable x Keeping x = 0 (nonbasic), x cannot be increased infinitely x 0, x = 4 x 4 x 0 x 4 0, x 4 = 5 x 5 x 0 Gauss elimination x = x + x 4 Iteration : z = 8 x 4 x 4 x = x + x 4 x = + x + x 4 All costs in objective are negative, hence stop implying x 5 When x = 5 we have x 4 = 0 (x 4 leaves basis) Gauss elimination x = 5 x x 4 Iteration : z = 5 + x x 4 x = x + x 4 x = 5 x x 4 Most promising variable x Keeping x 4 = 0 (nonbasic), x cannot be increased infinitely x 0, x = x x 0 x 0, x = 5 x 5 x 0 implying x When x = we have x = 0 (x leaves basis) 4

7 Simplex, in tabular form Taha is using the equations this leads to tabular form Iteration 0: z x x = 0 x + x + x = 4 x 4 + x + x = 5 basic z x x x x 4 solution z x x leaving variable is x Multiply second row by and add to first row Multiply second row by and add to third row Multiply second row by Iteration : basic z x x x x 4 solution 4 z x 0 0 x 0 0 All reduced costs in objective are positive, hence stop Entering variable x maximum value of entering variable min{ 4, 5 } = 5 leaving variable is x 4 Multiply third row by and add to first row Multiply third row by and add to second row Multiply third row by Iteration : basic z x x x x 4 solution z x 0 0 x 0 0 Entering variable x maximum value of entering variable min{, 5 } = Initial solution Simplex moves from basis solution to neighboring basis solution Initial basis solution: If m slack variables, set these to zero or add m variables x having a large negative cost in objective function maximize cx M x subject to Ax+ x = b x, x 0 then x is a basis feasible solution Iterative step Reduced cost used for choosing the next variable to enter basis Termination criteria All reduced costs are positive (TAHA) All reduced costs are negative (Cormen) LP basis solution The solution to A B x B = b is uniquely determined The solution is x B = A B b where A B is the inverse matrix of A B x B = A B is called a basis solution m variables in x B are called basic variables n m remaining variables nonbasic variables If all basic variables are nonnegative, then x B is called basic feasible solution If some of the basic variables are zero, we talk of a degenerate solution If some of the basic variables are negative, the basic solution is infeasible A B x B = b has infinitely many solutions A B x B = b has no solutions Proof of optimality Shown at DATA 7 8

8 LP basis solution, example LP basis solution 4x + x + 5x + 4x 4 + 0x 5 + 6x 6 = 0 x + 5x + 6x + 4x 4 + x 5 + x 6 = 4 Case x B = (x,x ) Basic feasible solution x =,x = 4 (nondegenerate) Case x B = (x,x ) Basic feasible solution x = 0,x = 4 (degenerate) Case x B = (x,x 4 ) Basic infeasible solution x =,x 4 = 7 Case x B = (x,x 5 ) Infinitely many solutions Case x B = (x,x 6 ) No solution An LP-problem may have a) No feasible solutions (in case of conflicting constraints) b) Feasible solutions but no optimal solution (in case of unboundedness such as max x, x ) c) Feasible and one or more optimal solutions In this case the problem also has an optimal basic solution 4x + x = 0 x + 5x = 4 x + 6x = 4 4x + 5x = 0 x + 4x 4 = 4 4x + 4x 4 = 0 5x + 0x 5 = 0 6x + x 5 = 4 x + x 6 = 4 4x + 6x 6 = Bigger example, Add slack variables x,x 4,x 5,x 6 maximize 5x + 4x subject to 6x + 4x 4 x + x 6 x + x x x,x 0 Bigger example, Write up equation form (Cormen style) Iteration 0: z = 5x + 4x x = 4 6x 4x x 4 = 6 x x x 5 = + x x x 6 = x maximize 5x + 4x subject to 6x + 4x + x = 4 x + x + x 4 = 6 x + x + x 5 = x + x 6 = x,x,x,x 4,x 5,x 6 0 Iteration : Iteration : z = 0 + x 5 6 x x = 4 x 6 x x 4 = 4 x + 6 x x 5 = x 6 x x 6 = x z = 4 x x 4 x = 4 x + x 4 x = + 8 x 4 x 4 x 5 = 5 8 x x 4 x 6 = 8 x + 4 x 4

9 Bigger example, CPLEX Iteration 0: basic z x x x x 4 x 5 x 6 solution z x x x x Iteration : basic z x x x x 4 x 5 x 6 solution z x x x x Iteration : basic z x x x x 4 x 5 x 6 solution z x x x x LP-solver simplex dual simplex network simplex interior point methods IP/MIP-solver branch-and-bound cuts Modes of use interactive mode file mode callable library called from AMPL 4 License bach and kand computers rlogin bach- cplex ILOG CPLEX 900, licensed to "university-copenhagen", options: e m b q Interactive use add variables add constraints change bounds File mode several file formats lp is natural form type in data to file read file in CPLEX optimize display solution 5 CPLEX Commands help displays a list of all commands read filenamelp reads a file in natural format write write problem or solution info to a file optimize run optimizer (LP or IP depending on formulation) display problem all show the problem display solution variables - show primal solution display solution dual - show dual solution display sensitivity objective - show objective sensitivity range display sensitivity rhs - show right-hand-side sensitivity range display sensitivity lb - show lower bound sensitivity range display sensitivity ub - show upper bound sensitivity range quit leave CPLEX Commands may be abbreviated eg dis sol var - 6

10 LP file format Using CPLEX everything following a backslash is a comment variables have formx, y, antalskibe minimize or maximize objective function subject to constraints should be in linear form with constant on right side x+ x <= 5 relational operators <, <=, >=, > are interpreted as,,, bounds 0 <= x <=+infty (warning: bounds have no associated dual variables!) integer (warning: means binary, so do not use!) general variables which must be integer variables binary variables which must be 0- variables end 7 Reddy Mikks, assuming decision variables are integers: maximize 5 x + 4 x subject to 6 x + 4 x <= 4 x + x <= 6 - x + x <= x <= bounds x >= 0 x >= 0 general x x end kand- > cplex ILOG CPLEX 900, licensed to "university-copenhagen", options: e m b q Welcome to CPLEX Interactive Optimizer 90 with Simplex, Mixed Integer \& Barrier Optimizers Copyright (c) ILOG CPLEX is a registered trademark of ILOG Type help for a list of available commands Type help followed by a command name for more information on commands CPLEX> read reddymixlp Problem reddymixlp read Read time = 000 sec CPLEX> opt Tried aggregator time MIP Presolve eliminated rows and 0 columns MIP Presolve modified coefficients Reduced MIP has rows, columns, and 6 nonzeros Presolve time = 000 sec MIP emphasis: balance optimality and feasibility Root relaxation solution time = 000 sec 8 Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt * infeasible Cuts: Mixed integer rounding cuts applied: Integer optimal solution: Objective = e+0 Solution time = 000 sec Iterations = Nodes = 0 CPLEX> dis sol var - Variable Name Solution Value x All other variables in the range - are zero Using CPLEX from AMPL AMPL is a modeling language for mathematical programming AMPL is described in Appendix A of Taha AMPL can use several solvers To use CPLEX, see section A8 in Taha option solver cplex; option cplex options sensitivity ; Solvers available on CD-ROM: CPLEX KNITRO LPSOLVE LOQO MINOS 9 40