Optimization in Process Systems Engineering M.Sc. Jan Kronqvist Process Design & Systems Engineering Laboratory Faculty of Science and Engineering Åbo Akademi University
Most optimization problems in production planning, scheduling and design, involves some kind of discrete decision making. To model such decisions we need to use discrete variables. This session is intended as an introduction to some of the basic concepts in integer optimization. 25.1.2018 2
Background Standard problem formulation min f ( ) s.t. g ( ) 0 j 1,, m, j h ( ) 0 i 1,, m i N Conve if f and g i are conve functions and h i are all linear. Integer problems are classified as conve if an integer relaation results in a conve problem. 25.1.2018 3
Basic properties of linear programming 25.1.2018 4
Linear programming (LP) All functions defining the objective and constraints are linear. An important type of optimization problems. Efficient algorithms and software for solving these problems. We can handle problems with millions of variables. For eample minimization of l 1 -norm. 25.1.2018 5
Linear programming (LP) Consider the following eample min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, 25.1.2018 6
Linear programming (LP) Consider the following eample min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, The feasible region of an LP problem is always a conve set! 25.1.2018 7
Definition: An etreme point of a conve set is a point which cannot be obtained as a conve combination of two other points within the set. A conve combination z of the points and y is given by z = α + 1 α y, α 0,1. 25.1.2018 8
Etreme points 0, 3.5 min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, 2, 0 4.21, 0.553 25.1.2018 9
Theorem 1: Any point within a conve set can be obtained as a conve combination of the etreme points C = α 1 e 1 + α 2 e 2 + α n e n, n i=1 α i = 1, α i 0, where e i are the etreme points of the conve set C. 25.1.2018 10
Theorem 2: For an LP problem, one of the etreme points corresponds to an optimal solution. 25.1.2018 11
Back to our eample problem As stated by Theorem 1, any point within the feasible set can be obtained as a conve combination of the etreme points. We can, thus, rewrite the problems as: min st.. 2 0 4.21 1 3 0 3.5 0.553 2 3 1 3,, 0 3 min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, The etreme points are 2, 0, 0, 3.5 and 4.21, 0.553. 25.1.2018 12
By substituting 1 and 2 in the objective we obtain: min 2 3.5 4.7631 st.. 3 1, 3,, 0. 3 The optimal solution is obviously α 3 = 1. And the optimal solution to the LP problem is the etreme point 4.21, 0.553 25.1.2018 13
Solving LP problems in Matlab Gurobi is one of most powerful solvers for LP, QP, QPQC, MILP, MIQP and MIQCQP available. Free academic licenses! Do not use Matlab s own solvers! 25.1.2018 14
Solving LP problems in Matlab The problems must be written as min st.. T c A b lb ub 25.1.2018 15
Mied-integer linear programming (MILP) Linear optimization problems containing some discrete variables. Non-conve problems! NP-hard. Today many MILP problems can be solved efficiently. 25.1.2018 16
MILP Consider the following optimization problem min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, 25.1.2018 17
MILP How can we solve this problem? min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, Rela the problem by removing the integer restrictions. 25.1.2018 18
MILP We begin by solving the relaed problem min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 5 1 0 5 2, 25.1.2018 19
Branch and Bound Divide the problem into subproblems The search space is divided into subregions to force the variables to take on integer values. 25.1.2018 20
Branch and Bound 25.1.2018 21
Branch and Bound 25.1.2018 22
Branch and Bound 25.1.2018 23
Branch and Bound 25.1.2018 24
Branch and Bound The number of nodes to eplore can grow rapidly with the number of integer variables! Worst case: With 25 binary variables we have 2 25 possible solutions, and 2 26 70 000 000 nodes in the BB tree. How does Gurobi solve the eample problem? 25.1.2018 25
MILP How can Gurobi solve the problem without branching??? 25.1.2018 26
25.1.2018 27
Cutting planes We want to introduce cuts that reduce the search space without cutting of any feasible integer combination! These cuts are given by additional linear inequality constraints. 25.1.2018 28
Strengthening the relaation min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 54 1 0 35 2, First, let s eamine the variable bounds. 25.1.2018 29
Strengthening the relaation min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 54 1 0 35 2, First, let s eamine the variable bounds. 25.1.2018 30
Chvatal-Gomory Cuts A simple rounding procedure. Suppose we have a constraint of the type: a 1 + b 2 c, Where 1 and 2 are positive integer variables. We can round a and b down to take on integer values. a 1 + b 2 c Now the left hand side can only take on integer values! The Chvatal-Gomory Cut is given by: a 1 + b 2 c 25.1.2018 31
Strengthening the relaation min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 4 1 0 3 2, 25.1.2018 32
Chvatal-Gomory Cuts included to our problem min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 43 1 0 3 2 2 4 2 3 10 4, 25.1.2018 33
Now, solving the integer relaation gives us an integer solution min st.. 1.75 3.5 0.7 3.5 0.5 2 1 0 3 1 0 3 2 2 4 2 3 10 4, 25.1.2018 34
Chvatal-Gomory Cuts Consider a pure integer problem with the constraints A b, where A is a m n matri with columns {a 1, a 2,, a n }. All valid cuts for such a problem can be obtained by n where u. n j=1 ua j j ub, All such problems can be solved as an LP problem by adding a finite number of Chvatal-Gomory Cuts! Why are integer problems difficult then? How to find the correct set of cuts? The number of cuts needed may be enormous! 25.1.2018 35
Cuts for MILP problems Similar cuts can be made for problems containing both integer and continuous variables. Mied-integer rounding (MIR) cuts or mied-integer Gomory (MIG) cuts. Many different types of cuts! Lift and project cuts, Clique cuts, Disjunctive cuts, Cover Cuts, Network cuts,.. Still an active research area. 25.1.2018 36
Why are MILP problems an important type of optimization problems? Possible to model discrete quantities. Possible to incorporate logic decisions in the optimization problem! 25.1.2018 37
Consider the following optimization problem: min st.. 1.751 2 3.5 1 0.92 9 0.7 3.5 or 2.5 5 10 0.51 22 1 1 1.52 9 0 10 1 0 10 2, How can we formulate this as an MILP problem? 25.1.2018 38
Let s start with a simple case: 1 2 10 or 1 + 2 3 We want to write these constraints as an MILP problem 25.1.2018 39
Let s use the same technique here! min st.. 1.751 2 3.5 1 0.92 9 0.7 3.5 or 2.5 5 10 0.51 22 1 1 1.52 9 0 10 1 0 10 2, 25.1.2018 40
Big-M formulation min st.. 1.75 3.5 M (1 b ) 1 0.7 3.5 M (1 b ) 1 0.5 2 1 M (1 b ) 1 0.9 9 M (1 b ) 2 2.5 5 10 M (1 b ) 2 1.5 9 M (1 b ) b 2 b 1 2 1 0 10 0 10 b {0,1}, b {0,1}, 25.1.2018 41
How to choose M? M has to be big enough! Choice of M also affects the integer relaation! A too large value for M can make the problem difficult to solve! min st.. 1.75 3.5 M (1 b ) 1 0.7 3.5 M (1 b ) 1 0.5 2 1 M (1 b ) 1 0.9 9 M (1 b ) 2 2.5 5 10 M (1 b ) 2 1.5 9 M (1 b ) b 2 b 1 2 1 0 10 0 10 b {0,1}, b {0,1}, 25.1.2018 42
How to choose M? M = 4 M = 7 25.1.2018 43
How to choose M? M = 7 z = 11.3 M = 10 z = 12.7 M = 20 z = 18 z denotes the optimal objective value of the integer relaed problem. The optimal objective value for the problem is -9.3. 25.1.2018 44
The big-m formulation is not the only techinque by which we can epress the problem as an MILP problem. Conve hull formulation The conve hull formulation gives a tighter integer relaation, but the formulations requires more variables and constraints. The LP subproblems becomes more difficult. 25.1.2018 45
Disjunctive programming min z st.. A 0 0 Y1 Y2 A b A b 1 1 2 2 Y1 Y2 T T c1 z c2 z Y Y Y Y Y 2 1 1 2 Y lb ub b N {True,false} {True,false} 25.1.2018 46
Disjunctive programming Eample: We want to produce two products p1 and p2. In order to produce these products we need to buy some new equipment, and we can choose between two machines m1 and m2. However, we can only choose one of the machines. The machines have the following restrictions: m1 is limited to 600 tones of p1 and 500 tones p2 and the total production is limited to 1000 tones. m2 is limited to 300 tones of p1 and 700 tones p2 and the total production is limited to 900 tones. Furthermore, the machines have different operating costs: The operating cost of m1 is given by: z = 10000 + 150 p 1 +70 p 2. The operating cost of m2 is given by: z = 5000 + 195 p 1 +80 p 2. Selling p1 gives a profit of 250 /ton and p2 gives a profit of 130 /ton. Determine the most profitable production strategy! 25.1.2018 47
ma 250 p 130 p z st.. Y1 Y2 p 600 p 300 1 1 p2 500 p2 700 p1 p2 1000 p1 p2 900 z 10000 150 p 70 p z 5000 195p 80 p Y p Y Y 1 2 Y, p {True,false} {True,false} 25.1.2018 48
ma 250 p 130 p z st.. Y1 Y2 p1 600 p1 300 p2 500 p2 700 p1 p2 1000 p1 p2 900 z 10000 150 p 70 p z 5000 195p 80 p Y p 1 2 Y, p Y {True,false} Y {True,false} ma 250 p 130 p z st.. p p 600 (1 b ) M 1 1 500 (1 b ) M 2 1 p p 1000 (1 b ) M 1 z 10000 150 p 70 p (1 b ) M p p 300 (1 b ) M 700 (1 b ) M 2 2 p p 900 (1 b ) M 2 1 z 5000 195p 80 p (1 b ) M b p b b 1, p {0,1}, b {0,1} 2 25.1.2018 49
A scheduling problem We are producing 3 products and we want to complete the production as fast as possible. The products are produced in 4 process steps, and the times for each step are given in the table. Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 25.1.2018 50
A scheduling problem We begin by introducing variables for the starting time of each product in each production step t p,s. 25.1.2018 51
Process step 1 Only product 2. Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 25.1.2018 52
Process step 2 Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 Product 1 and 3 cannot be handled simultaneously. t 1,2 t 3,2 + 9 or t 3,2 t 1,2 + 5 (1 b 1 )M + t 1,2 t 3,2 + 9 b 1 M+ t 3,2 t 1,2 + 5 b 1 =1 Product 3 before product 1 b 1 =0 Product 1 before product 3 25.1.2018 53
Process step 3 Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 The previous processing step has to be completed before the new step starts: t 1,3 t 1,2 + 5, t 2,3 t 2,1 + 3, t 3,3 t 3,2 + 9. 25.1.2018 54
Process step 3 Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 Processing order: t 2,3 t 1,3 + 3 or t 1,3 t 2,3 + 4 t 3,3 t 1,3 + 3 or t 1,3 t 3,3 + 5 t 3,3 t 2,3 + 4 or t 2,3 t 3,3 + 5 2 5 0 9 3 3 4 5 4 4 0 6 25.1.2018 55
Process step 3 Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 Processing order: 1 b 2 M + t 2,3 t 1,3 + 3 b 2 M + t 1,3 t 2,3 + 4 1 b 3 M + t 3,3 t 1,3 + 3 b 3 M + t 1,3 t 3,3 + 5 2 5 0 9 3 3 4 5 4 4 0 6 b 2 =1 Product 1 before product 2 b 2 =0 Product 2 before product 1 b 3 =1 Product 1 before product 3 b 3 =0 Product 3 before product 1 1 b 4 M + t 3,3 t 2,3 + 4 b 4 M +t 2,3 t 3,3 + 5 b 4 =1 Product 2 before product 3 b 4 =0 Product 3 before product 2 25.1.2018 56
Process step 4 Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 The previous processing step has to be completed before the new step starts: t 1,4 t 1,3 + 3, t 3,4 t 3,3 + 5. t 3,4 t 1,4 + 4 or t 1,4 t 3,4 + 6 (1 b 5 )M + t 3,4 t 1,4 + 4 b 5 M+ t 1,4 t 3,4 + 6 b 5 =1 Product 1 before product 3 b 5 =0 Product 3 before product 4 25.1.2018 57
Processing times in minutes. Process step Product 1 Product 2 Product 3 1 0 3 0 2 5 0 9 3 3 4 5 4 4 0 6 Finally we want to minimize the longest production time t final t 1,4 + 4 t final t 2,3 + 4 t final t 3,4 + 6 25.1.2018 58
min t final s.t. (1 b 1 )M + t 1,2 t 3,2 + 9 b 1 M+ t 3,2 t 1,2 + 5 t 1,3 t 1,2 + 5 t 2,3 t 2,1 + 3 t 3,3 t 3,2 + 9 1 b 2 M + t 2,3 t 1,3 + 3 b 2 M + t 1,3 t 2,3 + 4 1 b 3 M + t 3,3 t 1,3 + 3 b 3 M + t 1,3 t 3,3 + 5 1 b 4 M + t 3,3 t 2,3 + 4 b 4 M +t 2,3 t 3,3 + 5 t 1,4 t 1,3 + 3 t 3,4 t 3,3 + 5 (1 b 5 )M + t 3,4 t 1,4 + 4 b 5 M+ t 1,4 t 3,4 + 6 t final t 1,4 + 4, t final t 2,3 + 4, t final t 3,4 + 6 b 1,, b 5 {0,1}, t 1,2, t 3,2,, t final R + 25.1.2018 59
Other types of integer optimization problems MIQP Ok, if conve. Gurobi. MIQCQP More difficult, but tractable if conve. Gurobi. MINLP Several techniques are available for conve MINLP. 25.1.2018 60
Conve MINLP Nonlinear branch and bound. Solve a conve NLP in each node. Linear approimation techniques. Construct an equivalent MILP problem. 25.1.2018 61
Thank you! Any questions? jan.kronqvist@abo.fi 25.1.2018 62