Solving LP and MIP Models with Piecewise Linear Objective Functions

Transcription

1 Solving LP and MIP Models with Piecewise Linear Obective Functions Zonghao Gu Gurobi Optimization Inc. Columbus, July 23, 2014

2 Overview } Introduction } Piecewise linear (PWL) function Convex and convex relaxation } Modeling Variables for pieces SOS2, binary formulations for non-convexity Direct handing } Convex PWL obective How to extend primal and dual simplex } Non-convex PWL obective How to extend branch-and-bound algorithm } Possible future work 2

3 Introduction 3

4 Definition A linear program with separable PWL obec4ve func4on is an op4miza4on problem of the form 4 n piecewise linear are Where n m i to Subect Minimize x c u x l b x a x c i n i n, 1,, ) (, 1,,, 1,, ) ( 1 1 = = = = = =

5 Types } Convex c (x ) Treated as LP } Non-convex c (x ) Treated as MIP 5

6 Motivations } Demands Models with true piecewise linear structures Approximation of nonlinear functions A lot of different applications Customer models and requests } Traditional approaches One variable for each piece SOS2 or binary variables for non-convex function } New approach Can we handle it directly to improve performance? 6

7 Previous Work quick incomprehensive survey } Convex case Fourer and Marsten, Solving Piecewise-Linear Programs: Experiments with a Simplex Approach, 1992 Extend primal simplex to handle variables with piecewise obective function directly No piece variables Use XMP subroutine library } Non-convex case SOS2 formulation, Beale and Tomlin, 1970 Branch-and-cut without binary variables, Keha, de Farias and Nemhauser, 2006 Any work without adding piece variables? 7

8 Piecewise Linear Function 8

9 Piecewise Linear Function } Definition c(x) = a k x +b k, p k x p k+1, k = 1,,t-1 where a k, b k, p k are constants for k = 1,,t and p 1 < p 2 < < p t } Convex c((x+y)/2) (c(x) + c(y))/2 9

10 Convex PWL Function } Continuous } Slopes are non-decreasing 10

11 Non-convex PWL Function } There can be umps at breaking points 11

12 Convex Relaxation } Convex relaxation r(x) is convex and r(x) c(x) for all x } Strongest convex relaxation A convex relaxation For all x, there exist x 1 and x 2 such that r(x) = α c(x 1 ) + (1-α) c(x 2 ) with x = αx 1 +(1- α) x 2, 0 α 1 Hereafter, relaxation always means strongest one } Relaxation of a PWL function is a convex PWL function May contain fewer pieces 12

13 Convex Relaxation 13

14 Finding Convex Relaxation } Algorithm Step 1: initialize a set of ordered points S = {(p 1, c(p 1 )), (p 2, c(p 2 ))} Step 2: Loop k from 3 to t Find max, such that the slope between (p k, c(p k )) and point in S is larger than the slope between points and -1 in S If no such, set = 1 Remove the points after from S and add (p k, c(p k )) to S } Results S defines PWL convex relaxation Complexity O(t) Linear, because each breaking point can be only removed from S once Quite similar to Graham scan algorithm for convex hull of a finite points, but no sorting needed 14

15 Modeling 15

16 Direct PWL Formulation 16 n piecewise linear are Where n m i to Subect Minimize x c u x l b x a x c i n i n, 1,, ) (, 1,,, 1,, ) ( 1 1 = = = = = =

17 Commonly Used Approach λ Formulation } One variable for each piece of PWL function Suppose variable x have a PWL function c(x) defined by t points, (p 1, c 1 ), (p 2, c 2 ),, (p t, c t ) We introduce variables λ 1,λ 2,, λ t for the points, such that x t = = 1 p λ (1) c( x) = c t = 1 λ (2) t λ = 1 = 1 λ 0, for = 1,..., t (3) (4) 17

18 Convex PWL Functions } Translation Substitute c(x) in the direct formulation by using equations (2) Add equations (1) and (3), and inequalities (4) } Pure LP Size can be much bigger, if PWL functions have a lot of pieces Direct handling of PWL by extending simplex may have a big advantage 18

19 Non-Convex PWL Functions } MIP formulations (SOS2 and binary) Substitute c(x) in the direct formulation by using equations (2) Add equations (1) and (3), and inequalities (4) Add either SOS2 constraints or binary variables SOS2 formulation: add SOS2 constraints on λ variables Binary formulation: add binaries y 1, y 2,,y t-1 with following constraints λ 1 y 1 λ y -1 + y, for = 2,, t-1 λ t y t-1 Σ y = 1 } Relaxations Direct formulation: replace c(x) with r(x) Relaxations of the three formulations have the same obective value Model size Binary: biggest; SOS2: smaller; Direct: smallest (could be much smaller) 19

20 Simplex For convex PWL Formulation 20

21 Primal Simplex for LP } Important aspects Crash basis and phase I Pricing to find enter variable Ratio test to find leaving variable Linear algebra to compute and update basis factorization and to solve equations (ftran, btran) 21

22 Primal Simplex for PWL LP } Important aspects Crash basis and phase I Pretty much the same Pricing to find enter variable Need to consider both directions for a nonbasic variable at a breaking point of PWL function Ratio test to find leaving variable Different Longer step Linear algebra to compute and update basis factorization and to solve equations (ftran, btran) Pretty much the same 22

23 Ratio Test of PWL Primal Simplex } Example, consider the dictionary for ratio test x 1 = x 3 + a 1 x x 2 = 10 + x 3 + a 2 x x 1 : basic, PWL with points (0, 0), (1, 1), (2, 3), (3, 6), (4, 10) x 2 : basic, no PWL, lb = 0, ub = inf x 3 : nonbasic at 0, entering with reduced cost -1.7, PWL with points (0, 0), (2, 1), (4, 4), (6, 10) } Ratio test with a shorter step x 1 : (2.5-2)/0.5 = 1 x 2 : (10-0)/1 = 10 x 3 enters the basis with step length 1, x 1 leaves. It can be mapped to the λ formulation No need to consider obective or reduced costs 23

24 Ratio Test of PWL Primal Simplex } Ratio test with a longer step x 1 : (2.5-2)/0.5 = 1, (2.5 1)/0.5 =3, (2.5-0)/0.5 = 5 x 2 : (10-0)/1 = 10 (> 5, eliminated) x 3 : 2, 4, 6 (> 5 eliminated) Possible steps 1, 2, 3, 4, 5 with corresponding unit obective changes -1.7, -1.2 ( ), -0.2(-1.2+1), 0.3( ), 1.8( ) x 3 enters the basis with step length 3, x 1 leaves to 1. This is equivalent to 3 iterations for the λ formulation We can use the median algorithm to do the ratio test. Its complexity is O(m log(t)), which is usually much cheaper than solving (ftran, btran) 24

25 Dual Simplex for LP } Important aspects Crash basis and phase I Bounded variables don t matter much, cheaper Pricing to find leaving variable Ratio test to find entering variable Basically solve an LP with a single constraint on variables with possible lower and upper bounds Linear algebra to compute and update basis factorization and to solve equations (ftran, btran) 25

26 Dual Simplex for PWL LP } Important aspects Crash basis and phase I Different, not important, ust do something simple Pricing to find leaving variable Pretty much the same Ratio test to find leaving variable Solve a PWL LP with a single constraint on variables with possible lower and upper bounds Median algorithm Linear algebra to compute and update basis factorization and to solve equations (ftran, btran) Pretty much the same Values for basic variables may change from one piece to another for each iteration It needs to address 26

27 Preliminary Computational Test } Model set 100 easy models from our LP set, many from netlib Replace obective of every variable with a PWL function with 10 pieces } Method Primal simplex No presolve To avoid removing pieces } Comparison between direct PWL and λ formulations All 100 models: 5.91X fewer iterations, 3.28X faster (too fast to be reliable) 21 models with > 1s runtime 5.89X fewer iterations, 7.13X faster Still some issues in the code Possible further fewer iterations and better runtime performance 27

28 Advantages of PWL Simplex } Faster iteration Because of smaller model size Especially for dual simplex and for primal devex or steepest edge pricing } Fewer iterations One iteration often equals several iterations on the λ formulation 28

29 Branch-and-bound For non-convex PWL Formulation 29

30 MIP Bound-and-Bound Solver } Important aspects Presolve Stronger primal reductions like bound strengthening Harder for dual reductions Solve relaxation Select variable to branch Cutting planes Heuristics Any relaxation solution is MIP feasible 30

31 Solving Relaxation } Root relaxation Replace c(x) with convex relaxation r(x) and solve the relaxation, the obective value is the same as that for the λ formulation before adding cuts } Node relaxations Use changed bounds to update r(x) It is primal feasible for one branch Warm start with primal or dual depending on the situation } Comment The advantages of PWL simplex carry over 31

32 Selecting Variable } Variable Candidates Every variable x with r(x) < c(x) is a candidate Basic (not at a breaking point) vs nonbasic (at a breaking point) } Selecting Pick x with max c(x) r(x)? Extend current pseudo, strong and reliability branching? } Where to split Not good at non-breaking point Easy for nonbasic variable Two choices for basic variable, left or right breaking point 32

33 Choosing Split (1) } Pick right breaking point to x = a c(x) r(x) doesn t change at x = a 33

34 Choosing Split (2) } Pick left breaking point to x = a c(x) r(x) becomes zero at x = a 34

35 Cutting Planes } Primal feasible regions Same for the relaxation and the original models Impossible to have primal cuts } Cutting planes with dual arguments Possible and how? 35

36 Possible Future Work 36

37 Separable PWL Coefficients } Much more useful for approximating nonlinear programs } The λ formulation Model size may be too big } Extend simplex To handle the convex case directly to have faster iterations and fewer iterations Harder } Extend branch-and-bound algorithm To handle the non-convex case directly 37

38 Non-separable PWL Function } Separating We can separate Q by factorization and introduction of new variables } Mixed-integer models for nonseparable piecewise linear optimization: Unifying framework and extensions, Vielma, Ahmed and Nemhauser,

39 Computational Tests } Translation the λ models back } Approximate convex QP and non-convex QP } Ask you and our customers to try and to send us models } Ask you for ideas 39

40 Thank You