Symmetry in Linear Programming

Transcription

1 Roland Wunderling, Jean-Francois Puget ISMP 2015 Symmetry in Linear Programming

2 CPLEX Optimization studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: Cuts for non-convex (MI)QP Improvements for SOCP Improvement to convex MIQCP/MISOCP Exploit symmetry for LP sec 100 sec 10 sec

3 CPLEX Optimization studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: Cuts for non-convex (MI)QP Improvements for SOCP Improvement to convex MIQCP/MISOCP Exploit symmetry for LP: LP performance improvement on problem instances with symmetry sec 100 sec sec [0,1) [1,10) [10,100) [100,1k) [1k,10k) 3

4 Overview Motivation Symmetries in LPs Transformation to Graph Automorphism problem AUTOM Solving symmetric LPs Aggregation Basic vs Non-basic Solutions Performance analysis Conclusion 4

5 Motivation Consider: min c T x + g y + g y s.t. Ax + s T y + s T y = b y + y = 1 Simplest form of Symmetry identical columns swapping variables leaves problem unchanged MIP: x, y, y {0, 1} y = 0 y = 1 Ax = b-s root y = 1 y = 0 Ax = b-s Combinatorial explosion! MIP solvers deal with it Orbital fixing [Margot 2002] Orbital branching [Linderoth, Ostroski 2009] Substantial improvement B&C B&C 5

6 Motivation Consider: min c T x + g y + g y s.t. Ax + s T y + s T y = b y + y = 1 LP: 0 x, y, y 1 y Optimal Solution (x*,y*,y *) LP solver not bothered by symmetry Duality theory gives optimality proof Nothing to be gained? 6 y

7 Motivation Consider: min c T x + g y + g y s.t. Ax + s T y + s T y = b y + y = 1 y - y = 0 LP: 0 x, y, y 1 y Optimal Solution (x*,y*,y *) aggregate min c T x + 2 g y s.t. Ax + 2s T y = b 2 y = 1 0 x 1, 0 y 1/2 Let s try anyway Pick ONE: y = y All convex combinations 7 Symmetric Solution (x*,y *,y*) y

8 Relative size of aggregated model Symmetry % of models Using internal set of 2128 LP problem instances < 25% have symmetry ~ 5% considerable

9 Symmetry We are interested in symmetries in the problem definition Given LP = min {c T x : Ax = b, x 0} Find row and column permutation matrices R and C, respectively, such that the permuted problem data equals the original data, i.e. R A C = A, c T C = c T, R b = b For all feasible primal solutions x, x = C x is a feasible primal solution of LP For all feasible dual solutions y, y = R T y is a feasible dual solution of LP All such permutations form a subgroup G of the symmetry group Find orbits orb(x,y) = {(x,y ) : (R,C) such that x = C x and y = R T y} Convexity of LP and its dual allows us to aggregate all variables in each orbit: add x = x for all x in orb(x,y) to LP and all duals in each orbit: add y = y for all y in orb(x,y) to dual LP 9

10 Example Min s.t. x1 + x2 + x3 + x4 + x5 + x6 c1: 5x1 + 4x2 + 4x3 + 5x4 + 4x5 + 4x6 >= 10 c2: x1 + x2 + x3 >= 2 c3: x1 + x2 + x3 <= 20 c4: x4 + x5 + x6 >= 2 c5: x4 + x5 + x6 <= 20 xj >= 0, for j = 1,,6 10 Group generators (x1 x4)(x2 x5)(x3 x6)(c2 c4)(c3 c5) (x2 x3) (x5 x6) Orbits {x1 x4} {x2 x3 x5 x6} {c2 c4} {c3 c5}

11 Example 11 Min s.t. x1 + x2 + x3 + x4 + x5 + x6 c1: 5x1 + 4x2 + 4x3 + 5x4 + 4x5 + 4x6 >= 10 c2: x1 + x2 + x3 + s >= 2 c3: x1 + x2 + x3 + t <= 20 c4: x4 + x5 + x6 - s >= 2 c5: x4 + x5 + x6 - t <= 20 a1: x1 - x4 = 0 Aggregated problem a2: x2 - x3 = 0 a3: Min x2 2x1 + -4x2 x5 = 0 a4: s.t. x2 - x6 = 0 xj >= 0, for j = 1,,6; s,t free Orbits c1: 10x1 + 16x2 >= 10 c2: 2x1 + 4x2 >= 4 c3: 2x1 + 4x2 <= 40 x1, x2 >= 0 {x1 x4} {x2 x3 x5 x6} {c2 c4} {c3 c5}

12 Detecting Symmetry Success of using Symmetry for LP depends on overhead for computing symmetries (orbits) Transform problem to graph Compute graph automorphism in graph NAUTY [McKay 1981] SAUCY [Darga et al 2004] AUTOM [Puget 2005] BLISS [Junttila et all 2007] [Darga et al 2008] Needs work limits to hedge against bad cases Do not exceed (deterministic) time of presolve 12

13 Example: Creating symmetry graph Min s.t. x1 + x2 + x3 + x4 + x5 + x6 c1: 5x1 + 4x2 + 4x3 + 5x4 + 4x5 + 4x6 >= 10 c2: x1 + x2 + x3 >= 2 c3: x1 + x2 + x3 <= 20 c4: x4 + x5 + x6 >= 2 c5: x4 + x5 + x6 <= 20 xj >= 0, for j = 1,,6 13 Create a node per variable (column) Colored by bounds and objective Create a node per constraint (row) Colored by sense and rhs value Create a node per non-zero Colored by coefficient Link row/column nodes with coefficient nodes

14 Example: Preprocessing symmetry graph In each row (or column) Merge nodes with same color Symmetry preserving operation 14

15 time AUTOM Performance NAUTY SAUCY AUTOM Problem instance

16 Disaggregating Solutions Min s.t. 2x1 + 4x2 c1: 10x1 + 16x2 >= 10 c2: 2x1 + 4x2 >= 4 c3: 2x1 + 4x2 <= 40 x1, x2 >= 0 Optimal Solution x = (2,0), y = (0,1,0) Disaggregated Solution x = (2,0,0,2,0,0),,,2,,, ) y = (0,1,0,1,0),,, 16 Min s.t. x1 + x2 + x3 + x4 + x5 + x6 c1: 5x1 + 4x2 + 4x3 + 5x4 + 4x5 + 4x6 >= 10 c2: x1 + x2 + x3 >= 2 c3: x1 + x2 + x3 <= 20 c4: x4 + x5 + x6 >= 2 c5: x4 + x5 + x6 <= 20 xj >= 0, for j = 1,,6

17 Disaggregating Basis 17 Min s.t. Min s.t. 2x1 + 4x2 c1: 10x1 + 16x2 >= 10 c2: 2x1 + 4x2 >= 4 c3: 2x1 + 4x2 <= 40 x1, x2 >= 0 Aggregated Model Basis a2: x2 = x3 Constraint non-basic Defines x3, i.e. basic Optimal Basis Disaggregated Basis x = (B,N,B,B,B,B),,B,,, ) y = (B,N,B,N,N),,, ) x1 + x2 + x3 + x4 + x5 + x6 x = (B,N), y = (B,N,B) c1: 5x1 + 4x2 + 4x3 + 5x4 + 4x5 + 4x6 >= 10 c2: x1 + x2 + x3 >= 2 c3: x1 + x2 + x3 <= 20 c4: x4 + x5 + x6 >= 2 c5: x4 + x5 + x6 <= 20 xj >= 0, for j = 1,,6

18 Uncrushing Symmetry Aggregations Uncrushing yields complementary primal/dual solution vector pair Uncrushing does not yield basis (let alone optimal one) Need crossover to get to an optimal solution 18

19 Extra work for Exploiting Symmetry Barrier 1. Compute Orbits and build aggregated problem 2. Solve with barrier 3. Disaggregate solution vectors Extra Work Barrier + Crossover 1. Compute Orbits and build aggregated problem 2. Solve with barrier 3. Disaggregate solution vectors 4. Crossover to optimal basis Simplex 1. Compute Orbits and build aggregated problem 2. Solve with Simplex 3. Disaggregate solution vectors 4. Crossover to optimal basis Expected Savings 19

20 Performance Analysis Cost of symmetry detection 1545 models without symmetry barrier without crossover 4 threads geometric means OK 23 losses by > 10% [0-1) [1-10) [10-100) [100-1k) [1k-10k) 20

21 Performance Analysis Barrier without crossover 583 models with symmetry 81 wins by > 10% 53 losses by > 10% performance variability correlation with reduction from symmetry aggregation [0-1) [1-10) [10-100) [100-1k) [1k-10k)

22 Performance Analysis Barrier with crossover 583 models with symmetry 79 wins by > 10% 58 losses by > 10% performance variability correlation with reduction from symmetry aggregation [0-1) [1-10) [10-100) [100-1k) [1k-10k)

23 Performance Analysis Dual Simplex to basic solution [0-1) [1-10) [10-100) [100-1k) [1k-10k) 583 models with symmetry 58 wins by > 10% 143 losses by > 10% excluded 145 timelimit hits no trend but some impressive wins cost of crossover offsets gains from symmetry?

24 Performance Analysis Dual Simplex to vector solution models with symmetry 115 wins by > 10% 62 losses by > 10% excluded 138 time limit hits still slower than barrier for hard problems overall trendline reveals same benefit as barrier [0-1) [1-10) [10-100) [100-1k) [1k-10k)

25 Controlling CPLEX CPX_PARAM_Presolve_Symmetry Default: On for Barrier (with or without crossover) On for Simplex if CPX_PARAM_Solution_Type set to CPX_NONBASIC_SOLN Off for Simplex otherwise CPX_PARAM_Solution_Type Controls crossover if needed CPX_BASIC_SOLN Default setting CPX_NONBASIC_SOLN Is allowed to produce basic solution as well, e.g. for Simplex in absence of symmetries 25

26 Conclusion An interesting percentage of LPs contains considerable symmetry Exploitation for performance requires efficient symmetry detection (AUTOM) Applying symmetry aggregation generally leads to performance improvement Symmetry aggregation is not basis preserving Crossover can be used to generate basic solution May be slower than disregarding the symmetry and solving directly Only applies when Simplex is the algorithm of choice 26

27 Further talks by IBM Optimization and friends Failure-directed search in Constraint-based Scheduling P. Vilim, Mon 11:20 (MB25) Recent advances in CPLEX for Mixed Integer Nonlinear Optimization P. Bonami, Mon 17:30 (MF16) Symmetry in Linear Programming R. Wunderling, Tue 10:20 (TB07) Accelerating the Development of Efficient CP Optimizer Models P. Laborie, Tue 14:45 (TD19) Advances in the CPLEX Distributed Solver L. Ladanyi, Thu 13:10 (ThC15) On Mathematical Programming with Indicator Constraints A. Lodi, Thu 16:35 (ThE01) Max Clique Cuts for Standard Quadratic Programs J. Schweiger, Fr 10:50 (FB29) Zero-Half Cuts for solving Nonconvex Quadratic Programs with Box Constraints J. Linderoth, Fr 11:20 (FB29) 27

28 Legal Disclaimer IBM Corporation All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus Sametime Unyte ). Subsequent references can drop IBM but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to for guidance on which trademarks require the or symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. If you reference Adobe in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. If you reference Java in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. If you reference Microsoft and/or Windows in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. If you reference Intel and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. If you reference UNIX in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. If you reference Linux in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only. 28

29