Mixed-Integer PDE-Constrained Optimization

Transcription

1 Mixed-Integer PDE-Constrained Optimization IMA Workshop on Uncertainty Quantification Pelin Cay, Bart van Bloemen Waanders, Drew Kouri, Anna Thuenen and Sven Leyffer Lehigh University, Universität Magdeburg, Argonne National Laboratory, and Sandia National Laboratories February 22, 2016

2 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 2 / 42

3 Mixed-Integer PDE-Constrained Optimization (MIPDECO) PDE-constrained MIP... u = u(t, x, y, z) infinite-dimensional! t is time index; minimize u,w x, y, z are spatial dimensions F(u, w) subject to C(u, w) = 0 u U, and w Z p (integers), u(t, x, y, z): PDE states, controls, & design parameters w discrete or integral variables MIPDECO Warning w = w(t, x, y, z) Z may be infinite-dimensional integers! 3 / 42

4 Mixed-Integer PDE-Constrained Optimization (MIPDECO) PDE-constrained MIP... u = u(t, x, y, z) infinite-dimensional! t is time index; minimize u,w x, y, z are spatial dimensions F(u, w) subject to C(u, w) = 0 u U, and w Z p (integers), u(t, x, y, z): PDE states, controls, & design parameters w discrete or integral variables MIPDECO Warning w = w(t, x, y, z) Z may be infinite-dimensional integers! It s a MIP, Jim, but not as we know it! 3 / 42

5 Grand-Challenge Applications of MIPDECO Topology optimization [Sigmund and Maute, 2013] Nuclear plant design: select core types & control flow rates [Committee, 2010] Well-selection for remediation of contaminated sites [Ozdogan, 2004] Design of next-generation solar cells [Reinke et al., 2011] Design of wind-farms [Zhang et al., 2013] Scheduling for disaster recovery: oil-spills [You and Leyffer, 2010] & wildfires [Donovan and Rideout, 2003] Design & control of gas networks, [De Wolf and Smeers, 2000, Martin et al., 2006, Zavala, 2014]... also as optimization under uncertainty 4 / 42

6 Uncertainty Quantification and MIPDECO Design of experiments, e.g. discrete sensor placement Akaike s Information Criterion: parameter & structure est. AIC: maximize log-likelihood & minimize nonzeros u minimize w {0,1} N l e k (u) T R 1 e k (u)+ k=1 i=1 w i s.t. Mw i u i Mw i where R is known co-variance 5 / 42

7 Source Inversion as MIP with PDE Constraints Simple Example: Locate number of sources to match observation ū minimize J = 1 (u ū) 2 dω least-squares fit u,w 2 Ω subject to u = w kl f kl in Ω Poisson equation k,l w kl S and w kl {0, 1} source budget k,l with Dirichlet boundary conditions u = 0 on Ω. E.g. Gaussian source term, σ > 0, centered at (x k, y l ) ( (xk, y l ) (x, y) 2 ) f kl (x, y) := exp, Motivated by porous-media flow application to determine number of boreholes, [Ozdogan, 2004, Fipki and Celi, 2008] σ 2 6 / 42

8 Source Inversion as MIP with PDE Constraints Consider 2D example with Ω = [0, 1] 2 and discretize PDE: 5-point finite-difference stencil; uniform mesh h = 1/N Denote u i,j u(ih, jh) approximation at grid points minimize u,w J h = h2 2 N (u i,j ū i,j ) 2 i,j=0 subject to 4u i,j u i,j 1 u i,j+1 u i 1,j u i+1,j h 2 = u 0,j = u N,j = u i,0 = u i,n = 0 N w kl S and w kl {0, 1} k,l=1 finite-dimensional (convex) MIQP N w kl f kl (ih, jh) k,l=1 7 / 42

9 Source Inversion as MIP with PDE Constraints Potential source locations (blue dots) on mesh Create target ū using red square sources 8 / 42

10 Source Inversion as MIP with PDE Constraints 1 Contours of TARGET ubar(x,y) 0.02 Relative error in States ubar(x,y) - u(x,y) & Source Location Target (3 sources), reconstructed sources, & error on mesh 9 / 42

11 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 10 / 42

12 Mixed-Integer PDE-Constrained Optimization (MIPDECO) minimize u,w F(u, w) subject to C(u, w) = 0 u U, and w Z p (integers), u(t, x, y, z): PDE states, controls, & design parameters w discrete or integral variables Towards a problem characterization Type of PDE: different classes of PDEs e.g. elliptic, parabolic, hyperbolic, nonlinear,... Class of Integers: binary, general integers, etc Type of Objective: functional form of objective Type of Constraints: characterize c/s other than PDE Discretization: discretization method & CUTEr classification 11 / 42

13 Mesh-Independent & Mesh-Dependent Integers Definition (Mesh-Independent & Mesh-Dependent Integers) 1 The integer variables are mesh-independent, iff number of integer variables is independent of the mesh. 2 The integer variables are mesh-dependent, iff the number of integer variables depends on the mesh. Mesh-Independent Mesh-Dependent Manageable tree Theory possible Exploding tree size Theory??? 12 / 42

14 Theoretical Challenges of MIPDECO Functional Analysis (mesh-dependent integers) Denis Ridzal: What function space is w(x, y) {0, 1}? Consistently approximate w(x, y) {0, 1} as h 0? Conjecture: {w(x, y) {0, 1}} L 2 (Ω)... e.g. binary support of Cantor set not integrable Likely need additional regularity assumptions Coupling between Discretization & Integers Discretization scheme (e.g. upwinding for wave equation) depends on direction of flow (integers). Application: gas network models with flow reversals... open postdoc position at Argonne! 13 / 42

15 Computational Challenges of MIPDECO Approaches for humongous branch-and-bound trees... e.g. 3D topology optimization with 10 9 binary variables Warm-starts for PDE-constrained optimization (nodes) Guarantees for nonconvex (nonlinear) PDE constraints... factorable programming approach hopeless for 10 9 vars! + * ^ x1 log x 2 3 x2... f (x 1, x 2 ) = x 1 log(x 2 ) + x / 42

16 MIPDECO: Two Cultures Collide Observation PDE-optimization & MIP developed separately different assumptions, methodologies, and computational kernels! PDE-Optimization Obtain good solutions efficiently Nonlinear optimization: Newton s method Iterative Krylov solvers Run on bleeding-edge HPC Mixed-Integer Programming Deliver certificate of optimality Combinatorial optimization: branch-and-cut Factors & rank-one updates Limited HPC developments Potential for Disaster, or Opportunity for Innovation! 15 / 42

17 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 16 / 42

18 Source Inversion as MIP with PDE Constraints Find number and location of sources to match observation ū minimize J = 1 (u(w) ū) 2 dω least-squares fit u,w 2 Ω subject to u = w kl f kl in Ω Poisson equation k,l w kl S and w kl {0, 1} source budget k,l MIP with convex quadratic objective on Ω = [0, 1] 2 5-point finite-difference stencil; uniform mesh h = 1/N Denote u i,j u(ih, jh) approximation at grid points 17 / 42

19 Cool MIPDECO Trick: Eliminating the PDE Discretized PDE constraint (Poisson equation) 4u i,j u i,j 1 u i,j+1 u i 1,j u i+1,j h 2 = k,l w kl f kl (ih, jh), i, j Au = w kl f kl, where w kl {0, 1} only appear on RHS! Elimination of PDE and states u(x, y, z) Au = w kl f kl u = A 1 w kl f kl = k,l k,l k,l w kl A 1 f kl Solve n 2 2 n PDEs: u (kl) := A 1 f kl Eliminate u = k,l w klu (kl) from Source Inversion 18 / 42

20 Cool MIPDECO Trick: Eliminating the PDE Eliminating u = k,l w klu (kl) in MINLP gives: minimize w subject to N J h = h2 w kl u (kl) ij ū i,j 2 i,j=0 k,l N w kl S and w kl {0, 1} k,l=1 2 Eliminates the states u (N 2 variables) Eliminates the PDE constraint (N 2 constraints)... generalizes to other PDEs (with integer controls on RHS) Simplified model is quadratic knapsack problem 19 / 42

21 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 20 / 42

22 Numerical Experience with Source Inversion Find number and location of sources to match observation ū minimize J = 1 (u(w) ū) 2 dω least-squares fit u,w 2 Ω subject to u = w kl f kl in Ω Poisson equation k,l w kl S and w kl {0, 1} source budget k,l MIP with convex quadratic objective Computational Experiments: 1 Test NLP-plus-rounding heuristic versus MINLP 2 Effect of mesh-dependent vs. mesh-independent integers Mesh-independent: pick sources from 36 potential locations Mesh-dependent: all nodes are potential locations 3 Effect of state-elimination trick 21 / 42

23 1st Example Mixed-Integer PDE-Constrained Optimization Potential source locations (blue dots) on mesh Create target ū using red square sources 22 / 42

24 Approach 1: NLP-Solve, Knapsack Rounding, and MIP Knapsack Rounding 1 Solve continuous relaxation using NLP solver 2 Round largest S locations, w i, to one & set all others to zero 23 / 42

25 Approach 1: NLP-Solve, Knapsack Rounding, and MIP Knapsack Rounding 1 Solve continuous relaxation using NLP solver 2 Round largest S locations, w i, to one & set all others to zero Error in States ubar(x,y) - u(x,y) & Source Location Error in States ubar(x,y) - u(x,y) & Source Location Knapsack-rounded NLP (left) and MINLP (right) MINLP solution better: NLP-err = > = MIP-err 23 / 42

26 Mesh-Independent Source Inversion: MINLP Solvers Number of Nodes and CPU time for Increasing Mesh Sizes Nodes CPU Time Mesh-Size BonminOA MINLP Minotaur Mesh-Size BonminOA MINLP Minotaur Number of Nodes independent of mesh size! MINLP & Minotaur: filtersqp runs out of memory for N 32 BonminOA takes roughly 100 iterations... quadratic objective 24 / 42

27 Mesh-Dependent (all) Source Inversion: MINLP Solvers Number of Nodes and CPU time for Increasing Mesh Sizes Nodes CPU Time Mesh-Size BonminOA BonminBB MINLP Minotaur Mesh-Size BonminOA BonminBB MINLP Minotaur Number of nodes explodes with mesh size! OA <BREAK> after 130,000 seconds 25 / 42

28 Elimination of States & PDEs: Source Inversion CPU Time for Increasing Mesh Sizes: Simplified vs. Original Model CPU Time Mesh-Size MINLP MINLP-Simple Eliminating PDEs is two orders of magnitude faster! 26 / 42

29 Elimination of States & PDEs: Source Inversion CPU Time for Increasing Mesh Sizes: Simplified vs. Original Model Presolve Time Simplified Model Total Simplified Full PDE Model using NLP solve for PDE (inefficient) Presolve is cheap... simplified model solves much faster! 27 / 42

30 First Conclusions: Source Inversion Numerical Results Solve mesh-independent problems with coarse discretization Mesh-dependent instances cannot be solved Outer Approximation (Bon-OA) inefficient for these instances Trick # 1: elimination of states and PDE constraint Nonlinear solvers run into storage issues 28 / 42

31 First Conclusions: Source Inversion Numerical Results Solve mesh-independent problems with coarse discretization Mesh-dependent instances cannot be solved Outer Approximation (Bon-OA) inefficient for these instances Trick # 1: elimination of states and PDE constraint Nonlinear solvers run into storage issues... not surprising: MIPDECO trees grow like tribbles! 28 / 42

32 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 29 / 42

33 Control Regularization: Not All Norms Are Equal Poisson with Distributed Control [OPTPDE, 2014] & [Tröltzsch, 1984] minimize u u d 2 u,w L 2 (Ω) + e Γ u ds + α w 2 L x Γ subject to u + u = w + e Ω in Ω u = 0 on boundary Γ n w(t) {0, 1} L 1 or L 2 regularization term for control w(t) {0, 1}? Good Norms for MIPs MIP ers prefer polyhedral norms... promote integrality Old MIP trick: w 2 (t) = w(t) for w(t) {0, 1} L 1 -norm same as L 2 -norm on binary variables! 30 / 42

34 Not All Norms Are Equal Consider Distributed Control for increasing mesh-size CPU for L 2 Regularization Mesh Minotaur B-BB B-Hyb B-OA 8x x Time 32x32 Time Time Time Time 31 / 42

35 Not All Norms Are Equal Consider Distributed Control for increasing mesh-size CPU for L 2 Regularization Mesh Minotaur B-BB B-Hyb B-OA 8x x Time 32x32 Time Time Time Time CPU for L 1 Regularization Mesh Minotaur B-BB B-Hyb B-OA 8x x x L 1 regularization is equivalent to L 2, but faster Many fewer nodes in tree-searches solve up to / 42

36 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 32 / 42

37 Problem 2: Actuator Placement and Operation [Falk Hante] Goal: Control temperature with actuators Select sequence of control inputs (actuators) Choose continuous control (heat/cool) at locations Match prescribed temperature profile... de-mist bathroom mirror with hair-drier Potential Actuator Locations l = 1,..., L 33 / 42

38 Problem 2: Actuator Placement and Operation Find optimal sequence of actuators, w l (t), and controls, v l (t): where minimize u,v,w u(t f, ) 2 Ω + 2 u 2 T Ω v 2 T L subject to u t κ u = w l (t) {0, 1}, l=1 v l (t)f l in T Ω L w l (t) W, t T l=1 Lw l (t) v l (t) Uw l (t), l = 1,..., L, t T f l (x, y) = 1 ( (x, y) (xl, y l ) 2 ) ) exp 2πσ 2σ point-source for actuators at (x l, y l )... movies! 34 / 42

39 Outline 1 Introduction Problem Definition and Applications Source Inversion as MIP with PDE Constraints Problem Classification and Challenges 2 Early Theoretical & Numerical Results Eliminating the PDE & State Variables Numerical Experience with Source Inversion Control Regularization: Not All Norms Are Equal Heat Equation: Actuator Design 3 Rounding-Based Heuristic for Cloaking 4 Conclusions 35 / 42

40 Topology Design of Cloaking Devices/Scatterers Design of cloaking device on domain Ω Cloak subdomain Ω 0 (red dashes) by preventing (complex) wave from entering domain Design scatterer in subdomain ˆΩ...w(x, y) {0, 1} PDE: 2D Helmholtz (over C) with Robin boundary conditions Incident wave is exp(ik 0 y) for wavelength k 0 = 6π where i = 1 Romulan Warbird Scatterer 36 / 42

41 Topology Design of Cloaking Devices/Scatterers Control: w = w(x, y) in ˆΩ States: u = u(x, y) in Ω Target: u 0 = u 0 (x, y) in Ω 0 minimize J(u) = 1 u,v,w 2 u + u 0 2 2,Ω 0 subject to u k0 2(1 + qw)u = k2 0 qwu 0 in Ω u n ik 0u = 0 on Ω w {0, 1} in ˆΩ. Discretization: finite-differences with l = 3 nodes per scatter element, w(x, y). 37 / 42

42 Strip Rounding Heuristic Cannot solve on reasonable mesh/domain with any MINLP solver. Algorithm: Strip Rounding Heuristic Solve continuous relaxation & initialize i = 1 for i=1,...,n do Round a strip w(x i, y j ) for all j Resolve relaxation with w(x k, y) fixed for all k i end Round fractional w(x, y) following direction of wave 38 / 42

43 Strip Rounding Heuristic Cannot solve on reasonable mesh/domain with any MINLP solver. Algorithm: Strip Rounding Heuristic Solve continuous relaxation & initialize i = 1 for i=1,...,n do Round a strip w(x i, y j ) for all j Resolve relaxation with w(x k, y) fixed for all k i end Round fractional w(x, y) following direction of wave 38 / 42

44 Strip Rounding Heuristic Cannot solve on reasonable mesh/domain with any MINLP solver. Algorithm: Strip Rounding Heuristic Solve continuous relaxation & initialize i = 1 for i=1,...,n do Round a strip w(x i, y j ) for all j Resolve relaxation with w(x k, y) fixed for all k i end Round fractional w(x, y) following direction of wave 38 / 42

45 Results for Strip Rounding Scatterer, w(x, y) States u(x, y)... resolve PDE on finer mesh for fixed controls 39 / 42

46 ... Solution Not Physical! Coarse States Resolved States... not clear we re getting the correct physics! 40 / 42

47 Conclusions Mixed-Integer PDE-Constrained Optimization (MIPDECO) Class of challenging problems with important applications Subsurface flow: oil recovery or environmental remediation Design and operation of gas-/power-networks Classification: mesh-dependent vs. mesh-independent On-going work: Building library of test problems... formulation matters: interplay of binary and continuous Elimination of PDE and state variables u(t, x, y, z) Discretized PDEs huge MINLPs... push solvers to limit Outlook and Extensions Consider multi-level in space (network) and time Move toward truly multi-level approach similar to PDEs Interested in new UQ applications involving MIP & PDEs / 42

48 Our five-year mission To boldly go where no optimizer has gone before to explore strange new PDEs & MIPs! 42 / 42

49 Committee, T. (2010). Advanced fueld pellet materials and fuel rod design for water cooled reactors. Technical report, International Atomic Energy Agency. De Wolf, D. and Smeers, Y. (2000). The gas transmission problem solved by an extension of the simplex algorithm. Management Science, 46: Donovan, G. and Rideout, D. (2003). An integer programming model to optimize resource allocation for wildfire contrainment. Forest Science, 61(2). Fipki, S. and Celi, A. (2008). The use of multilateral well designs for improved recovery in heavy oil reservoirs. In IADV/SPE Conference and Exhibition, Orlanda, Florida. SPE. Martin, A., Möller, M., and Moritz, S. (2006). Mixed integer models for the stationary case of gas network optimization. Mathematical Programming, 105: OPTPDE (2014). OPTPDE a collection of problems in PDE-constrained optimization. Ozdogan, U. (2004). Optimization of well placement under time-dependent uncertainty. Master s thesis, Stanford University. 42 / 42

50 Reinke, C. M., la Mata Luque, T. M. D., Su, M. F., Sinclair, M. B., and El-Kady, I. (2011). Group-theory approach to tailored electromagnetic properties of metamaterials: An inverse-problem solution. Physical Review E, 83(6): Sigmund, O. and Maute, K. (2013). Topology optimization approaches: A comparative review. Structural and Multidisciplinary Optimization, 48(6): Tröltzsch, F. (1984). The generalized bang-bang-principle and the numerical solution of a parabolic boundary-control problem with constraints on the control and the state. Zeitschrift für Angewandte Mathematik und Mechanik, 64(12): You, F. and Leyffer, S. (2010). Oil spill response planning with MINLP. SIAG/OPT Views-and-News, 21(2):1 8. Zavala, V. M. (2014). Stochastic optimal control model for natural gas networks. Computers & Chemical Engineering, 64: Zhang, P., Romero, D., Beck, J., and Amon, C. (2013). Integration of AI and OR Techniques in Contraint Programming for Combinatorial Optimization Problems, chapter Solving Wind Farm Layout Optimization with Mixed Integer Programming and Constraint Programming. Springer Verlag. 42 / 42