Mflop/s per Node. Nonlinear Iterations

Size: px

Start display at page:

Download "Mflop/s per Node. Nonlinear Iterations"

Joan Sullivan
5 years ago
Views:

1 Lagrange-Newton-Krylov-Schur-Schwarz: Methods for Matrix-free PDE-constrained Optimization D. Hovland Paul & Computer Science Division, Argonne National Laboratory Mathematics E. Keyes David & Statistics Department, Old Dominion University Mathematics Institute for Scientic Computing Research, Lawrence Livermore National Laboratory Institute for Computer Applications in Sci. & Eng., NASA Langley Research Center C. McInnes Lois & Computer Science Division, Argonne National Laboratory Mathematics Samyono Widodo & Statistics Department, Old Dominion University Mathematics

2 Perspective \Have parallel PDE solver, will optimize" Years of two-sided (from architecture up, from applications down) algorithms has put us in a position to solve implicit PDE problems reasonably research scalably, with \Newton-Krylov-Schwarz" iterative methods. Eective parallel implicit solver for large-scale nonlinear problems derived from PDEs: aerodynamics, radiation transport, porous media, semiconductors, geophysics, astrophysics Implemented in parallel matrix-free object-oriented framework, including both FD and AD distributed matvecs, in PETSc (from Argonne) Applied to unstructured computational aerodynamics problem on 644 processors of ASCI Red for a 999 Gordon Bell Prize; see Anderson, Gropp, Kaushik, Keyes & Smith (999) PDEs are equality constraints on the state variables in many optimization hardly \auxiliary", the PDE system may contain a million or more problems; degrees of freedom. Optimization is easily incorporated into a Newton-like parallel PDE framework that accommodates substructuring. Editorial: A PDE solver not part of an optimization framework is probably short of what the client really wants. far

3 Example of NKS on Aerodynamics Problem

4 of NKS on Aerodynamics Problem Example ow on a tetrahedral grid of 2,76,744 vertices, based on Euler implementation of NASA code FUN3D run on KMeTiS-PETSc to 3072 nodes of ASCI Red at Top/s up Avg. Vertices per Node Mflop/s per Node Aggregate Gflop/s Execution Time (sec) Nonlinear Iterations.2 Implementation Efficiency

5 References on NKS Methods Recent with others at (on-line Globalized Newton-Krylov-Schwarz Algorithms and Software for Parallel Implicit CFD (with Gropp, McInnes & Tidriri), Int. J. High Performance Computer Applications 4: , software focus, overview 3D structured-grid transonic Euler example On the Interaction of Architecture and Algorithm in the Domain-Based Parallelization of an Unstructured Grid Incompressible Flow Code (with Kaushik & Smith), 998, in \Proc. of the 0th Intl. Conf. on Domain Decomposition Methods", AMS, pp HPC focus 3D unstructured-grid incompressible Euler example Convergence Analysis of Pseudo-Transient Continuation (with Kelley), 998, SIAM J. Num. Anal. 35: theory focus 2D unstructured-grid subsonic Euler example Parallel Newton-Krylov-Schwarz Algorithms for the Transonic Full Potential Equation (with Cai, Gropp, Melvin & Young), SIAM J. Sci. Comp. 9: , application focus 2D structured-grid transonic full potential example Newton-Krylov-Schwarz Methods for Aerodynamics Problems: Compressible and Incompressible Flows on Unstructured (with Kaushik & Smith), 998, \Proc. of the th Intl. Conf. on Domain Decomposition Methods", pp Grids multi-platform comparisons 3D unstructured-grid Euler example (Mach 0 to Mach.2) How Scalable is Domain Decomposition in Practice? 998, \Proc. of the th Intl. Conf. on Domain Decomposition pp Methods", parallel complexity focus 3D unstructured-grid incompressible Euler example

6 Implications of NKS for Optimization Equality constrained optimization leads, through the Lagrangian to a multivariate nonlinear rootnding problem for formulation, the gradient (rst-order necessary conditions) Canonical framework: choose m design variables u to minimize objective function, c(u; x), subject to n state constraints, h(u; x) = 0, where x is the vector of state variables Lagrange framework: nd stationary point of the Lagrangian L(x; u; ) c(x; u) + T h(x; u) Natural \outer" partitioning: controls are often of lower dimension than states and multipliers, suggesting Schur-complement preconditioning Natural \inner" partitioning: states and their multipliers are high dimension and corresponding matrix blocks are sparse, of suggesting Schwarz-like domain decomposition

7 Reduced or Full Systems? As in domain decomposition, choice to be made between: exact elimination of the states and multipliers by satisfying feasibility at every step (reduced system), or constraint making progress in all variables, possibly violating constraints convergence (full system) until Advantage of the former: existence of quality, robust \black box" optimization software Advantages of the latter: reuse of quality, ecient parallel PDE software employment of inexact solves while retaining \exact" Jaco- in outer iteration bian ease of application of automatic dierentiation software, with- out having to dierentiate through subiterations

8 Examples of PDE-constrained Optimization Design optimization (esp. shape optimization): u parameterizes the domain of the PDE (e.g., lifting surface) and c is cost-to-benet ratio of forces, energy expenditures, etc. Typically, a m is small compared to n, and does not scale directly with it. But m may still be several hundred. Optimal control: u parameterizes a continuous control function acting on the surface of the domain, and c is the norm of the between desired and actual responses of the system. dierence m / n 2=3. Typically, Parameter identication/data assimilation: u parameterizes an unknown continuous constitutive or forcing function throughout the domain, and c is the norm of the dierence dened between measurements and simulation results. Typically, m / n.

9 Optimality Conditions We look for saddle points of the Lagrangian: nding a correction, h = x u A using Newton's + = + = 0 to the iterate x u A

10 Optimality Conditions subscript notation for partial derivatives, the Newton (Karush- With equations are: Kuhn-Tucker) or (c;xx + T h;xx) (c;xu + T h;xu) h T ;x + T h;ux) (c;uu + T h;uu) h T ;u (c;ux h;u 0 h;x Wxx W T ux JT x Wuu J T u Wux Ju 0 Jx x u + A x u =, A =, c;x + T h;x + T h;u c;u h W + and g for where a; b 2 fx; ug, and where + = +. gx gu h A ; A

11 Newton Reduced SQP Design Step (Schur complement for middle blockrow): H u = f ; H f are the reduced Hessian and gradient, resp.: where and W uu, J T u J,T x W T ux + J T u J,T x W xx, W ux H, x J u J + J T u J,T x gx, J T u J,T x Wxx, Wux J, x h,gu f State Step (last blockrow): Adjoint Step (rst blockrow): In each overall iteration: must form and solve with H J x x =,h, J u u J T x + =,g x, W xx x, W T ux u must solve with J x and J T x (negligible compared with the cost of forming H) Number of overall iterations is few (asymptotically independent of m) Cost of forming H at each design iteration is m solutions with J x potentially concurrent, but prohibitive

12 Quasi-Newton Reduced SQP Design Step (severe approximation for middle blockrow): u =,g u + J T u J,T x g x ; Q where Q is a quasi-newton approximation to the reduced Hessian State Step (last blockrow): J x x =,h, J u u Adjoint Step (approximation to rst blockrow): In each overall iteration: J T x + =,g x must perform low rank update on Q or its inverse must solve with J x and J T x of overall iterations is many. Since BFGS is equivalent to unpreconditioned CG for quadratic Number functions, O(m p ) sequential cycles (p > 0, p 2 ) may be anticipated objective RSQP is not scalable in the number of design variables, and no ready form of parallelism can Hence, this address

13 Proposed Full System Approach Conventional RSQP methods apply a (quasi-)newton method the optimality conditions: to solving an approximate m m system to update u, updating x and consistently (to eliminate them), iterating Exact linearized analyses for updates to x and appear in the inner loop Consider replacing the exact elimination steps of RSQP with steps in an outer loop preconditioning Earlier proposed in this or closely related context by Batterman Heinkenschloss (996), Biros & Ghattas (998) & Algebraically identical to handling of interface constraints in domain decomposition by Keyes & Schur-complement-based Gropp (987)

14 Full Space Lagrange-NKS Method Apply KS directly to the (2n + m) (2n + m) KKT system: Wxx W T ux JT x Wuu J T u Wux Ju 0 Jx x u + A =, Need action of the full matrix on the full space vector Need a good full system preconditioner, for algorithmic scalability One Newton SQP iteration is a perfect preconditioner a block solver, based on forming the reduced Hessian of the factored Lagrangian H, but, of course, far too expensive Backing o wherever things get impractical in the preconditioner generates a family of methods gx gu h A

15 of Parameter Identication Example Radiation Transport in General = r ((x)t rt ) Initial conditions: impulsive boundary heating for T (Marshak wave) Use D steady example with jump in material properties Cost function is simple temperature matching based on a given (x) prole, T : c(u; x) = 2 jjt (x), T (x)jj 2, where, \Brisk-Spitzer" = 2:5; (x) = ; 0 x 0:5; (x) = 0; 0:5 < x :0 more generally, T (x) is a desired or experimental prole

16 Implementation in Matlab and ADMAT ADMAT (Verma & Coleman) is an automatic dierentiation for Matlab, based on operator overloading framework After supplying an m-le for the cost function and constraint all gradients and Jacobians and Hessians (as well functions, their transposes and their contracted action on vectors) are as without further user eort computed Preconditioner is RSQP block factorization, except that reduced is replaced with cost function Hessian alone, sparing Hessian sensitivity computations in a preconditioner Reduced Hessian should be replaced with quasi-newton reduced in the future Hessian Simple Newton method without robustication of any kind

17 LNKS for Radiation Diusion Example T (x) at initial (green) and nal (blue) (; R ) initially (0:5; :5) initially (0:5; 2) initially (2:8; 8) initially (2:9; 5)

18 LNKS Convergence History R ) trajectories from four KKT Norm Convergence History (; (all converging to (2:5; 0)) for the case (0:5; :5) cases Beta Right Alpha Iteration Norm of KKT residual

19 Preconditioning the Full System The shopping list of matrix actions in preconditioning is: Wxx; Wuu; Wux, and W T ux Ju and J T u J, x and J,T H, x are also needed in the full ) matrix-vector multiply. We require \working accuracy" system The rst six (together with Jx and J T x comparable to the state of the art in numerical dierentiation. accurate action of the last three is required in RSQP, but not The the full system preconditioner. We use approximate factoriza- in of lower quality approximations, including possibly just Wuu tions H, or a traditional quasi-newton rank-updated approximation for to the inverse.

20 Jacobian and Hessian Arithmetic Complexity estimate the complexity of applying each Jacobian block to a vector, assuming only that h(x; u) is We in subroutine form and that all dierentiated blocks are from AD tools, such as ADIC. We available the complexity of applying each Hessian block to a vector. Assume that h(x; u) and c(x; u) estimate available and that all dierentiated blocks are results of AD tools. We assume that J x is needed, are element-by-element, in order to factor it; hence, J T x is also available. Dene: C h, the cost of evaluating h p x, + the chromatic number of J x h; x p u, + the chromatic number of J u h; u C c, the cost of evaluating c q, + number of nonzero rows in c 00 r, implementation-dependent constant, approx. 350 Cost: forward mode Cost: fastest (hybrid) mode Object x ;J T x p x C h p x C h J 2Ch 2Ch Juv T u v p uc h rc h J W T uxv p x C h + qc c r(c h + C c ) Wxxv; W ux v p u C h + qc c r(c h + C c ) Wuuv;

21 Overall Arithmetic Complexity the inverse blocks, we need only low quality approximations, For low-ll (or low-rank correction inverse updates) of the square and systems: J, and J,T x x H, is only linear in subsystem dimensions n and m. Summarizing, Complexity all operations required to apply the full system matrix- product and its preconditioner are at worst linear in n or vector with coecients that depend upon: m, chromatic numbers (aected by stencil connectivity and intercomponent coupling of the PDE, and by separability structure of the objective function) implementation eciency of AD tools

22 Lagrange-Newton-Krylov-Schur-Schwarz: Parallel Optimizer for BVP-constrained Problems A Lagrange optimization formulation Newton nonlinear accelerator Krylov linear accelerator Schur subspace preconditioner Schwarz subdomain preconditioner

23 on Full Space Lagrange-Newton Method Remarks As with any Newton method, globalization strategies are important: parameter continuation (physical and algorithmic) mesh sequencing and multilevel iteration (for the PDE subsystem, at least; controls, too) probably discretization order progression model delity progression KKT system is a preconditioning challenge, but an exact factored preconditioner is known, and departures of preconditioned eigenvalues from unity be quantied with comparisons of original blocks with blockwise substitutions can in inexact models and solves (see, e.g., E. Sachs et al.) Orders of magnitude of savings may be available by converging the state and the design variables within the same outer iterative process, variables than a conventional SQP process that exactly satises the \auxiliary" rather constraints state With the extra work of forming Jacobian transposes and Hessian blocks, but extra work in Jacobian preconditioning, any parallel analysis code may no converted into a (limited) parallel optimization code and automatic be tools will shortly make this relatively painless dierentiation

24 Future Prospects Increased penetration of object-oriented coding practices will it easier to reuse subspace solvers as preconditioners in a make tightly-coupled Newton process. Automatic dierentiation tools will remove the burden of coding KKT blocks that are \missing" from a parallel solver, as well the as some that now require hand coding for the solver. Mathematical challenges and practical importance will attract numerical analysts to the preconditioning and globaliza- more tion problems of large-scale PDE-constrained optimization. for papers, talks, and workshop See summaries

Perspective \Have parallel PDE solver, will optimize" Years of two-sided (from architecture up, from applications down) research has put us in a posit

$Perspective \Have parallel PDE solver, will optimize Years of two-sided (from architecture up, from applications down) research has put us in a posit$ in Solving Inexactness Preconditioning and Systems: Partitioned Workshop Laundry List A David E. Keyes & Statistics Department, Mathematics Dominion University Old for Scientic Computing Research, Institute