Scalable Algorithms for Optimal Control of Systems Governed by PDEs Under Uncertainty

Scalable Algorithms for Optimal Control of Systems Governed by PDEs Under Uncertainty Alen Alexanderian 1, Omar Ghattas 2, Noémi Petra 3, Georg Stadler 4 1 Department of Mathematics North Carolina State University 2 Institute for Computational Engineering and Sciences The University of Texas at Austin 3 School of Natural Sciences University of California, Merced 3 Courant Institute of Mathematical Sciences New York University Advances in Uncertainty Quantification Methods, Algorithms and Applications Annual Meeting of KAUST SRI-UQ Center King Abdullah University of Science and Technology, Saudi Arabia January 5 10, 2016

From data to decisions under uncertainty Uncertain parameter m Mathematical Model A(m, u) = f Quantity of Interest (QoI) q(m) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 2 / 23

From data to decisions under uncertainty Uncertain parameter m prior posterior Bayesian inversion π post(m y) π like(y m)π 0(m) Experimental data y πlike(y m) = πnoise(bu y) Mathematical Model A(m, u) = f Quantity of Interest (QoI) q(m) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 2 / 23

From data to decisions under uncertainty Uncertain parameter m prior posterior Bayesian inversion π post(m y) π like(y m)π 0(m) Experimental data y πlike(y m) = πnoise(bu y) Mathematical Model A(m, u) = f experiments Quantity of Interest (QoI) q(m) Design of experiments min Ψ [ π post(m y; ξ) ] π(y)dy ξ ξ : experimental design Ψ: design objective/criterion Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 2 / 23

From data to decisions under uncertainty Uncertain parameter m prior posterior Bayesian inversion π post(m y) π like(y m)π 0(m) Experimental data y πlike(y m) = πnoise(bu y) Mathematical Model A(m, u) = f( ; z) z := control function experiments Quantity of Interest (QoI) q(z, m) Design of experiments min Ψ [ π post(m y; ξ) ] π(y)dy ξ ξ : experimental design Ψ: design objective/criterion Optimal control/design under uncertainty e.g. min q(z, m)µ(dm) z Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 2 / 23

Example: Groundwater contaminant remediation Source: Reed Maxwell, CSM Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 3 / 23

Example: Groundwater contaminant remediation Inverse problem Infer (uncertain) soil permeability from (uncertain) measurements of pressure head at wells and from a (uncertain) model of subsurface flow and transport Prediction (or forward) problem Predict (uncertain) evolution of contaminant concentration at municipal wells from (uncertain) permeability and (uncertain) subsurface flow/transport model Optimal experimental design problem Where should new observation wells be placed so that permeability is inferred with the least uncertainty? Optimal design problem Where should new remediation wells be placed so that (uncertain) contaminant concentrations at municipal wells are minimized? Optimal control problem What should the rates of extraction/injection at remediation wells be so that (uncertain) contaminant concentrations at municipal wells are minimized? Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 4 / 23

Optimal control of systems governed by PDEs with uncertain parameter fields PDE-constrained control objective Control of injection wells in porous medium flow (SPE10 permeability data) q = q(u(z, m)) where A(u, m) = f(z) q: control objective A: forward operator m: uncertain parameter field z: control function Problem: given the uncertainty model for m, find z that optimizes q(u(z, m)) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 5 / 23

Optimization under uncertainty (OUU) H : parameter space, infinite-dimensional separable Hilbert space q(z, m): control objective functional m H : uncertain model parameter field, Optimization under uncertainty (OUU): min z q(z, m) z: control function Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 6 / 23

Optimization under uncertainty (OUU) H : parameter space, infinite-dimensional separable Hilbert space q(z, m): control objective functional m H : uncertain model parameter field, z: control function Risk-neutral optimization under uncertainty (OUU): min E m {q(z, m)} z E m {q(z, m)} = H q(z, m) µ(dm) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 6 / 23

Optimization under uncertainty (OUU) H : parameter space, infinite-dimensional separable Hilbert space q(z, m): control objective functional m H : uncertain model parameter field, Risk-averse optimization under uncertainty (OUU): min E m {q(z, m)} + β var m {q(z, m)} z E m {q(z, m)} = q(z, m) µ(dm) H z: control function var m {q(z, m)} = E m {q(z, m) 2 } E m {q(z, m)} 2 Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 6 / 23

Optimization under uncertainty (OUU) H : parameter space, infinite-dimensional separable Hilbert space q(z, m): control objective functional m H : uncertain model parameter field, Risk-averse optimization under uncertainty (OUU): min E m {q(z, m)} + β var m {q(z, m)} z E m {q(z, m)} = q(z, m) µ(dm) H z: control function var m {q(z, m)} = E m {q(z, m) 2 } E m {q(z, m)} 2 Main challenges: Integration over infinite/high-dimensional parameter space Evaluation of q requires PDE solves Standard Monte Carlo approach (Sample Average Approximation) is prohibitive Numerous (n mc) samples required, each requires PDE solve Resulting PDE-constrained optimization problem has n mc PDE constraints Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 6 / 23

Some existing approaches for PDE-constrained OUU Methods based on stochastic collocation, sparse/adaptive sampling, POD,... Schulz & Schillings, Problem formulations and treatment of uncertainties in aerodynamic design, AIAA J, 2009. Borzì & von Winckel, Multigrid methods and sparse-grid collocation techniques for parabolic optimal control problems with random coefficients, SISC, 2009. Borzì, Schillings, & von Winckel, On the treatment of distributed uncertainties in PDE-constrained optimization, GAMM-Mitt. 2010. Borzì & von Winckel, A POD framework to determine robust controls in PDE optimization, Computing and Visualization in Science, 2011. Gunzburger & Ming, Optimal control of stochastic flow over a backward-facing step using reduced-order modeling, SISC 2011. Gunzburger, Lee, & Lee, Error estimates of stochastic optimal Neumann boundary control problems, SINUM, 2011. Kunoth & Schwab, Analytic Regularity and GPC Approximation for Control Problems Constrained by Linear Parametric Elliptic and Parabolic PDEs, SICON, 2013. Tiesler, Kirby, Xiu, & Preusser, Stochastic collocation for optimal control problems with stochastic PDE constraints, SICON, 2012. Kouri, Heinkenschloss, Ridzal, & Van Bloemen Waanders, A trust-region algorithm with adaptive stochastic collocation for PDE optimization under uncertainty, SISC, 2013. Chen, Quarteroni, & Rozza, Stochastic optimal Robin boundary control problems of advection-dominated elliptic equations, SINUM, 2013. Kouri, A multilevel stochastic collocation algorithm for optimization of PDEs with uncertain coefficients, JUQ, 2014. Chen & Quarteroni, Weighted reduced basis method for stochastic optimal control problems with elliptic PDE constraint, JUQ, 2014. Chen, Quarteroni, & Rozza, Multilevel and weighted reduced basis method for stochastic optimal control problems constrained by Stokes equations, Num. Math. 2015. Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 7 / 23

Control of injection wells in a porous medium flow 4 3 2 1 m = mean of log permeability field q = target pressure at production wells state PDE: single phase flow in a porous medium n c (e m u) = z i f i (x) with Dirchlet lateral & Neumann top/bottom BCs uncertain parameter: log permeability field m: control variables: z i, mass source at injection wells; f i, mollified Dirac deltas control objective: q(z, m) := 1 2 Qu(z, m) q 2, q: target pressure dimensions: n s = n m = 3242, n c = 20, n q = 12 i=1 Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 8 / 23

Porous medium with random permeability field Distribution law of m: µ = N ( m, C) (Gaussian measure on Hilbert space H ) Take covariance operator as square of inverse of Poisson-like operator: C = ( κ + αi) 2 κ, α > 0 C is positive, self-adjoint, of trace-class; µ well-defined on H (Stuart 10) κ α correlation length; the larger α, the smaller the variance Random draws for κ = 2 10 2, α = 4 Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 9 / 23

OUU with linearized parameter-to-objective map Risk-averse optimal control problem (including cost of controls) min z E m {q(z, m)} + β var m {q(z, m)} + γ z 2 Linear approximation to parameter-to-objective map g m (z, ) := dq(z, ) dm q lin (z, m) = q(z, m) + g m (z, m), m m is the gradient with respect to m The moments of the linearized objective: E m {q lin (z, )} = q(z, m), var m {q lin (z, )} = g m (z, m), C[g m (z, m)] ( ) q lin (z, ) N q(z, m), g m (z, m), C[g m (z, m)] Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 10 / 23

Risk-averse optimal control problem with linearized parameter-to-objective map State-and-adjoint-PDE constrained optimization problem (quartic in z): min z Z J (z) := 1 2 Qu q 2 + β 2 gm( m), C[gm( m)] + γ 2 z 2 with g m( m) =e m u p, where n c (e m u) = z if i i=1 (e m p) = Q (Qu q) state equation adjoint equation Lagrangian of the risk-averse optimal control problem with q lin : L (z, u, p, u, p ) = 1 2 Qu q 2 + β e m u p, C[e m u p] + γ 2 2 z 2 + e m u, u n c z i f i, u i=1 + e m p, p + Q (Qu q), p Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 11 / 23

Gradient computation for risk averse optimal control State problem for risk-averse optimal control problem with q lin : e m u, ũ n c = z i f i, ũ i=1 e m p, p = Q (Qu q), p for all test functions p and ũ Adjoint problem for risk-averse optimal control problem with q lin : e m p, p = β e m u p, C[e m u p] e m u, ũ = Q (Qu q), ũ β e m ũ p, C[e m u p] Q Qp, ũ for all test functions p and ũ J Gradient: = γz j f j, u, j = 1,..., n c z j Cost of objective = 2 PDE solves; cost of gradient = 2 PDE solves Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 12 / 23

Risk-averse optimal control with linearized objective distribution 0.04 0.03 0.02 Θ(z 0, ) Θlin(z 0, ) Θquad(z 0, ) 0.01 0 20 0 20 40 60 80 100 120 140 initial (suboptimal) control z 0 distrib. of exact & approx objectives at z 0 Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 13 / 23

Risk-averse optimal control with linearized objective distribution 0.04 0.03 0.02 Θ(z 0, ) Θlin(z 0, ) Θquad(z 0, ) 0.01 0 20 0 20 40 60 80 100 120 140 initial (suboptimal) control z 0 distrib. of exact & approx objectives at z 0 5 4 Θ(z opt lin, ) Θlin(z opt lin, ) Θquad(z opt lin, ) distribution 3 2 1 0 0 2 4 6 8 10 12 14 16 optimal control z opt lin based on q lin distrib. of exact & approx objectives at z opt lin Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 13 / 23

Quadratic approximation to parameter-to-objective map Quadratic approximation to the parameter-to-control-objective map: q quad (z, m) = q(z, m) + g m (z, m), m m + 1 2 H m(z, m)(m m), m m g m : gradient of parameter-to-objective map H m : Hessian of parameter-to-objective map Observations: Quadratic approximation does not lead to a Gaussian control objective However, can derive analytic formulas for the moments of q quad in the infinite-dimensional Hilbert space setting Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 14 / 23

Analytic expressions for mean and variance with q quad Mean: E m {q quad (z, )} = q(z, m)+ 1 2 tr[ Hm (z, m) ] Variance: var m {q quad (z, )} = g m (z, m), C[g m (z, m)] + 1 2 tr[ Hm (z, m) 2] where H m = C 1/2 H m C 1/2 is the covariance-preconditioned Hessian Risk averse optimal control objective with q quad : J(z) = q(z, m) + 1 2 tr[ Hm (z, m) ] + β { g m (z, m), C[g m (z, m)] + 12 2 tr[ Hm (z, m) 2]} Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 15 / 23

Randomized trace estimator Randomized trace estimation: tr( H m ) 1 n tr tr( H 2 m) 1 n tr n tr n tr Hm ξ j, ξ j = 1 n tr H m ζ j, C[H m ζ j ] n tr H m ζ j, ζ j where ζ j = C 1/2 ξ j In computations, we use draws ζ j N (0, C) =: ν Straightforward to show: H m ζ, ζ ν(dζ) = tr( H m ), H H H m ζ, C[H m ζ] ν(dζ) = tr( H 2 m) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 16 / 23

Risk-averse optimal control with quadraticized objective 1 min z Z 2 Qu q 2 2 + 1 n tr 2n tr ζ j, η j + β 2 { g m ( m), C[g m ( m)] + 1 n tr 2n tr } C 1/2 η j 2 with g m ( m) =e m u p η j = e m (ζ j u p + υ j p + u ρ j ) }{{} H mζ j j {1,..., n tr} where (e m u) = N i=1 z if i (e m p) = Q (Qu q) (e m υ j ) = (ζ j e m u) (e m ρ j ) = Q Qυ j + (ζ j e m p) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 17 / 23

Risk-averse optimal control with quadraticized objective 1 min z Z 2 Qu q 2 2 + 1 n tr 2n tr ζ j, η j + β 2 { g m ( m), C[g m ( m)] + 1 n tr 2n tr } C 1/2 η j 2 with g m ( m) =e m u p η j = e m (ζ j u p + υ j p + u ρ j ) }{{} H mζ j j {1,..., n tr} where (e m u) = N i=1 z if i (e m p) = Q (Qu q) (e m υ j ) = (ζ j e m u) (e m ρ j ) = Q Qυ j + (ζ j e m p) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 17 / 23

Risk-averse optimal control with quadraticized objective 1 min z Z 2 Qu q 2 2 + 1 n tr 2n tr ζ j, η j + β 2 { g m ( m), C[g m ( m)] + 1 n tr 2n tr } C 1/2 η j 2 with g m ( m) =e m u p η j = e m (ζ j u p + υ j p + u ρ j ) }{{} H mζ j j {1,..., n tr} where (e m u) = N i=1 z if i (e m p) = Q (Qu q) (e m υ j ) = (ζ j e m u) (e m ρ j ) = Q Qυ j + (ζ j e m p) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 17 / 23

Risk-averse optimal control with quadraticized objective 1 min z Z 2 Qu q 2 2 + 1 n tr 2n tr ζ j, η j + β 2 { g m ( m), C[g m ( m)] + 1 n tr 2n tr } C 1/2 η j 2 with g m ( m) =e m u p η j = e m (ζ j u p + υ j p + u ρ j ) }{{} H mζ j j {1,..., n tr} where (e m u) = N i=1 z if i (e m p) = Q (Qu q) (e m υ j ) = (ζ j e m u) (e m ρ j ) = Q Qυ j + (ζ j e m p) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 17 / 23

Risk-averse optimal control with quadraticized objective 1 min z Z 2 Qu q 2 2 + 1 n tr 2n tr ζ j, η j + β 2 { g m ( m), C[g m ( m)] + 1 n tr 2n tr } C 1/2 η j 2 with g m ( m) =e m u p η j = e m (ζ j u p + υ j p + u ρ j ) }{{} H mζ j j {1,..., n tr} where (e m u) = N i=1 z if i (e m p) = Q (Qu q) (e m υ j ) = (ζ j e m u) (e m ρ j ) = Q Qυ j + (ζ j e m p) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 17 / 23

Lagrangian for risk-averse optimal control with q quad L (z, u, p,{υ j} ntr, {ρ j} ntr, u, p, {υ j } ntr, {ρ j } ntr ) = 1 2 Qu q 2 2 + 1 2n tr n tr ζj, [ e m (ζ j u p + υ j p + u ρ j) ] + β e m u p, C[e m u p] 2 + β n tr C 1/2 [ e m (ζ j u p + υ j p + u ρ ] 2 j) 4n tr + e m u, u N i=1 zi fi, u + e m p, p + Q (Qu q), p n tr [ e + m υ j, υ j + ζje m u, υ ] j n tr [ e + m ρ j, ρ j + Q Qυ j, ρ j + ζje m p, ρ ] j Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 18 / 23

Adjoint & gradient for risk-averse optimal control w/q quad Adjoint problem for q quad approximation (e m ρ j ) = b (j) 1 j {1,..., n tr } (e m υ j ) + Q Qρ j = b (j) 2 j {1,..., n tr } n tr (e m p ) (ζ j e m ρ j ) = b 3 n tr (e m u ) + Q Qp (ζ j e m υ j ) = b 4 Gradient for q quad approximation L z j = γz j f j, u, j = 1,..., n c Cost of objective = 2 + 2n tr PDE solves; cost of gradient = 2 + 2n tr PDE solves Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 19 / 23

Risk-averse optimal control with quadraticized objective distribution 0.04 0.03 0.02 Θ(z 0, ) Θlin(z 0, ) Θquad(z 0, ) 0.01 0 20 0 20 40 60 80 100 120 140 initial (suboptimal) control z 0 distrib. of exact & approx objectives at z 0 Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 20 / 23

Risk-averse optimal control with quadraticized objective distribution 0.04 0.03 0.02 Θ(z 0, ) Θlin(z 0, ) Θquad(z 0, ) 0.01 0 20 0 20 40 60 80 100 120 140 initial (suboptimal) control z 0 distrib. of exact & approx objectives at z 0 0.6 Θ(z opt quad, ) Θquad(z opt quad, ) distribution 0.4 0.2 optimal control z opt quad based on q quad 0 0 2 4 6 8 10 12 14 16 distrib. of exact & approx objectives at z opt quad Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 20 / 23

Effect of number of trace estimator vectors on distribution of control objective evaluated at optimal controls distribution 0.4 0.2 n tr = 5 n tr = 20 n tr = 40 n tr = 60 0 0 1 2 3 4 5 6 7 8 value of control objective Optimal controls z opt quad computed for each value of trace estimator using quadratic approximation of control objective, q quad Each curve based on 10,000 samples of distribution of q(z opt quad, m) (control objective evaluated at optimal control) Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 21 / 23

Comparison of distribution of control objective for optimal controls based on linearized and quadraticized objective distribution 0.6 0.4 0.2 sample mean 5 4 3 2 1 1000 5000 10000 sample size sample variance 15 10 5 1000 5000 10000 sample size Θ(z opt quad, ) Θ(z opt lin, ) 0 0 1 2 3 4 5 6 7 8 value of control objective Comparison of the distributions of q(z opt lin, m) with q(zopt quad, m) β = 1, γ = 10 5 and n tr = 40 trace estimation vectors KDE results are based on 10,000 samples Inserts show Monte Carlo sample convergence for mean and variance Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 22 / 23

Summary Some concluding remarks: Optimal control of PDEs with infinite-dimensional parameters Use of local approximations to parameter-to-objective map Quadratic approximation is effective in capturing the distribution of the control objective for the target problem Computational complexity of objective & gradient evaluation is independent of problem dimension (n m = 3242) objective + gradient cost = 125 PDE solves objective/gradient cost for Monte Carlo/SAA: 10,000 PDE solves differences even more striking for nonlinear PDE state problems Possible extensions: Higher moments (e.g., for forward UQ) Alternative risk measures Higher order approximations, e.g., third order Taylor expansion with proper tensor contraction of third order derivative tensor A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, Mean-variance risk-averse optimal control of systems governed by PDEs with random coefficient fields using second order approximations, in submission. Omar Ghattas (ICES, UT Austin) Optimal control under uncertainty UQAW 2016 23 / 23