Polynomial Chaos Based Uncertainty Propagation

Transcription

1 Polynomial Chaos Based Uncertainty Propagation Lecture 2: Forward Propagation: Intrusive and Non-Intrusive Bert Debusschere Sandia National Laboratories, Livermore, CA Aug 12, 2013 Summer School, USC

2 Acknowledgement Cosmin Safta Khachik Sargsyan Katy Jarvis Habib Najm Omar Knio Roger Ghanem Olivier Le Maître and many others... Sandia National Laboratories Livermore, CA Duke University, Raleigh, NC University of Southern California Los Angeles, CA LIMSI-CNRS, Orsay, France This work was supported by the US Department of Energy (DOE), Office of Advanced Scientific Computing Research (ASCR), Scientific Discovery through Advanced Computing (SciDAC) and Applied Mathematics Research (AMR) programs. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL85000.

3 Goals of this tutorial 2 Lectures Lecture 1: Context and Fundamentals Lecture 2: Forward Propagation Intrusive Approaches Non-Intrusive Approaches Sensitivity Analysis High-dimensional Systems

4 Outline 1 Introduction 2 Forward Propagation of Uncertainty 3 Sensitivity Analysis 4 Sparse Quadrature Approaches for High-Dimensional Systems 5 References

5 This section focuses on propagating uncertainty through computational models. Experiments D, P(λ D, I) = Inference P(D λ, I)P(λ, I) P(D) P(λ) (λ, (λ) Predictive Simulation Theory du dt = f (u; λ) Forward Propagation du t P(u) u Assume uncertain parameters λ have been characterized with PCEs The goal is to obtain PCEs for output quantities u

6 Propagation of Uncertain Inputs Represented with PCEs Galerkin Projection u ψ u(ξ) 1 Collocation ũ u 1 u o ψ o u k = uψ k Ψ 2 k, k = 0,..., P Residual orthogonal to space covered by basis functions Match PCE to random variable at chosen sample points: interpolation or regression ξ

7 Galerkin projection methods are either intrusive or non-intrusive Use same projection but in different ways u k = uψ k Ψ 2 k, k = 0,..., P Intrusive methods apply Galerkin projection to governing equations Results in set of equations for the PC coefficients Requires redesign of computer code PCEs for all uncertain variables in system Non-intrusive approaches apply Galerkin projection to outputs of interest Sampling to evaluate projection operator Can use existing code as black box Only computes PCEs for quantities of interest

8 Collocation approaches are non-intrusive and minimize errors at sample points u k Ψ k (ξ i ) = u(ξ i ) k=0 i = 1,..., N c 1.5 u(ξ) Use functional representation point of view Can use interpolation, e.g. Lagrange interpolants Or use regression approaches: P + 1 degrees of freedom to fit N c points Can position points where most accuracy desired ξ

9 Remainder of this section focuses on Galerkin projection methods Intrusive Galerkin projection Non-intrusive Galerkin projection

10 Intrusive Galerkin projection reformulates original equations Assume v = f (u; a, λ), with a deterministic parameter(s) λ uncertain parameter(s) u, v variables of interest (deterministic or uncertain) Represent uncertain variables with PCEs λ = λ k Ψ k (ξ), v = k=0 v k Ψ k (ξ) k=0 Apply Galerkin projection to get PC coefficients of v v k = vψ k Ψ 2 k = f (u; a, λ)ψ k Ψ 2 k, k = 0,..., P Results in larger, but deterministic set of equations

11 Projection of linear operations: v = γ + λ Assume v = γ + λ, with γ = P k=0 γ kψ k, λ = P k=0 λ kψ k, and v = P k=0 v kψ k Galerkin projection v k = vψ k Ψ 2 k = (γ + λ)ψ k Ψ 2 k, k = 0,..., P (γ + λ)ψ k = v k = γ k + λ k = γ j Ψ j Ψ k + λ j Ψ j Ψ k j=0 γ j Ψ j Ψ k + j=0 j=0 λ j Ψ j Ψ k j=0 = γ k Ψ 2 k + λk Ψ 2 k

12 Projection of linear operations: v = a + λ Special case of v = γ + λ, with γ = a Ψ 0 = a λ = P k=0 λ kψ k v = P k=0 v kψ k Resulting in v 0 = a + λ 0, v k>0 = λ k Deterministic parameter is a special case of a PCE with 0 th term only Adding a deterministic value to a PCE just shifts its mean

13 Projection of linear operations: v = a λ Assume v = a λ, with a deterministic, λ = P k=0 λ kψ k, and v = P k=0 v kψ k Galerkin projection v k = vψ k Ψ 2 k = (a λ)ψ k Ψ 2 k, k = 0,..., P v k = a λ k (a λ)ψ k = a λ j Ψ j Ψ k j=0 = a λ j Ψ j Ψ k j=0 = a λ k Ψ 2 k

14 Projection of product: v = γ λ Assume v = γ λ, with γ = P k=0 γ kψ k, λ = P k=0 λ kψ k, and v = P k=0 v kψ k Galerkin projection v k = vψ k = (γ λ)ψ k, k = 0,..., P Ψ 2 k Ψ 2 k (γ λ)ψ k = = = γ i Ψ i λ j Ψ j Ψ k i=0 i=0 i=0 j=0 j=0 γ i λ j Ψ i Ψ j Ψ k j=0 γ i λ j Ψ i Ψ j Ψ k

15 Projection of product: v = γ λ v k = i=0 j=0 γ i λ j Ψ i Ψ j Ψ k Ψ 2 k = i=0 γ i λ j C ijk C ijk = Ψ i Ψ j Ψ k Ψ 2 k The C ijk tensor can be computed up front for any given PC order and dimension and stored for use whenever two RVs are multiplied This tensor is sparse, i.e. many of its elements are zero j=0

16 1D 4 th -Order C ijk Example : Hermite polynomials Ψ i Ψ j Ψ k value Ψ 0 Ψ 0 Ψ 0 1 Ψ 0 Ψ 1 Ψ 1 1 Ψ 0 Ψ 2 Ψ 2 2 Ψ 0 Ψ 3 Ψ 3 6 Ψ 0 Ψ 4 Ψ 4 24 Ψ 1 Ψ 1 Ψ 2 2 Ψ 1 Ψ 2 Ψ 3 6 Ψ 1 Ψ 3 Ψ 4 24 Ψ 2 Ψ 2 Ψ 2 8 Ψ 2 Ψ 2 Ψ 4 24 Ψ 2 Ψ 3 Ψ 3 36 Ψ 2 Ψ 4 Ψ Ψ 3 Ψ 3 Ψ Ψ 4 Ψ 4 Ψ k Ψ 2 k C ijk = Ψ i Ψ j Ψ k / Ψ 2 k and, Ψ i Ψ j Ψ k = Ψ i Ψ k Ψ j = Ψ j Ψ i Ψ k = Ψ j Ψ k Ψ i = Ψ k Ψ i Ψ j = Ψ k Ψ j Ψ i with other not-reported Ψ i Ψ j Ψ k zero

17 1D 4 th -Order C ijk Example : Legendre polynomials Ψ i Ψ j Ψ k value Ψ 0 Ψ 0 Ψ 0 1 Ψ 0 Ψ 1 Ψ 1 1/3 Ψ 0 Ψ 2 Ψ 2 1/5 Ψ 0 Ψ 3 Ψ 3 1/7 Ψ 0 Ψ 4 Ψ 4 1/9 Ψ 1 Ψ 1 Ψ 2 2/15 Ψ 1 Ψ 2 Ψ 3 3/35 Ψ 1 Ψ 3 Ψ 4 4/63 Ψ 2 Ψ 2 Ψ 2 2/35 Ψ 2 Ψ 2 Ψ 4 2/35 Ψ 2 Ψ 3 Ψ 3 4/105 Ψ 2 Ψ 4 Ψ Ψ 3 Ψ 3 Ψ Ψ 4 Ψ 4 Ψ k Ψ 2 k /3 2 1/5 3 1/7 4 1/9 C ijk = Ψ i Ψ j Ψ k / Ψ 2 k and, Ψ i Ψ j Ψ k = Ψ i Ψ k Ψ j = Ψ j Ψ i Ψ k = Ψ j Ψ k Ψ i = Ψ k Ψ i Ψ j = Ψ k Ψ j Ψ i with other not-reported Ψ i Ψ j Ψ k zero

18 Projection of triple product: v = γ λ u Assume v = γ λ u, with γ = P k=0 γ kψ k, λ = P k=0 λ kψ k, u = P k=0 u kψ k, and v = P k=0 v kψ k Galerkin projection v k = vψ k = (γ λ u)ψ k, k = 0,..., P Ψ 2 k Ψ 2 k (γ λ u)ψ k = = = γ l Ψ l λ i Ψ i u j Ψ j Ψ k l=0 l=0 i=0 l=0 i=0 j=0 i=0 j=0 γ l λ i u j Ψ l Ψ i Ψ j Ψ k j=0 γ l λ i u j Ψ l Ψ i Ψ j Ψ k

19 Projection of product: v = γ λ u v k = l=0 i=0 j=0 γ l λ i u j Ψ l Ψ i Ψ j Ψ k Ψ 2 k = i=0 γ l λ i u j D lijk D lijk = Ψ l Ψ i Ψ j Ψ k Ψ 2 k The D lijk tensor can also be computed up front for any given PC order and dimension and stored for use whenever three RVs are multiplied While this tensor is sparse, it can be expensive to compute and store This fully spectral formulation is even less practical for higher order products j=0

20 Pseudo-spectral triple product: v = γ λ u Assume v = γ λ, with γ = P k=0 γ kψ k, λ = P k=0 λ kψ k, u = P k=0 u kψ k, and v = P k=0 v kψ k Perform product in sub-steps v = γ λ u = ((γ λ)u) = ṽu Each sub-step can be performed with regular binary product formula ṽ = γ λ v = ṽu

21 Pseudo-spectral product readily generalizes to higher order products Decompose higher order products or powers in sequences of binary products Equivalent to successive multiplication (accounting for terms up to order 2p) and projecting back to order p Efficient and convenient Can lead to aliasing errors due to loss of information in higher order modes See also [Debusschere et al., SIAM J. Sci. Comp, 2004]

22 Intrusive propagation through non-polynomial functions Addition, subtraction, and product allow (pseudo-)spectral evaluation of all polynomial functions How to propagate PC expansions ({u k } {v k }) through transcendental functions v = 1, v = ln u, or v = eu u Use local polynomial approximations, e.g. Taylor series Rework operation into a system of equations Integration approach Borchardt-Gauss Algorithm: Arithmetic-Geometric Mean (AGM) series [Debusschere et al., SISC 2004; McKale, Texas Tech, M.S. Thesis, 2011]

23 Taylor series allows computation of many transcendental functions Provides local polynomial approximation E.g. e u = 1 + u 1! + u2 2! + u3 3! +... Expanding the series for f (u) around u 0 speeds up convergence Works well in most cases, especially for small uncertainties Not very robust for larger uncertainties Series can take too long to converge High-order PC multiplications lead to aliasing Instabilities if Taylor series range of convergence exceeded; e.g. log(u) expanded around u 0 only converges for u u 0 < 1

24 Inversion and division can be computed through a system of equations Assume three uncertain variables u, v, and w w = u v v w = u Mode k of the stochastic product C ijk v i w j = u k i=0 j=0 System of P + 1 linear equations in w j with known u k and v i, V kj = C ijk v i Vw = u i=0 More robust than Taylor series expansion for 1/u

25 Integration approach for non-polynomial functions Consider the ODE dv = v, with solution v = eu du f (u) = e u can be obtained from dv = v du e u e uo = Similarly for e u2, and ln(u) u u o v du e u2 e u2 o = u u o 2uv du, ln(u) ln(u o ) = u u o du u Accurate if PC order is high enough to properly capture the random variable v = f (u)

26 More general formulation of integration approach for irrational functions To evaluate v(u), u = P k=0 u kψ k, v = P k=0 v kψ k, Use a deterministic IC u a such that v(u a ) is known Express v = dv/du = f (v, u);... require: f is a rational function... ensures that ( v) k are found from v k and u k coeffs Evaluate the integral: (ub ) j v k (u b ) v k (u a ) = C ijk ( v) i du j j=0 (u a) j ok for e u, e u2, and ln(u), with v = v, 2uv, and 1/u resp. but not for e sin u, with v = v cos u More robust than Taylor series, but CPU-intensive i=0

27 Overloading of operations Construction allows for a general representation using pseudo-spectral (PS) overloaded operations. E.g. multiplication operation w = λ u u v Each deterministic function multiplication is transformed into a corresponding PC product Potential meta-code: take a general deterministic code function F(u), produce a pseudo-spectral stochastic function F(ũ) Possibility of transforming legacy deterministic code into corresponding pseudo-spectral stochastic code. Toolkit: contains library of utilities for operations on random variables represented with PCEs

28 Surface Reaction Model 3 ODEs for a monomer (u), dimer (v), and inert species (w) adsorbing onto a surface out of gas phase. du dt = az cu 4duv dv dt = 2bz 2 4duv dw dt = ez fw z = 1 u v w u(0) = v(0) = w(0) = 0.0 a = 1.6 b = c = 0.04 d = 1.0 e = 0.36 f = Species Mass Fractions [-] u v w Time [-] Oscillatory behavior for b [20.2, 21.2] [Vigil et al., Phys. Rev. E., 1996; Makeev et al., J. Chem. Phys., 2002]

29 Surface reaction model shows wide range of dynamics (Vigil et al., Phys. Debusschere Rev. E., SNL1996; Makeev et al., J. Chem u(t) 0.20 v(t) b =19.4 b = b =19.4 b = b =21.0 b =21.8 b =21.0 b =21.8 b = t b = t 0.80 w(t) b =19.4 b =20.2 b =21.0 b =21.8 a = 1.6 b = [ ] c = 0.04 d = 1.0 e = 0.36 f = b = t

30 Surface Reaction Model: Intrusive Spectral Propagation (ISP) of Uncertainty Assume PCE for uncertain parameter b and for the output variables, u, v, w Substitute PCEs into the governing equations Project the governing equations onto the PC basis functions Multiply with Ψ k and take the expectation Apply pseudo-spectral approximations where necessary

31 Surface Reaction Model: Specify PCEs for inputs and outputs Represent uncertain inputs with PCEs with known coefficients: b = b i Ψ i (ξ) i=0 Represent all uncertain variables with PCEs with unknown coefficients: u(t) = w(t) = u i (t)ψ i (ξ) v(t) = v i (t)ψ i (ξ) i=0 w i (t)ψ i (ξ) z(t) = i=0 i=0 i=0 z i (t)ψ i (ξ)

32 Surface Reaction Model: Substitute PCEs into governing equations and project onto basis functions Ψ k d dt d dt du dt = az cu 4duv u i Ψ i = a z i Ψ i c u i Ψ i 4d i=0 u i Ψ i i=0 = i=0 aψ k 4dΨ k P i=0 P i=0 i=0 z i Ψ i u i Ψ i P j=0 cψ k v j Ψ j P i=0 u i Ψ i i=0 u i Ψ i P j=0 v j Ψ j

33 Surface Reaction Model: Reorganize terms d dt u k Ψ 2 k = azk Ψ 2 k cuk Ψ 2 k 4d d dt u k = az k cu k 4d d dt u k = az k cu k 4d i=0 i=0 j=0 i=0 u i v j Ψ i Ψ j Ψ k j=0 Ψ i Ψ j Ψ k u i v j Ψ 2 k u i v j C ijk j=0 Triple products C ijk = Ψ i Ψ j Ψ k Ψ 2 k and stored for repeated use can be pre-computed

34 Surface Reaction Model: Substitute PCEs into governing equations and project onto basis functions Ψ k d dt d dt dv dt = 2bz 2 4duv v i Ψ i = 2 b h Ψ h i=0 v i Ψ i i=0 = h=0 2Ψ k 4dΨ k P h=0 P i=0 P i=0 b h Ψ h u i Ψ i z i Ψ i P i=0 P j=0 P j=0 z i Ψ i z j Ψ j 4d P j=0 v j Ψ j z j Ψ j u i Ψ i i=0 P j=0 v j Ψ j

35 Surface Reaction Model: Reorganize terms d dt v k Ψ 2 k = 2 d dt v k = 2 d dt v k = 2 h=0 i=0 j=0 h=0 i=0 j=0 b h z i z j Ψ h Ψ i Ψ j Ψ k 4d h=0 i=0 j=0 b h z i z j Ψ h Ψ i Ψ j Ψ k Ψ 2 k 4d b h z i z j D hijk 4d i=0 j=0 i=0 j=0 i=0 j=0 u i v j Ψ i Ψ j Ψ k u i v j C ijk Ψ i Ψ j Ψ k u i v j Ψ 2 k Pre-computing and storing the quad product D hijk becomes cumbersome Use pseudo-spectral approach instead

36 Surface Reaction Model: Pseudo-Spectral approach for products Introduce auxiliary variable g = z 2 g = z 2 f = 2bz 2 = 2bg g k = f k = 2 i=0 i=0 z i z j C ijk j=0 b i g j C ijk Limits the complexity of computing product terms Higher products can be computed by repeated use of the same binary product rule Does introduce errors if order of PCE is not large enough j=0

37 Surface Reaction Model: ISP results Species Mass Fractions [-] u v w Time [-] Assume 0.5% uncertainty in b around nominal value Legendre-Uniform intrusive PC Mean and standard deviation for u, v, and w Uncertainty grows in time

38 Surface Reaction Model: ISP results u 0 u 2 u 4 u 1 u 3 u 5 u i [-] Time [-] Modes of u Modes decay with higher order Amplitudes of oscillations of higher order modes grow in time

39 Surface Reaction Model: ISP results: PDFs Prob. Dens. [-] 350 t = t = Prob. Dens. [-] 16 t = t = u u Pdfs of u at maximum mean (left) and maximum standard deviation (right) Distributions get broader and multimodal as time increases Effect of accumulating uncertainty in phase of oscillation

40 Remainder of this section focuses on Galerkin projection methods Intrusive Galerkin projection Non-intrusive Galerkin projection

41 Non-intrusive Galerkin projection u k = uψ k = 1 Ψ 2 k Ψ 2 k uψ k (ξ)w(ξ)dξ, k = 0,..., P Evaluate projection integrals numerically Pick samples of uncertain parameters, e.g. b(ξ) by sampling ξ Run deterministic forward model for each of the sampled input parameter values b i = b(ξ i ) Integration using random sampling or quadrature methods Reconstruct uncertain model output u(x, t; θ) = u k (x, t)ψ k (ξ(θ)) k=0

42 Random sampling approaches for Galerkin projection Evaluate integral through sampling uψ k (ξ)w(ξ)dξ = 1 N s N s u(ξ i )Ψ k (ξ i ) Samples are drawn according to the distribution of ξ Monte-Carlo (MC) Latin-Hypercube-Sampling (LHS) Pros: i=1 Can be easily made fault tolerant Sometimes random samples is all we have Cons: slow convergence, but less dependent on number of stochastic dimensions

43 Quadrature approaches for Galerkin projection Use numerical quadrature rules to evaluate integrals N q uψ k (ξ)w(ξ)dξ = q i u(ξ i )Ψ k (ξ i ) The N q ξ i are quadrature points, with corresponding weights q i Choice of quadrature points important for accuracy Also referred to as deterministic sampling approach Pros: Can use existing codes as black box to evaluate u(ξ i ) Embarrassingly parallel Cons: Tensor product rule for d dimensions requires N d q samples i=1

44 Gauss quadrature rules are very efficient N q uψ k (ξ)w(ξ)dξ = q i u(ξ i )Ψ k (ξ i ) i=1 N q quadrature points can integrate polynomial of order 2N q 1 exactly Gauss-Hermite and Gauss-Legendre quadrature tailored to specific choices of the weight function w(ξ) As a rule of thumb, p + 1 quadrature points are needed for Galerkin projection of PCE of order p If both u and Ψ k are of order p, then integrand is of order 2p 2p 2N q 1 or N q p Only exact if u is indeed a polynomial of order p

45 Surface Reaction Model 3 ODEs for a monomer (u), dimer (v), and inert species (w) adsorbing onto a surface out of gas phase. du dt = az cu 4duv dv dt = 2bz 2 4duv dw dt = ez fw z = 1 u v w u(0) = v(0) = w(0) = 0.0 a = 1.6 b = c = 0.04 d = 1.0 e = 0.36 f = Species Mass Fractions [-] u v w Time [-] Oscillatory behavior for b [20.2, 21.2] [Vigil et al., Phys. Rev. E., 1996; Makeev et al., J. Chem. Phys., 2002]

46 Surface Reaction Model: NISP results Species Mass Fractions [-] NISP Quadrature u v w Time [-] Species Mass Fractions [-] NISP MC u v w Time [-] Mean and standard deviation for u, v, and w Quadrature approach agrees well with ISP approach using 6 quadrature points Monte Carlo sampling approach converges slowly With a 1000 samples, results are quite different from ISP and NISP

47 Surface Reaction Model: Comparison ISP and NISP u i [-] u 4,ISP u 4,NISP u 5,ISP u 5,NISP Time [-] Lower order modes agree perfectly Very small differences in higher order modes Difference increases with time

48 Surface Reaction Model: Comparison ISP and NISP Prob. Dens. [-] ISP, t = NISP, t = u All pdf s based on 50K samples each and evaluated with Kernel Density Estimation (KDE) No difference in PDFs of sampled PCEs between NISP and ISP

49 Surface Reaction Model: Comparison ISP, NISP, and MC Prob. Dens. [-] ISP, t = NISP, t = MC, t = u All pdf s based on 50K samples each and evaluated with Kernel Density Estimation (KDE) Good agreement between intrusive, non-intrusive projection, and Monte Carlo sampling

50 ISP pros and cons Pros: Elegant One time solution of system of equations for the PC coefficients fully characterizes uncertainty in all variables at all times Tailored solvers can (potentially) take advantage of new hardware developments Cons: Often requires re-write of the original code Reformulated system is factor (P+1) larger than the original system and can be challenging to solve Challenges with increasing time-horizon for ODEs Many efforts in the community to automate ISP

51 NISP pros and cons Pros: Cons: Easy to use as wrappers around existing codes Embarassingly parallel Most methods suffer from curse of dimensionality N q = n N d Many development efforts for smarter sampling approaches and dimensionality reduction (Adaptive) Sparse Quadrature approaches Compressive Sensing... Sampling methods have found very wide spread use in the community

52 Software for intrusive and non-intrusive DAKOTA: Wide variety of non-intrusive methods for uncertainty propagation, sensitivity analysis, etc. Mature and well-supported Toolkit (Tk): Intrusive and non-intrusive Geared towards algorithm development and educational use Sundance: -enabled finite-element solution of PDEs Stokhos: Package for intrusive Galerkin based...

53 Extra Material on Forward Propagation Alternative derivation of v = a + λ Simple ODE example Intrusive of incompressible flow Uncertainty Quantification Toolkit (Tk) implementation of intrusive and non-intrusive in surface reaction example (3 ODE system)

54 Projection of linear operations: v = a + λ Assume v = a + λ, with a deterministic, λ = P k=0 λ kψ k, and v = P k=0 v kψ k Galerkin projection v k = vψ k Ψ 2 k = (a + λ)ψ k Ψ 2 k, k = 0,..., P (a + λ)ψ k = aψ k + λ j Ψ j Ψ k = a Ψ k + v 0 = a + λ 0, v k>0 = λ k j=0 λ j Ψ j Ψ k j=0 = a δ 0k + λ k Ψ 2 k

55 Intrusive Spectral Stochastic Formulation: ODE Example Sample ODE with parameter λ: du dt = λu Let λ be uncertain; introduce ξ N (0, 1). Express λ and u using PCEs in ξ: λ = λ k Ψ k (ξ), u(t) = k=0 u k (t)ψ k (ξ) k=0 Substitute in ODE and apply a Galerkin projection on Ψ i (ξ),

56 Galerkin Projection on Ψ i (ξ) ( P ) d u k (t)ψ k (ξ) = dt k=0 du k (t) Ψ k (ξ) = dt k=0 P du k (t) Ψ k (ξ)ψ i (ξ) = dt k=0 k=0 du k (t) dt Ψ k (ξ)ψ i (ξ) = du i dt Ψ 2 i = λ p Ψ p (ξ) u q (t)ψ q (ξ) p=0 p=0 q=0 P q=0 λ p u q (t)ψ p (ξ)ψ q (ξ) p=0 q=0 p=0 q=0 p=0 q=0 λ p u q (t)ψ p (ξ)ψ q (ξ)ψ i (ξ) λ p u q (t) Ψ p (ξ)ψ q (ξ)ψ i (ξ) λ p u q Ψ p Ψ q Ψ i

57 Resulting Spectral ODE system (P + 1)-dimensional ODE system du i dt = p=0 q=0 λ p u q C pqi, i = 0,..., P where C pqi = Ψ p Ψ q Ψ i / Ψ 2 i The tensor C pqi can be evaluated once and stored for any given PC order and dimension This tensor is sparse, i.e. many elements are zero

58 Pseudo-Spectral Construction 1 w = λu 2 v, u = Spectral: u k Ψ k, similarly for λ & v k=0 w i = λu 2 v i = λ j u k u l v m Ψ j Ψ k Ψ l Ψ m i, j=0 k=0 l=0 m=0 i = 0,..., P The corresponding tensor of basis product expectations becomes too large to pre-compute and store Pseudo-Spectral: Project each PC product onto a (P + 1)-polynomial before proceeding further, thus:

59 Pseudo-Spectral Construction 2 w = uv : w i = uv i = u k v j Ψ k Ψ j i, i = 0,..., P j=0 j=0 k=0 ŵ = u w : ŵ i = u w i = u k w j Ψ k Ψ j i, i = 0,..., P k=0 w = λŵ : w i = λŵ i = λ k ŵ j Ψ k Ψ j i, i = 0,..., P j=0 k=0 Aliasing errors Efficiency, and convenience [Debusschere et al., SIAM J. Sci. Comp., 2004.]

60 Spectral : Incompressible Flow - Stochastic Projection Method (P + 1) Galerkin-Projected Mom./Cont. Eqns, q = 0,..., P: v q + vv t q = p q + 1 µ[( v) Re + ( v) T ] q v q = 0 Projection: for q = 0,..., P : ṽ q v n q t = C n q + D n q v n+1 q 2 p q = 1 t ṽ q ṽ q = p q t P + 1 decoupled Poisson Eqns for the pressure modes [Le Maître et al., J. Comp. Phys., 2001.]

61 Laminar 2D Channel Flow with Uncertain Viscosity Incompressible flow Gaussian viscosity PDF ν = ν 0 + ν 1 ξ Streamwise velocity v = v i Ψ i i=0 v 0 : mean v i : i-th order mode σ 2 = i=1 v 2 i Ψ 2 i v0 v1 v2 v3 sd v 0 v 1 v 2 v 3 σ

62 Uncertainty Quantification Toolkit (Tk) A library of C++ and Matlab functions for propagation of uncertainty through computational models Mainly relies on Polynomial Chaos Expansions (PCEs) for representing random variables and stochastic processes Target usage: Rapid prototyping Algorithmic research Tutorials / Educational Version 2.0 about to be released under the GNU Lesser General Public License C++ Tools for intrusive and non-intrusive Matlab tools for intrusive and non-intrusive Karhunen-Loève decomposition Bayesian inference tools Downloadable from

63 PCEs in the Toolkit // Initialize PC class int ord = 5; // Order of PCE int dim = 1; // Number of uncertain parameters PCSet mypcset("isp",ord,dim,"lu"); // Legendre-Uniform PCEs // Initialize PC class int ord = 5; // Order of PCE int dim = 1; // Number of uncertain parameters PCSet mypcset("nisp",ord,dim,"lu"); // Legendre-Uniform PCEs Currently support Wiener-Hermite, Legendre-Uniform, and Gamma-Laguerre (limited), Jacobi-Beta (development version) PCSet class initializes PC basis type and pre-computes information needed for working with PC expansions

64 Operations on PCEs in the Toolkit // PC coefficients in double* double* a = new double[npc]; double* b = new double[npc]; double* c = new double[npc]; // PC coefficients in Arrays Array1D<double> aa(npc,0.e0); Array1D<double> ab(npc,0.e0); Array1D<double> ac(npc,0.e0); // Initialization a[0] = 2.0; a[1] = 0.1;... // Perform some arithmetic mypcset.subtract(a,b,c); mypcset.prod(a,b,c); mypcset.exp(a,c); mypcset.log(a,c); // Initialization aa(0) = 2.0; aa(1) = 0.1;... // Perform arithmetic mypcset.subtract(aa,ab,ac); mypcset.prod(aa,ab,ac); mypcset.exp(aa,ac); mypcset.log(aa,ac); PC coefficients are either stored in double* vectors or in more advanced custom Array1D<double> classes Functions can take either data type as argument

65 Surface Reaction Model: Tk implementation // Build du/dt = a*z - c*u - 4.0*d*u*v apcset.multiply(z,a,dummy1); // dummy1 = a*z apcset.multiply(u,c,dummy2); // dummy2 = c*u apcset.subtractinplace(dummy1,dummy2); // dummy1 = a*z - c*u apcset.prod(u,v,dummy2); // dummy2 = u*v apcset.multiplyinplace(dummy2,4.e0*d); // dummy2 = 4.0*d*u*v apcset.subtract(dummy1,dummy2,dudt); // dudt = a*z - c*u - 4.0*d*u All operations are replaced with their equivalent intrusive counterparts Results in a set of coupled ODEs for the PC coefficients u, v, w, z represent vector of PC coefficients This set of equations is integrated to get the evolution of the PC coefficients in time

66 Surface Reaction Model: Second equation implementation // Build dv/dt = 2.0*b*z*z - 4.0*d*u*v apcset.prod(z,z,dummy1); // dummy1 = z*z apcset.prod(dummy1,b,dummy2); // dummy2 = b*z*z apcset.multiply(dummy2,2.e0,dummy1); // dummy1 = 2.0*b*z*z apcset.prod(u,v,dummy2); // dummy2 = u*v apcset.multiplyinplace(dummy2,4.e0*d); // dummy2 = 4.0*d*u*v apcset.subtract(dummy1,dummy2,dvdt); // dvdt = 2.0*b*z*z - 4.0*d*u // Build dw/dt = e*z - f*w apcset.multiply(z,e,dummy1); apcset.multiply(w,f,dummy2); apcset.subtract(dummy1,dummy2,dwdt); // dummy1 = e*z // dummy2 = f*w // dwdt = e*z - f*w Dummy variables used where needed to build the terms in the equations Data structure is currently being enhanced to provide the operation result as the function return value Will allow more elegant inline replacement of operators with their stochastic counterparts

67 Surface Reaction Model: NISP implementation in Tk Quadrature: // Get the quadrature points int nqdpts=mypcset.getnquadpoints(); double* qdpts=new double[nqdpts]; mypcset.getquadpoints(qdpts);... // Evaluate parameter at quad pts for(int i=0;i<nqdpts;i++){ bval[i]=mypcset.evalpc(b,&qdpts[i]); }... // Run model for all samples for(int i=0;i<nqdpts;i++){ u_val[i] =... } // Spectral projection mypcset.galerkprojection(u_val,u); mypcset.galerkprojection(v_val,v); mypcset.galerkprojection(w_val,w); Monte-Carlo Sampling: // Get the sample points int nsamples=1000; Array2D<double> sampts(nsamples,dim); mypcset.drawsamplevar(sampts);... // Evaluate parameter at sample pts for(int i=0;i<nsamples;i++){... // select sampt from sampts bval[i]=mypcset.evalpc(b,&sampt) }... // Run model for all samples for(int i=0;i<nsamples;i++){ u_val[i] =... } // Spectral projection mypcset.galerkprojectionmc(sampts,u_val,u); mypcset.galerkprojectionmc(sampts,v_val,v); mypcset.galerkprojectionmc(sampts,w_val,w);

68 Outline 1 Introduction 2 Forward Propagation of Uncertainty 3 Sensitivity Analysis 4 Sparse Quadrature Approaches for High-Dimensional Systems 5 References

69 Sensitivity Analysis Obtaining global sensitivity analysis from PCEs Identify dominant sources of uncertainty Attribution

70 PC postprocessing: global sensitivity information is readily obtained from PCE g(x 1,..., x d ) = c k Ψ k (x) k=0 Global sensitivity analysis Variance decomposition Total variance Var[g(x)] = k>0 c 2 k Ψ k 2

71 PC postprocessing: global sensitivity information is readily obtained from PCE Main effect sensitivity indices S i = Var[E(g(x x i)] k I = i ck 2 Ψ k 2 Var[g(x)] k>0 c2 k Ψ k 2 I i is the set of bases with only x i involved. S i is the uncertainty contribution that is due to i-th parameter only. Joint sensitivity indices S ij = Var[E(g(x x i, x j )] S i S j = Var[g(x)] k I ij ck 2 Ψ k 2 k>0 c2 k Ψ k 2 I ij is the set of bases with only x i and x j involved. S ij is the uncertainty contribution that is due to (i, j) parameter pair.

72 PC postprocessing: sampling-based approaches g(x 1,..., x d ) = c k Ψ k (x) k=0 In some cases, need to resort to Monte-Carlo estimation, e.g. Piecewise-PC with irregular subdomains Output transformations, e.g. build PC for log g(x), but inquire sensitivity with respect to g(x) A brute-force sampling of Var[E(g(x x i )] is extremely inefficient.

73 PC postprocessing: sampling-based approaches Tricks are available, given a single set of sampled input [Saltelli, 2002]. E.g., use E[g(x x i ) 2 ] = E[g(x x i )g(x x i )] = 1 N 1 N g(x (r) )g( x (r) ), r=1 where x is x with i-th element replaced by x i. Similar formulae available for joint sensitivity indices. Con: as all Monte-Carlo algorithms, converges slowly. Pro: sampling is cheap.

74 Outline 1 Introduction 2 Forward Propagation of Uncertainty 3 Sensitivity Analysis 4 Sparse Quadrature Approaches for High-Dimensional Systems 5 References

75 Sparse Quadrature Approaches for High-Dimensional Systems Need for sparse quadrature Sparse quadrature grids Application to surface reaction example

76 Full product quadrature is wasteful in high-dimensional systems Define the precision as the highest order of a polynomial that is integrated exactly by the quadrature rule. Gaussian quadratures are optimal in 1d N points achieve the highest possible precision of 2N 1. In multi-d, full product quadrature is wasteful: a 5 ppd (point per dimension) rule is of precision P = 9, but it integrates a polynomial x 9 y 9 exactly.

77 Sparse quadrature drastically reduces the number of function evaluations Gauss-Hermite Sparse, level 2, total 21 Full, ppd 7, total 49 Gauss-Legendre Sparse, level 2, total 17 Full, ppd 5, total 25 Sparse grids are built to achieve maximal precision with fewest possible points

78 Clenshaw-Curtis nested quadrature rules allow reuse of function evalutions Clenshaw-Curtis, level 1, total 5

79 Clenshaw-Curtis nested quadrature rules allow reuse of function evalutions Clenshaw-Curtis, level 2, total 13

80 Clenshaw-Curtis nested quadrature rules allow reuse of function evalutions Clenshaw-Curtis, level 3, total 29

81 Clenshaw-Curtis nested quadrature rules allow reuse of function evalutions Clenshaw-Curtis, level 4, total 65

82 Clenshaw-Curtis nested quadrature rules allow reuse of function evalutions Clenshaw-Curtis, level 5, total 145

83 The number of function evaluations is drastically reduced compared to full quadrature Table : The number of Clenshaw-Curtis sparse grid quadrature points for various levels and dimensionalities. Level Precision N N N N(d) L p = 2L 1 (d = 2) (d = 5) (d = 10) General d d + 2d d + 2d d d d d d

84 The number of function evaluations is drastically reduced compared to full quadrature Number of CC sparse grid points, N 1e+06 1e p=7 p=5 p= Dimensionality, d Sparse grid: polynomial growth, O(d p 1 2 ) Number of CC full grid points, N 1e+100 1e+80 1e+60 1e+40 1e+20 p=7 p=5 p= Dimensionality, d Full grid: exponential growth, (p + 1) d

85 Surface Reaction Model 3 ODEs for a monomer (u), dimer (v), and inert species (w) adsorbing onto a surface out of gas phase. du dt = az cu 4duv dv dt = 2bz 2 4duv dw dt = ez fw z = 1 u v w u(0) = v(0) = w(0) = 0.0 a = 1.6 b = c = 0.04 d = 1.0 e = 0.36 f = Species Mass Fractions [-] u v w Time [-] Oscillatory behavior for b [20.2, 21.2] [Vigil et al., Phys. Rev. E., 1996; Makeev et al., J. Chem. Phys., 2002]

86 Surface Reaction Model: 6d results f 10 1 e 2 0 PDF (u ss ) 6 4 d c b a u ss a b c d e f Output observable: time averaged u at steady state u ss Assume all input parameters have Gaussian distributions with σ/µ = 0.01, i.e. 1% deviation. 6-d, level 3 Gauss-Hermite sparse quadrature point set includes 713 distinct points (a 2-d case is plotted) Output PDF generated from 100K samples of 3 rd order PC Variance-based sensitivity information comes for free with the PC expansion

87 Advantages and caveats of sparse quadrature approaches Pro: number of required samples scales much more gracefully with number of dimensions than full tensor product quadrature rule Caveats: Function to be integrated needs to be smooth Due to negative quadrature weights, integrating a noisy positive function can give a negative answer For very high dimensions, even sparse quadrature is too expensive

88 Taking Advantage of Sparsity in the System For really high dimensional systems, even sparse quadrature requires too many function evaluations For 80-dimensional climate land model, L = 4 requires 10 6 points Such systems can only be tackled with dimensionality reduction and/or adaptive order Sensitivity analysis High Dimensional Model Representation (HDMR) Adaptive sparse quadrature approaches More generally, use only the basis terms needed to represent the physics / information in the system / data (Bayesian) Compressive Sensing (CS) approaches If information content is sparse, it can be represented at reasonable cost If not, you need to pay the price

89 Further Reading N. Wiener, Homogeneous Chaos", American Journal of Mathematics, 60:4, pp , M. Rosenblatt, Remarks on a Multivariate Transformation", Ann. Math. Statist., 23:3, pp , R. Ghanem and P. Spanos, Stochastic Finite Elements: a Spectral Approach", Springer, O. Le Maître and O. Knio, Spectral Methods for Uncertainty Quantification with Applications to Computational Fluid Dynamics", Springer, D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach", Princeton U. Press, O.G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann, On the convergence of generalized polynomial chaos expansions, ESAIM: M2AN, 46:2, pp , J. Li and Y. Marzouk, Multivariate semi-parametric density estimation using polynomial chaos expansions, in preparation, D. Xiu and G.E. Karniadakis, The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations", SIAM J. Sci. Comput., 24:2, Le Maître, Ghanem, Knio, and Najm, J. Comp. Phys., 197:28-57 (2004) Le Maître, Najm, Ghanem, and Knio, J. Comp. Phys., 197: (2004) B. Debusschere, H. Najm, P. Pébay, O. Knio, R. Ghanem and O. Le Maître, Numerical Challenges in the Use of Polynomial Chaos Representations for Stochastic Processes", SIAM J. Sci. Comp., 26:2, S. Ji, Y. Xue and L. Carin, Bayesian Compressive Sensing", IEEE Trans. Signal Proc., 56:6, A. Saltelli and S. Tarantola and F. Campolongo and M. Ratto, Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. John Wiley & Sons, K. Sargsyan, B. Debusschere, H. Najm and O. Le Maître, Spectral representation and reduced order modeling of the dynamics of stochastic reaction networks via adaptive data partitioning". SIAM J. Sci. Comp., 31:6, K. Sargsyan, B. Debusschere, H. Najm and Y. Marzouk, Bayesian inference of spectral expansions for predictability assessment in stochastic reaction networks," J. Comp. Theor. Nanosc., 6:10, D.S. Sivia, Data Analysis: A Bayesian Tutorial, Oxford Science, 1996.