Introduction to uncertainty quantification (UQ). With applications.

(UQ). With applications. J. Peter ONERA including contributions of A. Resmini, M. Lazareff, M. Bompard ONERA ANADE Workshop, Cambridge, September 24 th, 2014

OUTLINE Introduction Probability basics and Monte-Carlo Orthogonal/orthonormal polynomials Non-intrusive polynomial chaos methods in 1D Intrusive polynomial chaos method in 1D Extension to high dimensions Examples of application Conclusion

INTRODUCTION Example I : drag minimization Deterministic minimization of airplane cruise shape : define shape That minimizes total drag Cd at M=0.82 Satisfying constraints on lift, pitching moment, inner volume... Actually, variations of cruise flight Mach number Waiting for landing slot Speeding up to cope with pilot maximum flight time Variable Mach number described by D(M) Robust definition of airplane cruise shape Minimize Cd(M)D(M)dM, satisfying constraints for all values of M present in D(M)

INTRODUCTION Example II : fan design Fan operational conditions subject to changes in wind conditions Manufacturing subject to tolerances Robust design accounts for variability of external parameters tolerances for internal parameters Figure 1: Robust design (from cenaero.be)

INTRODUCTION Example III : validation process Unkown data in experiment Upwind Mach number not fully controled in wind tunnels dm = 0.001 Unknown physical constant needed in numerical model Wall roughness constant (milled, brazed, eroded surface...) Discrepancy in a computational/experimental validation process! Compute the mean and standard deviation of the output of interest due to the uncertain inputs

INTRODUCTION Definition of uncertain inputs UNCERTAINTY QUANTIFICATION : describe the stochastic behaviour of OUTPUTS of interest due to uncertain INPUTS Overview of CFD actual uncertain INPUTS : Geometrical (manufacturing tolerance) Operational: flow at boundaries (far field, injection...) Reference: Proceedings of RTO-MP-AVT-147 Evans T.P., Tattersall P. and Doherty J.J.: Identification and quantification of uncertainty sources in aircraft related CFD-computations - An industrial perspective Stochastic behaviour of OUTPUTS. Presently, most often, mean and variance are calculated

INTRODUCTION Three issues in (UQ) 1 terminology Lack of agreement on the definition of error, uncertainty... AIAA Guide G-077-1998 Uncertainty is a potential deficiency in any phase are activity of the modeling process that is due to the lack of knowledge. Error is a recognizable deficiency in any phase or activity of the modelling process that is not due to the lack of knowledge ASME Guide V& V 20 (in its simpler version adopted for the Lisbon Workshops on CFD uncertainty). The validation comparison error is defined as the difference between the simulation value and the experimental data value. It is split in numerical, model, input and data errors (assumed to be independant). Numerical (resp. input, model, data) uncertainty is a bound of the absolute value of numerical (resp. input, model, data) error

INTRODUCTION Three issues in (UQ) 2 (UQ) validation and verification (UQ) CFD-based exercise leads to a standard deviation on some outputs Compare this standard deviation to the numerical error Richardson method, GCI... Pierce et al. Venditti et al. adjoint based formulas for functional outputs Compare this standard deviation to the modeling error Run several (RANS) models Run better models than (RANS) Numerical (UQ) investigation only make sense if standard deviation due to uncertain inputs not much smaller than modelling or discretization error

INTRODUCTION Three issues in (UQ) 3 lack of shared well-defined problems Most often mean and variance of some outputs are computed in (UQ) exercises as this is feasible with usual methods? as this is actually required for applications? from EDF s specialits point of view, three stochastic criteria of interest: central dispersion (mean and variance of output) range (min and max possible values of output) probability of exceeding a threshold Get information from industry in order to define relevant (UQ) exercises Share mathematical industrial test cases and mathematical exercices to split the CFD influence / the one of (UQ) method

INTRODUCTION Intrusive vs non-intrusive methods Non-intrusive methods. No change in the analysis code Post-processing of deterministic simulations Intrusive methods. Changes in the analysis code Stochastic expansion of state/primitive variables Galerkin projections Probably not feasible for large industrial codes

MONTE-CARLO Basics of probability (1) A classical introduction to probability basics involves a sample space Ω (dice values, interval of Mach number values) events ξ i.e. element of Ω or set of elements of Ω (even dice values, subinterval of Mach number values) event space A, set of subsets of Ω, stable by union, intersection, including null set and Ω (also called σ-algebra) [INPUT] a probability function P on A such that P (Ω) = 1,P ( ) = 0, plus natural properties for complementary parts and union of disjoint parts (then denoted D keeping P for polynomials) [OUPUT] random variables X depending of the event ξ (like CDp or CLp of an airfoil depending on the far-field Mach number through Navier-Stokes equations )

MONTE-CARLO Basics of probability (2) Discrete example : regular 6-face Dice thrown once events ξ = 1,2,3,4,5 or 6 sample space Ω ={1,2,3,4,5,6} probability space Ω = null set plus all discrete sets of these numbers {, {1}, {2}, {3}, {4}, {5}, {6}, {1, 2}, {1, 3}, {1, 4}, {1, 5}, {1, 6}, {2, 3}...{1, 2, 3, 4, 5, 6}} probability function P (later denoted D) : P ( ) = 0, P ({1}) = 1./6., P ({2}) = 1./6.,... P ({1, 2}) = 1./3., P ({1, 3}) = 1./3, P ({1, 4}) = 1./3... P ({1, 2, 3, 4, 5, 6}) = 1. random variables X, for example dice value to the power three...

MONTE-CARLO Basics of probability (3) Discrete example : Mach number in [0.81,0.85] events ξ = a Mach number value in [0.81,0.85] sample space Ω = [0.81,0.85] probability space Ω = all subparts of [0.81,0.85] probability function P to be defined (later denoted D). For example from a beta distribution with power three : P φ (φ) = 35 32 (1. φ2 ) 3 φ [ 1, 1] φ = (ξ 0.83)/0.02 P (ξ) = 1 0.02 P φ(φ) = 1 0.02 35 0.83 (1. (ξ 32 0.02 )2 ) 3 random variables X, aerodynamic pitching moment of a wing with variable Mach number M ( event ξ) in the farfield

MONTE-CARLO Set of probability density functions of β distributions (on [0,+1] with the α 1 β 1 convention for exponants)

MONTE-CARLO Basics of probability (4) Allows to define probablities for both discrete and continuous spaces Quantities of interest = functions of the stochastic variables random variables Example = pitching moment of wing J (random variable) depending via NS equations of uncertain far-field Mach number (uncertain input/event) Mean Variance Z E(J ) = J (ξ)d(ξ)dξ Ω Z V ar(j ) = (J (ξ) E(J )) 2 D(ξ)dξ Ω Covariance of two random variables Cov(J, G) = Z Ω (J (ξ) E(J ))(G(ξ) E(G))D(ξ)dξ

MONTE-CARLO Basics Monte-Carlo mimics the law of the event in a series of calculations Reference method for all uncertainty propagation methods Generation of a sampling (ξ 1, ξ 2..., ξ p..., ξ N ) of the p.d.f D(ξ) Example : sampling for normal distribution. a k, b k indep. uniformly distributed in [0.,1.] ξ k = 2 ln(a k )cos(2πb k ) Computation of corresponding flow fields W (ξ p ), p [1, N] Computation of functional outputs J (ξ p ) = J(W (ξ p ), X(ξ p )) Estimation of following statistical quantities : E(J ) = Z J (ξ)d(ξ)dξ J N = 1 N p=n X J (ξ p ) p=1 σ 2 J = E((J E(J ))2 ) σ 2 J N = 1 N 1 Need to quantify accuracy of estimation p=n X p=1 (J (ξ p ) J N ) 2

MONTE-CARLO Accuracy of estimation: mean (1) Scalar case. variance σ J is known Reminders about N (0, 1) N J N E(J ) σ J N (0, 1) (Normal distribution) Probability density function (p.d.f.) of N (0, 1)- D N (x) = 1 2Π e x2 2 Symmetric cumulative distribution function - Φ N (x) = 1 2Π x x e t2 2 dt With ɛ confidence : E(J ) [ J uɛ σ N u J ɛ N, JN σ + u J ɛ N ] ɛ = 1 2Π e t 2 2 dt u ɛ ɛ 0.5 0.9 0.95 0.99 u ɛ 0.68 1.65 1.96 2.58 With 99% confidence : 2.58 E(J ) [ J N 2.58 σ J N, JN + 2.58 σ J N ] (0.99 = 1 2Π 2.58 e t2 2 dt)

MONTE-CARLO Accuracy of estimation: mean (2) Scalar case: variance σ J is unknown - N J N E(J ) σ JN S(N 1) (Student dist.) With ɛ confidence : E(J ) [ J N u ɛ(n 1) σ JN N, JN + u ɛ(n 1) σ JN N ] u ɛn as function of ɛ and N found in tables Student distribution converges to Normal distribution for large N Tables for u ɛn 1 ɛ N 2 3 20 30 0.95 12.7 4.30 2.09 2.04 1.96 0.99 63.7 9.93 2.86 2.75 2.58 Figure 3: Value of u ɛ(n 1) for Student distribution S(N 1) N 2

MONTE-CARLO Accuracy of estimation: mean (3) Scalar case: variance σ J is unknown - N J N E(J ) σ JN With ɛ confidence : E(J ) [ J N u ɛ(n 1) σ JN N, JN + u ɛ(n 1) σ JN N ] u ɛn u ɛn as function of ɛ and N found in tables converges towards u ɛ of normal law for large N S(N 1) (Student dist.) Density function of Student distribution S(N). D S(N) (x) = N+1 Γ( 2 ) N+1 x2 (1 + Γ( N 2 ) ) 2 NΠ N Definition of u ɛn. ɛ = Z uɛn u ɛn D S(N) (t)dt (ɛ ]0, 1.[)

MONTE-CARLO Accuracy of estimation: variance (1) (skpd) Scalar case: mean E J is known Notations SJ 2 N = 1 N Chi-square χ 2 N i=n i=1 (J (ξp ) E J ) 2 probability distribution defined on [0, [ with p.d.f. : D χ 2 N (x) = 1 Γ(N/2)2 N/2 x N/2 1 e x/2 Chi-square cumulative d.f. : Φ χ 2 N (x) = x 0 D χ 2 N (t)dt Stochastic variable N S2 J N σ 2 J χ 2 N

MONTE-CARLO Chi-square probabilistic density functions D χ 2 N and cumulative density functions Φ χ 2 N (skpd)

MONTE-CARLO Accuracy of estimation: variance (2) (skpd) Scalar case: mean E(J ) is known - N S2 J N σ 2 J With ɛ = 1 α confidence : Φ 1 ( α χ 2 2 ) N S2 J N N σj 2 With ɛ = 1 α confidence : σ 2 J [N S2 J N Φ 1 χ 2 N χ 2 N Φ 1 χ 2 N (1. α 2 ) 2 J N Φ 1 χ 2 N (1 α 2 ), N S ( α 2 )] x N 2 20 30 0.005 10.597 39.997 53.672 0.995 0.0100 7.434 13.787 Figure 4: Value of Φ 1 χ 2 N (x)

MONTE-CARLO Quality of estimation: variance - 99 % confidence Application N = 2 σ 2 J [0.189 S2 J 2, 200 S 2 J 2 ] N = 20 σ 2 J [0.500 S2 J 20, 2.69 S 2 J 20 ] N = 30 σ 2 J [0.558 S2 J 30, 2.17 S 2 J 30 ] N = 100 σ 2 J [0.713 S2 J 100, 1.49 S 2 J 100 ] Convergence speed of bounds towards 1. The cumulative distribution of the Chi-Square law Φ N (x) can be expressed as Φ χ x/2 1 2 N (x) = Γ(N/2) t N/2 e t γ(n/2, x/2) dt = (γ Γ(N/2) lower incomplete Γ function) Check properties of (the inverse of) Φ χ 2 N Check convergence speed of N/Φ 1 (1 α χ 2 2 ) and N/Φ 1 ( α N χ 2 2 ) N 0

MONTE-CARLO Accuracy of estimation: variance (3) (skpd) Scalar case: mean E(J ) is unknown - Stochastic variable (N 1) σ2 J N σ 2 J With ɛ = (1 α) confidence : σ 2 J [(N 1) σ 2 J N Φ 1 χ 2 N 1 (1 α ), (N 1) σ 2 Φ 1 2 J N χ 2 N 1 ( α 2 )] χ 2 N 1 x N 3 4 20 30 0.005 10.597 12.838 38.582 52.336 0.995 0.0100 0.0717 6.844 13.121 Figure 5: Value of Φ 1 χ 2 N 1(x)

MONTE-CARLO Restriction for Monte-Carlo method. Meta-models Convergence speed of Monte-Carlo for mean value estimation is 1 N Increasing precision of Monte-Carlo estimation by a factor of 10 requires multiplying the number of evaluations by a factor of 100 Not usable in the context of complex aeronautics simulation Cost of one evaluation very high (requires resolution of (RANS) equations) Aerodynamic function relatively smooth replace flow field / aerodynamic function by a model built with a database of flow fields / function values

Figure 6: Monte-Carlo method with meta-models Introduction to uncertainty quantification METAMODEL BASED MONTE-CARLO Use meta-models with Monte-Carlo method Monte Carlo (large) stochastic sampling Sampling and m.m. outputs Meta model (small) Sampling Sampling and outputs Build meta model (optimize inner parameters..) Meta model CFD code Compute flow Compute functional outputs Evaluate meta model based mean, variance,...

METAMODEL BASED MONTE-CARLO Meta-models Most often, approximation of a function of interest Meta-models used at ONERA for CFD: Kriging, Radial Basis Function, Support Vector Regression Other meta-models of specific interest for UQ: Adjoint based linear or quadratic Taylor expansion Discussed in more details in the extension to multi-dimension section Influence of meta-model accuracy on mean and variance accuracy?

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Presentation Search confidence interval on aerodynamic function C L with uncertainty on AoA Nominal configuration: NACA0012, M = 0.73, Re = 6M, AoA = 3 elsa (a) (RANS+(k-w) Wilcox turbulence model) solver (Roe flux+van Albada lim.) Figure 7: Mesh (a) The elsa CFD software: input from research and feedback from industry Mechanics and Industry 14(3) L. Cambier, S. Heib, S. Plot

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Distribution of uncertainty Beta distribution (parameters (2.,2.)) [pdfb(ξ) = 15 16 (1 ξ)2 (1 + ξ) 2 ] over [-1,1] p.d.f of angle of attack AoA [pdfa(α) = pdfb(10.(α 3.))] over [2.9,3.1] Figure 8: Beta distribution of AoA

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method: application on C L (1) 0.498 0.496 Estimated mean value 90 percent 95 percent 99 percent Best MC with MM value 0.494 0.492 0.49 0.488 0.486 0.484 1 10 100 1000 Figure 9: Mean of C L coefficient and confidence interval

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method: application on C L (2) 0.0002 0.00018 0.00016 Estimated variance value 90 percent 95 percent 99 percent Best MC with MM value 0.00014 0.00012 0.0001 8e-05 6e-05 4e-05 2e-05 0 1 10 100 1000 Figure 10: Variance of C L coefficient and confidence interval

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method with meta-models: learning sample Use learning sample based on Tchebyshev polynomials 1 0 1 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 Figure 11: Tchebychev distribution (11 points)

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method with meta-models: reconstruction of C L 0.505 0.5 sample11.dat outputok.dat outputrbf.dat outputsvr.dat 0.495 0.49 0.485 0.48 0.475 0.79 0.792 0.794 0.796 0.798 0.8 0.802 0.804 0.806 0.808 0.81 Figure 12: C L

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method with meta-models: application on C L (1) 0.498 0.496 Estimated mean value 90 percent 95 percent 99 percent Best MC with MM value 0.494 0.492 0.49 0.488 0.486 0.484 1 10 100 1000 10000 100000 1e+06 1e+07 Figure 13: Mean of C L coefficient and confidence interval

APPLICATION OF (METAMODEL BASED) MONTE-CARLO Monte-Carlo method with meta-models: application on C L (2) 0.0002 0.00018 0.00016 Estimated variance value 90 percent 95 percent 99 percent Best MC with MM value 0.00014 0.00012 0.0001 8e-05 6e-05 4e-05 2e-05 0 1 10 100 1000 10000 100000 1e+06 1e+07 Figure 14: Variance of C L coefficient and confidence interval

ORTHOGONAL POLYNOMIALS Orthogonal/orthonormal polynomials Polynomial/spectral expansion and stochastic post-processing Polynomials orthogonal for the product defined by p.d.f D(ξ) : = P l (ξ)p m (ξ)d(ξ)dξ = δ lm Basic references Orthogonal polynomials Abramowitz and Stegun: Handbook of Mathematical functions. (1972). Chapter 22 Spectral expansions J. P. Boyd: Chebyshef and Fourier spectral methods (2001) Expand fields on this basis In non-intrusive PC method fields of interest In intrusive PC method state/primitive variables

ORTHOGONAL POLYNOMIALS Orthonormal polynomials (1/3) Families of polynomials Normal distribution D n (ξ) = 1 2Π e ξ2 2 on R Hermitte polynomials Gamma distribution D g (ξ) = exp( ξ) on R + Laguerre polynomials Uniform distribution D u (ξ) = 0.5 on [ 1, 1] Legendre polynomials Chebyshev distribution D cf (ξ) = 1/Π/ 1 ξ 2 on [ 1, 1] Chebyshev (first-kind) polynomials Chebyshev distribution D cs (ξ) = 1 ξ 2 on [ 1, 1] Chebyshev (second-kind) polynomials Beta distribut. D β (ξ) = (1 ξ) α (1 + ξ) β / 1 1 (1 u)α (1 + u) β du α > 1., β > 1. on [ 1, +1] Jacobi polynomials (incl. Chebyshev polynomials) Non-usual probabilistic density functions, D l (ξ) computed by Gram- Schmidt orthogonalisation process.

ORTHOGONAL POLYNOMIALS Orthonormal polynomials (2/3) Example : 1D problem. Hermite polynomials for normal law Probability density function (p.d.f.) of D n (ξ) = 1 2Π e ξ2 2 Family of orthonormal polynomials for < f, g >= R + f(ξ)g(ξ)d n(ξ)dξ P H 0 (ξ) = 1 P H 1 (ξ) = ξ P H 2 (ξ) = ξ 2 1 P H 3 (ξ) = ξ 3 3ξ P H 4 (ξ) = ξ 4 6ξ 2 + 3 Normalization of Hermite polynomials P H j (ξ) = 1 j!(2π) (1/4) P H j (ξ) Orthonormality relation for P H j : = + P H j(ξ)p H k (ξ)d n (ξ)dξ = δ jk

ORTHOGONAL POLYNOMIALS Orthonormal polynomials (3/3) Example : 1D problem on [-1,1]. First-kind Chebyshef polynomials 1 1 ξ 2 Probability density function (p.d.f.) of D cf (ξ) = 1 Π Family of orthonormal polynomials for < f, g >= R 1 1 f(t)g(t)d cf (t)dt T 0 (ξ) = 1 T 1 (ξ) = ξ T 2 (ξ) = 2ξ 2 1 T 3 (ξ) = 4ξ 3 3ξ T 4 (ξ) = 8ξ 4 8ξ 2 + 1 Normalization of Chebyshev polynomials T 0 = T 0... T n = 2 T n (n 1) Orthonormality relation for T j : < T j, T k >= + T j(ξ)t k (ξ)d cf (ξ)dξ = δ jk specific property T n (cos(θ)) = cos(nθ) (hence T n 1. )

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Basics Polynomial/spectral expansion and stochastic post-processing Polynomials orthogonal for the product defined by p.d.f D(ξ) : = P l (ξ)p m (ξ)d(ξ)dξ = δ lm Expand field of in interest (entire flow field, flow field at the wall...). i is the discrete space variable (mesh index) Spectral expansion : W (i, ξ) P CW (i, ξ) = l=m 1 l=0 C l (i)p l (ξ) Stochastic post-processing for P CW instead of W References [Wiener 1938][Ghanem et al. 1991]

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Stochastic post-processing Expansion of flow field on structured mesh depending on stochastic variable ξ W (i, ξ) P CW (i, ξ) = l=m 1 l=0 C l (i)p l (ξ) Stochastic post-processing (mean and variance) done for the expansion P CW instead of W straightforward evaluation of mean value E(P CW (i, ξ)) = l=m 1 l=0 C l (i)p l (ξ)d(ξ)dξ = C 0 (i) E(P CW (i, ξ)) == C 0 (i) straightforward evaluation of variance E((P CW (i, ξ) C 0 (i)) 2 ) = ( l=m 1 l=1 C l (i)p l (ξ)) 2 D(ξ)dξ E((P CW (i, ξ) C 0 (i)) 2 ) = l=m 1 l=1 C l (i) 2

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Accuracy of truncated spectral extension (1/3) General idea : the highest the regularity of the function of interest the fastest the convergence of the truncated spectral expansion Discussed for Chebyshef first-kind polynomials Classical expension of a continuous fonction over [-1,1] f(ξ) = n=0 C nt n (ξ) Definition of coefficients C 0 = 1 +1 1 Π 1 f(ξ)t 1 ξ 2 o(ξ)dξ C n = 2 +1 Π 1 1 f(ξ)t 1 ξ 2 n(ξ)dξ (n > 0) Convergence speed of truncated series f(ξ) = P N n=0 C nt n (ξ)?

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Accuracy of truncated spectral extension (2/3) Cos Fourier series of g periodic over [ Π, +Π] : g(θ) = P n=0 A ncos(nθ) R A 0 = 1 +Π f(θ)dθ A R 2Π Π n = 1 +1 g(θ)cos(nθ)dθ Π 1 Integration by part of A n for a C k function A n = 1 nπ A n = 1 n 2 Π... A n = 1 n k Π +1 1 g (θ)sin(nθ)dθ +1 1 g(2) (θ)cos(nθ)dθ +1 1 g(k) (θ)cos(nθ + kπ/2)dθ Upper bound of the coefficients for a C k function A n 1 n k Π +1 1 g(k) (θ) dθ 2 n k g (k) Upper bound of the residual g(θ) N n=0 A ncos(nθ) N+1 A n...... 2 g (k) 1 du 2 g(k) N u k (k 1)N k 1

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Accuracy of truncated spectral extension (3/3) Chebyshev expansion of a C k function f(ξ) over [-1,1] f(ξ) = k=0 A kt k (ξ) Change variable: ξ = cos(θ), g = f cos is C k over [ Π, Π] g(θ) = f(cosθ) = k=0 A kt k (cos(θ)) g(θ) = k=0 A kcos(k θ) is a Fourier series!!! All results concerning Fourier series have a counterpart for Tchebyshev expansions Express g (k) from f (k)... f Express the upper bound of A k, use T k = 1. Find the upper bound of truncation error of spectral expansion of f Extension to f(i, ξ) = P k=0 A k(i)t k (ξ)

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Coefficient computations (1/4) Expansion of flow field on structured mesh depending on stochastic variable ξ W (i, ξ) P CW (i, ξ) = l=m 1 l=0 C l (i)p l (ξ) Straightforward stochastic post-processing for mean and variance (presented before) Accuracy of ideal P CW depending on degree and regularity - theory of spectral expansions (just discussed) Computation of C l coefficients? Accuracy of actual P CW expansion? (articles on this topic?)

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Coefficients computation - Gaussian quadrature (2/4) Expansion of flow field on structured mesh depending on stochastic variable ξ W (i, ξ) P CW (i, ξ) = l=m 1 l=0 C l (i)p l (ξ) From orthonormality property C l (i) = Under regularity assumptions C l (i) =< W, P l > Gaussian quadrature C l (i) ==< W, P l >= W (i, ξ)p l (ξ)d(ξ)dξ Computation by Gaussian quadrature associated to D with G points. f(ξ)d(ξ)dξ k=g k=1 w kf(ξ k ) (w k,ξ k ) depend on D(ξ) Exact quadrature if f(ξ) is a polynomial of degree (2G 1). Polynomial chaos : C l (i) exact if W (i, ξ)p l (ξ) polynomial of ξ of degree lower than/equal to (2G 1).

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Coefficients computation - collocation (3/4) Other way : collocation - least-square collocation Identify W (i, ξ l ) and P CW (i, ξ l ) for M values of ξ. Collocation W (i, ξ k ) = l=m 1 l=0 C l (i)p l (ξ k ) k {1, M} solved for C l (i) Solve least-square problem for more values of W (i, ξ l ) than coefficients in the expansion Less general accuracy results than for Gauss quadrature

GENERALIZED POLYNOMIAL CHAOS METHOD (N.-I.) Coefficients computation - collocation (4/4) Theorical result for Chebyshev series (see Boyd pages 95-96). Expansion of a C k function f(ξ) over [-1,1] f(ξ) = P k=0 A kt k (ξ) Collocation at x k = cos( kπ M 1 ) k [0, M 1] (Chebyshev extrema) obtain P M 1 (ξ) = M 1 k=0 Ā k T k (ξ) Collocation at x k = cos( (2k+1)Π 2M ) k [0, M 1] (Chebyshev roots) compute Q M 1 (ξ) = M 1 k=0 Ã k T k (ξ) Upper bounds of truncation error f(ξ) N n=0 ĀnT n (ξ) 2 N+1 A n... f(ξ) N n=0 ÃnT n (ξ) 2 N+1 A n... Factor TWO w.r.t. truncation of exact series

PROBABILISTIC COLLOCATION (N.-I.) Basics (1/2) Another approach for non-intrusive polynomial chaos based on Lagrangian polynomial expansion. [Tatang 1995] [Xiu et al. 2005] [Loeven et al. 2007] for compressible CFD Presented for 1D problem ( i is the discrete space variable mesh index) W (i, ξ) SCW (i, ξ) = l=n l=1 W l(i)h l (ξ) H l (ξ) = Π m=n (ξ ξ m ) m=1,m<>l ( polynomial of degree (N 1) ) (ξ l ξ m ) Note that actually SCW (i, ξ l ) = W l (i). no coefficient computation, define W l (i) = W (i, ξ l ) Definition of (ξ 1, ξ 2,..., ξ N ) (most often, not absolutely necessary) N points of the N-point quadrature associated to D(ξ) m=n f(ξ)d(ξ)dξ m=1 ω mf(ξ m )

PROBABILISTIC COLLOCATION (N.-I.) Basics (2/2) Expansion of flow field depending on stochastic variable ξ Stochastic post-processing (mean and variance) done for the expansion SCW instead of W straightforward evaluation of mean value E(SCW (i, ξ)) =< SCW (i, ξ), 1 >= m=n m=1 ω mw m (i) straightforward evaluation of variance E((SCW (i, ξ) E(SCW (i))) 2 ) = m=n m=1 ω mw m (i) 2 ( m=n m=1 ω mw m (i)) 2 Both exact for SCW as N -point Gaussian quadrature is exact for polynomial of degree up to 2N 1

PROBABILISTIC COLLOCATION (N.-I.) Link between polynomial chaos and probabilistic collocation Two polynomial expansions Case when the methods merge same degree (M 1) for both expansions coefficents of chaos expansion computed by collocation same set of M collocation points polynomials of degree (M 1) having same values in M distinct abscissas SCW = P CW (expressed in two different basis) exact evaluation of mean and variance for and SCW and P CW same evaluation of mean and variance by the two methods

INTRUSIVE POLYNOMIAL CHAOS References A stochastic projection method for fluid flow. I Basic formulation. Le Maître OP, Knio OM, Najm, HN, Ghanem RG. JCP 173 (2001) A stochastic projection method for fluid flow. II Random process. Le Maître OP, Reagan MT, Najm HN, Ghanem RG, Knio OM. JCP 181 (2002) Intrusive polynomial chaos method for the compressible Navier Stokes equations. Chris Lacor. Slides presented at ONERA Scientific Day on error approximation and uncertainty quantification. 3/12/2010 (available on ONERA s web site) Comparison and intrusive and non-intrusive polynomial chaos methods for CFD applications in aeronautics. Onorato G, Loeven GJA, Ghobarnias G, Bijl H, Lacor C. Proceedings of ECCOMAS CFD 2010 [Application] Assessment of intrusive and non-intrusive non-deterministic CFD methodologies based on polynomial chaos expansions. Dinsecu C, Smirnov S, Hirsch C, Lacor C. IJESMS 2 (2010)

INTRUSIVE POLYNOMIAL CHAOS Principle Expand primitive variables (not state-variables! unless you get ratio of polynomial expansions) on PC basis U(x, t, ξ) = l=m 1 l=0 U l (x, t)p l (ξ) Input expansions in stochastic partial differential equations Do a Galerkin projection for all P l (l [0, M 1]) : Multiply all equations by P l (ξ) Apply the mean (in other words, multiply all equations by P l (ξ) D(ξ) sum over ξ domain) Discretize resulting system of coupled partial differential equations Stochastic post-processing

INTRUSIVE POLYNOMIAL CHAOS Example 1 (1/3) Steady incompressible flow. Continuity equation One uncertain parameter (angle of attack or...) following D cf (ξ) = 1 Π Order 2 expansion of primitive variables U(x, ξ) = l=2 l=0 U l(x)t l (ξ) p(x, ξ) = l=2 l=0 p l(x)t l (ξ) 1 1 ξ 2 reminder T 0 (ξ) = 1 T 1 (ξ) = 2ξ T 2 (ξ) = 2(2ξ 2 1) Continuity equation div (U(x, ξ)) = div ( l=2 ) l=0 U l(x)t l (ξ) div (U(x, ξ)) = l=2 l=0 div(u l(x))t l (ξ)

INTRUSIVE POLYNOMIAL CHAOS Example 1 (2/3) Steady incompressible flow. Continuity equation. One uncertain parameter (angle of attack or...) following D cf (ξ) = 1 Π 1 1 ξ 2 Continuity equation div (U(x, ξ)) = l=2 l=0 div(u l(x))t l (ξ) = 0 Galerkin projection for T 0 (ξ),t 1 (ξ) and T 2 (ξ) (For T 1 for example) div (U(x, ξ)) T 1 (ξ)d cf (ξ) = P l=2 l=0 div(u l(x))t l (ξ)t 1 (ξ)d cf (ξ) = 0 A = R 1 1 div (U(x, ξ)))t 1(ξ)D cf (ξ) = 0

INTRUSIVE POLYNOMIAL CHAOS Example 1 (3/3) Steady incompressible flow. Continuity equation. One uncertain parameter (angle of attack or...) following D cf (ξ) = 1 Π 1 1 ξ 2 Continuity equation div (U(x, ξ)) = l=2 l=0 div(u l(x))t l (ξ) = 0 Galerkin projection for T 0 (ξ),t 1 (ξ) and T 2 (ξ) (For T 1 for example) A = R 1 P l=2 1 l=0 div(u l(x))t l (ξ)t 1 (ξ)d cf (ξ)dξ = 0 A = P l=2 l=0 div(u l(x)) R 1 1 T l(ξ)t 1 (ξ)d cf (ξ)dξ = div(u 1 (x)) = 0 Using orthonormality property : div(u 1 (x)) = 0 (of course div(u 0 (x)) and div(u 2 (x)) are also proved to be equal to zero using corresponding Galerkin projection)

INTRUSIVE POLYNOMIAL CHAOS Example 2 (1/3) Unsteady incompressible flow. Euler equations. Momentum equation. One uncertain parameter following D cf (ξ) = 1 Π 1 1 ξ 2 Order 2 expansion of primitive variables ( final number of equations multiplied by three) U(x, t, ξ) = l=2 l=0 U l(x, t)t l (ξ) p(x, t, ξ) = l=2 l=0 p l(x, t)t l (ξ) Momentum equation U t + (U. )U + p = 0 t (P l=2 l=0 U l(x, t)t l (ξ))+ P l=2 l=0 P n=2 n=0 (U l(x, t). )U n (x, t)t l (ξ)t n (ξ)+ P l=2 l=0 p l(x, t)t l (ξ) = 0

INTRUSIVE POLYNOMIAL CHAOS Example 2 (2/3) Steady incompressible flow. Euler equations. Momentum equation. One uncertain parameter following D cf (ξ) = 1 Π Momentum equation t (P l=2 + P l=2 l=0 1 1 ξ 2 l=0 U l(x, t)t l (ξ)) P n=2 n=0 (U l(x, t). )U n (x, t)t l (ξ)t n (ξ) + P l=2 l=0 p l(x, t)t l (ξ) = 0 Multiplying by T k k [0, 2] and taking the expected value for D cf t (P l=2 + P l=2 l=0 l=0 U l(x, t) < T l T k > P n=2 n=0 (U l(x, t). )U n (x, t) < T l T n T k > + P l=2 l=0 p l(x, t) < T l T k >= 0

INTRUSIVE POLYNOMIAL CHAOS Example 2 (3/3) Steady incompressible flow. Euler equations. Momentum equation. One uncertain parameter following D cf (ξ) = 1 Π Momentum equation t (P l=2 + P l=2 l=0 1 1 ξ 2 l=0 U l(x, t) < T l T k >) P n=2 n=0 (U l(x, t). )U n (x) < T l T n T k > + P l=2 l=0 p l(x, t) < T l T k >= 0 Using orthogonality property t U k(x, t) + P l,n=2 l,n=0 (U l(x, t). )U n (x, t) < T l T n T k > + p k (x, t) = 0 Set of p.d.e for (p 0, p 1, p 2 ) (U 0, U 1, U 2 ). Coupling through momentum equation

INTRUSIVE POLYNOMIAL CHAOS Stochastic post-processing Expand primitive variables on PC basis U(x, t, ξ) = l=m 1 l=0 U l (x, t)p l (ξ) Straightforward post-processing at continuous level E(U(x, t, ξ)) = U 0 (x, t) E((U(x, t, ξ) U 0 (x, t)) 2 ) = l=m 1 l=1 U l (x, t) 2 Actual stochastic post-processing at discrete level Use corresponding formulas at discrete level Derive mean and variance of (linearized) outputs of interest

INTRUSIVE POLYNOMIAL CHAOS Compressible Euler equations Including several technical details Pseudo spectral approach for cubic terms Truncation of higher order terms Three applications Feasibility for large codes?

EXTENSION TO HIGH DIMENSIONS Basics of multidimensional probability (1) Several R values stochastic variables a sample space Ω R d events i.e. element of Ω or set of elements of Ω elementary event ξ R d valued vector event space A, set of subsets of Ω, stable by union, intersection, including null set and Ω (also called σ-algebra) For the sake of simplicity Ω R 2 Define, cumulative distribution of (ξ 1, ξ 2 ), probabilty density function, marginal distributions, independance, covariance

EXTENSION TO HIGH DIMENSIONS Basics of multidimensional probability (2) Cumulative density function Φ(u 1, u 2 ) = P ({ξ 1 u 1 } {ξ 2 u 2 }) Probabilistic density function D ξ (ξ 1, ξ 2 ) = 2 Φ ξ x y (ξ 1, ξ 2 ) probability of ξ [ξ 1 dx/2, ξ 1 + dx/2][ξ 2 dy/2, ξ 2 + dy/2] is D ξ (ξ 1, ξ 2 )dxdy

EXTENSION TO HIGH DIMENSIONS Basics of multidimensional probability (3) Marginal distribution w.r.t. the two variables Φ 1 (ξ 1 ) = Φ 2 (ξ 2 ) = Z ξ1 Z + Z + Z ξ2 «D ξ (x, y)dy dx «D ξ (x, y)dx dy Marginal probabilistic density function D 1 (ξ 1 ) = Z + D ξ (ξ 1, y)dy D 2 (ξ 2 ) = Z + D ξ (x, ξ 2 )dx

EXTENSION TO HIGH DIMENSIONS Basics of multidimensional probability (4) Independant ξ 1 and ξ 2 = one variable provides no information about the other : φ ξ (ξ 1, ξ 2 ) = Φ 1 (ξ 1 )Φ 2 (ξ 2 ) D ξ (ξ 1, ξ 2 ) = D 1 (ξ 1 )D 2 (ξ 2 ) Null covariance is ensured for independant stochastic variable. Null covariance is (by far) a less stronger property than independance of stochatic variables Example : independant variables ξ 1 ξ 2 on [ 1, +1] 2 : D ξ (ξ 1, ξ 2 ) = D ξ (ξ 1, ξ 2 ) = 35 32 (1. ξ2 1) 3 15 16 (1. ξ2 2) 2

EXTENSION TO HIGH DIMENSIONS Basics of multidimensional probability (5) Example : independant variables ξ 1 ξ 2 on [ 1, +1] 2 : D ξ (ξ 1, ξ 2 ) = D ξ (ξ 1, ξ 2 ) = 35 32 (1. ξ2 1) 3 15 16 (1. ξ2 2) 2 Example = dependant variables ξ 1 ξ 2 on [ 1, +1] 2 : D a ξ (ξ 1, ξ 2 ) = D ξ (ξ 1, ξ 2 ) = 3/4 (1 1/4 (ξ 1 ξ 2 ) 2 ) D b ξ(ξ 1, ξ 2 ) = D ξ (ξ 1, ξ 2 ) = 3/4 (1 1/4 (ξ 1 + ξ 2 ) 2 ) Exercise = calculate all previous quantities : Dξ a 1 (ξ 1 ) = 3/4((5/6 1/2ξ1) 2 Dξ a 2 (ξ 2 ) = 3/4(5/6 1/2ξ2) 2 Dξ b 1 (ξ 1 ) = 3/4((5/6 1/2ξ1) 2 Dξ b 2 (ξ 2 ) = 3/4(5/6 1/2ξ2) 2 E a (ξ 1 ) = 0 E a (ξ 2 ) = 0 E b (ξ 1 ) = 0 E b (ξ 2 ) = 0 cov a (ξ 1, ξ 2 ) = 1/6 cov b (ξ 1, ξ 2 ) = 1/6

EXTENSION TO HIGH DIMENSIONS Changes from 1D to dimension d Several uncertainties considered simultaneously ξ is a vector ξ = (ξ 1, ξ 2.., ξ d ) Caution : subscript = components <> exponent = number in a sampling What does not change: Monte-Carlo method What does not change very much: Monte-Carlo plus metamodel Polynomial chaos method for independant uncertain ξ k and tensorial basis of polynomials. Size of polynomial basis (collocation grid) G d What does change: Polynomial chaos method for dependant uncertain ξ k or degree-m multivariate polynomials

EXTENSION TO HIGH DIMENSIONS Monte-Carlo method R d Basic Monte-Carlo No difference between R and R d Monte-Carlo plus Kriging, RBF, SVM/SVR... Number of inner parameters of the meta-model prop. to d Cost of the definition of an accurate meta-model increasing with d Monte-Carlo plus adjoint-based first or second order Taylor expansion First order : solve one adjoint equation to compute first-order derivative Second order : solve d direct equations plus one adjoint equation to compute first- and second-order derivatives

EXTENSION TO HIGH DIMENSIONS Independant uncertain variables. Tensorial-product polynomial basis (1/2) D(ξ) = D 1 (ξ 1 )D 2 (ξ 2 ) For 2D problem (i generic grid index) W (i, ξ) P CW (i, ξ) = l 1 =M,l 2 =M l 1 =0,l 2 =0 C l1,l 2 (i)p l1 (ξ 1 )Q l2 (ξ 2 ) P orthonormal polynomials associated to D 1 (ξ 1 ) Q orthonormal polynomials associated to D 2 (ξ 2 ) Calculate C l1,l 2 (i) by collocation Calculate C l1,l 2 (i) by quadrature using : Z C l1,l 2 (i) == P CW (i, ξ)p l1 (ξ 1 )Q l2 (ξ 2 )D 1 (ξ 1 )D 2 (ξ 2 )dξ 1 dξ 2 C l1,l 2 (i) < W, P l1 Q l2 >= Z W (i, ξ)p l1 (ξ 1 )Q l2 (ξ 2 )D 1 (ξ 1 )D 2 (ξ 2 )dξ 1 dξ 2 using a G G grid based on Gaussian quadrature points

EXTENSION TO HIGH DIMENSIONS Independant uncertain variables. Tensorial-product polynomial basis (2/2) Accuracy issue. How close is W (i, ξ)p l1 (ξ 1 )Q l2 (ξ 2 ) from a (2G 1) in ξ 1 times a (2G 1) polynomial in ξ 2? Stochastic post-processing done for the expansion P CW instead of W straightforward evaluation of mean value E(P CW (i, ξ)) == C 0,0 (i) straightforward evaluation of variance E((P CW (i, ξ) E(P CW (i))) 2 ) = l 1 =M,l 2 =M l 1 =0,l 2 =0 (l 1,l 2 )<>(0,0) C l 1,l 2 (i) 2

EXTENSION TO HIGH DIMENSIONS Independant uncertain variables. Tensorial-grid probabilistic collocation (1/2) D(ξ) = D 1 (ξ 1 )D 2 (ξ 2 ) Lagrangian polynomial expansion For 2D problem W (i, ξ) SCW (i, ξ) = l 1 =N,l 2 =N H l (ξ j ) = Π m=n m=1,m<>l l 1 =1,l 2 =1 W l 1,l 2 (i)h l1 (ξ 1 )H l2 (ξ 2 ) (ξ j ξ j,m ) (ξ j,l ξ j,m ) (j = 1 or 2) Note that actually SCW (i, ξ l1, ξ l2 ) = W l1,l 2 (i). no coefficient computation, define W l1,l 2 (i) = W (i, ξ l1, ξ l2 ) Definition of (ξ j,1, ξ j,2,..., ξ j,n ) (j = 1 or 2) (most often, not absolutely necessary) N points of the N-point quadrature associated to D j (ξ j )

EXTENSION TO HIGH DIMENSIONS Independant uncertain variables. Tensorial-grid probabilistic collocation (2/2) Stochastic post-processing (mean and variance) done for the expansion SCW instead of W straightforward evaluation of mean value E(SCW (i, ξ)) =< SCW (i, ξ), 1 >= l 1 =N,l 2 =N l 1 =1,l 2 =1 ω 1,l 1 ω 2,l2 W l1,l 2 (i) straightforward evaluation of variance E((SCW (i, ξ) E(SCW (i))) 2 ) = l 1 =N,l 2 =N l 1 =1,l 2 =1 ω 1,l 1 ω 2,l2 W l1,l 2 (i) 2 ( l 1 =N,l 2 =N l 1 =1,l 2 =1 ω 1,l 1 ω 2,l2 W l1,l 2 (i)) 2

EXTENSION TO HIGH DIMENSIONS Dependant uncertain variables. True degree M polynomials in R d Number of terms for a degree M polynomial in dimension d (M+d)! M!d! Number of continuous equations for intrusive approach Multiplied by (M+d)! M!d! instead of (M + 1) Computation of coefficients for non-intrusive approach Collocation : from at least (M+d)! M!d! flow computations Exact quadrature for degree-m polynomial [Smolyak 1963]... Define clever enrichment... Huge interest the community. Huge literature......attend (UQ) course level two

Some outputs of NODESIM-CFD M. Lazareff NODESIM-CFD = EU project 2007-2009 Some important outputs of NODESIM-CFD Split (UQ) issues and CFD analysis issues. Test (UQ) propagation method on well defined CFD test cases AND mathematical models Compare standard deviation due to uncertainty to numerical errors and modelization errors Need for accurate definition of the input p.d.f. Actually, most often non intrusive methods for complex test cases

RAE2822 profile Configuration characteristics RAE2822 airfoil Re = 6.510 6, M = 0.734, α = 2.79 o Classical test case of high-re transonic CFD Complex flow with upper and lower shocks Strong dependance on turbulence model and mesh density

RAE2822 profile, k ω model, Re = 6.510 6 around M = 0.734, α = 2.79 o TPS C L response surface with SST correction medium CFD grid, low-density collocation (blue spheres)

RAE2822 profile, k ω model, Re = 6.510 6 around M = 0.734, α = 2.79 o TPS C L response surface without (transparent) and with (opaque, lower) SST correction medium CFD grid, low-density collocation (white/blue spheres)

NASA Rotor37 compressor Configuration characteristics Isolated rotor blade. Transonic flow. Subsonic outlet. Uncertainty propagation exercises β distribution (bounded support) uncertainty on outlet static-pressure and tip-clearance uncertainty quantification for several points of characteristic curve (several outlet static pressure <> several mass flows) polynomial chaos (gaussian quadrature) applied to functional outputs good quadrature convergence is observed computed order-4 moments almost identical to order-3 ones use order-3 Jacobi polynomials (associated to β distributions) only 4 CFD points needed for near-complete characterisation

NASA Rotor37 compressor Barplots for uncertainties on outlet pressure (left) and tip clearance (right) variations

NASA Rotor37 compressor Tip clearance : order-1 (mean) moments for global quantities

NASA Rotor37 compressor Tip clearance : order-2 (stdev) moments for global quantities

NASA Rotor37 compressor Application of intrusive polynomial chaos method Assessment of intrusive and non-intrusive non-deterministic CFD methodologies based on polynomial chaos expansions. [Dinsecu C, Smirnov S, Hirsch C, Lacor C. IJESMS 2 (2010)] Comparison between intrusive PC and probabilistic collocation NASA Rotor 37 (Compressor map) Uncertain parameters : outlet static pressure, height of tip clearance Output parameters : local static pressure (middle gridline, mid-span), pressure ration, isentropic efficiency Very consistent results for mean and standard deviation

Conclusion Uncertainty quantification more and more interest and projects (EU, RTO... ) get definition of industry relevant problems robust design optimization process Methods 1D definition multi-d, sparsity, clever enrichment... Challenges Get relevant p.d.f. for uncertain parameters Deal efficiently with large number of uncertain parameters Deal efficiently with geometrical uncertainties