DTU 2010 - Lecture I Uncertainty Quantification in Computational Science Jan S Hesthaven Brown University Jan.Hesthaven@Brown.edu
Objective of lectures The main objective of these lectures are To offer an overview of Uncertainty Quantification (UQ) and discuss its importance in modern predictive computational science. To introduce Polynomial Chaos (PC) as an attractive and efficient way of dealing with such challenges in the context of complex dynamical systems. To provide enough background to allow the audience to evaluate the importance within their own research area To inspire and suggest interesting research directions
The global picture Lecture 1 - Introduction to UQ Motivation, terminology, background, Wiener chaos expansions. Lecture II - Stochastic Galerkin methods Formulation, extensions, polynomial chaos, and examples. Lecture III - Stochastic Collocation methods Motivation, formulation, high-d integration, and examples. Lecture IV - Extensions, challenges, and open questions Geometric uncertainty, ANOVA expansions, and discussion of open questions. Thank you for taking the time
The local picture A few examples to motivate the need for UQ Classification of types of uncertainty Probability 101 Overview of classic computational techniques to deal with uncertainty. The Wiener Chaos expansion Summary
Motivation for UQ - V&V Consider the classic problem of Verification and Validation (V&V) in computational science Verification - the need to make sure the problem is solved correctly : Convergence, constructed solutions, analysis, stability etc Validation - the need to make sure the right problem is solved : Comparison against other codes, experimental data etc While the former is well known, the latter quickly gets dicey - and a source of trouble!
Motivation for UQ - V&V Imagine this outcome All are pleased and happy Imagine instead this outcome Let the blame game begin
Motivation for UQ - V&V.. but what is/could really be going on? Imagine the problem is sensitive in some way We could simply be solving different problem due to - initial conditions - boundary conditions - parameters
Motivation for UQ - unmeassuables Many types of problem have inaccessible or unmeasurable parameters and characterization Biological systems Modeling of complex environments Micro/nano scale materials One can question the value of deterministic modeling of such systems - but then what?
Motivation for UQ - optimization/design In computer assisted optimization and design, one seeks to minimize a cost-function min x Ω J(x, µ) Why is this problematic? A better approach may be min x Ω E[J(x, µ)] + κvar(j(x, µ)).. but this requires us to be able to evaluate the impact of the parametric uncertainty - quickly.
Motivation for UQ Suggests that we need to re-evaluate our computational approach to achieve a true predictive capability
Motivation for UQ We need to consider a more complex situation This raises important questions such as How do we do this reliably? What is the cost? Do we need to develop everything from bottom up?
A little terminology Before embarking on this, let us revisit the sources of uncertainty and how we can hope to control them Uncertainty can be caused by a number of things Initial and boundary conditions Geometries Parametric variations Modeling errors Sources etc
A little terminology Aleatory uncertainty is inherent variation of the physical system and the environment - variability, irreducible uncertainty, random uncertainty etc Examples: Rapid variations in parameters, inherent randomness in a microstructure etc Epistemic uncertainty is caused by insufficient knowledge of parameters or processes - subjective uncertainty, reducible uncertainty, model uncertainty Examples: Insufficient experimental results, poor understanding of system, microstructure etc
A little terminology With the need to consider systems subject to both types of uncertainty, there is no alternative but to model the uncertainty in some way Aleatory uncertainty is often modeled by some assumed probability measure. Epistemic uncertainty is more problematic as it is grounded in insufficient knowledge of the system. It is often modeled by intervals of possible values... but it is, in principle, reducible at added cost.
Probability 101 Before we continue, let us make sure we recall the necessary background and terminology. The outcome of experiment is an event - Ex: Flipping a coin gives head or tails. We assign a number to the outcome to recover a Random variable - Ex: The event space, the empty set, and a number of set combinations is called the -field - called We assign probabilities to measure likelyhood of the outcome of the random variables - Ex: X(ω) [0, 1] X = X(ω) P (ω : X(ω) = 0) = P (ω : X(ω) = 1) = 0.5 σ ω Ω Ω = {head, tail} F P (ω : X(ω)) [0, 1]
Probability 101 Definition 2.4 (Probability space). Aprobabilityspaceisatriplet(, F,P) where is a countable event space, F 2 is the σ -field of, andp is a probability measure such that 1. 0 P(A) 1, A F. 2. P( ) = 1. 3. For A 1,A 2,... F and A i A j =, i = j, ( ) P A i = P(A i ). i=1 Definition 2.5 (Distribution function). The probability distribution F X (x) =P (X x) =P ({ω : X(ω) x}), x R The probability density F X (x) = f X (x) 0, x f X (y) dy f X (y) dy =1 i=1
Probability 101 We can now more appropriately define The mean or expectation µ X = E[X] = xf X dx E[g(X)] = g(x)f X dx The variance σ 2 X = var(x) = The m th moment E[X m ]= The standard deviation (x µ X ) 2 f X dx x m f X dx σ 2 X = E[X 2 ] µ 2 X
Probability 101 Let us recall a couple of widely used densities The normal/gaussian distribution - N(µ, σ 2 ) N f X (x) = [ 1 exp (x µ)2 2πσ 2 2σ [ ] 2 ], The uniform distribution - U(a, b) f X (x) = { 1 b a, x (a, b), 0, otherwise, de.
Probability 101 The last concepts we will recall are The correlation between two random variables is corr(x 1,X 2 ) = cov(x 1,X 2 ) 1 corr(x σ X1 σ 1,X 2 ) 1. X2 cient. Animmediatefactfollow ables are uncorrelated if cor cov(x 1,X 2 )=E[(X 1 µ X1 )(X 2 µ X2 )] They are uncorrelated if corr(x 1,X 2 )=0 The variables are independent if P(A 1 A 2 ) = P(A 1 )P (A 2 ). f X1,X 2 (x 1,x 2 )=f X1 (x 1 )f X2 (x 2 ) Independence Uncorrelated
Classic methods for UQ Let us now - finally - return to the quest at hand: How do we account for the impact of the uncertainty on the output of a dynamical system To briefly recall a few classic methods, let us consider the following problem (α, β) (α, β) du dt (t, ω) = α(ω)u, u(t, ω) : [0,T] Ω R independent then du(t, ω) dt dependent then u(0, ω) = β(ω), u(t, X 1,X 2 ) : [0,T] R 2 R = X 1 u, u(0, ω) =X 2 u(t, X, g(x)) : [0,T] R R du(t, ω) dt = Xu, u(0, ω) = g(x)
Monte Carlo methods The truly classic and simple approach 1. Create M iid s from the assumed distribution - (α (i), β (i) ) 2. Solve problem for each set of iid s - 3. Compute required statistics - u (i) (t) =u(t, X (i) ) ū(t) = 1 M u(t, X (i) ) E[u] M As samples are based on iid s, the central limit theorem implies ū(t) E[u] 1 M i=1 This result is both the curse and the strength The convergence is slow, i.e., expensive for good accuracy The convergence does not depend on dimension Note: Lots of games in town to improve
Moment methods Consider the following equation µ(t) =E[u] dµ dt The unknown can be computed de[αu] dt.. and can be continued = E[α 2 u], µ(0) = E[αβ] = E[αu], µ(0) = E[β] Tempting dµ dt = E[α]µ, µ(0) = E[β] µ k = E[α k u] dµ k dt = µ k+1, µ k (0) = E[α k β] but the closure problem remains - often solved by assuming µ k+1 = g(µ 0,...,µ k ) Accuracy is unclear in model Complex for a large problem
Perturbation methods Let us assume that The perturbation method c ɛ = O(α(ω)) σ α 1. power series of, Then we can express the solution as u(t, ω) = u 0 (t) + α(ω)u 1 (t) + α 2 (ω)u 2 (t) +, Matching orders in the expansion yields : = : = O(1): du 0 dt = 0, O(ɛ): du 1 dt = u 0, O(ɛ 2 ): du 2 dt = u 1, Only applies for small variance(s) Derivation is problem specific
Challenges in classic methods There are serious problems with these methods Moment/Perturbation methods are too complex and intrusive to serve as a general tool. Monte Carlo methods are flexible/ general - the convergence rate is problematic: 1 digit requires 100 simulations How can be strive to improve this? MC approximates the density using piecewise constant samples - resulting in the slow pointwise convergence
Challenges in classic methods But recall the context here - we will Make assumption on the nature of the aleatory uncertainty through the input variables. Make some assumptions on the character of the epistemic uncertainty, possibly using just uniformly distributed variables. These are random variables with a smooth density From an approximation standpoint, using piecewise constant functions to represent a smooth function is a poor choice
Detour on global expansions Consider the smooth period function Using a Fourier series P N u(x) = û n = 1 2π n N/2 2π 0 û n e inx, u(x)e inx dx P N u 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 u(x) N = 16 N = 8 N = 4 re u(x) = u P N u 10 0 10 2 10 4 10 6 10 8 10 10 10 12 3 5 4 cos(x). N = 4 N = 8 N = 16 N = 32 N = 64 0.0 0.0 0.2 0.4 0.6 0.8 1.0 x/2p Figure 2.1 Rigorous theory confirms that 10 14 0.0 0.2 0.4 0.6 0.8 1.0 x/2p u P 2N u W q p [0,2π] s identity yields C N r q u W r p [0,2π]. u P 2N u L C 1 q N q 1 2 Challenge: How to take advantage of this for UQ? u (q) L 2 [0,2π].
The Wiener Chaos expansion Define the space of square integrable functions L 2 F X = {f : I R E[f 2 ] < } Provided f L 2 F X f(x) = n=0 we have ˆf n Φ n (X) ˆf n = 1 γ n E[f(X)Φ n (X)] γ n = E[Φ 2 n(x)] Where the basis - the Chaos Polynomial - satisfies E[Φ m (X)Φ n (X)] = Φ m (X(x))Φ n (X(x)) df X (x) =γ n δ mn Basis depends on the distribution of the random variable!
The Wiener Chaos expansion Consider Gaussian variables with f X (x) = 1 2π e x2 /2 The corresponding polynomials are the Hermite Polynomials H 0 (X) =1,H 1 (X) =X, H 2 (X) =X 2 1, H 3 (X) =X 3 3X
The Wiener Chaos expansion If we now consider the truncated expansion N P N f(x) = ˆf n Φ n (X) n=0 then strong convergence follows directly from classic theory f P N f L 2 FX 0, N If we now consider a more general problem Z N = N n=0 X L 2 â n Φ n (X) F X Z L 2 F Z Weak convergence can be achieved by defining â n = 1 E X [F 1 Z γ (F X(X))Φ n (X)] n We can use one random variable to approximate another
The Wiener Chaos expansion Xiu, 2010 0.6 0.5 exact 1st order 3rd order 5th order 0.45 0.4 exact 1st order 3rd order 5th order 0.35 0.4 0.3 0.25 0.3 0.2 0.2 0.15 0.1 0.1 0.05 0 4 3 2 1 0 1 2 3 4 0 4 3 2 1 0 1 2 3 4 It is clear that choosing the right basis -- associated with the nature of the random variable -- is key to performance This the advantage and the curse -- as we shall see
The Wiener Chaos expansion The extension to multiple random variables follows Define F Xi (x i )=P(X i x i ) X =(X 1,...,X d ) F X = F X1... F Xd and the multi-dimensional polynomial chaos x i I Xi Φ i (X) =Φ i1 (X 1 )... Φ id (X d ) i N E[Φ i (X)Φ j (X)] = Φ i (x)φ j (x) df X (x) =γ i δ ij γ i = E[Φ 2 i ] The homogeneous Chaos expansion is f N (X) = N i =0 ˆf i Φ i (X) P d N dim P d N = N + d N = (N + d)! N!d! The curse of dimension shows its face!
The Wiener Chaos expansion Assuming the Chaos expansion is known, we need statistics N f N (X) = ˆf i Φ i (X) P d N i =0 The expectation follows from µ = E[f] E[f N ]= N i =0 ˆf i Φ i df X = ˆf 0 In a similar fashion, the variance is N var(f) =E[(f µ) 2 )] γ ˆf 2 i i i >0 Other moments can be obtained in a similar fashion. Functions of the expansion can also be estimated through Monte Carlo sampling
Summary We have achieved quite a bit Motivated the need for UQ in Computational Science Discussed in some detail the shortcomings of classic methods such as Monte Carlo methods. Realizing that smoothness in the behavior of the random variables should be explored.. and introduced the Chaos expansion to achieve this We have still to use this insight to solve differential equations - and demonstrate the promised benefits