Polynomial Chaos and Karhunen-Loeve Expansion

Transcription

1 Polynomial Chaos and Karhunen-Loeve Expansion 1) Random Variables Consider a system that is modeled by R = M(x, t, X) where X is a random variable. We are interested in determining the probability of the response, p(r). If there is only one random variable, we write p(r) = p(x) dr/dx (1) If there are two random variables, X 1, X 2, then the determination of p(r) is more difficult, but in principle possible. For more than two random variables, or if p(x) is complicated, it it unlikely that we can obtain an analytic expression for p(r). 2) Monte Carlo and Surrogate Models Instead of analytic solutions, which are rarely possible, we will make use of the Monte Carlo approach in which we draw samples of X i from their distributions, compute R and from these obtain approximate distributions p(r). Unfortunately MC requires a very large number of samples to converge and if the model is complex and computationally difficult, this approach is usually not a realistic possibility. In this case we use surrogate models, that is we sample R for a small number of values of X and assume that behavior of the samples R(X) is a reasonable representation of the complete response for all values of X. One approach is to represent M(x, t, X) by a reduced order form, for example using Proper Orthogonal Decomposition (POD) and then sampling this reduced form using Latin Hypercube or a similar approach. 3) Polynomial Chaos, PC A Polynomial Chaos expression (PCE) is a way of representing an arbitrary random variable as a function of another random variable with a given distribution, and representing that function as a polynomial expansion. For example we might choose Ξ to have a uniform distribution, or a normal distribution. We will represent X in much the same way that a time series, f(t), can be represented by a Fourier Series of sines and cosines. Let X be the random variable and Ξ be the random variable in terms of which X will be expressed. In PC theory Ξ is often called the germ. In other words we write X = f(ξ) (2) and we seek an appropriate function f(.) such that given that Ξ has the desired germ distribution, that X will have its required distribution. 1

2 Polynomial Chaos expands f(ξ) as a polynomial series in terms of basis functions. For example if a sample from p(ξ) is denoted by ξ, we might use f(ξ) = a 0 + a 1 ξ +, a n ξ n. The specific form of this series depends upon the distribution of Ξ. While there is considerable freedom in choosing the basis functions of the series, it is most convenient to use orthogonal functions, just like sines and cosines are convenient. Let the polynomial expression be X = f(ξ) = (ψ 0 = 1, ψ 1, ψ 2, ) (3) where ψ j (ξ) is a polynomial or order j in terms of ξ which represents a sample (also called a realization) from the distribution p(ξ). With orthogonal functions we have the orthogonality relationship < ψ i, ψ j >= ψ i (ξ)ψ j (ξ)p(ξ)dξ = 0 (4) where the operation < a, b > is defined by Eq. 4. Since ψ i (ξ) are orthogonal to all other ψ j (ξ) including to ψ 0, all ψ j (ξ) for j 1 have zero mean and because of orthogonality they are uncorrelated, i.e., independent. Now using the orthogonal basis functions we write X = f(ξ) = x j ψ j (Ξ) (5) The expansion in the form of Eq. 5 is called a Polynomial Chaos Expansion (PCE). Given f(ξ) and ψ j, and making use of orthogonality we have In practice, the series in Eq. 5 is truncated x j =< f, ψ j > / < ψ j, ψ j > (6) X = f M (Ξ) x j ψ j (Ξ) (7) Thus, given a random variable X and knowing its distribution p(x), we can express a realization of X by Eq. 7. While in principle it is possible to go from X Ξ, in practice, one posits a polynomial with specific values of x i and creates the corresponding X, see Sec ) Examples Some common PC expansions are 1. If the basis functions are to represent uniformly distributed distributions of ξ in [-1,1], they are the Legendre functions 2. If the basis functions are to represent normal distributions of ξ, they are the Hermite polynomials, ψ 0 (ξ) = 1, ψ 1 (ξ) = ξ, ψ 2 (ξ) = ξ 2 1 2

3 3. If they represent an exponential random variable in [0, ) they are the Laguerre polynomials A common case is when X is assumed to be normally distributed. Then we write X = x 0 ψ 0 + x 1 ψ 1 (ξ) = µ + σξ (8) where µ and σ are the mean and standard deviation of X. 3.2) Multivariate PC Theory Both X and Ξ can be vectors, If Ξ is a vector, then the polynomial ψ j (Ξ) is multivariate, i.e., a vector, and is usually written as Ψ j (Ξ). When the components of Ξ are independent (not correlated), then Ψ j is a tensor product of the polynomial bases for each Ξ i and we write 3.3) Random Fields Ψ j (ξ) = m ψ j (ξ i ) (9) Consider a random process (also called a field), X(t). We then write X(t) 3.4) The output R = M(x, t, X) x j (t)ψ j (Ξ) (10) Since X is uncertain, so also is R. In practice both X and R are generally vectors, or may even be processes. We then write the output as or in practice R = g(ξ) = r j ψ j (Ξ) R g M (Ξ) = r j ψ j (Ξ) Although not necessary, it is usual to use the same base functions (i.e., the same germ) for g as for f and the same order of truncation, M. The final result is then r j ψ j (Ξ) M(x, t, x j (t)ψ j (Ξ)) (11) 3

4 4) Karhunen-Loeve Expansions, K-L The Karhunen-Loeve expansion is written as X(t) = E[X(t)] + λj φ j (t)ξ j (12) Comparing with Eq. 5 we see the the K-L expansion is a PCE because it expresses X(t) as a function of Ξ and looks just like the multi-variate form, Eq. 9, but only up to polynomial order 1. The importance of the K-L expansion because the λ i and φ j (t) are the eigenvalues and eigenvectors of the covariance of X(t), that is we can include correlations between the different random variables. 4.1) Isotropic Correlation Consider the case where the function R(.) depends on two random variables, X and Y. Suppose that X, Y are correlated and that the correlation depends only on the distance between the two arguments, i.e., on x y for scalars or x y for vector arguments. Then we are saying that the correlation j=1 correlation(x, Y )) = E[(X E(X])(Y E[Y ]) σ(x)σ(y) (13a) = c( x y ) (13b) Such a correlation is said to be isotropic. Given some reference correlation length ρ 0, and defining ρ = x y, ˆρ = ρ/ρ o, some popular correlation functions are Table 1: Isotropic Correlation Functions Constant c(ρ)=1 everywhere Exponential c(ρ) = e ˆρ Gaussian c(ρ) = e ˆρ2 Linear c(ρ) = max(1 ˆρ, 0) White Noise c(ρ) = 1 if ρ = 0, 0 otherwise 4.2) SVD Every real m by n matrix A has a singular value decomposition A = USV T (14) where 4

5 1. U is an m by m orthogonal matrix (U T U = I) 2. S is a m by n diagonal matrix with non-negative entries ordered in decreasing values 3. V is an n by n orthogonal matrix where r A = λi u i vi T λ i = eigenvalue of AA T r = rank of A (15a) (15b) (15c) If A k is the sum that extends only to k terms, then it can be shown that A A k 2 = λ max of (A A k ) T (A A k ) A A k 2 2 = λ k+1 (16a) (16b) In other words, A k is the best possible representation of A in a least squares sense. Comparing Eq. 15a with Eq. 12 we see that the SVD is the discrete version of the K-L expansion. 5) How does it work? The most common application of PCE for engineering problems uses the K-L expansion where the material property is assumed to vary spatially. The K-L treats the random property as having a Gaussian distribution, that is P(x) = E[P(x)] + λj φ j (t)ξ j (17) where ξ i is a sample from a Gaussian distribution. Consider a finite element program that solves Ku = f where K is called the stiffness matrix (conductance matrix for thermal problems). For linear problems K can be represented as j=1 Ku = (K 0 + K ξ )R = K 0 R 0 + ξ i K ξ R ξ = f (18) where K 0 is based upon the average value E[P(x)] and K ξ is based on the random part of Eq. 17. Thus we have M + 1 problems to solve. If M is large, this may be expensive, hence the frequent use of reduced order methods. 5