Adaptive Collocation with Kernel Density Estimation

Transcription

1 Examples of with Kernel Density Estimation Howard C. Elman Department of Computer Science University of Maryland at College Park Christopher W. Miller Applied Mathematics and Scientific Computing Program University of Maryland at College Park with

2 Examples of 1 Background and Goals Examples of 5 with

3 Examples of Background and Goals Example: The Diffusion Equation The Joint Density Function Collocation Methods Example: the diffusion equation (a(x, ω) u) = f on D R d with suitable boundary conditions Uncertainty / randomness a = a(x, ω) is a random field For each fixed x D, a(x, ω) is a random variable Assumptions Coercivity: 0 < α 1 a α 2 < Problem is well posed Spatial correlation of random field: For x, x 1, x 2 D Mean µ(x) = E(a(x, )) Variance σ(x) 2 = E(a(x, ) 2 ) µ 2 Covariance function c(x 1, x 2 ) = E([a(x 1, ) µ(x 1 )] [a(x 2, ) µ(x 2 )]) is finite with

4 Examples of Example: The Diffusion Equation The Joint Density Function Collocation Methods Aims: Efficient computation of statistical properties of solution u(x, ω) Moments mean µ = Ω u(x, ω) dp(ω) = u(x, ξ) ρ(ξ)dξ Γ(Ω) variance Ω u(x, ω)2 dp(ω) µ 2 = Γ(Ω) u(x, ξ)2 ρ(ξ)dξ µ 2 Cumulative distribution functions, e.g., P(u(x, ω)) > α Issues addressed in this study: Costs of collocation methods Lack of knowledge of density function ρ(ξ) with

5 Examples of The Joint Density Function Example: The Diffusion Equation The Joint Density Function Collocation Methods Possible sources of diffusion coefficient Obtained from samples of a(x, ) Assume covariance function for a(x, ) is of standard type E.g. c(x 1, x 2 ) = exp ( x 1 x 2 1 /l) c(x 1, x 2 ) = exp ( ( x 1 x 2 2 /l) 2) Use to generate (truncated) Karhunen-Loève expansion a(x, ξ) = a 0 + σ m r=1 λr a r (x) ξ r Generate covariance matrix from samples, then as above Model itself has form of KL-expansion In all these scenarios The joint density function ρ(ξ) is generally not known with

6 Examples of Collocation Methods Example: The Diffusion Equation The Joint Density Function Collocation Methods Monte-Carlo (sampling) method: find u HE 1 (D) s.t. a(x, ξ k ) u vdx for all v HE 1 0 (D) D for a collection of samples {ξ k } L 2 (Γ) Collocation (Xiu, Hesthaven, Babuška, Nobile, Tempone, Webster) Choose {ξ k } in a special way (sparse grids), then construct discrete solution u (hp) (x, ξ) S E h T (p) to interpolate {u h (x, ξ k )} Advantages (vs. stochastic Galerkin): decouples algebraic system (like MC) applies in a straightforward way to nonlinear random terms vs. Monte Carlo: faster convergence for moderate number of parameters with

7 Examples of But Collocation may still be costly Example: The Diffusion Equation The Joint Density Function Collocation Methods Compare with Stochastic Galerkin (E., Miller, Phipps, Tuminaro): For comparable accuracy: # stochastic dof (collocation) 2 p (# stochastic dof (Galerkin)) { total polynomial degree for Galerkin Here p = level (depth) of sparse grid for collocation m = 4 uniform density Accuracy p = 1 p = 2 p = 3 p = 4 p = 5 Degrees of freedom p = 6 p = 5 p = 4 m = 5 uniform density Time p = 3 Error p = 2 p = 1 with

8 Examples of Adaptive Sparse Grid Collocation Influence of Density Function 1 Background and Goals 2 Adaptive Sparse Grid Collocation Influence of Density Function 3 4 Examples of 5 with

9 Examples of Adaptive Sparse Grid Collocation Adaptive Sparse Grid Collocation Influence of Density Function Collocation based on interpolation Consider function u(ξ) at left piecewise linear interpolant u I (ξ) ξ = child, node at next level uniform grid w(ξ ) = interpolation error at ξ Ma & Zabaras: If u i (ξ) is represented using hierarchical basis (left), then w(ξ ) = coefficient of basis function of interpolant on refined grid = hierarchical surplus Strategy: Refine grid using child ξ iff w(ξ ) > tolerance with

10 Examples of Adaptive Sparse Grid Collocation Adaptive Sparse Grid Collocation Influence of Density Function Collocation based on interpolation Consider function u(ξ) at left piecewise linear interpolant u I (ξ) ξ = child, node at next level uniform grid w(ξ ) = interpolation error at ξ Ma & Zabaras: If u i (ξ) is represented using hierarchical basis (left), then w(ξ ) = coefficient of basis function of interpolant on refined grid = hierarchical surplus Strategy: Refine grid using child ξ iff w(ξ ) > tolerance with

11 Examples of Adaptive Sparse Grid Collocation Influence of Density Function To Use this Idea with Diffusion Equation (a(x, ξ) u) = f Levels Start with ξ (0), compute solution u(x, ξ (0) ) Determines interpolant [A (0) u](x, ξ) Identify children {ξ (1) j } of ξ (0) Compute solutions {u(x, ξ (1) j )}, add {ξ (1) j } to set of collocation points according to adaptive strategy with tolerance test wj k > τ Determines interpolant [A (1) u](x, ξ) Repeat: identify children of level-1 points, compute solutions, etc. Result: Collocation solution A (k) u Approximate moments, distributions of u using A (k) u with

12 Examples of Adaptive Sparse Grid Collocation Influence of Density Function Algorithm: Adaptive interpolation with hierarchical basis functions Set A 0 (u)(ξ) = 0 Set k = 1 Set θadaptive 1 = θ1 repeat θ k+1 adaptive = for ξ k j w k j θadaptive k = u(ξj k ) A k 1 (u)(ξj k ) Hierarchical surplus if wj k > τ then θ k+1 adaptive = θk+1 adaptive child(ξ k endif endfor Set A k (u)(ξ) = k k = k + 1 until max( w k 1 j ) < τ i=1 j w i j ψi j (ξ) j ) Augment interpolant refined grid nodes only where interpolation error is large Interpolant = sum(levels) all level-k basis functions with

13 Examples of Adaptive Sparse Grid Collocation Influence of Density Function Will only add children of a collocation point if w k j > τ For an oscillatory function (below): points in refinement would tend to lie in oscillatory region u(ξ) = { ξ sin(1/ξ) ξ 0 0 ξ = 0 ξ with

14 Examples of Influence of Density Function Adaptive Sparse Grid Collocation Influence of Density Function What if density function is small on the oscillatory region? There may be no need to refine in this region u(ξ) = { ξ sin(1/ξ) ξ 0 0 ξ = 0 ξ Change criterion for refinement: refine if wj k ρ(ξj i ) is large with

15 Examples of Informally: only add children of a point if (i) the functon is changing in the region of that point, and (ii) the region is statistically significant What if we don t know ρ? Approximate ρ from the data using kernel density estimation Given N samples {ξ (i) } drawn from the distribution of ξ ˆρ(ξ) = 1 ( ) N Nh M k=1 K ξ ξ (i) h The kernel K satisfies R M K(ξ)dξ = 1 R M K(ξ)ξdξ = 0 R M K(ξ) ξ 2 dξ = k < K(ξ) 0 Comment: From here on: {ξ (i) } nearly always refers to samples with

16 Examples of KDE approximates the density by placing a bump over each observation Parameter h (bandwidth) controls smoothness Oversmoothed h too large Undersmoothed h too small with

17 Examples of In our examples: for bandwidth h, use maximum likelihood cross-validation: maximize CV (h) 1 N log(ˆρ i (ξ (i) )) N where ˆρ i (ξ) = 1 Nh M i=1 N k=1,k i ( ) ξ ξ (k) K h For K(ξ), use Epanechnikov kernel ( ) M 3 M K(ξ) = (1 ξi 2 )1 { 1 ξi 1} 4 i=1 Compactly supported, helps ensure coercivity for diffusion problem KDE material from Silverman with

18 Examples of One-Parameter Example Two-Parameter Example 1 Background and Goals Examples of One-Parameter Example Two-Parameter Example 5 with

19 Examples of One-Parameter Example Two-Parameter Example One-Parameter Example from Above Adaptive criterion and KDE: refine when w k j ˆρ(ξj i ) > τ: Collocation points are added only in statistically significant portions of the parameter space, determined from samples ξ with

20 Examples of Two-Parameter Example One-Parameter Example Two-Parameter Example Two dependent random variables, two statistically significant regions u(ξ 1, ξ 2 ) = ξ 1 ξ e (1 r) e (1 ˆr)2.1 r(ξ 1, ξ 2 ) = ξ ξ2 2 ˆr(ξ 1, ξ 2 ) = (ξ 1 5) 2 + (ξ 2 5) 2 ξ 1 log-normal with marginal density function ρ 1 (ξ 1 ) = ξ 2 = ξ 1 + η, η uniform (0, 1) (log ξ 1 1 ) 2 ξ 1 2π(.42 ) e 2(.4 2 ) with

21 Examples of One-Parameter Example Two-Parameter Example Plot of this function: Large near ξ = 1 and ξ 5 = 1 with

22 Examples of One-Parameter Example Two-Parameter Example Experiment Take 10,000 samples of (ξ 1, ξ 2 ) Shown at right, above Generate kernel density estimate ˆρ(ξ) by the method above Contour plot of ˆρ, shown at right below Observation: More samples near (0, 0) than (5, 5) = that large component plays more of a role in KDE with

23 Examples of One-Parameter Example Two-Parameter Example Experiment, continued So far: samples have been generated and used to define ˆρ Next: perform adaptive collocation with different values of τ Collocation points for τ = 10 3, τ = 10 4 shown at right N.B. Needed 5 global refinements to see large parts of u with

24 Examples of One-Parameter Example Two-Parameter Example Experiment, continued Use N = 10K samples {ξ (i) } N i=1 to compute sample mean E(u) µ MC 1 N N i=1 u(ξ(i) ) Compute sample variance in analogous manner From these: can get Monte-Carlo error with 95% confidence bound E(u) µ MC = Alterative: estimate mean using the collocation solution A(u) E(u) µ coll 1 N N i=1 (Au)(ξ(i) ) evaluated at Monte Carlo samples Comparison τ µ MC µ coll # coll. pts. (73) (101) (188) (3984) with

25 Examples of Error Analysis Example: Stochastic PDE 1 Background and Goals Examples of 5 Error Analysis Example: Stochastic PDE with

26 Error Analysis Background and Goals Examples of Error Analysis Example: Stochastic PDE Consider Chebyshev inequality error for Monte Carlo sample mean: That is: For a > 0, P ( E(u) µ MC a) Var(u) Take, say, P =.05, get a Var(u).05N N a 2. With 95% confidence, error of MC sample mean is bounded by Var(u).05N Collocation: Estimates the mean by µ coll = 1 N N i=1 (Au)(ξ(i) ) Error: E(u) µ coll E(u) µ MC + µ MC µ coll First term: MC error as above Second term: 1 N N i=1 (u(ξ(i) ) (Au)(ξ (i) )) Mean interpolation error with

27 Examples of Example: Stochastic PDE Error Analysis Example: Stochastic PDE Test problem: d dx a M(x, ξ) d dx u(x, ξ) = 1 x (0, 1) u(0, ξ) = u(1, ξ) = 0 a M = µ + M/2 1 k=0 λ k (ξ 2k cos(2πkx) + ξ 2k+1 sin(2πkx)) µ = 3, λ k = exp( k) ξ k uniformly distributed on [0, 1] Experiment: As above Generate N samples of ξ Use them to generate estimate ˆρ(ξ) Use ˆρ to generate adaptive collocation solution Au Compare with: Use the same N samples to perform MC simulation Differences from above: M > 2 random variables Function is solution to PDE with

28 Examples of Error Analysis Example: Stochastic PDE Representative Results: M = 10 parameters N τ (76) (1026) (1655) (5026) (8111) (92) (1170) (1189) (5773) (9404) (59) (1216) (1989) (5996) (9664) (93) (1120) (2041) (6095) (9787) (93) (1187) (2127) (6050) (9942) Monte-Carlo Mean interpolation error error 1 N N i=1 u(x, ξ(i) ) A(u)(x, ξ (i) ) l 2 (D) Parens: Number of DE solves needed for collocation Green: This number is smaller than N and yields smaller error with

29 Examples of Error Analysis Example: Stochastic PDE For different parameter sizes: For a smaller number, M = 5 Adaptive collocation is more effective N τ (28) (212) (301) (813) (1169) (28) (211) (315) (777) (1210) (33) (200) (297) (762) (1207) (33) (172) (286) (745) (1104) (33) (180) (294) (780) (1107) Monte-Carlo Mean interpolation error error 1 N N i=1 u(x, ξ(i) ) A(u)(x, ξ (i) ) l 2 (D) with

30 Examples of Error Analysis Example: Stochastic PDE For larger M = 20: Collocation is still effective N τ (41) (878) (1299) (4126) (6958) (41) (1045) (1738) (5545) (9106) (119) (1618) (2622) (8580) (14012) (156) (2459) (4169) (13389) (22276) (193) (3108) (4991) (15963) (26081) Monte-Carlo Mean interpolation error error 1 N N i=1 u(x, ξ(i) ) A(u)(x, ξ (i) ) l 2 (D) with

31 Examples of Concluding Remarks Adaptive collocation with kernel density estimation Enables adaptive construction of collocation points Avoids need to know density function explicitly Appears to be competitive with, potentially superior to, Monte Carlo methods with