Application of QMC methods to PDEs with random coefficients

Application of QMC methods to PDEs with random coefficients a survey of analysis and implementation Frances Kuo f.kuo@unsw.edu.au University of New South Wales, Sydney, Australia joint work with Ivan Graham and Rob Scheichl (Bath), Dirk Nuyens (KU Leuven), Christoph Schwab (ETH Zurich), Elizabeth Ullmann (Hamburg), Josef Dick, Thong Le Gia, James Nichols, and Ian Sloan (UNSW) Based on a survey of the same title, written jointly with Dirk Nuyens Frances Kuo @ UNSW Australia p.1

Outline Motivating example a flow through a porous medium Quasi-Monte Carlo (QMC) methods Component-by-component (CBC) construction Application of QMC theory to PDEs with random coefficients estimate the norm choose the weights Software http://people.cs.kuleuven.be/~dirk.nuyens/qmc4pde Concluding remarks Frances Kuo @ UNSW Australia p.2

Motivating example Uncertainty in groundwater flow eg. risk analysis of radwaste disposal or CO 2 sequestration Darcy s law q + a p = f in D R d, d = 1,2,3 mass conservation law q = 0 together with boundary conditions [Cliffe, et. al. (2000)] Uncertainty in a(x, ω) leads to uncertainty in q(x, ω) and p(x, ω) Frances Kuo @ UNSW Australia p.3

Expected values of quantities of interest To compute the expected value of some quantity of interest: 1. Generate a number of realizations of the random field (Some approximation may be required) 2. For each realization, solve the PDE using e.g. FEM / FVM / mfem 3. Take the average of all solutions from different realizations This describes Monte Carlo simulation. Example: particle dispersion release point p n = 0 p = 1 p = 0 1 0.8 0.6 0.4 0.2 p n = 0 0 0 0.2 0.4 0.6 0.8 1 Frances Kuo @ UNSW Australia p.4

Expected values of quantities of interest To compute the expected value of some quantity of interest: 1. Generate a number of realizations of the random field (Some approximation may be required) 2. For each realization, solve the PDE using e.g. FEM / FVM / mfem 3. Take the average of all solutions from different realizations This describes Monte Carlo simulation. NOTE: expected value = (high dimensional) integral 10 1 use quasi-monte Carlo methods 10 2 MC n 1/2 error 10 3 10 4 10 5 n 1 or better 10 3 10 4 10 5 10 6 n QMC s = stochastic dimension Frances Kuo @ UNSW Australia p.4

Frances Kuo @ UNSW Australia p.5 MC v.s. QMC in the unit cube [0,1] s F(y)dy 1 n n i=1 F(t i ) Monte Carlo method Quasi-Monte Carlo methods t i random uniform t i deterministic n 1/2 convergence close to n 1 convergence or better 0 1 1 64 random points 0 1 1 First 64 points of a 2D Sobol sequence 0 1 1 A lattice rule with 64 points more effective for earlier variables and lower-order projections order of variables irrelevant order of variables very important use randomized QMC methods for error estimation

Frances Kuo @ UNSW Australia p.6 QMC Two main families of QMC methods: (t,m,s)-nets and (t,s)-sequences lattice rules 0 1 1 First 64 points of a 2D Sobol sequence 0 1 1 A lattice rule with 64 points (0,6,2)-net Having the right number of points in various sub-cubes A group under addition modulo Z and includes the integer points Niederreiter book (1992) Sloan and Joe book (1994) Important developments: component-by-component (CBC) construction, fast CBC higher order digital nets Dick and Pillichshammer book (2010) Dick, Kuo, Sloan Acta Numerica (2013) Nuyens and Cools (2006) Dick (2008)

Application of QMC to PDEs with random coefficients [0] Graham, K., Nuyens, Scheichl, Sloan (J. Comput. Physics, 2011) [1] K., Schwab, Sloan (SIAM J. Numer. Anal., 2012) [2] K., Schwab, Sloan (J. FoCM, 2015) [3] Graham, K., Nichols, Scheichl, Schwab, Sloan (Numer. Math., 2015) [4] K., Scheichl, Schwab, Sloan, Ullmann (in review) [5] Dick, K., Le Gia, Nuyens, Schwab (SIAM J. Numer. Anal., 2014) [6] Dick, K., Le Gia, Schwab (in review) [7] Graham, K., Nuyens, Scheichl, Sloan (in progress) Also Schwab (Proceedings of MCQMC 2012) Le Gia (Proceedings of MCQMC 2012) Dick, Le Gia, Schwab (in review) Harbrecht, Peters, Siebenmorgen (Math. Comp., 2015) Gantner, Schwab (Proceedings of MCQMC 2014) Robbe, Nuyens, Vandewalle (Master thesis of Robbe, 2015) Ganesh, Hawkings (SIAM J. Sci. Comput., 2015) Frances Kuo @ UNSW Australia p.7

Application of QMC to PDEs with random coefficients Three different QMC theoretical settings: [1,2] Weighted Sobolev space over [0,1] s and randomly shifted lattice rules [3,4] Weighted space setting in R s and randomly shifted lattice rules [5,6] Weighted space of smooth functions over [0,1] s and (deterministic) interlaced polynomial lattice rules AIM OF THIS TALK: to survey [1,2], [3,4], [5,6] in a unified view Uniform Lognormal Lognormal KL expansion KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances Kuo @ UNSW Australia p.9

5-minute QMC primer Common features among all three QMC theoretical settings: Separation in error bound (rms) QMC error (rms) worst case error γ norm of integrand γ Weighted spaces Pairing of QMC rule with function space Optimal rate of convergence Rate and constant independent of dimension Fast component-by-component (CBC) construction Extensible or embedded rules Application of QMC theory Estimate the norm (critical step) Choose the weights Weights as input to the CBC construction Frances Kuo @ UNSW Australia p.10

Application of QMC to PDEs with random coefficients Three different QMC theoretical settings: [1,2] Weighted Sobolev space over [0,1] s and randomly shifted lattice rules [3,4] Weighted space setting in R s and randomly shifted lattice rules [5,6] Weighted space of smooth functions over [0,1] s and (deterministic) interlaced polynomial lattice rules Uniform Lognormal Lognormal KL expansion KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances Kuo @ UNSW Australia p.11

Frances Kuo @ UNSW Australia p.12 Lattice rules Rank-1 lattice rules have points t i = frac ( i n z ), i = 1,2,...,n z Z s the generating vector, with all components coprime to n frac( ) means to take the fractional part of all components 0 1 1 A lattice rule with 64 points

Lattice rules Rank-1 lattice rules have points ( ) i t i = frac n z, i = 1,2,...,n z Z s the generating vector, with all components coprime to n frac( ) means to take the fractional part of all components quality determined by the choice of z n = 64 z = (1,19) t i = frac ( ) i 64 (1,19) 1 10 37 20 47 3 13 30 57 40 23 50 6 33 60 16 26 43 53 9 36 63 2 19 29 46 56 12 39 5 22 49 32 15 42 59 25 52 8 18 35 45 62 1 28 55 11 21 38 48 4 31 58 14 41 24 7 34 51 61 17 44 64 27 54 0 1 A lattice rule with 64 points Frances Kuo @ UNSW Australia p.12

Frances Kuo @ UNSW Australia p.13 Randomly shifted lattice rules Shifted rank-1 lattice rules have points t i = frac ( i n z + ), i = 1,2,...,n [0,1) s the shift shifted by = (0.1,0.3) 0 1 1 A lattice rule with 64 points 0 1 1 A shifted lattice rule with 64 points use a number of random shifts for error estimation

Component-by-component construction Want to find z with (shift-averaged) worst case error as small possible. Exhaustive search is practically impossible - too many choices! CBC algorithm [Sloan, K., Joe (2002);... ] 1. Set z 1 = 1. 2. With z 1 fixed, choose z 2 to minimize the worst case error in 2D. 3. With z 1,z 2 fixed, choose z 3 to minimize the worst case error in 3D. 4. etc. Cost of algorithm is only O(nlogns) using FFTs. [Nuyens, Cools (2006)] Optimal rate of convergence O(n 1+δ ) in weighted Sobolev space, with the implied constant independent of s under an appropriate condition on the weights. [K. (2003); Dick (2004)] Averaging argument: there is always one choice as good as average! Extensible/embedded variants. [Cools, K., Nuyens (2006); Dick, Pillichshammer, Waterhouse (2007)] http://people.cs.kuleuven.be/~dirk.nuyens/fast-cbc/ http://people.cs.kuleuven.be/~dirk.nuyens/qmc-generators/ Frances Kuo @ UNSW Australia p.14

Fast CBC construction [Nuyens, Cools (2006)] n = 53 Natural ordering of the indices 1 Generator ordering of the indices 53 Matrix-vector multiplication with a circulant matrix can be done using FFT Images by Dirk Nuyens, KU Leuven Frances Kuo @ UNSW Australia p.15

Fast CBC construction [Nuyens, Cools (2006)] n = 128 = 2 7 1 Natural ordering of the indices 4 Symmetric reduction after application of B 2 kernel function 1 2 4 8 16 32 64 128 2 Grouping on divisors 3 Generator ordering of the indices 1 2 4 8 16 32 64 128 Images by Dirk Nuyens, KU Leuven Frances Kuo @ UNSW Australia p.16

Standard QMC theory today Worst case error bound F(y)dy 1 [0,1] s n n F(t i ) ewor γ (t 1,...,t n ) F γ i=1 Weighted Sobolev space [Sloan, Woźniakowski (1998)] F 2 γ = 1 u 2 F (y u ;0) dy y u u 2 s subsets u {1:s} γ u weights [0,1] u anchor at 0 (also unanchored ) Mixed first derivatives are square integrable Small weight γ u means that F depends weakly on the variables y u Choose weights to minimize the error bound [K., Schwab, Sloan (2012)] ( ) 1/(2λ) ( ) 1/2 2 ( ) γ λ u n a b 1/(1+λ) u bu u γ u = u {1:s} }{{} bound on worst case error (CBC) u {1:s} γ u } {{ } bound on norm Construct points (CBC) to minimize the worst case error a u POD weights Frances Kuo @ UNSW Australia p.17

Application of QMC to PDEs with random coefficients Three different QMC theoretical settings: [1,2] Weighted Sobolev space over [0,1] s and randomly shifted lattice rules [3,4] Weighted space setting in R s and randomly shifted lattice rules [5,6] Weighted space of smooth functions over [0,1] s and (deterministic) interlaced polynomial lattice rules Uniform Lognormal Lognormal KL expansion KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances Kuo @ UNSW Australia p.18

QMC theory for integration over R s Change of variables s F(y) φ(y j )dy = R s j=1 [0,1] s F(Φ 1 (w))dw Norm with weight function [Nichols & K. (2014)] F 2 γ = 1 u F (y u ;0) y u u {1:s} γ u R u 2 j u ϕ 2 j (y j)dy u weight function Also unanchored variant Randomly shifted lattice rules CBC error bound for general weights γ u Convergence rate depends on the relationship between φ and ϕ j Fast CBC for POD weights Frances Kuo @ UNSW Australia p.19

Application of QMC to PDEs with random coefficients Three different QMC theoretical settings: [1,2] Weighted Sobolev space over [0,1] s and randomly shifted lattice rules [3,4] Weighted space setting in R s and randomly shifted lattice rules [5,6] Weighted space of smooth functions over [0,1] s and (deterministic) interlaced polynomial lattice rules Uniform Lognormal Lognormal KL expansion KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances Kuo @ UNSW Australia p.20

Higher order digital nets [Dick (2008)] Classical polynomial lattice rule [Niederreiter (1992)] n = b m points with prime b An irreducible polynomial with degree m A generating vector of s polynomials with degree < m Interlaced polynomial lattice rule [Goda, Dick (2012)] Interlacing factor α An irreducible polynomial with degree m A generating vector of αs polynomials with degree < m Digit interlacing function D α : [0,1) α [0,1) (0.x 11 x 12 x 13 ) b,(0.x 21 x 22 x 23 ) b,..., (0.x α1 x α2 x α3 ) b becomes (0.x 11 x 21 x α1 x 12 x 22 x α2 x 13 x 23 x α3 ) b Weighted spaces with norm involving higher order mixed derivatives Variants of fast CBC construction available [Dick, K., Le Gia, Nuyens, Schwab (2014)] SPOD weights Frances Kuo @ UNSW Australia p.21

Properties of higher order digital nets 16 points obtained from a 4D Sobol sequence with interlacing factor 2: Each rectangle contains exactly 2 points. Frances Kuo @ UNSW Australia p.22

Properties of higher order digital nets 16 points obtained from a 4D Sobol sequence with interlacing factor 2: The shaded area contains exactly 2 points. Frances Kuo @ UNSW Australia p.22

Frances Kuo @ UNSW Australia p.22 Properties of higher order digital nets 16 points obtained from a 4D Sobol sequence with interlacing factor 2: The shaded area contains exactly half of the points.

Projections of higher order digital nets Images by Dirk Nuyens, KU Leuven Frances Kuo @ UNSW Australia p.23

Uniform v.s. lognormal (a(x,y) u(x,y)) = f(x), x D R d, d = 1,2,3 u(x,y) = 0, x D Uniform a(x,y) = a 0 (x) + j 1y j ψ j (x), y j U[ 1 2, 1 2 ] Goal: ( 1 2,1 2 )N G(u(,y))dy G bounded linear functional Key assumption: 0 < a min a(x,y) a max < Lognormal a(x,y) = exp ( a 0 (x) + j 1y ) j ψ j (x), y j N(0,1) Goal: R N G(u(,y)) j 1 φ(y j )dy = (0,1) N G(u(,Φ -1 (w)))dw Key assumption: restrict to y such that j 1 y j ψ j L < Frances Kuo @ UNSW Australia p.25

Single-level v.s. multi-level (the uniform case) I = G(u(,y))dy ( 1 2,1 2 )N Single-level algorithm 1 n Deterministic: G(u s h n (,t i 1 2 )) i=1 (1) dimension truncation (2) FE discretization (3) QMC quadrature Multi-level algorithm L ( 1 Deterministic: n l=0 l n l i=1 I = (I I L ) + L l=0 (I l I l 1 ), I 1 := 0 [Heinrich (1998); Giles (2008)] G((u s l h l u s l 1 ) h l 1 )(,t l,i 1 2 )) Frances Kuo @ UNSW Australia p.26

Estimate the norm (the uniform case) Single-level algorithm G(u s h ) γ Multi-level algorithm G(u s l h l u s l 1 h l 1 ) γ Frances Kuo @ UNSW Australia p.27

Estimate the norm (the uniform case) Single-level algorithm G(u s h ) γ u ( G(u s h y (,y)) u ) = G u s h u y (,y) G H -1 u u s h u y (,y) u linearity and boundedness of G H 1 0 ν u(,y) H 1 0 = ν u(,y) L2 ν!b ν f H-1 a min, b j := ψ j L a min G(u s h ) γ f H-1 G H -1 a min ( u {1:s} ( u!) 2 j u b2 j γ u ) 1/2 Frances Kuo @ UNSW Australia p.27

Choose the weights (the uniform case) Single-level algorithm G(u s h ) γ b j := ψ j L a min ms-error ( 2 n u {1:s} γ u λ [ρ(λ)] u ) 1/λ( Minimize the bound POD weights γ u = If j 1 bp j <, take λ = p 2 p The convergence rate is n min(1 δ, 1 p 1 2 ). The implied constant is independent of s. u {1:s} ( u! j u ( u!) 2 j u b2 j Frances Kuo @ UNSW Australia p.28 γ u for all λ ( 1 2,1] ) ) b 2/(1+λ) j ρ(λ) 1 2 2δ for δ (0, 1 2 ) when p (0, 2 3 ], when p ( 2 3,1).

Estimate the norm (the uniform case) Single-level algorithm G(u s h ) γ b j := ψ j L a min Lemma 1 ν u(,y) L2 ν!b ν f H-1 a min Multi-level algorithm G(u s l h l u s l 1 h l 1 ) γ Lemma 2 b j := max( ψ j L, ψ j L ) a min ν u(,y) L2 ( ν + 1)!b ν f L2 Lemma 3 ν (u u h )(,y) L2 h( ν + 2)!b ν f L2 Lemma 4 Important: FE projection depends on y ν G((u u )(,y)) h h 2 ( ν + 5)!b ν f L2 G L2 duality argument Frances Kuo @ UNSW Australia p.29

Estimate the norm (the lognormal case) Single-level algorithm G(u s h ) γ β j := ψ j L = µ j ξ j L Lemma 5 ( ) β ν -1 ν f H -1 u(,y) L2 ν! ln2 a min (y) induction on a 1/2 (,y) ν u(,y) L2 Multi-level algorithm G(u s l h l u s l 1 h l 1 ) γ Lemma 6 β j := max( ψ j L, ψ j L ) induction on a 1/2 (,y) (a(,y) ν u(,y)) L2 ν u(,y) L2 T(y)( ν + 1)!(2β) ν f L2 Lemma 7 a 1/2 (,y) ν (u u h )(,y) L2 ht(y)a 1/2 max (y)( ν +2)!(2β)ν f L2 Lemma 8 ν G((u u )(,y)) h h 2 T 2 (y)a max (y)( ν +5)!(2β) ν f L2 G L2 Frances Kuo @ UNSW Australia p.30

Generic bound on mixed derivative Mixed derivative bound For all multi-indices ν {0,α} s we have ν F(y) ( ( ν + a 1 )! ) d 1 s (a 2 B j ) ν j exp(a 3 B j y j ) j=1 for integers α 1, a 1 0, real numbers a 2 > 0, a 3 0, d 1 0, B j > 0. Particular cases: G(u s h F(y) = ) single-level G(u s h l u s h l 1 ) multi-level B j is b j,b j,β j, or β j a 3 = 0 = uniform case, a 3 > 0 = lognormal case Specify these parameters as input to the construction scripts: http://people.cs.kuleuven.be/~dirk.nuyens/qmc4pde Frances Kuo @ UNSW Australia p.31

MLQMC numerical results (the lognormal case) D = (0,1) 2, f 1, homogeneous Dirichlet boundary G(u(,y)) = 1 D D u(x,y)dx, D = ( 3 4, 7 8 ) (7 8,1) Piecewise linear finite elements, off-the-shelf randomly shifted lattice rule Matérn kernel: smoothness ν, variance σ 2, correlation length scale λ C 10 4 ν = 2.5, σ2 = 0.25, λ = 1 MC MLMC QMC 10 3 MLQMC 10 4 ν = 1.5, σ2 = 1, λ = 0.1 MC MLMC QMC 10 3 MLQMC Cost (in sec) 10 2 10 1 2 3 Cost (in sec) 10 2 10 1 2 3 10 0 1 10 0 10-1 10-4 10-3 10-2 ǫ 10-1 10-4 10-3 10-2 [K., Scheichl, Schwab, Sloan, Ullmann (in review)] ǫ Frances Kuo @ UNSW Australia p.32

Application of QMC to PDEs with random coefficients [0] Graham, K., Nuyens, Scheichl, Sloan (J. Comput. Physics, 2011) [1] K., Schwab, Sloan (SIAM J. Numer. Anal., 2012) [2] K., Schwab, Sloan (J. FoCM, 2015) [3] Graham, K., Nichols, Scheichl, Schwab, Sloan (Numer. Math., 2015) [4] K., Scheichl, Schwab, Sloan, Ullmann (in review) [5] Dick, K., Le Gia, Nuyens, Schwab (SIAM J. Numer. Anal., 2014) [6] Dick, K., Le Gia, Schwab (in review) [7] Graham, K., Nuyens, Scheichl, Sloan (in progress) Uniform Lognormal Lognormal KL expansion KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances Kuo @ UNSW Australia p.33

Concluding remarks QMC error bound can be independent of dimension Three weighted space settings, fast CBC construction Application of QMC theory: estimate the norm, choose the weights Pairing of method and setting: best rate with minimal assumption Theory versus practice Affine parametric operator equations [Schwab (2013)] Special orthogonality Cost reduction: single-level versus multi-level nsh d Circulant embedding nh d log(h d ) Fast QMC matrix-vector multiplication [Dick, K., Le Gia, Schwab (SIAM J. Sci. Comput., 2015)] nlognh d Multi-index methods [Haji-Ali, Nobile, Tempone (Numer. Math., 2015)] Frances Kuo @ UNSW Australia p.34