Hot new directions for QMC research in step with applications

Transcription

1 Hot new directions for QMC research in step with applications Frances Kuo University of New South Wales, Sydney, Australia Frances UNSW Australia p.1

2 Outline Motivating example a flow through a porous medium QMC theory Setting 1 Setting 2 Setting 3 Applications of QMC Application 1 Application 2 Application 3 Cost reduction strategies Saving 1 Saving 2 Saving 3 Concluding remarks Frances UNSW Australia p.2

3 Motivating example Uncertainty in groundwater flow eg. risk analysis of radwaste disposal or CO 2 sequestration Darcy s law q + a p = κ in D R d, d = 1,2,3 mass conservation law q = 0 together with boundary conditions Uncertainty in a(x, ω) leads to uncertainty in q(x, ω) and p(x, ω) Frances UNSW Australia p.3

4 Expected values of quantities of interest To compute the expected value of some quantity of interest: 1. Generate a number of realizations of the random field (Some approximation may be required) 2. For each realization, solve the PDE using e.g. FEM / FVM / mfem 3. Take the average of all solutions from different realizations This describes Monte Carlo simulation. Example: particle dispersion release point p n = 0 p = 1 p = p n = Frances UNSW Australia p.4

5 Expected values of quantities of interest To compute the expected value of some quantity of interest: 1. Generate a number of realizations of the random field (Some approximation may be required) 2. For each realization, solve the PDE using e.g. FEM / FVM / mfem 3. Take the average of all solutions from different realizations This describes Monte Carlo simulation. NOTE: expected value = (high dimensional) integral 10 1 use quasi-monte Carlo methods 10 2 MC n 1/2 error n 1 or better n QMC s = stochastic dimension Frances UNSW Australia p.5

6 Frances UNSW Australia p.6 MC v.s. QMC in the unit cube [0,1] s f(y)dy 1 n n i=1 f(t i ) Monte Carlo method Quasi-Monte Carlo methods t i random uniform t i deterministic n 1/2 convergence close to n 1 convergence or better random points First 64 points of a 2D Sobol sequence A lattice rule with 64 points more effective for earlier variables and lower-order projections order of variables irrelevant order of variables very important use randomized QMC methods for error estimation

7 Frances UNSW Australia p.7 QMC Two main families of QMC methods: (t,m,s)-nets and (t,s)-sequences lattice rules First 64 points of a 2D Sobol sequence A lattice rule with 64 points (0,6,2)-net Having the right number of points in various sub-cubes A group under addition modulo Z and includes the integer points Niederreiter book (1992) Sloan and Joe book (1994) Important developments: component-by-component (CBC) construction, fast CBC higher order digital nets Dick and Pillichshammer book (2010) Dick, K., Sloan Acta Numerica (2013) Nuyens and Cools (2006) Dick (2008) Contribute to: Information-Based Complexity Tractability Traub, Wasilkowski, Woźniakowski book (1988) Novak and Woźniakowski books (2008, 2010, 2012)

8 Application of QMC to PDEs with random coefficients [0] Graham, K., Nuyens, Scheichl, Sloan (J. Comput. Physics, 2011) [1] K., Schwab, Sloan (SIAM J. Numer. Anal., 2012) [2] K., Schwab, Sloan (J. FoCM, 2015) [3] Graham, K., Nichols, Scheichl, Schwab, Sloan (Numer. Math., 2015) [4] K., Scheichl, Schwab, Sloan, Ullmann (in review) [5] Dick, K., Le Gia, Nuyens, Schwab (SIAM J. Numer. Anal., 2014) [6] Dick, K., Le Gia, Schwab (SIAM J. Numer. Anal., to appear) [7] Graham, K., Nuyens, Scheichl, Sloan (in progress) Also Schwab (Proceedings of MCQMC 2012) Le Gia (Proceedings of MCQMC 2012) Dick, Le Gia, Schwab (in review, in review, in progress) Harbrecht, Peters, Siebenmorgen (Math. Comp., 2015) Gantner, Schwab (Proceedings of MCQMC 2014) Ganesh, Hawkings (SIAM J. Sci. Comput., 2015) Robbe, Nuyens, Vandewalle (in review) Scheichl, Stuart, Teckentrup (in review) Gilbert, Graham, K., Scheichl, Sloan (in progress) Graham, Parkinson, Scheichl (in progress) etc. Frances UNSW Australia p.8

9 Application of QMC to PDEs with random coefficients Three different QMC theoretical settings: [1,2] Weighted Sobolev space over [0,1] s and randomly shifted lattice rules [3,4] Weighted space setting in R s and randomly shifted lattice rules [5,6] Weighted space of smooth functions over [0,1] s and (deterministic) interlaced polynomial lattice rules K., Nuyens (J. FoCM, to appear) a survey of [1,2], [3,4], [5,6] in a unified view Accompanying software package Uniform Lognormal Lognormal KL expansion Circulant embedding Experiments only [0]* First order, single-level [1] [3]* [7]* First order, multi-level [2] [4]* Higher order, single-level [5] Higher order, multi-level [6]* The * indicates there are accompanying numerical results Frances UNSW Australia p.9

10 5-minute QMC primer (most important slide) Common features among all three QMC theoretical settings: Separation in error bound (rms) QMC error (rms) worst case error γ norm of integrand γ Weighted spaces Pairing of QMC rule with function space Optimal rate of convergence Rate and constant independent of dimension Fast component-by-component (CBC) construction Extensible or embedded rules Application of QMC theory Estimate the norm (critical step) Choose the weights Weights as input to the CBC construction Frances UNSW Australia p.10

11 Lattice rules Rank-1 lattice rules have points ( ) i t i = frac n z, i = 1,2,...,n z Z s the generating vector, with all components coprime to n frac( ) means to take the fractional part of all components quality determined by the choice of z n = 64 z = (1,19) t i = frac ( ) i 64 (1,19) A lattice rule with 64 points Frances UNSW Australia p.11

12 Frances UNSW Australia p.12 Randomly shifted lattice rules Shifted rank-1 lattice rules have points t i = frac ( i n z + ), i = 1,2,...,n [0,1) s the shift shifted by = (0.1,0.3) A lattice rule with 64 points A shifted lattice rule with 64 points use a number of random shifts for error estimation

13 Component-by-component construction Want to find z for which some error criterion is as small possible Exhaustive search is practically impossible - too many choices! CBC algorithm [Korobov (1959); Sloan, Reztsov (2002); Sloan, K., Joe (2002),... ] 1. Set z 1 = With z 1 fixed, choose z 2 to minimize the error criterion in 2D. 3. With z 1,z 2 fixed, choose z 3 to minimize the error criterion in 3D. 4. etc. [K. (2003); Dick (2004)] Optimal rate of convergence O(n 1+δ ) in weighted Sobolev space, independently of s under an appropriate condition on the weights Averaging argument: there is always one choice as good as average! [Nuyens, Cools (2006)] Cost of algorithm for product weights is O(nlogns) using FFT [Hickernell, Hong, L Ecuyer, Lemieux (2000); Hickernell, Niederreiter (2003)] Extensible/embedded variants [Cools, K., Nuyens (2006)] [Dick, Pillichshammer, Waterhouse (2007)] Frances UNSW Australia p.13

14 Fast CBC construction n = 53 Natural ordering of the indices 1 Generator ordering of the indices 53 Matrix-vector multiplication with a circulant matrix can be done using FFT Frances UNSW Australia p.14

15 Setting 1: standard QMC for unit cube Worst case error bound f(y)dy 1 [0,1] s n n f(t i ) ewor γ (t 1,...,t n ) f γ i=1 Weighted Sobolev space [Sloan, Woźniakowski (1998)] f 2 γ = 1 u 2 f (y u ;0) dy y u u 2 s subsets u {1:s} γ u [0,1] u anchor at 0 (also unanchored ) weights Mixed first derivatives are square integrable Small weight γ u means that f depends weakly on the variables y u Pair with randomly shifted lattice rules Choose weights to minimize the error bound [K., Schwab, Sloan (2012)] ( ) 1/(2λ) ( ) 1/2 2 ( γ λ u n a b u bu u γ u = u {1:s} }{{} bound on worst case error (CBC) u {1:s} Frances UNSW Australia p.15 γ u } {{ } bound on norm Construct points (CBC) to minimize the worst case error a u ) 1/(1+λ) POD weights

16 Setting 2: QMC integration over R s Change of variables s f(y) φ(y j )dy = R s j=1 f(φ 1 (w))dw [0,1] s Norm with weight function [Wasilkowski, Woźniakowski (2000)] f 2 γ = 1 u 2 f (y u ;0) ϕ 2 y u j (y j)dy u u {1:s} γ u R u j u also unanchored weight function [K., Sloan, Wasilkowski, Waterhouse (2010); Nichols, K. (2014)] Pair with randomly shifted lattice rules CBC error bound for general weights γ u Convergence rate depends on the relationship between φ and ϕ j Fast CBC for POD weights Important for applications: φ, ϕ j, γ u are up to us to choose/tune Frances UNSW Australia p.16

17 Setting 3: smooth integrands in the cube Norm involving higher order mixed derivatives Pair with higher order digital nets [Dick (2008)] Classical polynomial lattice rule [Niederreiter (1992)] n = b m points with prime b An irreducible polynomial with degree m A generating vector of s polynomials with degree < m Interlaced polynomial lattice rule [Goda, Dick (2012)] Interlacing factor α An irreducible polynomial with degree m A generating vector of αs polynomials with degree < m Digit interlacing function D α : [0,1) α [0,1) (0.x 11 x 12 x 13 ) b,(0.x 21 x 22 x 23 ) b,..., (0.x α1 x α2 x α3 ) b becomes (0.x 11 x 21 x α1 x 12 x 22 x α2 x 13 x 23 x α3 ) b [Dick, K., Le Gia, Nuyens, Schwab (2014)] Variants of fast CBC construction available SPOD weights Frances UNSW Australia p.17

18 Overview Application 1 Option pricing Application 2 GLMM maximum likelihood Application 3 PDEs with random coefficients Setting 1 domain [0,1] s randomly shifted lattice rules mixed first derivatives first order convergence Setting 2 domain R s randomly shifted lattice rules mixed first derivatives first order convergence Setting 3 domain [0,1] s interlaced polynomial lattice rules mixed higher derivatives higher order convergence Frances UNSW Australia p.18

19 Application 1: QMC for option pricing Path-dependent option [Giles, K., Sloan, Waterhouse (2008)] ( 1 max R s s s j=1 s = f(y) φ nor (y j )dy R s j=1 = g(w)dw [0,1] s ) exp( 1 2 S j (z) K,0 zt Σ 1 z) (2π) s det(σ) dz Write Σ = AA T Sub. z = Ay Sub. y = Φ -1 nor (w) In the cube, g is unbounded and has a kink standard error MC QMC + PCA naive QMC n ANOVA decomposition: f(y) = u {1:s} f u(y u ) All f u with u {1 : s} are smooth under BB [Griebel, K., Sloan (2010, 2013, 2016)] Smoothing by preintegration (P k f)(y) = R [Griewank, Leövey, K., Sloan (in progress)] f(y)φ nor (y k )dy k is smooth for some choice of k [Ian Sloan Mon 4:50pm Berg A] Frances UNSW Australia p.19

20 Application 2: QMC for maximum likelihood Generalized linear mixed model [K., Dunsmuir, Sloan, Wand, Womersley (2008)] R s ( s j=1 exp(τ j (β + z j ) e β+z j) τ j! s = f(y) φ(y j )dy R s j=1 = g(w)dw [0,1] s ) exp( 1 2 zt Σ 1 z) (2π) s det(σ) dz Re-center and re-scale, and multiply and divide by φ Sub. z = A y + z Sub. y = Φ -1 (w) cf. importance sampling no re-centering (worst) no re-scaling (bad) φ normal (good) φ logistic (better) φ Student-t (best) In the cube, g and/or its derivatives are unbounded New weighted space setting over R s [K., Sloan, Wasilkowski, Waterhouse (2010)] [Nichols, K. (2014)] Differentiate f to estimate the norm [Sinescu, K., Sloan (2013)] Frances UNSW Australia p.20

21 Application 3: QMC for PDEs with random coeff. (a(x,y) u(x,y)) = κ(x), x D R d, d = 1,2,3 u(x,y) = 0, x D Uniform [Cohen, DeVore, Schwab (2011)] a(x,y) = a(x) + j 1y j ψ j (x), y j U[ 1 2, 1 2 ] Goal: ( 1 2,1 2 )N G(u(,y))dy G bounded linear functional Lognormal a(x,y) = exp(z(x,y)) Karhunen Loève expansion: Z(x,y) = Z(x) + j 1y j µj ξ j (x), y j N(0,1) Goal: R N G(u(,y)) j 1 φ nor (y j )dy Frances UNSW Australia p.21

22 Application 3: QMC for PDEs with random coeff. Error analysis G(u(,y))dy 1 ( 1 2,1 2 )N n n G(u s h (,t i )) i=1 Balance three sources of error [K., Nuyens (2016) survey] (1) dimension truncation (2) FE discretization (3) QMC quadrature QMC error Estimate the norm G(u s h ) γ by differentiating the PDE Choose the weights to minimize (bound on e wor γ ) (bound on G(u s h ) γ) Get POD weights (product and order dependent) feed into fast CBC construction j 1 ψ j p < O(n min( 1 p 1 2,1 δ) ) convergence, independently of s Improved to O(n 1 p ) convergence, independently of s, with higher order QMC and SPOD weights (smoothness-driven product and order dependent) Lognormal results depend on smoothness of covariance function Pairing of QMC rule with function space is crucial Want the best convergence rate under the least assumption Want the convergence rate and implied constant to be independent of s Frances UNSW Australia p.22

23 Application 3: QMC for PDEs with random coeff. Extensions Lognormal a(x, y) = exp(z(x, y)) circulant embedding [Graham, K., Nuyens, Scheichl, Sloan (in progress)] Z(x i,y) for i = 1,...,M M M covariance matrix R Assume stationary covariance function and regular grid Extend R to an s s circulant matrix R ext = AA T Compute matrix-vector multiplication Ay using FFT H-matrix [Feischl, K., Sloan (in progress)] Does not require stationary covariance function or regular grid Approximate R by an H 2 -matrix of interpolation order p, R p = BB T Iterative method to compute By to some prescribed accuracy [Michael Feischl Fri 3:00pm Berg C] QMC for PDE eigenvalue problems (neutron transport) [Gilbert, Graham, K., Scheichl, Sloan (in progress)] QMC for wave propagation models QMC for Bayesian inverse problems [Ganesh, K., Nuyens, Sloan (in progress)] [several research groups] Frances UNSW Australia p.23

24 Overview Application 1 Option pricing Application 2 GLMM maximum likelihood Application 3 PDEs with random coefficients uniform lognormal smoothing by preintegration integral transformation KL expansion circulant embedding H-matrix Cost reduction Saving 1 Saving 2 Saving 3 Setting 1 domain [0,1] s randomly shifted lattice rules mixed first derivatives first order convergence Setting 2 domain R s randomly shifted lattice rules mixed first derivatives first order convergence Setting 3 domain [0,1] s interlaced polynomial lattice rules mixed higher derivatives higher order convergence Frances UNSW Australia p.24

25 Saving 1: multi-level and multi-index Multi-level method [Heinrich (1998); Giles (2008);... ; Giles (2016)] Telescoping sum f = (f l f l 1 ), f 1 := 0 A ML (f) = l=0 l=0 I(f) A ML (f) I(f) I L (f L ) + }{{} 1 2 ε choose L L Q l (f l f l 1 ) L (I l Q l )(f l f l 1 ) l=0 }{{} 1 2 ε specify parameters for Q l Successive differences f l f l 1 become smaller Require less samples for higher dimensional sub-problems Lagrange multiplier to minimize cost subject to a given error threshold Multi-index method [Haji-Ali, Nobile, Tempone (2016)] Apply multi-level simultaneously to more than one aspect of the problem e.g., quadrature, spatial discretization, time discretization, etc. Frances UNSW Australia p.25

26 Saving 2: MDM Multivariate decomposition method [... ; Wasilkowski (2013);... ] [K., Nuyens, Plaskota, Sloan, Wasilkowski (in progress)] Decomposition f = u A MDM (f) = u {1,2,...}, u < f Q u (f u ) u A I(f) A MDM (f) I u (f u ) + (I u Q u )(f u ) u/ A }{{} 1 2 ε choose A Frances UNSW Australia p.26 u A }{{} 1 2 ε specify parameters for Q u Assume bounds on the norms f u Fu B u are given Q u can be QMC or other quadratures including Smolyak Cardinality of active set A might be huge, but cardinality of u A is small Implementation [Gilbert, K., Nuyens, Wasilkowski (in progress)] Anchored decompositionf u (x u ) = v u ( 1) u v f(x v ;a) Explore nestedness/embedding in the quadratures Efficient data structure to reuse function evaluations Application to PDEs with random coefficients [Alexander Gilbert Mon 2:25pm Berg A]

27 Saving 3: QMC fast matrix-vector mult. QMC fast matrix-vector multiplication [Dick, K., Le Gia, Schwab (2015)] Want to compute y i A for all i = 1,...,n Row vectors y i = ϕ(t i ) where t i are QMC points y 1 C (n 1) (n 1) is circulant Write Y =.. = CP P has a single 1 in each column y n 1 and 0 everywhere else Can compute Y a in O(nlogn) operations for any column a of A When does/doesn t this work? deterministic lattice points, deterministic polynomial lattice points inverse cumulative normal, tent transform does not work with random shifting, scrambling, interlacing Where does this appy? Generating normally distributed points with a general covariance matrix PDEs with uniform random coefficients PDEs with lognormal random coefficients involving FE quadrature Frances UNSW Australia p.27

28 Summary and concluding remarks Application 1 Option pricing Application 2 GLMM maximum likelihood Application 3 PDEs with random coefficients uniform lognormal smoothing by preintegration integral transformation KL expansion circulant embedding H-matrix Setting 1 domain [0,1] s randomly shifted lattice rules mixed first derivatives first order convergence Setting 2 domain R s randomly shifted lattice rules mixed first derivatives first order convergence Setting 3 domain [0,1] s Cost reduction Saving 1 multi-level and multi-index methods Saving 2 multivariate decomposition method (MDM) Saving 3 QMC fast matrix-vector multiplication interlaced polynomial lattice rules mixed higher derivatives higher order convergence Frances UNSW Australia p.28

29 Setting 4: higher order QMC for R s??? Dick, Leobacher, Irrgeher, Pillichshammer [Christian Irrgeher Mon 5:20pm Berg A] Cools, Nguyen, Nuyens [Dong Nguyen MCM Linz 2015] Stretch the unit cube Truncate the integrand Apply higher order QMC without Φ -1 mapping s dependence??? Frances UNSW Australia p.29

30 Thanks to my collaborators AU Austria Belgium Ecuador Germany NZ Poland Switzerland UK USA Frances UNSW Australia p.30