Stochastic methods for solving partial differential equations in high dimension

Transcription

1 Stochastic methods for solving partial differential equations in high dimension Marie Billaud-Friess Joint work with : A. Macherey, A. Nouy & C. Prieur marie.billaud-friess@ec-nantes.fr Centrale Nantes, Laboratoire de Mathématiques Jean Leray July 2 nd, th International Conference in Monte Carlo & Quasi-Monte Carlo Methods in Scientific Computing

2 General context High-dimensional problem Find u solution of L(u) = g in D, u = f on D. (1) u : D R multivariate function of x = (x 1,..., x d ) on D R d L linear differential operator f, g : D R boundary condition and source term respectively M. Billaud-Friess MCQMC 18 Introduction 2/28

3 General context High-dimensional problem Find u solution of L(u) = g in D, u = f on D. (1) u : D R multivariate function of x = (x 1,..., x d ) on D R d L linear differential operator f, g : D R boundary condition and source term respectively Main challenges : 1 How to approximate u up to precision ε with reasonable computational cost? 2 In particular for high dimensions (d 1) while overcoming the so-called curse of dimensionality M. Billaud-Friess MCQMC 18 Introduction 2/28

4 Solving high dimensional problems Deterministic way... Framework : Nonlinear and sparse approximation M. Billaud-Friess MCQMC 18 Introduction 3/28

5 Solving high dimensional problems Deterministic way... Framework : Nonlinear and sparse approximation 1 Tensor based methods : General tensor networks with application in physics [Verstarete,Vidal]... Low-rank approximation methods [Bachmayer, Dahmen, Grasedyck, Hackbush, Khoromskij, Kressner, Matthies, Nouy, Oseledets, Schwab, Schneider, Uschmajew... ] M. Billaud-Friess MCQMC 18 Introduction 3/28

6 Solving high dimensional problems Deterministic way... Framework : Nonlinear and sparse approximation 1 Tensor based methods : General tensor networks with application in physics [Verstarete,Vidal]... Low-rank approximation methods [Bachmayer, Dahmen, Grasedyck, Hackbush, Khoromskij, Kressner, Matthies, Nouy, Oseledets, Schwab, Schneider, Uschmajew... ] 2 Sparse (tensor) approximation : Contributions for parameter dependent PDEs [Chkifa, Cohen, De Vore, Nobile, Schwab,... ] u(x) u Λ(x) = α νϕ ν(x) = ν Λ ν Λ α ν j 1 ϕ ν ν j (x j), How to compute u Λ? Polynomial expansions e.g [Chkifa 13, Cohen 15] Projection based methods : Galerkin e.g. with multilevel FE, wavelets [Schwab 11], least-square or polynomial interpolation [Chkifa 14] M. Billaud-Friess MCQMC 18 Introduction 3/28

7 Solving high dimensional problems Deterministic way... Framework : Nonlinear and sparse approximation 1 Tensor based methods : General tensor networks with application in physics [Verstarete,Vidal]... Low-rank approximation methods [Bachmayer, Dahmen, Grasedyck, Hackbush, Khoromskij, Kressner, Matthies, Nouy, Oseledets, Schwab, Schneider, Uschmajew... ] 2 Sparse (tensor) approximation : Contributions for parameter dependent PDEs [Chkifa, Cohen, De Vore, Nobile, Schwab,... ] u(x) u Λ(x) = ν Λ α νx ν = ν Λ α ν j 1 x ν j j, How to compute u Λ? Polynomial expansions e.g [Chkifa 13, Cohen 15] Projection based methods : Galerkin e.g. with multilevel FE, wavelets [Schwab 11], least-square or polynomial interpolation [Chkifa 14] Sparse polynomial interpolation : ϕ ν(x) = x ν Sample based methods requiring evaluation of u at some points of D Adaptive selection of Λ leading to sparse polynomial space [Chkifa 13, Chkifa 14] M. Billaud-Friess MCQMC 18 Introduction 3/28

8 Solving high dimensional problems Stochastic way... Single point estimations of u at x D : Combine probabilistic representation of u(x) together with Monte Carlo estimation [Graham 13, Gobet 13] u(x) u M (x) = 1 M M ψ(f, g, X x (ω m)), with X x (ω m) a realization of X x the diffusion process starting from x at t = 0. m=1 M. Billaud-Friess MCQMC 18 Introduction 4/28

9 Solving high dimensional problems Stochastic way... Single point estimations of u at x D : Combine probabilistic representation of u(x) together with Monte Carlo estimation [Graham 13, Gobet 13] u(x) u M (x) = 1 M M ψ(f, g, X x (ω m)), with X x (ω m) a realization of X x the diffusion process starting from x at t = 0. Toward a global approximation of u over D : m=1 Interpolation combined together with sequential variance reduction technique [Gobet 04, Gobet 09] Limited to small dimension! Deep learning based on artificial neural network approximations for linear and nonlinear parabolic high-dimensional problems [Beck 17, Weinan 17, Beck 18] Efficient method, but no rigorous analysis! M. Billaud-Friess MCQMC 18 Introduction 4/28

10 Outline Goal Gather a probabilistic approach for pointwise estimation of the solution together with a sparse interpolation method to compute global approximation to solution of highdimensional partial differential equations. 1 A sequential algorithm for variance reduction 2 A sequential algorithm in high dimension 3 A perturbed sparse adaptive algorithm M. Billaud-Friess MCQMC 18 Introduction 5/28

11 Outline 1 A sequential algorithm for variance reduction 2 A sequential algorithm in high dimension 3 A perturbed sparse adaptive algorithm M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 6/28

12 Pointwise estimation of u(x) Let define L in (1) by ( L = 1 d (σ(x)σ(x) T ) ij x 2 2 i x j + i,j=1 ) d b i(x) xi u(x) + k(x) as the infinitesimal generator of X x the d-dimensional diffusion process given by dx x t = b(x x t )dt + σ(x x t )dw t, X 0 = x D. where W is a d-dimensional brownian motion. i=1 M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 7/28

13 Pointwise estimation of u(x) Let define L in ( (1) by L = 1 d (σ(x)σ(x) T ) ij x 2 2 i x j + i,j=1 ) d b i(x) xi u(x) + k(x) as the infinitesimal generator of X x the d-dimensional diffusion process given by i=1 dx x t = b(x x t )dt + σ(x x t )dw t, X 0 = x D. where W is a d-dimensional brownian motion. Feynman-Kac formula Assuming that i) b, σ are lipschitz, ii) f, g : D R are continuous functions, iii) there exists u : D R in C 2 on all open subsets of D solution of (1), then for x D we have u(x) = E(ψ(f, g, X x )) := E with τ x = inf{s > 0 : X x s / D} the first exit time of D. ( f(x x τ x )e τx ) τx 0 k(x x r )dt + g(x x s )e s 0 k(x x r )dr ds. 0 M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 7/28

14 Pointwise estimation of u(x) Let define L in ( (1) by L = 1 d (σ(x)σ(x) T ) ij x 2 2 i x j + i,j=1 ) d b i(x) xi u(x) + k(x) as the infinitesimal generator of X x the d-dimensional diffusion process given by i=1 dx x t = b(x x t )dt + σ(x x t )dw t, X 0 = x D. where W is a d-dimensional brownian motion. Feynman-Kac formula Assuming that i) b, σ are lipschitz, ii) f, g : D R are continuous functions, iii) there exists u : D R in C 2 on all open subsets of D solution of (1), then for x D we have u(x) = E(φ(u, X x )) := E ( u(x x τ x )e τx ) τx 0 k(x x r )dt + Lu(X x s )e s 0 k(x x r )dr ds. 0 with τ x = inf{s > 0 : X x s / D} the first exit time of D. M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 7/28

15 In practice u(x) u t,m (x) Estimation of the expectation by Monte-Carlo simulation Noting 0 = t 0 < t 1 <... with t n = n t, n = 0, 1,..., let X x X x, t where = Xx n is given by Euler-Maruyama scheme X t,x t n where W n = W t n+1 W t n. X x n+1 = X x n + t b(x x n) + σ(x x n) W n, Let { X t,x (ω } M m), M independent realisations of X t,x m=1 t n u t,m (x) = 1 M M u(x t,x τx t m=1 N 1 (ω m)) + t i=0 L(u)(X t,x (ω t m))1 i t i τx t. M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 8/28

16 In practice u(x) u t,m (x) Estimation of the expectation by Monte-Carlo simulation Noting 0 = t 0 < t 1 <... with t n = n t, n = 0, 1,..., let X x X x, t where = Xx n is given by Euler-Maruyama scheme X t,x t n where W n = W t n+1 W t n. X x n+1 = X x n + t b(x x n) + σ(x x n) W n, Let { X t,x (ω } M m), M independent realisations of X t,x m=1 t n u t,m (x) := 1 M M φ(u, X t,x )(ω m). m=1 M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 8/28

17 In practice u(x) u t,m (x) Estimation of the expectation by Monte-Carlo simulation Noting 0 = t 0 < t 1 <... with t n = n t, n = 0, 1,..., let X x X x, t where = Xx n is given by Euler-Maruyama scheme X t,x t n where W n = W t n+1 W t n. X x n+1 = X x n + t b(x x n) + σ(x x n) W n, Let { X t,x (ω } M m), M independent realisations of X t,x m=1 t n u t,m (x) := 1 M φ(u, X t,x )(ω m). M m=1 Error : Integration error O( ( ) 1 t) + MC error O M Slow convergence w.r.t. M with large error for large variance V(u t,m (x)) M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 8/28

18 In practice u(x) u t,m (x) Estimation of the expectation by Monte-Carlo simulation Noting 0 = t 0 < t 1 <... with t n = n t, n = 0, 1,..., let X x X x, t where = Xx n is given by Euler-Maruyama scheme X t,x t n where W n = W t n+1 W t n. X x n+1 = X x n + t b(x x n) + σ(x x n) W n, Let { X t,x (ω } M m), M independent realisations of X t,x m=1 t n u t,m (x) := 1 M φ(u, X t,x )(ω m). M m=1 Error : Integration error O( ( ) 1 t) + MC error O M Slow convergence w.r.t. M with large error for large variance V(u t,m (x)) Improving the convergence : Multilevel MC [Giles 08], Variance reduction : control variate, importance sampling, antithetic sampling [Gobet 13, Graham 13],... M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 8/28

19 Sequential algorithm for variance reduction Gobet-Maire Algorithm [Gobet 04, Gobet 09] Notations : Let v be a smooth real-valued function defined on D the solution of an EDP under the form (1) Stochastic approximation of v at x D : v t,m (x) Approximation of v at step k of the algorithm : ṽ k Linear approximation (e.g. interpolant) of v using pointwise evaluations at {z i} N i=1 D for some basis functions {l i} N i=1 : I(v) = N v(z i)l i(x). i=1 M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 9/28

20 Algorithm 1. Gobet & Maire algorithm 1. Set ũ 0 = For k = 1,..., K : Compute e k t,m (z i) e k (z i), i = 1,..., N where e k = u ũ k is s.t. Define ẽ k = L(x)e k (x) = g(x) Lũ k (x), x D, e k (x) = f(x) ũ k (x), x D. N e k t,m (z i)l i(x) i=1 Update ũ k+1 = ũ k + ẽ k M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 10/28

21 Algorithm 1. Gobet & Maire algorithm 1. Set ũ 0 = For k = 1,..., K : Remarks : Compute e k t,m (z i) e k (z i), i = 1,..., N where e k = u ũ k is s.t. Define ẽ k = L(x)e k (x) = g(x) Lũ k (x), x D, e k (x) = f(x) ũ k (x), x D. N e k t,m (z i)l i(x) i=1 Update ũ k+1 = ũ k + ẽ k For t small enough, max i=1,...,n E(ũk (z i) z i) and V(ũ k (x i)) converge geometrically with k, up to threshold term. [Gobet 09]. Algorithm designed for small d. M. Billaud-Friess MCQMC 18 A sequential algorithm for variance reduction 10/28

22 Outline 1 A sequential algorithm for variance reduction 2 A sequential algorithm in high dimension 3 A perturbed sparse adaptive algorithm M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 11/28

23 Going to high dimension Sparse interpolation (in brief) [Chkifa 14] Let u : D R where D = [ 1, 1] d. M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 12/28

24 Going to high dimension Sparse interpolation (in brief) [Chkifa 14] Let u : D R where D = [ 1, 1] d. 1 Given a finite set of multi-indices ν = (ν 1,..., ν d ) noted Λ N d that is downward closed ν Λ, µ ν µ Λ, construct by tensorisation the multivariate polynomial P ν associated to ν P ν(x) = d p νi (x i) where x i p νi (x i) are univariate polynomial basis (e.g. Legendre). i=1 M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 12/28

25 Going to high dimension Sparse interpolation (in brief) [Chkifa 14] Let u : D R where D = [ 1, 1] d. 1 Given a finite set of multi-indices ν = (ν 1,..., ν d ) noted Λ N d that is downward closed ν Λ, µ ν µ Λ, construct by tensorisation the multivariate polynomial P ν associated to ν P ν(x) = d p νi (x i) where x i p νi (x i) are univariate polynomial basis (e.g. Legendre). 2 The interpolant I Λ(u) of u in P Λ is given by i=1 I Λ(u) = ν Λ α νp ν. It is associated with interpolation points (e.g. Leja, Magic Points [Maday 09]) {z ν} ν Λ [ 1, 1] d unisolvent for P Λ = span{p ν, ν Λ}, M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 12/28

26 Going to high dimension Sparse interpolation (in brief) [Chkifa 14] Let u : D R where D = [ 1, 1] d. 1 Given a finite set of multi-indices ν = (ν 1,..., ν d ) noted Λ N d that is downward closed ν Λ, µ ν µ Λ, construct by tensorisation the multivariate polynomial P ν associated to ν P ν(x) = d p νi (x i) where x i p νi (x i) are univariate polynomial basis (e.g. Legendre). 2 The interpolant I Λ(u) of u in P Λ is given by i=1 I Λ(u) = ν Λ u(z ν)l ν. It is associated with interpolation points (e.g. Leja, Magic Points [Maday 09]) {z ν} ν Λ [ 1, 1] d unisolvent for P Λ = span{p ν, ν Λ}, and {l ν} ν is a basis of P Λ s.t. l ν(z µ) = δ νµ. M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 12/28

27 Going to high dimension Adaptive selection of Λ and construction of I Λ (u) [Chkifa 13, Chkifa 14] Algorithm 2. Adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and n = While n < N and ε n 1 > ε : Define M n. Set Λ = Λ n 1 M n and compute I Λ (u). Select N n = {ν M n; E ν(i Λ (u)) θe Mn (I Λ (u))} Update Λ n = Λ n 1 N n. Compute I Λn (u) and ε n Update n = n + 1 M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 13/28

28 Going to high dimension Adaptive selection of Λ and construction of I Λ (u) [Chkifa 13, Chkifa 14] Algorithm 2. Adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and n = While n < N and ε n 1 > ε : Define M n. Set Λ = Λ n 1 M n and compute I Λ (u). Select N n = {ν M n; E ν(i Λ (u)) θe Mn (I Λ (u))} Update Λ n = Λ n 1 N n. Compute I Λn (u) and ε n Update n = n + 1 Remarks : 1 Using the reduced margin of Λ n we define M n = {ν / Λ n 1, ν j 0 ν e j Λ n 1}. M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 13/28

29 Going to high dimension Adaptive selection of Λ and construction of I Λ (u) [Chkifa 13, Chkifa 14] Algorithm 2. Adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and n = While n < N and ε n 1 > ε : Define M n. Set Λ = Λ n 1 M n and compute I Λ (u). Select N n = {ν M n; E ν(i Λ (u)) θe Mn (I Λ (u))} Update Λ n = Λ n 1 N n. Compute I Λn (u) and ε n Update n = n + 1 Remarks : 2 N n selected using a bulk chasing algorithm, ensuring Λ n downward closed, where E S(I Λ (u)) = β 2 ν i ν i S measures the norm of the interpolant coefficients {β ν j } ν j Λ decreasing values, associated to multi-indices in S. of IΛ (u) ordered by M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 13/28

30 Going to high dimension Adaptive selection of Λ and construction of I Λ (u) [Chkifa 13, Chkifa 14] Algorithm 2. Adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and n = While n < N and ε n 1 > ε : Define M n. Set Λ = Λ n 1 M n and compute I Λ (u). Select N n = {ν M n; E ν(i Λ (u)) θe Mn (I Λ (u))} Update Λ n = Λ n 1 N n. Compute I Λn (u) and ε n Update n = n + 1 Remarks : 3 Here ε n = ν M n α 2 ν ν Λ α 2 ν M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 13/28

31 Gobet-Maire algorithm in high-dimension Algorithm 1. (High dimension Gobet & Maire algorithm) 1. Set ũ 0 = For k = 1,..., K : Set e k = u ũ k is s.t. Interpolate ẽ k Update L(x)e k (x) = g(x) Lũ k (x), x D, e k (x) = f(x) ũ k (x), x D. ũ k+1 = ũ k + ẽ k Remark : Here ẽ k = I ε Λ k (e k t,m ) computed using Algorithm 2 for given precision ε using realizations {e k t,m (z ν)} ν. M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 14/28

32 Numerical results Problem setting Laplacian in d dimensions in D = [ 1, 1] d u(x) = g(x) x D, u(x) = f(x) x D, (2) M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 15/28

33 Numerical results Problem setting Laplacian in d dimensions in D = [ 1, 1] d u(x) = g(x) x D, u(x) = f(x) x D, (2) Tested methods : Let Λ = {ν N d, ν p}. Method 1 : No adaptive selection of multi-indices : ẽ k = I Λ(e k t,m ) Method 2 : Adaptive selection of multi-indices with Λ k Λ s.t. θ = 0.5 : ẽ k = I ε Λ (e k k t,m ) where ε = M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 15/28

34 Numerical results Problem setting Laplacian in d dimensions in D = [ 1, 1] d u(x) = g(x) x D, u(x) = f(x) x D, (2) Tested methods : Let Λ = {ν N d, ν p}. Method 1 : No adaptive selection of multi-indices : ẽ k = I Λ(e k t,m ) Method 2 : Adaptive selection of multi-indices with Λ k Λ s.t. θ = 0.5 : ẽ k = I ε Λ (e k k t,m ) where ε = Test configurations : Test 1 : u(x) = x 2 2, and d = 5, p = 2 Test 2 : u(x) = x sin(x 2) + exp(x 3) + sin(x 4)(x 5 + 1), and d = 5, p = 10. M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 15/28

35 Numerical results Method 1, test n 1, d = 5, p = 2, #Λ = M= M=500 M=1000 M=1000 u ũ k M=2000 u ũ k M= iteration k iteration k 10 1 dt= dt=0.1 dt=0.01 dt=0.01 u ũ k dt=0.001 dt= u ũ k dt=0.001 dt= iteration k iteration k Figure Test n 1 : Evolution of errors w.r.t. t (left) and M (right). 1 Convergence in few iterations 2 Error decreases with respect to M and t but finally stagnates M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 16/28

36 Numerical results Method 2, test n 1 d = 5, p = 2 ũ k u dt=0.1 dt=0.01 dt=0.001 ũ k u dt=0.1 dt=0.01 dt=0.001 u ũ k iteration k M=500 M=1000 M=2000 u ũ k iteration k M=500 M=1000 M= iteration k iteration k Figure Test n 1 : Evolution of errors w.r.t. t (left) and M (right). 1 Convergence with respect to M and t 2 Slower convergence w.r.t. to k then for non adapted Λ M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 17/28

37 Numerical results Method 2, test n 2 d = 5, p = 10 and t = 0.001, M = ũ k u 2 ũ k u #Λk iteration k iteration k Figure Test n 2 : Errors (left) and evolution of #Λ k (right). 1 The method converges with respect to k 2 Reasonnable #Λ k 200 during iterations M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 18/28

38 Error analysis for Λ k = Λ 1 Pointwise error Notations : Let a : D R be a smooth function, the integration error is e(a, t, x) = E(φ(a, X t,x ) φ(a, X x )). M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 19/28

39 Error analysis for Λ k = Λ 1 Pointwise error Notations : Let a : D R be a smooth function, the integration error is Pointwise error : We have e(a, t, x) = E(φ(a, X t,x ) φ(a, X x )). E(ũ k+1 (z ν) u(z ν)) = ν Λ E(u(z ν) ũ k (z ν))e(l ν, t, z ν) + e(u I Λ(u), t, z ν) M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 19/28

40 Error analysis for Λ k = Λ 1 Pointwise error Notations : Let a : D R be a smooth function, the integration error is Pointwise error : We have e(a, t, x) = E(φ(a, X t,x ) φ(a, X x )). E(ũ k+1 (z ν) u(z ν)) = ν Λ E(u(z ν) ũ k (z ν))e(l ν, t, z ν) + e(u I Λ(u), t, z ν) Taking absolute value and the supremum over ν with m k = sup E(ũ k+1 (z ν) u(z ν)), ν ρ m = sup e(l ν, t, zν), ν ν Λ r( t, u I Λ (u)) = sup e(u I Λ (u), t, z ν) ν m k+1 m k ρ m + r( t, u I Λ(u)). (3) M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 19/28

41 Error analysis for Λ k = Λ 1 Pointwise error Notations : Let a : D R be a smooth function, the integration error is Pointwise error : We have e(a, t, x) = E(φ(a, X t,x ) φ(a, X x )). E(ũ k+1 (z ν) u(z ν)) = ν Λ E(u(z ν) ũ k (z ν))e(l ν, t, z ν) + e(u I Λ(u), t, z ν) Taking absolute value and the supremum over ν m k+1 m k ρ m + r( t, u I Λ(u)). (3) Convergence theorem [Gobet 09] For t small enough s.t. ρ m < 1, {m k } k 0 converges geometrically at rate ρ m up to a threshold term that vanishes as e(u I(u), t, z ν) goes to 0 : 1 lim sup m k r( t, u I Λ(u)). k 1 ρ m M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 19/28

42 Error analysis for Λ k = Λ 2 Global error Starting from ( ) E ũ k+1 I Λ(u)(x) = E(ũ k+1 (z ν) u(z ν))l ν(x) (4) ν Λ we get by taking the absolute value and then the supremum over D x D ν Λ sup E(ũ k+1 I Λ(u))(x) m k+1 L Λ (5) x D were L Λ = sup l ν(x) denotes the Lebesgue constant. Then sup E(ũ k+1 u)(x) m k+1 L Λ + I Λ(u) u,d x D M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 20/28

43 Error analysis for Λ k = Λ 2 Global error Starting from ( ) E ũ k+1 I Λ(u)(x) = E(ũ k+1 (z ν) u(z ν))l ν(x) (4) ν Λ we get by taking the absolute value and then the supremum over D x D ν Λ sup E(ũ k+1 I Λ(u))(x) m k+1 L Λ (5) x D were L Λ = sup l ν(x) denotes the Lebesgue constant. Then sup E(ũ k+1 u)(x) m k+1 L Λ + I Λ(u) u,d x D Remark : When t is small enough s.t. ρ m < 1 the approximation error converges geometrically with k up to the a threshold term i.e. ( ) lim sup sup E(ũ k+1 u)(x) L Λ r( t, u I(u)) + I Λ(u) u,d. k x D 1 ρ m M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 20/28

44 Summary Pros and cons : Method 1 Method 2 Λ fixed Λ k adapted at each step Error analysis Convergence up to threshold Require additional assumptions Convergence Convergence in few iterations Slower convergence Problems Only small d Adapted for large d M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 21/28

45 Summary Pros and cons : Method 1 Method 2 Λ fixed Λ k adapted at each step Error analysis Convergence up to threshold Require additional assumptions Convergence Convergence in few iterations Slower convergence Problems Only small d Adapted for large d Alternative strategy : Perturbed sparse adaptive algorithm FK based evaluation instead of the true solution provided via GM algorithm Possible control up of the error of the approximation to a precision ε (?) M. Billaud-Friess MCQMC 18 A sequential algorithm in high dimension 21/28

46 Outline 1 A sequential algorithm for variance reduction 2 A sequential algorithm in high dimension 3 A perturbed sparse adaptive algorithm M. Billaud-Friess MCQMC 18 A perturbed sparse adaptive algorithm 22/28

47 Proposed algorithm Algorithm 3. Perturbed adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and set n = While n < N and ε n 1 > ε : Define M n. Set Λ = Λ n 1 M n and compute ũ Λ. Select N n = {ν M n; E ν(ũ Λ ) θe Mn (ũ Λ )}. Update Λ n = Λ n 1 N n. Compute ũ Λn and ε n. Set n = n + 1. M. Billaud-Friess MCQMC 18 A perturbed sparse adaptive algorithm 23/28

48 Proposed algorithm Algorithm 3. Perturbed adaptive sparse interpolation algorithm 1. Set Λ 1 = {0 d } and set n = While n < N and ε n 1 > ε : Remarks : Define M n. Set Λ = Λ n 1 M n and compute ũ Λ. Select N n = {ν M n; E ν(ũ Λ ) θe Mn (ũ Λ )}. Update Λ n = Λ n 1 N n. Compute ũ Λn and ε n. Set n = n + 1. Here ũ Λ, ũ Λn can be computed with Algorithm 1. stopped either for a stopping criterion based on fixed number of iterations K or given precision δ. For error analysis, the second choice is preferable but we need a practical error estimate ε (e.g. variance?). M. Billaud-Friess MCQMC 18 A perturbed sparse adaptive algorithm 23/28

49 First numerical results : exact vs. perturbed algorithm Test n 2 : t = 0.001, M = 2000, δ = ε = #Λ n ε n ε n K e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e-05 4 Table #Λ n, ε n, ε n and K w.r.t. #Λ n Error L 2 -norm L -norm u I Λ20 (u) e e-03 u ũ δ Λ e e-03 Table Error to exact solution Remarks : Since δ = ε, approximation ũ δ Λ 20 as accurate as I Λ20 (u). 15 first iterates : low impact of the error due to ũ δ Λ n since governed by the interpolation error (> δ) Last iterates : δ is reached with few iterations using exact error stopping criterion M. Billaud-Friess MCQMC 18 A perturbed sparse adaptive algorithm 24/28

50 Conclusion Summary : Stochastic approaches for computing global approximation to solution of high-dimensional partial differential equations. M. Billaud-Friess MCQMC 18 Conclusions 25/28

51 Conclusion Summary : Stochastic approaches for computing global approximation to solution of high-dimensional partial differential equations. Ongoing work :? Clarify error analysis : for fixed Λ especially for the stagnation terms (w.r.t. t, M) and for the variance, for varying Λ k (nested sets)? Improve Algorithm 3. with better control of the error for the estimate provided by the approximation ũ Λn? Study the convergence of the perturbed sparse adaptive interpolation algorithm M. Billaud-Friess MCQMC 18 Conclusions 25/28

52 Conclusion Summary : Stochastic approaches for computing global approximation to solution of high-dimensional partial differential equations. Ongoing work :? Clarify error analysis : for fixed Λ especially for the stagnation terms (w.r.t. t, M) and for the variance, for varying Λ k (nested sets)? Improve Algorithm 3. with better control of the error for the estimate provided by the approximation ũ Λn? Study the convergence of the perturbed sparse adaptive interpolation algorithm Thanks for attention! M. Billaud-Friess MCQMC 18 Conclusions 25/28

53 References I [Bachmayer 16] M., Bachmayr, R. Schneider & A., Uschmajew Tensor Networks and Hierarchical Tensors for the Solution of High-Dimensional Partial Differential Equations, Found Comput Math [Beck 18] C. Beck, S. Becker, P. Grohs, N. Jaafari & A. Jentzen Solving stochastic differential equations and Kolmogorov equations by means of deep learning ArXiv 2018 [Beck 17] C. Beck & A. Jentzen Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations ArXiv 2017 Christian Beck, Weinan E, Arnulf Jentzen [Chkifa 13] A. Chkifa, A., Cohen, R., DeVore, R., & C. Schwab, Sparse adaptive Taylor approximation algorithms for parametric and stochastic elliptic PDEs. ESAIM : Mathematical Modelling and Numerical Analysis [Chkifa 14] A. Chkifa, A. Cohen & C.Schwab, High-dimensional adaptive sparse polynomial interpolation and applications to parametric PDEs, Found. Comput. Math [Cohen 15] A., Cohen, A., & R. DeVore Approximation of high-dimensional parametric PDEs, Acta Numerica [Nouy 17] A. Nouy, Low-Rank Methods for High-Dimensional Approximation and Model Order Reduction Model Reduction and Approximation, Chapter [Giles 08] M. Giles Multi-level Monte Carlo path simulation. Operations research, 2008 M. Billaud-Friess MCQMC 18 Conclusions 26/28

54 References II [Grasedyck 13] L. Grasedyck, D. Kressner & C. Tobler, A literature survey of low- rank tensor approximation techniques, GAMM-Mitteilungen [Gobet 04] E. Gobet and S. Maire, A spectral Monte Carlo method for the Poisson equation, Monte Carlo Methods Appl [Gobet 09] E. Gobet and S. Maire, Sequential control variates for functionals of Markov processes, SIAM Journal on Numerical Analysis [Gobet 13] E. Gobet, Méthodes de Monte-Carlo et processus stochastiques : du linéaire au non linéaire, Editions de l école polytechnique 2013 [Graham 13] G. Graham & D. Talay, Stochastic simulation and Monte Carlo methods Stochastic Modelling and Applied Probability, Springer [Hackbush 14] W. Hackbusch. Numerical tensor calculus Acta numerica 2014 [Khoromskij 12] B. Khoromskij, Tensors-structured numerical methods in scientific computing : Survey on recent advances, Chemometrics and Intelligent Laboratory Systems [Kolda 09] T. G. Kolda & B. W. Bader, Tensor decompositions and applications. SIAM Review [Kloeden 99] P.E. Kloeden & E. Platen, Numerical Solution of Stochastic Differential Equations, Springer Verlag 1999 M. Billaud-Friess MCQMC 18 Conclusions 27/28

55 References III [Maday 09] Y. Maday, N. C. Nguyen, A. T. Patera, & G. S. H. Pau. A general multipurpose interpolation procedure : The magic points, Communications on Pure & Applied Analysis, 2009 [Oseledets 11] I. Oseledets. Tensor-train decomposition. SIAM J. Sci. Comput [Schwab 11] C. Schwab, & C. J. Gittelson, Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs, Acta Numerica [Weinan 17] E. Weinan, H. Jiequn & A. Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations Arxiv M. Billaud-Friess MCQMC 18 Conclusions 28/28