Part III. Quasi Monte Carlo methods 146/349

Transcription

1 Part III Quasi Monte Carlo methods 46/349

2 Outline Quasi Monte Carlo methods 47/349

3 Quasi Monte Carlo methods Let y = (y,...,y N ) be a vector of independent uniform random variables in Γ = [0,] N, u(y) : Γ V a given Hilbert-valued function and Q : V R a functional on V. We are interested in computing I N = Q(u(y))ρ(y)dy, Γ = [0,] N, ρ(y) = Γ A Monte Carlo approximation uses M points (ξ,...,ξ M ) sampled independently from the density ρ(y) and approximates I N as I N,M = M Q(u(ξ i )) (*) which can be seen as a quadrature formula with equal weights M. Question: while keeping the approximation form (*) with equal weights, is there a better distribution of points than the purely random one? Random points Sobol s points 48/349

4 Quasi Monte Carlo methods and Discrepancy Let P = {ξ,...ξ M } be a set of points ξ i [0,] N and f : [0,] N R a continuous function. A Quasi Monte Carlo method (QMC) to approximate I N (f) = [0,] N f(y)dy is an equal weight cubature formula of the form I N,M (f) = M f(ξ i ) The key concept in the analysis of QMC methods is the one of discrepancy. Let x [0,] N and [0,x] = [0,x ]... [0,x N ]. Local discrepancy: P (x) = # points in [0,x] M In formulae: P (x) = M Vol([0,x]) [0,x] (ξ i ) N x i x Star discrepancy : P,N = sup x [0,] N P (x) 49/349

5 Low discrepancy sequences Definition: a sequence (ξ,ξ 2,...), ξ i [0,] N is a low discrepancy sequence if the set of points P M = (ξ,...,ξ M ) satisfies P M,N C N (logm) N M. QMC methods use equal weight cubature formulae with low discrepancy sequences of points. Low order projections: let α {,...,N} a subset of indices, with #(α) = s; denote by ξ α the vector {ξ j, j α} and (ξ α,) the vector ξ with the components not included in α set to. The projected sequence P α M = (ξα,...,ξ α M) onto the α directions has discrepancy P α M,s P M,N Proof: P α M,s(x) = sup x α [0,] s ( M [0,x α ](ξ α i ) x i ) i α = sup y=(x α,) [0,] N ( M [0,y] (ξ i ) N y i ) P M,N 50/349

6 Implication: if P M,N (logm)n /M, the discrepancy of any projection on dimension s < N has to behave the same way or better! Exercice: How does the star discrepancy scale in M for: M random iid uniformly distributed points in [0,] N uniform grid of M points per edge /349

7 Low discrepancy sequences Several low discrepancy sequences exist, available also in Matlab (see e.g. [Niederreiter 92]): Halton, Hammersley, Sobol, Faure, Niederreiter,... The simplest example: Halton sequence. For integers i and b 2, we first define the radical inverse function φ b (i) as follows: if i = i n b n, i n {0,,...,b }, then φ b (i) = i n b n, n= so, in base b = 0, the radical inverse of i = 542 is φ 0 (i) = Then, if p,...,p N denote the first N prime numbers, the Halton sequence P = {ξ,ξ 2,...} is given by and its star-discrepancy satisfies ξ i = (φ p (i),φ p2 (i),...,φ pn (i)) N := sup x [0,] N P (x) = O ( (logm) N M ). n= Other (better) sequences such as Hammersley, Sobol, Faure,..., have ( ) (logm) a star-discrepancy N N = O. M 52/349

8 Having low discrepancy in high dimension is difficult from [Joe-Kuo, 2008]: two dimensional projections of 4096 Sobol points in dimension /349

9 Lattice rules Let M be the number of points we want to use and z N N be an integer vector whose components have no factor in common with M. { } iz rank-one lattice rule ξ i =, i = 0,...,M M where { } denotes the fractional part (of each component of the vector). The lattice rule quality depends heavily on the choice of the vector z. In 2D a good choice is M = fibonacci(k) and z = (,fibonacci(k )) z/m z/m M = 3, z = (,8) M = 55, z = (,34) 54/349

10 Lattice rules In higher dimension, there is no obvious generalization of the Fibonacci lattice. Let gcd(x,y) be the greatest common divisor of x and y. Each component z i of the vector z should be looked for in the set U M := {z Z : z M, gcd(z,m) = } so that each one-dimensional projection of the lattice has M distinct points. If M is prime, there are U M = M possible choices for each component z i. Component-by-Component (CBC) construction. Set z = for i = 2,...,N with z,...,z i held fixed, choose z i U M to minimize a desired error criterion in dimension i. end 55/349

11 A key formula for Error analysis D Let us consider first the N = dimensional case. The following error representation holds: M f(ξ i ) [0,] f(y)dy = P ()f() [0,] P (y)f (y)dy Proof: y [0,], and M f(y) = f() f(ξ i ) [0,] y f (t)dt = f() [0,] (y) 0 f (t) [0,t] (y)dt ( ) f(y)dy = f() [0,] (ξ i ) [0,] (y)dy M [0,] }{{} P ()=0 ( f ) (t) [0,t] (ξ i ) [0,t] (y)dy dt [0,] M [0,] }{{} P (t) 56/349

12 A key formula for Error analysis D Hence, the error in the QMC integration is bounded by f(ξ i ) f(y)dy M = P (y)f (y)dy P f L (0,) [0,] [0,] Remark : the norm f L can be replaced by V(f), the total variation of f in the sense of Hardy and Krause. Remark 2: One could use Holder with different exponents to obtain f(ξ i ) f(y)dy M P L p (0,) f L q (0,), with p + q =. [0,] 57/349

13 A key formula for error analysis The previous formula generalizes to arbitrary dimension in the following way: let [ : N] denotes the set of subsets of {,2,...,N}, including the empty set. For α [ : N], let y α denote the vector {y j, with j α} and (y α,) the vector y with the components not included in α set to. Then, letting #(α) be the cardinality of α, we have M f(ξ i ) [0,] N f(y)dy = α [:N] ( ) #(α) [0,] #(α) #(α) f y α (y α,) P (y α,)dy α (2) Define now the following space V,p = {f : [0,] N R, f,p < + }, with norm f,p = #(α) p p f (y α,) y α dy α α [:N] [0,] #(α) For p = 2, V,2 is a Hilbert space isomorphic to H mix ([0,]N ). It measures mixed first derivatives of the function f. 58/349

14 The Hmix k (Γ) space Space of functions with k mixed square integrable derivatives: let β = (β,...,β n ) N N and the seminorms The Hilbert space v 2 H k mix (Γ,V) = β =k Γ k v(y) y β yβ n n 2 ρ(y)dy. H k mix(γ,v) = {v L 2 ρ(γ,v) : k s=0 v 2 Hmix s (Γ,V) < + } is endowed with the norm k v 2 Hmix k (Γ,V) = s=0 v 2 H s mix (Γ,V). 59/349

15 Koksma-Hlawka inequality Let p <. Then, using Hölder s inequality in (2) we have f(y)dy [0,] M N f(ξ i ) P,q f,p, with p + q = In the case p = and q = this is known as Koksma-Hlawka inequality. The Quasi Monte Carlo error is bounded by the sum of the q-norm of the local discrepancies in all dimensions,...,n, provided the function has p-integrable mixed first derivatives. 60/349

16 Other equivalent norms (but the equivalence constant depends on N!) can be considered as well and analogous results obtained. See [Dick-Kuo-Sloan 3] for an extensive discussion. For example: Anchored Sobolev space: for c [0,] N f,2 = α [:N] [0,] #(α) #(α) f (y α,c) y α 2 dy α 2 Unanchored Sobolev space: f,2 = α [:N] [0,] #(α) ( [0,] N #(α) #(α) f y α (y α,y α )dy α ) 2 dy α 2 where y α denotes the vector {y j, j / α}. 6/349

17 Randomly shifted lattice rules To estimate the error in a Quasi Monte Carlo computation, the following stategy can be adopted. Let η U([0,] N ) be a uniformly distributed random vector (shift) and I N,M (f,η) = M f({ξ i +η}) the QMC formula with the random shift η. Observe that the shift preserves the lattice structure and E[f({ξ i +η})] = I N (f), for all i =...,M. Take s iid shifts η,...,η s U([0,] N ). Then Estimate of the integral I N,M (f) = s s j= I N,M (f,η s ) = sm s j= f({ξ i +η j }) Estimate of the MCS error based on the η sample standard deviation, (observe that there is no bias anymore!) i.e. e N,M (f) = c o s (s ) s ( IN,M (f,η j ) I N,M (f) ) 2 j= 62/349

18 Randomly shifted lattice rules Since we no longer have a bias error, we need to understand the resulting variance in the randomly shifted rules. The analysis goes through the reinterpretation of the induced Sobolev space as a Reproducing Kernel Hilbert Space (RKHS) and a representation of the worst mean square error in terms of the Kernel function. (See [Dick-Kuo-Sloan 3] for details) 63/349

19 Infinite dimensional approximation We have seen that the QMC error can be bounded by f(y)dy f(ξ [0,] M i ) P,q f,p (22) N Moreover, good low discrepancy sequences (Sobol, Faure, lattice rules,...) have a star-discrepancy P,N = P, = O(log(M) N /M). Is there hope for an infinite dimensional approximation N? Note: if all variables have the same importance in (22), we cannot take the limit N. However, infinite dimensional approximation can be achieved if the function f belongs to a weighted space. Let {γ α } α [:N] be a sequence of positive weights Anchored weighted Sobolev space f,2,γ = α [:N] γ α [0,] #(α) #(α) f (y α,c) y α Analogous definition for the unanchored version. 2 dy α 2 64/349

20 Infinite dimensional approximation Then, from the Zaremba identity we have f(y)dy f(ξ [0,] M i ) P,2, N γ f,2,γ The weights γ should be chosen such that f,2,γ < P,2, γ as small as possible 65/349

21 Infinite dimensional approximation in weighted space Product weights [Sloan-Woźniakowski] γ α = i αγ i, γ i < + Each variable is weighted differently. The intuition behind is that the variables y i for i become unimportant in the computation of the integral. Order dependent weights [Dick et al 06] γ α = β α, with β n n 0 Here we think that the terms involving many variables at the same time become unimportant in the computation of the integral as the number of involved variables increases. Product and Order Dependent (POD) weights [Kuo et al 2] n γ α = β α γ i, with β n 0 i α Efficient constructions of points by the CBC (Component-by-component) techniques corresponding to γ weights have been proposed in [Kuo-Joe, 03]. 66/349

22 A convergence result for Quasi Monte Carlo see [Dick-Kuo-Sloan 3, Theorem 5.0]: the error given by a shifted averaged lattice rule with CBC optimal construction of the generating vector z satisfies e N,M (f) = O ( loglogm M ) 2λ, λ ( 2, ] provided the function f belongs to the (anchored or unanchored) weighted space with weights γ α such that =α [:N] γ α λ ( ) #(α) 2ζ(2λ) (2π 2 ) λ +βλ < + where ζ( ) is the Riemann zeta function (observe that ζ(2λ) as λ 2 ) and β = 0 for the unanchored space and β = c2 c + 3 for the anchored space. This result shows that a limit rate M is possible even in infinite dimension if the integrand function belongs to a suitable weighted space, with sufficiently fast decaying weights. 67/349

23 Example elliptic PDE with random diffusivity coefficient We consider once again the model problem { div(a(x,y) u(x,y)) = f(x), x D, u(x,y) = 0, x D, y Γ := [ 3, 3] N with a(x,y) = ā+ N λi y i b i (x) (here N could be ), y i U( 3, 3) i.i.d., N 3λi b i δā for some 0 < δ <, so that u(y) L 2 (D) C u := C P Moreover, setting β i = ( δ)ā f L 2 (D). λi b i ( δ)ā, we assume that exists 0 < p s.t. N β p i C, p p, with C independent of N (23) Considering a linear funtional Q(u): will the random variable ψ(y) = Q(u(y)) belong to a certain weighted space W,2,γ for some suitable choice of weights γ α? ψ,2,γ = #(α) 2 γ α ψ(y α,0) y α (2 3) #(α)dy α α [:N] [ 3, 3] #(α) 2 < 68/349

24 Observe that #(α) ( ψ #(α) ) y α = Q u(y) y α Q V #(α) u(y) y α V so we have to check if u(y) belongs to the space W,2,γ (Γ;V), where V = H 0 (D) which we endow with the norm v V = v L 2 (D) We (formally) differentiate the equation with respect to y i at y = 0: first derivative yi. For all x D we have div(a(x,0) yi u(x,0)) = div( yi a(x,0) u(x,0)) = div( λ i b i (x) u(x,0)). Multiplying by yi u(x,0) and integrating by parts in x yields ( δ)ā yi u(,0) 2 L 2 (D) λ i b i u(,0) L 2 (D) y i u(,0) L 2 (D) so that yi u(,0) V β i C u second mixed derivative yi y j. For all x D we have div(a(x,0) y 2 i y j u(x,0)) = div( λ j b j (x) yi u(x,0))+div( λ i b i (x) yj u(x,0)). so that 2 y i y j u(,0) V β j yi u(,0) V +β i yj u(,0) V 2β i β j C u 69/349

25 Iterating the procedure, yields that for any α [ : N] we have #(α) y α u(,0) V C u #(α)! j αβ j Following [Kuo-Schwab-Sloan ], this in turn implies that u W,2,γ (Γ;V) with corresponding γ α = #(α)! j αβ j 2 p (*) with p as in (23). Indeed u 2 W,2,γ (Γ;V) C u 2 γ (#(α)!)2 α C 2 u C 2 u C 2 u α [:N] α [:N] α [:N] α N N (#(α)!) p j α #(α)! α! α! j j α β p α j j j α β p2 j β p j β 2 j = C 2 u j α p p α [:N] (#(α)!) p j α β p( p) j [use Hölder with p and p ] α [:N] j α α [:N] j α β p j β p j p p β p j = C 2 u ( j β p j )p j (+β p j ) p < + 70/349

26 Finally, for this choice of weights, the quantity ( 2ζ(2λ) γα λ (2π 2 ) λ +βλ =α [:N] ) #(α) is bounded for any λ = max{ p 2 p, 2 2δ }, δ > 0 (here β = /2 and with p as in (23)). Indeed, set σ(λ) = 2ζ(2λ) (2π 2 ) λ +2 λ. Then, since p (2 p)λ γα λ σ(λ)#(α) = (#(α)!) (2 p)λ α [:N] α [:N] σ(λ) j β (2 p)λ j j α σ(λ)β (2 p)λ j < since (2 p)λ p α [:N] #(α)! j α σ(λ)β (2 p)λ j Theorem [Kuo-Schwab-Sloan ] The Quasi Monte Carlo method with CBC construction of the lattice rule and weights (*) converges at rate O(M +δ ) for p < 2 3 2δ, O(M 2 p ) for 2 3 2δ < p with p as in (23) and for any δ > 0 (with a constant that blows up when δ 0). 7/349

27 Complexity analysis We now discretize the problem in space by finite elements and assume that y Γ, the discretization error Q(u(y)) Q(u h (y)) C h α The computational work to solve for each u h (ξ j ) is O(h dβ ). As in Monte Carlo, we can therefore choose M and h so as to { min h,m Mh dβ s.t. C h α +C 2 M γ TOL with γ = δ if p < 2 3 2δ and γ = p 2 if 2 3 2δ < p. which leads to a resulting complexity O(TOL γ dβ α ). Hence W TOL 2 δ dβ α, for p < 3 2δ W TOL 2p 2 2 p dβ α, for 3 2δ < p 72/349

28 Extensions of Quasi Monte Carlo Multilevel QMC [Kuo et al, 203] Higher order QMC, which aim to achieve a rate of convergence faster than linear, lattice rules that are extensible in number of points or dimension, more general pdfs and unbounded domains... 73/349