Quasi-Monte Carlo Methods for Applications in Statistics

Transcription

1 Quasi-Monte Carlo Methods for Applications in Statistics Weights for QMC in Statistics Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

2 Quasi-Monte Carlo Methods for Applications in Statistics Joint work with Ian Sloan and Frances Kuo Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

3 Quasi-Monte Carlo Methods for Applications in Statistics Outline: Background on QMC A class of problems in Statistics Approach and preliminary results Future research Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

4 Integrals over Euclidean space Many problems involve evaluation of integrals given by I d (f, ρ) = f (x)ρ(x) dx, R d where ρ(x) is a probability density. Hence ρ(x) 0 for any x R d and R d ρ(x) dx = 1. Here we assume that the density is expressed as the product of univariate densities, hence d ρ(x) = φ(x j ). j=1 Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

5 Transforming the integral Integrals over unbounded regions are usually mapped to the unit cube. Then, a QMC method (for instance a shifted lattice rule) can be used to approximate the integral. In the 1-dimensional case, we can use the following transform: u = Φ(x) = x φ(t) dt, x R. The inverse mapping will be Φ 1 : (0, 1) R, Φ 1 (u) = x. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

6 Mapping to the unit cube In the d-dimensional case, if x = (x 1, x 2,..., x d ) R d, then Φ(x) = (Φ(x 1 ), Φ(x 2 ),..., Φ(x d )). In the same manner, the inverse mapping will also be applied component-wise. The integral becomes I d (f, ρ) = f (Φ 1 (u)) du = [0,1] d g(u) du = I d (g), [0,1] d where g = f Φ 1 (applied component-wise). These integrals can be approximated by quadrature rules of the form Q n,d (g) = 1 n 1 g(w k ) = 1 n 1 f (t k ), n n k=0 k=0 where t k = Φ 1 (w k ) R d, for all 0 k n 1. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

7 QMC error The typical error bound is expressed as I d (g) Q n,d (g) e worst n,d,γ (P) g (0,1) d, where P is the set of quadrature points. Note that we have g (0,1) d = f R d. Provided that g <, the vast majority of research papers was focused on obtaining the optimal convergence rate for the worst-case error. It has been proved that good lattice rules do exist and that there are efficient algorithms (such as the CBC construction) to construct such lattice rules under various settings. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

8 Weights A widely used setting is provided by weighted reproducing kernel Hilbert spaces (RKHS) and L 2 -type discrepancies. Typical examples are weighted Sobolev and weighted Korobov spaces of functions. The concepts of weights is related to the fact that variables or group of variables have different importance. Accordingly, we associate a weight γ u, for each subset of coordinates u D := {1, 2,..., d}. It is known that under suitable conditions on the weights, the weighted discrepancy or the worst-case error is independent of the dimension. A typical example of convergence rate on the worst-case error is O(n 1+δ ) with the involved constant independent of the dimension. It is also known how to construct lattice rules that achieve such a convergence rate. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

9 Our approach Instead of analysing only the worst-case error, we look at the error bound itself, namely we analyse the product between the worst-case error and the norm of the integrand. This idea is not new and there are however some earlier works by Dick, Sloan, Wang and Woźniakowski (2004) and Larcher, Leobacher and Scheicher (2003). More recent contributions coming from the analysis of the error bound is due to Kuo, Schwab and Sloan who obtained the optimal weights for a class of QMC methods suitable for PDE. We follow a similar idea, but for a class of important problems in statistics. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

10 A class of problems in Statistics We are interested in a class of highly structured generalised response models in Statistics, in particular on the so-called generalised linear mixed model (GLMM). The Poisson-type likelihood of such a model can be expressed by L(β, Σ) = R d d exp(y j (w j + β) e w j +β exp ( 1 2 wt Σ 1 w ) dw. y j (2π) d det(σ) j=1 However a naïve approach consisting of diagonalising the covariance matrix and then mapping to the unit cube gives poor results!! See Kuo, Dunsmuir, Sloan, Wand and Womersley (2008) for details. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

11 Recentering and rescaling Good results however have been obtained after an appropriate linear transformation consisting of recentering and rescaling. Denote F (w) = d j=1 ( y j (w j + β) e w j +β ) 1 2 wt Σ 1 w. We use a change of variables of the form w A x + w, where w satisfies F (w ) = 0 and A A T = ( 2 F (w )) 1. That is, w is the stationary point, while the matrix A A T describes the convexity of F around the stationary point. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

12 The transformed integral The integral we want to evaluate becomes d exp F (w) dw = det(a ) F (A x + w ) R d R d where φ(x) is a univariate probability density. j=1 1 φ(x j ) } {{ } f (x) d φ(x j ) dx, This integral can then be mapped to the unit cube as explained earlier. j=1 Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

13 Our function We want to evaluate the norm of functions f given by d ( ) f (x) := exp y j ((A x) j + β + wj ) e (A x) j +β+wj exp j=1 [ 1 ] 2 (A x + w ) T Σ 1 (A x + w ) d h(x j ), where h(x) := 1/φ(x). The norm of such functions is defined by f := ( ) 2 γ 1 u 1/2 u f ((x u, 0)) dx u. x u u D R u j=1 Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

14 How to find a bound on the norm of f? The task of finding bounds is also difficult. However we follow three main steps: 1 We find the partial mixed first derivatives of f. 2 We bound the derivatives. 3 We bound the norm. Following these steps, we obtained a generic bound on the norm f and a specific bound in the case of an assumed logistic density. Similar ideas can be used to deduce corresponding bounds for other densities (future research). Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

15 The partial derivatives of f After some tedious calculations, by applying Faà di Bruno formula and the formula for differentiating products of functions, we obtain: u f (x) = exp(g(x)) d e (A x+w ) j +β aji + T w (x) x u v u π Π(v) w π j=1 i w d d h(x i ) h (x i ), where i=1 i u v i=1 i u v Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

16 The partial derivatives of f g(x) := d j=1 ( ) y j ((A x) j + β + wj ) e (A x) j +β+wj and 1 2 (A x + w ) T Σ 1 (A x + w ), d j=1 y ja ji (A ) i Σ 1 (A x + w ), if T w (x) := (A ) T i Σ 1 (A ) k, if w = {i, k}, 0, if w 3. w = {i}, Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

17 Expressing the bounds on the derivatives Theorem The partial derivatives of f can be bounded by ( u f (x) d (A ) T i τ exp J(x) + Σ 1 A x ) x u α i i=1 d B v v!ω v α i h(x i ) v u 2 i v i=1 i u v d i=1 i u v h (x i ), where B m denotes the Bell number of order m, that is the number of partitions of a set with cardinality m. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

18 Expressing the bounds on the derivatives J(x) := d j=1 y j ((A x) j + β + w j ) 1 2 (A x + w ) T Σ 1 (A x + w ), α i := max j=1,...,d a ji, d τ := exp 1 d y j aji + (A ) T i Σ 1 w, α i i=1 ω := 1 + Σ 1 1, j=1 where by 1, we mean the entrywise norm of the matrix. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

19 Bounding the norm Theorem The norm of f can be bounded by f 2 τ 2 e r u D R u v u k u γ 1 u (h(0)) 2(d u ) exp( λ min x 2 k + µ kx k + θ k x k ) B v v!ω v 2 i v α i h(x i ) h (x i ) i v i u v 2 dx u. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

20 Bounding the norm where λ min is the smallest eigenvalue of the matrix A T Σ 1 A (which is positive definite), and r := 2 d j=1 y j (β + w j ) w T Σ 1 w, µ k := 2(w T Σ 1 A ) k + 2 θ k := 2 d i=1 d y j ajk, j=1 ((A ) T i Σ 1 A ) k α i. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

21 Logistic density The logistic univariate density is defined by This leads to φ(x) = e x/ν ν(1 + e x/ν ) 2. h(x) = ν(e x/ν e x/ν ) 4νe x /ν, and h (x) = e x/ν e x/ν e x /ν. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

22 Logistic density We then have Denote also h(x i ) h (x i ) (4ν) v e x i /ν. i v i u v i u θ k := θ k + 2 ν. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

23 Bounding the norm for the logistic density Theorem f 2 τ 2 e ( ) u π r γ 1 u (4ν) 2(d u ) B u 2 λ ( u!)2 min u D ) 2 (1 + 4νω 1/2 α k k u ( ( ) ( )) ( θ k µ k ) 2 ( θ k + µ k ) 2 exp + exp. 4λ min 4λ min k u Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

24 On the possible weights We remark that there are two distinctive parts in the RHS of the bound. One part is depending only on the cardinality of the set u and the second one is depending on the coordinates belonging to u. Consequently, the optimal weights might be a hybrid between product weights and order dependent weights. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

25 POD weights POD weights who were first introduced by Kuo, Schwab and Sloan (2011) in connection with the use of QMC methods for a class of elliptic PDE. Such weights are defined by γ u := Γ u γ j, where Γ u relates only to the cardinality of u (the order dependent part), while γ j is a weight associated with the individual coordinate j. It remains to figure out the expression of the optimal weights for our problem. j u Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

26 Worst-case error Here, it was the norm of the integrand that received full attention. We already know that there exist shifted lattice rules such that the worst-case error satisfies en,d,γ sh 1 (z) (ϕ(n)) 2λ γ λ u(ϱ(λ)) u u D 1 2λ, where n is the number of points of the lattice, z is the generating vector, λ ( 1 2, 1] and ϱ is a function depending only on λ. These lattice rules can be constructed via the CBC algorithm. Contributions by Kuo, Nichols, Schwab, Sloan, Wasilkowski, Waterhouse. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

27 Error bound The error bound is given by (ϕ(n)) 1 2λ 1 2λ γ λ u(ϱ(λ)) u u D u D The weights that minimise such error bounds are γ u = ( β u ( ϱ(λ)) u k u γ 1 u b k ) 2 1+λ. β u k u b k 1 2. Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

28 Future research 1 Finding the optimal weights for other statistical models. 2 Develop similar analysis for other important applications, such as finance (work in progress with some preliminary results). Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24

29 Thank you! Vasile Sinescu (UNSW) Weights for QMC in Statistics MCQMC February / 24