QUASI-MONTE CARLO METHODS IN NON-CUBICAL SPACES

Transcription

1 QUASI-MONTE CARLO METHODS IN NON-CUBICAL SPACES A DISSERTATION SUBMITTED TO THE DEPARTMENT OF STATISTICS AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Kinjal Basu May 2016

2 2016 by Kinjal Basu. All Rights Reserved. Re-distributed by Stanford University under license with the author. This work is licensed under a Creative Commons Attribution- Noncommercial 3.0 United States License. This dissertation is online at: ii

3 I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Art Owen, Primary Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Emmanuel Candes I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Sourav Chatterjee Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost for Graduate Education This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives. iii

4 Abstract Monte Carlo integration is a widely used technique for approximating high dimensional integrals. However, due to the inherent randomness of this method, the convergence is typically slow. Quasi-Monte Carlo (QMC) on the other hand gives a much better rate of convergence by using low-discrepancy sequences. These point sets are much more uniformly distributed than random samples. Most QMC research focuses on sampling from the unit cube. However, many problems in real-world applications are defined over much more general spaces, such as triangle, spheres, spherical triangles and discs. This dissertation deals with solving such problems of numerical integration defined over non-cubical domains. We introduce two QMC constructions in the triangle with a vanishing discrepancy. The first is a version of the van der Corput sequence customized to the unit triangle. The second construction rotates an integer lattice through an angle whose tangent is a quadratic irrational number. We then generalize the van der Corput construction to study the problem of numerical integration over the Cartesian products of s spaces of dimension d via scrambled geometric nets. We also show the asymptotic normality of the scrambled geometric net estimator. We further discuss some of the issues of why transformations of the unit cube to the domain of interest fails to give good results. Since products of simplices is throughout of special interest to us, we end the dissertation with few results of QMC tractability on that domain. iv

5 Acknowledgements I would sincerely like to thank many people both within and outside Stanford, without whose support, this thesis would not have been possible. First and foremost, I would like to express my deepest gratitude and heartfelt appreciation to Professor Art B. Owen, for being an exceptional advisor, a remarkable teacher and an amazing person to work with. I have learnt a lot from him, not only on the academic front, but also on almost all other aspects of life. He has continuously encouraged me to work on challenging problems. His critical way of thinking has greatly helped me in understanding which problems are worth tackling. I am indebted to him for his patience and for giving me the freedom to work on some of my own projects. I will cherish the long drawn discussions and his ever-welcoming attitude throughout my life. His mentorship is the strongest pillar for the successful completion of this thesis. Second, I would like to thank my committee members. I am grateful to Professor Emmanuel Candès for his valuable comments during my proposal meeting. His class Stats 300C is one of the best classes I have taken at Stanford and it has encouraged me to work on theoretical problems. Professor Sourav Chatterjee has been a true inspiration throughout my years at Stanford. His excellent notes of Stein s Method from a prior class, helped develop the asymptotic results in Chapter 5. Special thanks to Professor Persi Diaconis for his willingness to listen, advice and take the time to be a part of my thesis committee. Moreover, I value having several discussions with him on orthonormal polynomials, a founding stone for the results in Chapter 7. I also thank Professor Peter Glynn for chairing my dissertation committee. I express my profound gratitude to all my teachers here at Stanford. I thank v

6 Professor Stephen Boyd for introducing me to the world of convex optimization and Professors Darrell Duffie for the exquisite exposition to the world of mathematical finance. I also thank all the Stanford faculty members whom I came across at some point or the other, as a graduate student. During my years at Stanford, I have been fortunate to travel abroad and meet other researchers working in a similar field. Many thanks to Professors Luca Brandolini, Giancarlo Travaglini, Josef Dick, Dmitriy Bilyk, and Michael T. Lacey for the many interactions and fruitful discussions. Special shout out for the entire Quasi-Monte Carlo and Discrepancy Theory community for the excellent meetings, which I was honored to be a part of. A huge thanks goes to Ajit Singh, Shaunak Chatterjee and Ankan Saha, my mentors at LinkedIn (during my internships in 2014 and 2015) for being excellent collaborators and great friends. I also thank my teachers from ISI, especially Debapriya Sengupta for encouraging me to take this path. An integral part of the Stanford experience is the company of my wonderful friends. A big thank you to all members of the Owen research group, and other students of the department for creating an incredibly fun and intellectually stimulating environment to spend four years in. I am thankful to each and everyone in the cohort of students who entered Stanford with me, for being great friends and a source of support. Special thanks to Amir, Cyrus, Lucas, Naftali and Xiaoying for being with me through thick and thin and gifting me countless fond memories. I am also thankful to my fellow graduate students who have made my stay at Stanford immensely enjoyable, epsecially, Subhabrata, Snigdha, Pragya, Murat, Josh, Hera and Qingyuan. A heartfelt thank you to Basak-da, Pallavi-di, Sumit-da, Kausturi-di, Gourab-da and Rahul-da. My Stanford experience would not be the same without them. It would be a crime to not mention Bhaswar-da and Ramu, for not only being great collaborators, but also for being the elder brothers which I never had! Thank you. None of this would have been possible without the eternal support and endless love of my parents. They have always stood by me and made sacrifices for me so that I could travel half way around the world to follow my dreams. I would not be the person I am without them - I dedicate this thesis to them, for all that they have done. Last but not the least, I thank Devleena for her love, continued support and vi

7 for being my emotional pillar. I cannot thank her enough for all that she has done for me. Her companionship is better than the best gift I could have ever asked for! vii

8 Contents Abstract Acknowledgements iv v 1 Introduction 1 2 Background on QMC and Randomized QMC Discrepancy Multi-dimensional Variation Scrambling QMC Tractability I Low-Discrepancy Constructions 17 3 Low-Discrepancy Constructions on the Triangle Introduction Background Discrepancy Koksma-Hlawka Transformations Triangular van der Corput points Discrepancy of triangular van der Corput points Proof of Theorem Triangular Kronecker Lattices viii

9 3.5 Riemann integrable functions Discussion II Scrambled Geometric Nets 45 4 Theory of Scrambled Geometric Nets Introduction Splits and geometric van der Corput sequences Splits and recursive splittings Splitting the disk and spherical triangle Geometric van der Corput sequences Geometric nets and scrambled geometric nets Measure preservation Results in L 2 not requiring smoothness ANOVA and multiresolution for X 1:s ANOVA of X 1:s Multiresolution Variance and gain coefficients Smoothness and Extension Sobol extension Whitney extension Smoothness of ANOVA components Scrambled net variance for smooth functions Discussion Asymptotic Distribution of Scrambled Geometric Net Quadrature Introduction Main Results Smooth functions on X 1:s Lower Bound on Variance Asymptotic Normality ix

10 5.3 Numerical Results Proof of Theorem Lower bound on Gain Coefficients Supporting Lemmas Completing Proof of Theorem Specific domains Proof of Corollary Proof of Corollary Alternative approach for s = 1, X = T Stein s Method and Sketch of Proof of Theorem Discussion Effect of Transformations on Hardy-Krause Variation Introduction Smoothness conditions Function composition A counter-example Faa di Bruno formulas Necessary and sufficient conditions Counter-Examples Map from [0, 1] 3 to an equilateral triangle Inverse Gaussian map to the hypersphere Mappings from Fang and Wang (1994) Mapping from [0, 1] d to A d Mapping from [0, 1] d to B d Mapping from [0, 1] d 1 to U d Mapping from [0, 1] d to V d Mapping from [0, 1] d 1 to T d Efficient mapping from [0, 1] d 1 to U d Nonuniform transformations Importance Sampling x

11 6.7.2 Sequential inversion Importance sampling QMC for the simplex III Tractability QMC Tractability on Products of Simplices Introduction Numerical integration over products of simplices and tractability Preliminaries The d-dimensional Simplex and Orthogonal Polynomials The Sobolev Space H Reproducing Kernel Hilbert Space The Sobolev Space H s QMC tractability Upper Bound for e 2 n,s Lower Bound on e 2 n,s Tractability Conclusion References 174 xi

12 List of Tables 3.1 Signed discrepancies for subtriangles crossed horizontally by L b and not touched by L a. Line Inv+ passes just above the centroid of the inverted subtriangle, Inv passes just below it. Upr± are similarly defined with respect to the centroid of the upright subtriangle. For each subtriangle we record the number of centroid points and the fraction of its volume below each line as well as n times the signed discrepancy contribution of that subtriangle and the total signed discrepancy of the trapezoid they form Signed discrepancies for an upright subtriangle T touched by L a and L b. Rows designate 4 relevant horizontal lines, columns the slanted lines. The main table shows the signed discrepancy of T. The rightmost column shows the signed discrepancy of trapezoids to the right of T. The bottom row shows the signed discrepancy of trapezoids below T. 35 xii

13 List of Figures 2.1 The left panel shows 32 points sampled independently from U[0, 1] 2. The center and right panel show two different low-discrepancy sequences. (Source : Owen (2013)) Each panel shows the unit square divided into 5-adic boxes. Panels in the left, middle and right columns are divided into 1, 5, and 25 vertical strips respectively. Panels in the top and bottom rows are divided into 1 and 5 horizontal strips respectively. (Source : Owen (2013)) This figure shows two digital nets in the unit square in base 5. The one on the left has 125 points. The one on the right has 625 points. Dark reference lines 1/5 apart and light ones 1/25 apart show boundaries of some 5-adic boxes. (Source : Owen (2013)) Nested scrambling algorithm. (Source : Dick and Pillichshammer (2010)) The construction of the parallelogram T a,b,c = CDF E A labeled subdivision of (A, B, C) into 4 and then 16 congruent subtriangles. Next are the first 32 triangular van der Corput points followed by the first This figure illustrates the four cases that can arise when both L a and L b touch E This figure shows a trapezoid made up of one upright subtriangle and one inverted subtriangle. Each subtriangle has area 1/n and contains 1 point of P at its centroid, as shown xiii

14 3.5 Triangular lattice points for target n = 64. Domain is an equilateral triangle. Angles 3π/8 and 5π/8 have badly approximable tangents. Angles π/4 and π/2 have integer and infinite tangents respectively and do not satisfy the conditions for discrepancy O(log(n)/n) Parallel discrepancy of triangular lattice points for angle α = 3π/8 and various targets n. The number of points was always n or n + 1. The dashed reference line is 1/n. The solid line is log(n)/n Figure to illustrate decomposition of signed discrepancies of upright (respectively inverted) subtriangle A in terms of parallelograms Splits of a triangle X for bases b = 2, 3 and 4. The subtriangles X j are labeled by the digit j Z b The base b splits from Figure 4.1 carried out to k = 6 or 3 or 4 levels A recursive binary equal area splitting of the unit disk, keeping the aspect ratio close to unity Sobol extensible regions. At left, X is the triangle with vertices (0, 0), (0, 2), ( 2, 0) and the anchor is c = (0, 0). At right, X is a circular disk centered its anchor c. The dashed lines depict some rectangular hulls joining selected points to the anchor Non-Sobol extensible regions. At left, X is an annular region centered at the origin. At right, X is the unit square exclusive of an ɛ-wide strip centered on the diagonal Decay of estimated variance as a function of sample size in a loglog scale. The solid and dashed black line show the log of estimated variance using scrambled geometric nets and Monte Carlo sampling respectively Empirical verification of asymptotic normality for scrambled geometric net estimator. The x-axis shows the centered (with the true mean µ) and scaled (with the estimated standard deviation) scrambled geometric net estimator xiv

15 replications of 95% confidence intervals for µ constructed using scrambled geometric net estimators. The true value is denoted by the horizontal black line. The confidence intervals which do not contain the true µ are shown in red Confidence Intervals for µ generated by the two different sampling techniques. The solid and dashed lines show the confidence intervals using scrambled geometric nets and Monte Carlo respectively with n = A labeled subdivision of T 2 into 4 and then 16 congruent subtriangles The labelling of each cell X (k,t,c) for a level k triangulation of T 2. Subfigure (a) and (b) denotes the upright and inverted cases of X (k,t) respectively The plot on the left shows the square partition P which is repeated in a recursive manner. The right figure shows the function as a 2- dimensional projection for k = 3. Each such pyramidal structure has a height of half the length of its base square xv

16 Chapter 1 Introduction A principal task of modern statistical sampling theory is to choose point sets which reduce the error in high dimensional integration. Quasi-Monte Carlo (QMC) sampling has been well developed for the purposes of integration over the unit cube [0, 1] s. Sampling over other regular shapes is a much more challenging problem that is receiving a lot more focus in the recent era. Motivated by real-world applications, researchers are especially interested in domains such as triangles, spherical triangles, spheres, hemispheres and disks. Integration over such sets is important in graphical rendering (Arvo et al., 2001). For instance, when the domain is a triangle, an integral of the form (T 2 ) 2 f(x 1, x 2 ) dx 1 dx 2 describes the potential for light to leave one triangle and reach another. The function f incorporates the shapes and relative positions of these triangles as well as whatever lies between them. This thesis is dedicated to study such problems of numerical integration over a non-cubical domain D. Our main goal is to accurately estimate by the equal weight rule µ = 1 f(x)dx vol(d) D ˆµ = 1 n n f(x i ), (1.1) i=1 1

17 CHAPTER 1. INTRODUCTION 2 by appropriately choosing the points x i D for i = 1,..., n. Towards that goal, this thesis covers three connected topics based on a series of papers by us; and can be classified under the following divisions: I Low-discrepancy constructions Here we introduce two QMC point sets in the unit triangle. One is a lattice like construction that is the first construction to attain a discrepancy O(log(n)/n) in that space. The other is a generalization of the van der Corput sequence that makes a recursive partition of the triangle. This is based on the joint work with Art Owen and appears in our paper Basu and Owen (2015a). II Scrambled Geometric Nets We split this section into three chapters. (i) Theory of Scrambled Geometric Nets In this chapter we introduce scrambled geometric nets to study the problem of quasi-monte Carlo integration over product spaces of the form X s where X is a bounded set of dimension d. We show that under certain smoothness conditions on f and a sphericity constraint on the partitioning of X, ( ) (log n) s 1 Var(ˆµ) = O. (1.2) n 1+2/d where ˆµ is defined in (1.1) and x i are the points of the scrambled geometric net. This is based on the joint work with Art Owen and appears in our paper Basu and Owen (2015b). (ii) Asymptotic Distribution of Scrambled Geometric Net Quadrature Here we show that the properly scaled and centered scrambled geometric net estimator is asymptotically normal. The result follows via the exchangeable pair technique of Stein s method. Our main result is to show

18 CHAPTER 1. INTRODUCTION 3 that for a class of functions, we attain the same rate in the lower bound on Var(ˆµ) as (1.2). This is based on the joint work with Rajarshi Mukherjee and appears in our article Basu and Mukherjee (2016). (iii) Effect of Transformations Measure preserving mapping from the unit cube to other domains work very well for plain Monte Carlo. Unfortunately, the composition of the integrand with the mapping may fail to have even the mild smoothness properties that QMC exploits. In this chapter, we study a lot of common transformations and their effect on the Hardy-Krause variation. Thus, justifying our proposed scrambled geometric nets. This is based on the joint work with Art Owen and appears in our paper Basu and Owen (2016). III Tractability Since products of simplices is of special interest to us, we develop the theory of QMC tractability on the product of s copies of the simplex T d R d. We find a function class H s, such that the number of function evaluations needed to reduce the initial error by a factor of ɛ is bounded independently of s. We also prove that error in numerical integration is O(n 1/2 ) for functions belonging to the class H s. This is based on my paper Basu (2015).

19 Chapter 2 Background on QMC and Randomized QMC In this chapter we give an overview of QMC and randomized QMC (RQMC) on the unit cube [0, 1] s. Two of the core concepts of QMC are discrepancy of a point set and the variation of a function. We introduce the notion of discrepancy in Section 2.1 and focus on a particular low-discrepancy sequence known as digital nets. In Section 2.2 we describe the concept of multi-dimensional variation in the sense of Hardy and Krause. Using these, we describe the Koksma-Hlawka inequality which gives an error bound for QMC integration over the unit cube. A particular scrambling algorithm is described in Section 2.3 to develop the ideas of randomization. We finally end the chapter with certain notions of tractability in Section 2.4. All of the concepts introduced in this chapter are used to develop the ideas for the rest of the thesis. 2.1 Discrepancy Plain Monte Carlo sampling of [0, 1] s takes x i U[0, 1] s. The left panel of Figure 2.1 shows a set of 32 pseudorandom points. These points form clusters and have gaps. QMC sampling improves upon MC by taking x i more uniformly distributed in [0, 1] s than random points usually are. The uniformity is measured via discrepancy. 4

20 CHAPTER 2. BACKGROUND 5 Figure 2.1: The left panel shows 32 points sampled independently from U[0, 1] 2. The center and right panel show two different low-discrepancy sequences. (Source : Owen (2013)) The local discrepancy of x 1,..., x n [0, 1] s at point a [0, 1] s is δ(a) = δ(a; x 1,..., x n ) = 1 n n 1 xi [0,a) vol([0, a)). i=1 The star discrepancy of those points is D n(x 1,..., x n ) = D n = sup δ(a). a [0,1] s There are certain other important notions of discrepancy. See Niederreiter (1987); Dick and Pillichshammer (2010) for more details. Digital nets and sequences Of special interest here are QMC constructions known as digital nets (Niederreiter, 1987; Dick and Pillichshammer, 2010). We describe them through a series of definitions. Throughout these definitions b 2 is an integer base, s 1 is an integer dimension and Z b = {0, 1,..., b 1}.

21 CHAPTER 2. BACKGROUND 6 Definition For k j N 0 and c j Z b k j for j = 1,..., s, the set s j=1 [ cj b, c j + 1 k j b k j ) is a b-adic box of dimension s. Figure 2.2: Each panel shows the unit square divided into 5-adic boxes. Panels in the left, middle and right columns are divided into 1, 5, and 25 vertical strips respectively. Panels in the top and bottom rows are divided into 1 and 5 horizontal strips respectively. (Source : Owen (2013)) Figure 2.2 taken from Owen (2013) shows some 5-adic boxes for s = 2. Note that in the upper left corner the entire box [0, 1) 2 is trivially, a 5-adic box. Definition For integers m t 0, the points x 1,..., x b m [0, 1) s are a (t, m, s)-net in base b if every b-adic box of dimension s with volume b t m contains precisely b t of the x i. Figure 2.3 show some (0, m, 2)-nets in base 5. The nets have good equidistribution

22 CHAPTER 2. BACKGROUND 7 Figure 2.3: This figure shows two digital nets in the unit square in base 5. The one on the left has 125 points. The one on the right has 625 points. Dark reference lines 1/5 apart and light ones 1/25 apart show boundaries of some 5-adic boxes. (Source : Owen (2013)) (low discrepancy) because boxes [0, a) can be efficiently approximated by unions of b-adic boxes. Digital nets can attain a discrepancy of O((log(n)) s 1 /n). Definition For integer t 0, the infinite sequence x 1, x 2, [0, 1) s is a (t, s)-sequence in base b if the subsequence x 1+rb m,..., x (r+1)b m is a (t, m, s)-net in base b for all integers r 0 and m t. The (t, s)-sequences (called digital sequences) are extensible versions of (t, m, s)- nets. They attain a discrepancy of O((log(n)) s /n). It improves to O((log(n)) s 1 /n) along the subsequence n = λb m for integers m 0 and 1 λ < b. 2.2 Multi-dimensional Variation A detailed account of multi-dimensional variation for QMC can be found in Owen (2005). Here we describe the variation in the sense of Hardy and Krause. We begin with some notation.

23 CHAPTER 2. BACKGROUND 8 For x R s, we write its j-th component as x j. That is x = (x 1,..., x s ). For a, b R s we write a < b or a b if the inequality holds for all s components. For any a, b R s, with a b the hyperrectangle [a, b] is the set {x R s a x b}. For u {1,..., s} we write u as the cardinality of u and u as the compliment of u. The expression x u denotes a u -tuple representing the components x j for j u. Suppose u, v {1,..., s} and x, z [a, b] with u v =. Then the symbol x u : z v represents the point y [a u v, b u v ] with y j = x j for j u and y j = z j for j v. With these notation we can now describe the multi-dimensional variation. Let f be any function defined over [a, b] for a, b R s. The s-fold alternating sum of f over [a, b] is (f; a, b) = v {1,...,s} ( 1) v f(a v : b v ). (2.1) For each j = 1,..., s, let Y j be a partition of [a j, b j ]. A multi-dimensional grid on [a, b] has the form Y = s j=1 Yj. For y Y, the successor point y + is defined by taking y j + to be the successor of y j in Y j. The variation of f over Y is V Y (f) = y Y (f; y, y + ). (2.2) Let Y j denote the set of all partitions on [a j, b j ] and put Y = s j=1 Yj. Then, Definition The variation of f on the hyper-rectangle [a, b] in the sense of Vitali, is V IT (f) = V [a,b] (f) = sup V Y (f). Y Y However, variation in the sense of Vitali is not adequate to study the error in QMC sampling. Instead variation is the sense of Hardy and Krause is used which sums the Vitali variation over [a, b] and its upper faces. Definition The variation of f on the hyper-rectangle [a, b] in the sense of

24 CHAPTER 2. BACKGROUND 9 Hardy and Krause, is V HK (f) = V HK (f; a, b) = V [a u,b u ](f(x u ; b u )). u {1,...,s} The function f has bounded variation in the sense of Hardy and Krause (BVHK) if V HK (f) <. It is easy to check that the function f is BVHK if the complete mixed partial derivative exists. Let 1:s = {1, 2,..., s} and u f denote the partial derivative of f taken once with respect to each variable j u. By convention f = f. If the mixed partial derivative 1:s f exists then, V IT (f) 1:s f(x) dx, and [0,1] s V HK (f) u f(x u :1 u ) dx u. u [0,1] u These and related results are presented in Owen (2005). The most important use of variation in QMC is in the Koksma-Hlawka inequality which we state below. Theorem (Koksma-Hlawka inequality). Let f : [0, 1] s R be a function of bounded variation in the sense of Hardy and Krause. Then for any set of points x 1,..., x n [0, 1] s with n 2, we have 1 n n f(x i ) f(x) dx [0,1] D nv HK (f) s i=1 where Dn is the star discrepancy of the points x 1,..., x n. Proof. See Niederreiter (1992). Numerous constructions are known for which Dn = O((log n) s 1 /n) (Niederreiter, 1992) and so QMC is asymptotically much more accurate than MC when V HK (f) <.

25 CHAPTER 2. BACKGROUND Scrambling Here we consider scrambling of digital nets and state several theorems for [0, 1) s that we can generalize to a general domain D. Let a [0, 1) have base b expansion a = k=1 a kb k where a k Z b. If a has two base b expansions, we take the one with a tail of 0s, not a tail of b 1s. We apply random permutations to the digits a k yielding x k Z b and deliver x = k=1 x kb k. There are many different ways to choose the permutations (Owen, 2003). Here we present the nested uniform scramble from Owen (1995). In a nested uniform scramble, x 1 = π (a 1 ) where π is a uniform random permutation (all b! permutations equally probable). Then x 2 = π a1 (a 2 ), x 3 = π a1 a 2 (a 2 ) and x k+1 = π a1 a 2...a k (a k+1 ) where all of these permutations are independent and uniform. Notice that the permutation applied to digit a k+1 depends on the previous digits. A nested uniform scramble of a = (a 1,..., a s ) [0, 1) s applies independent nested uniform scrambles to all s components of a, so that x j,k+1 = π j aj1 a j2,...,a jk (a j,k+1 ). A nested uniform scramble of a 1,..., a n [0, 1) s applies the same set of permutations to the digits of all n of those points. We illustrate the scrambling algorithm in Figure 2.4 taken from Dick and Pillichshammer (2010). The permutations are applied to each coordinate independently. To illustrate the procedure, we first scrambled the abscissa and then the ordinate in Figure 2.4. Propositions and are from Owen (1995). Proposition Let a [0, 1) s and let x be the result of a nested uniform random scramble of a. Then x U[0, 1) s. Proposition If the sequence a 1,..., a n is a (t, m, s)-net in base b, and x i are a nested uniform scramble of a i, then x i are a (t, m, s)-net in base b with probability 1. Similarly if a i is a (t, s)-sequence in base b, then x i is a (t, s)-sequence in base b with probability 1. In scrambled net quadrature we estimate µ = [0,1) s f(x) dx by ˆµ = 1 n n f(x i ), (2.3) i=1

26 CHAPTER 2. BACKGROUND 11 Figure 2.4: Nested scrambling algorithm. (Source : Dick and Pillichshammer (2010))

27 CHAPTER 2. BACKGROUND 12 where x i are a nested uniform scramble of a digital net a i. It follows from Proposition that E(ˆµ) = µ for f L 1 [0, 1) s. When f L 2 [0, 1) s we can use independent random replications of the scrambled nets to estimate the variance of ˆµ. If V HK (f) < then we obtain Var(ˆµ) = O(log(n) 2(s 1) /n 2 ) = O(n 2+ɛ ) for any ɛ > 0 directly from the Koksma-Hlawka inequality. Surprisingly, scrambling the net has the potential to improve accuracy: Theorem Let f : [0, 1] s R with continuous 1:s f. Suppose that x i are a nested uniform scramble of the first n = λb m points of a (t, s)-sequence in base b, for λ {1, 2,..., b 1}. Then for ˆµ given by (2.3), as n for any ɛ > 0. ( log(n) s 1 ) Var(ˆµ) = O = O(n 3+ɛ ) n 3 Proof. Owen (1997b) has this under a Lipschitz condition. Owen (2008) removes that condition and corrects a Lemma from the first paper. Smoothness is not necessary for scrambled nets to attain a better rate than Monte Carlo. Bounded variation is not even necessary: Theorem Let x 1,..., x n be a nested uniform scramble of a (t, m, s)-net in base b. Let f L 2 ([0, 1] s ). Then for ˆµ given by (2.3), ( 1 Var(ˆµ) = o n) as n. Proof. This follows from Owen (1998). The case t = 0 is in Owen (1997a). The factor log(n) s 1 is not necessarily small compared to n 3 for reasonable sizes of n and large s. Informally speaking those powers cannot take effect for scrambled nets until after they are too small to make the result much worse than plain Monte Carlo:

28 CHAPTER 2. BACKGROUND 13 Theorem Let x 1,..., x n be a nested uniform scramble of a (t, m, s)-net in base b. Let f L 2 ([0, 1] s ) with Var(f(x)) = σ 2 when x U[0, 1] s. Then for ˆµ given by (2.3), ( b + 1 ) s Var(ˆµ) b t σ 2 b 1 n. If t = 0, then Var(ˆµ) eσ 2 /n. = 2.718σ 2 /n. Proof. The first result is in Owen (1998), the second is in Owen (1997a). 2.4 QMC Tractability As we have seen in Section 2.2 the accuracy of a QMC method can be measured using the Koksma-Hlawka inequality which suggests to use low-discrepancy sequences in order to reduce the error of numerical integration. However, they have a cost (in terms of the number of function evaluations) which grows exponentially with s. The idea of QMC tractability is to find a class of functions for which the cost is bounded independently of s. We use a slightly different notation for simplicity. We write the integral as µ s (f) := f(x) dx, (2.4) [0,1] s where the integrand is assumed to belong to some function space H s. We approximate the integral (2.4) using the following QMC method: ˆµ n,s (f) := 1 n n f(t i ) = 1 n i=1 n f(t i,1,..., t i,s ), (2.5) i=1 where t 1,... t n [0, 1] s. Consider the worst-case error of ˆµ n,s which is the worst-case performance of ˆµ n,s over the unit ball of H s ; i.e., e n,s := e(ˆµ n,s ) = sup µ s (f) ˆµ n,s (f), (2.6) f H s, f s 1 where s denotes the norm in H s. For n = 0, we formally set ˆµ 0,s := 0. The

29 CHAPTER 2. BACKGROUND 14 corresponding worst-case error is the initial error e 0,s := sup µ s (f). f H s, f s 1 The aim of the problem is to reduce the initial error by a factor of ɛ, where ɛ (0, 1). Thus, we are looking for the smallest n = n(ɛ, s) for which t 1,..., t n exist such that e n,s ɛe 0,s. We can now define what we mean by QMC tractability. The integration problem (in the worst-case setting) is said to be polynomial tractable iff there exist non-negative C, q, and p such that n(ɛ, s) Cɛ p s q s = 1, 2,... ; ɛ (0, 1). (2.7) If (2.7) holds, then the infima of q and p are called the s-exponent and ɛ-exponent of tractability. The problem is said to be strongly polynomial tractable if (2.7) holds with q = 0. Some years ago, a third relevant notion of tractability was introduced, namely weak tractability (see Novak and Woźniakowski (2010) for details). The integration problem is said to be weakly tractable iff log n(ɛ, s) lim ɛ 1 +s ɛ 1 + s = 0. (2.8) The general notion of tractability can be found in Novak et al. (1997); Woźniakowski (1994a,b). For a detailed account of tractability of multivariate problems, we refer the reader to the trilogy by Novak and Woźniakowski (2008, 2010, 2012). We end this section by giving some results on the necessary and sufficient conditions for polynomial, strong polynomial and weak tractability. We begin defining the function class. Let H 1 denote a reproducing kernel Hilbert space (RKHS). Thus, exists a kernel K 1 : [0, 1] [0, 1] R associated with H 1 which has the reproducing property; namely K 1 (x, y) = K 1 (y, x) for all x, y [0, 1], K 1 (x, ) H 1 for all x [0, 1] and f, K 1 (, y) 1 = f(y) f H 1, y [0, 1].

30 CHAPTER 2. BACKGROUND 15 In some cases, this kernel can be associated with a weight γ. Few popular examples are as follows. K 1 (x, y) = K 1,γ (x, y) = 1 + γ min(x, y) K 1 (x, y) = K 1,γ (x, y) = γ sinh γ cosh( γ(1 max(x, y))) cosh( γ min(x, y)) K 1 (x, y) = K 1,γ (x, y) = 1 + γ 2 (B 2( x y ) + 2(x 1/2)(y 1/2)) where B 2 (x) is the Bernoulli polynomial of degree 2. Each of these kernels correspond to a different RKHS, H 1 = H 1,γ with some inner product. Now, define H s to be a weighted RKHS over [0, 1] s as the tensor product H s = H s,γs := H 1,γs,1 H 1,γs,2 H 1,γs,s, where γ s = (γ s,1,..., γ s,s ) denote the weights used in the kernels. A typical result in this field is of the following form. Theorem Let H s be a tensor product of s weighted reproducing kernel Hilbert spaces (RKHS), where the jth weighted RKHS is parametrized by weights γ s,j for j = 1,..., s. If γ s,j are positive and uniformly bounded, then strong polynomial tractability holds iff lim sup s s γ s,j <, j=1 polynomial tractability holds iff and weak tractability holds iff lim sup s s j=1 γ s,j log(s + 1) < lim s s j=1 γ s,j s = 0. See Chapter 7 for more details of RKHS. The proof of the above result is usually done on a case by case basis by considering different spaces H s. Sloan and

31 CHAPTER 2. BACKGROUND 16 Woźniakowski (1998) prove the result for the weighted Sobolev space W (1,1,...,1) 2 ([0, 1] s ). Hickernell and Woźniakowski (2001) show the result for certain weighted Korobov spaces. Sloan and Woźniakowski (2002) show the result for six different periodic and non-periodic weighted tensor product Hilbert spaces. Many more such results are available in literature. For a detailed account we refer to the trilogy by Novak and Woźniakowski (2008, 2010, 2012).

32 Part I Low-Discrepancy Constructions 17

33 Chapter 3 Low-Discrepancy Constructions on the Triangle 3.1 Introduction The problem we consider here is numerical integration over a triangular domain, using quasi-monte Carlo (QMC) sampling. Such integrals commonly arise in graphical rendering. Classical quadrature methods find a set of points x 1,..., x n in the triangle and weights w i R so that n i=1 w if(x i ) correctly integrates a class of polynomials f. Lyness and Cools (1994) give a survey. Classical rules often do poorly on non-smooth integrands. It is also difficult to estimate error for them and there is little freedom to choose n. As a result, QMC sampling, which equidistributes sample points through the domain of interest is attractive. In recent years there has been renewed interest in integration over other domains. Among these are the sphere (Aistleitner et al., 2012) and the simplex (Brandolini et al., 2013a). The usual approach to sampling these domains is to apply a mapping φ from [0, 1] s to the domain D of interest. The mapping is such that if x U([0, 1] s ) (uniform distribution) then φ(x) U(D). There are typically several choices for such mappings and the dimension s of the cube is not necessarily equal to the dimension of D. With such a mapping in hand we may generate QMC points x i [0, 1] s and 18

34 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 19 use φ(x i ) as sample points in D. Using this approach we may estimate µ = D g(x) dx by (1/n) n i=1 f(x i) where f(x) = g(φ(x)). The difficulty is that the composite function f = g φ may not be well suited to QMC; it may have cusps or singularities or discontinuities. These features may diminish the performance of QMC. At a minimum, they make it more difficult to analyze QMC s performance. Recently Brandolini et al. (2013a) presented a version of the Koksma-Hlawka inequality for the simplex. They devised a measure of variation for the simplex and a discrepancy measure for points in the simplex. But they did not present a sequence of points with vanishing discrepancy. Pillards and Cools (2005) also studied QMC integration over the simplex. They mention that the Koksma-Hlawka bound can be applied using the discrepancy of the original points x i and the variation of the composite function g φ, but do not give conditions for that variation to be finite. They also devised a measure of variation for functions on the simplex, a corresponding discrepancy measure for points inside the simplex, and a Koksma-Hlawka bound using these two factors. But they did not obtain a link between the cube discrepancy of their original points and the simplex discrepancy of the image of those points under φ. Neither Brandolini et al. (2013a) nor Pillards and Cools (2005) provide a QMC construction for the simplex with a vanishing discrepancy. In this chapter we present two constructions for points in the triangle. The first is an extensible digital construction that mimicks the van der Corput sequence and exploits a recursive partitioning of the triangle. The second resembles a hybrid of lattice points (Sloan and Joe, 1994) and the Kronecker construction (Larcher and Niederreiter, 1993). A rectangular grid of points is rotated through a judiciously chosen angle and those that intersect the triangle are retained. We combined recent theorems of Chen and Travaglini (2007) and Brandolini et al. (2013a) to show that our points have vanishing discrepancy. This second construction has better discrepancy but the digital one is extensible and is amenable to digital scrambling among other things. For both of these constructions, the discrepancy of Brandolini et al. (2013a) vanishes as the number n of points increases. The discrepancy of Pillards and Cools

35 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 20 (2005) also vanishes. We believe that these are the first constructions of points in the triangle to which a Koksma-Hlawka inequality applies. An outline of this chapter is as follows. Section 3.2 presents results from the literature that we need along with notation to describe those results. We show there that the discrepancy of Pillards and Cools (2005) is no larger than twice that of Brandolini et al. (2013a) so that the former vanishes whenever the latter does. In Section 3.3 we adapt the van der Corput sequence from the unit interval to an arbitrary triangle. The result is an extensible sequence. We show that the parallelogram discrepancy of Brandolini et al. (2013a) is at most 12/ n when using the first n points of our triangular van der Corput sequence and it is exactly 2/(3 n) 1/(9n) when n = 4 k. Section 3.4 develops a second explicit construction. It rotates a scaled copy of Z 2 through a carefully chosen angle, keeping only those points that lie within a right angle triangle. The resulting points have parallelogram discrepancy O(log(n)/n) and retain that discrepancy when mapped to an arbitrary nondegenerate triangle. Integration over a triangle is an important sub-problem in computer graphics. But there the integrands are often discontinuous and of infinite variation. Quasi-Monte Carlo over the cube has vanishing error so long as f is Riemann integrable (Niederreiter, 1992). Section 3.5 shows that triangular van der Corput points yield integral estimates with vanishing error whenever the integrand is merely Riemann integrable over the triangle. Section 3.6 has some final discussion. Further references We conclude this section by describing some of the literature. Fang and Wang (1994) give volume preserving mappings from the unit cube to the ball, sphere and simplex in d dimensions all of which can be used to generate QMC samples in those other spaces. Pillards and Cools (2005) present 5 different mappings from the unit cube to the simplex. Additionally they consider an approach that embeds the simplex within a cube and ignores any QMC points from the cube that do not also lie in the simplex. Arvo (1995) gives a mapping for spherical triangles. Further mappings are based on probabilistic identities, such as those in Devroye

36 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 21 (1986). These mappings are equivalent when applied to uniform random inputs, but they differ for QMC points. Some are many-to-one, raising the dimension of the input points needed. Others yield points whose coordinates have unfavorably large mixed partial derivatives with respect to the transformed input points in [0, 1] s. Aistleitner et al. (2012) study QMC in the sphere. More generally, Aistleitner and Dick (2015) prove a correspondence principle between multi-variate functions of bounded variation in the sense of Hardy and Krause and signed measures of finite total variation. This gives a simple proof of a generalized Koksma-Hlawka inequality for nonuniform measures. Brandolini et al. (2013b) prove a Koksma-Hlawka-type inequality which applies to piecewise smooth functions f XΩ with f smooth and Ω a Borel subset of [0, 1] s. They improve their main theorem in Brandolini et al. (2013a), which yields results closer to the original Koksma-Hlawka theorem when Ω is an arbitrary parallelepiped or a simplex in R s. 3.2 Background Here we follow the notation that we introduced in the previous Chapter in Section 2.2 and describe some of previous results. Some computations and expressions are simpler with one triangle than they are with another. Let A, B, and C be three non-collinear points in R s. Those points define the non-degenerate triangle (A, B, C) = {ω 1 A + ω 2 B + ω 3 C min(ω 1, ω 2, ω 3 ) 0, ω 1 + ω 2 + ω 3 = 1}. The simplex is usually defined via with corners (0, 0, 1) T (0, 1, 0) T, and (1, 0, 0) T. For some computations it is convenient to use the equilateral triangle defined by A = (0, 0) T, B = (1, 0) T, and C = (1/2, 3/2) T. For some purposes we may scale the points so that our triangle has unit area. At other times one scales the triangle to have area equal to the number n of points in a quadrature rule. Pillards and Cools

37 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 22 (2005) used the right-angle triangle T PC = ((0, 0) T, (0, 1) T, (1, 1) T ). (3.1) Our lattice construction uses ((0, 0) T, (0, 1) T, (1, 0) T ) Discrepancy Here we define the notions of discrepancy that we need, for quadrature problems over a set Ω following the idea of Section 2.1. We follow Brandolini et al. (2013a) in taking Ω to be a bounded Borel subset of R s. We use vol( ) to denote d-dimensional Lebesgue measure. If Ω is contained in a linear flat subset of R s then we interpret volumes as Lebesgue measure with respect to the lowest-dimensional such linear flat. To exclude uninteresting cases, we assume that vol(ω) > 0. For n 1, let P = (x 1,..., x n ) be a list of (not necessarily distinct) points in R s. For a set S R s, we let A n be the counting function, A n (S; P) = n i=1 1 x i S. The signed discrepancy of P at the measurable set S R s is δ n (S; P, Ω) = vol(s Ω) vol(ω) A n(s; P). n The signed discrepancy has a useful additive property. If S 1 S 2 =, then δ n (S 1 S 2 ; P, Ω) = δ n (S 1 ; P, Ω) + δ n (S 2 ; P, Ω). (3.2) Also, δ n ( ; P, Ω) = 0. The absolute discrepancy of points P for a class S of measurable subsets of Ω is D n (S; P, Ω) = sup D n (S; P, Ω), where D n (S; P, Ω) = δ n (S; P, Ω). S S For general Ω it is helpful to extend P by all integer shifts, that is by considering all x i + m Ω for i = 1,..., n and m Z s. Because Ω is bounded, the extension

38 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 23 still has finitely many points. We define the extended count n Ā n (S; P) = m Z s i=1 1 xi +m S and then take D n (S; P, Ω) = sup δ n (S; P, Ω), (3.3) S S where δ n (S; P, Ω) = vol(s Ω)/vol(Ω) Ān(S; P)/n. notice that Ān is divided by n, and not the number of extended points lying in Ω. When Ω is understood, we may simplify the discrepancies to D n (S; P), Dn (S; P). Likewise S can be omitted. Standard quasi-monte Carlo sampling (Niederreiter, 1992) works with Ω = [0, 1) s and takes for S the set of anchored boxes [0, a) with a [0, 1) s. Then D n (S; P) above is the star-discrepancy D n(p). Now let a real-valued function f be defined on [0, 1] s (not just [0, 1) s ) with variation V HK (f) in the sense of Hardy and Krause. Then the Koksma-Hlawka inequality is 1 n n f(x i ) f(x) dx [0,1) D n(p)v HK (f). s i=1 If the needed derivatives are continuous, then V HK (f) = u 1:s,u [0,1] u u x u f(xu :1 u ) dxu. Brandolini et al. (2013a) provide a Koksma-Hlawka inequality for parallelepipeds. Unlike the usual Koksma-Hlawka inequality, their variation measure sums integrals over all faces of all dimensions of the parallepiped. They then represent the indicator function of a simplex defined by s + 1 corner points as the weighted sum of indicators of s + 1 parallepipeds. The j th parallepiped has one vertex at the j th corner of the simplex and its s defining vectors extend from that j th vertex to the other s corners. Their non-negative weighting function varies spatially, summing to 1 within

39 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 24 B F L b E L a D T a,b,c b A a C Figure 3.1: The construction of the parallelogram T a,b,c = CDF E. the simplex. They then obtain a Koksma-Hlawka inequality for the simplex based on their inequality for parallelepipeds. Here we present their discrepancy measure for the case of a triangle with corners A, B and C. For real values a and b, let T a,b,c be the parallelogram defined by the point C with vectors a(a C) and b(b C). One such parallelogram is illustrated in Figure 3.1 where it has vertices C, D, F and E. Let S C = {T a,b,c 0 < a < A C, 0 < a < B C } (3.4) and define S A and S B analogously. Then the parallelogram discrepancy of points P for Ω = (A, B, C) is D P n(p; Ω) = D n (S P ; P, Ω), for S P = S A S B S C. Pillards and Cools (2005) also define a discrepancy for simplices. For simplices with three vertices, their Ω is the triangle T PC from (3.1). They measure discrepancy using anchored boxes, studying D PC n (P; T PC ) = D n (S I, P, T PC ) where S I = {[0, a) a [0, 1) 2 }. (3.5) Lemma Let T PC be the triangle from (3.1) and for n 1, let P be the list of points x 1,..., x n T PC. Then Dn PC (P, T PC ) 2Dn(P, P PC T PC ) and D n (P, T PC ) 2 D n(p, P T PC ).

40 CHAPTER 3. LOW-DISCREPANCY CONSTRUCTIONS 25 Proof. Let [0, a 1 ) [0, a 2 ) be an anchored box in [0, 1] 2. difference [0, a 1 ) [0, 1) [0, a 1 ) [a 2, 1) of sets in S C We may write it as the taking C to be the vertex (0, 1) T of T PC. Then Dn PC (P; T PC ) 2D n (S C, P, T PC ) 2Dn(P, P T PC ). The same argument holds for D n. From Lemma we see that a sequence with vanishing parallel discrepancy will also have vanishing discrepancy in the sense of Pillands and Cools Koksma-Hlawka Brandolini et al. (2013a) define a corresponding variation measure that we will call V P (f). The specialization of this measure to the triangle appears on the last page of their article. Rather than reproduce it here we remark that it is a weighted sum of some integrals over the triangle, some integrals over the edges of the triangle, and function evaluations at the corners of the triangle. The corner evaluations are absolute values of f at those corners. The edge integrals are averages of f plus the absolute value of the interior directional derivative of f along that edge. The integrand on the whole triangle sums the absolute value of 3f as well as first order directional derivatives of 2f and second order directional derivatives of f. The entire sum is multiplied by a constant C 2 > 0 known to be finite. Note that their variation is positive for (nonzero) constant functions. The numerical treatment of sample points x i is different when those points are on the boundary of Ω. Let Ω be a closed polytope in R s not lying in a flat of dimension s 1 or less. Then we define the weight function 0, x Ω, w Ω (x) = 1, x the interior of Ω, 2 k s x a k-dimensional face of Ω. The integer k is understood to be the smallest dimension of any face of Ω that contains x. When Ω lies in a lower dimensional flat we work instead with the relative interior of Ω and similarly replace s by the smallest containing dimension. Given P with