Hochdimensionale Integration

Transcription

1 Oliver Ernst Institut für Numerische Mathematik und Optimierung Hochdimensionale Integration 4-tägige Vorlesung im Wintersemester 2/ im Rahmen des Moduls Ausgewählte Kapitel der Numerik

2 Contents. Introduction. An Example.2 A Selection of Strategies 2. Monte Carlo Integration 2. Convergence and Accuracy 2.2 Sampling Methods 2.3 Variance Reduction Methods 3. Sparse Grids 4. Quasi-Monte Carlo Integration 5. Extensions 5. ANOVA Decomposition 5.2 Concentration of Measure Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/

3 Inhalt. Introduction. An Example.2 A Selection of Strategies 2. Monte Carlo Integration 2. Convergence and Accuracy 2.2 Sampling Methods 2.3 Variance Reduction Methods 3. Sparse Grids 4. Quasi-Monte Carlo Integration 5. Extensions 5. ANOVA Decomposition 5.2 Concentration of Measure Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 2

4 Object of Study In these lectures we will study numerical methods for approximating definite integrals [,] d f(x ) dx. [, ] d d-dimensional unit cube, d large f : [, ] d R continuous. Today: Why is this useful? What methods are out there? Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 3

5 Inhalt. Introduction. An Example.2 A Selection of Strategies 2. Monte Carlo Integration 2. Convergence and Accuracy 2.2 Sampling Methods 2.3 Variance Reduction Methods 3. Sparse Grids 4. Quasi-Monte Carlo Integration 5. Extensions 5. ANOVA Decomposition 5.2 Concentration of Measure Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 4

6 Example : Pricing Derivatives [Traub & Passkov (995)]: Goldman Sachs CMO Fannie Mae REMIC Trust CMO : Collateralized Mortgage Obligation REMIC : Real Estate Mortgage Investment Conduit Pool of mortgages with 3-year maturity and monthly payment, i.e., 36 cash flows Cash flows distributed across tranches A, B, C, D, E, G, H, J, R, Z. Notation C : monthly payment on underlying pool of mortgages, a 36 k+ : remaining annuity after month k, i k : interest rate in month k, random w k : percentage prepaying in month k, random a k = + v + + v k, v = + i, k =,..., 36, i : current monthly interest rate. Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 5

7 Example : Pricing Derivatives Interest rate model: i k = K e iξ k i k = K k i e ξ+ +ξ k, {ξ k } 36 k= i.i.d. random variables, ξ k N(, σ 2 ), σ = 4 4. Prepayment model: w k = w k (ξ,..., ξ k ) = K + K 2 arctan(k 3 i k + K 4 ), K j constants Cash flow in month k: M k = M k (ξ,..., ξ k ) = C ( w (ξ ) )( w 2(ξ, ξ 2) ) ( w k (ξ,..., ξ k ) ) + w k (ξ,..., ξ k )a 36 k+ Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 6

8 Example : Pricing Derivatives Cash flow distributed to tranches according to CMO rules. Portion of M k directed to tranche T in month k described by function G k,t = G k,t (ξ,..., ξ k ) (continuous, composition of smooth functions with min, complicated). Present value of tranche T in month k: multiply G k,t with discount factor u k = u k (ξ,..., ξ k ) := v v (ξ )... v k (ξ,..., ξ k ), v j (ξ,..., ξ j ) = + i j (ξ,..., ξ j ) = + K j j,..., 359. eξ+ +ξj Sum up values over all months: present value PV of tranche T given by 36 P V T (ξ,..., ξ 36 ) = G k,t (ξ,..., ξ k ) u k (ξ,..., ξ k ). k= Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 7

9 Example : Pricing Derivatives Expected value E(P V T ) = E ( P V T (ξ,..., ξ 36 ) ) after change of variables y i := y i (x i ), x i = 2πσ yi e t2 2σ given by E(P V T ) = P V T (y (x ),..., y 36 (x 36 )) dx dx 36 [,] 36 dt, Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 8

10 Inhalt. Introduction. An Example.2 A Selection of Strategies 2. Monte Carlo Integration 2. Convergence and Accuracy 2.2 Sampling Methods 2.3 Variance Reduction Methods 3. Sparse Grids 4. Quasi-Monte Carlo Integration 5. Extensions 5. ANOVA Decomposition 5.2 Concentration of Measure Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 9

11 Numerical Quadrature Numerical quadrature (integration) of univariate functions is essentially a solved problem. Denoting the desired integral of f by I(f) := f(x) dx quadrature formulas for approximating I(f) have the form I(f) Q N (f) := with nodes {ξ j } N j= and weights {w j} N j=. N w j f(ξ j ) j= Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/

12 Numerical Quadrature Some Simple Quadrature Rules Trapezoidal Rule Composite Trapezoidal Rule Q T N (f) = h f() N + 2 j=2 N-Point Gauß-Legendre Rule Q T (f) = Q GL N (f) = f(jh) + f() 2 N j= f() + f(). 2 w j 2 f, h = N, N 2. ( ) ξj + with {ξ j } N j= [, ] the zeros of the Legendre polynomial of degree N and w j chosen such that the quadrature formula is interpolatory. 2 Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/

13 Numerical Quadrature Some Quadrature Rules Example: f(x) = cos x. 2 Composite Trapezoidal Gauss Legendre 4 (If Q N f)/if N Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 2

14 Multivariate Quadrature, d = 2 Simplest idea: product rule of n-point D quadrature rule: I(f) = f(x ) dx = f(x, y) dy dx [,] 2 x= y= n w j f(x, η j ) dx x= k= j= j= n n w k w j f(ξ k, η j ) Result: N = n 2 -point quadrature rule in 2 dimensions. Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 3

15 Numerical Quadrature Benchmark Problems A widely used set of benchmark problems for multivariate integration schemes [Genz (984)], [Genz (987)] uses the functions ( ) d f (x ) = cos 2πu + a i x i Oscillatory f 2 (x ) = f 3 (x ) = i= i= d a 2 Product Peak i + (x i u i ) 2 ( f 4 (x ) = exp + ( d i= a i x i ) d+ ) d a 2 i (x i u i ) 2 i= Corner Peak Gaussian Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 4

16 Numerical Quadrature Benchmark Problems (cont d) f 5 (x ) = exp ( ) d a i x i u i i= { x > u x d > u d, f 6 (x ) = ( d ) exp i= a ix i, otherwise. C Function Discontinuous u, u = [u,..., u d ] chosen randomly from [, ], inneffective parameters integration problem more difficult with increasing a Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 5

17 f Numerical Quadrature Benchmark Problems Oscillatory: a = 2*rand(d,); u= x y Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 6

18 f Numerical Quadrature Benchmark Problems Product Peak: a = 2*ones(d,); u =.5*ones(d,); y.4.2 Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ x.6.8

19 Numerical Quadrature Benchmark Problems Corner Peak: a = 2*ones(d,); u =.5*ones(d,); f y x Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 8

20 f Numerical Quadrature Benchmark Problems Gaussian: a = 5*ones(d,); u =.5*ones(d,); y.4.2 Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ x.6.8

21 f Numerical Quadrature Benchmark Problems C Function: a = 3*ones(d,); u =.5*ones(d,); y.4.2 Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ x.6.8

22 f Numerical Quadrature Benchmark Problems Discontinuous: a = 5*ones(d,); u =.5*ones(d,); y.8 Oliver Ernst (TU Freiberg) Hochdimensionale Integration x Wintersemester 2/ 2

23 Numerical Quadrature 2D Benchmark Examples With Product Gauss-Legendre (If Q N f)/if (If Q N f)/if 8 (If Q N f)/if N Corner Peak N Gaussian N C Function Summary: For small d numerical integration is tractable. Issues not mentioned: Geometry Adaptivity Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 22

24 Numerical Quadrature in Higher Dimensions For the CMO valuation example d = 36. Assuming integrand smooth (it is not) such that 2 nodes in each direction suffice, a product rule requires function evaluations. N = The supercomputer currently leading the Top 5 list runs at a theoretical peak performance of 2.3 Petaflops, i.e., operations per second. This machine would require 93 seconds, or 3 85 years. The estimated numer of atoms in the observable universe: 8. This fundamental limitation is known as the curse of dimensionality [Bellman (957)]. Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 23

25 Numerical Quadrature in Higher Dimensions Monte Carlo Integration [Neumann, Ulam & Metropolis (946)] Idea: Choose N quadrature nodes {ξ j } N j= [, ]d sampled from a uniform distribution and approximate I(f) = f(x ) dx Q N (f) := N f(ξ [,] N j ). d j= Terminology: I(f) : mean of f, σ 2 (f) := I(f 2 ) [I(f)] 2 : variance of f 64 random nodes. Known: error estimate Figure random points Carlo method is that it does not suffer from th curse of dimensionality: in particular the O(N /2 convergence rate, while slow and erratic, does no depend on the dimension d so long as f is squar I(f) Q N (f) σ(f) (independent of d!), σ(f) Q N (f 2 ) (Q N f) 2. N Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 24

26 Numerical Quadrature in Higher Dimensions Monte Carlo Integration D Examples 2 MC error MC error estimate 2 MC error MC error estimate If Q N f /2 If Q N f 3 4 / N f(x) = cos x N f(x) = x 2 Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 25

27 Numerical Quadrature in Higher Dimensions Monte Carlo Integration 2D Examples 2 MC error MC error estimate 2 MC error MC error estimate If Q N f /2 If Q N f 3 4 / N Corner Peak N C Function Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 26

28 ntroduction concludes with a tion Numerical of the 36-dimensional Quadrature in Higher Dimensions above, since many features of mmon Sparse to problems Gridsfrom math- Figure. Product rule with 64 points. oncerned with the valuation of ge-backed securities held by a [Smolyak (963)] mers of the bank borrow money rs. Each month every customer ay the loan, and of course re- sparse-grid methods, which are generalizations of Idea: Organize points in a product rule in a hierarchical way and only use a construction first devised by Smolyak. Figure 2 certain levels of nodes. shows an example of a regular sparse grid with 49 rly will reduce its value to the odel, the proportion of those ay will depend on the interest e higher the interest rate, the e to repay the loan. The interto follow a (geometric) Brownth-by-month changes in the indom variables, so the present of mortgages is a (suitably dissional expected value, because ble repayment occasions. This -dimensional Euclidean space into an integral over the 36- be by an appropriate variable y other high-dimensional probluding options of all varieties) sional expected values, with the ng either from discretization in cause there are multiple assets acteristics, or both. One feasible strategy is to organize the points of a product rule in a hierarchical way and use only a few levels of points. This is the principle behind points. Figure 2. Sparse grid with 49 points. Smolyak sparse grid with 49 points Modern sparse-grid methods are dimensionadaptive: they find the important dimensions automatically and use more integration points in those dimensions. For details on sparse-grid methods, we refer readers to the recent survey article re Possible? e can approximate an integral by Bungartz and Griebel [2]. Another possible strategy is the Monte Carlo Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 27

29 ntroduction concludes with a tion Numerical of the 36-dimensional Quadrature in Higher Dimensions above, since many features of mmon Sparse to problems Gridsfrom math- Figure. Product rule with 64 points. oncerned with the valuation of ge-backed securities held by a [Smolyak (963)] mers of the bank borrow money rs. Each month every customer ay the loan, and of course re- sparse-grid methods, which are generalizations of Idea: Organize points in a product rule in a hierarchical way and only use a construction first devised by Smolyak. Figure 2 certain levels of nodes. shows an example of a regular sparse grid with 49 rly will reduce its value to the odel, the proportion of those ay will depend on the interest e higher the interest rate, the e to repay the loan. The interto follow a (geometric) Brownth-by-month changes in the indom variables, so the present of mortgages is a (suitably dissional expected value, because ble repayment occasions. This -dimensional Euclidean space into an integral over the 36- be by an appropriate variable y other high-dimensional probluding options of all varieties) sional expected values, with the ng either from discretization in cause there are multiple assets acteristics, or both. One feasible strategy is to organize the points of a product rule in a hierarchical way and use only a few levels of points. This is the principle behind points. Figure 2. Sparse grid with 49 points. Smolyak sparse grid with 49 points Modern sparse-grid methods are dimensionadaptive: they find the important dimensions automatically and use more integration points in those dimensions. For details on sparse-grid methods, we refer readers to the recent survey article re Possible? e can approximate an integral by Bungartz and Griebel [2]. Another possible strategy is the Monte Carlo Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 28

30 Quasi-Monte Carlo methods are equal weight rules, just like the Monte Carlo method, except Numerical Quadrature in Higher Dimensions Figure 5. Fibonacci lattice rule with 5 In this article we focus on lattice rule that the points x,...,x N are now designed in a Quasi-Monte continuing we should acknowledge a disa clever way Carlo to be Methods more uniformly distributed size, each rectangle than will include exactly one point. random of all deterministic methods, such as sp points so that a convergence Details rate on close both totheory and construction of nets and O(N ) is possible. or quasi-monte Carlo, when compared to t (Note, however, that sequences the implied can be found in the book of Niederreiter on but d.) [3]. Figure design 4 the nodes to be more evenly Carlo method namely, that they come Idea: constant Use equal can depend weights exponentially like in MC shows the first 64 points of a 2-dimensional Lattice Sobol rules (deterministically) distributed than random. are a any different practical kind information of quasi-monte about the erro sequence, priori estimates involving, for example, h the first example of the Carlo now method. widelythe points x,...,x N of a lattice rivatives of f are essentially never useful renowned concept of (t,m,s)-nets rule and are so (t,s) regular - that they form a group under tical error estimation.) This has led to a Can achieve sequences O(/N) established rate, by Niederreiter. but constant the operation generally of addition dependent modulo the integers. d. Figure 5 shows a lattice rule with 55 points. interest in hybrid methods that are essen terministic but that also have some el randomness, thereby seeking to capture efits of both approaches. We shall see an Figure random points. of a hybrid method later in the article. More on Lattice Rules There are many kinds of lattice rules (ind the product of left-rectangle rules is a lat but for our purposes it is enough to con the oldest and simplest kind, technical now as a rank- lattice rule, which takes (2) Q N f = N ({ f k z }) Figure 4. First Sobol 64 sequence points of 2D with Sobol 64 Figure sequence. 5. Fibonacci lattice rule with with points.. N N k= Informally, the basic points are now designed idea in ais to have In this the article right we focus points on lattice rules. Before Here z Z number of points in various subcubes. continuing For example, if in Figure 4 we divide the unit of all square deterministic into methods, such as sparse-grid we should acknowledge d is the a disadvantage generating vector, and t indicate that each component is to be rep its fractional part in [, ). In this case the strips of size by /64, then there or is quasi-monte exactly onecarlo, when compared to the Monte group formed by the points is the cyc point in each of the 64 strips, with Carlo any method namely, point on that they come without generated by {z/n}. Without loss of gene method is that it does not suffer from the of dimensionality: in particular the O(N /2 ) rgence rate, while slow and erratic, does not d on the dimension d so long as f is square able. Furthermore, it is cheap and easy to pron effective error estimate, since the first term the variance can be estimated by making use same function values as already used for apmating If. asi-monte Carlo methods are equal weight just like the Monte Carlo method, except he points x,...,x N way to be more uniformly distributed than m points so that a convergence rate close to ) is possible. (Note, however, that the implied ant can depend exponentially on d.) Figure 4 Oliver Ernst (TU Freiberg) Hochdimensionale any practical Integration information about the Wintersemester error. (The 2/ a 29

31 Numerical Quadrature in Higher Dimensions Effective Dimension Consider function f(x ) = f(x, x 2, x 3, x 4 ) = x cos x 2 + x 3. What is the dimensionality of f? (2, 3, 4?) Generally: can write any d-dimensional function as the 2 d -term sum f(x ) = f α (x α ), α {,...,d} where f α depends on only α of the original d variables. Uniquess from requirement f α (x α ) dx j = j α (ANOVA decomposition). Idea: Approximate f by neglecting unimportant terms for efficient quadrature.. Oliver Ernst (TU Freiberg) Hochdimensionale Integration Wintersemester 2/ 3