Numericl Integrtion Wouter J. Den Hn London School of Economics c 2011 by Wouter J. Den Hn June 3, 2011
Qudrture techniques I = f (x)dx n n w i f (x i ) = w i f i i=1 i=1 Nodes: x i Weights: w i
Qudrture techniques Two versions: I = f (x)dx n w i f (x i ) i=1 Newton Cotes: equidistnt nodes & "best" choice for the weights w i Gussin Qudrture: "best" choice for both nodes nd weights
Monte Crlo techniques pseudo: implemetble version of true Monte Crlo qusi: looks like Monte Crlo, but is something different nme should hve been chosen better
Power Newton-Cotes: With n nodes you get exct nswer if f is (n 1) th -order polynomil ccurte nswer f is close to n th -order polynomil Gussin: With n nodes you get exct nswer if f is (2n 1) th -order polynomil ccurte nswer f is close to (2n 1) th -order polynomil
Power (Pseudo) Monte Crlo: ccurcy requires lots of drws Qusi Monte Crlo: definitely better thn (pseudo) Monte Crlo nd will dominte qudrture methods for higher-dimensionl problems
Ide behind Newton-Cotes function vlues t n nodes = you cn fit (n 1) th -order polynomil & integrte the pproximting polynomil f (x)dx P 2 (x)dx It turns out tht this cn be stndrdized see Section "extr" for derivtion
Simpson for one intervl (3 nodes) f (x)dx ( 1 3 f 0 + 4 3 f 1 + 1 ) 3 f 2 h
Simpson for multiple intervls (n+1 nodes) = ( 1 f (x)dx 3 f 0 + 4 3 f 1 + 1 ) 3 f 2 h ( 1 + 3 f 2 + 4 3 f 3 + 1 ) 3 f 4 h + ( 1 + 3 f n 2 + 4 3 f n 1 + 1 ) 3 f n h ( 1 3 f 0 + 4 3 f 1 + 2 3 f 2 + 4 3 f 3 + 2 3 f 4 + 2 3 f n 2 + 4 3 f n 1 + 1 ) 3 f n h
Gussin qudrture Could we do better? Tht is, get better ccurcy with sme mount of nodes? Answer: Yes, if you re smrt bout choosing the nodes This is Gussin qudrture
Guss-Legendre qudrture Let [, b] be [ 1, 1] cn lwys be ccomplished by scling Qudrture 1 1 f (x)dx n ω i f (ζ i ). i=1 Gol: Get exct nswer if f (x) is polynomil of order 2n 1 Tht is with 5 nodes you get exct nswer even if f (x) is 9 th -order polynomil
Implementing Guss-Legendre qudrture Get n nodes nd n weights from computer progrm ζ i, i = 1,, n, ω i, i = 1,, n Clculte the function vlues t the n nodes, f i i = 1,, n Answer is equl to Anybody could do this n ω i f i i=1 How does the computer get the nodes nd weights?
2n equtions for nodes nd weights To get right nswer for f (x) = 1 1 1 1dx = To get right nswer for f (x) = x 1 1 xdx = To get right nswer for f (x) = x 2 n ω i 1 i=1 n ω i ζ i i=1 etc 1 1 x 2 dx = n ω i ζ 2 i i=1
2n equtions for nodes nd weights To get right nswer for f (x) = x j for j = 0,, 2n 1 1 1 x j dx = n ω i ζ j i j = 0, 1,, 2n 1 i=1 This is system of 2n equtions in 2n unknowns.
Wht hs been ccomplished so fr? By construction we get right nswer for f (x) = 1, f (x) = x,, f (x) = x 2n 1 But this is enough to get right nswer for ny polynomil of order 2n 1 2n 1 f (x) = i x i i=0 Why?
Guss-Hermite Qudrture Suppose we wnt to pproximte f (x)e x2 dx with n ω i f (ζ i ) i=1 The function e x2 is the weighting function, it is not used in the pproximtion but is cptured by the ω i coeffi cients
Guss-Hermite Qudrture We cn use the sme procedure to find the weights nd the nodes, tht is we solve them from the system: x j e x2 dx = n ω i ζ j i for j = 0, 1,, 2n 1 i=1 Note tht e ζ2 i is not on the right-hnd side
Implementing Guss-Hermite Qudrture Get n nodes, ζ i, i = 1,, n, nd n weights, ω i, i = 1,, n, from computer progrm Clculte the function vlues t the n nodes, f i i = 1,, n Answer is equl to n ω i f i i=1
Expecttion of Normlly distributed vrible How to clculte E [h(y)] with y N(µ, σ 2 ) Tht is, we hve to clculte ) 1 σ (y µ)2 h(y) exp ( 2π 2σ 2 dy Unfortuntely, this does not exctly fit the Hermite weighting function, but chnge in vrible will do the trick
Chnge of vribles If y = φ(x) then g(y)dy = φ 1 (b) φ 1 () g(φ(x))φ (x)dx Note the Jcobin is dded
Chnge of vribles The trnsformtion we use here is x = y µ σ 2 or y = σ 2x + µ
Chnge of vribles E [h(y)] = 1 σ 2π ) (y µ)2 h(y) exp ( 2σ 2 dy = 1 ( σ 2π h( 2σx + µ) exp x 2) σ 2dx = 1 h( ( 2σx + µ) exp x 2) dx π
Wht to do in prctice? Obtins n Guss-Hermite qudrture weights nd nodes using numericl lgorithm. Clculte the pproximtion using E [h(y)] n i=1 1 ω GH π i ( ) h 2σζ GH i + µ Do not forget to divide by π! Is this mzingly simple or wht?
Extr mteril Derivtion Simpson formul Monte Crlo integrtion
Lgrnge interpoltion Let L i (x) = (x x 0) (x x i 1 )(x x i+1 ) (x x n ) (x i x 0 ) (x i x i 1 )(x i x i+1 ) (x i x n ) f (x) f 0 L 0 (x) + + f n L n (x). Wht is the right-hnd side? Do I hve perfect fit t the n + 1 nodes?
Simpson: 2nd-order Newton-Cotes x 0 =, x 1 = ( + b)/2, x 2 = b, or x 1 = x 0 + h, x 2 = x 0 + 2h Using the Lgrnge wy of writing the 2 nd -order polynomil, we get f (x)dx f 0 L 0 (x) + f 1 L 1 (x) + f 2 L 2 (x) = f 0 L 0 (x)dx + f 1 L 1 (x)dx + f 2 L 2 (x)dx
Amzing lgebr Why mzing? L 0 (x)dx = 1 3 h L 1 (x)dx = 4 3 h L 2 (x)dx = 1 3 h formul only depends on h, not on vlues x i nd f i Combining gives f (x)dx P 2 (x)dx = ( 1 3 f 0 + 4 3 f 1 + 1 ) 3 f 2 h.
True nd pseudo Monte Crlo To clculte n expecttion Let x be rndom vrible with CDF F(x) Monte Crlo integrtion: h(x)df(x) T t=1 h(x t), T Use rndom number genertor to implement this in prctice
True nd pseudo Monte Crlo Wht if integrl is not n expecttion h(x)dx = (b ) h(x)f b (x)dx, where f b is the density of rndom vrible with uniform distribution over [, b], tht is, f b = (b ) 1. Thus, one could pproximte the integrl with h(x)dx (b ) T t=1 h(x t), T where x t is generted using rndom number genertor for vrible tht is uniform on [, b].
Qusi Monte Crlo Monte Crlo integrtion hs very slow convergence properties In higher dimensionl problems, however, it does better thn qudrture (it seems to void the curse of dimensionlity) But why? Pseudo MC is simply deterministic wy to go through the stte spce Qusi MC tkes tht ide nd improves upon it
Qusi Monte Crlo Ide: Fill the spce in n effi cient wy Equidistributed series: A sclr sequence {x t } T t=1 is equidistributed over [, b] iff T b b lim T T f (x t ) = f (x)dx t=1 for ll Riemn-integrble f (x). Equidistributed tkes the plce of uniform
Qusi Monte Crlo. Exmples ξ, 2ξ, 3ξ, 4ξ, is equidistributed modulo 1 for ny irrtionl number ξ. 1 The sequence of prime numbers multiplied by n irrtionl number (2ξ, 3ξ, 5ξ, 7ξ, ) 1 Frc(x) (or x Modulo 1) mens tht we subtrct the lrgest integer tht is less thn x. For exmple, frc(3.564) = 0.564.
Multidimensionl For d-dimensionl problem, n equidistributed sequence {x t } T t=1 D Rd stisfies T µ(d) lim T T f (x t ) = f (x)dx, t=1 D where µ(d) is the Lebesque mesure of D.
Multidimensionl equidistributed vectors Exmples for the d-dimensionl unit hypercube: Weyl: x t = (t p 1, t p 2,, t p d ) modulo 1, where p i is the i th positive prime number. Neiderreiter: x t = (t2 1/(d+1), t2 2/(d+1),, t2 d/(d+1) ) modulo 1
References Den Hn, W.J., Numericl Integrtion Most text books on numericl methods will hve chpter on this topic