Modern Methods for high-dimensionl qudrture WS 213/14 A. Griewnk, H. eövey V: Die. 11-13 RUD 25, 1.13 Mi. 9-11 RUD 25, 1.13 UE: Die. 13-15 RUD 25, 1.13 http://www2.mthemtik.huberlin.de/ gggle/w1314/mqi/uebung/qudrture.pdf I Review of 1-dimensionl Integrtion - Polynomil interpoltion Newton nd grnge Error of interpoltion Hermite interpoltion - Integrtion bsed on interpoltion: Bsic rules Coefficient determintion Error nlysis Chnge of intervls - Orthogonl Polynomils nd Gussin Integrtion: Approximtion theory nd orthogonl systems Prcticl Construction of orthogonl polynomils: 3-term recursion Bsic Ide nd differences with numericl integrtion bsed on interpoltion Golub/Eigenvlue formultion of 3-term recursion Bsic theorems of Gussin integrtion - Romberg nd dptive integrtion (if time permits): Romberg lgorithm: Bsic description nd convergence Adptive qudrture: Bsic description of subdivision schemes, qudrture selection nd stop-criteris - Generl error theory from Peno (trunction error): Peno s theorem for Newton-Cotes nd Guss rules II Monte Crlo nd Qusi Monte Crlo Methods - Product rules, curse of dimensionlity - Monte Crlo (MC) methods: Clssicl/plin Monte Crlo integrtion Pseudo-rndom number genertion
Inverse-trnsform method nd cceptnce rejection Importnce smpling Vrince reduction techniques - Discrepncy nd uniform distributed sequences: Bsic results nd definitions Reltion to high dimensionl integrtion (Koksm-Hlwk inequlity) - Qusi-Monte Crlo (QMC) Methods: Elementl intervls nd low discrepncy sequences (t,m,d)-nets nd (t,d)-sequences. Digitl nets nd sequences Discrepncy upper-bounds for (t,m,d)-nets nd (t,d)-sequences Bsic constructions: Hlton, Sobol. ttice Rules Integrtion error for lttice rules for periodic integrnds - Applictions of QMC to Finnce, prticle physics, stochstic progrmming Brownin motion pths genertion methods The qurtic oscilltor Two-stge stochstic progrms - Weighted Reproducing Kernel Hilbert Spces (WRKHS) (if time permits) Setting nd trctbility Results for ttice rules nd digitl nets III Vritions nd Extensions of QMC - Rndomiztion of QMC (if time permits) Rndomly shifted lttice rules nd errors in (WRKHS) of non-periodic nture Rndomly digitlly shifted (t,m,d)-nets nd (t,d)-sequences. Rndom Scrmbling of (t,m,d)-nets nd (t,d)-sequences Prcticl error estimtion - Sprse grids (if time permits) Bsic constructions Error bounds for integrtion - Anlysis of Vrince (ANOVA): Definition of functionl ANOVA decomposition Properties Effective dimensions
1 Review of 1-D integrtion 1.1 grngin Interpoltion Proposition 1.1 Suppose (x i, y i ) with x < x 1 < x n b nd y i = f(x i ) with f C n,1 ([, b]). Then the unique interpolnt P n (x) = n f(x i )l i (x) with l i (x) = i j= x x j x i x j stisfies f(x) P n (x) where is ipschitz constnt of f (n), possibly (x x i ) (n + 1)!, }{{} q n(x) sup f (n+1) (x). x b Proof: If x = x i for some i n, then equlity holds. Suppose now x x i for ll i n. Then q n (x) nd we cn define the function φ(t) = f(t) P n (t) λ x q n (t), with λ x := f(x) Pn(x) q n(x). Since φ C n,1 ([, b]), nd φ vnishes t n + 2 distinct points x, x, x 1,..., x n, by Rolle s theorem we hve tht φ hs t lest n + 1 distinct zeros in [, b]. By following this rgument n times we obtin tht φ n hst two distinct zeros, sy ξ 1, ξ 2, in [, b]. But then we hve tht = φ n (ξ 1 ) φ n (ξ 2 ) = f n (ξ 1 ) f n (ξ 2 ) + P n n (ξ 2 ) P n n (ξ 1 ) + λ x (q n n(ξ 2 ) q n n(ξ 1 )) Since the deg(p n ) = n, then P n n (ξ 2 ) P n n (ξ 1 ) =. Since deg(q n ) = n + 1 nd q n monic, then q n n(t) = (n + 1)!t +, for some R. Therefore it follows = f n (ξ 1 ) f n (ξ 2 ) + λ x (n + 1)!(ξ 2 ξ 1 ) Due to the ipschitz condition on f n, we hve f(x) P n (x) = λ x q n (x) = f n (ξ 2 ) f n (ξ 1 ) (n + 1)!(ξ 2 ξ 1 ) q n(x) (n + 1)! q n(x). Remrk: Smoothness = Differentibility Requirement usully quite high for clssicl higher order methods, but much lower for mny modern pplictions nd suitble methods. Consequence: Bound of qudrture t Chebyshev points f P n = sup f(x) P n (x) q n x b (n + 1)!, where q n (x) = (x x )(x x 1 ),..., (x x n ) is rbitrry except for being monic, i.e., hving the leding term x n.
emm 1.2 For [, b] = [, 1], q n is minimized by the Chebyshev polynomil Proof: See ny numerics text book. q n (x) = 1 2 n T n+1 (x) where T n (x) = cos(n rccos(x)) [, 1]. Substitution in bove formul yields f P n (n + 1)!2 n, when interpoltion is crried out t Chebyshev points, i.e., n + 1 roots of T n+1. 1.2 Simple Qudrtures bsed on Interpoltion Suppose we cn pick A i for i =,..., n such tht Q n (f) := n A i f(x i ) f(x)dx holds exctly for ll polynomils of degree n. Then, if P interpoltes f on the points (x i ),...,n, we hve f(x)dx Q n (f) f(x)dx b q (n+1)! n (x) dx (b ) q (n+1)! n (x). P n (x) When [, b] = [, 1], the lst bound is minimized ccording to emm 1.2 by emm 1.3 When [, b] = [, 1], of the second kind for which 2 (n + 1)!2 = n (n + 1)! 2 n q n (x) = 1 2 n+1 U n+1(x) with U n+1 = q n (x) dx = 1 2 n+1 +1 q n (x) dx is miniml by the Chebyshev polynomil U n+1 dx = 1 2 n. sin((n + 2) rccos(x)) 1 x 2 Consequence Estimte for qudrture points (x i ),...,n, tken s the roots of the Chebyshev polynomil of second kind f(x)dx Q n (f) (n + 1)!2 n.
Hence, ll we gined is fctor of 2. Newton-Cotes formuls for uniform grid sup x 1 x i = + 2i/n for i =,..., n ( 2i (x x i ) (x i + 1) = n i=1 i=1 ) = 2n n n n! Consequence Estimte for Newton-Cotes f(x)dx Q n (f) (n + 1)! 2 n n n n! = (n + 1) ( ) n 2 n Rtio Chebyshev points to uniform grid Significnt, but not overwhelming gin. Exmple Trpezoidl Rule n = 1, x =, x 1 = +1 n n (n + 1) ( n ) n ( ( e n 2 (n + 1)!2 n 2 = /n! n 4 Stirling 4) 3 = A = f(x) = A f() + A 1 f(1) l (x) = = 1 x2 4 A 1 = 1 by symmetry 2 = fter liner trnsformtion q n (x) dx = 1(1 x)dx 2 +1 = 1 1 4 1 4 = 1 2 (1 x 2 )dx = 2 x3 3 (x + 1)(x 1) dx 1 ) n = 2 1 3 1 3 = 4 3 f(x)dx 1 2 (f() + f(+1)) 4 2 3 = 2 3 f(x)dx 1 2 [f() + f(b)] (b ) 3 12 Composite trpezoidl rule f(x)dx [ 1 2 n [f() + f(b)] + (b )3 12n 2 f( + ih)] i=1 f 2 (ξ) (b )3 = (b ) 12n 2 12 h2
This is second Order Method, i.e., doubling of n for even n developed by n leds to qurtering of errors. Exmple Composite Simpson f(x)dx h [ n 2 n 2 f(x ) + 2 f(x 2i 2 ) + 4 f(x 2i ) + f(x n ) ] 3 i=2 i=1 (b ) 18 h4 Remrk Composite Newton-Cotes methods hve the dvntge tht, when n is doubled, ll old vlues cn be reused. Not possible for Chebyshev qudrture. Question: Is it possible to design qudrture using n + 1 points tht is exct for polynomils of degree m n? Answer: Yes. By Gussin qudrture yielding m = 2n + 1. Summry of I.1 nd I.2 = f C n,1 ([, b]), q n = n (x x i ) monic f(x)dx = n ( ) A i f(x i ) + O qn (n+1)! A i = l i (x)dx with l i = n i j= Exctness : f polynomil of degree n = n f(x)dx = A i f(x i ). (x x j ) (x i x j ) Question: Cn the nodes = (n + 1) degrees of freedom be chosen such tht exctness holds for ll f Π 2n+1, i.e., of degree 2n + 1? Answer: Yes. By Gussin integrtion formul. 1.3 Guss Integrtion Generlized gol: For weight function < w(x) C[, b] (or piecewise continuous) try to chieve n f(x)w(x)dx = A i f i (x) for ll polynomils f of degree 2n + 1. First order cse: n = 2n + 1 = 1 w(x) = A 1 xw(x)dx = A x x = xw(x)dx/ w(x)dx [, b]
Exmple: w(x) = e x = e e + 1 = 1, x = e 1.7 xe x dx = xe x 1 e x dx e x dx = e 1 Generl Derivtion: Π[, b] = polynomils on [, b] form inner product spce w.r.t. f, g = f 2 = f(x)g(x)w(x)dx f(x) 2 w(x)dx Π n [, b] = {P Π[, b] : deg(p ) n} is Hilbert spce of finite dimension IR n+1. Monic representtion: f(x) = n ϕ i x i, g(x) = n γ i x i f, g = f Hḡ, f = (ϕi ),...,n, ḡ = (γ i ),...,n H = (h ij ) = ( x i x j dx ) ( ) = 1 1+i+j H is clled Hilbert mtrix nd is terribly ill-conditioned. j,,...,n emm 1.4 Grm-Schmidt orthogonliztion If {f i } n Π re linerly independent nd nonzero, i.e. bsis of their spn, then the recursion υ = f υ 1 = f 1 υ, f 1 υ υ k = f k k υ j, f k υ j k = 2,..., n genertes n orthogonl bsis {v i } n, tht is { υj υ j, υ k = 2 > if j = k, otherwise. Proof: Simple induction. j=1 Proposition 1.5 For the genertion of orthogonl polynomils with respect to the inner product f, g = Define P (x) =, P (x) = 1, nd where f(x)g(x)w(x)dx, one obtins 3-term recurrence in the following form: P k+1 (x) = (x α k+1 )P k (x) γ k P k (x) k =, 1,..., α k+1 = xp k,p k for k =, 1,... γ k = for k = 1, 2,.... Then the sequence P, P 1, P 2,... generted in this form is orthogonl.
Proof: By induction in k P 1, P = x xp,p P,P P, P if orthogonlity holds for k 1, k 2, then In ddition, for j < k 2 we hve = xp, P xp, P = P k, P k = xp k, P k α k P k, P k γ k P k, P k 2 = xp k, P k α k P k, P k = P k, P k 2 = xp k, P k 2 γ k P k 2, P k 2 = P k, xp k 2 P k, P k = = P k, P k + αp k 2 + γp k 3 P k, P k = P k, P j = (x α k )P k + γ k P k 2, P j = xp k, P j = P k, xp j =, since xp j is in spn{p,..., P k 2 }, which is orthogonl to P k by inductive ssumption. Exmple: egendre Polynomils: [, b] = [, 1], w(x) = 1, α 1 = α 2 = γ 1 = xdx/ x 3 dx/ x 2 1/ 1dx =, P 1 = x x 2 dx = 1dx = 1 3 P 2 = x 2 1 3 nd son on P 3 = x 3 3 5 x, P 4 = x 4 6 7 x2 + 3 35 Theorem 1.6 et < w(x) C[, b], nd consider the qudrture Q n (f) = n A if(x i ), where x i, i n, re defined s the n + 1 zeros of the orthogonl polynomil P n+1 (orthogonl to Π n [, b]). Then Q n is exct for ll polynomils in Π 2n+1 [, b]. The coefficients A i re given by nd sum to w(x)dx. A i = Proof: By polynomil division w(x)l 2 i (x)dx > for i =, 1,..., n f Π 2n+1 [, b] = P n (x)s(x) + R(x)
with deg(s(x)) n nd deg(r(x)) n f(x)w(x)dx = q n (x)w(x)s(x)dx + R(x)w(x)dx = Q n (R) = Q n (f). The squred grngins l 2 i (x) hve degree 2n nd re therefore integrble exctly. The exctness on f(x) = 1 implies tht n A i = w(x)dx. Remrk 1.7 The theorem holds lso for discontinuous w, s long s the inner product is well defined for polynomils. Corollry 1.8 When w 1 for f C ([, b]) nd P Π 2n+1 [, b], then the Guss qudrture Q n yields f(x)dx Q n (f) 2 (b ) f P Proof: n (f(x) P (x))dx Q n (f P ) (b ) f P + A i f(x i ) P (x i ) 2(b ) f P. Question: Which re cndidtes for P? Tke Chebychev of order (2n + 1) when f C 2n+1,1, then we hve by emm I.2 for [, b] = [, 1]. f P (2n + 2)!2 2n+1, Resulting estimte for qudrture f(x)dx Q n (f) (2n + 2)!2 2n corresponds to tht for simple qudrture t (2n+1) Chebyshev points. Shrp estimtion by Hermite interpoltion Proposition 1.9 If f C 2n+2 ([, b]) nd x < < x n < x n b, then there exists unique polynomil of degree 2n+1 such tht } P (x i ) = f(x i ) P (x i ) = f for i n (x i ) nd for ll x [, b] for some men-vlue < ξ < b. f(x) P (x) = f (2n+2) (ξ) (2n + 2)! (x x i ) 2
Corollry 1.1 The error of Gussin qudrture is given by f(x)w(x)dx Q n (f) = f (2n+2) (ξ ) b (x x i ) 2 w(x)dx. (2n + 2)! If w(x) = 1 nd [, b] = [, 1], we hve by emm 1.3 (x x i ) 2 dx 1 2 2n i.e., we gin t most fctor of 2 compred to simple rgument. Clculting Guss Points nd Weights One possibility is to compute the roots of the polynomil P n+1 (x) by Newton method. Slick lterntive: Reformultion s symmetric eigenvlue problem. Recursion P (x) =, P (x) = 1, nd where P k+1 (x) = (x α k+1 )P k (x) γ k P k (x) k =, 1,..., α k+1 = xp k,p k for k =, 1,... γ k = for k = 1, 2,.... cn be rewritten using the normlized polynomils ˆP k = P k P k (i.e. replcing P k = ˆP k P k ) nd dividing the recurrence by P k s ˆP (x) =, ˆP (x) = 1/(β ) with β = w(x)dx, nd βk+1 ˆPk+1 (x) = (x α k+1 ) ˆP k (x) β k ˆPk (x) k =, 1,..., where α k+1 = x ˆP k, ˆP k = xp k,p k for k =, 1,... β k = for k = 1, 2,.... Thus we hve tht α k+1 is the sme s in the orthogonl 3-term recurrence, nd now β k = γ k. The 3-term recurrence cn be written in mtrix form s following ˆP (x) α 1 β1 β1 ˆP (x) α 2 β2 β2 x = α 3 β3 + βk βk ˆP k (x) α k+1 ˆP }{{} k (x) βk+1 ˆPk+1 (x) T For the roots x i of ˆP k+1 (being the sme roots of P k+1 ) we obtin the eigenvlue problem P (x i ) T v (i) = v (i) x i with v (i) =. IR n+1. P k (x i ) All eigenvlues re rel since T is symmetric mtrix.
Proposition 1.11 (Golub nd Welsh 1968) The (n + 1) roots x i for i =,..., n of P n+1 re distinct, rel nd cn be clculted by eigenvlue decomposition of T. Weights A i re given by ( A i = v (i) 1 ) 2 w(x)dx where v (i) 1 is the first component of the normlized eigenvector. Proof: Golub nd Welsh Remrk: Eigenvlue decomposition is computed by OR lgorithm on T. Symmetry nd boundedness is mintined so tht everything cn hppen in (n) opertions per itertion.