1/31 Orthogonal Polynomials, Quadratures & Sparse-Grid Methods for Probability Integrals Dr. Abebe Geletu May, 2010 Technische Universität Ilmenau, Institut für Automatisierungs- und Systemtechnik Fachgebiet Simulation und Optimale Prozesse
2/31 Topics Interpolatory Quadrature Rules Orthogonal Polynomials Embedded Quadrature Rules Quadrature Rules and Probability Integrals Full-Grid Tensor Product Multidimensional Integration Sparse-Grid Tensor Product Multidimensional Integration Part - I: Quadrature Rules; Part - II: Sparse-Grid Integration Methods
3/31 1. Interpolatory Quadrature Rules For an integrable function f : R R to compute the integral I[f] = b a f(x)φ(x)dx, where - [a, b] R is a bounde or unbounde interval; - φ : R R is a nonnegative weight function. Sometimes the weight function need to be adjusted to conform to the the properties of the integrand f, e.g. oscillatory behaviors, etc.; and the interval of integration Ω 1.
4/31 1.2. Some integrals with standard weight functions and intervals of integration (a) I 1 [f] = (b) I 2 [f] = (c) I 3 [f] = 1 1 1 1 1 1 f(x)dx φ(x) = 1;Ω 1 = [ 1, 1] f(x)(1 x) α (1+x) β dx φ(x) = (1 x) α (1+x) β,α,β > 1; ( f(x) 1 x 2) 1 2 dx φ(x) = Ω 1 = [ 1, 1] ( 1 x 2) 1 2 ;Ω 1 = [ 1, 1]
5/31 (d) I 4 [f] = (e) I 5 [f] = Note: + 0 + f(x)x α e x dx φ(x) = x α e x,α > 1;Ω 1 = [0,+ ) f(x)e x2 dx φ(x) = e x2 ;Ω 1 = [0,+ ) transform other types of integrals to the standard ones. Example: 1 1 sin(x)e x2 dx 1 1 (sin(x)e x2 1 x 2 ) } {{ } =f(x) ( 1 x 2) 1 2 dx
6/31 1.3. Interpolatory Quadrature Formulas To construct an N-point quadrature formula (rule): N Q 1 N [f] := w k f(x k ), k=1 where the integration nodesx = {x 1, x 2,...,x N } and weights {w 1, w 2,...w N } are constructed based on Ω 1 = [a, b] and the weight function φ. Question: How to determine the 2N unknowns x 1,..., x N and w 1,...,w N? Requirement: the weights w 1, w 2,..., w N should be non-negative to avoid numerical cancelations.
7/31 1.4. Efficient interpolatory quadrature rules 1 Gauss qudrature rules and their extensions 2 Curtis-Clenshaw quadrature rules
8/31 1.5. Polynomial Exactness One measure of quality for a quadrature rule is its polynomial exactness. The N-point quadrature QN 1 [ ] is said to be exact for a polynomial p m of degree m if I[p m ] = b a p m (x)φ(x)dx = Q 1 N [p m] = N w k p m (x k ) k=1 the degree of exactness or degree of accuracy d of an N-point quadrature rule is equal to the maximum degree polynomial for which QN 1 is exact.
1.6.Gauss Quadrature Rules and Orthogonal Polynomials An N-point Gauss quadrature rule is constructed to achieve the largest possible polynomial exactness. Theorem (see Davis & Rabinowitz [1]) An N-point quadrature formula QN 1 has degree of exactness d = 2N 1 if the integration nodes x 1, x 2,...,x N are zeros of the N th degree orthogonal polynomial p N (x) w.r.t. φ and Ω 1 = [a, b]. Hence, there is a polynomial p m with degree m > 2N 1, such that I[p m ] = Q 1 N [p m]. Example: a) A 3-point Gauss quadrature rule Q 1 3 has a polynomial exactness d = 2 3 1 = 5 9/31
10/31 Orthogonal Polynomials Two polynomials p n and p m are orthogonal w.r.t. φ and Ω 1 = [a, b] if < p n, p m >:= b a p n (x)p m (x)φ(x)dx = 0. In Theorem 1, the N th degree orthogonal polynomial p N is orthogonal to all polynomials p m degree m N 1. Examples: (a) p 0 = 1, p 1 (x) = x are orthogonal w.r.t. φ(x) = 1 and Ω 1 = [ 1, 1]. (b) H 1 = 2x, H 2 (x) = 4x 2 2 are orthogonal w.r.t. φ(x) = e x2 and Ω 1 = (,+ ).
11/31 Characterization of Orthogonal Polynomials Theorem (Recurrence relationship, see Gatushi [2]) Suppose that p 0 = 1, p 1,... are orthogonal polynomials with deg(p n ) = n and leading coefficient equal to 1. For every integer n where p n+1 (x) = (x a n )p n (x) b n p n 1 (x), n = 1, 2... ( ) a n = b n = b a xp2 n (x)φ(x)dx b a p2 n(x)φ(x)dx, n = 1,...; b a xp2 n(x)φ(x)dx b n = 1,... a p2 n 1 (x)φ(x)dx,
12/31 (Continued...) Characterization of Orthogonal Polynomials Theorem (Continued...Recurrence relation) and p 1 (x) = (x a 0 )p 0 (x) = x a 0, b a a 0 = xp2 0 (x)φ(x)dx b b a p2 0 (x)φ(x)dx = a xφ(x)dx b a φ(x)dx. Remark: Given a nonnegative weight function φ and a set Ω, you can construct your own set of orthogonal polynomials if you can efficiently compute the recurrence coefficients a 0, a n, b n, n = 1, 2,...
13/31 1.7. Algorithms to determine the recurrence coefficients Algorithms for the computation of the recurrence coefficients are commonly known as Steleje s Procedures. Currently an efficient and stable algorithm for the computation of the coefficients a 0, a n, b n, n = 1, 2,... is given by Gander & Karp [5]. Software: FORTRAN: ORTHOPOL by Gatushi [4] C++ implmentation of ORTHOPOL Fernandes & Atchley [4]. Various Matlab codes by Gatushi [1]
14/31 Some known sets of orthogonal polynomials Jacobi polynomials orthogonal w.r.t. φ(x) = (1 x) α (1+x) β,α,β > 1 and Ω 1 = [ 1, 1]: p (α,β) n (x) = 1 2 n n ( n+α k k=0 )( n+β n k ) (x 1) n k (1+x) k, n = 0, 1, 2,... Lagendre polynomials are the Jacobi polynomials for α = β = 0; i.e. φ(x) = 1: L n (x) = 1 2 n n k=0 ( )( ) n n (x 1) n k (1+x) k, k n k n = 0, 1, 2,...
15/31 Hermite Polynomials w.r.t. φ = e x2 and Ω 1 = (,+ ) [n/2] ( 1) k H n (x) = n! k!(n 2k)! (2x)n 2k, n = 0, 1, 2,... k=1 The first 6 terms of Hermite polynomials: H 0 (x) = 1, H 1 (x) = 2x, H 2 (x) = 4x 2 2 H 3 (x) = 8x 3 12x, H 4 (x) = 16x 4 48x 2 + 12, H 5 (x) = 32x 5 160x 3 + 120x H 6 (x) = 64x 6 480x 4 + 720x 2 120, etc.
16/31 Chebychev Polynomials orthogonal w.r.t. φ(x) = (1 x 2 ) 1 2 and Ω 1 = [ 1, 1] C 0 (x) = 1 2 T 0 (x), C n (x) = π π T n(x), n = 1, 2,... where T n (x) = cos(n acrcos(x)), known as Chebychev polynomials of first kind. Few terms are given by T 0 (x) = 1, T 1 (x) = x, T 2 (x) = 2x 2 1, T 3 (x) = 4x 3 3x, T 4 (x) = 8x 4 8x 2 + 1 T 5 (x) = 16x 5 = 8x 4 20x 3 + 5x, T 6 (x) = 32x 6 48x 4 + 18x 2 1 Corresponding to each set of orthogonal polynomials we have the quadrature rules: Gauss-Jacobi, Gauss-Legendre, Gauss-Hermite, Gauss-Chebychev, etc.
17/31 1.8. Computation of quadrature nodes and weights Theorem (see Gatushi [2, 3]) The quadrature nodes x 1,...,x N and weights w 1, w 2,...,w N can be obtained from the spectral factorization J n = V ΛV; Λ = diag(λ 1,λ 2,...,λ N ), VV = I N ; of the symmetric matrix tridiagonal Jacobi matrix a 0 b1 b1 a 1 b2 J n =. b2........ an 2 bn 1 bn 1 a N 1,
18/31 Theorem (continued...) where a k, b k, k = 1, 2,...,N 1 are known coefficients from the recurrence relation ( ). In particular x k = λ k, k = 1, 2,...,N; ( 2, w k = e1 k) Ve k = 1, 2,...,N. Observe that: all the weights w k obtained above are nonnegative. Exercise: Write a Matlab code not longer than 10 lines to compute x k s and w k s.
19/31 1.9. Gauss Quadrature Rules with Preassigned nodes In some application integration nodes need to be prefixed (pre-given), eg. due to boundary conditions, constraints, etc. Such nodes are usually needed in collocation methods in the solution of ODEs and PDEs. Objective: To construct an N point quadrature rule Q 1 N with x 1,...,x m, m N, are preassigned (fixed) nodes in [a, b]: determine the remaining N m nodes ; correspoindg weights w 1, w 2,..., w N so that QN 1 has the maximum possible degree of exactness d.
20/31 Classical Gauss quadrature rules with preassigned nodes In a closed bounded interval [a, b]: 1 Gauss-Radau rule: - One end point of [a, b] is prefixed to be a node; i.e. either x 1 = a or x N = b; the rest of the nodes x 2, x 3,...,x N (a, b). - Degree of polynomial exactness d = 2N 2. 2 Gauss-Lobatto rule: - Both end points of the interval [a, b] are prefixed to be nodes; i.e. x 1 = a and x N = b. There rest x 2,...,x N (a, n). - Degree of polynomial exactness d = 2N 3. In general, prefixing nodes reduces the degree of polynomial exactness by the number of integration nodes prefixed.
21/31 Recently, Bultheel et. al. [2] give Gauss qudrature rules with prefixed nodes and positive weights in any interval Ω 1 R bounded or unbounded. Software: See Gatushi [1] for Matlab codes.
1.10 How good are Gauss Quadrature Rules? Disadvantage: Different Gauss quadrature rules Q 1 N 1 and Q 1 N 2 have only a few common nodes, for N 1 < N 2 ; i.e. the set of nodes X 1 = {x 1, x 2,..., x N1 } [a, b] and X 2 = {z 1, z 2,..., z N2 } [a, b]. = it is difficult to estimate the convergence of the limit: lim I[f] QN 1. [f] N Note that: had it been X 1 X 2, we could have obtained I[f] QN 1 2 [f] I[f] QN 1 1 [f]. Solution: Either extended Gauss quadrature rules so that the nodes for a lower accuracy can be reused to construct quadrature rules of higher accuracy; or construct embedded quadrature rules from the start. 22/31
Hence, X N X 2N+1. 23/31 1.11. Embedded quadrature rules - Extensions of Gauss quadrature rules A) Kronord s extension [6]: Given Gauss quadrature nodes x 1, x 2,..., x N, between every two nodes add one new node: z 1 (a, x 1 ), z 2 (x 1, x 2 ),... z N (x N 1, x n ), z n+1 (x N, b) so that the N + N + 1 = 2N + 1 points z 1, x 1, z 2, x 2,..., x N 1, z N, x N, z N+1 are the nodes of the new quadrature rules; and and the new weights w 1, w 2,..., w 2N+1 are all nonnegative. = The degree to exactness of a Gauss-Kronord rule is: { 3N + 1, if N is even d = 3N + 2, if N is odd
B) Patterson s extension [2]: To an existing N Gauss quadrature nodes x 1, x 2,...,x N, add m new integration nodes z 1, z 2,...,z m so the new quadrature rule has the maximum degree of accuracy d = 2(m+N) N 1 = N + 2m 1 and the weights w 1, w 2,..., w N+m are nonnegative. Hence, X N X N+m. Note: In general, for a given weight function φ and integration domain [a, b], the construction of convenient embedded Gauss-quadrature rules is not a trivial task.(see, [3, 3, 5]) Software: FORTRAN, C++, Matlab codes in the book of Kythe & Schäferkotter [1]. 24/31
Embedded quadrature rules- the Curtis-Clenshaw quadrature rule For integrals on [ 1, 1] (in general on a closed and bounded interval [a, b]) the set of nodes { } (k 1)π X N = x k x k = cos, k = 1, 2,..., N N 1 and the weights are computed from the orthogonal Chebychev polynomials of first type T n (x) = cos(n acrcos(x)) using Theorem 4. Properties: degree accuracy N 1; X N X 2N 1 ; nodes and weight are very simple to compute; (see Trefethen [3],Waldvogel[4] for Matlab codes) despite lower polynomial exactness, comparable efficiency with Gauss quadrature rules ( Trefethen [3]); extensive applications in spectral and pseudo-spectral 25/31
26/31 1.12. Quadrature Rules and Probability Integrals For a well behaved nonnegative function ρ(x), if ρ(x)dx = 1, then the expression μ(x) = ρ(x)dx defines a probability measure. And ρ is termed a probability density function of x. In particular, for a random set A where Pr{x A} = μ(x A) = 1 A (x) = { 1, if x A 0, if x / A. 1 A (x)ρ(x)dx, For function f of the random variable x, its expected value is E[f] = f(x)ρ(x)dx.
27/31 Standard probability density functions Given a nonnegative weight function φ(x) with the property { > 0, for x [a, b] φ(x) = 0, if x / [a, b]. such that φ 0 = b a φ(x) φ(x)dx set ρ(x) := φ 0, then the expression μ(x) = ρ(x)dx defines a probability measure. Example: For φ(x) = 1 on [a, b]: φ 0 = b a ρ(x) = 1 = 1 the uniform density function. φ 0 b a
28/31 For φ(x) = e x2 on (,+ ): φ 0 = + e x2 = π ρ(x) = φ(x) φ 0 = e x2. π Hence, F(x) = x e z2 x e 1 2 ( 2z) 2 dz = 2dz π 2 π Setting u = 2z we have du = 2dz and F(x) = x e z2 x e 1 2 u2 dz = du. π 2π F defines the standard normal distribution.
29/31 Literature Davis, P. J.; Rabinowitz, P. Methods of numerical integration. Dover Publications, 2nd ed., 2007. Bultheel, A.; Cruz-Barroso, R.; Van Barel, M. On Gauss-type quadrature formulas with prescribed nodes anywhere on the real line. Calcolo, 47(2010) 21 48. Elhay, S.; Kautsky J. Generalized Kronrod Patterson type embedded quadratures. Aplikace Matematiky, 37(1992) 81 103. Fernandes, A. D.; Atchley, W. R. Gaussian quadrature formulae for arbitrary positive measure. Evolutionary Bioinformatics, 2(2006) 261 269. Gander, M. J.; Karp, A. H. Stable computation of high order Gauss quadrature rules using discretization for measures in radiation transfer. J. of Molecular Evolution, 53(4-5):47.
30/31 Gatushi, W. Orthogonal polynomials (in Matlab). J. Comp. Appl. Math. 178(2005) 215 234. Gatushi, W. Orthogonal polynomials: computation and approximation. Oxford University Press, New York, 2004. Gatushi, W. Orthogonal polynomials and quadrature. Electronic Trans. of Num. Math. 9(1999) 65 76. Gatushi, W. Algorithm 726: ORTHOPOL - A package of routines for generating orhtogonal polynomials and Gauss-type quadrature rules. ACM Trans. on Math. Software, 20(1994) 21 62. Laurie, D. P. Calculation of Gauss-Kronord quadrature rules. Math. Comp. 66(1997) 1133 1145. Kronord, A. S. Nodes and weights of quadrature formulas. Consultants Bureau, New York, 1965.
31/31 Kythe, P. K.; Schäferkotter, M. R. Handbook of computational methods for integration. Chapman & Hall/CRC, 2005. Patterson, T. N. L. The optimum addition of points to quadrature formulae. Math. Comp. 22(1968) 847 856. Errata: Math. Comp. 23(1969) 892. Trefethen, L. N. Is Gauss Better than Clenshaw-Curtis? SIAM Review 50(2008) 67 87. Waldvogel, J. Fast construction of the Fejer and Clenshaw-Curtis quadrature rules. BIT Numerical Mathematics 43(2004) 1 18.