Orthogonal Polynomials and Gaussian Quadrature

Transcription

1 Orthogonal Polynomials and Gaussian Quadrature 1. Orthogonal polynomials Given a bounded, nonnegative, nondecreasing function w(x) on an interval, I of the real line, we consider the Hilbert space L 2 (I,dw). That is, we consider a Borel measure on a subinterval of the real line. Typically, we restrict to the cases of an integrable weight function, i.e., an absolutely continuous measure, or a discrete measure supported on a countable subset of R. The polynomial functions {1,x,x 2,...,x n,...} are defined on I. Form the real inner product space, with f,g = f(x)g(x)dw(x). I Nowapply Gram-Schmidt to getanons, {ψ 0,ψ 1,...}, where degψ n = n, n 0. In particular, with w(i) = 1dw we have ψ 0 = 1/ w(i). We proceed to consider some of the main features of a sequence of orthogonal polynomials. I We focus on two specific sequences of orthogonal polynomials for a given dw. The sequence {ψ n } will denote the ONS. The sequence {φ n } will denote the monic polynomials, i.e., with leading coefficient equal to 1. In particular, φ 0 = 1 in every case. Finally, denote φ n 2 = φ n (x) 2 dw(x) = γ n the sequence of squared norms. I

2 Three-term recurrences. Proposition 1.1. There exist sequences {a k } k 0 and {b k } k 0 such that the polynomials {φ k } k 0 satisfy, for k 0, the recurrence xφ k = φ k+1 +a k φ k +b k φ k 1 with b 0 = 0, eliminating the third term for k = 0. Proof. Since xφ k is a polynomial of degree k+1, we have an expansion xφ k = φ k+1 +a k φ k +b k φ k 1 + c j φ j. j<k 1 We will check that all of the coefficients c j, j < k 1 are zero. We have xφ k,φ j = φ k,xφ j = 0 = c j γ j by orthogonality, if j < k 1, since then degxφ j < k. Since γ j 0, we conclude that c j = 0. First, the recurrence relation gives xφ k,φ k = a k. Now, observe that by the recurrence relation xφ k,φ k 1 = b k 1 and, dropping k to k 1, for k 1, We thus have, with γ 0 = w(i), 1 = b k Now, the ONS, {ψ n }, satisfies xφ k 1,φ k = = = ( k ) b i γ0. i=1 φ k = ψ k γk = ψ k γ0 b 1 b 2 b k. Substituting into the recurrence relation yields xψ k γk = bk+1 ψ k+1 +a k γk ψ k +b k γk /b k ψ k 1. Clearing the common factor we have Proposition 1.2. The orthonormal polynomials satisfy, for k 0, the recurrence xψ k = b k+1 ψ k+1 +a k ψ k + b k ψ k 1 if the monic polynomials satisfy the recurrence of Proposition 1.1.

3 An interesting feature of this latter recurrence is that it can be written in matrix form in terms of a symmetric tridiagonal matrix. Namely, for n 1, x ψ 0 ψ 1 ψ 2. ψ n 1 a 0 b b1 a 1 b = 0 b2 a 2 b bn 1 a n 1 ψ 0 ψ 1 ψ 2. ψ n bn ψ n (1) 1.2. Zeros. A sequence of polynomials orthogonal on an interval I has some special properties. Here {φ n } denotes a sequence of polynomials forminganorthogonalsysteminl 2 (I,dw)withoutanyfurtherassumptions, i.e., we do not require them to be monic nor normalized. Proposition 1.3. For each n > 0, the polynomial φ n has n real roots lying in the interval I. Proof. Letλbearealrootofφ n lyingoutsideofi. Then, forx I, x λ doesnotchange sign. Ifit isnegative, we replace itby λ x, inanycase φ n (x)/ x λ isapolynomial ofdegree n 1 andφ n (x)(φ n (x)/ x λ ) 0 on I. By orthogonality, φ n (x)(φ n (x)/ x λ )dw = 0 I so that φ n (x) 2 / x λ = 0 on I. But then φ n (x) would be identically zero. Similarly, if there are complex conjugate roots, they yield a quadratic factor x 2 + ax + b that does not change sign on I. And φ n (x)/(x 2 +ax+b) is a polynomial of degree n 2 orthogonal to φ n, again leading to the contradiction φ n vanishing identically. A similar argument applies to a root λ I with multiplicity greater than1. Form φ n (x)/(x λ) 2 which would be a polynomial orthogonal to φ n, leading to a contradiction as before. 2. Gaussian quadrature A quadrature rule is a linear functional on C(I) designed to approximate I f dw. For convenience we will simply write f. The quadrature rule

4 4 has the form f n w k f(x k ) k=1 with x k I, 1 k n, called the nodes. The coefficients {w k } are the weights. Let {φ n } be a sequence of polynomials orthogonal on I with respect to the measure dw with degφ n = n, for n 0. Unless otherwise stated, we will assume that dw is normalized so that 1dw = 1 I with φ 0 = 1. If the quadrature formula holds for polynomials up to degree n, then we have 1 = 1 = w k and φ i = w k φ i (x k ) = 0,for 0 i < n. k k Fix a degree/order n > 1. If f is a polynomial of degree degf > n, we can use the division algorithm, dividing f by φ n, and write f = qφ n +r withquotientq andtheremainder r satisfyingdegr < n. Ifdegf < 2n, then degq < n as well and we have, by orthogonality, f = qφ n + r = r whereas by the quadrature rule, assuming it holds for polynomials of degree at most n, hence for r, f(x k )w k f = w k r(x k ) = ( w k q(xk )φ n (x k )+r(x k ) ) k k k That is, w k q(x k )φ n (x k ) = 0 k for polynomials q of degree less than n. By choosing the nodes to be the (n distinct) zeros of φ n, the quadrature rule becomes exact for polynomials of degree less than 2n. We will henceforth assume the nodes to be so chosen. This, then, is Gaussian quadrature.

5 The quadrature rule is now exact for polynomials of degree less than 2n. So take polynomials φ i and φ j, with 0 i,j < n. Then φ i φ j is a polynomial of degree less than 2n and we have φ i φ j = γ i δ ij = w k φ i (x k )φ j (x k ) (2) k with {γ i } the squared norms. It is convenient to introduce matrices to express these and subsequent useful relations. Let Φ ij = ( φ i 1 (x j ) ) 1 i,j n, W = diag(w 1,...,w n ), and Γ = diag(γ 0,...,γ n 1 ). Then equation (2) takes the form where the indicates transpose. ΦWΦ = Γ (3) We can also see at this point that the weights w k are positive. Let l j (x) = cφ n (x)/(x x j ) (4) a polynomial vanishing at all nodes except x j with c chosen so that l j (x j ) = 1. Sincedegl j = n 1 < n, degl 2 j < 2nsothatthequadrature rule holds for l 2 j. Thus, since this non-zero polynomial vanishes at all nodes except x j, we have 0 < l 2 j = w k l j (x k ) 2 = w j (5) k as required. Note that we have a formula for the weights from this discussion. Find c by letting x x j in equation (4): l j (x j ) = 1 = c lim x xj φ n (x) φ n (x j ) x x j = cφ n (x j) by definition of the derivative of φ n at x = x j, where we use the fact that φ n (x j ) = 0. So c = 1/φ n(x j ). Thus, using the equalities in equation (5) with l j instead of l 2 j, we have φ n (x) w j = φ n (x (6) j)(x x j ) 5

6 6 Remark. With the assumption 1 = 1, the sum in the quadrature rule is a convex combination of the values {f(x k )} 1 k n which we can formintoacolumnvectorf. Alternatively, definef = diag(f(x 1 ),...,f(x n )). Then the rule takes the form f trfw. Thus, the weights define a discrete probability distribution. Taking determinants in equation (3) (detφ) 2 detw = detγ > 0 so that Φ and W are invertible with detw > 0, as we have seen from positivity of the weights. Now we can define Then equation (3) reads V = Γ 1/2 ΦW 1/2 with Φ = Γ 1/2 VW 1/2 (7) VV = I. That is, V is an orthogonal matrix. And we have the dual relation V V = I = Φ Γ 1 Φ = W 1. (8) 2.1. Recurrence formula. Nodes and weights. In this discussion we will use the orthonormal polynomials {ψ n }. The nodes are the same, and we set with corresponding weights Ψ ij = ( ψ i 1 (x j ) ) 1 i,j n W = (Ψ Ψ) 1. and Γ = I Recall the matrix form of the recurrence formula, equation(1). Replacing x successively by the rootsx j of ψ n, the vector involving ψ n will vanish. Setting a 0 b b1 a 1 b A = 0 b2 a 2 b bn 1 a n 1

7 for x = x j we have, ψ j = (ψ 0 (x j ),...,ψ n 1 (x j )), as a column vector, satisfying A ψ j = x j ψj. Sincetherearendistinctvalues{x j },thesearepreciselytheeigenvalues of A with corresponding eigenvectors { ψ j }. Nowconstruct Ψ fromthecolumns ψ j andtherecurrence formula takes the matrix form AΨ = ΨΛ with Λ = diag(x 1,...,x n ), the diagonal matrix of eigenvalues. Since the eigenvalues of A are distinct, the matrix Ψ has orthogonal columns, being the eigenvectors of a symmetric matrix corresponding to distinct eigenvalues. For orthonormal polynomials, equation (3) has the form ΨWΨ = I so that ΨW 1/2 is an orthogonal matrix satisfying A(ΨW 1/2 ) = ΨΛW 1/2 = (ΨW 1/2 )Λ. Thatis, U = ΨW 1/2 isanorthogonalmatrixdiagonalizing A. Since the eigenvalues are distinct, adjusting signs so the top row is all positive, this matrix is unique up to permutation of its columns, corresponding to sorting the eigenvalues of A. Since Γ has diagonal entries equal to the squared norms of the orthogonal system, we see that Ψ = Γ 1/2 Φ. Thus, U = ΨW 1/2 = Γ 1/2 ΦW 1/2. (9) Comparing with equation (7), we identify U = V. Note that the recurrence relation AΨ = ΨΛ can be written AΓ 1/2 Φ = Γ 1/2 ΦΛ = Γ 1/2 AΓ 1/2 Φ = ΦΛ Takingfor{φ }themonicpolynomials, callthislastmatrixofcoefficients A monic. Thus, we have A = A orthonormal = Γ 1/2 A monic Γ 1/2 7

8 8 so that one could start directly from the original recurrence relation for the monic polynomials, construct A monic from the coefficients, calculate the values γ n via the coefficients b n and continue from there. Remark. When the eigenvectors of A are found by computation, form Ũ taking as columns a basis of eigenvectors. Then Ũ Ũ = D a diagonal matrix. Thus U = ŨD 1/2 will be an orthogonal matrix diagonalizing A. If necessary, adjust the signs so that the top row is all positive. Then, sorting the eigenvalues x 1 < x 2 < < x n produces a unique matrix U which must then equal ΨW 1/2 after suitable arrangement of its columns. The top row of U is U 1j = ψ 0 (x j ) w j With γ 0 = w(i), ψ 0 = 1/ γ 0 and solving for the weights, we have w j = γ 0 U 2 1j (10) directly from the top row of U. Any of the formulas (6), (8), or (10) can be used to evaluate the weights {w j } Christoffel-Darboux identity. Weights revisited. Proposition 2.1. For monic orthogonal polynomials {φ n }, we have n φ k (x)φ k (y) = 1 1 γ n x y φ n+1(x) φ n+1 (y) φ n (x) φ n (y) the vertical delimiters denoting determinant. Proof. For induction, start with n = 0. We have, using the recurrence formula at k = 0, φ 0 (x)φ 0 (y) =? 1 1 γ 0 γ 0 x y φ 1(x) φ 1 (y) φ 0 (x) φ 0 (y) = 1 1 γ 0 x y x a 0 y a = 1 γ 0

9 since φ 0 (x) = φ 0 (y) = 1. The inductive step is similar: 1 1 γ n x y φ n+1(x) φ n+1 (y) φ n (x) φ n (y) = 1 1 γ n x y (x a n)φ n (x) b n φ n 1 (x) (y a n )φ n (y) b n φ n 1 (y) φ n (x) φ n (y) = 1 1 γ n x y xφ n(x) yφ n (y) φ n (x) φ n (y) + b n γ n (x y) φ n(x) φ n (y) φ n 1 (x) φ n 1 (y) (switching rows) = φ n(x)φ n (y) γ n γ n 1 x y φ n(x) φ n (y) φ n 1 (x) φ n 1 (y) = φ n(x)φ n (y) n 1 + γ n = as required. n φ k (x)φ k (y) φ k (x)φ k (y) (by induction hypothesis) Now we have another formula for the Gaussian quadrature weights. Proposition 2.2. For monic polynomials, {φ n }, we have γ n 1 w j = φ n(x j )φ n 1 (x j ). Proof. Letting y = x j, the j th root of φ n, in the Christoffel-Darboux formula, with n 1 replacing n, yields n 1 φ k (x)φ k (x j ) = 1 γ n 1 φ n (x)φ n 1 (x j ) φ n (x j )φ n 1 (x) x x j with φ n (x j ) vanishing. Now let x = x i. If i j, then the factor φ n (x i ) = 0 yielding zero. If i = j, use L Hôpital s rule to get n 1 φ k (x j )φ k (x j ) = φ n 1(x j )φ n(x j ) γ n 1. Now the left side reads (Φ Γ 1 Φ) jj. By equation (8), we get which immediately gives the result. 1 = φ n 1(x j )φ n (x j) (11) w j γ n 1 9

10 10 Corollary 2.3. Let {p n } be an OPS. Let p k have leading coefficient α k, k 0. Then the weights are given by w j = α n γ n 1 α n 1 p n(x j )p n 1 (x j ) where now the squared norms are given by γ n 1 = p n 1 2. Proof. Let φ n = p n /α n denote the corresponding monic polynomials. Then in the Proposition, note that φ n 1 2 = p n 1 2 /α 2 n 1 with γ n 1 now referring to p n 1 2. Problem. Consider the Chebyshev polynomials of the first kind on dx [ 1, 1] with measure π 1 x Use the recurrence formula to show that the leading coefficient of T n is 2 n 1, for n Verify that T n 2 = 1/2 for n 1, while T 0 2 = 1. (2j 1)π 3. Check that the zeros of T n (x) are given by x j = cos, 2n 1 j n. 4. Show that T n (x) = nu n 1(x), Chebyshev polynomials of the second kind. 5. For n 2, use the formula in Corollary 2.3 to verify that the weights w j all equal 1/n Convergence. Now we check that if we increase n, the quadrature estimates indeed converge to f. In this section we take I to be a closed bounded interval. Start by denoting thenodes and weights at order nby {x nk } and {w nk } respectively. Then the quadrature rule at stage n is n P n (f) = w nk f(x nk ). k=1 Since the quadrature rule is exact for polynomials of degree less than 2n, we have P n (x m ) = x m for n > m. Hence lim P n(x m ) = n x m

11 11 for all m 0, hence, by linearity, for all polynomials p(x). Let f C(I). Then by Weierstrass Approximation Theorem, we can find a polynomial p such that f p < ε, denoting the sup norm on I. So, using the fact that P n (1) = γ 0, P n (f) f P n (f) P n (p) + P n (p) p + f p f p γ f p γ 0 2εγ 0 whenever n exceeds degp. So the error is the same order of magnitude as that in Weierstrass approximation. 3. Interpolation We are given a function f on I. We can interpolate its values at the nodes{x j }producingapolynomialwhichagreeswithf atthosepoints. Start by defining the column vector f = (f(x k )) 1 k n. If f has an expansion in the OPS {φ } up to order n 1, n 1 f(x) = c k φ k (x) (12) and, with c the vector with components (c 0,...,c n 1 ), we have n 1 f(x j ) = c k φ k (x j ) or Φ c = f. From equation (3), we have Thus, ΦWΦ = Γ = (Φ ) 1 = Γ 1 ΦW. c = Γ 1 ΦWf (13) gives the coefficients in the expansion of f. With the usual assumption φ 0 = 1, the integral of f is simply c 0 γ 0. Writing out the matrix relation yields the formula c k = 1 n φ k (x j )w j f(x j ) (14) j=1

12 12 for 0 k < n. By orthogonality, multiplying by φ k (x) and integrating in equation (12), c k = 1 f(x)φ k (x)dw I and the quadrature formula recovers equation (14) for polynomials of degree less than n. This shows that the orthogonal expansion, equation (12), agrees with the interpolation expansion. For general continuous, or piecewise continuous bounded, functions these give the truncated orthogonal expansion in L 2 (I,dw). Remark. When doing computations by machine, the eigenvalue problem for A yields the nodes as eigenvalues and the weights via the top row of the orthogonal matrix diagonalizing A. If we have the entire matrix U at hand, we can use it to do interpolation directly via equations (13) and (9): and for the orthonormal expansion we have as Γ is simply the identity matrix. c = Γ 1/2 UW 1/2 f n 1 f(x) = α k ψ k (x) α = UW 1/2 f Problem. Do Chebyshev interpolation on [ 1, 1] with the weight function(1 x 2 ) 1/2 /π. Foreachfunctionfindtheinterpolatingpolynomial as an expansion in Chebyshev polynomials. Then plot the interpolant and the function on [ 1,1]. Do for n = 5, n = 10, n = 20, for each function: (i) 1/(1+x 2 ) and (ii) cosx. Notes on References. The matrix approach to finding the nodes and weights is thoroughly discussed in [1]. [2] presents the matrix approach along with MATLAB code examples. Code is available at the web site.

13 [3] presents an overview with various formulas for the weights. See the material on Gaussian quadrature in[4], where the formula (10) is one of the exercises. References [1] Calculation of Gauss Quadrature Rules: Gene H. Golub, John H. Welsch, Mathematics of Computation, Vol. 23, No. 106 (Apr., 1969), pp s1-s10, American Mathematical Society, [2] John A. Gubner, [3] See [4] Herbert S. Wilf, Mathematics for the Physical Sciences, wilf/website/mathematics for the Physical Sciences.html 13