Numericl Methods I Orthogonl Polynomils Aleksndr Donev Cournt Institute, NYU 1 donev@cournt.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fll 2014 Nov 6th, 2014 A. Donev (Cournt Institute) Lecture IX 11/2014 1 / 27
Outline 1 Review 2 Orthogonl Polynomils on [ 1, 1] 3 Spectrl Approximtion 4 Conclusions A. Donev (Cournt Institute) Lecture IX 11/2014 2 / 27
Review Lgrnge bsis on 10 nodes 2 A few Lgrnge bsis functions for 10 nodes 1 0 1 2 3 4 5 6 φ 5 φ 1 7 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 φ 3 A. Donev (Cournt Institute) Lecture IX 11/2014 3 / 27
Review Runge s phenomenon f (x) = (1 + x 2 ) 1 1 Runges phenomenon for 10 nodes 0 1 y 2 3 4 5 5 4 3 2 1 0 1 2 3 4 5 x A. Donev (Cournt Institute) Lecture IX 11/2014 4 / 27
Function Spce Bsis Review Think of function s vector of coefficients in terms of set of n bsis functions: {φ 0 (x), φ 1 (x),..., φ n (x)}, for exmple, the monomil bsis φ k (x) = x k for polynomils. A finite-dimensionl pproximtion to given function f (x): f (x) = n c i φ i (x) i=1 Lest-squres pproximtion for m > n (usully m n): c f = rg min (x) f (x), c 2 which gives the orthogonl projection of f (x) onto the finite-dimensionl bsis. A. Donev (Cournt Institute) Lecture IX 11/2014 5 / 27
Lest-Squres Fitting Review Discrete cse: Think of fitting stright line or qudrtic through experimentl dt points. The function becomes the vector y = f X, nd the pproximtion is y i = n c j φ j (x i ) y = Φc, j=1 Φ ij = φ j (x i ). This mens tht finding the pproximtion consists of solving n overdetermined liner system Φc = y Note tht for m = n this is equivlent to interpoltion. MATLAB s polyfit works for m n. A. Donev (Cournt Institute) Lecture IX 11/2014 6 / 27
Norml Equtions Review Recll tht one wy to solve this is vi the norml equtions: (Φ Φ) c = Φ y A bsis set is n orthonorml bsis if (φ i, φ j ) = φ i (x k )φ j (x k ) = δ ij = k=0 { 1 if i = j 0 if i j Φ Φ = I (unitry or orthogonl mtrix) c = Φ y c i = φ X i f X = f (x k )φ i (x k ) k=0 A. Donev (Cournt Institute) Lecture IX 11/2014 7 / 27
Orthogonl Polynomils on [ 1, 1] Orthogonl Polynomils Consider function on the intervl I = [, b]. Any finite intervl cn be trnsformed to I = [ 1, 1] by simple trnsformtion. Using weight function w(x), define function dot product s: (f, g) = b w(x) [f (x)g(x)] dx For different choices of the weight w(x), one cn explicitly construct bsis of orthogonl polynomils where φ k (x) is polynomil of degree k (tringulr bsis): (φ i, φ j ) = b w(x) [φ i (x)φ j (x)] dx = δ ij φ i 2. A. Donev (Cournt Institute) Lecture IX 11/2014 8 / 27
Orthogonl Polynomils on [ 1, 1] Legendre Polynomils For equl weighting w(x) = 1, the resulting tringulr fmily of of polynomils re clled Legendre polynomils: φ 0 (x) =1 φ 1 (x) =x φ 2 (x) = 1 2 (3x2 1) φ 3 (x) = 1 2 (5x3 3x) φ k+1 (x) = 2k + 1 k + 1 xφ k(x) k k + 1 φ k 1(x) = 1 2 n n! These re orthogonl on I = [ 1, 1]: 1 1 φ i (x)φ j (x)dx = δ ij 2 2i + 1. d n [ (x 2 dx n 1 ) ] n A. Donev (Cournt Institute) Lecture IX 11/2014 9 / 27
Orthogonl Polynomils on [ 1, 1] Interpoltion using Orthogonl Polynomils Let s look t the interpolting polynomil φ(x) of function f (x) on set of m + 1 nodes {x 0,..., x m } I, expressed in n orthogonl bsis: φ(x) = i φ i (x) Due to orthogonlity, tking dot product with φ j (wek formultion): (φ, φ j ) = i (φ i, φ j ) = i δ ij φ i 2 = j φ j 2 This is equivlent to norml equtions if we use the right dot product: (Φ Φ) ij = (φ i, φ j ) = δ ij φ i 2 nd Φ y = (φ, φ j ) A. Donev (Cournt Institute) Lecture IX 11/2014 10 / 27
Orthogonl Polynomils on [ 1, 1] Guss Integrtion j φ j 2 = (φ, φ j ) j = Question: Cn we esily compute (φ, φ j ) = b w(x) [φ(x)φ j (x)] dx = ( φ j 2) 1 (φ, φj ) b w(x)p 2m (x)dx for polynomil p 2m (x) = φ(x)φ j (x) of degree t most 2m? Let s first consider polynomils of degree t most m b w(x)p m (x)dx =? A. Donev (Cournt Institute) Lecture IX 11/2014 11 / 27
Orthogonl Polynomils on [ 1, 1] Guss Weights Now consider the Lgrnge bsis {ϕ 0 (x), ϕ 1 (x),..., ϕ m (x)}, where you recll tht ϕ i (x j ) = δ ij. Any polynomil p m (x) of degree t most m cn be expressed in the Lgrnge bsis: p m (x) = p m (x i )ϕ i (x), b w(x)p m (x)dx = [ b ] p m (x i ) w(x)ϕ i (x)dx where the Guss weights w re given by w i = b w(x)ϕ i (x)dx. = w i p m (x i ), A. Donev (Cournt Institute) Lecture IX 11/2014 12 / 27
Orthogonl Polynomils on [ 1, 1] Bck to Interpoltion For ny polynomil p 2m (x) there exists polynomil quotient q m 1 nd reminder r m such tht: p 2m (x) = φ m+1 (x)q m 1 (x) + r m (x) b w(x)p 2m (x)dx = b [w(x)φ m+1 (x)q m 1 (x) + w(x)r m (x)] dx = (φ m+1, q m 1 ) + b w(x)r m (x)dx But, since φ m+1 (x) is orthogonl to ny polynomil of degree t most m, (φ m+1, q m 1 ) = 0 nd we thus get: b w(x)p 2m (x)dx = w i r m (x i ) A. Donev (Cournt Institute) Lecture IX 11/2014 13 / 27
Orthogonl Polynomils on [ 1, 1] Guss nodes Finlly, if we choose the nodes to be zeros of φ m+1 (x), then r m (x i ) = p 2m (x i ) φ m+1 (x i )q m 1 (x i ) = p 2m (x i ) b w(x)p 2m (x)dx = w i p 2m (x i ) nd thus we hve found wy to quickly project ny polynomil of degree up to m onto the bsis of orthogonl polynomils: (p m, φ j ) = w i p m (x i )φ j (x i ) A. Donev (Cournt Institute) Lecture IX 11/2014 14 / 27
Orthogonl Polynomils on [ 1, 1] Interpoltion using Guss weights Recll we wnt to compute j = where for Legendre polynomils ( φ j 2) 1 (φ, φj ), φ j 2 = 2 2j + 1. Now we know how to compute fst: (φ, φ j ) = w i φ(x i )φ j (x i ) = w i f (x i )φ j (x i ) ( ) 2j + 1 m j = w i f (x i )φ j (x i ), 2 where w i nd φ j (x i ) re coefficients tht cn be precomputed nd tbulted. A. Donev (Cournt Institute) Lecture IX 11/2014 15 / 27
Orthogonl Polynomils on [ 1, 1] Guss-Legendre polynomils For ny weighting function the polynomil φ k (x) hs k simple zeros ll of which re in ( 1, 1), clled the (order k) Guss-Legendre nodes, φ m+1 (x i ) = 0. The interpolting polynomil φ(x i ) = f (x i ) on the Guss nodes is the Guss-Legendre interpolnt φ GL (x). The orthogonlity reltion cn be expressed s sum insted of integrl: (φ i, φ j ) = w i φ i (x i )φ j (x i ) = δ ij φ i 2 We cn thus define new weighted discrete dot product f g = w i f i g i A. Donev (Cournt Institute) Lecture IX 11/2014 16 / 27
Orthogonl Polynomils on [ 1, 1] Discrete Orthogonlity of Polynomils The orthogonl polynomil bsis is discretely-orthogonl in the new dot product, φ i φ j = (φ i, φ j ) = δ ij (φ i φ i ) This mens tht the mtrix in the norml equtions is digonl: { Φ Φ = Dig φ 0 2,..., φ m 2} i = f φ i. φ i φ i The Guss-Legendre interpolnt is thus esy to compute: φ GL (x) = f φ i φ i φ i φ i (x). A. Donev (Cournt Institute) Lecture IX 11/2014 17 / 27
Orthogonl Polynomils on [ 1, 1] Chebyshev Interpoltion There re other fmilies of orthogonl polynomils tht re lso very useful in prctice (Guss-Lobto, Guss-Hermite, etc.). A notble exmple re the Chebyshev polynomils on [ 1, 1], with weight function 1 w(x) = 1 x 2 defined recursively vi: φ 0 = 1 φ 1 = x φ k+1 = 2xφ k φ k 1. Orthogonlity reltion, for i, j not both zero, (φ i, φ j ) = δ ij π 2 nd (φ 0, φ 0 ) = π. A. Donev (Cournt Institute) Lecture IX 11/2014 18 / 27
Orthogonl Polynomils on [ 1, 1] Chebyshev Nodes They cn lso be defined s the unique polynomils stisfying the trig reltion: φ k (cos θ) = cos (kθ). This mens tht their roots φ k (x i ) = 0, the Chebyshev nodes, re esy to find, ( ) 2i 1 x i = cos 2k π, i = 1,..., k, which hve simple geometric interprettion s the projection of uniformly spced points on the unit circle. The Chebyshev-Guss weights re lso esy to compute, w i = b w(x)ϕ i (x)dx = π k. Polynomil interpoltion using the Chebyshev nodes elimintes Runge s phenomenon. A. Donev (Cournt Institute) Lecture IX 11/2014 19 / 27
Spectrl Approximtion Hilbert Spce L 2 w Consider the Hilbert spce L 2 w of squre-integrble functions on [ 1, 1]: f L 2 w : (f, f ) = f 2 = 1 1 w(x) [f (x)] 2 dx <. Legendre polynomils form complete orthogonl bsis for L 2 w: f L 2 w : f (x) = f i φ i (x) f i = (f, φ i) (φ i, φ i ). The lest-squres pproximtion of f is spectrl pproximtion nd is obtined by simply truncting the infinite series: φ sp (x) = f i φ i (x). A. Donev (Cournt Institute) Lecture IX 11/2014 20 / 27
Spectrl Approximtion Spectrl pproximtion Continuous (spectrl pproximtion): φ sp (x) = Discrete (interpolting polynomil): φ GL (x) = (f, φ i ) (φ i, φ i ) φ i(x). f φ i φ i φ i φ i (x). If we pproximte the function dot-products with the discrete weighted products (f, φ i ) w j f (x j )φ i (x j ) = f φ i, j=0 we see tht the Guss-Legendre interpolnt is discrete spectrl pproximtion: φ GL (x) φ sp (x). A. Donev (Cournt Institute) Lecture IX 11/2014 21 / 27
Spectrl Approximtion Discrete spectrl pproximtion Using spectrl representtion hs mny dvntges for function pproximtion: stbility, rpid convergence, esy to dd more bsis functions. The convergence, for sufficiently smooth (nice) functions, is more rpid thn ny power lw ( d f (x) φ GL (x) C 1/2 N d f (k) 2), k=0 where the multiplier is relted to the Sobolev norm of f (x). For f (x) C 1, the convergence is lso pointwise with similr ccurcy (N d 1/2 in the denomintor). This so-clled spectrl ccurcy (limited by smoothness only) cnnot be chived by piecewise, i.e., locl, pproximtions (limited by order of locl pproximtion). A. Donev (Cournt Institute) Lecture IX 11/2014 22 / 27
Regulr grids Spectrl Approximtion =2; f = @( x ) cos (2 exp ( x ) ) ; x f i n e=l i n s p c e ( 1,1,100); y f i n e=f ( x f i n e ) ; % Equi spced nodes : n=10; x=l i n s p c e ( 1,1, n ) ; y=f ( x ) ; c=p o l y f i t ( x, y, n ) ; y i n t e r p=p o l y v l ( c, x f i n e ) ; % Guss nodes : [ x,w]=glnodewt( n ) ; % See webpge f o r code y=f ( x ) ; c=p o l y f i t ( x, y, n ) ; y i n t e r p=p o l y v l ( c, x f i n e ) ; A. Donev (Cournt Institute) Lecture IX 11/2014 23 / 27
Spectrl Approximtion Guss-Legendre Interpoltion 2 Function nd pproximtions for n=10 2 Error of interpolnts/pproximnts for n=10 1 1 0 0 1 1 2 2 3 3 4 4 Actul Equi spced nodes Stndrd pprox Guss nodes Spectrl pprox 5 Stndrd pprox Spectrl pprox 5 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 6 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 A. Donev (Cournt Institute) Lecture IX 11/2014 24 / 27
Spectrl Approximtion Globl polynomil interpoltion error Error for equispced nodes for n=8,16,32,..128 Error for Guss nodes for n=8,16,32,..128 10 0 10 0 10 2 10 2 10 4 10 4 10 6 10 6 10 8 10 8 10 10 10 10 10 12 10 12 10 14 10 14 10 16 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 10 16 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 A. Donev (Cournt Institute) Lecture IX 11/2014 25 / 27
Spectrl Approximtion Locl polynomil interpoltion error Error for liner interpolnt for n=8,16,32,..256 10 1 10 0 10 0 10 2 10 1 Error for cubic spline for n=8,16,32,..256 10 2 10 3 10 4 10 6 10 4 10 8 10 5 10 10 10 6 10 12 10 7 10 14 10 8 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 10 16 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 A. Donev (Cournt Institute) Lecture IX 11/2014 26 / 27
Conclusions Conclusions/Summry Once function dot product is defined, one cn construct orthogonl bsis for the spce of functions of finite 2 norm. For functions on the intervl [ 1, 1], tringulr fmilies of orthogonl polynomils φ i (x) provide such bsis, e.g., Legendre or Chebyshev polynomils. If one discretizes t the Guss nodes, i.e., the roots of the polynomil φ m+1 (x), nd defines suitble discrete Guss-weighted dot product, one obtins discretely-orthogonl bsis suitble for numericl computtions. The interpolting polynomil on the Guss nodes is closely relted to the spectrl pproximtion of function. Spectrl convergence is fster thn ny power lw of the number of nodes nd is only limited by the globl smoothness of the function, unlike piecewise polynomil pproximtions limited by the choice of locl bsis functions. One cn lso consider piecewise-spectrl pproximtions. A. Donev (Cournt Institute) Lecture IX 11/2014 27 / 27