Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling continuous rel function f on some intervl [, b] We will begin by illustrting this in the context of polynomil interpoltion where we will construct polynomils tht exctly mtch the function f t certin fixed points of the intervl [, b] We will then investigte n other lterntive to interpoltion tht is more suited to pproximte polynomils To this we will define the continuous nlogue of the discrete liner lest-squres studied in Chpter 2 1 Polynomil interpoltion The problem of interpolting function f : [, b] R cn be stted s: Given f continuous on the intervl [, b] nd n + 1 points {x 0, x 1,, x n } stisfying x 0 < x 1 < < x n b, determine polynomil P n R n [X] such tht P n (x i ) = f(x i ), i = 0,, n, where P n R n [X] mens tht nd we sy tht the polynomil P n hs degree n P n (x) = 0 + 1 x + + n x n = i x i We require n + 1 dt points to construct n interpolting polynomil of degree n since if the number of points is smller thn n, then we could construct infinitely mny degree n interpolting polynomils nd if it is lrger, then there would be no degree n interpolnt Monomil bsis The most strightforwrd wy of solving the interpoltion problem is to notice tht for the choice of the interpolnt P n (x) = 0 + 1 x + n x n = n ix i we hve 0 + 1 x 1 + + n x n 1 = f(x 1 ) 0 + 1 x 2 + + n x n 2 = f(x 2 ) 0 + 1 x n + + n x n n = f(x n ) In mtrix form this cn be rewritten s 1 x 0 x 2 0 x n 0 1 x 1 x 2 1 x n 1 1 x n x 2 n x n n 0 1 n = f(x 0 ) f(x 1 ) f(x n ) Mtrices of this form re clled Vrdemonde mtrices nd they re invertible since their determinnt is given by n n j=i+1 (x j x i ) which is non-zero if the x i s re distinct In prticulr, this implies tht there is unique interpolnt to f stisfying the bove conditions To solve the liner system bove, we require O(n 3 ) Despite being strightforwrd, the use of the monomil bsis 1, x, x 2, for interpolting gives rise to unplesnt numericl problems when ttempting to determine the coefficients i on computer The min issue stems for the fct tht the monomil bsis functions look incresingly similr s we tke higher nd higher powers This in turn might led to the coefficients c i to become very lrge in mgnitude even if the function f remins of modest size on [, b] To understnd ( ) this phenomen, ( we cn ) use ( n nlogy ) with 1 1 1 liner lgebr In prticulr, try to express the vector in the bsis 1 10 10 nd 0 1 of 7
Dr M DRAIEF Lgrnge bsis An lterntive to the monomil bsis is to express the polynomil P n in different bsis L 0, L 1,, L n of polynomils of degree t most n in R n [X], where L i (x k ) = 0 for ll k i nd L i (x i ) = 1 If such bsis exists then it is not difficult to see tht if we wnt to express P n (x), the interpolnt of f in x 0,, x n, s liner combintion of the L i s then we hve tht P n (x) = i L i (x) nd i = P n (x i ) It is not difficult to check tht the Lgrnge polynomils defined by L i (x) = n k=0,k i (x x k) n k=0,k i (x i x k ), does the trick With the Lgrnge bsis, finding the coefficients for the interpoltion does not require to solve liner eqution Newton bsis The monomil nd the Lgrnge bses represent the two extreme cses from numericl perspective to perform interpoltion Moreover, both shre the shortcoming tht one needs to hve ll the dt points nd the vlues of f in dvnce to be ble to perform the interpoltion In prctise, these dt points my rrive one fter the other requiring the recomputtion of the interpolnt when one gets new dt point It would therefore be useful if one could derive the interpolnts recursively s the points re mde vilble to us In wht follows we will construct bsis Q 0, Q 1, Q n such tht P n (x) = n iq i (x) nd P k (x) = k iq i (x) interpoltes f in the points {x 0,, x k } for k = 0,, n Let us strt by constructing the polynomil P 0 of degree 0 tht interpoltesf in x 0 clerly P 0 (x) = f(x 0 ) Let P 0 = 0 Q 0 where Q 0 (x) = 1 for ll x nd 0 = f(x 0 ) Now, we use P 0 to find P 1 nd Q 1 in R 1 [X] tht interpoltes F in x 0 nd x 1, ie Note tht P 1 (x) = P 0 (x) + 1 Q 1 (x) P 1 (x 0 ) = P 0 (x 0 ) + 1 Q 1 (x 0 ) = f(x 0 ) + 1 Q 1 (x 0 ) As we require tht P 1 (x 0 ) = f(x 0 ), this implies tht 1 Q 1 (x) = 0, ie either 1 = 0 (this only hppens when f(x 0 ) = f(x 1 ) while we re seeking generic bsis tht works for ll f) of Q 1 (x 0 ) = 0 The ltter condition yields Q 1 (x) = x x 0 nd P 1 (x) = 0 + 1 (x x 0 ) To determine 1, we use the interpoltion condition for x 1 which implies tht 1 = f(x 1) 0 x 1 x 0 Next, we determine P 2 in R 2 [X] tht interpoltes f t x 0, x 1, x 2 where P 2 (x) = P 1 (x) + 2 Q 2 (x) nd Q 2 in R 2 [X] to be determined The interpoltion conditions together with the properties of the polynomil P 1 imply tht Q 2 (x 0 ) = Q 2 (x 1 ) = 0 Q 2 (x) = (x x 0 )(x x 1 ), nd from Q 2 (x 2 ) = 0 we get tht 2 = f(x 2) P 1 (x 2 ) Q 2 (x 2 ) = f(x 2) 1 (x 2 x 0 ) (x 2 x 0 )(x 2 x 1 ) 2 of 7
Dr M DRAIEF Following the sme pttern, it is not difficult to see tht one cn construct Q 1, Q 2, Q 3,, Q n bsis of R n [X] such tht, for k = 0, n k 1 Q k (x) = (x x i ) where P k (x) the interpolnt of f t x 0,, x k is given by P k (x) = k iq i (x) nd i = f(x i) i 1 j=0 jq j (x i ) Q i (x i ) The bove bsis is known s the Newton bsis The entire procedure of constructing P n cn be written in mtrix form s follows 1 0 0 0 f(x 0 ) 1 (x 1 x 0 ) 0 0 1 (x 2 x 0 ) (x 2 x 0 )(x 2 x 1 ) 1 f(x 1 ) 0 2 = f(x 2 ) 0 n 1 1 (x n x 0 ) (x n x 0 )(x n x 1 ) (x n x i ) n f(x n ) With the newton form of the interpolnt, one cn esily updte P n to P n+1 to incorporte new dt point (x n+1, f(x n+1 )) Note tht solving the liner system to find the coefficients i tkes O(n 2 ) steps We would like emphsise tht the three methods bove led the sme polynomil P n It is just expressed in three different bses 2 Orthogonl polynomils Interpoltion with high degree polynomils is not lwys the best wy to pproximte function s the polynomil might diverge form the function is is ment to pproximte t points in the intervls (x i, x i+1 ) (between the points of interpoltion) s the degree of the polynomils increse In fct the interpoltion only ccounts for the points of interest x 0, x 1, nd ignores ll other points In this section, we consider n lterntive to polynomil interpoltion, nmely polynomil pproximtion where we wnt to find polynomil P n R n [X] such tht P n (x i ) f(x i ) for i = 1,, m where m n When m = n we hve seen tht such polynomil exists nd is unique when m > n we must settle with n pproximtion, together with method for quntifying the error of this pproximtion For exmple, we could choose to minimise the mximum error min mx f(x i) P (x i ) 0 i m or the sum of the squres of the errors P R n[x] min P R n[x] m (f(x i ) P (x i )) 2 Alterntively, one could ignore the specific points nd mesure the entire intervl of interest [, b] min mx P R n[x] x [,b] f(x) P (x) or min P R n[x] (f(x) P (x)) 2 dx Polynomil pproximtion In this section we will focus the ltter error tht cn be seen s the continuous nlogue of the stndrd liner lest-squres method Exmple 1 Find polynomil P R 1 [X] tht minimises 1 0 (ex P (x)) 2 dx 3 of 7
Dr M DRAIEF In generl suppose tht we re interested in expressing the polynomil P n tht minimises (f(x) P (x))2 dx in bis φ 1,, φ n of R n [X], ie P (x) = k φ k (x) In this cse E( 0,, n ) = (f(x) P (x)) 2 dx = f, f 2 i f, φ i + k=0 i k φ i, φ k where g, h = h, g = g(x)h(x)dx In prticulr to minimise E( 0,, n ) we need to find i such tht E/ x i = 0 which implies f, φ i = k φ i, φ k In mtrix form this cn be rewritten s k=0 φ 0, φ 0 φ 0, φ 1 φ 0, φ n φ 1, φ 0 φ 1, φ 1 φ 1, φ n φ n, φ 0 φ n, φ 1 φ n, φ n Suppose = 0, b = 1 nd φ k (x) = x k then φ i, φ j = 1 0 x i+j dx = 0 1 n = 1 i + j + 1 f, φ 0 f, φ 1 f, φ n The corresponding mtrix is the well-known Hilbert mtrix This is poor bsis (supporting our erlier findings when using the monomils for interpolting) In fct Hilbert mtrices hve very lrge condition numbers, eg κ(h) 10 14 where H is the Hilbert mtrix of order 10 nd it increses the lrger the mtrix Orthogonl polynomils nd continuous lest squres Here we will focus on more generl errors by investigting polynomils tht minimise (f(x) P (x))2 w(x)dx for w continuous, positive function on [, b] nd we let g, h = h, g = g(x)h(x)w(x)dx Definition 1 We will sy tht two functions g nd h re orthogonl if g, h = 0 function φ 0, φ 1,, φ n is system of orthogonl polynomils if Moreover, set of for ll k φ k is polynomil of exct degree k for ll k j, φ j, φ k = 0 In prticulr if φ 0, φ 1,, φ n re orthogonl polynomils then (φ 0, φ 1,, φ n ) is bsis of R n [X] Moreover if P Spn(φ 0, φ 1,, φ n 1 ) then P, φ n = 0 We now describe mechnism for constructing orthogonl polynomils The Grm-Schmidt process used to orthogonlise vectors in R n cn in fct be employed here s well More precisely, suppose tht we hve bsis of R n [X], P 0, P 1,, P n where the degree of P k is exctly k, s mtter of exmple this could be the monomil bsis P i (x) = x i then the Grm-Schmidt lgorithm tkes the following form 4 of 7
Dr M DRAIEF Pseudocode: Grm-Schmidt for polynomils 1 Let φ 0 = P 0 2 For k = 1,, n φ k = p k k 1 p k,φ i φ i,φ i φ i Now suppose tht one hs set of orthogonl polynomils φ 0, φ n nd seeks the next orthogonl polynomil φ n+1 Since the degree of φ n is exctly n, then the degree of xφ n (x) is n + 1 nd one could perform the Grm-Schmidt procedure on φ 0 (x), φ n (x), xφ n (x) which forms bsis of R n+1 [X] In prticulr First notice tht φ n+1 (x) = xφ n (x) xφ n (x), φ i (x) = = i= xφ n (x), φ i (x) φ i (x) φ i, φ i xφ n (x)φ i (x)w(x)dx φ n (x)xφ i (x)w(x)dx = φ n (x), xφ i (x) Since xφ i (x)r i+1 [X], then for i < n 1 we hve φ n (x), xφ i (x) = 0 this yields substntil simplifiction in the Grm-Schmidt procedure In prticulr Theorem 1 Given positive, continuous weight function w on [, b] nd n ssocited inner product g, h = h, g = g(x)h(x)w(x)dx, then system of orthogonl polynomils cn be generted s follows φ 0 (x) = 1 φ 1 (x) = x x, 1 1, 1 φ k (x) = xφ k 1 (x) xφ k 1(x), φ k 1 (x) φ k 1 (x), φ k 1 (x) φ k 1(x) xφ k 1(x), φ k 2 (x) φ k 2 (x), φ k 2 (x) φ k 2(x) Exmple 2 On [ 1, 1] with wight function w(x) = 1 the orthogonl polynomils re knowns s Legendre polynomils: φ 0 (x) = 1 φ 1 (x) = x φ 2 (x) = x 2 1 3 φ 3 (x) = x 3 3 5 x A generl expression for these polynomils is s follows d n φ n (x) = 1 2 n n! dx n (x2 1) n, where dn dx n (x 2 1) n stnds for the n-th derivtive of (x 2 1) n 5 of 7
Dr M DRAIEF Exmple 3 If we chnge the intervl [, b] nd the weight we hve number of interesting fmilies of orthogonl polynomils [, b] = [ 1, 1], nd w(x) = 1 1 x 2 we hve the notorious Chebyshev polynomils T n tht stisfy T n (cos(θ)) = cos(nθ) [, b] = (, ), nd w(x) = e x2 we hve the notorious Hermite polynomils H n where H n (x) = ( 1) n e x2 dn 2 dx n e x Let us go bck to find the best pproximtion to function f, given by φ 0, φ 0 φ 0, φ 1 φ 0, φ n 0 φ 1, φ 0 φ 1, φ 1 φ 1, φ n 1 = φ n, φ 0 φ n, φ 1 φ n, φ n n f, φ 0 f, φ 1 f, φ n In φ 0,, φ n is n orthogonl bsis of R n [X] then one cn crete n orthonorml bsis ψ 0,, ψ n where ψ k = φ k φ k, φ k 1/2 nd the coefficients k such tht P n (x) = n k=0 kψ k (x) where P n given the minimum of f(x) P (x), f(x) P (x) re then given by k = f, ψ k Theorem 2 The unique L 2 pproximtion to f, ie the one tht minimises the distnce P f 2 L 2 = (f(x) P (x)2 w(x)dx is given by where f, ψ k = f(x)ψ k(x)w(x)dx P = f, ψ k ψ k, k=0 To prove this note tht f P, ψ k = 0 for ll k = 0,, n nd by consequence for ll Q R n [X] which cn be rewritten s liner combintion of the ψ k s we hve f P, Q = 0 Exmple 4 Approximte f(x) = e x for [, b] = [0, 1] nd w(x) = 1 6 of 7
Dr M DRAIEF 3 Exercises Exercise 1 Consider the following inner product on R [X]: P, Q = 1 Show tht this is in fct n inner product 1 1 P (x) Q (x) 1 x 2 dx 2 For every n N there exists unique polynomil T n such tht T n (cos x) = cos (nx) for ll x R Verify this for n = 1,, 4 Write then recursive representtion for the generl cse (ny n) 3 Verify: ( 1 X 2) T n (X) X T n (X) = n 2 T n (X) by using X = cos (x) 4 Consider now P = 0 + 1 x 1 + + n x n R n [X] We represent it s P = ( 0, 1, n ) T Given this expression, find mtrix M such tht P ( 1 x 2) P xp Wht re its rnge nd kernel for n = 3? 5 Determine the eigenvlues md eigenvectors of M 6 Show the orthogonlity of the polynomils Hint: Use x = cos (σ) nd substitute T n (x) = cos (nσ) Exercise 2 (2011) We consider the set R n [X] of polynomils with rel coefficients nd degrees less or equl to n endowed with the inner product P, Q = 1 1 P (t)q(t)dt 1 Show tht P, Q = 1 1 P (t)q(t)dt is indeed n inner product on R n[x] 2 Give the expression of P, Q when P nd Q re polynomils in R 2 [X] in terms of the coefficients of both P nd Q 3 Let L be the ppliction on R n [X] such tht L(P ) = d dx [ (X 2 1) dp ] dx () Show tht if P R n [X] then L(P ) R n [X] nd tht L is liner trnsformtion on R n [X] (b) Prove tht, for ll P, Q in R n [X], we hve Hint: Perform integrtions by prts L(P ), Q = P, L(Q) 4 Let P 0 = 1 nd for k = 1,, n, define the polynomil P k of degree k s follows the k-th derivtive of (X 2 1) k () Compute P 1 nd P 2 P k = dk dx k ( (X 2 1) k), 7 of 7
Dr M DRAIEF (b) Derive n expression of L(P k ) in terms of P k respectively (c) Prove the following identity (X 2 1) d[( X 2 1) k] dx nd P k the first nd second derivtives of P k, 2kX(X 2 1) k = 0 (d) By differentiting (k + 1) times the bove expression, estblish tht (X 2 1)P k (X) + 2XP k(x) = k(k + 1)P k (X) Hint: Use Leibniz s formul k+1 ( ) k + 1 (fg) (k+1) = f (i) g (k+1 i), i i=1 where f (i) is the i-th derivtive of f (e) Find the eigenvlues nd eigenvectors of the trnsformtion L 5 Let k, l two integers between 0 nd n () Express L(P k ), P l nd L(P l ), P k in terms of P k, P l (b) Prove tht (P 0, P 1,, P n ) is n orthogonl bsis of R n [X] when endowed with the inner product 1 1 P (t)q(t)dt These polynomils re known s Legendre polynomils 8 of 7