WEEK 4 Abstrct inner product spces Definition An inner product spce is vector spce V over the rel field R equipped with rule for multiplying vectors, such tht the product of two vectors is sclr, nd the following properties hold: IP) (ũ, ṽ) (ṽ, ũ) for ll ũ, ṽ V ; IP) (ũ + ṽ, w ) (ũ, w ) + (ṽ, w ) for ll ũ, ṽ w V ; IP3) (kũ, ṽ) k(ũ, ṽ) for ll ũ, ṽ V nd ll k R; IP4) (ũ, ũ) for ll ũ V, nd if (ũ, ũ) then ṽ The sclr (u, v) is clled the inner product of the vectors u nd v It ws pointed out in Lecture 6 tht if A is ny n n symmetric mtrix with positive eigenvlues, then the rule (ũ, ṽ) ũ T Aṽ defines n inner product on R n In the cse tht A is the identity mtrix then this construction gives the stndrd inner product on R n ; tht is, the dot product Now consider C[, b], the spce of continuous functions from the intervl [, b] to R, nd for f, g C[, b] define (f, g) f(x)g(x) dx Checking tht (IP), (IP) nd (IP3) re stisfied is quite strightforwrd For exmple, if f, g nd h re rbitrry elements of C[, b], then (f + g, h) (f + g)(x) h(x) dx f(x)h(x) + g(x)h(x) dx (f(x) + g(x))h(x) dx f(x)h(x) dx + which shows tht (IP) holds The first prt of (IP4) is lso cler: (f, f) f(x) dx g(x)h(x) dx (f, h) + (g, h), is obviously nonnegtive The other ssertion of (IP4) tht (f, f) implies f is lso intuitively resonble: if f(x) is nonzero t ny point in the intervl [, b], then continuity of f gurntees tht there is some subintervl of [, b] nd some α > such tht f(x) > α t ll points of this subintervl, nd it follows tht f(x) dx > We omit further detils of the proof, since the clculus involved is somewht removed from the topics tht re the min focus of this course We cn lso use the formul f(x)g(x) dx to define n inner product on P[, b], the polynomil functions on [, b] (which is subspce of C[, b]) If V is n inner product spce nd f V, then, s in R n, we define f (f, f), nd cll this the length of f The Cuchy-Schwrz Inequlity sys tht (f, g) f g for ll f, g V, nd the proof of this inequlity for inner product spces in generl is just the sme s its proof in R n, since the proof uses nothing beyond the properties
(IP) (IP4) As in R n, we define the ngle between elements f nd g of n inner product spce to be rccos((f, g)/ f g ) We sy tht f nd g re orthogonl if (f, g) Pythgors Theorem holds for ny inner product spce, nd is proved in the sme wy s it is for R n And our discussion of projections lso goes through unchnged Exmple Let f, f nd f 3 be the functions [, ] R defined by f (x), f (x) x, f 3 (x) 6x + 6x These form n orthogonl set in P[, ] To see this, we must check tht (f, f ), (f, f 3 ) nd (f, f 3 ) re ll zero We hve (f, f ) (f, f 3 ) (f, f 3 ) s required ( x) dx x x ], ( 6x + 6x ) dx x 3x + x 3] ( x)( 6x + 6x ) dx, 8x + 8x x 3 ) dx x 4x + 6x 3 3x 4], Projections in inner product spces Let {ã, ã,, ã k } be bsis for finite-dimensionl subspce W of n inner product spce V For ech ṽ V there is unique W with the property tht (x, ṽ p ) for ll W ; tht is, ṽ is orthogonl p to ll elements of W The element is clled the x projection of ṽ onto p W The condition tht (x, ṽ p ) for ll W cn p be reformulted s (x, ṽ) (x, p ) for ll W, nd this in turn is equivlent to x the condition tht (ã x i, ṽ) (ã i, p ) for ll i,,, k We my write s liner combintion of the bsis elements of W, p λ ã + λ ã + + λ k ã k, () p nd then the condition becomes (ã, ṽ) (ã, p ) (ã, λ ã + λ ã + + λ k ã k ) (ã, ṽ) (ã, p ) (ã, λ ã + λ ã + + λ k ã k ) (ã k, ṽ) (ã k, p ) (ã k, λ ã + λ ã + + λ k ã k ) (ã, ã )λ + (ã, ã )λ + + (ã, ã k )λ k (ã, ã )λ + (ã, ã )λ + + (ã, ã k )λ k (ã k, ã )λ + (ã k, ã )λ + + (ã k, ã k )λ k (ã, ã ) (ã, ã ) (ã, ã k ) λ (ã, ã ) (ã, ã ) (ã, ã k ) λ (ã k, ã ) (ã k, ã ) (ã k, ã k ) λ k
In other words, we cn find by solving the system of liner equtions p λ (ã, ṽ) λ G (ã, ṽ), λ k (ã k, ṽ) where the mtrix G is the Grm mtrix of the bsis {ã, ã,, ã k }, nd then is given by Eq () bove p The bove clcultion is essentilly the sme s clcultion we did when discussing projections in the context of R n in the first week of lectures The formul bove is the sme s the formul we derived then, since the mtrix G bove coincides with the mtrix A T A tht we hd before As in R n, projections become simpler when orthogonl bses re used So you should generlly not use the bove formul to compute projections Insted, first pply the Grm- Schmidt process to obtin n orthogonl bsis {b, b,, b k} for the subspce W, nd then use the formul p (b, ṽ) (b, b ) b + (b, ṽ) (b, b ) b + + (b k, ṽ) (b k, b k) b k () to compute the projection of ṽ onto W Remember tht Eq () is only vlid when the bsis {b, b,, b k} is othogonl! Legendre polynomils by Let C be the spce of continuous functions [, ] R, with inner product defined (f, g) f(x)g(x) dx Let f, f, f,, f k C be defined by f n (x) x n for ll x [, ], nd let P k be the subspce of C spnned by f, f,, f k Tht is, P k is the subspce of C given by polynomil functions of degree t most k Given continuous function f: [, ] R, the projection of f onto P k is the polynomil function of degree t most k tht is the best pproximtion to f on [, ], in the lest squres sense: it gives the miniml vlue for (f(x) p(x)) dx, subject to p being polynomil of degree t most k As explined bove, for the purpose of computing such projections conveniently, we need n orthogonl bsis for P k To obtin one, we pply the Grm-Schmidt process, strting with the bsis {f, f,, f k } The formuls re s follows: g f g f (f, g ) (g, g ) g g f (f, g ) (g, g ) g (f, g ) (g, g ) g g 3 f 3 (f 3, g ) (g, g ) g (f 3, g ) (g, g ) g (f 3, g ) (g, g ) g 3
nd so on Let us compute these Firstly, we hve (f, g ) nd so it follows tht g f Next, (f, g ) (g, g ) (f, g ) (g, g ) x dx x] x dx 3 x3] ] dx x x x dx 4 x4] x dx 3,, 3 nd so Similrly, we find tht nd this gives g f (/3) g (/3) g f 3 g (f 3, g ) (f 3, g ) (f 3, g ) x 3 dx x 4 dx 5 x 3 (x 3 ) dx, g 3 f 3 g (/5) (/3) g g f 3 3 5 g Students re invited to compute further terms for themselves The polynomils we hve been clculting re known s Legendre polynomils Expressed in terms of x, the first few re s follows: g (x) g (x) x g (x) x 3 g 3 (x) x 3 3 5 x g 4 (x) x 4 6 7 x + 3 35 g 5 (x) x 5 9 x3 + 5 x g 6 (x) x 6 5 x4 + 5 x 5 3 4
Strictly speking, the Legendre polynomils re not these, but sclr multiples of these, the sclrs being chosen so tht the polynomils tke the vlue t x Performing this scling gives P (x) P (x) x In fct, the generl formul is but we shll not prove this P (x) (3x ) P 3 (x) (5x3 3x) P 4 (x) 8 (35x4 3x + 3) P 5 (x) 8 (63x5 7x 3 + 5x) P 6 (x) 6 (3x6 35x 4 + 5x 5) P n (x) [n/] k () k (n k)! n k!(n k)!(n k)! xn k, Here is some MAGMA code for clculting Legendre polynomils It utilizes the MAGMA functions Integrl nd Evlute: if f is polynomil in x then Integrl(f) is the polynomil with zero constnt term whose derivtive is f; if f is polynomil in x nd t number then Evlute(f,t) is the number obtined by evluting f when x is given the vlue t (It is f(t), so to spek, lthough f(t) is not correct MAGMA syntx) > R : RelField(); > P<x> : PolynomilAlgebr(R); > polyip : func< f,g Evlute(Integrl(f*g),) - > Evlute(Integrl(f*g),-) >; > // > // So polyip(f,g) is the integrl of f*g from - to, > // which is exctly the inner product of f nd g > // > f : []; > for i in [6] do for> f[i+] : x^i; for> end for; > f; [ x x^ x^3 x^4 x^5 x^6 ] 5
> g : [ ]; > for i in [6] do for> g[i+]:f[i+]; for> for j in [i] do for for> g[i+] : g[i+] - (polyip(g[i+],g[j])/ for for> polyip(g[j],g[j]))*g[j]; for for> end for; for> end for; > g; [, x, x^ - /3, x^3-3/5*x, x^4-6/7*x^ + 3/35, x^5 - /9*x^3 + 5/*x, x^6-5/*x^4 + 5/*x^ - 5/3 ] > q:[ ]; > for i in [7] do for> q[i] : g[i]/evlute(g[i],); > end for; >q; [, x, 3/*x^ - /, 5/*x^3-3/*x, 35/8*x^4-5/4*x^ + 3/8, 63/8*x^5-35/4*x^3 + 5/8*x, 3/6*x^6-35/6*x^4 + 5/6*x^ - 5/6 ] Exmple Let us use the orthogonl bsis {g, g, g } to compute the projection onto the spce W of the function f: [, ] R given by f(x) e x (for ll x [, ]) By the formul, the projection p is given by p (f, g ) (g, g ) g + (f, g ) (g, g ) g + (f, g ) (g, g ) g This formul, of course, is only pplicble since the g i form n orthogonl set We need to clculte few integrls: (f, g ) (f, g ) (f, g ) e x dx e x] e e, e x x dx xe x e x] e, e x (x 3 ) dx x e x xe x + 5 3 ex] 6 3 e 4 3 e
We found bove tht (g, g ) nd (g, g ) 3, nd we lso need (g, g ) The finl nswer is (x 3 ) dx 8 45 p(x) e e + e (/3)e (4/3)e x + (x (/3) (8/45) 3 ) 996 + 4x + 537x Note tht the Tylor series for e x bout x is + x + x + 6 x3 +, nd in prticulr the polynomil we hve found is not much different from the degree Tylor polynomil, + x + x The difference is due to the fct tht the ccurcy of the Tylor polynomil s n pproximtion to e x improves the closer x is to zero, wheres the lest squres pproximtion gives equl weight to ll vlues of x in the intervl [, ] Fourier series Let C[, π] be the spce of continuous functions [, π] R, with inner product (f, g) f(x)g(x) dx Let c, c, c, nd s, s, be the elements of C[, π] defined by c n (x) cos(nx) s n (x) sin(nx) for x in the intervl [, π] Note tht s n is not defined for n, while c is the constnt function c (x) (since cos(x) cos for ll x) To compute inner products involving these, we need to use some stndrd trigonometric identities For ll nonnegtive integers n nd m, (s n, c m ) sin(nx) cos(mx) dx (sin(n + m)x sin(n m)x) dx ( n+m cos(n + m)x n m cos(n m)x) ( m cos(m x)) if n m if n m since the term sin(n m)x vnishes when n m The crucil point to observe now is tht, by the periodicity of the cosine function, cos(kx) tkes the sme vlue t x s t x π, whenever k is n integer So in both cses the bove integrl vnishes, nd 7
we deduce tht (s n, c m ) Similrly, if n, m re positive integers, (s n, s m ) sin(nx) sin(mx) dx (cos(n m)x cos(n + m)x) dx ( n m sin(n m)x n+m sin(n + m)x) (x m sin(m x)) { if n m π if n m, if n m if n m where gin the periodicity of the sin function simplifies the clcultions (Note tht in the cse n m bove the term cos(n m)x becomes simply cos, nd integrting this gives x The formul n m sin(n m)x for the integrl of cos(n m)x is not vlid when n m, but it is vlid for ll other vlues of n nd m) The clcultion of (c n, c m ), when n nd m re positive integers, is nlogous to the clcultion of (s n, s m ): (c n, c m ) cos(nx) cos(mx) dx (cos(n m)x + cos(n + m)x) dx ( n m sin(n m)x + n+m sin(n + m)x) (x + m sin(m x)) { if n m π if n m if n m if n m When n nd m re both zero the bove clcultion does not pply (becuse of the n+m ), nd in fct (c, c ) dx π These clcultions hve shown, in prticulr, tht {c, s, c, s, c, s 3, c 3,, s k, c k } is n orthogonl set in C[, π] (for ny vlue of k) If we define W k to be the subspce spnned by this set, then we cn clculte the projection of ny f C[, π] onto this subspce by mens of the generl formuls we hve obtined Specificlly, if p is the projection of f then p is given by the formul p (f, c ) (c, c ) c + (f, s ) (s, s ) s + (f, c ) (c, c ) c + + (f, c k) (c k, c k ) c k In view of the formuls for (c n, c n ) nd (s n, s n ) this yields, for ll x [, π], p(x) ( π ( + π ) ( f(t) dt + π ) f(t) sin(t) dt ) ( f(t) sin t dt sin x + π ( sin(x) + + π 8 ) f(t) cos t dt cos x ) f(t) cos(kt) dt cos(kx)
This is the best pproximtion in the lest squres sense to the function f on the intervl [, π] by function in the subspce W k The lrger k is, the better the pproximtion Letting k tend to yields n infinite series known s the Fourier series of f Exmple 3 Let us find the Fourier series for the function f(x) x on the intervl [, π] It is useful to remember tht if function g hs the property tht g( x) g(x) for ll x [, ], then g(x) dx In prticulr, the function g defined by g(x) x cos(nx) hs this property (for ny vlue of n), nd so x cos(nx) dx Thus the coefficient of cos nx in the Fourier series of x is zero for ll n The coefficient of sin(nx) is π π x sin(nx) dx π ( xn cos(nx) n ( x πn cos(nx) + πn sin(nx) ) cos(nπ) + πn πn cos(n()) ()n+ n So the Fourier series of x on [, π] is ) cos(nx) dx (sin x sin(x) + 3 sin(3x) 4 sin(4x) + 4 sin(5x) ) It cn be shown tht this series converges to x when x [, π] When x / [, π] the series still converges, but, rther thn x, the limit is x kπ, where k is defined by the requirement tht x kπ [, π] We conclude the section of the course on inner product spces with two more exmples of clcultions with spces of continuous functions The first of these ws done incorrectly in lectures: I indvertently omitted couple of squre root signs, so tht the quntities which I sid were the length of f nd the length of g were in fct the squres of these lengths Exmple 4 We verify the Cuchy-Schwrz inequlity for the functions f(x) x nd g(x) e x in C[, ] Recll tht the Cuchy-Schwrz inequlity sys tht (fg) f g 9
Verifying this for f nd g s given is simply mtter of evluting some integrls: f ] (f, f) x dx 3 x3 8 3, g ] (g, g) e x dx ex (e4 ), (f, g) ( xe x ) dx xe x + e x ] (e e ) e e Thus (f, g) + e 4 839, which is less thn f g 3 (e4 ) 845 Note tht, by definition, the ngle between f nd g is cos ( (f,g) f g ), lthough this (f,g) quntity hs no geometricl interprettion In this cse f g is firly close to, nd so the ngle is firly close to zero In fct it is pproximtely 7 rdins, or 79 degrees Exmple 5 For our finl exmple we compute the best pproximtion to cos x in P [, π] (the spce of polynomil functions of degree t most on the intervl [, π] The necessry first step is to find n orthogonl bsis for P [, π] We do this by pplying the Grm-Schmidt process to the bsis {f, f, f }, where f i (x) x (for ll x [, ]) This is very similr to the clcultion of the Legendre polynomils, but the numbers come out differently since we re working over different intervl now The new bsis is g f g f (f, g ) (g, g ) g g f (f, g ) (g, g ) g (f, g ) (g, g ) g We hve (f, g ) x dx, nd so g f Now Thus (f, g ) (f, g ) (g, g ) x dx 3 π3, x 3 dx, dx π g f (/3)π3 g f π π 3 g, so tht g (x), g (x) x nd g (x) x π 3, for ll x [, π] Since {g, g, g } is n orthogonl bsis for the spce, the projection of cos onto this spce is given by (cos, g ) (g, g ) g + (cos, g ) (g, g ) g + (cos, g ) (g, g ) g
We find tht (cos, g ) cos x dx sin x And (cos, g ) x cos x dx, since the function f(x) x cos x stisfies f( x) f(x) for ll x [, π] Now nd we lso hve tht (cos, g ) (g, g ) (cos x)(x π 3 ) dx x (cos x) dx x (sin x) x(sin x) dx x (sin x) ( x(cos x) + x (sin x) + x(cos x) (sin x) π() ()() 4π, (cos x) dx ) x 4 π 3 x + π4 9 dx 5 π5 4 9 π5 + 9 π5 8 45 π5 4π So the projection of cos onto P [, π] is (8/45)π (x π 5 3 ) 45 π (x π 4 3 ) The digrm below is firly ccurte grph of cos nd its projection