Inner-product spces Definition: Let V be rel or complex liner spce over F (here R or C). An inner product is n opertion between two elements of V which results in sclr. It is denoted by u, v nd stisfies: 1. For ech v V, v, v 0.. For ech v V, v, v = 0 iff v = 0.. For ech u, v, w V nd, b F, u + bv, w = u, w + b v, w. 4. For ech u, v V, u, v = v, u. Exercise: Show tht: 1. For ech u, v, w V nd, b C, u, v + bw = u, v + b u, w.. For ech v V, C, v, v = v, v.. For ech v V, 0, v = 0. Exmple 1: In (R n, R), x, y = n i=1 x iy i defines n inner product. Exmple : In (C(, b), C), f, g = b f(t)g(t)dt defines n inner product. Exmple : Let l be the spce of ll complex sequences {x n } for which the series n=1 x ny n converges. Then x, y = i=1 x ny n defines n inner product. Definition: Let V be liner spce. A norm on V is function V R + denoted by with the properties: 1. For ech v V, v 0.. v = 0 iff v = 0.. For ech v V nd F, v = v. 4. For ech u, v V, u + v u + v (tringle inequlity). 1
Exmple 4: If V = R or C, for ech x = (x 1, x,..., x n ) V define x = n i=1 x i. ( Eucliden norm ). Exmple 5: For V s in Exmple 4 define x = mx,,...,n x k ( infinity or uniform norm ). Exmple 6: For V s in Exmple 4 define x 1 = n x k ( 1-norm ). Exmple 7: If V = C(, b) then for ech f V, f = mx x b f(x) is norm. Theorem 1: If V is n inner-product spce, then v = v, v, v V, defines norm on V. Theorem : (Cuchy-Schwrtz inequlity) Let V be n inner-product spce. Then for ech u, v V we hve u, v u v. Proof: Assume tht = u, v 0, otherwise proof is obvious. For every λ R we hve 0 u λv = u λv, u λv = u, u λ u, v λ v, u + λ v, v = u λ u, v λ v, u + λ v, v = u λ + λ v The qudrtic in λ is minimized when λ = 1/ v which gives 0 u v s required. The Cuchy-Schwrtz inequlity my be used to prove Theorem 1. Proof (of Theorem 1): Norm conditions 1- re immedite. To show condition 4 (tringle inequlity) note tht u + v = u + v, u + v = u, u + u, v + u, v + v, v = u + Re( u, v ) + v u + u, v + v u + u v + v = ( u + v ) which shows tht u + v u + v.
Exmple 8: Consider the spce (R n, R) with inner product defined s in Exmple 1. Then the Cuchy-Schwrtz inequlity sttes tht x k y k n x n k yk Exmple 9: Consider the spce (C(, b), C) with inner product s defined in Exmple. Then the Cuchy-Schwrtz inequlity implies tht b f(t)g(t)dt ( b ) ( b ) f(t) dt g(t) dt Definition: Let V be n inner product spce nd u, v V. We sy tht u nd v re orthogonl if u, v = 0, in which cse we write u v. Definition: Let V n inner product spce. A (possible infinite) sequence of vectors {u k } in V is clled n orthogonl system iff u k 0 for ll k nd u k u j for ll k j. If in ddition u k = 1 for ll k, the system is clled orthonorml. Exercise: Show tht the vectors in n orthogonl system re linerly independent. Exercise: Assume V is n inner product spce nd {e 1, e,..., e n } n orthonorml system in V. Show tht if u = n ke k, then for ech k we hve k = u, e k. (The k s re the generlised Fourier coefficients of u reltive to the orthonorml system). Exercise: Let V is n inner product spce nd {e 1, e,..., e n } n orthonorml system in V. If { k } n nd {b k} n re ny two sequences of (complex) sclrs, show tht n ke k, n b ke k = n kb k nd hence, for u, v spn{e 1, e,..., e n } tht u, v = n u, e k v, e k. Theorem: Let V be n inner product spce. Then: () Let {u 1,..., u n } n orthogonl system in V nd { 1,..., n } sclrs. Then n i=1 ku k = n k u k. (b) Let {e 1,..., e n } n orthonorml system in V. spn{e 1,..., e n }, u = n u, e k. Then for every u
Proof: (b) Follows by direct clcultions: k u k = k u k, = = In this cse (from previous exercise), j=1 j u j j=1 k j u k, u j k u k u = u, u = u, e k u, e k = u, e k. Let {e 1,..., e n } be n orthonorml system in vector spce V. Let W = spn{e 1,..., e n } be (proper) subspce of V. Let u V, u / W. In this cse u n u, e k e k. Nevertheless there is n intimte reltion between u nd û = n u, e k e k, i.e. û is the orthogonl projection of u on W: Theorem: () u û, w = 0 for ll w W (u û W). (b) u w = u û + û w for ll w W. Hence Proof: () First show tht u û, e j = 0: u û, e j = u, e j = u, e j u, e k e k, e j u, e k e k, e j u û, e j = u, e j u, e j e j, e j = u, e j u, e j = 0 4
Tke w W; then w = n j=1 b je j for certin b j s nd u û, w = u û, b j e j = j=1 b j u û, e j = 0 j=1 (b) From prt () we hve tht (u û) w for every w W. (u û) (û w) nd hence Thus u w = u û + û w = u û + û w u u-u^ w w-u^ u^ W Figure 1: Orthogonl projection on subspce Theorem: Let V be n inner product spce nd {e 1,..., e n } n orthonorml system in V. Let W = spn{e 1,..., e n }. Then, the vector û = n u, e k e k is the (unique) closest vector to u in W. Proof: From previous Theorem, min u w W w = min { u w W û + û w } = u û 5
since û W, the unique minimiser being w = û. Exmple: For the linet spce C( 1, 1) define the inner product f, g = 1 f(x)g(x)dx nd the ssocited norm 1 f = 1 1 f(x) dx. Determine the closest function to x to the subspce of C( 1, 1) spnned by {1, x}. It is esy to check tht e 1 (x) = 1 nd e (x) = x form n orthonorml system in C( 1, 1). Thus x (e 1 (x) + be (x)) is minimized by nd = x, e 1 = b = x, e = 1 1 1 1 x 1 dx = 0 x xdx = 6 5 nd thus the closest function to x in the given subspce is 6 5 x = x 5. In generl, given list of n linerly independent vectors {v 1,..., v n }, we cn find n orthonorml bsis of their liner spn {e 1,..., e n } using the following procedure: Grm-Schmidt orthonormlistion process: S 1 : Set e 1 = v 1 v 1 (why is v 1 = 0?). Clerly e 1 = 1 nd spn{v 1 } = spn{e 1 }. S : Let W 1 = spn(e 1 ) nd set ˆv = v, e 1 e 1 be the orthogonl projection of v on W 1. Then v ˆv e 1 nd v ˆv 0 (why?). Define e = v ˆv v ˆv We hve tht e e 1, e = 1 nd spn{e 1, e } = spn{v 1, v }. S k : Let W k 1 = spn{e 1,..., e k 1 } nd set ˆv k = k 1 j=1 v k, e j e j be the orthogonl projection of v k on W k 1. Then v k ˆv k W k 1 nd v k ˆv k 0. Define e k = v k ˆv k v k ˆv k Hence spn{e 1,..., e k } = spn{v 1,..., v k } nd {e 1,..., e k } is orthonorml. 6
The process continues until we hve obtined the required orthonorml system spn{e 1,..., e n }. 7