Chpter 4 Hilbert Spces 4.1 Inner product spces In the following we will discuss both complex nd rel vector spces. With L denoting either R or C we recll tht vector spce over L is set E equipped with ddition, i.e., mp (x, y) x + y from E E to E nd multipliction by sclrs in L, i.e., mp (λ, x) λx from L E to E stisfying the well known vector spce xioms. Exmples of rel vector spces re known from liner lgebr they include R k nd F(M, R) of rel functions defined on set M with pointwise ddition nd multipliction by rel numbers. Likewise we hve the complex vector spces C k nd F(M, C) of complex vlued functions on set M with pointwise ddition nd multipliction by complex numbers. If M is subset of R k then the set C(M, C) of continuous functions is subspce of F(M, C), since ddition of continuous functions results in continuous function nd multipliction of continuous function by number is lso continuous function. We will encounter mny other interesting exmples of subspces of F(M, C). We turn to the definition of inner product spces. Definition 4.1 (Inner product spces). Let E be vector spce over L (= R or C). An inner product (lso clled (sclr product) on E is mp (, ) : E E L, stisfying the following conditions (where 0 denotes the null-vector in E): i) x E \ {0} : (x, x) > 0, ii) x, y E : (x, y) = (y, x), iii) x, y, z E : (x + y, z) = (x, z) + (y, z), iv) λ L x, y E : (λx, y) = λ(x, y). If (, ) is n inner product on E we cll the pir (E, (, )) n inner product spce. Note tht (x, x) > 0 in i) mens tht (x, x) is rel positive number. In the cse L = R, complex conjugtion in ii) is of course superfluous. 1
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 2 The lst two conditions bove express tht the mp x (x, y) from E into L is liner for ech fixed y E. If we combine this with ii) we see tht (x, y + z) = (x, y) + (x, z), (4.1) (x, λy) = λ(x, y), (4.2) for ll x, y, z E nd λ L. We sy tht the inner product is conjugte liner in the second vrible. In the cse L = C mp from E E C, which is liner in the first vrible nd conjugte liner in the second vrible is often sid to be sesqui-liner form on E. Thus we hve seen tht n inner product on complex vector spce E is sesquiliner form on E. Conversely, ny sesqui-liner form, stisfying the property i), which is referred to s sying tht the form is positive definite, is n inner product. To see this we only hve to show tht ii) is stisfied. Note first tht by the linerity in the first vrible we hve (0, x) = 0 for ll x E. In prticulr, we obtin (0, 0) = 0, nd thus from i) we find x E : (x, x) = 0 x = 0. (4.3) Together with i)this shows tht (x, x) R for ll x E. Applying (x + y, x + y) = (x, x) + (y, y) + (x, y) + (y, x) we obtin tht (x, y) + (y, x) R, i.e., Im(x, y) = Im(y, x) for ll x, y E. Replcing x by ix nd using the sesqui-linerity we find i(x, y) i(y, x) R, i.e., Re(x, y) = Re(y, x), which is wht we wnted to show. Given n inner product we define the norm x of x E by x = (x, x). From i) bove it follows tht x > 0 for x 0, nd from iv) nd (4.2) it follows tht λx 2 = (λx, λx) = λλ(x, x) = λ 2 x 2. i.e., λx = λ x. (4.4) Below, s consequence of the Cuchy-Schwrz inequlity in Theorem 4.10 we will show the tringle inequlity x + y x + y, x, y E. (4.5) Positivity nd the two previous reltions re the chrcteristic properties of the norm. The norm defines nturl generliztion of the Eucliden distnce in R 3 (see (1.1) in Chpter 1) by d(x, y) = x y. (4.6) Of specil importnce in wht follows is tht the norm defines notion of convergence of sequences in E:
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 3 Definition 4.2 (Convergence of sequences). Let (x n ) n N be sequence in n inner product spce E, i.e., mp n x n from N into E. We sy tht the sequence converges to x E, or hs the limit x, nd write x n x for n or lim n x n = x, if x n x 0 for n. Finlly, we shll need the importnt notion of Cuchy sequence. Definition 4.3 (Cuchy sequence). We sy tht the sequence (x n ) n N in E is Cuchy sequence, if for ll ε > 0 there is n N N such tht x n x m < ε for ll n, m > N, (which is sometimes written s x n x m 0 s n, m ). Proposition 4.4. Every convergent sequence in n inner product spce E is Cuchy sequence Proof. Exercise 4.1 If the converse holds we sy tht the spce E is complete or tht it is Hilbert spce. Definition 4.5 (Hilbert Spce). If every Cuchy sequence in nd inner product spce E is convergent we sy tht E is complete inner product spce or tht it is Hilbert spce. It is fundmentl property of the rel numbers (equivlent to the supremum property, tht every non-empty set of rel numbers with n upper bound hs supremum, i.e., lest upper bound) tht R is complete wrt. the usul inner product, i.e., the usul product. It is simple consequence of this tht R k s well s C k re complete for ll k wrt. the usul inner products defined in the next exmple (see Exercise 4.2). Exmple 4.6. 1. The usul inner product on C k (corresponding to the usul inner product on R k ) is defined by ((x 1,..., x k ), (y 1,..., y k )) = x 1 y 1 +... x k y k. It is left s n exercise for he reder to show tht the requirements i) iv) re stisfied. 2. On the vector spce C([, b]) of continuous complex functions on the intervl [, b] we define n inner product by (f, g) = b f(x)g(x)dx (4.7) for f, g C([, b]). In generl, for ny positive continuous function ρ : [, b] R + = ]0, + [ we get n inner product (, ) ρ by (f, g) ρ = b f(x)g(x)ρ(x)dx. (4.8)
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 4 We remind the reder tht the integrl of continuous complex function is obtined by integrting the rel n imginry prt, i.e., if f = Ref + iimf then b f(x)dx = b Ref(x)dx + i b Imf(x)dx. (4.9) Then f b f(x)dx defines liner mp from C([, b]) into C, which immeditely implies tht (, ) ρ stisfies iii) nd iv) in Definition 4.1. Property ii) follows from b f(x)dx = b f(x)dx, nd i) is consequence of the fct tht if f is continuous nd f 0 then b f(x)dx = 0 if nd only if f = 0. Note tht if ρ nd ρ denote the mx nd the min of the positive convex function ρ on [, b] such tht 0 < ρ ρ ρ < +, then ρ f f ρ ρ f for f C([, b]), where is the norm given by the inner product (4.7)nd ρ is the norm given by (4.8). 3. Let l 0 (N) denote the set of complex sequences (x n ) n N tht vnish from certin point, i.e., for which only finite number of elements re different from 0. The set l 0 (N) is then subspce of the vector spce of ll complex sequences, which with our previous nottion is the spce F(N, C). On l 0 (N) we define n inner product by ((x n ), (y n )) = x n y n, where the sum on the right hs only finitely mny terms different from 0 (nd thus obviously is convergent). Tht this defines n inner product on l 0 (N) is seen s in the cse of C k. 4.2 Orthogonlity Let now (E, (, )) be n inner product spce. We sy tht two vectors x, y E re orthogonl nd write x y if (x, y) = 0. We generlize this notion nd sy tht vector x E is orthogonl to subset A E, nd write x A, if x is orthogonl to ll vectors in A. The orthogonl complement A of A is defined to be the set of ll vectors orthogonl to A, i.e., A = {x E (x, y) = 0 for ll y A}. (4.10) We note tht by iii) nd iv) in Definition 4.1 A is subspce of E for ny subset A E. For the sme reson A = (spna). (4.11) Here spna is the subspce of E spnned by ll elements in A, i.e., the spce consisting of ll liner combintions of vectors from A.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 5 A fmily (x i ) i I of vectors from E, where I is ny index set is sid to be n orthogonl fmily if (x i, x j ) = 0, whenever i j, i.e., if the vectors in the set re pirwise orthogonl. If moreover x i = 1 for ll i I we sy tht the fmily is orthonorml. A fmily (x i ) i I of vectors from E is sid to be linerly independent if ny finite subfmily of (x i ) i I is linerly independent. Lemm 4.7. Let (x i ) i I be n orthogonl fmily in E, such tht x i 0 for ll i I. Then (x i ) i I is linerly independent fmily. Proof. Let (x i1,..., x in ) be finite subfmily of (x i ) i I nd ssume tht the sclrs λ 1,..., λ n stisfy λ 1 x i1 +... + λ n x in = 0. If we for ll j {1,..., n} clculte the inner product with x ij on both sides of this eqution we rrive t λ 1 (x i1, x ij ) +... + λ n (x in, x ij ) = 0. By the ssumption tht (x ik, x ij ) = 0 for k j we rrive t λ j (x ij, x ij ) = 0. Hence λ j = 0 becuse (x ij, x ij ) 0, since x ij 0. We lso hve the following generliztion of the Pythgoren Theorem. Theorem 4.8 (Pythgors). Let (x 1,..., x n ) be finite orthogonl fmily. Then x i 2 = x i 2. Proof. We hve x i 2 = x i, = j=1 (x i, x i ) = x j = (x i, x j ) i,j=1 x i 2, where the third equlity used tht only the digonl terms corresponding to i = j contribute to the sum by the orthogonlity ssumption. The next theorem is lso well known from elementry liner lgebr. Theorem 4.9 (Orthogonl projections nd Bessel s inequlity). Let (e 1,..., e n ) be finite orthonorml fmily in E. For ll vectors x E there is unique vector u spn{e 1,..., e n } such tht x u {e 1,..., e n }. It is given by u = (x, e i )e i. (4.12) nd my be chrcterized s the unique vector in spn{e 1,..., e n } with the shortest distnce to x wrt. the norm. The vector u given in (4.12) is clled the orthogonl projection of x on the subspce spn{e 1,..., e n }.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 6 Moreover, we hve Bessel s inequlity (x, e i ) 2 x 2 (4.13) for ll x E. Proof. All vectors u spn{e 1,..., e n } cn be written in the form u = λ 1 e 1 +... + λ n e n, where λ 1,..., λ n L. Clculting the inner product with e i on both sides of this eqution we rrive t (u, e i ) = λ i (e i, e i ) = λ i since (e i, e i ) = 1. Hence u = (u, e i )e i (4.14) for u spn{e 1,..., e n }. But x u {e 1,..., e n } is equivlent to (x u, e i ) = 0 for ll i = 1,..., n. Thus (x, e i ) = (u, e i ) for i = 1,..., n. Together with (4.14) this gives the first prt of the theorem. If v spn{e 1,..., e n } then x v = (x u) + (u v) where (x u) (u v). Thus x v 2 = x u 2 + u v 2 x u 2, with equlity if nd only if u = v. Bessel s inequlity follow from Theorem 4.8 if we note tht x = u + (x u) where u (x u), since then x 2 = u 2 + x u 2 u 2 = (x, e i ) 2. In the lst step we hve used tht (x, e i )e i 2 = (x, e i ) 2. From Bessel s inequlity we shll now derive the Cuchy-Schwrz inequlity Theorem 4.10 (Cuchy-Schwrz inequlity). If E is n inner product spce then for ll x, y E we hve (x, y) x y. (4.15) Proof. In fct, if y = 0 then both sides of the inequlity re zero nd if y 0 then 1 y y = 1 nd it follows from (4.13) with n = 1 nd e 1 = 1 y y tht ( ) 1 x, y y x, from which (4.15) follows fter multipliction by y on both sides. From the Cuchy-Schwrz inequlity we derive the tringle inequlity (4.5) x + y 2 = (x + y, x + y) = x 2 + (x, y) + (y, x) + y 2 = x 2 + 2Re(x, y) + y 2 x 2 + 2 (x, y) + y 2 x 2 + 2 x y + y 2 = ( x + y ) 2.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 7 4.3 Continuity of the inner product In the following we will repetedly mke use of the fct tht the inner product (, ) : E E L is continuous. This is seen s follows. Let x 0, y 0 E be given nd choose x, y E such tht x x 0 δ nd y y 0 δ where δ > 0 is given. Then (x, y) (x 0, y 0 ) = (x, y y 0 ) + (x x 0, y 0 ) (x, y y 0 ) + (x x 0, y 0 ) x y y 0 + x x 0 y 0 δ( x + y 0 ) δ( x 0 + δ + y 0 ). (4.16) Here we hve used the Cuchy-Schwrz inequlity nd tht x = (x x 0 ) + x 0 x x 0 + x 0 x 0 + δ. Since the lst expression in (4.16) tends to 0 s δ 0 we conclude tht for ll ε > 0 there is δ > 0 such tht (x, y) (x 0, y 0 ) < ε if x x 0 δ nd y y 0 δ. This is exctly the continuity of the inner product. The continuity my equivlently be stted s sying tht for ll sequences (x n ) nd (y n ) in E we hve tht (x n, y n ) (x 0, y 0 ) s n, (4.17) if x n x 0 nd y n y 0 s n. Definition 4.11 (Convergence of series). A series x n with terms in normed vector spce E is sid to be convergent with sum x = x n in E if the sequence of prtil sums (s k ) k N defined by converges to x s k. s k = k x n (4.18) In Exercise 4.19 n lterntive definition of Hilbert spce is given using the concept of bsolute convergence of series. From (4.17) nd iii) in Definition 4.1 it follows tht ( ) ( k ) k x n, y = lim x n, y = lim (x n, y) = (x n, y) (4.19) k k nd likewise tht ( y, ) x n = (y, x n ) (4.20) for ny convergent series x n in E nd ll y E. An other consequence of the continuity of the inner product is tht A = (A) for ny subset A E. Here A denotes the closure of the set A, i.e., the set of ll limit points of
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 8 sequences from A. In fct, if x A nd y A there is sequence (y n ) in A such tht y = lim y n. Thus we conclude tht (x, y) = lim(x, y n ) = 0, i.e., x y. Since y A ws rbitrry we see tht A (A). The opposite inclusion follows immeditely from A A. Together with (4.11) this implies tht A = (spna) = (spna). (4.21) We cll spna the closed subspce spnned by A. Likewise it it seen in Exercise 4.5 tht A is closed subspce of E (i.e., A = A for ny subset A E). 4.4 Hilbert spces It is well known from liner lgebr tht ny finite dimensionl inner product spce E hs orthonorml bses. If we let (e 1,..., e n ) denote such bsis. If x E we denote by x = (x 1,..., x n ) L n the coordintes of x wrt. this bsis, i.e., x = x 1 e 1 +... + x n e n, According to Pythgors Theorem 4.8 we hve x = ( x 1 2 +... + x n 2 ) 1/2. It follows tht the mp (x 1,..., x n ) x 1 e 1 +... + x n e n is liner isometry from L n onto E nd the inverse mp is x ((x, e 1 ),..., (x, e n )). Since the vector spce L n is known to be complete wrt. the usul norm we conclude from tht E is lso complete, i.e., Hilbert spce. For generl Hilbert spces we will see tht the requirement of completeness will ensure tht severl properties of finite dimensionl spces will generlize to infinite dimensionl spces. We will first consider importnt exmples of Hilbert spces. Exmple 4.12. 1. We hve seen bove tht ny finite dimensionl inner product spce is Hilbert spce. This is in prticulr true for R k nd C k (see Exercise 4.2). 2. The spce l 0 (N) (see Exmple 4.6) is not complete. In fct, let x n l 0 (N) be given by x n = (1, 1 2, 1 3,..., 1 n, 0, 0,... ). Then for m n we see tht x n x m 2 = k=m+1 nd hence the sequence (x n ) n N is Cuchy sequence in l 0 (N) since the series is convergent. The sequence (x n ) n N is however obviously not convergent in l 0 (N). Thus l 0 (N) cnnot be Hilbert spce. 1 k 2 k=1 1 k 2
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 9 If we insted consider the lrger subspce l 2 (N) of F(N, C) consisting of sequences of complex numbers ( n ) n N tht re squre summble, i.e, n 2 < +. Tht l 2 (N) is subspce of F(N, C) follows from the well-known inequlity +b 2 2( 2 + b 2 ) for complex number, b (this inequlity is ctully the Cuchy-Schwrz inequlity for the vectors (1, 1) nd (, b)). For two sequences ( n ) nd (b n ) in l 2 (N) we conclude tht n + b n 2 2 n 2 + 2 b n 2 < +. Thus ( n ) + (b n ) l 2 (N). Since we lso hve λ( n ) = (λ n ) l 2 (N) if λ C nd ( n ) l 2 (N) we hve shown tht l 2 (N) is subspce of F(N, C). Using the inequlity b 1 2 ( 2 + b 2 ) for, b C we see tht (( n ), (b n )) = n b n defines mp (, ) : l 2 (N) l 2 (N) C. In fct, the series bove is bsolutely convergent (see the definition of bsolute convergence in Exercise 4.19). It is now cler tht the bove mp defines n inner product on l 2 (N). We will now show tht l 2 (N) with this inner product is Hilbert spce. Thus ssume tht (x n ) n N is Cuchy sequence in l 2 (N). We write x n = ( n 1, n 2,... ). For every k N we hve tht n k m k xn x m. Thus the sequence ( n k ) n N is Cuchy sequence in C for ech k N. Since C is complete we conclude tht this sequence converges, i.e., there is k such tht k = lim n n k for ll k N. Let x = ( 1, 2,... ). We will now show tht x l 2 (N) nd tht x n x s n. Given ε > 0 we cn find n N N, such tht K n k m k 2 k=1 n k m k 2 = x n x m 2 ε 2 k=1 for ll n, m N nd ll K N. For m we see from this tht K n k k 2 ε 2 for n N nd ll K N. For K we get x n x 2 = n k k 2 ε 2 k=1 for n N. This shows tht x N x l 2 (N) nd hence tht x = x N (x N x) l 2 (N) nd lso tht x n x s n s ws climed. k=1
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 10 3. The spce C([, b]) with inner product given by (4.8) is not complete s we shll now discuss. The norm on this spce is given by f 2 ρ = b f(x) 2 ρ(x)dx. Let f n denote the function on [0, 2] which is equl to 0 on [0, 1], grows linerly from 0 to 1 on [1, 1 + 1 n ] nd is equl to 1 on [1 + 1 n, 2] (drw the grph!), It is esy to see tht (f n ) is Cuchy sequence in C([0, 2]) w.r.t. the norm ρ, but tht it is not convergent in C([0, 2]). As in the discussion of l 0 bove we cn extend C([, b]) to Hilbert spce L 2 ([, b]) consisting of squre integrble functions on [, b], i.e., functions f : [, b] C, such tht b f(x) 2 dx <. We here refer to generliztion of the Riemnn integrl, clled the Lebesgue integrl. We lso generlize wht is ment by functions or rther wht is ment by two functions being equl. The detils of this is beyond the scope of these notes. We shll only need the following three fcts: i) Two functions f nd g in L 2 ([, b]) re considered to be equl if b f(x) g(x) 2 dx = 0 i.e., if f g = 0, nd we sy tht f is equl to g lmost everywhere on [, b], or tht f(x) = g(x) for lmost ll x [, b]. This is, in prticulr, true for ny two functions tht differ t only finitely mny points of the intervl. This strictly speking mens tht L 2 ([, b]) is not spce of functions, but of clsses of functions tht gree lmost everywhere. The proof tht L 2 ([, b]) is vector spce with inner product given by (4.7) cn now be completed s for l 2 (N) bove. Tht L 2 ([, b]) is Hilbert spce is one of the most fundmentl results in mesure theory known s the Riesz-Fischer Theorem. ii) L 2 ([, b]) is miniml extension of C([, b]) in the sense tht the closure (C([, b]) in L 2 ([, b]) is equl to ll of L 2 ([, b]). In other words to every f L 2 ([, b]) there is sequence (f n ) n N in C([, b]), such tht f n f 0 s n. iii) In contrst to the Riemnn integrl the Lebesgue integrl is defined on n rbitrry intervl I R nd for continuous positive functions it grees with the improper Riemnn integrl. We my thus define the Hilbert spce L 2 (I) for ll intervls I, in prticulr, we my consider L 2 (R). For this spce we hve insted of (ii) bove tht the closure of the subspce C 0 (R), consisting of ll continuous functions vnishing outside bounded intervl, is ll of L 2 (R). In Section 5.5 we will need tht the sme holds for the subspce C0 (R) consisting of C -functions vnishing outside bounded intervl. For more detils on mesure theory nd Lebesgue integrtion the interested reder my consult the book M.Reed nd B. Simon: Methods of modern mthemticl physics, Vol I, Acdemic press 1972.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 11 In the following we will let H denote Hilbert spce with inner product (, ). Any subspce X of H is n inner product spce with the inner product defined s the restriction of (, ) to X X. Then X H is Hilbert spce if nd only if X is closed subspce of H, i.e., if X = X (see Exercise 4.6). In prticulr, ll finite-dimensionl subspces of H re closed. We will need the following extension of Pythgors Theorem. Theorem 4.13 (Infinite Pythgoren Theorem). Let (x i ) i N be n orthogonl fmily in Hilbert spce H. Then x i is convergent in H if nd only if nd x i 2 < +, x i 2 = x i 2. (4.22) Proof. Since H is Hilbert spce x i is convergent in H if nd only if the sequence of prtil sums (s n ) is Cuchy sequence. This mens tht for ll ε > 0 there is n N N such tht s n s m 2 = x i 2 = x i 2 ε 2 (4.23) i=m+1 i=m+1 for ll n > m N where we hve used Theorem 4.8. Since R is lso complete we hve on the other hnd tht x i 2 is convergent if nd only if the sequence (r n ) given by r n = x i 2 is Cuchy sequence in R. This mens tht for ll ε > 0 there is n N N such tht r n r m = x i 2 < ε 2 (4.24) i=m+1 for ll n > m N. The first clim in the theorem follows by compring (4.23) nd (4.24). Finlly, we see from the continuity of x x nd Theorem 4.8 tht x i 2 = lim x i 2 = lim x i 2 = lim x i 2 = x i 2, n n n which implies (4.22). We wnt to extend the notion of orthonorml bsis to the infinite dimensionl cse. We my chrcterize n orthonorml bsis in finite dimensionl inner product spce H s n orthonorml fmily tht spns H. In the infinite dimensionl cse we will only require tht n orthonorml bsis spns dense subspce, i.e., set whose closure is the whole spce. Since s we just sw ny finite dimensionl subspce in n inner product spce is closed the following notion of orthonorml bsis spce is, indeed, n extention of the finite dimensionl notion.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 12 Definition 4.14 (Orthonorml bsis). An orthonorml bsis for A Hilbert spce H is n orthonorml fmily (e i ) i I in H such tht spn{e i i I} = H. We emphsize tht this notion of orthonorml bsis is different from the purely lgebric notion of bsis, where it is required tht the bsis spns the whole set. Since H = {0} it follows tht from (4.21)tht {e i i I} = {0}. for ny orthonorml bsis (e i ) i I for H. This mens ny orthonorml bsis is mximl orthonorml fmily in H, i.e., tht there is no vector e H with e = 1, such tht e together with (e i ) i I is n orthonorml fmily. The converse tht ny mximl orthonorml fmily in H is n orthonorml bsis will be shown in Theorem 4.16 below. Note tht the definition of orthonorml bsis implies tht ny orthonorml fmily (e i ) i I is n orthonorml bsis for spn{e i i I}, i.e., for the closure of the subspce spnned by the fmily. Any Hilbert spce hs n orthonorml bsis. This clim however relies on the wht is known s the xiom of choice, which we will not discuss here. In the following we shll restrict ttention to seprble Hilbert spces, i.e., Hilbert spces tht hve n orthonorml bsis which is either finite or countble, i.e., cn be indexed by N. We know from elementry liner lgebr tht in finite dimensionl vector spces ll bses hve the sme number of elements equl to the dimension of the spce. In Exercise 4.21 we show tht in n infinite dimensionl seprble Hilbert spce ll orthonorml bses re infinite nd hve countbly mny elements. Exmple 4.15. In the spce l 2 (N) discussed in Exmple 4.12 we cn get n orthonorml bsis (e i ) i N by letting e i be the sequence with ll elements equl to 0 except element number i which we choose to be 1, i.e., { 1 for j = i (e i ) j = 0 for j i. Tht this is n orthonorml bsis follows by observing tht it is obviously n orthonorml fmily nd tht spn{e i i N} = l 0 (N) whose closure is l 2 (N) (see Exercise 4.15) We cll (e i ) i N the nturl orthonorml bsis for l 2 (N). In Section 4.6 we shll determine n orthonorml bsis for the spce L 2 ([ L, L]). It will in this cse turn out to be more convenient to use ll integers Z s n index ser for the bsis. Of course if one cn use Z one could lso hve used N (why?). In Exercise 4.18 n lterntive construction of n orthonorml bsis for L 2 ([0, 1]) is given 4.5 Orthonorml expnsions Our gol in this section is to generlize the expnsion of vectors in orthonorml bses known in finite dimensionl Hilbert spces to the cse of infinite dimensionl seprble Hilbert spces. Thus we will in this section ssume tht H is n infinite dimensionl seprble Hilbert spce. All results in this section generlize to non-seprble spces but we shll not discuss this here.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 13 Let (e i ) i N be n orthonorml fmily in H (ssumed to be infinite dimensionl) nd consider vector x H. From Bessel s inequlity (4.13) we hve tht n (x, e i ) 2 x 2 for ll n N. This implies the generlized Bessel inequlity (x, e i ) 2 x 2. (4.25) By Theorem 4.13 together with (4.25) we therefore conclude tht the series (x, e i )e i is convergent in H. If we write u = (x, e i )e i (4.26) it is cler tht u spn{e i i N}. From Definition 4.11 for convergence of series the vector u could in principle depend on the order in which we perform the sum in (), i.e., on the order we chose for the bsis e 1, e 2, e 3,.... Tht this is not the cse follows from the next theorem. Theorem 4.16 (Projections in infinite dimensionl spces). Let (e i ) i N be n orthonorml fmily in the seprble Hilbert spce H. To ny vector x H there is unique vector u spn{e i i N} such tht x u {e i i N}, nd u is given by (4.5). In prticulr, u does not depend on the order of the summtion nd we lso write u = i N(x, e i )e i. Among ll vectors in spn{e i i N} the vector u hs the shortest distnce to x. Proof. The vector u given by (4.5) stisfies tht for ech j N ( ) (u, e j ) = (x, e i )e i, e j = (x, e i )(e i, e j ) = (x, e j ). Thus, (x u, e j ) = 0 for ll j N nd hence x u {e j j N}. This proves the existence of u we show the uniqueness below. If v spn{e i i N} then u v spn{e i i N} nd thus Pythgors Theorem gives x v 2 = x u 2 + u v 2 x u 2. This shows the lst clim of the theorem. If v moreover, ssumes tht x v {e i i N} then u v = (x v) (x u) {e i i N} = (spn{e i i N}). In prticulr, (u v, u v) = 0, i.e., u = v showing tht u is unique. We now hve the following importnt result on orthonorml bses.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 14 Theorem 4.17 (Properties of Orthonorml Bsis). For n orthonorml fmily (e i ) i N in seprble Hilbert spce H the following sttements re equivlent (i) (e i ) i N is n orthonorml bsis in H. (ii) {e i i N} = {0}. (iii) The orthonorml expnsion x = i N(x, e i )e i holds for ll x H. (iv) Prsevl s identity holds for ll x H. x 2 = (x, e i ) 2 (4.27) Proof. We hve previously seen tht (i) (ii). The impliction (ii) (iii) follows from Theorem 4.16 since with the nottion used there we hve x = u. Tht (iii) (iv) follows immeditely from (4.22). If we finlly ssume tht (iv) holds we find for ll x H x (x, e i )e i 2 = x 2 (x, e i ) 2 0 for n. Thus spn{e i i I} is dense in H nd hence (iv) (i). As further consequence of Theorem 4.16 we obtin the projection theorem. Theorem 4.18 (Projection Theorem). Let X be closed subspce of seprble Hilbert spce H. Then for ech x H there exist unique vectors u X nd v X such tht x = u + v. (4.28) Proof. Since X is closed subspce of the seprble Hilbert spce H then X is seprble Hilbert spce (see Exercise 4.17). We my thus choose nd orthonorml bsis (e i ) i I for X, where I is finite or equl to N. Hence X = spn{e i i I}. The clim is now n immedite consequence of Theorems 4.9 nd 4.16. As we hve seen u in (4.28) my lso be chrcterized s the vector in X with the shortest distnce to x. In generl if two subspces V nd W of vector spce E stisfy tht ny vector x E cn be uniquely written s x = v + w, where v V nd w W we sy tht V nd W re complementry nd write E = V W. We lso express this s sying tht E is the direct sum of V nd W. We my thus rephrse Theorem 4.18 s H = X X
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 15 for ny closed subspce X of H. The vector u in (4.28) is clled the orthogonle projection of x onto X nd s we hve seen it cn be clculted s u = i I (x, e i )e i, where (e i ) i I is n orthonorml bsis for X. Since X is lso closed subspce of H we hve H = X X. If we use tht ny vector x X my be written uniquely s x = u + v, with u X nd v X nd the obvious inclusion X X (why?) we see tht, in fct, u = x nd v = 0. Thus for every closed subspce X of H. projection of x onto X. X = X It then follows tht v in (4.28) is the orthogonl Proposition 4.19. Let X be closed subspce in seprble Hilbert spce H. Assume we hve orthonorml bses (e i ) i I for X nd (e j ) j J for X. Then the combined fmily (e i ) i I J is n orthonorml bsis for H Proof. Exercise 4.22 4.6 Fourier series The celebrted theory of Fourier series which hs its origin in J. Fourier s 1807 nlysis of the het eqution my be conveniently formulted in terms of orthonorml expnsions. We shll now discuss this very briefly. The relevnt Hilbert spce is H = L 2 ([ L, L]) with inner product (f, g) = 1 2L L L f(θ)g(θ) dθ. 1 We hve included the fctor 2L for convenience. The key result bout Fourier series is stted in the next theorem. We will give proof bsed on the Stone-Weierstrss Theorem (see, e.g., W. Rudin: Functionl nlysis, Chpter 5). Theorem 4.20 (Fourier bsis). Let the functions e n C([ L, L]) be given by e n (θ) = e in π L θ, θ [ L, L], n Z. Then (e n ) n Z is n orthonorml bsis in H = L 2 ([ L, L]). Proof. Tht (e n ) n Z is n orthonorml fmily is seen from the clcultion (e n, e m ) = 1 2L L L L e n (x)e m (x)dx = 1 e i(n m) π L θ dθ 2L. L [ 1 L = 2L i(n m)π ei(n m) π θ] L L = 0 for n m L 1 2L [x]l L = 1 for n = m,
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 16 Here we hve used tht e n is periodic function of θ R with period 2L. It remins to prove tht spn{e n n Z} = H. To conclude this we note tht it is enough to show tht C([ L, L]) spn{e n n Z} since C([ L, L]) = H s lredy remrked. It is thus enough to show tht for ll functions f C( L, L) nd ll ε > 0 there is n f 1 spn{e n n Z} such tht f f 1 < ε. (4.29) To see this we first choose function (why is this possible?) f 2 C([ L, L]) such tht f 2 ( L) = f 2 (L) = 0 nd f f 2 < ε 2. (4.30) Then f 2 my be extended to continuous periodic function on R with period 2L. It the follows from the Stone-Weierstrss Theorem tht there is function f 1 spn{e n n Z}, such tht f 2 (θ) f 1 (θ) < ε, θ [ L, L]. 4L Hence f 2 f 1 < ε 2. (4.31) From (4.30), (4.31), nd the tringle inequlity we rrive t (4.29). If for f L 2 ([ L.L]) we define the Fourier coefficients c n (f) by the bove theorem nd Theorem 4.17 imply tht c n (f) = 1 L f(θ)e in π L θ dθ, (4.32) 2L L f(θ) = n Z c n (f)e in π L θ. (4.33) This series is clled the Fourier series for f. We emphsize tht it converges in sense of the Hilbert spce L 2 ([ L, L]), i.e., with respect to the L 2 -norm. More precisely, this mens tht N c n (f)e n f 0 for N. n= N It is importnt to relize tht this is not the sme s uniform or pointwise convergence. For discussion on uniform nd pointwise convergence of Fourier series we refer the reder to more dvnced texts on the subject. Exercises Opgve 4.1. Proof Proposition 4.4. Hint: Use the tringle inequlity (4.5).
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 17 Opgve 4.2. () Show tht sequence (x n ) n N in R k (or C k ) is convergent or Cuchy sequence wrt. the stndrd norm if nd only if ech coordinte sequence (x i n) n N, i = 1,..., k, is convergent or Cuchy sequence respectively (in R or C). nottion x n = (x 1 n,..., x k n). We hve used the (b) Use () to conclude tht R k nd C k re complete, i.e., Hilbert spces, for ll k 2. The completeness of R is ssumed known. Opgve 4.3. Consider the finite dimensionl complex Hilbert spce H = C k, with the inner product from Exmple 4.6 (1). Show tht ((1, 0,..., 0), (0, 1, 0..., 0),..., (0,..., 0, 1)) is n orthonorml bsis for H. Opgve 4.4. Show tht n inner product on complex Hilbert spce Hilbert stisfies the polriztion identity (x, y) = 1 4 ( x + y 2 x y 2 + i x + iy 2 i x iy 2 ). Opgve 4.5. Let H be Hilbert spce. Show tht for ny subset A H the subset A is closed subspce of H. Opgve 4.6. Let H be Hilbert spce. Show tht subspce X of H is Hilbert spce if nd only if X is closed. Show lso tht the closure of subspce of H is itself subspce. (here you should use the continuity of ddition nd of sclr multipliction). Opgve 4.7. Show tht (sin nθ) n N is n orthogonl fmily in C([0, π]) with the inner product given in (4.7). Opgve 4.8. Determine 1, 2, 3 C such tht is s smll s possible. π 0 cos θ 3 n sin nθ 2 dθ Opgve 4.9. Let the polynomils p 0 (x), p 1 (x),... be given such tht p n (x) is polynomil of degree n in the vrible x with the coefficient of x n being 1 nd (p 0 (x), p 1 (x),...) being n orthonorml fmily in L 2 ([0, 1]). Find p 0 (x), p 1 (x) nd p 2 (x). Opgve 4.10. Show tht (sin(n 1 2 )θ) n N is n orthogonl fmily in C([0, π]) with inner product given by (4.7). Opgve 4.11. Let H be n infinite dimensionl seprble Hilbert spce nd let (e n ) n N be n orthonorml fmily in H. (1) Show tht the series 1 n e n is convergent in H nd determine for which α R the series nα e n is convergent in H.
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 18 (2) Determine the orthogonl projection of the vectors e 1 ± 2e 2 on the subspce spnned by the vector n 1 e n. You my use tht 1 = π2 n 2 6. π Opgve 4.12. Show tht lim n 0 log θ sin nθdθ = 0. Hint: This cn be seen s corollry to Bessel s inequlity (4.25) used on the the orthogonl fmily in Exercise 4.7. Opgve 4.13. Let (e i ) i N be n orthonorml bsis for the Hilbert spce H. Show tht the following generliztion of Prsevl s eqution holds for ll x, y H: (x, y) = (x, e i )(y, e i ). Opgve 4.14. Consider the inner product spce l 0 (N) defined in Exmple 4.6 (3). Let { } 1 X = (x n ) n N l 0 (N) x n n = 0. Show tht X is closed subspce of l 0 (N) nd tht X X l 0 (N). Opgve 4.15. Argue tht the closure of l 0 (N) is equl to l 2 (N), i.e., tht ny sequence x in l 2 (N) is the limit of convergent sequence (x n ) n N in l 0 (N). Hint: Let x n be the sequence which is equl to x up to the n-th term nd zero fterwrds. Opgve 4.16. Let H be complex Hilbert spce nd let n N, C be such tht n = 1 nd 2 1. Show the generlized polriztion identity (x, y) = 1 n 1 ν x + ν y 2. n ν=0 Opgve 4.17. () Show tht the Hilbert spce H is seprble if nd only if it hs countble dense subset ( countble set whose closure is the whole spce). Hint to the if-prt: Construct n orthonorml bsis using the Grm-Schmidt procedure. (b) Let H be seprble Hilbert spce. Show tht ny subset M of H hs n t most countble dense subset (n t most countble subset of M whose closure is M) Hint: Assume tht M is not finite. Let {x i i N} be countble dense subset of H. For i, n N let x i,n denote ny rbitrrily chosen point in {x M x x i 1 n } ssuming tht this set is non-empty. Otherwise let x i,n be n rbitrrily chosen point in M. Now show tht {x i,n i, n N} is countble dense subset of M. (c) Show tht closed subspce X of seprble Hilbert spce is itself seprble Hilbert spce. Hint: Use the chrcteriztion of seprble in question () nd the result of (b). Opgve 4.18. (Difficult) Consider the Hilbert spce H = L 2 ([0, 1]) with the inner product (f, g) = f(θ)g(θ)dθ. Let F 0 : R R be given by F 0 (t) = 1 if 0 t < 1 nd F (t) = 1 if 1 t < 2. Define f n, n = 0, 1, 2,... by F n (t) = 2 n 1 F 0 (2 n t).
Chp. 4 Opertors on Hilbert Spces Version of 19.12.08 19 Let f n,k H, n = 0, 1, 2,..., k = 0, 1,..., 2 n 1 1 be given by f n,k (θ) = f n ( θ k2 n 1 ), θ [0.1] () Show tht the fmily (f n,k n = 0, 1, 2,..., k = 0, 1,..., 2 n 1 1) is countble orthonorml fmily. (b) Show tht the subspce spn{f n,k n = 0, 1, 2,..., N, k = 0, 1,..., 2 n 1 1} consists of ll functions tht re constnt on the intervls [k2 N, (k + 1)2 N ), k = 0, 1,..., 2 N 1. (c) Show tht 1 C 1 ([, b]) spn{f n,k n = 0, 1, 2,..., k = 0, 1,..., 2 n 1 1}. Hint: Use the Men Vlue Theorem. (d) Conclude from the informtion L 2 ([0, 1]) = C 1 ([, b]) tht (f n,k n = 0, 1, 2,..., k = 0, 1,..., 2 n 1 1) is n orthonorml bsis for L 2 ([0, 1]). Opgve 4.19. Show tht n inner product spce E is Hilbert spce if nd only if ny series x n with terms in E stisfying x n < is convergent. On Hilbert spce series x n stisfying x n < is sid to be bsolutely convergent. Opgve 4.20. Consider the Hilbert spce H = L 2 ([ 1, 1]) with the inner product (f, g) = 1 1 f(t)g(t)dt nd the elements f 1, f 2, g H given by f 1 (t) = 3/2 t, f 2 (t) = 5/2 t 2, nd g(t) = 1. Show tht f 1, f 2 re orthonorml nd find the closest vector to g in spn{f 1, f 2 }. Opgve 4.21. Show tht if H is n infinite dimensionl seprble Hilbert spce then ll orthonorml bses re infinite nd hve countbly mny elements. Hint: From the chrcteriztion of seprbility in Exercise 4.17 you know tht H hs countble dense set D. Given n orthonorml bsis show tht the blls centered t bsis vectors with rdius less thn 1 re disjoint. Argue tht these blls hve non-empty intersection with D nd then tht the orthonorml bsis is is t most countble. Opgve 4.22. Prove Proposition 4.19. 1 It is lso possible to show tht C([, b]) spn{f n,k n = 0, 1, 2,..., k = 0, 1,..., 2 n 1 1} but this is most esily done by fist showing tht continuous functions on [, b] re uniformly continuous. Uniform continuity mens tht for ny ε > 0 there is δ > 0, such tht x y < δ f(x) f(y) < ε, for ll x, y [0, 1].