Hilbert spaces Let H be a complex vector space. Scalar product, on H : For all x, y, z H and α, β C α x + β y, z = α x, z + β y, z x, y = y, x x, x 0 x, x = 0 x = 0 Orthogonality: x y x, y = 0 A scalar product defines a norm on H : x = x, x 1 2 Cauchy-Schwarz inequality: For all x, yinh x, y x y Convergence in norm: x n x 0 Continuity of the scalar product: x n x 0 x n, y x, y and x n x A sequence (x n ) is a Cauchy sequence if x n x m 0 as m, n. Hilbert space: A vector space H is a Hilbert space if it has a scalar product and it is complete, i.e. every Cauchy sequence (x n ) has a limit x H : x n x o as n. Fourier Theory I, Oct 9, 2003-1 -
Hilbert spaces Example: Let L 2 (Ω, F, P) be the space of all real-valued square integrable random variables defined on a probability space (Ω, F, P). Define X, Y = E(XY )., only scalar semiproduct: X, X = 0 if P(X = 0) = 1. Define equivalence classes: X Y if P(X = Y ) = 1. The space of all equivalence classes is a Hilbert space. Example: Let L 2 [ π, π] be the space of all complex-valued functions f on [ π, π] = such that f() 2 d <. The scalar product of f, g L 2 [ π, π] is defined by f, g = f() g() d. L 2 [ π, π] is a Hilbert space (again up to equivalence f g f = g a.e.). Fourier Theory I, Oct 9, 2003-2 -
Projection in Hilbert spaces Projection Theorem: If M is a closed subspace of H then there exists a unique ˆx M such that x ˆx = inf x y y M for any x M (x x ) M x = x ˆx = P M x is called the (orthogonal) projection of x onto M. Suppose that M = sp{e 1,..., e n }, where e 1,..., e n is an orthonormal subset of H, i.e. { 1 if i = j e i, e j = 0 if i j. Then P M x is determined by the equations P M x, e i = x, e i, i = 1,..., n. Furthermore, we have for all x H P M x = n x, e i e i i=1 P M x 2 = n n i=1 i=1 x, ei x 2 x, e i 2 (Bessel s inequality) Fourier Theory I, Oct 9, 2003-3 -
Fourier series In the following, we consider the complex Hilbert space L 2 [ π, π] with scalar product f, g = 1 f() g() d. Let Note: e k () = e ik, k Z. The set {e k, k Z} is orthonormal: e i, e j = 1 e i(j k) d = f, e k = 0 for all k Z f = 0 a.e. L 2 [ π, π] = sp{e k, k Z}. Fourier approximation: { 1 if j = k 0 if j k. The n-th order Fourier approximation of a function f is given by S n f = f, e k e k with Fourier coefficients f, e k = 1 f() e ik d. Theorem: S n f f 0 as n. Corollary: f 2 = k Z f, e k 2 f, g = k Z f, e k g, e k (Parseval s identity) Fourier Theory I, Oct 9, 2003-4 -
Fouries series Example: 1 α 2 f() = 1 + α 2 2α cos() C(k) = f() e ik d = α k S n f() = 1 C(k) e ik = 1 + 2 n α k cos(k) k=1 n=4 n=8 15 15 f() 10 f() 10 5 5 0 0 n=16 n=32 15 15 f() 10 f() 10 5 5 0 0 Fourier Theory I, Oct 9, 2003-5 -
Fouries series Example: f() = 1 [ 1,1] () C(k) = f() e ik d = 2 sin(k) k S n f() = 1 C(k) e ik = 1 π + 2 n k=1 2 sin(k) πk cos(k) 1.2 n=4 1.2 n=8 f() f() 1.2 1.2 n=32 f() f() Fourier Theory I, Oct 9, 2003-6 -
Fourier series Remarks: The Fourier series S n f can be written as where S n f() = = 1 D n () = f, e k e k () is the Dirichlet kernel. e i( µ)k f(µ) dµ = 1 D n ( µ) f(µ) dµ sin ( (n + 1 2 )) e i( µ)k = sin ( 1 2 ) if 0 2n + 1 if = 0 D5() D10() Fourier Theory I, Oct 9, 2003-7 -
Fourier series Remarks: If f is continuous with f(pi) = f(π) then S k f f n 1 1 n k=0 uniformly on [ π, π] as n. We have S k f() = 1 n 1 1 n k=0 = ( k ) 1 e i( µ)k f(µ) dµ n K n ( µ) f(µ) dµ where K n () = 1 n ( k ) 1 e i( µ)k = n 1 n n sin 2 ( 1 2 n) sin 2 ( 1 2 ) if 0 if = 0 K5() K10() Fourier Theory I, Oct 9, 2003-8 -
Fourier series Remarks: Suppose that f has derivative f L 2 π, π] such that f ( π) = f(π). Then f, e k = (ik) 1 f, e k S n f converges absolutely, f, e k < k Z S n f converges uniformly to f, Sn f() f() 0 sup [ π,π] as n Fourier Theory I, Oct 9, 2003-9 -
Hilbert space isomorphism The Fourier transform defines a mapping T f = { f, e k } from L 2 [ π, π] to the complex Hilbert space l 2 (C) with scalar product C 1, C 2 = C 1 (k)c 2 (k). k Z Properties: T is bijective T is linear, T (αf + βg) = αt (f) + βt (g) T f, T g = f, g A mapping with these properties is called an isomorphism of L 2 [ π, π] onto l 2 (C). The inverse mapping T 1, T 1 (C) = 1 C(k) e k k Z is then an isomorphism of l 2 (C) onto L 2 [ π, π]. Further properties: {e k } complete orthonormal set of L 2 [ π, π] {T e k } complete orthonormal set of l 2 (C). T F = f for all f L 2 [ π, π]. T F n T F 0 F n F 0. {T f n } Cauchy sequence {f n } Cauchy sequence. T P sp{fi,i I}(f) = P sp{t fi,i I}(T f) The two spaces L 2 [ π, π] and l 2 (C) are mathematically equivalent. Fourier Theory I, Oct 9, 2003-10 -
Autocorrelation and spectral densities Let X = {X(t)} be a stationary process with autocovariance function γ(u). Suppose that u Z γ(u) 2 <, that is, γ l 2 (R). In this case we define the second-order spectrum (or power spectrum) of X by f() = 1 γ(u) e iu. k Z The summability condition on γ implies that X(t) and X(t+u) become increasingly less dependent as u. A more common mixing condition is u Z γ(u) <, that is, well-separated (in time) values of the process are even less dependent than implied by the previous condition. Under this assumption the spectrum f of X is bounded and uniformly continuous. If the autocovariance function satisfies ( ) 1 + u k γ(u) <, u Z then the spectrum has bounded and uniformly continuous derivatives of order k. Fourier Theory I, Oct 9, 2003-11 -