Fourier and Wavelet Signal Processing

Transcription

1 Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring /25/ Outline Vector spaces, Hilbert spaces Linear operators, Projection operators Orthogonal and biorthogonal bases, frames Matrix representations, expansions Goal : Set the basics in a Hilbert space setup through geometric intuition Readings: Chapter 1,, of Fourier and Wavelet Signal Processing 2/25/2011 2

2 For a vector space, we need: A set of vectors V These can be vectors in, functions, etc Think of geometry in or, we will use pictures! A field of scalars F Real or complex numbers Vector addition + Scalar multiplication Vector Spaces Easy case: N finite, linear algebra, matrices Beware: N goes to infinity convergence! 2/25/ Vector Spaces: Axioms A vector space is defined over a field (think or ) as a set with two operations Vector addition: Scalar multiplication: That satisfies the following axioms 2/25/2011 4

3 Subspace Span is a subspace when it is closed under vector addition and scalar multiplication: Vector Spaces: Key notions : set of vectors (could be infinite) = set of all finite linear combinations of vectors in is always a subspace 2/25/ Linear independence is linearly independent when: If then If is infinite, we need every finite subset to be linearly independent Dimension Vector Spaces: Key notions Dim(V) = N if V contains a linearly independent set with N vectors and every set with N+1 or more vectors is linearly dependent V is infinite dimensional if no such finite N exists 2/25/2011 6

4 Inner Products Formalize the geometric notions of orientation and orthogonality Measure similarity between vectors An inner product for V is a function satisfying 2/25/ Inner Products: Examples 2/25/2011 8

5 Geometry in Inner Product Spaces In : (true in general) 2/25/ Orthogonality and are orthogonal when,, written is orthogonal when for all we have is orthonormal when it is orthogonal and for all is orthogonal to when for all,written and are orthogonal when every is orthogonal to, written 2/25/

6 Orthogonal complement If is a subspace of, the orthogonal complement of is the set If is closed (contains all limits) then given, there exists s.t. 2/25/ Norm Measure length, size of vectors A norm on is a function satisfying 2/25/

7 Norms: Examples 2/25/ Distances, norms and inner products A norm induces a distance An inner product induces a norm Not all norms are induced by an inner product 2/25/

8 Norms induced by inner products: Properties Cauchy-Schwarz inequality Parallelogram Law 2/25/ Norms induced by inner products: Properties Pythagorean theorem 2/25/

9 Normed vector spaces: Standard spaces : : square-summable sequences ( finite energy sequences ) : square-integrable functions ( finite energy functions ) 2/25/ Normed vector spaces: Standard spaces spaces : The norm :, for spaces :, for Extend norm to norm as : the only norm induced by an inner product 2/25/

10 Normed vector spaces: Standard spaces spaces : Extend to : norm : only norm induced by an inner product 2/25/ Geometry of spaces 2/25/

11 Space of random variables Random variables with finite second moment Inner product and norm Apply all the abstract theorems to random variables. : inner product space of complex, continuous functions over interval : inner product space of complex, continuous functions with p-continuous derivatives over interval Usual inner product, usual norm Spaces Example: set of polynomial functions over an interval forms a subspace of, for any in and p in. Why: closed under vector space operations, and polynomials are indefinitely differentiable 2/25/

12 Hilbert spaces: Completeness A sequence is a Cauchy sequence in a normed space when for any, there exists for all A normed vector space V is complete if every Cauchy sequence converges in V A complete inner product space = Hilbert space A complete normed vector space = Banach space 2/25/ Hilbert spaces: Examples is not a Hilbert space because it is not complete is a Hilbert space 2/25/

13 Hilbert spaces: Examples All finite dimensional spaces are complete and are complete and are Hilbert spaces are not complete except under norm Vector space of random variables as already defined is a Hilbert space. 2/25/ Summary on spaces 2/25/

14 Linear operators is a linear operator when for all 1. Additivity: 2. Scalability: Null space (subspace of ): Range space (subspace of ): Operator norm: A is bounded when: Inverse: 2/25/ Adjoint Operators Generalizing Hermitian transpose of matrices is the adjoint of when If is self-adjoint or Hermitian Intuitively, if has an effect, preserves the geometry of that effect while acting on reversed domain. 2/25/

15 Adjoint operators: local averaging 2/25/ Adjoint operators: Properties Let be a bounded linear operator 1. exists and is unique and are self-adjoint If invertible, 6. bounded, 7. bounded, 2/25/

16 Unitary Operators bounded linear operator is unitary when: 1. is invertible 2. preserves inner products: for every If is unitary, then is unitary iff 2/25/ Best approximation closed subspace of a Hilbert space Best approximation problem 2/25/

17 Orthogonal projection Existence of Orthogonality: Uniqueness of Linearity: where is a linear operator Idempotency: Self-adjointness: Example Consider: Find the closest degree 1 polynomial Solution: use orthogonality

18 Projection operators is idempotent when A projection operator is a bounded linear operator that is idempotent An orthogonal projection operator is a self-adjoint projection operator An oblique projection operator is not self adjoint 2/25/ Orthogonal projection operators A bounded linear operator or satisfies iff is an orthogonal projection operator Example: Orthogonal projection onto 1D subspace If bounded and is a left inverse of, then is a projection operator. If then, is an orthogonal projection 2/25/

19 Projection operators: Example Local averaging example is identity, is orthogonal projection Projections and direct sums projection on H, then If closed subspaces s.t then there exists projection on H s.t and 2/25/

20 Summary Set the basics in a Hilbert space setup through geometric intuition Vector spaces, subspaces Norms, inner products Hilbert spaces Linear operators Projections