Frames. Hongkai Xiong 熊红凯 Department of Electronic Engineering Shanghai Jiao Tong University

Size: px

Start display at page:

Download "Frames. Hongkai Xiong 熊红凯 Department of Electronic Engineering Shanghai Jiao Tong University"

Simon Newman
5 years ago
Views:

1 Frames Hongkai Xiong 熊红凯 Department of Electronic Engineering Shanghai Jiao Tong University

2 2/39 Frames Frames and Riesz Bases Translation-Invariant Dyadic Wavelet Transform Subsampled Wavelet Frames

3 Introduction 3/39 A signal representation may provide analysis coefficients that are inner products with a family of vectors, or synthesis coefficients that compute an approximation by recombining a family of vectors. Frames are families of vectors where analysis and synthesis representations are stable. Signal reconstructions are computed with a dual frame. Frames are potentially redundant and thus more general than bases, with a redundancy measured by frame bounds. They provide the flexibility needed to build signal representations with unstructured families of vectors. Complete and stable wavelet Fourier transforms are constructed with frames of wavelets.

4 4/39 Frames 1 Frames and Riesz Bases

5 Stable Analysis and Synthesis Operators Frame analysis operator 5/39 The frame theory was originally developed to reconstruct band-limited signals from irregularly spaced samples. The following frames definition gives an energy equivalence to invert the operator Φ defined by, Φf n = f, φ n. Definition 5.1: Frame and Riesz Basis. The sequence φ n is a frame of Η if there exist two constants B A > 0 such that fεη, A f 2 f, φ n 2 If satisfied B f 2. When A = B, the frame is said to be tight. If the φ n are linearly independent then the frame is not redundant and is called a Riesz basis. Then Φ is called a frame analysis operator.

6 Stable Analysis and Synthesis Operators Frame analysis operator 6/39 Definition 5.1: fεη, A f 2 f, φ n 2 B f 2. It is a necessary and sufficient condition guaranteeing that Φ is invertible on its image space, with a bounded inverse. Thus, a frame defines a complete and stable signal representation, which may also be redundant.

7 Stable Analysis and Synthesis Operators Frame synthesis operator Let us consider the space of finite energy coefficients l 2 Γ = a: a 2 = a n 2 < +. The adjoint Φ of Φ is defined over l 2 Γ and satisfies for any fεη and a l 2 Γ : 7/39 Φ a, f = a, Φf = a n f, φ n The frame synthesis operator: = a n φ n, f. Let a = Φf Φ a = a n φ n. Φ Φf = Φf n φ n = f, φ n φ n. Φ Φf, f = Φf, Φf = Φf 2 = f, φ n 2

8 Stable Analysis and Synthesis Operators Frame synthesis operator 8/39 Definition 5.1(Frame Condition): fεη, A f 2 f, φ n 2 B f 2. Φ Φf, f = Φf, Φf = Φf 2 = f, φ n 2 Definition 5.1(Frame Condition): fεη, A f 2 Φ Φf, f B f 2. A and B are the smallest and largest eigenvalues in finite dimension. The eigenvalues are also called singular values of Φ or singular spectrum. in finite dimension A f 2 λ i f, f B f 2 A f 2 λ i f 2 B f 2 A λ i B Φ Φf = λ i f λ i is the eigenvalues A and B are the infimum and supremum values of the spectrum of the symmetric operator Φ Φ.

9 Stable Analysis and Synthesis Operators Frame synthesis operator 9/39 Definition 5.1(Frame Condition): fεη, A f 2 f, φ n 2 B f 2. frame synthesis operator Φ a = a n φ n Stable! frame analysis operator Φf n = f, φ n Theorem 5.1: The family φ n is a frame with bounds B A > 0 if and only if aεimφ, A a 2 a n φ n 2 B a 2.

10 Stable Analysis and Synthesis Operators Frame synthesis operator 10/39 Proof of Theorem 5.1: Theorem 5.1: aεimφ, A a 2 a n φ n 2 B a 2. Theorem 5.1: Definition 5.1: aεimφ, A a 2 ΦΦ a, a B a 2. fεη, A f 2 Φ Φf, f B f 2. 2 a n φ n = Φ a 2 = Φ a, Φ a = a, ΦΦ a = ΦΦ a, a A and B are the infimum and supremum values of the spectrum of ΦΦ? A and B are the infimum and supremum values of the spectrum of Φ Φ In finite dimension, the maximum and minimum eigenvalues of ΦΦ and Φ Φ on ImΦ are identical.(imφ is the image space of all Φf)

11 Stable Analysis and Synthesis Operators Frame synthesis operator 11/39 When the frame vectors are normalized φ n measured by the frame bounds A and B. = 1, the frame redundancy is Theorem 5.2: In a space of finite dimension N, a frame of P N normalized vectors has frame bounds A and B, which satisfy For a tight frame A = B = P/N. A P N B. Proof: ΦΦ a p = Φ a, φ p = a n φ n, φ p = a n φ n, φ p ΦΦ = Since tr ΦΦ tr ΦΦ = φ, φ φ 1, φ 1 φ P, φ 1 p φ 1, φ P φ P, φ P n n = tr Φ Φ, and AN P BN A P N B n=1 N AN tr Φ Φ = λ i i=1 BN 2 = P A λ i B

12 Stable Analysis and Synthesis Operators Redundancy 12/39 Theorem 5.2: In a space of finite dimension N, a frame of P N normalized vectors has frame bounds A and B, which satisfy For a tight frame A = B = P/N. A P N B. If φ n is a normalized Riesz basis and is therefore linearly independent, then it proves that A 1 B. This result remains valid in infinite dimension. The frame is orthonormal if and only if B = 1, in which case A = 1. One can verify that a finite set of N vectors φ n 1 n N is always a frame of the space V generated by linear combinations of these vectors. When N increases, the frame bounds A and B may go respectively to 0 and +. This illustrates the fact that in infinite dimensional spaces, a family of vectors may be complete and not yield a stable signal representation.

13 Stable Analysis and Synthesis Operators Redundancy 13/39 Example 5.1: Let g 1, g 2 be an orthonormal basis of an N = 2 twodimensional plane Η. The P = 3 normalized vectors φ 1 = g 1, φ 2 = g g 2, φ 3 = g g 2. have equal angles of 2π/3 between each other. For any fεη, 3 f, φ n 2 = 3 2 f 2. n=1 Thus, these three vectors define a tight frame with A = B = 3/2.

14 Stable Analysis and Synthesis Operators Core Equations 14/39 Φf n = f, φ n Φ a = a n φ n 2 fεη, A f 2 f, φ n 2 B f 2. aεimφ, A a 2 a n φ n B a 2 fεη, A f 2 Φ Φf, f B f 2. aεimφ, A a 2 ΦΦ a, a B a 2.

15 Dual Frame and Pseudo Inverse Pseudo Inverse 15/39 The reconstruction of f from its frame coefficients Φf n is calculated with a pseudo inverse. This pseudo inverse is a bounded operator that implements a dual-frame reconstruction. For Riesz bases, this dual frame is a biorthogonal basis. Theorem 5.3: If φ n is a frame but not a Riesz basis, then Φ admits an infinite number of left inverses. Proof: ImU:the image space of all Uf and by NullU: the null space of all h, such that Uh = 0. NullΦ = (ImΦ) is the orthogonal complement of ImΦ in l 2 Γ. If Φ is a frame and not a Riesz basis, then φ n is linearly dependent a 0, Φ a = a n φ n = 0 a 0, a NullΦ = (ImΦ)

16 Dual Frame and Pseudo Inverse Pseudo Inverse Proof: 16/39 If Φf = 0 A f 2 Φf 2 = 0 (A > 0) f = 0 Φ admits a left inverse. There is an infinite inverses since the restriction of a left inverse to (ImΦ) {0} may be any arbitrary linear operator. The more redundant the frame φ n, the larger the orthogonal complement (ImΦ) of ImΦ in l 2 Γ. The pseudo inverse Φ +, is defined as the left inverse that is zero on (ImΦ) : fεη, Φ + Φf = f and aε ImΦ, Φ + a = 0. How to compute this pseudo inverse? Theorem 5.4

17 Dual Frame and Pseudo Inverse Pseudo Inverse 17/39 Theorem 5.4: Pseudo Inverse. If Φ is a frame operator, then Φ Φ is invertible and the pseudo inverse satisfies Proof: Φ + = (Φ Φ) 1 Φ a) If Φ Φf = 0 A f 2 f, φ n 2 = Φ Φf, f = 0 (A > 0) f = 0 Φ Φ is invertible. For all fεη, Φ Φ 1 Φ is a left inverse. b) (ImΦ) = NullΦ aϵ ImΦ, Φ a = 0 Φ Φ 1 Φ Φf = f aϵ ImΦ, Φ Φ 1 Φ a = 0 From a) and b), Φ Φ 1 Φ is the pseudo inverse. Pseudo inverse definition: fεη, Φ + Φf = f and aε ImΦ, Φ + a = 0.

18 Dual Frame and Pseudo Inverse Dual Frame 18/39 The pseudo inverse of a frame operator implements a reconstruction with a dual frame Theorem 5.5 Theorem 5.5: Let φ n be a frame with bounds B A > 0. The dual operator defined by, Φf n = f, φ n with φ n = (Φ Φ) 1 φ n satisfies Φ = Φ + (a) and thus f = f, φ n φ n = f, φ n φ n (b) It defines a dual frame as fεη, 1 B f 2 f, φ n 2 1 A f 2 (c) If the frame is tight (i.e. A = B), then φ n = A 1 φ n.

19 Dual Frame and Pseudo Inverse Dual Frame 19/39 Proof: (a)(b) Φf n = f, φ n = f, (Φ Φ) 1 φ n = (Φ Φ) 1 f, φ n = Φ(Φ Φ) 1 f n Φ = Φ Φ Φ 1 Φ = Φ Φ 1 Φ Φ + = (Φ Φ) 1 Φ Φ = Φ + (a) proved Φ Φ = Φ + Φ = Id Φ Φ = Id f = Φ Φf f = Φ Φf f = f, φ n φ n = f, φ n (b) proved φ n

20 Dual Frame and Pseudo Inverse Dual Frame Proof: (c) 20/39 Definition 5.1: fεη, A f 2 Φ Φf, f B f 2. fεη, B 1 f 2 (Φ Φ) 1 f, f A 1 f 2 fεη, A λ i B fεη, 1 B 1 λ i 1 A Φf 2 = Φ(Φ Φ) 1 f, Φ(Φ Φ) 1 f = f, (Φ Φ) 1 Φ Φ(Φ Φ) 1 f = f, (Φ Φ) 1 f λ i is the eigenvalues of Φ Φ 1 λ i is the eigenvalues of (Φ Φ) 1 fεη, 1 B f 2 Φf 2 = f, φ n 2 1 A f 2 (c) proved

21 Dual Frame and Pseudo Inverse Dual Frame 21/39 Dual frame vector Frame vector f = f, φ n φ n = f, φ n φ n = Φf n φ n = Φf n φ n frame coefficient Dual frame coefficient Theis theorem proves that f is reconstructed from frame coefficients Φf n = f, φ n with the dual frame φ n. The synthesis coefficients of f in φ n are the dual-frame coefficients Φf n = f, φ n. If the frame is tight (i.e. φ n = A 1 φ n ) then both decompositions are identical: f = 1 A f, φ n φ n

22 Dual Frame Biorthogonal Bases 22/39 A Riesz basis is a frame of vectors that are linearly independent, so its dual frame is also linearly independent. Theorem 5.5: f = f, φ n φ n = f, φ n 1 B f 2 f, φ n 2 1 A f 2 φ n Dual Riesz bases are biorthogonal families of vectors. f = φ p 1 B φ p φ p = φ p, φ n 2 φp, φ n 2 φ n 1 A φ p Linearly independent 2 If normalized ( φ n =1) φ p, φ n = δ[p n] A 1 B

23 Dual Frame Dual-Frame Analysis 23/39 Theorem 5.5: f = f, φ n φ n = f, φ n φ n φ n is a frame of the whole signal space and φ n its dual frame. What if φ n is a frame of a subspace V of the whole signal space and φ n its dual frame?? = f, φ n φ n = f, φ n φ n The best linear approximation of f in V orthogonal projection of f in V Theorem 5.6: Let φ n be a frame of V and φ n frame in V. The orthogonal projection of fεη in V is P V f = f, φ n φ n = f, φ n φ n its dual

24 Dual Frame Dual-Frame Analysis 24/39 Theorem 5.6: P V f = f, φ n φ n = f, φ n φ n = Φ Φf =Φ Φf Dual-frame synthesis operator Dual-frame analysis operator P V f = f, φ n φ n P V f = Φf[n]φ n = (ΦΦ ImΦ ) 1 Φf[n]φ n = f, φ n (Φ Φ V ) 1 φ n = (Φ Φ V ) 1 f, φ n φ n = (Φ Φ V ) 1 Φ Φf Φf = (ΦΦ ImΦ ) 1 Φf ΦP V f = ΦΦ Φf = Φf For sparse representation, the selection of φ n depends on the signal f, computing the dual frame φ n is inefficient.

25 25/39 Frames 2 Translation-Invariant Dyadic Wavelet Transform

26 Dyadic Wavelet Transform Translation-invariant wavelet transform 26/39 Translation-invariant wavelet dictionaries are constructed by sampling the scale parameter s while keeping all translation parameters u. ψ u,s t = 1 t u ψ s s wavelets s = 2 j Sampling s Wf u, 2 j = f, ψ u,2 j = f t D = ψ u,2 j t = 1 t u ψ 2j 2 j uεr,jεz Translation-invariant wavelet dictionaries 1 t u ψ 2j 2 j dt = f ψ 2 j u As in the case of orthogonal and biorthogonal wavelet bases, we construct a scaling function φ and the corresponding wavelet ψ with a Fourier transform: φ ω = 1 2 h ω 2 φ ω 2, ψ ω = 1 2 g ω 2 φ ω 2 Low-pass FIR filter band-pass FIR filter

27 Dyadic Wavelet Transform Vanishing moments 27/39 The number of vanishing moments of ψ Apply φ 0 = 1 = = the number of zeros of ψ ω at ω = 0. Apply ψ ω = 1 2 g ω 2 φ ω 2 the number of zeros of g ω at ω = 0.

28 Dyadic Wavelet Transform Reconstructing Wavelets 28/39 φ ω = 1 2 h ω 2 φ ω 2 ψ ω = 1 2 g ω 2 φ ω 2 dual dual φ ω = 1 2 h ω 2 φ ω 2 ψ ω = 1 2 g ω 2 φ ω 2 Reconstructing wavelets are calculated with a pair of finite impulse response dual filters h and g. What s the condition to guarantee that ψ is a reconstruction wavelet? Theorem 5.13: (Reconstruction condition) If the filters satisfy ωε π, π, h ω h ω + g ω g ω = 2, then ψ is a reconstruction wavelet

29 Dyadic Wavelet Transform Spline Dyadic Wavelets 29/39 A box spline of degree m: φ ω = sin(ω/2) ω/2 m+1 exp iεω 2 with ε = 1 if m is even 0 if m is odd choosing g ω = O(ω): g ω = i 2 sin ω 2 exp iεω 2 The number of vanishing moments of ψ = The number of zeros of g ω at ω = 0 ψ ω = 1 2 g ω 2 φ ω 2 one vanishing moment wavelet: ψ ω = iω 4 sin(ω/4) ω/4 m+2 exp iω(ε + 1 4

30 Dyadic Wavelet Transform Spline Dyadic Wavelets 30/39 ψ ω = iω 4 sin(ω/4) ω/4 m+2 exp iω(ε + 1) 4, φ ω = sin(ω/2) ω/2 m+1 exp iεω 2 Set m = 2, ε=1 and take inverse Fourier transform: ψ(t) φ(t)

31 Dyadic Wavelet Transform Fast Dyadic Transform 31/39 a j n = f φ 2 j n = f t, φ 2 j t n with φ 2 j t = 1 2 j φ t 2 j a j+1 n = f φ 2 j+1 + a j+1 (ω) = f k= + n (ω + 2kπ)φ 2 j+1 ω + 2kπ = f (ω + 2kπ) 2 j+1 φ 2 j+1 (ω + 2kπ) k= + = f (ω + 2kπ) 2 j h 2 j (ω + 2kπ) φ 2 j (ω + 2kπ) k= + = h 2 j ω f (ω + 2kπ) 2 j φ 2 j (ω + 2kπ) k= + = h 2 j ω f k= (ω + 2kπ)φ 2 j a j (ω) ω + 2kπ Apply φ ω = 1 2 h ω 2 φ ω 2 and let ω = 2 j+1 (ω + 2kπ) a j+1 (ω) = h 2 j ω a j (ω) Similarly, we can also get d j+1 (ω) = g 2 j ω a j (ω)

32 Dyadic Wavelet Transform Fast Dyadic Transform 32/39 a j+1 (ω) = h 2 j ω a j (ω), d j+1 (ω) = g 2 j ω a j (ω) a j+1 ω h 2 j ω + d j+1 ω g 2 j ω = a j ω h 2 j ω h 2 j ω + a j ω g 2 j ω g 2 j ω = a j ω h 2 j ω h 2 j ω + g 2 j ω g 2 j ω = 2a j ω Theorem 5.13: (Reconstruction condition) ωε π, π, h ω h ω + g ω g ω = 2 a j (ω) = 1 2 a j+1 ω h 2 j ω + d j+1 ω g 2 j ω

33 Dyadic Wavelet Transform Fast Dyadic Transform 33/39 a j+1 (ω) = a j (ω)h 2 j ω, d j+1 (ω) = a j (ω)g 2 j ω a j (ω) = 1 2 a j+1 ω h 2 j ω + d j+1 ω g 2 j ω Inverse Fourier Transform Theorem 5.14: for any j 0, a j+1 n = a j h j [n], d j+1 n = a j g j n, and a j n = 1 2 a j+1h j n + d j+1 g j [n] Filter h j [n] is obtained by inserting 2 j 1 zeros between each sample of h[n], its Fourier transform is h(2 j ω).

34 Dyadic Wavelet Transform Fast Dyadic Transform 34/39 a j+1 n = a j h j [n], d j+1 n = a j g j n a j n = 1 2 a j+1h j n + d j+1 g j [n]

35 Dual Frame Fast Dyadic Transform 35/39 The dyadic wavelet representation of a j is defined as the set of wavelet coefficients up to a scale 2 J plus the remaining low-frequency information a J : [{d j } 1 j J, a J ] Complexity : If the input signal a 0 [n] has N samples, The maximum scale 2 J =N, and J = log 2 N. Suppose that h and g have, respectively, K h and K g nonzero samples, h j and g j have the same nonzero coefficients. The calculation complexity is K h + K g N log 2 N. N samples K h nonzero samples K g nonzero samples J = log 2 N iterations

36 36/39 Frames 3 Subsampled Wavelet Frames

37 Subsampled Wavelet Frames Fast Dyadic Transform 37/39 ψ u,s t = 1 s wavelets ψ t u s s = 2 j D = ψ u,2 j t = 1 t u ψ 2j 2 j uεr,jεz Translation-invariant dyadic wavelet dictionaries ψ u,s t = 1 s wavelets ψ t u s s = a j u = nu 0 a j Intuitively, to construct a frame we need to cover the time-frequency plane with the Heisenberg boxes of corresponding discrete wavelet family. What s the conditions on ψ, a and u 0 so that ψ j,n t is a frame of L 2 (R)? (j,n)εz 2 D = ψ j,n t = 1 a j ψ t nu 0a j Wavelet Frames a j (j,n)εz 2

38 Subsampled Wavelet Frames Necessary Conditions 38/39 We suppose that ψ is real, normalized, and satisfies the admissibility condition: + ψ(ω) 2 C ψ = dω < + ω Theorem 5.15: Necessary Condition. If ψ j,n t L 2 (R), then the frame bounds satisfy A C ψ u 0 ln a B, 0 ωεr 0, A 1 ψ(a j ω) 2 B u 0 j= (j,n)εz 2 is a frame of

39 39 Q & A Many Thanks

Wavelets and multiresolution representations. Time meets frequency

Wavelets and multiresolution representations Time meets frequency Time-Frequency resolution Depends on the time-frequency spread of the wavelet atoms Assuming that ψ is centred in t=0 Signal domain + t