Multiresolution Analysis

Transcription

1 Multiresolution Analysis DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis Carlos Fernandez-Granda

2 Frames Short-time Fourier transform (STFT) Wavelets Thresholding

3 Definition Let V be an inner-product space A frame of V is a set of vectors F := { v 1, v 2,...} such that for every x V c L x 2, v F x, v 2 c U x 2, for fixed positive constants c U c L 0 The frame is a tight frame if c L = c U

4 Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...)

5 Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...) Then P span( v1, v 2 y,...) is nonzero and P span( v1, v 2,...) y, v 2 = v F

6 Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...) Then P span( v1, v 2 y,...) is nonzero and P span( v1, v 2,...) y, v 2 = 0 v F

7 Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof:

8 Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2,

9 Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2,

10 Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2, = x, 2 b 2 b b B,

11 Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2, = x, 2 b 2 b b B = x, 2 b b B,

12 Analysis operator The analysis operator Φ of a frame maps a vector to its coefficients Φ ( x) [k] = x, v k For any finite frame { v 1, v 2,..., v m } of C n the analysis operator is v 1 F := v 2... v m

13 Frames in finite-dimensional spaces v 1, v 2,..., v m are a frame of C n if and only F is full rank In that case, c U = σ 2 1 c L = σ 2 n Proof: m σn 2 F x 2 2 = x, v j 2 σ1 2 j=1

14 Pseudoinverse If an n m tall matrix A, m n, is full rank, then its pseudoinverse A := (A A) 1 A is well defined, is a left inverse of A A A = I and equals A = VS 1 U where A = USV is the SVD of A

15 Proof A := (A A) 1 A

16 Proof A := (A A) 1 A = (VSU USV A) 1 VSU

17 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU

18 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU

19 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U

20 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A

21 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A = VS 1 UV USV

22 Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A = VS 1 UV USV = I

23 Frames Short-time Fourier transform (STFT) Wavelets Thresholding

24 Motivation Spectrum of speech, music, etc. varies over time Idea: Compute frequency representation of time segments of the signal

25 Short-time Fourier transform The short-time Fourier transform (STFT) of a function f L 2 [ 1/2, 1/2] is STFT {f } (k, τ) := 1/2 1/2 where w L 2 [ 1/2, 1/2] is a window function Frame vectors: v k,τ (t) := w (t τ) e i2πkt f (t) w (t τ)e i2πkt dt

26 Discrete short-time Fourier transform The STFT of a vector x C n is STFT {f } (k, l) := x w [l], h k where w C n is a window vector Frame vectors: v k,l (t) := w [l] h k

27 STFT Length of window and shifts are chosen so that shifted windows overlap In that case the STFT is a frame We can invert it using fast algorithms based on the FFT Window should not produce spurious high-frequency artifacts

28 Rectangular window Signal Window = Spectrum =

29 Hann window Signal Window = Spectrum =

30 Frame vector l = 0, k = 0 Real part Imaginary part Spectrum

31 Frame vector l = 1/32, k = 0 Real part Imaginary part Spectrum

32 Frame vector l = 0, k = 64 Real part Imaginary part Spectrum

33 Frame vector l = 1/32, k = 64 Real part Imaginary part Spectrum

34 Speech signal

35 Spectrum

36 Spectrogram (log magnitude of STFT coefficients) Frequency Time

37 Frames Short-time Fourier transform (STFT) Wavelets Thresholding

38 Wavelet transform Motivation: Extracting features at different scales Idea: Frame vectors are scaled, shifted copies of a fixed function An additional function captures low-pass component at largest scale

39 Wavelet transform The wavelet transform of a function f L 2 [ 1/2, 1/2] depends on a choice of scaling function (or father wavelet) φ and wavelet function (or mother wavelet) ψ The scaling coefficients are W φ {f } (τ) := 1 s f (t) φ (t τ) dt The wavelet coefficients are W ψ {f } (s, τ) := 1 1 ( t τ f (t) ψ s s Wavelets can be designed to be bases or frames 0 ) dt

40 Haar wavelet Scaling function Mother wavelet Wavelets are band-pass filters, scaling functions are low-pass filters

41 Discrete wavelet transform The discrete wavelet transform depends on a choice of scaling vector φ and wavelet ψ The scaling coefficients are W φ {f } (l) := x, φ [l] The wavelet coefficients are W ψ {f } (s, l) := x, ψ [s,l], where [ ] ψ [s,l] [j] := ψ j l s Wavelets can be designed to be bases or frames

42 Orthonormal wavelet basis Scale Basis functions 2 0

43 Orthonormal wavelet basis Scale Basis functions 2 0

44 Orthonormal wavelet basis Scale Basis functions 2 0

45 Orthonormal wavelet basis Scale Basis functions

46 Orthonormal wavelet basis Scale Basis functions

47 Orthonormal wavelet basis Scale Basis functions

48 Orthonormal wavelet basis Scale Basis functions

49 Orthonormal wavelet basis Scale Basis functions

50 Orthonormal wavelet basis Scale Basis functions

51 Orthonormal wavelet basis Scale Basis functions (scaling vector)

52 Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ

53 Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ V K is the span of φ

54 Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ V K is the span of φ V K

55 Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1

56 Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0

57 Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0 W 2

58 Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0 W 2 P Vk x is an approximation of x at scale 2 k

59 Multiresolution decomposition Properties V 0 = C n (approximation at scale 2 0 is perfect) V k is invariant to translations of scale 2 k Dilating vectors in V j by 2 yields vectors in V j+1

60 Electrocardiogram Signal Haar transform

61 Scale 2 9 P W9 x P V9 x

62 Scale 2 8 P W8 x P V8 x

63 Scale 2 7 P W7 x P V7 x

64 Scale 2 6 P W6 x P V6 x

65 Scale 2 5 P W5 x P V5 x

66 Scale 2 4 P W4 x P V4 x

67 Scale 2 3 P W3 x P V3 x

68 Scale 2 2 P W2 x P V2 x

69 Scale 2 1 P W1 x P V1 x

70 Scale 2 0 P W0 x P V0 x

71 2D Wavelets Extension to 2D by using outer products of 1D atoms ( ) T ξs 2D 1,s 2,k 1,k 2 := ξs 1D 1,k 1 ξs 1D 2,k 2 The JPEG 2000 compression standard is based on 2D wavelets Many extensions: Steerable pyramid, ridgelets, curvelets, bandlets,...

72 2D Haar transform

73 2D wavelet transform

74 2D wavelet transform

75 Frames Short-time Fourier transform (STFT) Wavelets Thresholding

76 Denoising Aim: Extracting information (signal) from data in the presence of uninformative perturbations (noise) Additive noise model data = signal + noise y = x + z Prior knowledge about structure of signal vs structure of noise is required

77 Assumption Signal is a sparse superposition of basis/frame vectors Noise is not

78 Assumption Signal is a sparse superposition of basis/frame vectors Noise is not Example: Gaussian noise z with covariance matrix σ 2 I, distribution of F z?

79 Example Data Signal

80 Thresholding Hard-thresholding operator H η ( v) [j] := { v [j] if v [j] > η 0 otherwise

81 Denoising via hard thresholding Estimate Signal

82 Multisinusoidal signal y F y Data Signal Data

83 Denoising via hard thresholding Data: y = x + z Assumption: F x is sparse, F z is not 1. Apply the hard-thresholding operator H η to F y 2. If F is a basis, then x est := F 1 H η (F y) If F is a frame, x est := F H η (F y), where F is the pseudoinverse of F (other left inverses of F also work)

84 Denoising via hard thresholding in Fourier basis y F y Data Signal Data

85 Denoising via hard thresholding in Fourier basis F 1 H η (F y) H η (F y) Estimate Signal Estimate

86 Image denoising x F x

87 Image denoising ~z F ~z

88 Data (SNR=2.5) y F y

89 F y

90 H η (F y)

91 F 1 H η (F y)

92 y

93 Data (SNR=1) ~y F ~y

94 F y

95 H η (F y)

96 F 1 H η (F y)

97 y

98 Image denoising y F 1 H η (F y) x

99 Denoising via thresholding y F 1 H η (F y) x

100 Speech denoising

101 Time thresholding

102 Spectrum

103 Frequency thresholding

104 Frequency thresholding Data DFT thresholding

105 Spectrogram (STFT) Frequency Time

106 STFT thresholding Frequency Time

107 STFT thresholding Data STFT thresholding

108 Coefficients are structured Frequency Time

109 Coefficients are structured x F x

110 Block thresholding Assumption: Coefficients are group sparse, nonzero coefficients cluster together Partition coefficients into blocks I 1, I 2,..., I k and threshold whole blocks { v [j] if j Ij such that v Ij 2 > η,, B η ( v) [j] := 0 otherwise,

111 Denoising via block thresholding 1. Apply the hard-thresholding operator B η to F y 2. If F is a basis, then x est := F 1 B η (F y) If F is a frame, x est := F B η (F y), where F is the pseudoinverse of F (other left inverses of F also work)

112 Image denoising (SNR=2.5) y F y

113 F y

114 H η (F y)

115 B η (F y)

116 F 1 H η (F y)

117 F 1 B η (F y)

118 Image denoising (SNR=1) ~y F ~y

119 F y

120 H η (F y)

121 B η (F y)

122 F 1 H η (F y)

123 F 1 B η (F y)

124 Denoising via thresholding y F 1 H η (F y) x

125 Denoising via thresholding y F 1 B η (F y) x

126 Denoising via thresholding y F 1 H η (F y) x

127 Denoising via thresholding y F 1 B η (F y) x

128 Speech denoising

129 Spectrogram (STFT) Frequency Time

130 STFT thresholding Frequency Time

131 STFT thresholding Data STFT thresholding

132 STFT block thresholding Frequency Time

133 STFT block thresholding Data STFT block thresh.