Multiresolution Analysis DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda
Frames Short-time Fourier transform (STFT) Wavelets Thresholding
Definition Let V be an inner-product space A frame of V is a set of vectors F := { v 1, v 2,...} such that for every x V c L x 2, v F x, v 2 c U x 2, for fixed positive constants c U c L 0 The frame is a tight frame if c L = c U
Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...)
Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...) Then P span( v1, v 2 y,...) is nonzero and P span( v1, v 2,...) y, v 2 = v F
Frames span the whole space Any frame F := { v 1, v 2,...} of V spans V Proof: Assume y / span ( v 1, v 2,...) Then P span( v1, v 2 y,...) is nonzero and P span( v1, v 2,...) y, v 2 = 0 v F
Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof:
Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2,
Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2,
Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2, = x, 2 b 2 b b B,
Orthogonal bases are tight frames { Any orthonormal basis B := b1, } b 2,... is a tight frame Proof: For any vector x V x 2, = x, b b b B 2, = x, 2 b 2 b b B = x, 2 b b B,
Analysis operator The analysis operator Φ of a frame maps a vector to its coefficients Φ ( x) [k] = x, v k For any finite frame { v 1, v 2,..., v m } of C n the analysis operator is v 1 F := v 2... v m
Frames in finite-dimensional spaces v 1, v 2,..., v m are a frame of C n if and only F is full rank In that case, c U = σ 2 1 c L = σ 2 n Proof: m σn 2 F x 2 2 = x, v j 2 σ1 2 j=1
Pseudoinverse If an n m tall matrix A, m n, is full rank, then its pseudoinverse A := (A A) 1 A is well defined, is a left inverse of A A A = I and equals A = VS 1 U where A = USV is the SVD of A
Proof A := (A A) 1 A
Proof A := (A A) 1 A = (VSU USV A) 1 VSU
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A = VS 1 UV USV
Proof A := (A A) 1 A = (VSU USV A) 1 VSU = ( VS 2 V ) 1 VSU = VS 2 V VSU = VS 1 U A A = VS 1 UV USV = I
Frames Short-time Fourier transform (STFT) Wavelets Thresholding
Motivation Spectrum of speech, music, etc. varies over time Idea: Compute frequency representation of time segments of the signal
Short-time Fourier transform The short-time Fourier transform (STFT) of a function f L 2 [ 1/2, 1/2] is STFT {f } (k, τ) := 1/2 1/2 where w L 2 [ 1/2, 1/2] is a window function Frame vectors: v k,τ (t) := w (t τ) e i2πkt f (t) w (t τ)e i2πkt dt
Discrete short-time Fourier transform The STFT of a vector x C n is STFT {f } (k, l) := x w [l], h k where w C n is a window vector Frame vectors: v k,l (t) := w [l] h k
STFT Length of window and shifts are chosen so that shifted windows overlap In that case the STFT is a frame We can invert it using fast algorithms based on the FFT Window should not produce spurious high-frequency artifacts
Rectangular window Signal Window = Spectrum =
Hann window Signal Window = Spectrum =
Frame vector l = 0, k = 0 Real part Imaginary part Spectrum
Frame vector l = 1/32, k = 0 Real part Imaginary part Spectrum
Frame vector l = 0, k = 64 Real part Imaginary part Spectrum
Frame vector l = 1/32, k = 64 Real part Imaginary part Spectrum
Speech signal
Spectrum
Spectrogram (log magnitude of STFT coefficients) Frequency Time
Frames Short-time Fourier transform (STFT) Wavelets Thresholding
Wavelet transform Motivation: Extracting features at different scales Idea: Frame vectors are scaled, shifted copies of a fixed function An additional function captures low-pass component at largest scale
Wavelet transform The wavelet transform of a function f L 2 [ 1/2, 1/2] depends on a choice of scaling function (or father wavelet) φ and wavelet function (or mother wavelet) ψ The scaling coefficients are W φ {f } (τ) := 1 s f (t) φ (t τ) dt The wavelet coefficients are W ψ {f } (s, τ) := 1 1 ( t τ f (t) ψ s s Wavelets can be designed to be bases or frames 0 ) dt
Haar wavelet Scaling function Mother wavelet Wavelets are band-pass filters, scaling functions are low-pass filters
Discrete wavelet transform The discrete wavelet transform depends on a choice of scaling vector φ and wavelet ψ The scaling coefficients are W φ {f } (l) := x, φ [l] The wavelet coefficients are W ψ {f } (s, l) := x, ψ [s,l], where [ ] ψ [s,l] [j] := ψ j l s Wavelets can be designed to be bases or frames
Orthonormal wavelet basis Scale Basis functions 2 0
Orthonormal wavelet basis Scale Basis functions 2 0
Orthonormal wavelet basis Scale Basis functions 2 0
Orthonormal wavelet basis Scale Basis functions 2 0 2 1
Orthonormal wavelet basis Scale Basis functions 2 0 2 1
Orthonormal wavelet basis Scale Basis functions 2 0 2 1
Orthonormal wavelet basis Scale Basis functions 2 0 2 1 2 2
Orthonormal wavelet basis Scale Basis functions 2 0 2 1 2 2
Orthonormal wavelet basis Scale Basis functions 2 0 2 1 2 2 2 3
Orthonormal wavelet basis Scale Basis functions 2 0 2 1 2 2 2 3 (scaling vector)
Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ
Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ V K is the span of φ
Multiresolution decomposition Sequence of subspaces V 0, V 1,..., V K representing different scales Fix a scaling vector φ and a wavelet ψ V K is the span of φ V K
Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1
Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0
Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0 W 2
Multiresolution decomposition V k := W k V k+1 W k is the span of ψ dilated by 2 k and shifted by multiples of 2 k+1 W 0 W 2 P Vk x is an approximation of x at scale 2 k
Multiresolution decomposition Properties V 0 = C n (approximation at scale 2 0 is perfect) V k is invariant to translations of scale 2 k Dilating vectors in V j by 2 yields vectors in V j+1
Electrocardiogram Signal Haar transform
Scale 2 9 P W9 x P V9 x
Scale 2 8 P W8 x P V8 x
Scale 2 7 P W7 x P V7 x
Scale 2 6 P W6 x P V6 x
Scale 2 5 P W5 x P V5 x
Scale 2 4 P W4 x P V4 x
Scale 2 3 P W3 x P V3 x
Scale 2 2 P W2 x P V2 x
Scale 2 1 P W1 x P V1 x
Scale 2 0 P W0 x P V0 x
2D Wavelets Extension to 2D by using outer products of 1D atoms ( ) T ξs 2D 1,s 2,k 1,k 2 := ξs 1D 1,k 1 ξs 1D 2,k 2 The JPEG 2000 compression standard is based on 2D wavelets Many extensions: Steerable pyramid, ridgelets, curvelets, bandlets,...
2D Haar transform
2D wavelet transform
2D wavelet transform
Frames Short-time Fourier transform (STFT) Wavelets Thresholding
Denoising Aim: Extracting information (signal) from data in the presence of uninformative perturbations (noise) Additive noise model data = signal + noise y = x + z Prior knowledge about structure of signal vs structure of noise is required
Assumption Signal is a sparse superposition of basis/frame vectors Noise is not
Assumption Signal is a sparse superposition of basis/frame vectors Noise is not Example: Gaussian noise z with covariance matrix σ 2 I, distribution of F z?
Example Data Signal
Thresholding Hard-thresholding operator H η ( v) [j] := { v [j] if v [j] > η 0 otherwise
Denoising via hard thresholding Estimate Signal
Multisinusoidal signal y F y Data Signal Data
Denoising via hard thresholding Data: y = x + z Assumption: F x is sparse, F z is not 1. Apply the hard-thresholding operator H η to F y 2. If F is a basis, then x est := F 1 H η (F y) If F is a frame, x est := F H η (F y), where F is the pseudoinverse of F (other left inverses of F also work)
Denoising via hard thresholding in Fourier basis y F y Data Signal Data
Denoising via hard thresholding in Fourier basis F 1 H η (F y) H η (F y) Estimate Signal Estimate
Image denoising x F x
Image denoising ~z F ~z
Data (SNR=2.5) y F y
F y
H η (F y)
F 1 H η (F y)
y
Data (SNR=1) ~y F ~y
F y
H η (F y)
F 1 H η (F y)
y
Image denoising y F 1 H η (F y) x
Denoising via thresholding y F 1 H η (F y) x
Speech denoising
Time thresholding
Spectrum
Frequency thresholding
Frequency thresholding Data DFT thresholding
Spectrogram (STFT) Frequency Time
STFT thresholding Frequency Time
STFT thresholding Data STFT thresholding
Coefficients are structured Frequency Time
Coefficients are structured x F x
Block thresholding Assumption: Coefficients are group sparse, nonzero coefficients cluster together Partition coefficients into blocks I 1, I 2,..., I k and threshold whole blocks { v [j] if j Ij such that v Ij 2 > η,, B η ( v) [j] := 0 otherwise,
Denoising via block thresholding 1. Apply the hard-thresholding operator B η to F y 2. If F is a basis, then x est := F 1 B η (F y) If F is a frame, x est := F B η (F y), where F is the pseudoinverse of F (other left inverses of F also work)
Image denoising (SNR=2.5) y F y
F y
H η (F y)
B η (F y)
F 1 H η (F y)
F 1 B η (F y)
Image denoising (SNR=1) ~y F ~y
F y
H η (F y)
B η (F y)
F 1 H η (F y)
F 1 B η (F y)
Denoising via thresholding y F 1 H η (F y) x
Denoising via thresholding y F 1 B η (F y) x
Denoising via thresholding y F 1 H η (F y) x
Denoising via thresholding y F 1 B η (F y) x
Speech denoising
Spectrogram (STFT) Frequency Time
STFT thresholding Frequency Time
STFT thresholding Data STFT thresholding
STFT block thresholding Frequency Time
STFT block thresholding Data STFT block thresh.