Wavelets, Filter Banks and Multiresolution Signal Processing It is with logic that one proves; it is with intuition that one invents. Henri Poincaré Introduction - 1
A bit of history: from Fourier to Haar to wavelets Old topic: representations of functions 1807: Joseph Fourier upsets the French Academy = + + 1898: Gibbs paper 1899: Gibbs correction Introduction - 2
1910: Alfred Haar discovers the Haar wavelet dual to the Fourier construction = + + + Why do this? What makes it work? basic atoms form an orthonormal set Note sines/cosines and Haar functions are ON bases for both are structured orthonormal bases they have different time and frequency behavior L 2 ( R) Introduction - 3
1930: Heisenberg discovers that you cannot have your cake and eat it too! ft () ft () perfectly local in time t F( ω) F( ω) w t perfectly local in frequency w ft () F( ω) t trade-off w Uncertainty principle lower bound on TF product w t Introduction - 4
1945: Gabor localizes the Fourier transform STFT 1980: Morlet proposes the continuous wavelet transform frequency frequency time time short-time Fourier transform wavelet transform Introduction - 5
Analogy with the musical score Bach knew about wavelets! Introduction - 6
Time-frequency tiling for a sine + Delta t 0 T t f f FT f t t 0 T t 0 T f t STFT WT t t 0 T t 0 T t so... what is a good basis? Introduction - 7
1983: Lena discovers pyramids (actually, Burt and Adelson) coarse D I I x - + residual + x MR encoder MR decoder Introduction - 8
1984: Lena gets critical (subband coding) H 1 2 HH H 1 H L 2 H 0 2 HL x H 1 2 LH H Η 2 H 0 horizontal H 0 2 LL π f 2 vertical π π f 1 LL LH HL HH π Introduction - 9
1986: Lena gets formal... (multiresolution theory by Mallat, Meyer...) = + Introduction - 10
Wavelets, filter banks and multiresolution analysis Wavelets (applied mathematics) Filter banks (DSP) Construction of bases for signal expansions x = ψ i, x ψ i Multiresolution signal analysis (computer vision) = + Introduction - 11
Wavelets... All this time, the guard was looking at her, first through a telescope, then through a microscope, and then through an opera glass. Lewis Carroll, Through the Looking Glass Introduction - 12
... what are they and how to build them? Orthonormal bases of wavelets Haar s construction of a basis for L 2 ( R) (1910) Meyer, Battle-Lemarié, Stromberg (1980 s) Mallat and Meyer s multiresolution analysis (1986) Wavelets from iterated filter banks Daubechies construction of compactly supported wavelets smooth wavelet bases for L 2 ( R) and computational algorithms Relation to other constructions successive refinements in graphics and interpolation multiresolution in computer vision multigrid methods in numerical analysis subband coding in speech and image processing Goal: find ψ(t) such that its scales and shifts form an orthonormal basis for L 2 ( R). Introduction - 13
Why expand signals? Suppose = + = original coarse detail = + + signal block 1 block 2 signal = Σ projection x elementary signals Advantages easier to analyze signal in pieces: divide and conquer extracts important features pieces can be treated in an independent manner Introduction - 14
Example: Example: R 2 orthogonal basis biorthogonal basis tight frame e e 1 1 = ϕ ~ 1 ϕ 1 ϕ 0 ϕ 1 e 1 e 0 e 0 = ϕ 0 e 0 = ϕ 0 ϕ 1 ~ ϕ 0 ϕ 2 Note orthonormal basis has successive approximation property, biorthogonal basis and frames do not quantization in orthogonal case is easy, unlike in the other cases Introduction - 15
Why not use Fourier? Block Fourier transform: bad frequency localization Gabor transform: ill-behaved for critical sampling Balian-Low theorem: there is no local Fourier basis with good time and frequency localization however: good local cosine bases! t 1/ω ω t 1/ω α, α > 1 shift and modulation ω Introduction - 16
How do filter banks expand signals? analysis H 1 2 y 1 2 synthesis G 1 x 1 x + x^ H 0 2 G 0 2 y 0 x 0 Analogy beam of white light Introduction - 17
...and multiresolution analysis? IDEA: successive approximation/refinement of the signal original = coarse 2 + coarse 1 detail 2 + detail 1 detail 1 + = = Introduction - 18
mother wavelet ψ... how about wavelets? Who? families of functions obtained from mother wavelet by dilation and translation Why? well localized in time and frequency it has the ability to zoom Introduction - 19
Haar system Basis functions ψ() t = 1 0 t < 0.5 1 0.5 t < 1 0 else ψ mn, () t = 2 m/2 ψ( 2 m t n) Basis functions across scales m = 1 t m = 0 t m = -1 t Introduction - 20
Haar system...... as a basis for L 2 ( R) f (0) -8-7 -6-5 -4-3 -2-1 0 1 2 3 4 5 f (1) = 6 7 8 t geometric proof -8-7 -6-5 -4-3 -1 0 1 2 3 4 5 7 8 t + -8-5 -4-6 0 d (1) 2 4 8 t f (2) = -8-7 -6-5 -4-3 -2-1 0 1 2 3 4 5 6 7 = 8 t + -8-7 -5-3 d (2) -1 1 3 4 5 7 8 t Introduction - 21
Haar system...... scaling function and wavelet The Haar scaling function (indicator of unit interval) ϕ() t = 1 0 t < 1 0 else helps in the construction of the wavelet, since ψ() t = ϕ2t ( ) ϕ( 2t 1) and satisfies a two-scale equation 1 1 ϕ(t) 1 ϕ(t) 1 = 1 1 1 ϕ(2t) ϕ(2t 1) 1 ϕ() t = ϕ2t ( ) + ϕ( 2t 1) Note: Haar wavelet a bit too trivial to be useful... 1 ϕ(t) 1 = 1 ϕ(2t) + ϕ(2t-1) 1 Introduction - 22
Discrete version of the wavelet transform Compute WT on a discrete grid scale m = -1 shift m = 0 m = 1 Introduction - 23
Perfect reconstruction filter banks analysis H 1 2 y 1 2 synthesis G 1 x 1 x + x^ H 0 2 G 0 2 y 0 x 0 magnitude responses of analysis filters H 0 H 1 π/2 π Perfect reconstruction: Orthogonal system: G 0 H 0 + G 1 H 1 = I ( H 0 ) * H 0 + ( H 1 ) * H 1 = I G 0 = ( H 1 ) * Introduction - 24
Daubechies construction...... iterated filter banks Iteration will generate an orthonormal basis for the space of square-summable sequences l 2 ( I) 2 2 G 1 G 0 + () i i Consider equivalent basis sequences G 0 ( z) and G1 (generates octave-band frequency analysis) 2 2 G 1 G 0 + 2 2 G 1 G 0 + () z ( ) π/8 π/4 π/2 π Interesting question: what happens in the limit? Introduction - 25
Daubechies construction...... iteration algorithm At ith step associate piecewise constant approximation of length () i with g 0 [ n] 1/2 i i = 1 0.8 0.8 i = 2 0.6 0.6 Amplitude 0.4 0.2 Amplitude 0.4 0.2 0 0 0 1 2 Time -0.2 0 1 2 Time i = 3 0.8 0.8 i = 4 0.6 0.6 Amplitude 0.4 0.2 Amplitude 0.4 0.2 0 0-0.2-0.2 0 1 2 Time 0 1 2 Time Fundamental link between discrete and continuous time! Introduction - 26
Daubechies construction......scaling function and wavelet Haar and sync systems: either good time OR frequency localization Daubechies system: good time AND frequency localization scaling function Amplitude 1.25 1 0.75 0.5 0.25 Magnitude response 1 0.8 0.6 0.4 0 0.2-0.25 0.5 1.0 1.5 2.0 2.5 3.0 Time 0 8.0 16.0 24.0 32.0 40.0 48.0 Frequency [radians] wavelet 1.5 1 1 Amplitude 0.5 0-0.5 Magnitude response 0.8 0.6 0.4-1 0.2 0.5 1.0 1.5 2.0 2.5 3.0 Time 0 8.0 16.0 24.0 32.0 40.0 48.0 Frequency [radians] Finite length, continuous ϕ() t and ψt (), based on L=4 iterated filter Many other constructions: biorthogonal, IIR, multidimensional... Introduction - 27
Hat function Daubechies construction...... two-scale equation ϕ() t = c n ϕ( 2t n) n = Daubechies scaling function 1.25 1.5 Amplitude 1 0.75 0.5 0.25 0 = Amplitude 1 0.5 0-0.25 0.5 1.0 1.5 2.0 2.5 3.0 Time 0.5 1.0 1.5 2.0 2.5 3.0 Time Introduction - 28
Not every discrete scheme leads to wavelets 33 33 0 1 1 0 biorthogonal filter banks 0 0-1 -1 quadratic spline How do we know which ones will?... wait and see... Introduction - 29
Applications That which shrinks must first expand. Compression Communications Denoising Graphics Lao-Tzu, Tao Te Ching Introduction - 30
What is multiresolution? High resolution subtract info Low resolution = + add info Introduction - 31
... and why use multiresolution? A number of applications require signals to be processed and transmitted at multiple resolutions and multiple rates digital audio and video coding conversions between TV standards digital HDTV and audio broadcast remote image databases with searching storage media with random access MR coding for multicast over the Internet MR graphics Compression: still a key technique in communications Introduction - 32
Multiresolution compression...... the DCT versus wavelet game Question given Lena (you have never seen before), what is the best transform to code it? Fourier versus wavelet bases linear versus octave-band frequency scale DCT versus subband coding JPEG versus multiresolution Multiresolution source coding successive approximation browsing progressive transmission Introduction - 33
Compression systems based on linear transforms Goal: remove built-in redundancy, send only necessary info LT Q EC 0100101001 8 bits/pixel 0.5 bits/pixel LT: linear transform (KLT, WT, SBC, DCT, STFT) Q: quantization EC: entropy coding Introduction - 34
Gibbs phenomenon Blocking effect in image compression Wavelets smooth transitions multiscale properties multiresolution Introduction - 35
A rate-distortion primer... Compression: rate-distortion is fundamental trade-off more bitrate less distortion less bitrate more distortion Standard image coder operates at one particular point on D(R) curve Multiresolution coder (layered, scalable) travels rate-distortion curve (successive approximation) computation scalability distortion x x x x D(R) D SA (R) x x x rate Introduction - 36
Best image coder?... wavelet based! Shapiro s embedded zero-tree algorithm (EZW) DWT bit plane coding zero trees adaptive entorpy coding standard wavelet decomposition (biorthogonal) bit plane coding and zero-tree structure beats JPEG while achieving successive approximation distortion x x x x EZW JPEG x x x rate Introduction - 37
Next image coding standard... JPEG 2000 All the best coders based on wavelets 24 full proposals and a few partial ones 18 used wavelets, 4 used DCT and 5 used others top 75% are wavelet-based top 5 use advanced wavelet oriented quantization systems requirements ask for multiresolution Final JPEG 2000 standard is wavelet based Introduction - 38
Digital video coding MR processing block variety of scales and resolutions signal decomposition for compression compatible subchannels tight control over coding error easy joint source/channel coding robustness to channel errors easy random access for digital storage Introduction - 39
Conversion between TV formats 60Hz USA HDTV/NTSC interlaced/progressive Europe 50Hz Introduction - 40
Interaction of source and channel coding high priority high protection MR coder low priority little protection full reconstruction coarse reconstruction Introduction - 41
MR transmission for digital broadcast Embedding of coarse information within detail b i cloud with satellites a i cloud: carries coarse info satellite: carries detail blend MR transmission with MR coding Trade-off in broadcast ranges [miles] SR IR MR: λ = 0.5, 0.2 28 14 45 24 40 16 63 high/low resolution Introduction - 42
MR coding for multicast over the Internet I want to say a special welcome to everyone that s climbed into the Internet tonight, and has got into the MBone --- and I hope it doesn t all collapse! Mick Jagger, Rollings Stones on Internet,11/18/94 Motivation: Internet is a heterogeneous mess! Video multicast over Mbone video by VIC software encoder/decoder learning experience (seminars...) LAN 64 64 LAN 64 Heterogeneous user population On-going experience ATM 1M 1M 1M Introduction - 43
MR coding for multicast over the Internet Fact: different users receive different bit rates transmission heterogeneity Different users absorb different bit rates computation heterogeneity Solution: layered multicast trees different layers are transmitted over independent trees automatic subscribe/unsubscribe dynamic quality management Q 1 Q 2 Q 3 D S Q 1 Q 2 Q 3 Q 2 Q 3 Q 2 Q 2 R Introduction - 44
Remote image databases with browsing Introduction - 45
Multiresolution graphics Example: optimize quality (distortion) for a target rate distortion rate Introduction - 46
Skull page not reproduced due to copyright restrictions Introduction - 47