Directionlets. Anisotropic Multi-directional Representation of Images with Separable Filtering. Vladan Velisavljević Deutsche Telekom, Laboratories

Transcription

1 Directionlets Anisotropic Multi-directional Representation of Images with Separable Filtering Vladan Velisavljević Deutsche Telekom, Laboratories Google Inc. Mountain View, CA October 2006

2 Collaborators Prof. Martin Vetterli, EPFL, Switzerland & UC Berkeley, CA Prof. Baltasar Beferull-Lozano, University of Valencia, Spain Dr. Pier Luigi Dragotti, Imperial College, UK Directionlets 2

3 Outline Introduction Anisotropic transforms Skewed transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 3

4 Standard 2-D Wavelet Transform - Review H 1 ( z 1 ) h H 0 ( z 1 ) 2 2 H 1 ( z 1 ) v H 0 ( z 1 ) H 1 ( z 1 ) v H 0 ( z 1 ) h - horizontal v - vertical... Directionlets 4

5 Standard 2-D Wavelet Transform - Review Advantages: sparse representation, separability, simple filter design (1-D), low computational complexity - used in JPEG2000 Disadvantages: isotropy, only horizontal and vertical directions Directionlets 5

6 Previous Work & Motivation Several approaches with anisotropy or multi-directionality have been proposed: Curvelets (Candès & Donoho), Contourlets (Do et al.), Wedgelets (Donoho, Baraniuk), Wedgeprints (Wakin et al.), Edgeprints (Dragotti & Vetterli), Bandelets (Mallat), Multiscale transform (Cohen), Directional wavelets (Zuidwijk), Polynomial modeling & quadtree segmentation (Shukla et al.), Brushlets (Meyer & Coifman) Steerable pyramid (Simoncelli), Directional filter banks (Bamberger & Smith), Complex wavelets (Kingsbury). Disadvantages: non-separability (2-D convolution), continuous-time filter design, high computational complexity, oversampling, non-perfect reconstruction Our goals: anisotropy, multi-directionality, basis property, separable filter design Directionlets 6

7 Outline Introduction Anisotropic transforms Skewed transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 7

8 Mondrian Images Standard 2-D wavelet transform of Mondrian images is not sparse There are O((k 1 + k 2 )M) nonzero coefficients Directionlets 8

9 Fully Separable Decomposition Tensor wavelets - previously proposed by Westerink (1989), Nowak & Baraniuk (1999), Rosiene & Nguyen (1999) Ex.: biorthogonal D filters Stand. WT FSWT Freq. decomp. Imp. resp. Spectrum Directionlets 9

10 Fully Separable Decomposition - Approximation Order Fully separable transform changes order of approximation of Mondrian images Representation is sparser: O((k 1 + k 2 )(log 2 M) 2 ) nonzero coefficients Exponential gain over standard WT: (log 2 M) 2 /M Directionlets 10

11 Anisotropic Wavelet Transforms Apply n 1 horizontal and n 2 vertical steps, where n 1 n 2 Stand. WT AWT Freq. decomp. Imp. resp. Spectrum Better description of natural images Fine for dominant horizontal and vertical directions What if other directions are present? skewed transforms Directionlets 11

12 Outline Introduction Anisotropic transforms Skewed transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 12

13 Discretization of Directions Digital line L(r, n) approximates continuous line along slope r and with intercept n (Bresenham, 1965) L , 3... L , 0 Digital lines L(r, n), for a fixed rational r and n Z, partitions discrete space Z 2 Directionlets 13

14 Directional Interaction Apply filtering and subsampling along digital lines Ex.: two digital lines L( 1/2,n) and L(2/3,n) L , n L , n Haar high-pass filtering along lines of slope -1/2 Resulting nonzero coefficients produced by blue digital line are not aligned! Non-systematic iterated construction in general Directionlets 14

15 Lattice-based Filtering and Subsampling Let vectors d 1 and d 2 define two directions M Λ = [ d1 d 2 ] = [ ] a1 b 1 a 2 b 2 Filtering and subsampling are applied across integer lattice Λ defined by M Λ M 1 1 Λ = 1 1 s 0 = 00 s 1 = 01 M 2 2 Λ = 1 1 Subsampled lattice Λ is defined by M Λ =[2d 1, d 2 ] T Operations are applied in each coset separately Directionlets 15

16 How to Avoid Directional Interaction Given two digital lines L( 1/2,n) and L(2/3,n), apply filtering and subsampling in corresponding lattice defined by M Λ L , n L , n M 2 1 Λ = 3 2 Haar high-pass filtering along lines of slope -1/2 Resulting nonzero coefficients produced by blue digital line are aligned now! Directionlets 16

17 Skewed Anisotropic Wavelet Transforms Apply anisotropic transforms along vectors of lattice Λ S-AWT(M Λ,n 1,n 2 ) Basis functions (directionlets) have directional vanishing moments Stand. WT S-AWT Freq. decomp. Imp. resp. Spectrum Directionlets 17

18 Outline Introduction Anisotropic transforms Skewed transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 18

19 Approximation Power Non-linear approximation: retain N largest magnitude transform coefficients For C 2 /C 2 images mean-square error decays as: Standard WT: O(N 1 ) (Mallat) Optimal: O(N 2 ) (Mallat) Oversampled curvelets (Donoho) and contourlets (Do&Vetterli): O(N 2 (log N) 3 ) Oversampled bandelets (Le Pennec&Mallat) and wedgelets (Donoho, Baraniuk): O(N 2 ) Directionlets 19

20 Approximation Power of Directionlets Theorem: Given a C 2 /C 2 image and using spatial segmentation, directionlets achieve (a) MSE= O(N 1.55 ) in non-linear approximation (b) D = O(R 1.55 ) in compression. Transform directions are adapted to each spatial segment Adaptivity allows for a sparser representation Directionlets are critically sampled mean-square error is conserved Convenient for Lagrangian optimization-based compression methods Directionlets 20

21 Approximation Results db Directionlets Standard wavelets Stand. WT N =0.5%, PSNR=18.68dB Directionlets N =0.5%, PSNR=19.23dB % Directionlets 21

22 Adapted Segmentation and Directions Optimization of directions minimization of Lagrangian cost N + λ MSE Directionlets 22

23 Outline Introduction Anisotropic transforms Skewed transforms General multi-directional transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 23

24 Directionlets in Compression 2-D wavelet transform has been applied successfully in image compression algorithms: Scalar quantization and optimal bit allocation (Antonini et al., 1992) Embedded coding using zerotrees (Shapiro, 1993) Set partitioning in hierarchical trees (SPIHT) (Said and Pearlman, 1996) Space-frequency quantization (Xiong, 1998) Sparsity good compression! Directionlets substitute standard WT Multi-directionality and anisotropy improve sparsity Spatial segmentation extracts one or a few dominant directions per segment Given a target bitrate, the best segmentation and transform directions in each segment are found using Lagrangian optimization Directionlets 24

25 Zerotree Structure - Review Zerotree hierarchical structure of wavelet coefficients How to adapt this structure to directionlets? M 20 2 = M 1 = Directionlets 25

26 Skewed Anisotropic Zerotree Structure Directionlets coefficients can also be structured in directional zerotrees along any two directions Ex.: S-AWT(M Λ,2,1) M 44 Λ2 = M Λ1 = Directionlets 26

27 Computational Complexity Given an N N image and an L-tap filter: Bandelets: O(N 2 (log 2 N) 2 ) Wedgeprints: O(N 2 log 2 N) Contourlets: O(L 2 N 2 ) Directionlets: O(LN 2 ) Directionlets provide low-complex compression method! Directionlets 27

28 Deblocking Method Segmentation generates blocking effect (like in the old JPEG) Deblocking method (Xiong, Mallat) Directionlets 28

29 Compression Results - Lena db SFQ WT SFQ WT with extended Q SFQ Directionlets Stand. WT R =0.057bpp, PSNR=26.20dB Directionlets R =0.057bpp, PSNR=26.92dB bpp Directionlets 29

30 Optimal Segmentation - Lena Directionlets 30

31 Compression Results - Fingerprints db SFQ WT SFQ Directionlets Stand. WT R =0.114bpp, PSNR=20.03dB Directionlets R =0.114bpp, PSNR=22.53dB bpp Directionlets 31

32 Outline Introduction Anisotropic transforms Skewed transforms General multi-directional transforms Applications in image processing: Non-linear approximation Compression Denoising Directionlets 32

33 Denoising Algorithm Gaussian-scale mixtures in denoising (Wainwright et al. 2000, 2001; Portilla et al. 2003) We apply redundant undecimated directionlets in GSM model Enforce coherence across scales, space and directions in images S AWT( M 1, 21, ) GSM est. ( S AWT) 1 x est dir S AWT( M d, 21, ) GSM est. ( S AWT) 1 x est dir d x noisy Lin. comb. xˆ S AWT( I, 11, ) GSM est. ( S AWT) 1 S AWT( Q, 11, ) GSM est. ( S AWT) 1 mean x est smooth Directionlets 33

34 Isotropic and Anisotropic Neighborhoods Isotropic c 2 + δ δ m j 1 c 2 δ c 2 δ n c 1 δ c 1 c 1 + δ Anisotropic m j δ c 2 δ n θ c 1 Directionlets 34

35 Smooth and Oriented Denoising Orig. Noisy Smooth Oriented Directionlets 35

36 Combined Denoising For each pixel: Determine dominant direction using oriented high-pass output Combine smooth and oriented denoising outputs 32.93dB Smooth 33.57dB Combined Directionlets 36

37 Denoising Results Original Noisy 10.6dB UWT 25.1dB Dir.-lets 26.1dB UWT Directionlets Directionlets 37

38 Denoising Results - cont d Numerical results for Barbara Output db Standard WT Directionlets Portilla et al Input db Directionlets 38

39 Conclusion Anisotropic wavelet transforms Lattice-based skewed wavelet transforms S-AWT (directionlets) retains simplicity, separability and critical sampling of standard WT involves multi-directionality and anisotropy Achievable order of approximation MSE= O(N 1.55 ), while keeping critical sampling Straightforward implementation in compression with skewed anisotropic zerotrees Undecimated directionlets provide an efficient denoising tool enforcing coherence across scales, space and directions Directionlets 39

40 Further Applications Image classification Lightfield rendering 3-D view (moving viewpoint) Video processing Resolution enhancement Directionlets 40

41 References [1] V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, P. L. Dragotti, Low-rate reduced complexity image compression using directionlets, Proc. of the IEEE Int. Conf. on Image Proc., ICIP2006, Atlanta, GA, October [2] V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, Low bit-rate space-frequency quantization using directionlets, submitted to IEEE Trans. on Image Proc., September [3] V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, P. L. Dragotti, Directionlets: anisotropic multi-directional representation with separable filtering, IEEE Trans. on Image Proc., July [4] V. Velisavljevic, Directionlets: anisotropic multi-directional representation with separable filtering, Ph.D. Thesis, EPFL, Lausanne, Switzerland, October [5] V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, P. L. Dragotti, Approximation power of directionlets, Proc. of the IEEE Int. Conf. on Image Proc., ICIP2005, Genova, Italy, September Directionlets 41