Modulation-Feature based Textured Image Segmentation Using Curve Evolution

Transcription

1 Modulation-Feature based Textured Image Segmentation Using Curve Evolution Iasonas Kokkinos, Giorgos Evangelopoulos and Petros Maragos Computer Vision, Speech Communication and Signal Processing Group National Technical University of Athens, Greece School of Electrical and Computer Engineering URL:

2 Presentation Overview and Motivation Extracting texture features for segmentation: Intensity does not suffice. High dimensional features, common for texture description (e.g. Gabor filterbanks, orthonormal bases etc.) suboptimal segmentation In this work, we propose a low dimensional texture descriptor, based on the AM-FM image model. Unsupervised segmentation using state of the art techniques: Curve Evolution & Level-Set methods combine efficiency, elegance and mathematical tractability. Region Based Terms guarantee robustness, which is essential to texture segmentation.

3 Modulation Features for Texture Analysis (I) Image AM-FM Modulation Model ( ) ( ) Èf ( ) I xy, = ak xy, cos Î k xy,, k = 1 Estimate amplitude & frequency coefficients via Multiband Gabor filtering Narrowband image components Demodulation using D Energy Operator and the ESA Y K Â ( k ) k k1 ( ) = ( )* ( ) I x, y I x, y h x, y, k ( Ik ) ( I x) + Y( I y) k Y I x ( I ) ( x, y), k k ª (, ) k (, ) =W f k x y x y I I I I Y ( ) - a k ( x, y) Y I y Y ( I ) ª W ( x, y) Y Y ª W ( k ) k k Refs: P. Maragos & A.C. Bovik, JOSA 95, J.E. Daugman, JOSA 85, Havlicek, Harding & Bovik, 1996, 000.

4 Modulation Features for Texture Analysis (II) Dominant Components Analysis (DCA) proposes using at each pixel only the most prominent channel, j a( x, y) = aj ( x, y), W ( xy, ) = W j ( xy, ) G K Use maximization criterion for choosing among channels: Amplitude-Based DCA Teager Energy-Based DCA G (, ) = a ( x, y) k xy max H k ( W) k W j { } = arg max G 1 k K ( xy, ) È( I h)( xy, ) G k =Y Î * Refine results with texture vs. non-texture mask at various scales using multiple statistical hypothesis testing. Using a single channel amounts to locally modeling the texture with a Gabor-like texton whose features are described by the DCA components. Refs: Havlicek, Harding & Bovik, T-Image 000. Kokkinos, Evangelopoulos & Maragos, ICIP 04. k k

5 Modulation Features Extraction Examples Synthetic AM-FM Real Modulation Parameters a( x, y ) W1 ( xy, ) W ( xy), DCA Estimated Parameters ADCA EDCA Amplitude Frequency Magnitude

6 Modulation Features Extraction Examples EDCA Estimates: Amplitude and Spatial Frequencies Texture vs. Non-Texture

7 Region Based Segmentation by Active Contours Functional expressing segmentation cost (Region Competition Algorithm): N v EC [, p] = - log p Y dx+ LenC, ÂÚÚ Model multivariate features within each region using a Gaussian pdf : 1 1 Τ 1 Y µ Σ Y µ R 1 R 3 R p ( ( )) ( ) i i i i= 1 Ri ( I µ ) Y = [ Y,..., Y ] T ( π) i i i d / 1/ Σi p 3 p p 1 1 d C = { C1,..., C N } ( ) ( ) ;, Σ = exp i i i Refs: Zhu & Yuille, PAMI 96 Paragios & Deriche, JVCIR 0 Rousson, Brox & Deriche, CVPR 03

8 Functional Minimization Algorithm Euler-Lagrange equations lead to: C Ê i pi( Y) ˆ = vk i log t Á + Ë p ( Y) c N i Level Set Implementation: view front as zero set of embedding function: df Ê i pi( Y) ˆ = vk i log dt Á + F Ë pc ( Y) i F i Refs: Osher & Sethian, 1988.

9 Unsupervised Segmentation by Active Contours Unsupervised segmentation pdfs not known a priori Fronts are initiated so that the union of their interiors occupies the whole of the image Iterate: Estimate the parameters of the Gaussian p.d.f. for each region, using the front s current position: 1 1 µ Y Σ = Y µ Y µ Τ i = i i ( i i)( i i) R i R R i Ri i i Evolve fronts in the direction dictated by region competition (statistics force + geometrical information) i Refs: Zhu & Yuille, PAMI 1996, Rousson et. al., CVPR 003

10 Modulation Features for Texture Segmentation Dominant Component Features provide a low-dimensional and rich texture descriptor, that contains local information about Oscillation Amplitude Frequency (Scale) Orientation (Variation) Features for segmentation: Amplitude Horizontal & Vertical Frequency ( or ) Frequency Magnitude & Orientation Intensity Τ Y = [ a, Ω, Ω, I] Y = [ a, Ω, Ω, I] 1 Τ

11 Segmentation Examples Using Modulation Features Synthetic Textures DCA Estimated Texture Features a W W Segmentation Regions a W1 W

12 Segmentation Examples Using Modulation Features a DCA Estimated Texture Features W1 W Segmentation Regions

13 Segmentation Examples Using Modulation Features a DCA Estimated Texture Features W1 W a W Segmentation Regions

14 Segmentation Examples Using Modulation Features Segmentation Regions Segmentation Regions

15 Summary & Conclusions Texture representation by simple, information rich, lowdimensional feature vector. Important texture characteristics are captured with good localization. Unsupervised segmentation scheme that combines the merits of region competition and modulation/dca. Efficient segmentation of a wide variety of natural textured images.

16 Evolution of related ideas Zhu & Yuille, 1996: Region Competition (No level set implementation, fixed filterbank) Zray, Havlicek, Acton & Pattichis, 001: Modulation features & GAC (no region based term, curve evolution used for post-processing) Paragios & Deriche, 00: Textured Image Segmentation Using Level Set Methods (Supervised) Sagiv et al., 00. Vese et al, 00 : Unsupervised textured Image Segmentation using active contours (Mostly heuristic methods for dimensionality reduction) Rousson, Brox & Deriche, 003: Unsupervised Image Segmentation using structure tensor features. No scale information, anisotropic diffusion used for feature pre-processing. Kokkinos, Evangelopoulos, Maragos, ICIP 004: Modulation feature extraction and texture vs. non-texture decision as segmentation cues.

17 Appendix: D Energy Operator Continuous Discrete F c ( f)( xy, ) f( xy, ) - f( xy, ) f( xy, ) F d ( f)( i, j) = Ê ˆ Ê ˆ Ê ˆ f Á f Á Ê f ˆ f f f = Á Ë x Ë x Á Ë y Ë y f ( i, j) - f( i- 1, j) f( i+ 1, j) - f( i, j- 1) f( i, j+ 1)

18 Spatial AM-FM signal: j W ( ) 1 xy,, x W= W, W ( ) Energy tracking Appendix: D AM-FM Energy Tracking W ( xy) 1 [ cos( j) ] (, ) (, ) cos Èj (, ) f xy = axy Î xy, f y F ª W F ª a W1 W c a a : Instantaneous Frequency vector [ f ] c x

19 Appendix: D Continuous Energy Separation Algorithm D CESA ˆ Ê f ˆ W ( ) (, ) 1 = FcÁ F c f ªW1 xy Ë x ˆ Ê f ˆ W = F ( ) (, ) cá F c f ªW xy Ë y Fc ( f ) aˆ = ª a( x, y) F +F c ( f x) ( f y) c D cosine: CESA Æ exact estimates: f( x, y) = Acos( W x+w y+ j ) c1 c o W ( xy, ) =Wc W ( xy, ) =Wc axy (, ) =A 1 1

20 Appendix: D Gabor filterbanks and DCA D Gabor function hxy ax by Ux Vy (, ) = exp[ -( ) + ( ) s ] cos[ p( + )] DCA Block diagram, (by Havlicek et. al. 1996)

21 Appendix: Curve Evolution Γ() t is evolving curve (front) for t 0 Γ(0) is a simple smooth closed curve Position vector = Cpt (,) Normal vector = Npt (,) Speed: V Curvature = = F( K) Kpt (,) curve evolution PDE (flow) Cpt (,) t = VN(,) p t

22 Appendix: Level Set Method for Curve Evolution Embed curve Γ () t as zero-level curve of function Φ(,,) xyt : Γ ( t) = {( x, y) : Φ ( x, y, t) = 0} Φ 0( xy, ) = Φ ( xy,, 0) = signed distance tranform from Γ(0) Function evolution PDE: Φ = V t Φ Advantages of level set method: - Φ remains a function even if topology of Γt () changes - Compute curvature & normal of Γ() t from : κ = div ( Φ/ Φ ) = N Φ N = Φ / Φ - Efficient Numerics: entropy-satisfying finite differences - Extends easily to 3D Ref: Osher & Sethian 1988

23 Appendix: Region Competition N v EC [, p] = - log p I dx+ C ÂÚÚ ( ( )) i i i i= 1 Ri R 1 R i R R Q Forces acting on the contours, (by Zhu and Yuille, 1996)

24 Minimize Image functional (generalized energy ) Euler PDE: Gradient Descent: Solution reached at steady state: Example 1: Example : E[ u] F( x, y, u, u u ) dxdy = D Appendix: Functional Minimization x, u = Fu Fu F x u = t x y y Euler derivative [ F] u = [ F] t u y u 0 u/ t = 0 F = u ut = u t ( ( )/ ) F = u u = curv u u u

25 Appendix: Energy Tracking in AM-FM Signals Cont.-Time AM-FM Discrete-Time AM-FM ( ) = ( n ) ω τ τ xn [ ] = an [ ]cos Ω( mdm ) 0 t xt () at ()cos ( ) d 0 Cont.-Time TK Energy Operator: Discrete-Time TK Energy Operator: c [ ] x( t) = x ( t) x( t) x ( t) Ψ [ ] d Ψ xn [] = x[] n xn [ + 1][ xn 1] Energy Tracking Energy Tracking Ψc [ xt ()] α () tω () t Ψ [ ] δ xn [ ] α [ n]sin ( Ω[ n] )

26 Appendix: 1D Continuous & Discrete-Time ESA Ψ[ xt ( )] Ψ[ xt ( )] at () 1 Ψ[ xt ( )] π Ψ[ xt ( )] f() t Assumption: x(t) is a narrowband AM-FM Signal Ψ[ xn [ ]] [ xn [ 1] xn [ 1] ] Ψ + an [ ] [ xn [ 1] xn [ 1] ] Ψ[ xn] 1 Ψ + arcsin f [ n ] π 4 [ ]