graphics + usability + visualization Edge Aware Anisotropic Diffusion for 3D Scalar Data on Regular Lattices Zahid Hossain MSc.

Transcription

1 gruvi graphics + usability + visualization Edge Aware Anisotropic Diffusion for 3D Scalar Data on Regular Lattices Zahid Hossain MSc. Thesis Defense Simon Fraser University Burnaby, BC Canada.

2 Motivation Original Sheep s heart data 2 of 43

3 Motivation Existing gradient based anisotropic diffusion Original Sheep s heart data 2 of 43

4 Motivation Existing gradient based anisotropic diffusion Gradient magnitude map 2 of 43

5 Motivation Existing gradient based anisotropic diffusion Gradient magnitude map Proposed method Our method is consistent and free of undesirable artifacts 2 of 43

6 Previous Work Perona and Malik (PM) Model (TPAMI, 1990): f = div(h( f ) f ) t 3 of 43

7 Previous Work Perona and Malik (PM) Model (TPAMI, 1990): f = div(h( f ) f ) t Carmona et al.(tip, 1998) generalization in 2D: SF f t = {}}{ h ( ) af nn + bf vv, SF = Stopping Function where v = n. 3 of 43

8 Previous Work Perona and Malik (PM) Model (TPAMI, 1990): f = div(h( f ) f ) t Carmona et al.(tip, 1998) generalization in 2D: SF f t = {}}{ h ( ) af nn + bf vv, SF = Stopping Function where v = n. 3D extension by Gerig et al.(tmi, 1992): Isotropic on the tangent plane of the gradient. 3 of 43

9 Previous Work: Trouble in 3D Original 4 of 43

10 Previous Work: Trouble in 3D PM model in 3D Original 4 of 43

11 Previous Work: Trouble in 3D PM model in 3D Original Curvatures taken into account Therefore Weickert classified PM and higher order diffusion processes as non-linear rather than anisotropic. 4 of 43

12 Previous Work: In 3D Whitaker (VBC, 1994) and Krissian et al.(scale-space, 1997) (KM model), took curvatures into account The stopping function had gradient magnitude based parameter. 5 of 43

13 Previous Work: In 3D Whitaker (VBC, 1994) and Krissian et al.(scale-space, 1997) (KM model), took curvatures into account The stopping function had gradient magnitude based parameter. Level Set: The definition of edge is based on the curvature that is measured on the surface of the level set. 5 of 43

14 Previous Work: In 3D Whitaker (VBC, 1994) and Krissian et al.(scale-space, 1997) (KM model), took curvatures into account The stopping function had gradient magnitude based parameter. Level Set: The definition of edge is based on the curvature that is measured on the surface of the level set. De-noising: Bilateral filtering, mean-shift filtering and non-linear diffusion are equivalent: Barash et al., (IVC, 2004). A recent methods: SRNRAD and ORNRAD (TIP, 2009). 5 of 43

15 Contributions 1 Removal of the gradient magnitude based parameter 6 of 43

16 Contributions 1 Removal of the gradient magnitude based parameter 2 Dynamic adaptation of anisotropy 6 of 43

17 Contributions 1 Removal of the gradient magnitude based parameter 2 Dynamic adaptation of anisotropy 3 Application in de-noising 6 of 43

18 Contributions 1 Removal of the gradient magnitude based parameter 2 Dynamic adaptation of anisotropy 3 Application in de-noising 4 Implementation recipe in arbitrary regular lattices 6 of 43

19 The Proposed Method 7 of 43

20 Proposed Method: Edge The red solid curve is the error function while the dashed black curve is the second derivative of it. A common technique in 2D image processing. 8 of 43

21 Four Objectives of Our Diffusion Objectives 1 No diffusion along the gradient direction. This prevents blurring across an edge. 9 of 43

22 Four Objectives of Our Diffusion Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. Check for f nn = 0. No problem in constant homogeneous regions. 9 of 43

23 Four Objectives of Our Diffusion Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. Similar to Krissian et al.(1997), i.e the KM model. 9 of 43

24 Four Objectives of Our Diffusion Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. 4 Diffuse on the tangent plane where the local iso-surface has similar principal curvatures. For example, on the surface of a sphere or a slab. 9 of 43

25 Modelling the PDE Consider the 1D heat, also known as diffusion, equation: f t = cf xx 10 of 43

26 Modelling the PDE Consider the 1D heat, also known as diffusion, equation: f t = cf xx A 3D extension of the above: f t = hf r 1 r 1 + gf r2 r 2 + wf nn Where the orthonormal bases [r 1,r 2,n] are min curvature, max curvature and normal directions respectively. 10 of 43

27 Modelling the PDE Consider the 1D heat, also known as diffusion, equation: f t = cf xx A 3D extension of the above: f t = hf r 1 r 1 + gf r2 r 2 + wf nn Where the orthonormal bases [r 1,r 2,n] are min curvature, max curvature and normal directions respectively. Setting g = τh and w = ηh we can simplify the above f ( t = h f r1 r 1 + τf r2 r 2 + ηf nn ), τ,η [0,1] 10 of 43

28 Modelling the PDE: Objective 1 Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. 4 Diffuse on the tangent plane where the local iso-surface has similar principal curvatures. f ( t = h f r1 r 1 + τf r2 r 2 + ηf nn ), τ,η [0,1] 11 of 43

29 Modelling the PDE: Objective 1 Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. 4 Diffuse on the tangent plane where the local iso-surface has similar principal curvatures. Set η = 0 f ( t = h f r1 r 1 + τf r2 r 2 + ηf nn ), τ,η [0,1] f ( t = h f r1 r 1 + τf r2 r 2 ), τ [0,1] 11 of 43

30 Modelling the PDE: Objective 2 Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. 4 Diffuse on the tangent plane where the local iso-surface has similar principal curvatures. 12 of 43

31 Modelling the PDE: Objective 2 Define h h(α) = 1 (0.9) ( α σ ) 2, σ R h(α) α Plot of h(α) 13 of 43

32 Modelling the PDE: Objective 2 Define h h(α) = 1 (0.9) ( α σ ) 2, σ R h(α) Takes the directional second derivative f nn along the normal n as argument α Plot of h(α) 13 of 43

33 Modelling the PDE: Objective 2 Define h h(α) = 1 (0.9) ( α σ ) 2, σ R h(α) Takes the directional second derivative f nn along the normal n as argument. ) So far: f t = h(f nn) (f r1 r 1 + τf r2 r α Plot of h(α) 13 of 43

34 Modelling the PDE: Objective 2 Define h h(α) = 1 (0.9) ( α σ ) 2, σ R h(α) Takes the directional second derivative f nn along the normal n as argument. ) So far: f t = h(f nn) (f r1 r 1 + τf r2 r α Plot of h(α) Diffusion stops when f nn 0, i.e. around the edge locations. Homogeneous regions remains unaffected. 13 of 43

35 Modelling the PDE: Objectives 3 and 4 Objectives 1 No diffusion along the gradient direction. 2 Diffusion stopped around the edge locations. 3 Diffuse along the direction of the minimum curvature. 4 Diffuse on the tangent plane where the local iso-surface has similar principal curvatures. 14 of 43

36 Modelling the PDE: Objectives 3 and 4 f ) t = h(f nn) (f r1 r 1 + τf r2 r 2 15 of 43

37 Modelling the PDE: Objectives 3 and 4 where, f ) t = h(f nn) (f r1 r 1 + τf r2 r 2 ( ) 2λ κ1 τ = κ 2 where κ 2 > 0,λ Z 1 κ 2 = 0 κ 1 : κ 2 : minimum curvature maximum curvature 15 of 43

38 Modelling the PDE: Objectives 3 and 4 where, f ) t = h(f nn) (f r1 r 1 + τf r2 r 2 ( ) 2λ κ1 τ = κ 2 where κ 2 > 0,λ Z 1 κ 2 = 0 κ 1 : κ 2 : minimum curvature maximum curvature Usually λ = 2 works well Curvatures are measured from a blurred version of the data f ρ τ is replaced by τ ρ 15 of 43

39 Simplification So far we have Stopping Function f {}}{( ) t = h(f nn ) f r1 r 1 + τ ρ f r2 r }{{} 2 Isotropy Function 16 of 43

40 Simplification So far we have Which can be simplified to Stopping Function f {}}{( ) t = h(f nn ) f r1 r 1 + τ ρ f r2 r }{{} 2 Isotropy Function. f t = h(f nn) f (κ 1 + τ ρ κ 2 ) 16 of 43

41 Simplification So far we have Which can be simplified to Stopping Function f {}}{( ) t = h(f nn ) f r1 r 1 + τ ρ f r2 r }{{} 2 Isotropy Function. f t = h(f nn) f (κ 1 + τ ρ κ 2 ) Has a similar form to Mean Curvature Motion (MCM), yet essentially different. f MCM: t = f (κ 1 + κ 2 ) 16 of 43

42 Significance of the Stopping Function Original MCM Our method Our: ( ) fnn 2 f t = h(f nn) f (κ 1 + τ ρ κ 2 ), h(f nn ) = 1 (0.9) σ, σ R MCM: f t = f (κ 1 + κ 2 ) 17 of 43

43 Stability Courant-Friedrichs-Lewy (CFL) revealed the following necessary condition for stability: t C d2, d = min{ x, y, z}, C = Courant Number 2 C 0.8 is stable for most practical purposes. 18 of 43

44 Results 19 of 43

45 Results: Consistent Smoothing KM method: k = 40 Original Our method: σ = 1 20 of 43

46 Results: Consistent Smoothing KM method: k = 60 Original Our method: σ = of 43

47 De-noising Properties Note We used the same parameter settings for all our de-noising experiments across all datasets and noise types. 21 of 43

48 Results: De-noising (Gaussian) Noisy, SNR=12.89 Original Diffused, SNR= of 43

49 Results: De-noising (Gaussian) Noisy, SNR=12.89 Original Diffused, SNR=26.12 Original Noisy Diffused Profile 22 of 43

50 Results: De-noising (Gaussian) Noisy, SNR=12.89 Original Diffused, SNR=26.12 Original Noisy Diffused Profile 22 of 43

51 Results: De-noising (Salt and Pepper) Noisy, SNR=12.89 Original Diffused, SNR=26.12 Original Noisy Diffused Profile 23 of 43

52 De-noising Comparison Note Comparison with SRNRAD and ORNRAD (Krissian and Aja-Fernández, 2009). We did not change any parameter settings. 24 of 43

53 Quantitative Measures Mean Squared Error (MSE) Structural Similarity Index (SSIM) Quality Index Based on Local Variance (QILV) Original Blurred Gaussian noise SSIM: (Good) SSIM: (Bad) QILV: (Bad) QILV: (Good) 25 of 43

54 De-noising: Comparison Gaussian noise Original MSE SSIM QILV Noise SRNRAD ORNRAD Our Noisy Our SRNRAD ORNRAD 26 of 43

55 De-noising: Comparison Rician noise Original MSE SSIM QILV Noise SRNRAD ORNRAD Our Noisy Our SRNRAD ORNRAD 26 of 43

56 De-noising: Remarks Our method performs similarly if not better in many cases. 27 of 43

57 De-noising: Remarks Our method performs similarly if not better in many cases. Matlab implementation is 14 faster than a multi-threaded C/C++ based ORNRAD. Our formulation is simpler ORNRAD requires, beside complex statistical measurements, eigen decomposition of structure tensor. 27 of 43

58 Extension Framework for Regular Lattices 28 of 43

59 Lattices in 3D Cartesian Cubic (CC) Body Centric Cubic (BCC) Face Centric Cubic (FCC) A regular lattice can be characterized by a matrix L 29 of 43

60 Motivation For Other Lattices Tai Meng et al. (2011) and Entezari et al. (2008) CC BCC 30 of 43

61 Motivation For Other Lattices (Contd.) Alim et al. (2009) CC BCC 31 of 43

62 Taylor Expansion of Discrete Convolution Discrete convolution of f [k] with a filter [k] can be written as the following: f = fr w [k] = a }{{} n D n f [k] f [k] = f (Lk) n Taylor Coefficient 32 of 43

63 Taylor Expansion of Discrete Convolution Discrete convolution of f [k] with a filter [k] can be written as the following: f = fr w [k] = a }{{} n D n f [k] f [k] = f (Lk) n Taylor Coefficient And a n is a linear function of the filter weights [k] 32 of 43

64 Linear System f = fr w [k] = a n D n f [k], n f [k] = f (Lk) To extract the u-th derivative D u f [k], correct up to a polynomial order of n, the following must be satisfied { a 0, n η [0,n] n = d and n u 1, n = u. 33 of 43

65 Linear System (Contd.) Forms a linear system with the unknown filter weights w i = [k i ] a n 1 a n 2. a n m = of 43

66 Linear System (Contd.) Forms a linear system with the unknown filter weights w i = [k i ] a n 1 a n 2. a n m = g 1,1. g 1, g 1,m. g n,1 g n,2... g n,m w 1 w 2. w m = G W = C 34 of 43

67 Infinite Solution Space The linear system is often not full-rank 35 of 43

68 Infinite Solution Space The linear system is often not full-rank Finding a suitable solution by: Imposing symmetry/anti-symmetry in the filter geometry Minimizing error in the higher polynomial order 35 of 43

69 Infinite Solution Space The linear system is often not full-rank Finding a suitable solution by: Imposing symmetry/anti-symmetry in the filter geometry Minimizing error in the higher polynomial order Usually yields a filter that has small support 35 of 43

70 Results 36 of 43

71 Quality of Normals: Synthetic Data 2-EF 4-EF CC (Mag=3.13, Angle=49.4 ) (2.79, 44.1 ) (2.42, 28.7 ) BCC (2.88, Hossain, Simon Fraser University, Canada ) gruvi graphics + usability + visualization 37 of 43

72 Quality of Normals: Real Data (Iso-Surface) CC (4-EF) BCC (4-EF) 38 of 43

73 Effects of Higher Order Filters (CC) Original 2-EF 4-EF 39 of 43

74 Conclusion We developed... An anisotropic diffusion: Consistent smoothing Less effort to choose a parameter Performs similarly, if not better, to a recent de-noising method Fast and easy to implement 40 of 43

75 Conclusion We developed... An anisotropic diffusion: Consistent smoothing Less effort to choose a parameter Performs similarly, if not better, to a recent de-noising method Fast and easy to implement A framework to design discrete derivative filters: Works on arbitrary regular lattice Any derivative could be estimated Size of the filter could be specified PDE implementation on any regular lattice is made possible 40 of 43

76 Future Work Actual implementation in BCC and FCC De-noising performance in BCC and FCC Different functions for τ Alternate edge detectors 41 of 43

77 Acknowledgements I would like to thank: My supervisor, co-supervisor and the examining committee All the GrUVi members for their relentless support Natural Science and Engineering Research Council of Canada (NSERC) for funding my research 42 of 43

78 Thank you for your attention :) 43 of 43