Patch similarity under non Gaussian noise

Transcription

1 The 18th IEEE International Conference on Image Processing Brussels, Belgium, September 11 14, 011 Patch similarity under non Gaussian noise Charles Deledalle 1, Florence Tupin 1, Loïc Denis 1 Institut Telecom, Telecom ParisTech, CNRS LTCI, Paris, France Observatoire de Lyon, CNRS CRAL, UCBL, ENS de Lyon, Université de Lyon, Lyon, France September 13, 011 C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

2 Motivation Increasing use of patches to model images Texture synthesis, Inpainting, Image editing, Denoising, Super-resolution, Image registration, Stereo vision, Object tracking. Image model based on the natural redundancy of patches C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, 011 / 18

3 Motivation Increasing use of patches to model images Texture synthesis, Inpainting, Image editing, Denoising, Super-resolution, Image registration, Stereo vision, Object tracking. (a) Microscopy (b) Astronomy (c) SAR polarimetry C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, 011 / 18

4 Motivation Increasing use of patches to model images Texture synthesis, Inpainting, Image editing, Denoising, Super-resolution, Image registration, Stereo vision, Object tracking. How to compare noisy patches? }{{}? }{{}? How to take into account the noise model? }{{}? C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, 011 / 18

5 Table of contents 1 Limits of the Euclidean distance Variance stabilization approach 3 How to adapt properly to the noise distribution? 4 Evaluation of similarity criteria C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

6 Table of contents 1 Limits of the Euclidean distance Variance stabilization approach 3 How to adapt properly to the noise distribution? 4 Evaluation of similarity criteria C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

7 Noise models Gaussian noise assumption A pair (x 1, x ) of noisy patches can be decomposed as: = + and = +. }{{} x 1 } {{ } θ 1 }{{} n 1 }{{} x } {{ } θ }{{} n Beyond Gaussian noise Noise can be non-gaussian, e.g., Poisson or Gamma distributed, Non-additive decomposition for Poisson noise: }{{} x 1 = } {{ } θ 1 + }{{} n 1 The noise level is signal-dependent. and }{{} x = } {{ } θ + }{{} n C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

8 Euclidean distance under noisy conditions Why the Euclidean distance under Gaussian noise? 1 Euclidean distance estimates the dissimilarity between noise-free patches: E = + PatchSize σ, When θ 1 = θ = θ 1, the residue is statistically small and independent on θ 1: = 1 < τ, 3 When θ 1 θ, the residue is statistically higher: = 1 > τ. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

9 Limits of the Euclidean distance Limit of the Euclidean distance with Poisson noise 1 Euclidean distance does not estimate the dissimilarity between noise-free patches: E = , When θ 1 = θ = θ 1, the residue cannot be controlled and is dependent on θ 1: = 1? τ, 3 Then, when θ 1 θ, there is no guarantee that the residue is statistically higher: = 1? τ. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

10 Table of contents 1 Limits of the Euclidean distance Variance stabilization approach 3 How to adapt properly to the noise distribution? 4 Evaluation of similarity criteria C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

11 Variance stabilization approach Variance stabilization approach Use an application s which stabilizes the variance for a specific noise model, Evaluate the Euclidean distance between the transformed patches: ( ) ( ) s s =, (1) C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

12 Variance stabilization approach Variance stabilization approach Use an application s which stabilizes the variance for a specific noise model, Evaluate the Euclidean distance between the transformed patches: ( ) ( ) s s =, (1) Example Gamma noise (multiplicative) and the homomorphic approach: s(x ) = log X Var[s(X )] = Var[log X ] = Ψ(1, L) () where L is the shape parameter of the gamma distribution. Poisson noise and the Anscombe transform: s(x ) = X + 3 (θ 4 Var[s(X )] = 1). (3) 8 C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

13 Variance stabilization approach Why does it seem to work? 1 Euclidean distance estimates the dissimilarity between transformed noise-free patches: E s ( ) s ( ) = + Constant, When θ 1 = θ = θ 1, the residue is statistically small: = 1 < τ, 3 Then, when θ 1 θ, the residue is statistically higher: = 1 > τ. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

14 Variance stabilization approach Limits Only heuristic, No optimality results, Does not take into account the statistics of the transformed data, Does not exist for all noise distribution models. (a) Image with impulse noise (b) SAR cross correlation C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

15 Table of contents 1 Limits of the Euclidean distance Variance stabilization approach 3 How to adapt properly to the noise distribution? 4 Evaluation of similarity criteria C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

16 What are we looking for? Definitions and properties A similarity criterion can be based on the hypothesis test (i.e., a parameter test): H 0 : θ 1 = θ θ 1 (null hypothesis), H 1 : θ 1 θ (alternative hypothesis). Its performance can be measured as: P FA = P(answer dissimilar; θ 1, H 0) P D = P(answer dissimilar; θ 1, θ, H 1) (false-alarm rate), (detection rate). The likelihood ratio (LR) test minimizes P D for any P FA : L(x 1, x ) = p(x1, x; θ1, H0) p(x 1, x ; θ 1, θ, H 1). Problem: θ 1, θ 1 and θ are unknown. given by the noise distribution model C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

17 Patch similarity criteria Generalized likelihood ratio Generalized likelihood ratio (GLR) Estimate θ 1, θ 1 and θ with the maximum likelihood estimate (MLE), Define the (negative log) generalized likelihood ratio test: log GLR(x 1, x ) = log sup t p(x 1, x ; θ 1 = t, H 0) sup t1,t p(x 1, x ; θ 1 = t 1, θ = t, H 1) = log p(x1; θ1 = ˆt 1)p(x ; θ = ˆt 1) p(x 1; θ 1 = ˆt 1)p(x ; θ = ˆt ) C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

18 Patch similarity criteria Generalized likelihood ratio Generalized likelihood ratio (GLR) Estimate θ 1, θ 1 and θ with the maximum likelihood estimate (MLE), Define the (negative log) generalized likelihood ratio test: log GLR(x 1, x ) = log sup t p(x 1, x ; θ 1 = t, H 0) sup t1,t p(x 1, x ; θ 1 = t 1, θ = t, H 1) = log p(x1; θ1 = ˆt 1)p(x ; θ = ˆt 1) p(x 1; θ 1 = ˆt 1)p(x ; θ = ˆt ) Maximal self similarity Assume x 1 x, then: ( p log x 1 = ; θ 1 = ( p x 1 = ; θ 1 = ) ( p x = ; θ 1 = ) ( p x = ; θ = ) ) > 0 C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

19 Patch similarity criteria Generalized likelihood ratio Generalized likelihood ratio (GLR) Estimate θ 1, θ 1 and θ with the maximum likelihood estimate (MLE), Define the (negative log) generalized likelihood ratio test: log GLR(x 1, x ) = log sup t p(x 1, x ; θ 1 = t, H 0) sup t1,t p(x 1, x ; θ 1 = t 1, θ = t, H 1) = log p(x1; θ1 = ˆt 1)p(x ; θ = ˆt 1) p(x 1; θ 1 = ˆt 1)p(x ; θ = ˆt ) Equal self similarity Assume x 1 = x, then: ( p log x 1 = ; θ 1 = ( p x 1 = ; θ 1 = ) ( p x = ; θ 1 = ) ( p x = ; θ = ) ) = 0 C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

20 Patch similarity criteria Generalized likelihood ratio name pdf log GLR Stabilization Euclidean Gaussian Poisson Gamma e (x θ) σ (x πσ 1 x ) θ x e θ x! L L x L 1 e Lx θ Γ(L)θ L log x 1 +x x x 1 1 xx (x 1 +x ) x 1 +x ( ) x1 x log + log x x1 ( ) x1 +3/8 x +3/8 ( log x ) 1 x The three criteria for three noise models C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

21 Patch similarity criteria Generalized likelihood ratio name pdf log GLR Stabilization Euclidean Gaussian Poisson Gamma e (x θ) σ (x πσ 1 x ) θ x e θ x! L L x L 1 e Lx θ Γ(L)θ L log x 1 +x x x 1 1 xx (x 1 +x ) x 1 +x ( ) x1 x log + log x x1 ( ) x1 +3/8 x +3/8 ( log x ) 1 x The three criteria for three noise models Does it work? Illustration with Gamma noise 1 When θ 1 = θ = θ 1, the residue is statistically small: ( ) log GLR, = Then, when θ 1 θ, the residue is statistically higher: ( ) log GLR, = < τ 1 > τ 1 C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

22 Table of contents 1 Limits of the Euclidean distance Variance stabilization approach 3 How to adapt properly to the noise distribution? 4 Evaluation of similarity criteria C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

23 Detection 1 Gamma Generalized likelihood ratio Variance stabilization Euclidean distance Probability of detection L G 0. S 0.1 N Q G Probability of false alarm Maximum joint likelihood [Alter et al., 006] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

24 Detection 1 Poisson Generalized likelihood ratio Variance stabilization Euclidean distance Probability of detection Q G Probability of false alarm Maximum joint likelihood [Alter et al., 006] Mutual information kernel [Seeger, 00] Bayesian likelihood ratio [Minka, 1998, Minka, 000] Bayesian joint likelihood [Yianilos, 1995, Matsushita and Lin, 007] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18 L G K B S L B N Q B

25 Evaluation on denoising Non-local filtering with GLR Poisson Gamma (a) Noisy image (b) Euclidean distance (c) GLR NB: Variance stabilization provides visual quality very close to GLR. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

26 Evaluation on denoising Non-local filtering with GLR Poisson Gamma (a) Noisy image (b) Euclidean distance (c) GLR NB: Variance stabilization provides visual quality very close to GLR. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

27 Evaluation on denoising Iterative non-local filtering with GLR (a) SAR cross correlation (b) Non-local filtering with GLR GLR can be used when the variance stabilization approach cannot be applied. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

28 Stereo-vision Gamma (a) Noisy image (b) Euclidean distance (c) GLR Poisson (d) Ground truth (e) Euclidean distance (f) GLR C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

29 Motion tracking Glacier monitoring with a stereo pair of SAR images (g) Noisy image (h) Euclidean distance (i) GLR Glacier of Argentie re. With GLR, the estimated speeds matches with the ground truth: average over the surface of 1.7 cm/day and a maximum of 41.1 cm/day in the areas with crevasses. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

30 Conclusion Conclusion GLR behaves well to compare patches under non-gaussian noise conditions, It can be used when variance stabilization cannot be applied, Under high levels of gamma and Poisson noise, it outperforms six other criteria: Best probability of detection for any probability of false-alarm. We have shown the interest of GLR in: Patch-based denoising, Patch-based stereo vision, and Patch-based motion tracking. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

31 Conclusion Conclusion GLR behaves well to compare patches under non-gaussian noise conditions, It can be used when variance stabilization cannot be applied, Under high levels of gamma and Poisson noise, it outperforms six other criteria: Best probability of detection for any probability of false-alarm. We have shown the interest of GLR in: Patch-based denoising, Patch-based stereo vision, and Patch-based motion tracking. Future work Under high SNR the variance stabilization approach defeats GLR: Why such a change of relative behavior? Euclidean distance estimates the dissimilarity between noise-free patches, While GLR evaluates the equality of noise-free patches. Extend this approach to derive criteria with contrast invariance (e.g., for stereo-vision or flickering). C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

32 Conclusion Questions? C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

33 [Alter et al., 006] Alter, F., Matsushita, Y., and Tang, X. (006). An intensity similarity measure in low-light conditions. Lecture Notes in Computer Science, 3954:67. [Matsushita and Lin, 007] Matsushita, Y. and Lin, S. (007). A Probabilistic Intensity Similarity Measure based on Noise Distributions. In IEEE Conference on Computer Vision and Pattern Recognition, 007. CVPR 07, pages 1 8. [Minka, 1998] Minka, T. (1998). Bayesian Inference, Entropy, and the Multinomial Distribution. Technical report, CMU. [Minka, 000] Minka, T. (000). Distance measures as prior probabilities. Technical report, CMU. [Seeger, 00] Seeger, M. (00). Covariance kernels from Bayesian generative models. In Advances in neural information processing systems 14: proceedings of the 001 conference, page 905. MIT Press. [Yianilos, 1995] Yianilos, P. (1995). Metric learning via normal mixtures. Technical report, NEC Research Institute, Princeton, NJ. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

34 Evaluation on denoising Numerical performance Medium noise levels Noisy Q G L G S G Gamma barbara boat bridge cameraman couple fingerprint hill house lena man mandril peppers PSNR values: the higher the better Generalized likelihood ratio Variance stabilization Euclidean distance Maximum joint likelihood [Alter et al., 006] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

35 Evaluation on denoising Numerical performance Strong noise levels Noisy Q G L G S G Gamma barbara boat bridge cameraman couple fingerprint hill house lena man mandril peppers PSNR values: the higher the better Generalized likelihood ratio Variance stabilization Euclidean distance Maximum joint likelihood [Alter et al., 006] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

36 Evaluation on denoising Numerical performance Medium noise levels Noisy Q B Q G L B L G K B S G Poisson barbara boat bridge cameraman couple fingerprint hill house lena man mandril peppers PSNR values: the higher the better Generalized likelihood ratio Variance stabilization Euclidean distance Maximum joint likelihood [Alter et al., 006] Mutual information kernel [Seeger, 00] Bayesian likelihood ratio [Minka, 1998, Minka, 000] Bayesian joint likelihood [Yianilos, 1995, Matsushita and Lin, 007] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

37 Evaluation on denoising Numerical performance Strong noise levels Noisy Q B Q G L B L G K B S G Poisson barbara boat bridge cameraman couple fingerprint hill house lena man mandril peppers PSNR values: the higher the better Generalized likelihood ratio Variance stabilization Euclidean distance Maximum joint likelihood [Alter et al., 006] Mutual information kernel [Seeger, 00] Bayesian likelihood ratio [Minka, 1998, Minka, 000] Bayesian joint likelihood [Yianilos, 1995, Matsushita and Lin, 007] C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

38 Max. self sim. Eq. self sim. Id. of indiscernible Invariance Asym. CFAR Asym. UMPI Q B Q G L B L ( ) G K ( ) B G S ( ) ( ) ( ) ( ) ( ) Properties of the different studied criteria. Legend: ( ) the criterion holds, ( ) the criterion does not hold. Holds only if the observations are statistically identifiable ( ) through their MLE or ( ) through their likelihood (such assumptions are frequently true). ( ) Holds only for an exact variance stabilizing transform s( ) (such an assumption is usually wrong). The proofs of all these properties are available in the Online Resource 1. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, / 18

39 name pdf Q B Q G L B L G K B S G Gaussian Gamma e (x θ) σ e (x 1 x ) πσ LL x L 1 e Lx ( θ Γ(L)θ L 1 x 1 x x 1 x (x 1 +x ) ( ) L x 1 x log x ) 1 x (x 1 +x ) e Poisson θ x e θ x! Γ ( ) (x 1 +x ) x1 +x x1 +x x 1 +x x 1!x! (e) x 1 +x x 1!x! Γ ( ) (x 1 +x ) x1 +x x1 +x x 1 +x Γ (x 1 )Γ (x ) x 1 +x x x 1 1 xx Γ (x 1 +x ) Γ (x 1 )Γ (x ) e ( x 1 +a x +a ) Instances of the seven criteria for Gaussian, gamma and Poisson noise (parameters σ and L are fixed and known). All Bayesian criteria are obtained with Jeffreys priors (resp. 1/σ, L/θ, 1/θ). All constant terms which do not affect the detection performance are omitted. For clarity reason, we define Γ (x) = Γ(x + 0.5) and the Anscombe constant a = 3/8. C. Deledalle, F. Tupin, L. Denis (Telecom ParisTech) Patch similarity September 13, 011 / 18