Non-uniformity Correction of IR Images using Natural Scene Statistics

Transcription

1 Non-uniformity Correction of IR Images using Natural Scene Statistics Todd Goodall, Alan C. Bovik, and Haris Vikalo 1 Nicholas G. Paulter, Jr. 2 1 University of Texas at Austin 2 National Institute of Standards and Technology (NIST) December 14, 2015 LaboratoryforImage&VideoEngineering TheUniversityofTexasatAustin

2 Outline Introduction Non-uniformity model Non-uniformity prediction Non-uniformity prediction performance Recall steerable pyramid decomposition Gaussian Scale Mixture (GSM) denoising model Prediction of non-uniformity covariance, Ĉ NU Gaussian Scale Mixture Perceptual Pattern (GSMPP) denoiser Other non-uniformity denoisers Results Conclusion

3 Introduction Infrared (IR) Images capture 8 to 14 micrometer wavelengths. These images are similar to visible light images, but very smooth.

4 Introduction Many current IR imagers require non-uniformity correction, as in the early days of visible light imagers. Non-uniformity is a fixed pattern noise, appearing as one of vertical, horizontal, or gridlike striping patterns.

5 Introduction Goal Predict the degree to which real non-uniformity is present. Remove that non-uniformity. Before After

6 Non-uniformity model Pezoa and Medina s non-uniformity model ( ) ( ) (u u 0 ) 2 (v v 0 ) 2 ĨNU(u, v) = B u exp + B v exp ĨNU(u, v) U[ π, π] 2σ 2 u 2σ 2 v Amplitude controlled by σ V

7 Non-uniformity prediction Existing prediction models (SNR) Signal-to-noise ratio SNR(I ) = µ/σ where µ = I w and σ = I 2 w µ and w is a 2D Gaussian weighting function (Ro) Roughness Index Ro(I ) = h1 I 1+ h2 I 1 I 1 where h 1 = [1, 1] and h 2 = h 1 T (ERo) Effective Roughness Index ERo(I ) = Ro(g I ) where g I = I µ

8 Non-uniformity prediction New prediction model (PNU) Perceptual non-uniformity index Natural Scene Statistic features Î = I µ σ + C H(i, j) = Î (i, j)î (i, j + 1) V(i, j) = Î (i, j)î (i + 1, j) D1(i, j) = Î (i, j)î (i + 1, j + 1) D2(i, j) = Î (i, j)î (i + 1, j 1) SVR trained to predict σ V. Feature ID Feature Description Computation Procedure f 1 - f 2 α and σ 2 GGD fit to MSCN coefficients f 3 - f 4 α and σ 2 differences Asymmetries in MSCN coefficients pp 1 - pp 16 α, η, left σ 2, right σ 2 AGGD fit to each pairwise product pd 1 - pd 14 Shape and Variance GGD fit to each pairwise log-derivative

9 Non-uniformity prediction performance ˆσ V PNU prediction SRCC=0.969 SRCC= SNR SRCC= Ro SRCC= ERo PNU SNR Ro ERo ˆσ V in presence of AWN PNU prediction SRCC=0.968 SRCC= SNR SRCC= Ro SRCC= ERo PNU SNR Ro ERo

10 Recall steerable pyramid decomposition Input First level decomposition (contrast enhanced) eero/steerpyr/

11 Gausian Scale Mixture (GSM) denoising model For a given pixel location x, subband level b, and orientation θ, vectors ḡ(x, θ, b) follow Gaussian scale mixture model ḡ(x, θ, b) = z(x, θ, b) γ(x, θ, b) where z is a positive random scalar estimated, using ḡ maximimum likelihood, to be ẑ ML = T Cγ 1 ḡ M and γ N (0, C γ ). Note C γ is an MxM matrix. Collecting 9 coefficients in the same band, 5 across neighboring orientations, and 1 from the parent yields vectors of size 15 (M = 15).

12 Gausian Scale Mixture (GSM) denoising model If an image is contaminated by an additive noise N, we have ḡ N (x, θ, b) = ḡ(x, θ, b) + N(x, θ, b) The best mean-squared-error (MSE) estimate of the true value of ḡ(x, y, θ, b) given ḡ N (x, y, θ, b) and z(x, y, θ, b) is ĝ(x, θ, b) = E [ḡ(x, θ, b) ḡ N (x, θ, b), z(x, θ, b)] = ẑc γ [ẑc γ + C N ] 1 ḡ N (x, θ, b) where ẑ is computed from the noisy image. To estimate all ĝ, 19 noise covariance matrices C N must be estimated using C γ+n = C γ + C N.

13 Prediction of non-uniformity covariance, Ĉ NU Observations of C NU for different σ V Normalized Matrix coefficient Normalized Matrix coefficient Normalized Matrix coefficient Normalized Matrix coefficient C(σ V,, 0) C(σ V, 0, 1) C(σ V, 0, 2) C(σ V, 0, 3) Prediction, Ĉ NU, as a function of σ V Predicted Matrix coefficient Predicted Matrix coefficient Predicted Matrix coefficient Predicted Matrix coefficient Ĉ NU (σ V,, 0) Ĉ NU (σ V, 0, 1) Ĉ NU (σ V, 0, 2) Ĉ NU (σ V, 0, 3)

14 Gaussian Scale Mixture Perceptual Pattern denoiser Extract NSS, predict σ V. Estimate 19 C NU matrices. Decompose input image into 19 subbands. Apply GSM denoising using C NU as the noise matrices. Reconstruct.

15 Other non-uniformity denoisers (TVNUC) Total variation non-uniformity correction Uses iterative and weighted smoothing Uses Ro index as a stopping criterion (MIRE) Midway infrared equalization Spatial de-flicker algorithm Uses column-wise histograms, computes an average column-wise emperical CDF, then inverts to get the corrected result.

16 Results Original TVNUC MIRE GSMPP

17 Results Model MSE PSNR SSIM NONE TVNUC MIRE GSMPP full GSMPP w/σ V GSMPP w/c NU SSIM NONE TVNUC MIRE GSMPP GSMPP w/σv GSMPP w/cnu

18 Conclusion Works well for vertical NU. Later study other noise patterns. Investigate algorithm complexity and performance limitations.

19 Questions? Comments? Todd Goodall