No-reference (N-R) image quality metrics.

Transcription

1 No-reference (N-R) image quality metrics. A brief overview and future trends G. Cristóbal and S. Gabarda Instituto de Optica (CSIC) Serrano 121, Madrid, Spain gabriel@optica.csic.es Lisbon, March 10-11,

2 Contents Motivation IQ modeling. Measuring image quality Overview of N-R quality metrics N-R quality metric based on image anisotropy Conclusions 2

3 MSE as a quality reference All pairs have the same MSE=200, but differences in quality are quite dramatic! 3

4 IQ modeling Difficulties Perceptual by nature hard to measure in a standarized way Differ vastly between different domains 3-step approach (Bar82) & IQ circle Identification perceptual dimensions (attributes) Objective measures correlating with them Combination of attributes to predict overall quality Bartleson, J., The combined influence of sharpness and graininess 4 on the quality of colour prints, J. Photographic Science 30 (1982)

5 Measuring image quality Engeldrum, P.E. A theory of image quality: the image quality circle, J.Imaging and Techology 48,

6 IQ approaches Two approaches Bottom-up: model HVS to predict IQ Reproducible research. Perceptual quality Matlab toolbox: 6

7 IQ approaches Two approaches Top-down: high level assumptions on the method used by the HVS to develop a quality metric. Most freq. approach due to low complexity. Only the most relevant features are taken into account to evaluate IQ. 7

8 Overview N-R quality In the Fourier/wavelet domain (Saghri89) IQM based on HVS for compressed images (DCT) (Nill92) IQM derived from image power spectrum (Wang04) evaluating image blur using phase coherence (Firestone91) frequency threshold (Marichal99) histogram frequency (Shaked05) Shaked-Tastl metric (high pass/band pass ratio) (Ferzli05) Noise immune sharpness (NIS) (wavelet domain) (Saghri89) Saghri, J.A. et al, Image quality measure based on a human visual system model, Opt. Eng 28, (Nill92) Nill, N.B. Objective image quality measure derived from digital image power spectra, Opt. Eng (Wang04) Wang, Z. Local phase coherence and the perception of blur, NIPS, 16 (Firestone91) Firestone, L. et al. Comparison if autofocus methods for automated microscopy,cytometry 12: (Marichal99) Marichal,X. et al, Blur determination in the compressed domain using DCT information, ICIP 2: (Shaked05) Shaked, D. et al, Sharpness measure: towards automatic image enhancement, ICIP 81: (Ferzli05) Ferzli, R. et al, No-reference objective wavelet based noise immune image sharpeness metric, ICIP 1:

9 Overview N-R image quality In the spatial domain (Erasmus82) variance metric (Batten00) autocorrelation/derivative based (Caviedes04) kurtosis (Firestone91) histogram threshold (Chern01) histogram entropy (Ong03) blur metric (Canny edge detector) (Ferzli09) just noticeable blur (Erasmus82) Erasmus, S. et al. An automatic focusing and astigmatism..., J. Microscopy 127: (1982) (Batten00) Batten, C.F, Autofocusing and astigmatism correction in the scanning EM, MSc Thesis, Cambridge (Caviedes04) Caviedes, J. et al, A new sharpness metric based on local kurtosis, edge and energy information, Signal Proc.Imag. Comm. 2: (Firestone91) Firestone, L. et al. Comparison if autofocus methods for automated microscopy,cytometry 12: (Chern01) Chern, N.K. et al. Practical issues in pixel-based autofocusing for machine vision,icip 3: (Ong03) Ong, E.P, et al. No-reference quality metric,icip (Ferzli09) Ong, E.P, et al. No-reference objective wavelet noise immune image sharpness metric,icip 1:

10 Overview N-R image quality In the spatial domain (cont.) (Marzilinano02) edge bluriness (Ong03) bluriness gradient s direction (Li02) NR image quality (blur, noise and block artifacts) (Brandao08) NR for quantization noise lossy encoding (JPEG, MPEG) using DCT statistics (Gabarda07) N-R based on anisotropy (Cohen10) MIS modified image spectrum Marziliano, P. et al. A no-reference perceptual blur metric, ICIP 3:57-80 (2002) Ong, et al, A no-reference QM for measuring image blur, IEEE Signal Proc. and applications 1: (2003) Li, X. Blind image quality assessment, ICIP 1: (2002) Brandao, T. et al. No-reference image quality assessment based on DCT domain statistics, Signal Processing 88: (2008) 10 Gabarda, S. et al, Blind image quality assessment through anisotropy, J.Opt.Soc.Am.A 24:B42-B51 (2007) Cohen, E. et al, No-reference assessment of blur and noise impacts on image quality, SIViP 4: (2010)

11 Image anisotropy Diversity of edges and textures are responsible for the anisotropy of images Entropy evaluated at different directions will provide a measure of anisotropy Edges are responsible of the main differences on image entropy (pixel level) How the anisotropy is measured? The anisotropy is measured as the expectation value of the entropy for different orientations 11

12 Image anisotropy Wigner Distribution Rényi entropy Shannon entropy Entropy histograms Point-wise Rényi entropy 12 Quantum normalization

13 Image anisotropy 13 Gabarda, S. and Cristobal, G. Blind image quality assessment through anisotropy, J. Opt. Soc. Am. A 24:B42-B51 (2007)

14 Image anisotropy 14 Gabarda, S. and Cristobal, G. Blind image quality assessment through anisotropy, J. Opt. Soc. Am. A 24:B42-B51 (2007)

15 Image anisotropy Entropy variance as a N-R quality metric In focus, noise free images show an steady maximum Anisotropy decreases as more degradations are added to the image Reproducible research: Gabarda, S. and Cristobal, G. Blind image quality assessment through 15 anisotropy, J. Opt. Soc. Am. A 24:B42-B51 (2007)

16 Future research No-reference IQA is still in its infancy Focus should be concentrated on specific applications Related area: parameter optimization 2 Extension to include other perceptual cues Osaka plot e.g. color, depth, etc Multidimensional scaling 1, e.g. Osaka plot (a graphical IQ: the area and shape gives info about the type and amount of degradation) 1 Kayargadde, V. and Martens, J.B. Perceptual characterization of images degraded by blur and noise, J.Opt. Soc. Am. A, 13: (1996) 2 Zhu,X. and Milanfar,P Automatic parameter selection for denoising algorithms 16 using a no-reference measure of image content, IEEE TIP, 19: (2010)

17 Open issues Images with non-uniform saliency content: ROI or foveated regions 1 A content-based quality metric? 2 Substantial/correlated noise 3 3 (a) σ = 20; (b) variational; (c) Richard-Lucy (a) SAR; (b) denoised#1; (c) denoised#2 1 Narvekar, N.D. et al A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection,qomex, (2009) 2 Gang, J.L. et al Image coding quality assessment using fuzzy integrals with a three component image model, IEEE Trans. on Fuzzy Systems, 12:99-106(2004) 3 17 Zhao, W. et al Image restoration under significant additive noise, IEEE Signal Proc. Letters, 14: (2007)