IQA [, ] = 42. Information Content Redundancies of Image Quality Assessments. Jens Preiss and Philipp Urban

Transcription

1 Information Content Redundancies of Image Quality Assessments Jens Preiss and Philipp Urban IQA [, ] = Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss

2 Table of Contents Introduction Redundant Information Content Evaluation (RICE) Introduction Methods Evaluated Image Quality Assessments (IQAs) Color Image Difference Database Investigation of the Proposed RICE Methods Summary and Outlook Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 2 / 17

3 Introduction Image processing In image processing the assessment of image quality is essential Visual experiments are exhausting and expensive A computed image quality assessment (IQA) should accurately predict human judgments Reference image Distortion Gaussian noise JPEG compression Gaussian blur Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 3 / 17

4 Introduction Image quality assessment There are three possible types of IQAs: No-reference IQA: No information about the reference image is given Reduced-reference IQA: Partial information is available Full-reference IQA: The reference image is completely present A full-reference IQA method compares a distorted with a reference image and transforms them into a single number Famous representatives are the mean squared error (MSE) and the peak signal-to-noise ratio (PSNR) IQA [, ] = Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 4 / 17

5 Introduction The need for the comparison of IQA methods The human visual system (HVS) is not fully understood The existing IQAs do not correlate perfectly with human perception Each IQA method extracts and compares different information contents from the images, such as Structure Gradients Color attributes (Lightness, Chroma, hue, )... The detection of redundant information content extracted by different IQA algorithms provides a useful tool for developing IQAs Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 5 / 17

6 Redundant Information Content Evaluation Introduction The redundant information content evaluation (RICE) compares the predictions of several IQA methods IQAs that rely on similar information are likely to show a higher correlation than IQAs that are based on mutually different information A color image database is necessary to apply RICE The predictions of every IQA algorithm are written into a column vector RICE can be applied to a single IQA pair as well as to a whole group of IQAs Image Database Image Quality Asessments IQA algorithms img 1 img 2. img N img 1. img 2 img N IQA 1 IQA Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 6 / 17

7 Redundant Information Content Evaluation Method 1 1) Condition number method (CN) The condition number Cond(X) of a matrix X is the ratio of its largest and its smallest singular value Here, the matrix X is created from n different IQA vectors: X = [IQA 1, IQA 2,, IQA n ],n ϵ {2, 3, } A high condition number means that the points are not uniformly distributed within the n-dimensional space Hence, the n IQAs contain some redundant information CN = Cond(X) n Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 7 / 17

8 Redundant Information Content Evaluation Method 2 2) Determinant of covariance method (DC) The covariance matrix Cov(X) specifies the variation between the involved IQAs The determinant Det(X) is the product of the eigenvalues A large DC value implies a high variation of the involved IQAs RICE indicates low redundant information content DC = 10 (Det Cov X ) 1/n Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 8 / 17

9 Redundant Information Content Evaluation Method 3 3) Partial correlation method (PC) The partial correlation r AB.C is a correlation between two vectors A and B with respect to a set of control vectors C Vectors A and B are projected onto the vector subspace orthogonal to the control vectors C The information of the control vectors is removed A high PC value indicates a lot of redundant information r AB.C = r AB r AC r BC 1 r AC 2 1 r BC Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 9 / 17

10 Redundant Information Content Evaluation Method 4 4) Intraclass correlation method (ICC) The intraclass correlation is a modified Pearson correlation The mean of each single IQA is replaced by the mean of all IQAs A low ICC indicates a high variance of the predictions Therefore, it indicates a low redundancy  = 1 2N N n=1 A n,1 + A n,2 ICC = N 2 n=1 A n,1  A n,2  N n=1 A n,1  2 N + n=1 A n,2  Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 10 / 17

11 Redundant Information Content Evaluation Method 5 5) Principle angle method (PA) The principle angle specifies the minimal angle between two vector subspaces Here, we calculate the PA value of A = IQA 1 and a vector subspace spanned by B = {IQA 2,, IQA n } U is a orthonormal basis of this vector subspace as a matrix Svd(X) is the singular value decomposition of X A small principle angle originates from two close vector subspaces A lot of information content is redundant PA = cos 1 (Svd At A U ) Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 11 / 17

12 Evaluated Image Quality Assessments A set of common IQAs is provided by the MeTriX MuX package Additionally, two PSNR extensions are evaluated Number IQA MSE PSNR SSIM MSSIM VSNR VIF VIFP UQI Number IQA IFC NQM WSNR SNR PSNR-HVS PSNR-HVSM Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 12 / 17

13 Color Image Difference Database The publicly available Tampere Image Database 2008 is chosen 25 reference images 1700 distorted images (17 distortions with 4 levels) The mean opinion score (MOS) of the perceived image difference was obtained from 838 subjects Reference image Gaussian noise Gaussian blur JPEG compression MOS = PSNR = SSIM = VIF = MOS = PSNR = SSIM = VIF = MOS = PSNR = SSIM = VIF = Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 13 / 17

14 Investigation of the Proposed RICE Methods Analysis of the functionality Three model IQAs were computed for every image Peak signal-to-noise ratio (PSNR) as an exemplary IQA PSNR with low random noise added Uniformly distributed random number That way, redundant (PSNR vs. PSNR with noise) and non-redundant (PSNR vs. random number) information content is simulated The RICE values can roughly be seen as the lower and upper limits Method CN DC PC ICC PA PSNR vs. PSNR with noise High Low High High Low PSNR vs. random number Low High Low Low High Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 14 / 17

15 Investigation of the Proposed RICE Methods Results of a pairwise comparison The most redundant information can be found between IQAs based on similar approaches, such as {PSNR,SNR}, {VIF,VIFP} The RICE methods differ widely for choosing the IQA pair with lowest redundant information PSNR and MSE should extract the same information Only the PC and ICC methods detect this relation These two methods should be preferred for a pairwise analysis PSNR = 20 log I max MSE I max = maximal signal intensity Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 15 / 17

16 Summary and Outlook Five different methods for detecting redundant information content are introduced The functionality is verified Approximate upper and lower limits are provided RICE detects redundancies between similar IQA methods More applications of RICE are being investigated Pooling of IQAs into different types Sorting of extracted image difference features for blending Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss 16 / 17

17 Thank you for your kind attention Dipl.-Phys. Jens Preiss Technische Universität Darmstadt Institut für Druckmaschinen und Druckverfahren Magdalenenstr Darmstadt Institut für Druckmaschinen und Druckverfahren Dipl.-Phys. Jens Preiss