MYSTERY BEHIND SIMILARITY MEASURES MSE AND SSIM. Gintautas Palubinskas

Transcription

1 MYSTEY BEHID SIMILAITY MEASUES MSE AD SSIM Gintautas Palubinskas German Aerospace Center DL, emote Sensing Technolog Institute Oberpfaffenhofen, 834 Wessling, German ABSTACT Similarit or distance measures pla an important role in various pattern recognition applications such as classification, clustering, change detection, information retrieval, energ minimization and optimization problems We shall analze theoreticall the two most popular qualit measures MSE and SSIM used in image processing b showing their origin, similarities/differences and advantages/drawbacks Both measures depend on the same parameters: sample means, standard deviations and correlation coefficient It is shown that SSIM originates from two Dice measures and thus inherit their main drawback - dependence on the absolute mean and standard deviation values Similarl, MSE depends on the absolute standard deviation values A new similarit measure Composite qualit index based on Means, Standard deviations and Correlation coefficient (CMSC) is proposed inheriting advantages of the both measures but at the same time avoiding their drawbacks Index Terms Similarit, distance, Euclidian, Dice, composite, correlation coefficient, qualit index, image ITODUCTIO Similarit or distance measures are unavoidable in solving various signal/image processing problems such as restoration, de-noising, registration/matching, segmentation, classification, detection and recognition (for a surve see []) Usuall broadl recognized and accepted measure - Mean Squared Error (MSE) - is used ecent investigation [] shows its weakness in some applications eg visual perception of images A new Structural SIMilarit index (SSIM) proposed in [3-4] is spreading fast not onl in computer vision communit but also other communities such as remote sensing with a ver different tasks, eg pansharpening [5] The question is arising (despite recent publications [6-9]): can we simpl transfer SSIM to other applications requiring mainl relative comparison of data? To answer this question we have looked theoreticall what is reall behind MSE and SSIM measures This analsis allowed us to detect advantages and drawbacks of these two measures Moreover, this analsis resulted in the proposal of a new similarit measure - Composite qualit index based on sample moments (Mean, Standard deviation) and Correlation coefficient (CMSC), which inherits advantages of both measures at the same time avoiding their drawbacks Its variants CMSCam (averaging/multiplication), CMSCm (multiplication) and CMSCa (averaging) differ onl in how individual similarities are combined THEOY First, we shall introduce some notations used in this paper Let x xi i,, and i i,, denote two images or image patches or more generall signals to be compared, where x i, i - real numbers with a finite range of values min x i, i max (eg min=0 and max=55 for 8bit images), max min, - number of pixels/samples Distance or dissimilarit d is defined as a measure indicating how close/far apart are two samples/objects It exhibits high values for objects which are far from each other and low values for near objects For example, Euclidian measure is probabl the most popular distance measure The inverse measure to distance d is a similarit measure s, which exhibits high values for similar objects and low values for different objects Here we can mention a correlation coefficient as a most popular similarit measure For scaled/normalized measures hold the following relationships For distance d scaled to interval 0 d, eg using d norm ( d dmin ) /( dmax dmin ), the corresponding similarit is simpl equal to s d Analogousl, for similarit s normalized to interval 0 s the corresponding distance is simpl equal to d s Two or more similarit measures can be combined/composed b averaging, summation and multiplication operators For example, two distance measures d and d, each normalized to interval 0 d, d, first are transformed to individual similarities s d and s d, then a composite similarit is calculated b averaging s s ) / ( d ) / or multiplication s ave ( d s s ( d) ( d) s mult Due to 0 d, d the following relation holds smult save Of course, mixed composite measures are possible eg s ( s s) s3 All distances and similarities analzed in this work are summarized in Table MSE Mean squared error (MSE) is a ver popular distance measure (based on original data) and is defined as Proc ICIP, Paris, France, IEEE, pp , 04

2 d MSE i i i ( x ) We shall use the normalized version of MSE (nmse) in this work (, Table ) It was shown experimentall in [] that MSE can be rather poor in some cases especiall for visual image qualit perception Some properties of MSE were analzed, but it is still not clear the real reasons for such behavior of MSE In this paper we shall tr to look theoreticall what is reall behind MSE Using sampled statistics: x, x and correlation coefficient we can rewrite d nmse (, Table ) as alread proposed in [8] (3, Table ) We see that d nmse is a sum of two distances: d normalized squared Euclidian measure for means (, Table ) and d Because of 0 ( d d) the similarit for d nmse can be defined as ( d d ) s nmse SSIM Structural similarit (SSIM) measure proposed in [3-4] can be written as a composite measure (multiplication) of three similarities: luminance (mean) s, contrast (standard deviation) s and correlation coefficient s3 (3, Table ) Constants for avoiding singular case (zero in denominator) are omitted for simplicit We see that it is based on the same sample moments and correlation coefficient as MSE So this is the first observation/propert or mster revealed about MSE and SSIM: both measures are composed of the same parameters which are onl combined in a different wa We can see easil that the first two similarities of SSIM are Dice measures (, Table ), which were independentl introduced b botanists Dice [0] and Sørensen and can be easil extended to vector data [] The authors of SSIM never mention such origin of SSIM [6-7] So this is the second observation/propert or mster revealed about SSIM: SSIM is composed of two Dice measures one for means and another one for standard deviations In Fig Dice measure values are presented for nonnegative data values 0 x 55 and 0 55 So the following two observations about Dice measure are valid: it is unstable around zero point (0,0) and it cannot be used as a similarit measure for data with different signs So the third observation for SSIM is its instabilit around zero point (0,0) and the fourth one it can be used onl for data of the same sign The authors of SSIM solve these problems b introducing small constants and restricting the usage to non-negative data onl, respectivel It can be seen from (, Table ) directl that eg for x 0 and an 0,,55 the Dice similarit measure is equal to 0 Moreover, it depends not onl on the relative difference of two values (what is expected from the measure) but also on the absolute values as can be seen in Fig for different line profiles (Fig ) Figure Dice similarit measure for x 0,,55 0,,55 and Figure Line profiles of Dice and normalized squared Euclidian (nse) similarit measures for three lines (see Fig ) in dependence of mean difference ed color stands for large mean values, blue middle, green small and black dotted line all means (nse) Line profiles of Dice and normalized squared Euclidian (, Table ) similarities for constant mean difference 00 in dependence on the absolute mean value,,55 x x (ellow line in Fig ) are plotted in Fig 3 As expected nse exhibits a constant value whereas the Dice measure increases with the increase of the absolute value of one of the parameters The fifth observation for Dice measure and thus for SSIM too is that it depends on the absolute values of input parameters First, it is insensitive at all if one of the parameters is equal 0 Secondl, its sensitivit is decreasing b the increase of absolute parameter values

3 Figure 3 Line profiles of Dice and normalized squared Euclidian (nse) similarities for constant mean difference x 00 and x,,55 (ellow line in Fig ) Usuall objective similarit measure should be dependent onl on the relative difference of the two parameters and independent of the absolute parameter values ormalized squared Euclidian distance for means (, Table ) eg defined as 0 d nse and its corresponding similarit measure snse dnse depends onl on the relative mean difference Thus it fulfills objective measure requirement what is illustrated in Fig 4 Moreover, it is easil seen that it can be defined for real data (independent of sign) 3 DISCUSSIO Here we shall perform analsis of composite similarit measures s SSIM, s nmse based on the information presented in Table First, we observe that under the assumptions of x, x the SSIM similarit is simpl reduced to the correlation coefficient s SSIM Secondl, we see that under following assumptions of x, x, the nmse similarit is also reduced to the correlation coefficient s nmse what shows an identit of both measures under special (quite similar) conditions Analsis of nmse second similarit term based on distance d (3, Table ) shows its dependence on the absolute values of two standard deviation values (see Fig 5) Onl for it converges to a squared Euclidian measure as can be seen in Fig 5 and Table This last observation has served as an inspiration for a new composite similarit measure CMSC which exploits advantages of both MSE and SSIM measures at the same time avoiding their drawbacks Figure 4 Euclidian similarit measure (nse) for x 0,,55 and 0,,55 4 EW SIMILAITY MEASUE CMSC After analzing two most popular composite similarit measures MSE and SSIM we can propose a new Composite image qualit measure based on Means, Standard deviations and Correlation coefficient (CMSC) and consisting of the three components: two normalized squared Euclidian measures and one correlation coefficient (3-33, Table ) Depending on the wa of combination three versions are possible CSMCam (3) uses averaging and multiplication of individual similarities, CSMCm (3) - onl multiplication of similarities and CMSCa (33) onl averaging of similarities We have to note, that averaging of similarities gives a possibilit for weighting It is eas to prove that for the normalization of the second distance d (3, Table ) including standard deviations a two times smaller constant / can be used All proposed measures are free of drawbacks of MSE and SSIM and thus are more suitable as objective similarit/qualit measures not onl for the images but an signals 3 COCLUSIOS Theoretical analsis of MSE and SSIM similarit measures is performed, resulting in a new composite similarit measure CMSC Preliminar experiments on simulated and real data covering various image distortions: mean shift, contrast change, various tpes of noise (additive Gaussian, multiplicative speckle and impulsive salt&pepper) and blurring support theoretical results Further research can be conducted towards introducing additional gradient, texture, spectral information for CMSC, similarl as it was alread proposed in [] for SSIM or higher sample moments: skewness and kurtosis

4 0 0 5 Figure 5 nmse similarit measure for x, x 0,,55 and 0,,55 in dependence of Table Summar of distance and similarit measures used in this work is a normalization constant eg =55 for 8bit data Distance d Distance d Dist Comb Similarit d tpe 3 Dice measure x x x x Correlation coefficient CC Structural similarit SSIM x x Mult x x x x x x 4 SSIM x, x 0 0 Mult ormalized squared x d Euclidian nse ormalized MSE (original d data) nmse ( x i i ) i 3 ormalized MSE (sample - Sum moments) nmse x x x ( d d ) 4 nmse 0 ( ) - Sum x, x, 5 nmse 0 - Sum x x ( d ) d 6 nmse x ( x ) - Sum ( d d) 3 Composite qualit index (Means, Standard deviations, x ( x ) Ave/ ( d ) Mult d Correlation coefficient) / CMSCam 3 CMSCm x ( x ) Mult ( d ) ( d) / 33 CMSCa x ( x ) Ave d d / 3 3 3

5 4 EFEECES [] S-H Cha, Comprehensive surve on distance/similarit measures between probabilit densit functions, International Journal of Mathematical Models and Methods in Applied Sciences, (4), , 007 [] Z Wang, and A C Bovik, Mean squared error: love it or leave it? A new look at signal fidelit measures, IEEE Signal Processing Magazine, 6(), 98-7, 009 [3] Z Wang, and A C Bovik, A universal image qualit index, IEEE Signal Processing Letters, 9(3), 8 84, 00 [4] Z Wang, A C Bovik, H Sheikh, and E P Simoncelli, Image qualit assessment: from error visibilit to structural similarit, IEEE Trans Image Processing, 3(4), 600-6, 004 [5] G Palubinskas, Fast, simple and good pan-sharpening method, Journal of Applied emote Sensing, 7(), pp -, 03 [6] M P Sampat, Z Wang, S Gupta, A C Bovik, and M K Marke, Complex wavelet structural similarit: a new image similarit index, IEEE Trans Image Processing, 8(), , 009 [7] D Brunet, E Vrsca, and Z Wang, On the mathematical properties of the structural similarit index, IEEE Trans Image Processing, (4), , 0 [8] A Horé, and D Ziou, Image qualit metrics: PS vs SSIM, Proc 0 th Int Conference on Pattern ecognition ICP, 3-6 August, 00, Istanbul, Turke, pp , 00 [9] Dosselmann, and XD Yang, A comprehensive assessment of the structural similarit index, SIViP, 5, pp 8-9, 0 [0] L- Dice, Measures of the amount of ecologic association between species, Ecolog, 6, 97-30, 945 [] E Blasch, X Li, G Chen, and W Li, Image qualit assessment for performance evaluation of image fusion, Proc of th International Conference on Information Fusion, 30 June - 3 Jul, 008, Cologne, German, pp -6, 008