An Efficient Image Similarity Measure Based on Approximations of KL-Divergence Between Two Gaussian Mixtures

Transcription

1 An Efficien Image Similariy Measure Based on Approximaions of KL-Divergence Beween Two Gaussian Mixures Jacob Goldberger CUTe Sysems Ld Tel-Aviv, Israel Shiri Gordon Faculy of Engineering Tel-Aviv Universiy, Israel Hayi Greenspan Faculy of Engineering Tel-Aviv Universiy, Israel Absrac In his work we presen wo new mehods for approximaing he Kullback-Liebler (KL) divergence beween wo mixures of Gaussians The firs mehod is based on maching beween he Gaussian elemens of he wo Gaussian mixure densiies The second mehod is based on he unscened ransform The proposed mehods are uilized for image rerieval asks Coninuous probabilisic image modeling based on mixures of Gaussians ogeher wih KL measure for image similariy, can be used for image rerieval asks wih remarkable performance The efficiency and he performance of he KL approximaion mehods proposed are demonsraed on boh simulaed daa and real image daa ses The experimenal resuls indicae ha our proposed approximaions ouperform previously suggesed mehods 1 Inroducion Image maching is an imporan componen in many applicaions ha require comparing images based on heir conen The mos imporan examples are image daa base rerieval sysems Image maching echniques can be classified according o wo parameers The firs is he image represenaion mehod and he second is a definiion of appropriae disance measure o compare beween images in he seleced represenaion space A sandard represenaion mehod is color hisogram The advanages and disadvanages of his mehod are well sudied and many variaions exis A variey of measures have been proposed for dissimilariy beween wo hisograms (eg χ 2 saisics, KLdivergence) [9] An alernaive image represenaion is a coninuous probabilisic framework based on a Mixure of Gaussians model (MoG) [1] [3] The KL-divergence is a naural dissimilariy measure beween wo images represened by mixure of Gaussians However, since here is no closed form expression for he KL-divergence beween wo MoGs, compuing his disance measure is done using Mone-Carlo simulaions Mone-Carlo simulaions may cause a significan increase in compuaional complexiy which can be a maor drawback in real conen based image rerieval sysems In his work we aim o solve his drawback by presening wo new mehods for he approximaion of he KL-divergence beween wo mixures of Gaussians The firs one is an improved version of he approximaion suggesed by Vasconcelos [10] The mehod is based on maching beween he Gaussian elemens of he wo MoG densiies and on he exisence of a closed form soluion for he KL-divergence beween wo Gaussians The second mehod demands a lile more processing ime bu gives much beer resuls I is based on he unscened ransform inroduced by Juiler and Uhlmann [4] The res of he paper is organized as follows Image modeling via a mixure of Gaussians is reviewed in secion 2 In secion 3 we propose an easily compued approximaion of he KL-disance beween wo mixures of Gaussians In secion 4 we propose an alernaive approximaion based on he unscened ransform mechanism In secion 5 we compare boh he performance and he compuaional efficiency of he various KLdivergence approximaions The comparison is performed on boh simulaed daa and MoG densiies obained from modeling real images 2 Image modelling via MoG We model an image as a se of coheren regions Each homogeneous region in he image plane is represened by a Gaussian disribuion, and he se of all he regions in he image is represened by a Gaussian mixure model The image is herefore viewed as an insance of he generaive mixure of Gaussians model We focus here on he color feaure In paricular we model each image as a mixure of Gaussians in he color feaure space I should be noed ha he represenaion model is a general one, and can incorpo-

2 he Mone-Carlo based mehod 3 Maching based approximaion Figure 1 Inpu image (lef) Image modeling via a mixure of Gaussians (cener) Image segmenaion using he learned model (righ) rae any desired feaure space (such as exure, and shape) or combinaion hereof Color feaures are exraced by represening each pixel wih a hree-dimensional color descripor in a seleced color space In his work we choose o work in he (L,a,b) color space which was shown o be approximaely percepually uniform by Wyszecki and Siles [11], hus disances in his space are meaningful In order o include spaial informaion, he (x, y) posiion of he pixel is appended o he feaure vecor Including he posiion enables a localized represenaion Each pixel is represened by a five-dimensional feaure vecor (L,a,b,x,y) Pixels are grouped ino homogeneous regions, by grouping he feaure vecors in he seleced five-dimensional feaure space The Expecaion-Maximizaion (EM) algorihm is used o deermine he maximum likelihood parameers of a mixure of k Gaussians The Minimum Descripion Lengh (MDL) principle serves o selec among values of k In our experimens, k ranges from 4 o 8 Figure 1 shows an example of learning a MoG model for a given inpu image In his visualizaion he Gaussian mixure is shown as a se of ellipsoids Each ellipsoid represens he suppor, mean color and spaial layou, of a paricular Gaussian in he image plane Using he learned model (cener) each pixel of he original image is affiliaed wih he mos probable Gaussian, providing for a probabilisic image segmenaion (righ) Given he represenaion of an image by a densiy funcion, we can define a similariy measure beween wo images as he Kullback-Liebler divergence [8] beween he respecive densiy models of he images In he case of discree (hisogram) represenaions, he KL-divergence can be easily obained However, here is no closed-form expression for he KL-divergence beween wo mixures of Gaussians We can use, insead, Mone-Carlo simulaions o approximae he KL-divergence beween wo MoGs, f and g: KL(f g) = f log f g 1 n =1 log f(x ) g(x ) such ha x 1,, x n are sampled from f(x) The problem wih his approach is ha i can no be used in image rerieval sysems due o is large complexiy In he following secions we presen wo alernaive deerminisic approximaions ha can be compued much more efficienly han Le f(x) = n α if i (x) and g(x) = m =1 β g (x) be wo mixure densiies such ha α = {α 1,, α n } and β = {β 1,, β m } are discree disribuions and f i,g are arbirary coninuous densiies Assume ha i is no possible o obain a closed-form expression for he Kullback- Liebler divergence KL(f g) bu here is an analyical way o compue he KL-divergence beween each pair of componens f i,g In his secion we presen and moivae an approximaed expression for KL(f g) based on he KLdivergence beween he mixures componens KL(f i g ) The convexiy of he KL-divergence [2] implies ha: KL( α i f i β g ) α i β KL(f i g ) =1 i, The resulan weighed average approximaion is one possible approximaion for he KL-divergence This approximaion is oo crude, however, especially when each mixure densiy is composed of disribuions which are unimodal and he modes are far apar A beer approximaion can be obained by maching a single componen of g(x) o each componen of f(x) A maching funcion beween he componens of f(x) and g(x) is needed We propose he following, maching-based approximaion: KL(f g) = α i f i log f α i f i log g α i f i log α i f i = α i max f i log β g ( α i min KL(f i g ) + log α ) i β This approximaion is based on he assumpion ha he erm in he sum β g ha is mos proximal o f i dominaes he inegral f i log g The proposed approximaion yields a maching funcion beween elemens of f and elemens of g Define he maching funcion π : {1,, n} {1,, m} beween he componens of f(x) and he componens of g(x) as follows: π(i) = arg min (KL(f i g ) log β ) (1) Uilizing π we can wrie he suggesed approximaion in he following way: ( KL mach (f g) = α i KL(f i g π(i) ) + log α ) i β π(i) (2)

3 A mixure model f(x) = n α if i (x) can be viewed as a wo sep model In he firs sep we sample a laen discree random variable I according o p(i = i) = α i In he second sep we sample he observed coninuous random variable x according o f(x i) =f i (x) The complee daa is he union of he laen and he observed daa The complee daa densiy funcion associaed wih he mixure densiy f(x) is f(i, x) =f(i)f(x i) =α i f i (x) Noe ha if he laen variables of f and g share he same alphabe (ie n = m) hen he KL-divergence beween he complee daa densiies associaed wih f and g is well defined and has he following closed form expression: Figure 2 A possible mach beween a mixure of 3 Gaussians and a mixure of 4 Gaussians Figure 2 shows a possible maching funcion Several componens of f may be mached o he same componen of g There can be componens of g ha no componen of f is mached o We focus nex on he image rerieval applicaion The following siuaion is common in image rerieval sysems: Given a query mixure densiy f and a family of mixure densiies {g }, we wan o affiliae f o he densiy ha minimizes he disance crierion KL(f g ) (for a conen-based image rerieval sysem in which mixure densiies are used as he image models and he KL measure is used as he disance measure, see [3]) Since α i arg min KL(f g ) = arg max f i log g (3) for each MoG g, we need only o evaluae f log g We can apply he approximaion: f i log( β g ) max f i log β g =1 o obain a lower bound approximaion: f log g = α i f i log g α i f i log(β π(i) g π(i) ) where π is he maching funcion defined in expression (1) The suggesed approximaion will be usified empirically in secion 5 As a moivaion for he approximaion, we show nex ha he proposed approximaion (Equaion 2) can be viewed as a KL-divergence beween he complee versions of he wo MoGs KL(f(i, x) g(i, x)) = KL(f(i) g(i)) +KL(f(x i) g(x i)) = KL(α β) + α i KL(f i g i ) such ha: KL(α β) = α i log α i β i The chain rule for relaive enropy [2] implies ha: KL(f(x) g(x)) KL(f(i, x) g(i, x)) Thus we obain an upper bound for KL(f g) Since he MoG g(x) is invarian o a permuaion of he alphabe of he hidden random variable we can obain a igher bound: KL(f(x) g(x)) min s α i (KL(f i g s(i) ) + log α i ) β s(i) such ha he minimizaion is performed over all he n! permuaions on he se {1,, n} This approximaion, which is suiable only for he special case n=m, can be compued by he assignmen algorihm [7] whose complexiy is high (O(n 3 )) We reurn o he general case where g = m =1 β g Le π be he maching funcion defined in expression (1) We can build a new mixure densiy: g π (x) = 1 β π(i) g π(i) (x) C π such ha C π is he normalizaion scalar n β π(i) The MoG g π is a mixure densiy composed of he componens of g and i has he same number of componens as f(x) Sandard informaion heory manipulaions reveal ha he proposed approximaion (Equaion 2) can be rewrien in he following way: KL mach (f g) =KL(f(i, x) g π (i, x)) log(c π ) such ha f(i, x) is he densiy of he complee daa including he hidden discree variable of he mixure densiy, ie f(i, x) =α i f i (x) and g π (i, x) = 1 C π β π(i) g π(i) (x)

4 Therefore, he proposed approximaion is based on wo principles The firs one is a maching beween each componen of f(x) o one of he componens of g(x) The maching funcion ensures ha he hidden variables of he wo mixure models are defined on he same alphabe such ha i is meaningful o consider he KL-divergence beween he complee daa version of he densiies The second poin in he suggesed formula is approximaing he disance beween wo mixure densiies by he disance of he densiy funcions of he associaed complee daa densiies So far we developed an approximaion mehod for a general mixure model We shall now concenrae on he case which is mixure of Gaussians (MoG) The KL-divergence beween he Gaussians N(µ 1, Σ 1 ) and N(µ 2, Σ 2 ) is: 1 2 (log Σ 2 Σ 1 +Tr(Σ 1 2 Σ 1)+(µ 1 µ 2 ) T Σ 1 2 (µ 1 µ 2 )) (4) Given wo mixure of Gaussians f = α i N(µ 1,i, Σ 1,i ) and g = β N(µ 2,, Σ 2, ) =1 we can plug expression (4) ino approximaion (2) o obain he approximaion of he KL-divergence for he case of MoG Anoher approximaion of he KL-divergence beween wo MoGs was suggesed by Vasconcelos [10] The mehod is similar o he one presened in his secion The only difference is ha he maching funcion π beween he elemens of he wo MoGs, used in [10], is based on he Mahalanobis disance: ( π(i) = arg min (µ1,i µ 2, ) T Σ 1 2, (µ 1,i µ 2, ) ) (5) where as in our approach: (1 π(i) = arg min 2 (log Σ 2, Σ 1,i + Tr(Σ 1 2, Σ 1,i)+ (µ 1,i µ 2, ) T Σ 1 2, (µ 1,i µ 2, )) log β ) In secion 5 we empirically compare he performance of hese wo varians 4 Unscened Transform based approximaion The maching based mehod approximaes well he KLdivergence if he Gaussian elemens are far apar However, if here is a significan overlap beween he Gaussian elemens, hen he mach of a single componen of g(x) wih each componen of f(x) becomes less accurae Replacing he deerminisic maching by a sochasic one doesn help since we can easily observe ha he maching approximaion (2) can be wrien as: KL mach (f g) = min Ψ =1 α i Ψ i (log α i β +KL(f i g )) such ha Ψ is a n m sochasic marix, ie he minimizaion over all he sochasic marices yields a deerminisic maching To handle overlapping siuaions we propose anoher approximaion based on he unscened ransform The unscened ransformaion is a mehod for calculaing he saisics of a random variable which undergoes a non-linear ransformaion [4] I is successfully used for nonlinear filering The Unscened Kalman filer (UKF) [5] is more accurae, more sable and far easier o implemen han he exended Kalman filer (EKF) In cases where he process noise is Gaussian i is also beer han he paricle filer which is based on Mone-Carlo simulaions Unlike he EKF which uses he firs order erm of he Taylor expansion of he non-linear funcion, he UKF uses he rue nonlinear funcion and approximaes he disribuion of he funcion oupu In his secion we show how we can uilize he unscened ransform mechanism o obain an approximaion for he KL-divergence beween wo MoGs Le x be a d- dimensional normal random variable x f(x) =N(µ, Σ) and le h(x) :R d R be an arbirary non-linear funcion We wan o approximae he expecaion of h(x) which is f(x)h(x)dx The unscened ransform approach is he following A se of 2d sigma poins are chosen as follows: x k = µ +( dσ) k k =1,, d x d+k = µ ( dσ) k k =1,, d such ha ( Σ) k is he k-h column of he marix square roo of Σ Le UDU T be he singular value decomposiion of Σ, such ha U = {U 1,, U d } and D = diag{λ 1,, λ d } hen ( Σ) k = λ k U k These sample poins compleely capure he rue mean and variance of f(x) (seefigure3) The uniform disribuion over he poins {x k } 2d k=1 has mean µ and covariance marix Σ Given he sigma poins we define he following approximaion: f(x)h(x)dx 1 2d 2d k=1 h(x k ) (6) Alhough his approximaion algorihm resembles a Mone- Carlo mehod, no random sampling is used hus only a small number of poins are required The basic unscened mehod can be generalized The mean of he Gaussian disribuion µ can be also included in he se of sigma poins Scaling parameers can provide an exra degree of freedom o conrol he scaling of he sigma poins furher or owards µ

5 is o find: arg min KL(f g ) = arg max f log g Figure 3 The sigma poins of he unscened ransform [6] In he implemenaion presened in his paper in which he dimensionaliy of he disribuions is five (see secion 2), including µ in he se of sigma poin did no cause any improvemen in performance I can be verified ha if h(x) is a quadraic funcion hen he approximaion is precise For example in he case of h(x) = log N(µ 2, Σ 2 ), h(x) is a quadraic funcion of x Hence expression (6) is a mehod, alernaive o expression (4), o compue he exac KL-divergence beween wo Gaussian disribuions In he case ha h(x) is he log densiy funcion of MoG, expression (6) is an approximaion Given wo mixures of Gaussians: f = α i N(µ 1,i, Σ 1,i ) and g = β N(µ 2,, Σ 2, ) =1 he approximaion of f log g based on he unscened ransform is: such ha: 1 2d 2d α i k=1 log g(x i,k ) x i,k = µ 1,i +( dσ 1,i ) k k =1,, d, (7) x i,d+k = µ 1,i ( dσ 1,i ) k k =1,, d 5 Experimenal evaluaion In order o compare he accuracy of he proposed approximaions as well as heir processing efficiency we conduced he following simulaion of a rerieval ask based on image similariy In each rerieval session we sample a random MoG f as a query obec and four oher random MoGs {g } as a daa-se The ask is o find for a given f, he member of he daa-se ha is mos similar o i, ie, he rerieval ask Gaussian mixure models were randomly sampled according o he following rules The number of Gaussians wihin he MoG was chosen uniformly in he range 4-8 The dimension of all he Gaussians was 5 For each Gaussian N(µ, Σ), µ was sampled from N(0,I) and Σ was sampled from he Wishar disribuion as follows The enries of a marix A 5 5 were independenly sampled from N(0, 1) and we se Σ=ɛAA T The parameer ɛ conrols he size of he covariance marices As we decrease ɛ, he Gaussians ha compose he MoG are furher apar Hence approximaing he KL-divergence using maching beween he Gaussians becomes more relevan The approximaions we compare are: he Mahalanobis based maching mehod, denoed as Mahalanobis-mach (expression (5)), our maching based approximaion, denoed as KL-mach, (expressions (2) and (4)), he unscened ransform approximaion described in secion 4, and a Mone-Carlo simulaion based on 100 samples We considered he rerieval resuls based on a Mone- Carlo simulaion using 10,000 samples as he ground ruh For each of he four approximaion mehods we coun he percenage of rerieval resuls ha are consisen wih he ground ruh For each ɛ ha appears in Table 1, he simulaed rerieval ask was repeaed 10,000 imes All he approximaions were done o he expression f log g Table 1 summarizes he comparison resuls The bes resuls were obained via he unscened approximaion, followed by he resuls obained via he 100-sample Mone- Carlo simulaion As ɛ is increased he Gaussians become closer o each oher and he overlapping beween hem increases Approximaing he KL-divergence by maching a single Gaussian of g o each Gaussian componen of f becomes less accurae in ha case, as can be seen from he resuls of he Mahalanobis-mach and he KL-mach In all he ess ha were conduced, he KL-mach varian of approximaion via Gaussian maching ouperforms he Mahalanobis-mach The boom row of Table 1 indicaes he relaive processing ime needed o compue f log g for each approximaion mehod The mos efficien resuls are hose obained by he Mahalanobis-mach approximaion 1 A rade-off exiss beween efficiency and rerieval performance This rade-off srongly depends on he naure of he Gaussian mixures A more sraigh-forward crierion for he qualiy of an approximaion is is proximiy o he rue value of he KLdivergence The proximiy is measured as he average of he 1 I should be noed ha when comparing compuaional complexiy beween he proposed approximaions, we assume ha any compuaional sep needed for he approximaion ha can be done on a single image (eg invering he Gaussian marix, choosing he sigma poin) is considered as a pre-processing sep which is done as par of he MoG model learning

6 Table 1 Comparison beween rerieval simulaion resuls using differen approximaions o he KL-divergence beween mixure of Gaussians ɛ MC-100 Mahalanobis KL unscened mach mach Relaive ime Table 2 Proximiy o he rue KL-divergence ɛ MC-100 Mahalanobis KL unscened mach mach high-level semanic descripion In he following experimen we averaged rerieval resuls for 320 images, 20 images drawn randomly from each of he 16 labelled caegories we have in he daabase PR curves were calculaed for 10,20,30,40,50, and 60 rerieved images Figure 4 shows he PR curves obained by each of he approximaions The unscened based approximaion, which was shown o be more efficien in ime (see Table 1) achieves comparable resuls o he 500-sample Mone-Carlo simulaions The proposed maching based approximaion (KL-mach) significanly ouperforms he Mahalanobis-maching based mehod (Mahalanobis-mach) Combining Figure 4 and Table 1 provides an esimaion of he radeoff beween he rerieval accuracy and efficiency Precision MC 500 unscened KL mach Mahalanobis mach following meric: KL approximae (f g) KL rue (f g) KL rue (f g) Table 2 presens he accuracy resul for several ɛ values As in he former experimen i can be seen ha as he value of ɛ increases he accuracy of each of he KL-divergence approximaions decreases The mos accurae approximaion is he unscened based approximaion The wors resuls are hose obained by he Mahalanobis-mach approximaion In he final se of experimens we evaluae he rerieval resuls creaed by he various approximaions using precision versus recall (PR) curves Recall measures he abiliy of rerieving all relevan or percepually similar iems in he daabase I is defined as he raio beween he number of percepually similar iems rerieved and he oal relevan iems in he daabase Precision measures he rerieval accuracy and is defined as he raio beween he number of relevan or percepually similar iems rerieved and he oal number of iems rerieved The daabase used hroughou he experimens consiss of 1460 images selecively hand-picked from he COREL daabase o creae 16 caegories The images wihin each caegory have similar colors and color spaial layou, and can be labelled wih a Recall Figure 4 Precision vs recall for evaluaing rerieval resuls for differen KL-divergence approximaions Figure 5 displays he firs 20 images rerieved by each of he KL approximaions, for an example query aken from he Tigers caegory 2 The bes rerieval resuls are obained as before via he Mone-Carlo simulaion and he unscened based approximaion In hese wo cases more images wihin he firs 20 rerieved images are from he same caegory as he query 6 Conclusions In his paper we described wo new mehods for approximaing he KL-divergence beween mixure densiies The firs (mach-based) mehod can be applied o any mixure densiy while he second (unscened) is ailored for mixures of Gaussian densiies The efficiency and he perfor- 2 A color version may be found in hp://wwwengauacil/ hayi

7 Query MC 500 mance of hese mehods were demonsraed on image rerieval asks on a large daabase In all he experimens conduced, he unscened approximaion achieves he bes resuls, resuls ha are very close o large sample Mone-Carlo based ground ruh The Kl-mach based approximaion is faser bu less accurae han he unscened based mehod A fuure research direcion can be o combine he wo approximaion mehods ino a single scheme: In order o approximae KL(f g), we can check separaely for each Gaussian componen f i wheher he maching based approximaion is accurae enough This is he case if he componen of g mached o f i is significanly closer o f i as compared o he oher componens of g Oherwise we can uilize he unscened based approximaion o compue KL(f i g) Efficien approximaion of he KL-divergence beween MoGs is a maor sep owards coninuous and probabilisic image rerieval sysems References unscened KL mach Mahalanobis mach Figure 5 Rerieval example for a query image aken from he Tigers caegory [1] C Carson, S Belongie, H Greenspan and J Malik, Blobworld: Image Segmenaion Using Expecaion-Maximizaion and Is Applicaion o Image Querying, IEEE Transacions on PAMI, 24(8): , 2002 [2] T Cover and J Thomas, Elemens of informaion heory Wiley Series in Telecommunicaions, John Wiley and Sons, New-York, USA, 1991 [3] H Greenspan, J Goldberger and L Riddel, A coninuous probabilisic framework for image maching, Journal of Compuer Vision and Image Undersanding 84, pp , 2001 [4] S Julier and J K Uhlmann, A general mehod for approximaing nonlinear ransfromaions of probabiliy disribuions, Technical repor, RRG, Dep of Engineering Science, Universiy of Oxford, 1996 [5] S Julier and J K Uhlmann, A new exension of he Kalman filer o non-linear sysems, Proc of AeroSense: The 11h Inernaional Symposium on Aerospace/Defence Sensing, Simulaion and Conrol, Florida, 1997 [6] S Julier, The scaled unscened ransformaion, Auomaica, 2000 [7] H W Kuhn, The Hungarian mehod for he assignmen problem, Naval Research Logisics Quarerly2, pp 83-97,1955 [8] S Kullback, Informaion heory and saisics, Dover Publicaions, New York, 1968 [9] J Puzicha, Y Rubner, C Tomasi and J Buhmann, Empirical evaluaion of dissimilairy measure for color and exure, Proc of he In Conference on Compuer Vision, 1999 [10] N Vasconcelos, On he complexiy of probabilisic Image Rerieval, Proc of he In Conference on Compuer Vision, 2001 [11] G Wyszecki and W Siles, Color Science: Conceps and Mehods, Quaniaive Daa and Formulae, John Wiley and Sons, New-York, USA, 1982