KULLBACK-LEIBLER DISTANCE BETWEEN COMPLEX GENERALIZED GAUSSIAN DISTRIBUTIONS

Transcription

1 0th Europen Sgnl Proessng Conferene (EUSIPCO 0) uhrest, Romn, August 7-3, 0 KULLACK-LEILER DISTANCE ETWEEN COMPLEX GENERALIZED GAUSSIAN DISTRIUTIONS Corn Nfornt, Ynnk erthoumeu, Ion Nfornt, Alexndru Isr Poltehn Unverst of Tmsor, Romn, orn.nfornt@gml.om IMS- Dpt LAPS - UMR 58 CNRS, ENSEIR - Unversté de ordeux, Frne ASTRACT In texture lssfton, feture extrton n be mde n trnsform domn. A possblt to preserve the trnslton nvrne s to use omplex trnsform lke the Hpernlt Wvelet trnsform. It exhbts rulrl smmetr denst funton for subbnd oeffents so t n be modeled b prtulr form of the omplex generlzed Gussn (CGGD) dstrbuton funton. The Kullbk-Lebler (KL) dvergene, or dstne, n be used to mesure the smlrt between subbnds denst funton. We derve n ths pper losed-form expresson for the KL dvergene between two omplex generlzed Gussn dstrbutons. Index Terms Kullbk-Lebler dstne, dvergene, Complex Generlzed Gussn Dstrbuton. INTRODUCTION In probblt nd nformton theor, the Kullbk Lebler (KL) dvergene s non-smmetr mesure of the dfferene between two probblt denst funtons (pdf), p nd q. Ths s defned s []: px, D p q px, log dxd KL () qx, If the two pdfs re the sme (p=q), the dvergene s null. The KL dstne s used s smlrt mesure between textures, whh mkes t useful for texture lssfton []. In [], the uthors del wth omputton of KL dvergene for sttsts of rel wvelet subbnd oeffents. A wvelet subbnd s modeled usng the generlzed Gussn dstrbuton (GGD). sed on ths model, hperprmeters of the oeffents pdf from eh subbnd re estmted. The KL dvergene s omputed between the pdf of subbnds for two ompred textures. If ths lssfton s mde usng omplex wvelet trnsform, we need omplex model nd the losed-form for the KL dvergene. The generlzton for the GGD model n the omplex se ws proposed b Nove nd Adl whh pproxmtes the pdf bsed on hstogrm [3]. The omputton problem for the dstne between two pdf for omplex vrbles ws lso dsussed b Verdoolege [4]. He estblshed equtons for geodess n probblt spe. Unfortuntel, these reltons re not usble t ths moment. euse the hpernlt wvelet trnsform (HWT) produes omplex oeffents wth rulr dstrbuton we hve studed the smpler problem of KL dvergene for suh dstrbutons [5]. We derve n ths pper losed-form for the KL dvergene of prs of CGGD rndom vrbles nd we stud ts senstvt wth the shpe prmeter. The pper hs the followng struture. In seton we gve the defnton of HWT nd ts mn sttstl propertes. Seton 3 brefl presents the CGGD [3] nd we expln wh we hose ths model for HWT. Seton 4 presents the losed-form of the KL dvergene of two CGGD. The senstvt of ths KL dvergene wth the prmeters of the CGGDs s nlzed s well. Conlusons re presented n the lst seton.. HWT TRANSFORM In [5] new omplex wvelet trnsform ws proposed, bsed on the hperomplex mother wvelet x, ssoted to rel mother wvelet x, : x, x, Hx x, jhx, khx Hx, where j k, nd j j k [6], H x s the Hlbert trnsform omputed ross rows nd H ross olumns. The HWT of the mge f x, s: () HWT f x, f x,, x,. (3) Ths s omputed usng the D dsrete wvelet trnsform (D-DWT) of ts ssoted hperomplex mge, f :,, HWT f x DWT f x where f s defned s H f x, f x, x f x, (4) EURASIP, 0 - ISSN

2 jh f x, khx H f x, Ths mens tht HWT uses four trees, mplemented b D- DWT, beng dequte to mult-wvelet envronment [5]. The HWT dentfes sx orenttons, 3 postve nd 3 negtve, ±tn(/), ±/4 nd ±tn(): z z R jz I (5) A problem of nterest s the sttstl modelng of the HWT oeffents. For nput rndom proesses, rndom vrbles s Z, n be ssoted to the HWT oeffents z. The oeffents hve zero men, the ross-orrelton between ther rel nd mgnr prts s zero nd the vrnes of ther rel nd mgnr prts re estmted to be the sme, R I /, for n seond order sttonr bvrte nput rndom proess [7]. Therefore, we onsdered the reprttons of the rndom vrbles Z ± orrespondng to the HWT oeffents z to be lke rulrl smmetr. The ross-orrelton mtrx s: T / 0 Cb EZbZ b (6) 0 / where Z b =[Z R,Z I ] T s the bvrte vetor of the rel nd mgnr prts of the HWT oeffents. The ugmented form: Z =[Z, Z * ] T [3] n lso be used. 3. CGGD For omplex generlzed Gussn dstrbuton, CGGD, where the bvrte rndom vetor s Z b nd the ugmented vetor s Z [3], the generl form of the bvrte ovrne mtrx s: T R C E b Z Z b b (7) I where EZ Z R I s the ross-orrelton between the rel nd mgnr prt. The ugmented ovrne mtrx s estblshed b Nove nd Adl s: ( ) j H R I R I C E ZZ ( ) (8) j R I R I The probblt denst funton generlzes the GGD fml of denstes, x p x;, exp (9) X where () s the gmm funton, s the sle prmeter, nd s the shpe prmeter. The generlzed probblt denst funton for the ugmented vetor s [3]: p v v C v V C H exp (0) z / where v, nd / / / / /. In [3] Mtlb progrm s presented / whh gves the ML estmton for the vetor T R, I,,. Ths mens we n hve the ML estmton for the shpe prmeter nd the mtres C b nd C. We show n the followng the mportne of the qult of ths estmton. 4. KULLACK-LEILER DIVERGENCE FOR CGGD In the se of rulr vetors, wth R I / nd 0, whh orresponds to the HWT oeffents of n bvrte sttonr rndom proess [7], strtng from the ugmented pdf n (0), the bvrte pdf s: (, ) exp / x px / () For the pdf hvng the shpe prmeters, nd the vrnes usng reltonshp () nd the defnton n () we obtn the Kullbk-Lebler dstne: / / D p p ln KL / / () / / / / / The proof of ths relton n be found n Appendx. We plot the KL dstne between p nd p, for. In Fg., the shpe prmeter for p, tht s, s fxed, wth vlues 0.3, 0.5,,.5 nd. The shpe prmeter for p, tht s, vres from 0. to. In Fg., the shpe prmeter for p, s fxed, wth vlues 0.3, 0.5,,.5 nd. The shpe prmeter for p, tht s, vres from 0. to. It s essentl for n lssfton tht the dstne between the two pdf to be s dsrmnnt s possble. In other words, f nd re ver lose then KL should be lose to zero, nd f the hve dfferent vlues, ths dstne should be s hgh s possble. It n be observed, nlzng Fg. nd Fg. tht the KL beomes zero f = nd σ =σ. These prmeters re not pror known n textures lssfton ppltons nd the must be estmted. The suess of the lssfton depends on the qult of the estmtors used. For n effent lssfton, t s neessr tht the speed of vrton of the urves n Fg. nd Fg. round ther ntersetons wth the lne expressed b the equton D KL =0, to be s hgh s possble. 85

3 Fg.. KL dstne between p nd p ( ). The shpe prmeter for p, s fxed, wth vlues 0.3, 0.5,,.5 nd. The shpe prmeter for p, tht s, vres from 0. to. Fg.. KL dstne between p nd p ( ). The shpe prmeter for p, s fxed, wth vlues 0.3, 0.5,,.5 nd. The shpe prmeter for p, tht s, vres from 0. to. For the VsTex dtbse [8], usng 40 mges subdvded n 6 submges eh, resultng n 640 smller mges, we hve repeted the estmton of the shpe prmeter nd of the ovrne mtrx C, usng the progrms presented n [3]. Ths ws done n the HWT domn, usng one deomposton level nd Dubehes-3 mother wvelet. We hve noted tht the shpe prmeter vres n the rnge 0. 5 but ts vlues round 0.5 pper more frequentl. From Fg. t s esl noteble tht the KL dstne vres onl slghtl for vlues of between 0.8 nd.. It 85

4 s nterestng tht t responds better round the vlue =0.5. The KL dstne s more senstve for the plot =0.3 thn for Gussn se ( =). For Fg., where we plotted KL dstne wth fxed, the best se s for =0.3, s opposed to the se of = (Gussn se). The KL dstne vres onl slghtl for exmple n the rnge of As expeted, the KL dstne s non-smmetr wth respet to nd. 5. CONCLUSIONS In texture lssfton, when usng omplex trnsform suh s the HWT, modeled b the CGGD dstrbuton, the KL dstne n be used to mesure the smlrt between subbnd denst funtons. Ths s not lws stsftor beuse there re ntervls where KL dstne vres onl slghtl despte the ft tht the two pdfs re ver dfferent. It would be useful n the future to stud more mesures for texture lssfton. 6. ACKNOWLEDGMENTS Ths pper ws developed n the frmework of blterl progrm rnus 50/ Clssfton de textures fondée sur l théore des ondelettes hpernltques et les opules. 7. REFERENCES [] S. Kullbk nd R.A. Lebler, On Informton nd Suffen, Annls of Mthemtl Sttsts, (), pp , 95. [] M. N. Do nd M. Vetterl, Wvelet-bsed texture retrevl usng generlzed Gussn denst nd Kullbk-Lebler dstne, IEEE Trns. Imge Proessng, vol., pp , Feb. 00. [3] M. Nove, T. Adl, nd A. Ro, A omplex generlzed Gussn dstrbuton-chrterzton, generton, nd estmton, IEEE Trns. Sgnl Proessng, vol. 58, no. 3, prt., pp , Mrh 00. [4] G. Verdoolege, S. De ker, P. Sheunders, Multsle olour texture retrevl usng the geodes dstne between multvrte generlzed Gussn models, ICIP'008, pp [5] I. Frou, C. Nfornt, J.-M. ouher, A. Isr, Imge Denosng Usng New Implementton of the Hpernlt Wvelet Trnsform, IEEE Trns. on Instrumentton nd Mesurement, Aug. 009, vol. 58, no. 8, pp [6] C. Dvenport, Commuttve Hperomplex Mthemts, [7] C. Nfornt, I. Frou, D. Isr, J.-M. ouher, A. Isr, A Seond Order Sttstl Anlss of the Hpernlt Wvelet Trnsform, Pro. 9th Int. Smp. on Eletrons nd Teleommuntons, ISETC 00, Tmsor, Romn, Nov. 00, pp [8] MIT Vson nd modelng group, Vson texture, Avlble onlne: APPENDIX We ompute the KL dstne for the CGGD model, n the rulr se. The probblt denst funton s: / x px, exp / (A.) x A exp where x nd re the rel nd mgnr omponents, nd A nd (A.) We ompre two pdf: x p Aexp (A.3) nd x p Aexp (A.4) We strt from the KL dstne defnton: D p p px, log dxd KL Frst we hve: p x, (A.5) p x, ln p A x x ln (A.6) p A The ntegrnd s then: p x p ln A exp p (A.7) A x x ln A The KL dstne n be wrtten s sum of three terms, I, I nd I 3 : D p p I I I (A.8) The frst term s: KL 3 x A exp ln A I A dxd 853

5 euse: we obtn: r A exp ln 0 0 A I A rdrd A r A 0 A ln exp r rdr t rdr t dt A t I Aln t e dt A 0 A A ln A In the sme mnner, we hve: x x I A exp dxd r r A exp rdr 0 A nd x x I A 3 exp dxd r r A exp rdr 0 t I3 A t e t dt 0 t 3 I A t e tdt A The dstne beomes: 0 (A.9) (A.0) (A.) (A.) (A.3) A D p p A ln KL A A A where: A It results tht: / / D KL p p ln / / ;, / / / / / (A.6) We took nto ount tht: We verf tht the dstne s orret, for ; t should be zero: / / D KL p p ln / / / / / / / 0 / (A.4) (A.5) (A.7) (A.8) 854