CHI-SQUARE DIVERGENCE AND MINIMIZATION PROBLEM

Transcription

1 CHI-SQUARE DIVERGENCE AND MINIMIZATION PROBLEM PRANESH KUMAR AND INDER JEET TANEJA Abstrct The mnmum dcrmnton nformton prncple for the Kullbck-Lebler cross-entropy well known n the lterture In th pper we hve developed mnmum χ dvergence dcrete probblty dtrbuton gven pror dtrbuton nd prtl nformton n the form of verge or prtl nformton n the form of verge nd vrncethe probblty dtrbutons gven pror s unform Poson bnoml logrthmc nd geometrc dtrbutons re dcussed 1 Introducton The mxmum entropy prncple (MEP) due to Jyne (1957) nd the mnmum dcrmnton nformton prncple (MDIP) or mnmum cross-entropy prncple of Kullbck(1959) re well known to provde methodology for dentfyng nd chrcterzng the most unbed unvrte nd multvrte probblty dtrbutons [Kgn et l(1975) Kpur(1989) Kpur nd Kesvn(1989;199)] Kpur(198) used MEP nd MDIP to chrcterze unvrte dtrbutons: unform geometrc Gbb s dcrete norml gmm exponentl bet Cuchy Lplce norml lognorml nd Preto dtrbutons Kesvn nd Kpur(1989) descrbed generlztons of MEP nd MDIP nd presented formlm one n the MEP verson nd nother n the MDIP verson Recently Kwmur nd Iwse(00) ppled MEP to chrcterze dtrbutons of the power nverse Gussn power Brnum-Sunders nd generlzed Gumbel The equvlence of MDIP nd stttcl prncples lke mxmum lkelhood prncple nd Guss s prncple hs been dcussed by Cmpbell(1970) nd Shore nd Johnson(1980) Mnmzng cross entropy equvlent to mxmzng the lkelhood functon [Kpur(198)] nd the dtrbuton produced by n pplcton of Guss prncple lso the dtrbuton whch mnmzes the cross entropy Intheltertureonstttcstheχ dvergence due to Person (1900) well known In th pper we present methodology to derve the probblty dtrbutons usng the mnmum χ dvergence prncple when gven : () pror dtrbuton nd () prtl nformton n the form of verge or () prtl nformton n the form of verge nd vrnce The dcrete probblty dtrbuons consdered re: Unform Poson Bnoml Logrthmc nd Geometrc Let Mnmum χ dvergence for Dcrete Probblty Dtrbutons ( ) Γ n = P =(p 1 p p n ) p 0 p =1 n > be the set of ll complete fnte dcrete probblty dtrbutonsgven pror probblty dtrbuton Q Γ n m to estmte usng the prncple of mnmum drected dvergence probblty dtrbuton P Γ n when th underlyng probblty dtrbuton P stfes the usul probblty constrnts nd the prtl nformton n terms of verges Defnton 1 A probblty dtrbuton P Γ n termed s the mnmum χ dvergence probblty dtrbuton f t mnmzes the χ dvergence mesure (1) χ p (P Q) = 1 q gven: () pror probblty dtrbuton: Q Γ n 000 Mthemtcs Subject Clssfcton 94A17; 6D15 Key words nd phrses Dvergence mesure Reltve nformton Mnmum dcrmnton nformton prncple Person s ch-squre Chrcterzton Dcrete probblty dtrbutons Th reserch prtlly supported by the Nturl Scences nd Engneerng Reserch Councl s Dcovery Grnt to Prnesh Kumr

2 PRANESH KUMAR AND INDER JEET TANEJA () probblty constrnts: p 0 () prtl nformton: np p =1 np k p = m k k =1 r We present the mn result n the followng lemm: Lemm 1 Gven pror probblty dtrbuton ( ) Q = (q 1 q q n ) 0 q =1 n> q nd the constrnts () p 0 p =1 k p = m k k=1 r the mnmum χ dvergence probblty dtrbuton ( ) () P = (p 1 p p n ) 0 p p =1 n> à (4) p = q α 0 +! rx k α k nd (r +1) constnts α 0 nd α k k=1 r re determned from Ã! q rx (5) α 0 + k α k =1 nd (6) k q à α 0 + k=1 k=1! rx k α k = m k k=1 Proof We pply the fmlr method of fndng extrem of functon by ntroducng Lgrngn multplers one for ech constrnt Thus we mnmze the lner functon à n! à X p n! à X rx n! X (7) f = 1 α 0 p 1 α k k p m k q Dfferenttng f wth respect to p nd equtng the result to zero we get the followng n equtons: f (8) = p P α 0 q r k=1 α k k q =0 p q From (8) we obtn the desred vlue of p gven by (4) tht mnmzes χ (P Q) snce for ll s f p = > 0 q Expressons for the (r + 1) constnts α 0 nd α k k =1 r gven bove re obtned from the n + r + 1 equtons () nd (8) Usng the results of the lemm the mnmum χ dvergence mesure gven by: k=1 (9) χ (P Q) mn = q 4 à α 0 +! rx k α k 1 k=1

3 MINIMIZATION PROBLEM 1 Gven pror Dtrbuton nd Prtl Informton n the Form of Averge Suppose tht pror probblty dtrbuton Q nd the prtl nformton n the form of verge (m) e np p = m vlble Then we hve from Lemm 1 : Theorem 1 Gven pror probblty dtrbuton ( ) (10) Q = (q 1 q q n ) 0 q =1 n> q nd the constrnts (11) p 0 p =1 p = m the mnmum χ dvergence probblty dtrbuton ( ) (1) P = (p 1 p p n ) 0 p p =1 n> (1) p = P n µ P q m n P q + m n q q np P q n q Gven pror Dtrbuton nd Prtl Informton n the Form of Averge nd Vrnce When pror probblty dtrbuton Q nd the prtl nformton n the form of verge (m) np nd vrnce (σ P ) e p = m nd n p = m + σ re gven we get from Lemm 1: Theorem Gven pror probblty dtrbuton ( ) (14) Q = (q 1 q q n ) 0 q =1 n> q nd the constrnts (15) p 0 p =1 p = m p = m + σ the mnmum χ dvergence probblty dtrbuton ( ) (16) P = (p 1 p p n ) 0 p p =1 n> (17) p = q α0 + α 1 + α

4 4 PRANESH KUMAR AND INDER JEET TANEJA m ( P n q )( P n 4 q ) ( P n q )( P n q ) + (m + σ ) ( P n q ) ( P n q )( P n q ) +( P n α 0 = q ) ( P n 4 q )( P n q ) ( P n q )( P n q )( P n q ) ( P n q ) ( P n q ) ( P n 4 q )( P n q ) +( P n 4 q )( P n q ) m ( P n 4 q ) ( P n q ) +(m + σ ) ( P n q )( P n q ) P n q ( P n α 1 = q )( P n 4 q )+( P n q )( P n q ) ( P n q )( P n q )( P n q ) ( P n q ) ( P n q ) ( P n 4 q )( P n q ) +( P n 4 q )( P n q ) m ( P n q )( P n q ) P n q +(m + σ ) ( P n q ) ( P n q ) ( P n α = q ) +( P n q )( P n q ) ( P n q )( P n q )( P n q ) ( P n q ) ( P n q ) ( P n 4 q )( P n q ) +( P n 4 q )( P n q ) In wht follows now we dcuss specl cses of the mnmum χ dvergence probblty dtrbutons nd ther propertes χ Dvergence nd Dcrete Probblty Dtrbutons In th secton we shll consder the problem done n prevous secton the mnmzton of χ dvergence consderng prorome well known dcrete probblty dtrbutons such s unform Poson bnoml log seres etc 1 Gven Unform Dtrbuton A Pror nd Prtl Informton n the Form of Averge Proposton 1 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s unform probblty dtrbuton e q = 1 n =1 n nd the constrnts p 0 p =1 p = m (1) p = n(n 1) n 1 [(n m +1)+(m ) ] n +1 m n +1 n +1 The r th moment (r =1 ) bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by () n(n 1) [(n m +1)r +( m n 1 ) r+1 n +1 n +1 ] m n +1 In prtculr the frst four moments bout orgn re: M 1 = m (n +1)(6m n ) M = 6 (n +4)(n +1)m (n +1)(n +)(n +1) M = 10 nd M 4 = (n +1){6 4n +5n 1 m (n +) 6n +6n 1 } 0 n +1 n +1 m The men (µ) ndvrnce(σ ) of the mnmum χ dvergence probblty dtrbuton P for n+1 m n+1 re: n +1 µ n +1

5 MINIMIZATION PROBLEM 5 nd (11n + 1) (n 1) σ 144 respectvely µ = m nd σ = (n+1)(6m n ) 6 m Here below re some nterestng prtculr cses n 1 Cse 11 For m = n+1 P = Q nd the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the unform dtrbuton p = 1 n n Cse 1 For m = n+1 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q p = n n(n +1) In th cse the r th moment bout orgn of P gven by r+1 r =1 n(n +1) In prtculr the frst four moments bout orgn re: n +1 n(n +1) M 1 = M = M = (n +1) n +n 1 M 4 = n (n +1) n +n nd the men (µ) ndvrnce(σ )re n +1 µ = σ (n +)(n 1) = 18 Cse 1 Consder pror unform dtrbuton Q wth n =6 Suppose the prtl nformton bout verge (m) suchtht 7 m 1 Thus for m =5 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the unform dtrbuton p = However for 7 <m 1 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q not the unform dtrbuton Th dtrbuton p = 1 m(7 ) 15 5 (7 m) m 1 6 Gven Unform Dtrbuton A Pror nd Prtl Informton n the Form of Averge nd Vrnce Proposton The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s unform probblty dtrbuton e q = 1 n n nd the constrnts p 0 p =1 p = m nd p = m + σ () p = {10m 6m(n +1) +n (n +1)+(5σ +1)} 0m (n+1) m(8n+11)(n+1) +(n+1)(n+)(n+1)+0σ (n+1) (n+1)(n+) m 6m(n+1)+(n+)(n+1)+6σ (n+1)(n+) n (n 1) (n )

6 6 PRANESH KUMAR AND INDER JEET TANEJA subject to the smultneous suffcent restrctons on m nd σ n terms of n so tht p 0e 1 10 h(n +1) p (4) 6n(n +1) σ m 1 10 h(n +1)+ p 6n(n +1) σ 1 6 h(n +1) p (5) (n 1) 6σ m 1 6 h(n +1)+ p (n 1) 6σ (8n+11)(n+1) (n+1)(8n +56n n 59) 900σ (n+1) (6) 0(n+1) m (8n+11)(n+1)+ (n+1)(8n +56n n 59) 900σ (n+1) 0(n+1) The r th moment (r =1 ) bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by {10m 6m(n +1) +n (n +1)+(5σ +1)} r (7) 0m (n+1) m(8n+11)(n+1) +(n+1)(n+)(n+1)+0σ (n+1) (n+1)(n+) r m 6m(n+1)+(n+)(n+1)+6σ n (n 1) (n ) (n+1)(n+) r+ Now we dcuss some specl cses of the mnmum χ dvergence probblty dtrbuton P tht re for specfc vlues of verge (m) ndvrnce(σ ): Cse 1 For m = n+1 nd σ = n 1 1 Note tht m σ (n +1)(7 n) = 1 mplyng tht ½ σ m = n 7 <σ n > 7 In th cse α 0 =1 α 1 = α =0ndP = Q tht p = 1 =1 n n Cse For m = n+1 nd σ = (n+)(n 1) 18 Snce m σ 8+11n n = 18 ½ σ m = n 11 <σ else In th cse α 1 = n+1 α 0 = α =0nd P becomes p = =1 n n(n +1) Cse For m = n(n+1) (n+1) nd σ = (n 1)(n+)(n +n+) 0(n+1) We hve m σ = 4 n (n +1) n 57n 4 0 (n +1) mplyng tht ½ σ m = n 0 <σ else 6 In th cse α 0 = α 1 =0 α = (n+1)(n+1) nd P p = 6 =1 n n (n +1)(n +1)

7 MINIMIZATION PROBLEM 7 The r th moment bout orgn for r =1 gven by 6 r+ =1 n n (n +1)(n +1) Gven Poson Dtrbuton A Pror nd Prtl Informton n the Form of Averge Proposton The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s Poson probblty dtrbuton e q = e! > 0 =0 1 nd the constrnts p 0 p =1 p = m (8) p =[(1+ m)+( m ) ] q m 1+ The r th moment (r =1 ) bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by r + m r+1! e m 1+! In prtculr the frst four moments bout orgn for m 1+ re: M 1 = m M = m(1 + ) M = m( ) ( + ) (9) 1+ m e nd M 4 = m{1+ ( +7)(1+)} ( ) The men (µ) ndvrnce(σ ) of the mnmum χ dvergence probblty dtrbuton P re: µ = m nd σ = m (m ) Here below re some prtculr cses: µ 1+ nd σ Cse 1 For m = P = Q Thus the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the Poson dtrbuton e p = e > 0 =0 1! Cse For m =1+ the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q 1 e p = >0 =1 ( 1)! The r th moment bout orgn of P for r =1 gven by r 1 e ( 1)! Cse Let gven be pror Poson dtrbuton Q wth =nd the prtl nformton bout verge (m) besuchtht = m 1+ = Thusform = the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q Poson dtrbuton p = e =0 1!

8 8 PRANESH KUMAR AND INDER JEET TANEJA However for <m the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q not the Poson dtrbuton Th dtrbuton p =[(6 m)+(m )] 1 e <m! 4 Gven Poson Dtrbuton A Pror nd Prtl Informton n the Form of Averge nd Vrnce Proposton 4 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s Poson probblty dtrbuton e q = e! > 0 =0 1 nd the constrnts p 0 p =1 p = m nd p = m + σ (10) p = {m ( + )m +(+ + + σ )} {(1 + )m ( )m +(σ +σ + + )} + {m (1 + )m +( + σ ) subject to the smultneous suffcent restrctons on m nd σ n terms of so tht p 0 e (11) + p 1+4( σ ) m + + p 1+4( σ ) (1) + 1 p 1 1+4( σ ) m p 1 1+4( σ ) +( +1) 1+4(+)(+1) {σ(+1)} (1) ( +1) m +( +1) + 1+4(+)(+1) {σ(+1)} (+1) q The r th moment bout orgn of the mnmum χ dvergence probblty dtrbuton P for r = 1 gven by (14) e ( 1)! Now we dcuss some specl cses of the mnmum χ dvergence probblty dtrbuton P tht re for specfc vlues of verge (m) ndvrnce(σ ): Cse 41 For m = σ = In th cse α 0 =1 α 1 = α =0ndP = Q tht p = e {m ( + )m +(+ + + σ )} r 1 {(1 + )m ( )m +(σ +σ + + )} r + {m (1 + )m +( + σ )} r+1 Cse 4 For m =1+ vrnce by one Further α 1 = 1 α 0 = α =0nd P becomes > 0 =0 1! σ = Note tht m σ =1 mplyng tht the verge lrger thn p = 1 e > 0 =1 ( 1)! Cse 4 For m = σ = (++ ) (1+) It noted tht µ m σ =1+ 1+ scertnng tht the verge lrger thn the vrnce by more thn one 1 In th cse α 0 = α 1 =0 α = (1+) nd P 1 e p = > 0 =1 ( 1)!(1 + ) The r th moment bout orgn for r =1 gven by r+1 1 e ( 1)!(1 + ) > 0

9 MINIMIZATION PROBLEM 9 5 Gven Bnoml Dtrbuton A Pror nd Prtl Informton n the Form of Averge Proposton 5 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s bnoml probblty dtrbuton e q = n (1 ) n 0 <<1 = 0 1 n nd the constrnts p 0 p =1nd p = m n(1 + n m )+(m n) (15) p = q n m n +1 n(1 ) The r th moment (r =1 ) bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by P n n (1 ) n r (m n) (16) (1 + n m )+ n m n +1 1 n In prtculr the frst four moments bout orgn for n m n +1 re: M 1 = m M = m(1 + n ) n(n 1) M = m{1 6(1 )+n (n + ) } n (n 1) (n + 4) M 4 = m[1 + (n 1) {7+(n ) (n 6 +9)}]+n [7(1 n) 6(4 ) n n 6n +4n ] nd the men (µ) ndvrnce(σ )ofthemnmumχ dvergence probblty dtrbuton P re: n µ n +(1 ) nd(n 1)(1 ) σ (1 )[(1 + )+4(n 1)] 4 respectvely µ = m nd σ = m(1 m)+(n 1)(m n) Here below re some prtculr cses Cse 51 For m = n P = Q Thus the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the bnoml dtrbuton e µ n p = (1 ) n 0 <<1 =0 1 n Cse 5 For m = n +1 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q p = µ n 1 1 (1 ) n 0 <<1 =1 n 1 The r th moment bout orgn of P for r =1 gven by µ n 1 r 1 (1 ) n 1 Thus the men (µ) ndvrnce(σ )re µ = n +(1 ) nd σ =(n 1)(1 ) Cse 5 Consder pror bnoml dtrbuton Q wth n =10 =0 Suppose the prtl nformton bout verge (m) suchtht m 8 Thusform = the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q bnoml dtrbuton p = µ 10 (0) (08) 10 =0 1 10

10 10 PRANESH KUMAR AND INDER JEET TANEJA However for <m 8 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q not the bnoml dtrbuton Th dtrbuton µ 10 (8 m)+(05m 1) p = (0) (08) 10 = Gven Bnoml Dtrbuton A Pror nd Prtl Informton n the Form of Averge nd Vrnce Proposton 6 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s bnoml probblty dtrbuton e q = n (1 ) n 0 <<1 = 0 1 n nd the constrnts p 0 p =1 p = m nd p = m + σ (17) p =(α 0 + α 1 + α )q α 0 = (n 1) (n ) (m 1) (n ) +( m + m + σ ) (1 ) α 1 = (n 1) (m m σ ) +(4mn n 6m) n(n ) ª m(m 1) σ n(n 1) (1 ) α = (n 1) (n m) + (m + σ m) n(n 1) (1 ) subject to the smultneous suffcent restrctons on m nd σ n terms of so tht p 0e (n ) 1 p 1 4σ 4(n ) ( 1) m (n ) + 1 p 1 4σ 4(n ) ( 1) (n 1) p 1 4σ +4(n 1)(1 ) m (n 1) p 1 4σ +4(n 1)(1 ) L m U s ½ 4 +4σ 1+(n 1) (n +) (1 4σ 16 n + nσ ) (n 1) σ + ¾ +8 n 10n +9 4(4n 9) (n 1) 4 L = {1+(n 1)} s ½ 4 +4σ 1+(n 1) (n +) + (1 4σ 16 n + nσ ) (n 1) σ + ¾ +8 n 10n +9 4(4n 9) (n 1) 4 U = {1+(n 1)} The r th moment bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by µ n (1 ) n (α 0 + α 1 + α ) r r=1 Now we dcuss some specl cses of the mnmum χ dvergence probblty dtrbuton P tht re for specfc vlues of verge (m) ndvrnce(σ ): Cse 61 For m = n nd σ = n(1 ) Note tht m σ = n > 0 In th cse α 0 =1 α 1 = α =0ndP = Q tht µ n p = (1 ) n 0 <<1 =0 1 n

11 MINIMIZATION PROBLEM 11 Cse 6 For m =1+(n 1) nd σ =(n 1)(1 ) Note tht m σ =1+(n 1) > 0 In th cse α 0 = α =0α 1 = 1 n nd P becomes µ n 1 p = 1 (1 ) n 0 <<1 = 1 n 1 Cse 6 For m = (1 )(1 )+n( +n) 1+(n 1) nd σ = (n 1)(1 )[(1 ) +n( +n)] [1+(n 1)] It noted tht m σ = 1+(n 1) +(n 1)(n +1) +(n 1)(n ) +(n )(n 1) 4 {1+(n 1)} > 0 scertnng tht the verge lrger thn the vrnce for n 1 In th cse α 0 = α 1 =0 α = n{1+(n 1)} nd P µ n 1 1 (1 ) n p = 0 <<1 =1 n 1 1+(n 1) The r th moment bout orgn for r =1 gven by µ n 1 r+1 1 (1 ) n 0 <<1 1 1+(n 1) 7 Gven Logrthmc Seres Dtrbuton A Pror nd Prtl Informton n the Form of Averge Proposton 7 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s logrthmc seres probblty dtrbuton e q = K 0 <<1 K= > 0 nd the constrnts 1 log(1 ) p 0 p =1 p = m K{1 (1 )m} +(1 ){(1 )m K} K (18) p = q K(1 K) 1 m 1 1 The r th moment bout orgn of the mnmum χ dvergence probblty dtrbuton P for r = 1 gven by " # K K{1 (1 )m} r 1 +(1 ){(1 )m K} r K K(1 K) 1 m 1 1 The men (µ) ndvrnce(σ ) of the mnmum χ dvergence probblty dtrbuton P re: K 1 µ 1 1 (1 ) σ K (1 ) f 0 < 09 K (1 ) σ (1 ) f 09 <1 µ = m nd σ = (1 )(1+ K)m K m (1 ) (1 K) (1 K)(1 ) Equlty holds f =09 Then we hve K = 1 nd vrnce σ = 90 Here below re some prtculr cses

12 1 PRANESH KUMAR AND INDER JEET TANEJA Cse 71 For m = K 1 P = Q Thus the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the logrthmc seres dtrbuton p = 0 <<1 log(1 ) Cse 7 For m = 1 1 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q p =(1 ) 1 0 <<1 The r th moment bout orgn of P for r =1 gven by (1 ) 1 r Thus the men (µ) ndvrnce(σ )reµ = 1 1 nd σ = (1 ) respectvely Cse 7 Let there be pror logrthmc seres dtrbuton Q wth =08 Suppose the prtl nformton bout verge (m) such tht 485 m 5 Then form = 485 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q logrthmc seres dtrbuton p = (061)(08) However for 485 <m 5 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q not the logrthmc seres dtrbuton Th dtrbuton p =( )(m ) =1 8 Gven Logrthmc Seres Dtrbuton A Pror nd Prtl Informton n the Form of Averge nd Vrnce Proposton 8 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s logrthmc seres probblty dtrbuton e q = K 0 <<1 K= > 0 nd the constrnts 1 log(1 ) p 0 p =1 p = m (19) p =(α 0 + α 1 + α ) q p = m + σ α 0 = m( m) σ +(m + σ m) (m + m + σ ) K( +) K(m (1 ) + σ ) + (m + K) Km +(K 1)(m + σ ) ª {m( K) +(K +1)(m + σ )} + m(m 1) + σ α 1 = {K( +) } K m + K (K 1)(m (1 ) + σ ) ª + m(1 K)+K +(K +1)(m + σ ) ª {Km (K +)(m + σ )} m(m 1) σ α = {K( +) } K subject to the smultneous suffcent restrctons on m nd σ n terms of so tht p 0e q q σ ( 1) m σ ( 1) (1 ) (1 ) L 1 L m U 1 U

13 MINIMIZATION PROBLEM 1 s K( +1)(9 +1)+K 1+( +4 K) σ ( 1) (K 1) L 1 = (1 )(1 + K) s K( +1)(9 +1)+K 1+( +4 K) σ ( 1) (K 1) U 1 = (1 )(1 + K) q 1+(1 K) (1 + ) K( +1)+ K (1 + 4) 4σ ( 1) (K 1) L = (1 )(1 K) q 1+(1 K) (1 + ) K( +1)+ K (1 + 4) 4σ ( 1) (K 1) U = (1 )(1 K) The r th moment bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by K (0) (1 ) n ( ) r r =1! Now we dcuss some specl cses of the mnmum χ dvergence probblty dtrbuton P tht re for specfc vlues of verge (m) ndvrnce(σ ): Cse 81 For m = K 1 σ = K(1 K) (1 ) Note tht m σ K(K 1) = (1 ) mplyng tht ½ σ m = f K 1 <σ else In th cse α 0 =1 α 1 = α =0ndP = Q tht p = 0 <<1 log(1 ) Cse 8 For m = 1 1 σ = (1 ) Note tht m σ = 1 (1 ) mplyng tht ½ σ m = f 0 < 05 <σ f 05 <<1 In th cse α 0 = α =0α 1 = 1 K nd P becomes p =(1 ) 1 0 <<1 The r th moment bout orgn for r =1 gven by (1 ) r 1 0 <<1 Cse 8 For m = 1+ 1 σ = (1 ) Itnotedtht mplyng tht m σ = 1 (1 ) ½ σ m = f 0 < 1 <σ f 1 <<1 In th cse α 0 = α 1 =0 α = (1 ) K nd P

14 14 PRANESH KUMAR AND INDER JEET TANEJA p = (1 ) 1 0 <<1 The r th moment bout orgn for r =1 gven by (1 ) r <<1 9 Gven Geometrc Dtrbuton A Pror nd Prtl Informton n the Form of Averge Proposton 9 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s geometrc probblty dtrbuton e q =(1 ) 0 <<1 = 0 1 nd the constrnts p 0 p =1 p = m (1) p =[ (1 + m + m)+(1 )(m m ) ] (1 ) 1 1 m 1+ 1 The r th moment (r =1 ) bout orgn of the mnmum χ dvergence probblty dtrbuton P : () [ (1 + m + m)+(1 )(m m ) ] r (1 ) 1 In prtculr the frst four moments bout orgn re: M 1 = m M = m ( +1)(1 ) (1 ) 1 m 1+ 1 M = m (1 ) 6 (1 + ) (1 ) nd M 4 = m (1 ) (7 + +7) (1 ) 4 1 m 1+ 1 The men (µ) ndvrnce(σ ) of the mnmum χ dvergence probblty dtrbuton P re: 1 µ 1+ 1 nd (1 ) σ 1+6 4(1 ) respectvely µ = m nd σ = m ( 1) +m(1+ ) (1 ) Cse 91 For m = 1 P = Q nd the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q the geometrc dtrbuton p = (1 ) 0 <<1 =0 1 Cse 9 For m = 1+ 1 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q p = (1 ) 1 0 <<1 The r th moment bout orgn of P for r =1 gven by (1 ) 1 r

15 MINIMIZATION PROBLEM 15 Thus the men (µ) ndvrnce(σ )re µ = 1+ 1 nd σ = (1 ) Cse 9 Consder geometrc dtrbuton Q wth =08 Suppose the prtl nformton bout verge (m) such tht 4 m 9 Thus for m =4 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q geometrc dtrbuton p =08(0) 1 However for 4 <m 9 the probblty dtrbuton whch mnmzes the χ dvergence between P nd Q not the geometrc dtrbuton Th dtrbuton p =(0008)[4 (9 m)+ (m 4) ] (08) 1 4 <m<9 10 Gven Geometrc Dtrbuton A Pror nd Prtl Informton n the Form of Averge nd Vrnce Proposton 10 The probblty dtrbuton P whch mnmzes the χ dvergence between P nd Q gven pror dtrbuton Q s geometrc probblty dtrbuton e q =(1 ) 0 <<1 = 0 1 nd the constrnts p 0 p =1 p = m p = m + σ () p =(α 0 + α 1 + α ) q α 0 = (1 + + ) +m(1 )+(m + σ )(1 ) α 1 = m (1 + )(1+9)(1 ) (m + σ )( +1)(1 ) 4 α = (1 ) m (1 + )(1 ) +(m + σ ) (1 ) 4 4 subject to the smultneous suffcent restrctons on m nd σ n terms of so tht p 0e r ³ r ³ ( +1) σ ( 1) ( +1) σ ( 1) ( 1) m ( 1) r ³ (1 + )(1+9) (1 + )( ) 4σ ( +1) ( 1) ( +1)( 1) m r ³ (1 + )(1+9)+ (1 + )( ) 4σ ( +1) ( 1) ( +1)( 1) r ³ r ³ ( +1) σ ( 1) ( +1) σ ( 1) ( 1) m ( 1) The r th moment bout orgn of the mnmum χ dvergence probblty dtrbuton P gven by (4) (1 ) ( ) r r =1 Now we dcuss some specl cses of the mnmum χ dvergence probblty dtrbuton P tht re for specfc vlues of verge (m) ndvrnce(σ ):

16 16 PRANESH KUMAR AND INDER JEET TANEJA Cse 101 For m = 1 σ = (1 ) It noted tht m σ = (1 ) mplyng tht m<σ In th cse α 1 = α =0 α 1 =1 nd P = Q e p = (1 ) 0 <<1 =0 1 Cse 10 For m = 1+ 1 σ = (1 ) It noted tht m σ 1 = (1 ) Further t my be seen tht ½ σ m = f 0 < 1 <σ f 1 <<1 In th cse α 0 = α =0 α 1 = 1 nd P p =(1 ) 1 0 <<1 =1 The r th moment bout orgn for r =1 gven by r+1 1 (1 ) 0 <<1 Cse 10 For m = σ = 4(+ +1) (1+) (1 ) It noted tht m σ = Further m = (1 + ) (1 ) ½ σ f 0 < 074 <σ f 074 <<1 In th cse α 0 = α 1 =0 α = (1 ) (1+) nd P p = 1 (1 ) 0 <<1 = 1 1+ The r th moment bout orgn for r =1 gven by r+ 1 (1 ) 0 << Concludng Remrks The mnmum dcrmnton nformton or the mnmum cross entropy prncple (MDIP) of Kullbck nd the mxmum entropy prncple (MEP) due to Jyne hve been often used to chrcterze unvrte nd multvrte probblty dtrbutons Mnmzng cross entropy equvlent to mxmzng the lkelhood functon nd the dtrbuton produced by n pplcton of Guss prncple lso the dtrbuton whch mnmzes the cross entropy Thus gven pror nformton bout the underlyng dtrbuton n ddton to the prtl nformton n terms of the expected vlues MDIP provdes useful methodology for chrcterzng probblty dtrbutons We hve consdered the methodology for chrcterzng the dcrete probblty dtrbutons bsed on the Person χ -dtncegvenpror dtrbuton nd the prtl nformton n terms of moments We hve shown by consderng the dcrete probblty dtrbutons lke unform Poson bnoml logrthmc nd geometrc whch mnmze the Kullbck s mesure of the drected dvergence (1959) K(P Q) = p ln( p ) q tht these dtrbutons lso mnmze the Person s χ -dtnce Further wth condtons on men nd vrnce we hve obtned the new probblty dtrbutons tht mnmze the χ -dtnce Thework

17 MINIMIZATION PROBLEM 17 on mnmum χ -dvergencecontnuous probblty dtrbutons n progress nd would be reported else REFERENCES Cmpbell LL Equvlence of Guss s prncple nd mnmum dcrmnton nformton estmton of probbltes Annls of Mth Sttt Gokhle DV Mxmum entropy chrcterzton of some dtrbutons In Stttcl Dtrbuton n Scentfc Work Vol III Ptl Golz nd Old (eds) A Redel Boston Iwse K nd Hrno K Power nverse Gussn dtrbuton nd ts pplctons Jpn Jour Appled Sttt Jynes E T Informton theory nd stttcl mechncs Physcl Revews Kgn AM Lnnk V Ju nd Ro C R Chrcterzton Problems n Mthemtcl Stttcs1975 New York: John Wley Kpur JN Mxmum entropy probblty dtrbutons of contnuous rndom vrte over fnte ntervl Jour Mth nd Phys Scences Kpur JN Twenty-fve yers of mxmum entropy prncple Jour Mth nd Phys Scences Kwmur T nd Iwse K Chrcterztons of the dtrbutons of power nverse Gussn nd others bsed on the entropy mxmzton prncple Jour Jpn Sttt Soc Kesvn HK nd Kpur J The generlzed mxmum entropy prncple IEEE Trns Systems Mn nd Cybernetcs Kullbck S Informton Theory nd Stttcs 1959 New York: John Wley Person K On the Crteron tht gven system of devtons from the probble n the cse of correlted system of vrbles such tht t cn be resonble supposed to hve ren from rndom smplng Phl Mg Shore JE nd Johnson RW Axomtc dervton of the prncple of mxmum entropy nd the prncple of mnmum cross entropy IEEE Trns Informton Theory 1980 IT Tnej I J Generlzed Informton Mesures nd ther Applctons On lne book: tnej/book/bookhtml 001 Tnej IJ nd Kumr Prnesh Reltve Informton of Type s Cszá r f Dvergence nd Informton Inequltes Informton Scences 004 (to pper) E-ml ddress: kumrp@unbcc URL: kumrp E-ml ddress: tnej@mtmufscbr URL: tnej Mthemtcs Deprtment College of Scence nd Mngement Unversty of Northern Brth Columb Prnce George BC VN4Z9 Cnd Deprtmento de Mtemátc Unversdde Federl de Snt Ctrn Flornópol SC Brzl